I have a pandas data frame with several entries, and I want to calculate the correlation between the income of some type of stores. There are a number of stores with income data, classification of area of activity (theater, cloth stores, food ...) and other data. I tried to create a new data frame and insert a column with the income of all kinds of stores that belong to the same category, and the returning data frame has only the first column filled and the rest is full of NaN's.The code that I tired: corr = pd.DataFrame()for at in activity:    stores.loc[stores['Activity']==at]['income']I want to do so, so I can use .corr() to gave the correlation matrix between the category of stores.After that, I would like to know how I can plot the matrix values (-1 to 1, since I want to use Pearson's correlation) with matplolib.
Calculation and Visualization of Correlation Matrix with Pandas
python;statistics;visualization;pandas
I suggest some sort of play on the following:Using the UCI Abalone data for this example...import matplotlibimport numpy as npimport matplotlib.pyplot as plt%matplotlib inline# Read file into a Pandas dataframefrom pandas import DataFrame, read_csvf = 'https://archive.ics.uci.edu/ml/machine-learning-databases/abalone/abalone.data'df = read_csv(f)df=df[0:10]dfCorrelation matrix plotting function: # Correlation matric plotting function def correlation_matrix(df):    from matplotlib import pyplot as plt    from matplotlib import cm as cm    fig = plt.figure()    ax1 = fig.add_subplot(111)    cmap = cm.get_cmap('jet', 30)    cax = ax1.imshow(df.corr(), interpolation=nearest, cmap=cmap)    ax1.grid(True)    plt.title('Abalone Feature Correlation')    labels=['Sex','Length','Diam','Height','Whole','Shucked','Viscera','Shell','Rings',]    ax1.set_xticklabels(labels,fontsize=6)    ax1.set_yticklabels(labels,fontsize=6)    # Add colorbar, make sure to specify tick locations to match desired ticklabels    fig.colorbar(cax, ticks=[.75,.8,.85,.90,.95,1])    plt.show()correlation_matrix(df)Hope this helps!