<jupyter_start><jupyter_text>Summarising and Computing Descriptive Statspandas objects are equipped with a set of common mathematical and statistical methods. Most of these fall into the category of reductions or summary statistics, methods that extract a single value from a Series or a Series of values from the rows and columns of a DataFrame- have built0in handling for missing data<jupyter_code>import numpy as np
import pandas as pd
df = pd.DataFrame([[1.4, np.nan],[7.1, -4.5],
                    [np.nan, np.nan],[0.75,-1.3]],
                     index = ['a','b','c','d'],
                     columns=['one', 'two'])
df
df.sum()
df.sum(axis=1)
#NA values are excluded unless the entire slice is NA
#Skipping NA values can be disabled with the skipna option
df.mean(axis=1,skipna=False)<jupyter_output><empty_output><jupyter_text>The below method is extremely useful<jupyter_code>df.describe()<jupyter_output><empty_output><jupyter_text>List of Descriptive and Summary Stat Methods- count: Number of non-NA values- describe: set of summary stats- min, max- argmin, argmax: compute index locations at which min and max value obtained, respectively- idxmin, idxmax: return index value where the max and min values are contained - quantile: compute sample quantile ranging from 0 to 1- sum- mean- median- mad: mean absolute devaition from the mean value- prod: product of all values- var: simple variance of all values- std- skew: sample skewness of values- cumsum: cumlative sum of values- diff: compute first arthmetic difference(useful for time series)- pct_change: Compute percent changes (valuable for time series analysis)  Correlation & CovarianceCorrelation: a mutual relationship or connection between two or more thingsCovariance: In probability theory and statistics, covariance is a measure of the joint variability of two random variables. If the greater values of one variable mainly correspond with the greater values of the other variable, and the same holds for the lesser values, the covariance is positiveCorrelation and covaraince are computed for pairs of arguments. This example considers some DF’s of stock prices and volumes obtained from Yahoo! Finance using the add-on pandas-datareader package.<jupyter_code>import pandas_datareader.data as web
all_data = {ticker: web.get_data_yahoo(ticker)
           for ticker in ['AAPL', 'IBM', 'MSFT', 'GOOG']}

price = pd.DataFrame({ticker: data['Adj Close']
                     for ticker, data in all_data.items()})
volume = pd.DataFrame({ticker: data['Volume']
                      for ticker, data in all_data.items()})
#compute percent changes, a time series operation
returns = price.pct_change()
returns.tail()<jupyter_output><empty_output><jupyter_text>.corr: computes the correlation.cov:  computes the covariance<jupyter_code>returns['MSFT'].corr(returns['IBM'])
returns['MSFT'].cov(returns['IBM'])
#to return a full correaltion or covariance matrix
returns.corr()
returns.cov()<jupyter_output><empty_output><jupyter_text>corrwith methodUsing this, you can compute pairwise correlations. Can also pass axis=‘columns’ to get row-by-row results.<jupyter_code>returns.corrwith(returns.IBM)<jupyter_output><empty_output><jupyter_text>Unique Values, Value Counts<jupyter_code>#To return an array of the unique values in a Series, use .unique()
obj = pd.Series(['a','a','d','c','f','c','g','f'])
obj
uniques = obj.unique()
uniques
#To compute a Series containing value frequencies
obj.value_counts()<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.09259259259259259, 'fraction_numerical': 0.004558404558404558, 'mean_word_length': 5.001709401709402, 'pattern_counts': {'":': 0, '<': 25, '<?xml version=': 0, '>': 25, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '6', 'dir': 'jupyter-structured-clean-dedup', 'id': '504773', 'license': 'MIT', 'n_tokens_mistral': 1022, 'n_tokens_neox': 972, 'n_words': 392, 'path': 'Notes/Pandas/Getting Started with pandas/Summarising and Computing Descriptive Stats.ipynb', 'repo_name': 'yorken2/DataAnalysis-Notes', 'size': '3510'}