<jupyter_start><jupyter_text>Data Exploration in Pandas  Pandas is very useful for basic statistical exploration. Along the way, we will also learn some basic munging of data with Pandas. First, we download all the data.<jupyter_code>import pandas as pd
import matplotlib.pyplot as plt
% matplotlib inline
JPY = pd.read_csv('JPY.csv')
EUR = pd.read_csv('EUR.csv')
CNY = pd.read_csv('CNY.csv')<jupyter_output><empty_output><jupyter_text>We can check the type of data imported quite easily.<jupyter_code>type(JPY)<jupyter_output><empty_output><jupyter_text>Check how the data looks like<jupyter_code>JPY.head(3)<jupyter_output><empty_output><jupyter_text>As you can see, we need to get rid of one column, and set DATE as the index<jupyter_code>JPY = JPY.drop(['Unnamed: 0'], axis=1)
JPY.DATE = pd.to_datetime(JPY.DATE)
JPY = JPY.set_index('DATE')
EUR = EUR.drop(['Unnamed: 0'], axis=1)
EUR.DATE = pd.to_datetime(EUR.DATE)
EUR = EUR.set_index('DATE')
CNY = CNY.drop(['Unnamed: 0'], axis=1)
CNY.DATE = pd.to_datetime(CNY.DATE)
CNY = CNY.set_index('DATE')
JPY.head(2)<jupyter_output><empty_output><jupyter_text>And plotting is easy peasy, as we have seen previously.<jupyter_code>JPY.plot()
EUR.plot()
CNY.plot()
CURR = pd.concat([EUR, JPY, CNY], axis=1)
CURR.head(3)
CURR.columns = (['EUR', 'JPY', 'CNY'])
CURR.head(3)
CURR.plot()<jupyter_output><empty_output><jupyter_text>To get a quick overview of the key stats of each of the data columns<jupyter_code>CURR.describe()<jupyter_output><empty_output><jupyter_text>And for all the stats, such as -count - to count the number of items in each column<jupyter_code>CURR.count()<jupyter_output><empty_output><jupyter_text>To get the minimum level for each column<jupyter_code>CURR.min()<jupyter_output><empty_output><jupyter_text>To get the minimum level for each column<jupyter_code>CURR.max()<jupyter_output><empty_output><jupyter_text>To get the median for each column<jupyter_code>CURR.median()<jupyter_output><empty_output><jupyter_text>To get the variance of each column<jupyter_code>CURR.var()<jupyter_output><empty_output><jupyter_text>To get the standard deviation for each column<jupyter_code>CURR.std()<jupyter_output><empty_output><jupyter_text>The skew<jupyter_code>CURR.skew()<jupyter_output><empty_output><jupyter_text>And the kurtosis<jupyter_code>CURR.kurt()<jupyter_output><empty_output><jupyter_text>The correlation<jupyter_code>CURR.corr()<jupyter_output><empty_output><jupyter_text>And finally the covariance between all 3 currencies<jupyter_code>CURR.cov()<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.12470588235294118, 'fraction_numerical': 0.004705882352941176, 'mean_word_length': 8.590225563909774, 'pattern_counts': {'":': 0, '<': 65, '<?xml version=': 0, '>': 65, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 1, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '16', 'dir': 'jupyter-structured-clean-dedup', 'id': '352041', 'license': 'MIT', 'n_tokens_mistral': 1002, 'n_tokens_neox': 982, 'n_words': 199, 'path': 'Data Exploration in Pandas.ipynb', 'repo_name': 'playgrdstar/explore_data_pandas', 'size': '2550'}