<jupyter_start><jupyter_text>pandasを用いたデータ分析サンプルpandasに備わったAPIを活用したデータ分析の動作確認．  動作確認で使用するデータは，scikit-learnを用いて取得可能なひとつであるボストン住宅価格データセットを使用する． モジュールのインストール<jupyter_code>!pip install pandas
!pip install matplotlib
!pip install scikit-learn<jupyter_output>Requirement already satisfied: pandas in /opt/conda/lib/python3.9/site-packages (1.2.4)
Requirement already satisfied: python-dateutil>=2.7.3 in /opt/conda/lib/python3.9/site-packages (from pandas) (2.8.1)
Requirement already satisfied: pytz>=2017.3 in /opt/conda/lib/python3.9/site-packages (from pandas) (2021.1)
Requirement already satisfied: numpy>=1.16.5 in /opt/conda/lib/python3.9/site-packages (from pandas) (1.20.3)
Requirement already satisfied: six>=1.5 in /opt/conda/lib/python3.9/site-packages (from python-dateutil>=2.7.3->pandas) (1.16.0)
Requirement already satisfied: matplotlib in /opt/conda/lib/python3.9/site-packages (3.4.2)
Requirement already satisfied: cycler>=0.10 in /opt/conda/lib/python3.9/site-packages (from matplotlib) (0.10.0)
Requirement already satisfied: numpy>=1.16 in /opt/conda/lib/python3.9/site-packages (from matplotlib) (1.20.3)
Requirement already satisfied: pyparsing>=2.2.1 in /opt/conda/lib/python3.9/site-packages (from matplotlib) (2.4.7)
Requirement a[...]<jupyter_text>モジュールのインポート<jupyter_code>import pandas as pd
from sklearn.datasets import load_boston
import matplotlib.pyplot as plt
import seaborn as sns<jupyter_output><empty_output><jupyter_text>データセット読み込み<jupyter_code>dataset_boston = load_boston()
dataset_boston.keys()
df_dataset_boston_data = pd.DataFrame(dataset_boston['data'], columns=dataset_boston['feature_names'])
df_dataset_boston_data.head()
TARGET_TAG = 'SalePrice'
df_dataset_boston_target = pd.DataFrame(dataset_boston['target'], columns=[TARGET_TAG])
df_dataset_boston_target.head()
df_dataset_boston = pd.concat([df_dataset_boston_data, df_dataset_boston_target], axis=1)
df_dataset_boston.head()<jupyter_output><empty_output><jupyter_text>データ分析用APIの動作確認 要約([pandas.DataFrame.describe](https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.DataFrame.describe.html))データセットの要約を確認するための統計量を算出するAPI．|算出するパラメータ|説明||:--|:--||count|データ数||mean|平均||std|標準偏差||min|最小値||25%|$\frac{1}{4}$分位||50%|$\frac{1}{2}$分位(=中央値)||75%|$\frac{3}{4}$分位||max|最大値|<jupyter_code>df_dataset_boston_data.describe()<jupyter_output><empty_output><jupyter_text>作図([pandas.DataFrame.plot](https://pandas.pydata.org/docs/reference/api/pandas.DataFrame.plot.html))SeriesまたはDataFrameのグラフを作成できる．  デフォルトはmatplotlibだが，plotting.backendで使用する作図モジュールを指定することもできる．<jupyter_code>var = 'CRIM'
plot_data = pd.concat([df_dataset_boston_target[TARGET_TAG], df_dataset_boston_data[var]], axis=1)
plot_data.plot.scatter(x=var, y=TARGET_TAG)<jupyter_output><empty_output><jupyter_text>相関分析([pandas.DataFrame.corr](https://pandas.pydata.org/docs/reference/api/pandas.DataFrame.corr.html))pandasにはデータ間の相関係数を算出することができる．  相関分析手法は，ピアソン，ケンドール，スピアマンから選択可能で，デフォルトはピアソンである．DataFrameの各データ間の相関係数を相関マトリクスとして得ることができる．<jupyter_code>corrmat = df_dataset_boston.corr()
fig, ax = plt.subplots(figsize=(12, 9))
sns.heatmap(corrmat, square=True)<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.14331210191082802, 'fraction_numerical': 0.029936305732484077, 'mean_word_length': 8.971428571428572, 'pattern_counts': {'":': 0, '<': 24, '<?xml version=': 0, '>': 33, 'https://': 3, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '6', 'dir': 'jupyter-structured-clean-dedup', 'id': '558423', 'license': 'Apache-2.0', 'n_tokens_mistral': 1479, 'n_tokens_neox': 1438, 'n_words': 138, 'path': 'python/data_analysis/pandas/pandas.ipynb', 'repo_name': 'ryoma-jp/samples', 'size': '3140'}