<jupyter_start><jupyter_text>Independent Study Week 8 - Data cleaning test (Oct 13, 2021) Using the data of 3 province - Incheon, Busan, Jeju<jupyter_code>import matplotlib.pyplot as plt
import matplotlib
import seaborn as sns
import numpy as np
import pandas as pd
td = pd.read_csv('testing11.csv', thousands=',', skiprows=2)
td.head()<jupyter_output><empty_output><jupyter_text>**Remove the tourist_spot_kor**<jupyter_code>td2 = td.drop(columns='tourist_spot_kor')
td2.head()<jupyter_output><empty_output><jupyter_text>**only select the total number of tourists in each spot**<jupyter_code>td3 = td2[(td2.foreigner_local == 'Total')]
td3.head()<jupyter_output><empty_output><jupyter_text>Making 'sum' columns - there is an error message. Is this the right way to make columns?<jupyter_code>td3['2016_sum'] = td3['2016-Jan'] + td3['2016-Feb'] + td3['2016-Mar'] + td3['2016-Apr'] + td3['2016-May'] + td3['2016-Jun'] + td3['2016-Jul'] + td3['2016-Aug'] + td3['2016-Sep'] + td3['2016-Oct'] + td3['2016-Nov'] + td3['2016-Dec']
td3['2017_sum'] = td3['2017-Jan'] + td3['2017-Feb'] + td3['2017-Mar'] + td3['2017-Apr'] + td3['2017-May'] + td3['2017-Jun'] + td3['2017-Jul'] + td3['2017-Aug'] + td3['2017-Sep'] + td3['2017-Oct'] + td3['2017-Nov'] + td3['2017-Dec']
td3['2018_sum'] = td3['2018-Jan'] + td3['2018-Feb'] + td3['2018-Mar'] + td3['2018-Apr'] + td3['2018-May'] + td3['2018-Jun'] + td3['2018-Jul'] + td3['2018-Aug'] + td3['2018-Sep'] + td3['2018-Oct'] + td3['2018-Nov'] + td3['2018-Dec']
td3['2019_sum'] = td3['2019-Jan'] + td3['2019-Feb'] + td3['2019-Mar'] + td3['2019-Apr'] + td3['2019-May'] + td3['2019-Jun'] + td3['2019-Jul'] + td3['2019-Aug'] + td3['2019-Sep'] + td3['2019-Oct'] + td3['2019-Nov'] + td3['2019-Dec']
td3['2020_sum'] = td3['2020-Jan'] + td3['2020-Feb'] + td3['2020-Mar'] + td3['2020-Apr'] + td3['2020-May'] + td3['2020-Jun'] + td3['2020-Jul'] + td3['2020-Aug'] + td3['2020-Sep'] + td3['2020-Oct'] + td3['2020-Nov'] + td3['2020-Dec']
td3.head()
td4 = td3[['tourist_spot_eng', 'sub_category', '2016_sum', '2017_sum', '2018_sum', '2019_sum', '2020_sum']].sort_values('2020_sum', ascending=False)
td4.head(20)
td4.dropna().plot(x='tourist_spot_eng', y='2020_sum', kind='bar');
td5 = td4.groupby('sub_category').agg('sum')
td5
td5.reset_index(inplace=True)
td5
td5.set_index('sub_category', inplace=True)
td5
td6 = td5.transpose()
td6
td6.plot(figsize = (15,5), kind='bar');
td6.plot(figsize = (15,5), kind='bar', stacked = True);<jupyter_output><empty_output><jupyter_text>- **Can we change the name of the column 'sub_category' to 'year'?**- **Not possible to choose the column ['sub_category']. Why?**  Is it possible to make the fowlloing table using the data we have?melt() function? pivot_table()?<jupyter_code>i_total_number_grouped2<jupyter_output><empty_output><jupyter_text>2017-2021.March Monthly Total by Category<jupyter_code># Chart will be added.<jupyter_output><empty_output><jupyter_text>2017-2021.March Monthly Total by Sub_category<jupyter_code># Chart will be added.<jupyter_output><empty_output><jupyter_text>Making a barplot for sub_category (Year Data) Data was sorted by excel for test.<jupyter_code>i_total_number = pd.read_csv('total_number1.csv')
i_total_number.head()
i_total_number.groupby(['Year','Sub_category']).agg('sum')
# Make a 'sum' column 
i_total_number_grouped = i_total_number.groupby(['Year','Sub_category']).agg('sum')
i_total_number_grouped['sum'] = i_total_number_grouped['Incheon'] + i_total_number_grouped['Busan'] + i_total_number_grouped['Jeju']
i_total_number_grouped.reset_index(inplace=True)
i_total_number_grouped

# make a table only including 'Year', 'Sub_category', 'sum'
i_total_number_grouped2 = i_total_number_grouped[['Year', 'Sub_category', 'sum']]
i_total_number_grouped2
# barplot for sub_category
sns.set(rc={'figure.figsize':(15,5)})
sns.barplot(data = i_total_number_grouped2, x='Year', y='sum', hue='Sub_category');<jupyter_output><empty_output><jupyter_text>how to use vegalite in jupyter notebook<jupyter_code>from vega import VegaLite
VegaLite({
    "data" : {"url" : "total_number.csv"},
    "mark":"bar",
    "encoding" : {
        "y" : {"type" : "quantitative", "field" : "Incheon"},
        "x" : {"type" : "ordinal", "field" : "Year"},
        "color" : {"type" : "nominal", "field" : "Sub_category"}
    }
})<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.19842919842919843, 'fraction_numerical': 0.09678909678909679, 'mean_word_length': 6.930402930402931, 'pattern_counts': {'":': 1, '<': 37, '<?xml version=': 0, '>': 37, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '9', 'dir': 'jupyter-structured-clean-dedup', 'id': '82638', 'license': 'BSD-3-Clause', 'n_tokens_mistral': 1978, 'n_tokens_neox': 1699, 'n_words': 300, 'path': 'etc/Independent Study Week 8 - Data cleaning test (Oct 13, 2021).ipynb', 'repo_name': 'haesookma/haesookma.github.io', 'size': '4329'}