<jupyter_start><jupyter_text>tau-data IndonesiaExploratory Data Analysis-02: Data Visualizations(C) <NAME>https://tau-data.id/eda-02/ ~ <EMAIL><jupyter_code>import warnings; warnings.simplefilter('ignore')
import pandas as pd, matplotlib.pyplot as plt, seaborn as sns, numpy as np
import matplotlib.cm as cm
from collections import Counter
plt.style.use('bmh'); sns.set()
# Importing CSV data  https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.read_csv.html
try:
    # Running Locally 
    price = pd.read_csv('data/price.csv')
except:
    # Running in Google Colab
    !mkdir data
    !wget -P data/ https://raw.githubusercontent.com/taudata-indonesia/eLearning/master/data/price.csv
    price = pd.read_csv('data/price.csv')
# Dari EDA-01 - Bisa juga Load PreProcessed Data
price.drop("Observation", axis=1, inplace=True)
price.drop_duplicates(inplace=True)
price['Parking'] = price['Parking'].astype('category')
price['City_Category'] = price['City_Category'].astype('category')
price2 = price[np.abs(price.House_Price - price.House_Price.mean())<=(2*price.House_Price.std())]
price2.info()<jupyter_output><empty_output><jupyter_text>Statistika Deskriptif<jupyter_code># Statistika Sederhana dari data "Numerik"-nya
price2.describe(include='all').transpose()<jupyter_output><empty_output><jupyter_text>Apakah ada kecenderungan perbedaan harga rumah akibat dari tipe tempat parkir?<jupyter_code>p= sns.catplot(x="Parking", y="House_Price", data=price2)
# Apa yang bisa dilihat dari hasil ini?<jupyter_output><empty_output><jupyter_text>Tambah dimensi di Visualisasi untuk melihat insight yang lebih jelas/baik<jupyter_code># Bisa juga plot dengan informasi dari 3 variabel sekaligus
# (untuk melihat kemungkinan faktor interaksi)
p= sns.catplot(x="Parking", y="House_Price", hue="City_Category", kind="swarm", data=price2)<jupyter_output><empty_output><jupyter_text>Ada informasi apakah dari hasil diatas?<jupyter_code>plt.figure(figsize=(8,6)) # https://matplotlib.org/api/_as_gen/matplotlib.pyplot.figure.html#matplotlib.pyplot.figure
p = sns.countplot(x="Parking", data=price2)<jupyter_output><empty_output><jupyter_text>Adding labels? ... Hhhmmm...<jupyter_code>def groupedbarplot(df, width=0.8, annotate="values", ax=None, **kw):
    ax = ax or plt.gca()
    n = len(df.columns)
    w = 1./n
    pos = (np.linspace(w/2., 1-w/2., n)-0.5)*width
    w *= width
    bars = []
    for col, x in zip(df.columns, pos):
        bars.append(ax.bar(np.arange(len(df))+x, df[col].values, width=w, **kw))
        # matplotlib.pyplot.bar(x, height, width=0.8, bottom=None, *, align='center', data=None, **kwargs)
        for val, xi in zip(df[col].values, np.arange(len(df))+x):
            if annotate:
                txt = val if annotate == "values" else col
                ax.annotate(txt, xy=(xi, val), xytext=(0,2), 
                            textcoords="offset points",
                            ha="center", va="bottom")
    ax.set_xticks(np.arange(len(df)))
    ax.set_xticklabels(df.index)
    return bars
counts = price2.groupby(["Parking", "City_Category"]).size().unstack()
plt.figure(figsize=(12,8))
groupedbarplot(counts)
plt.show()
price2.groupby(["Parking", "City_Category"]).size().unstack()<jupyter_output><empty_output><jupyter_text>Horizontal? Why?<jupyter_code>ax = sns.countplot(y = 'Parking', hue = 'City_Category', palette = 'muted', data=price2)
tips=sns.load_dataset('tips')
categorical = tips.select_dtypes(include = ['category']).columns
fig, ax = plt.subplots(2, 2, figsize=(20, 10))
for variable, subplot in zip(categorical, ax.flatten()):
    sns.countplot(tips[variable], ax=subplot)<jupyter_output><empty_output><jupyter_text>Stacked/Segmented Chart<jupyter_code>CT = pd.crosstab(index=price2["City_Category"], columns=price2["Parking"])
p = CT.plot(kind="bar", figsize=(8,8), stacked=True)
# ini dilakukan jika kita ingin menyimpan plotnya ke dalam suatu file
p.figure.savefig('barChart.png')
# lihat di folder ipynb-nya akan muncul file baru.<jupyter_output><empty_output><jupyter_text>Mosaic Plot for multiple categorical data analysis<jupyter_code>from statsmodels.graphics.mosaicplot import mosaic

p = mosaic(tips, ['sex','smoker','time'])
# PieChart
plot = price2.City_Category.value_counts().plot(kind='pie')<jupyter_output><empty_output><jupyter_text>Show Values?<jupyter_code>data = price2['Parking']

proporsion = Counter(data)
values = [float(v) for v in proporsion.values()]
colors = ['r', 'g', 'b', 'y']
labels = proporsion.keys()
explode = (0.2, 0, 0, 0)
plt.pie(values, colors=colors, labels= values, explode=explode, shadow=True)
plt.title('Proporsi Tipe Parkir')
plt.legend(labels,loc='best')
plt.show()
# Jika ada outlier grafiknya menjadi tidak jelas (data = price, bukan price2)
p = sns.boxplot(x="House_Price", y="Parking", data=price)
# BoxPlots
p = sns.boxplot(x="House_Price", y="Parking", data=price2)
# Apa makna pola yang terlihat di data oleh BoxPlot ini?<jupyter_output><empty_output><jupyter_text>Boxplot dapat juga dipisahkan berdasarkan suatu kategori<jupyter_code>p = sns.catplot(x="Parking", y="House_Price", hue="City_Category", kind="box", data=price2)<jupyter_output><empty_output><jupyter_text>* Ada dugaan/interpretasi (baru) apakah dari boxPlot diatas?* Sebutkan beberapa kelemahan (PitFalls) Box Plot?  Swarn Plot & Violin Plot<jupyter_code>p= sns.catplot(x="day", y="total_bill", hue="sex", kind="swarm", data=tips)
p = sns.violinplot(x="day", y="total_bill", data=tips,palette='rainbow')
col = 'House_Price'
plot = sns.displot(data=price2, x=col, kde=True)
plot = sns.displot(data=price2, x=col, hue='Parking', kind="kde")
numerical = price2.select_dtypes(include = ['int64','float64']).columns

price2[numerical].hist(figsize=(15, 6), layout=(2, 4));
p = sns.scatterplot(x=price2['House_Price'], y=price2['Dist_Market'], hue = price2['Parking'])<jupyter_output><empty_output><jupyter_text>Joined<jupyter_code>p = sns.jointplot(x=price2['House_Price'], y=price2['Rainfall'])<jupyter_output><empty_output><jupyter_text>Conditional Plot<jupyter_code>cond_plot = sns.FacetGrid(data=price2, col='Parking', hue='City_Category')#, hue_order=["Yes", "No"]
p = cond_plot.map(sns.scatterplot, 'Dist_Hospital', 'House_Price').add_legend()<jupyter_output><empty_output><jupyter_text>Pairwise Plot<jupyter_code># Coba kita perhatikan sebagiannya saja dulu dan coba kelompokkan berdasarkan "Parking"
p = sns.pairplot(price2[['House_Price','Builtup','Dist_Hospital','Parking']], hue="Parking")
# Ada pola menarik?<jupyter_output><empty_output><jupyter_text>3D Visualization: 3D Scatter Plothttps://pythonprogramming.net/matplotlib-3d-scatterplot-tutorial/<jupyter_code>%matplotlib inline

fig = plt.figure(figsize=(12, 10))
ax = fig.add_subplot(111, projection='3d')
x = price2['House_Price']
y = price2['Dist_Hospital']
z = price2['Rainfall']
warna = cm.rainbow(np.linspace(0, 1, len(y)))

ax.scatter(x, y, z, s=50, c=warna, marker='o')
ax.set_xlabel('Harga')
ax.set_ylabel('Jarak ke RS')
ax.set_zlabel('Curah Hujan')
plt.show()
%matplotlib inline
fig = plt.figure(figsize=(12, 10))
ax = fig.add_subplot(111, projection='3d')
x = price2['House_Price']
y = price2['Dist_Hospital']
z = price2['Rainfall']
warna = cm.rainbow(np.linspace(0, 1, len(y)))

ax.scatter(x, y, z, s=50, c=warna, marker='o')
ax.set_xlabel('Harga')
ax.set_ylabel('Jarak ke RS')
ax.set_zlabel('Curah Hujan')
plt.show()<jupyter_output><empty_output><jupyter_text>3D Visualization:  3D Bar PlotsBar plots are used quite frequently in data visualisation projects since they’re able to convey information, usually some type of comparison, in a simple and intuitive way. The beauty of 3D bar plots is that they maintain the simplicity of 2D bar plots while extending their capacity to represent comparative information.https://towardsdatascience.com/an-easy-introduction-to-3d-plotting-with-matplotlib-801561999725<jupyter_code>import random

fig = plt.figure(figsize=(12, 10))
ax = plt.axes(projection="3d")

num_bars = 15
x_pos = random.sample(range(20), num_bars)
y_pos = random.sample(range(20), num_bars)
z_pos = [0] * num_bars

x_size = np.ones(num_bars)
y_size = np.ones(num_bars)
z_size = random.sample(range(20), num_bars)

ax.bar3d(x_pos, y_pos, z_pos, x_size, y_size, z_size, color='aqua')
plt.show()<jupyter_output><empty_output><jupyter_text>Checking Correlations<jupyter_code>price2.corr()
# HeatMap untuk menyelidiki korelasi
corr2 = price2.corr() # We already examined SalePrice correlations
plt.figure(figsize=(12, 10))
sns.heatmap(corr2[(corr2 >= 0.5) | (corr2 <= -0.4)], 
            cmap='viridis', vmax=1.0, vmin=-1.0, linewidths=0.1,
            annot=True, annot_kws={"size": 14}, square=True);
iris = sns.load_dataset("iris")
g = sns.pairplot(iris, hue="species")
pd.plotting.parallel_coordinates(iris, 'species', color=('r', 'g', 'b'))
plt.show()<jupyter_output><empty_output><jupyter_text>Time Series Plot<jupyter_code># Load an example dataset with long-form data
fmri = sns.load_dataset("fmri")
fmri.sample(10)
# Plot the responses for different events and regions
plot = sns.lineplot(x="timepoint", y="signal", data=fmri)
plot = sns.lineplot(x="timepoint", y="signal", hue="region", style="event", data=fmri)<jupyter_output><empty_output><jupyter_text>Spatial Visualization<jupyter_code>def generateBaseMap(default_location=[-0.789275, 113.921], default_zoom_start=5):
    base_map = folium.Map(location=default_location, control_scale=True, zoom_start=default_zoom_start)
    return base_map
# Load Data
try:
    # Running Locally, yakinkan module folium sudah terinstall
    df_loc = pd.read_csv('data/df_loc.csv')
except:
    # Running in Google Colab, yakinkan folder "data" sudah ada
    !wget -P data/ https://raw.githubusercontent.com/taudata-indonesia/eLearning/master/data/df_loc.csv
    df_loc = pd.read_csv('data/df_loc.csv')
    !pip install folium
df_loc.head()
import folium
from folium.plugins import HeatMap

base_map = generateBaseMap()
HeatMap(data=df_loc[['lat', 'lon', 'count']].groupby(['lat', 'lon']).sum().reset_index().values.tolist(), radius=8, max_zoom=13).add_to(base_map)
base_map<jupyter_output><empty_output><jupyter_text>Latihan Studi Kasus: Data Tips RestaurantSebuah dataset dari suatu Restaurant memuat variabel-variabel berikut:*	total_bill: Total bill (cost of the meal), including tax, in US dollars*	tip: Tip (gratuity) in US dollars*	sex: Sex of person paying for the meal (0=male, 1=female)*	smoker: Smoker in party? (0=No, 1=Yes)*	day: 3=Thur, 4=Fri, 5=Sat, 6=Sun*	time: 0=Day, 1=Night*	size: Size of the partyhttps://www.kaggle.com/ranjeetjain3/seaborn-tips-dataset<jupyter_code># Loading Contoh Data studi kasus pertama di atas
tips = sns.load_dataset('tips') # Loading dari SeaBorn library's dataset
# Ukuran Data
N, P = tips.shape
print('baris = ', N, ', Kolom = ', P)

tips.head()<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.1522481765303296, 'fraction_numerical': 0.01772689502354353, 'mean_word_length': 6.429355281207133, 'pattern_counts': {'":': 1, '<': 89, '<?xml version=': 0, '>': 88, 'https://': 8, 'lorem ipsum': 0, 'www.': 1, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '21', 'dir': 'jupyter-structured-clean-dedup', 'id': '147690', 'license': 'Apache-2.0', 'n_tokens_mistral': 4329, 'n_tokens_neox': 4083, 'n_words': 793, 'path': 'eda-02-codes.ipynb', 'repo_name': 'taudata-indonesia/elearning', 'size': '10850'}