<jupyter_start><jupyter_text>câu hỏi 1: đọc dữ liệu 2019 từ file parquet<jupyter_code>source_data_path = "challenge2/"
partition = "Year=2019"
full_path = f"{source_data_path}{partition}"
sale2019_df = spark.read.parquet(full_path)
sale2019_df.show()<jupyter_output>+--------------------+--------------------+--------------+-----+--------+------------+---------+-------------------+-----+
|             Product|        StoreAddress|          City|State|Order_ID|Quantity_pro|Price_pro|         Order_Date|Month|
+--------------------+--------------------+--------------+-----+--------+------------+---------+-------------------+-----+
|  Macbook Pro Laptop|136 Church St, Ne...| New York City|   NY|  295665|           1|     1700|2019-12-30 00:01:00|   12|
|  LG Washing Machine|562 2nd St, New Y...| New York City|   NY|  295666|           1|      600|2019-12-29 07:03:00|   12|
|USB-C Charging Cable|277 Main St, New ...| New York City|   NY|  295667|           1|       11|2019-12-12 18:21:00|   12|
|    27in FHD Monitor|410 6th St, San F...| San Francisco|   CA|  295668|           1|      149|2019-12-22 15:13:00|   12|
|USB-C Charging Cable|43 Hill St, Atlan...|       Atlanta|   GA|  295669|           1|       11|2019-12-18 12:38:00|   12|
|AA Batteries (4[...]<jupyter_text>câu hỏi 2 là: tháng nào thuận lợi nhất để sale<jupyter_code>!pip uninstall seaborn
!conda install seaborn=0.11
!pip install pandas
!pip install matplotlib
from pyspark.sql.functions import round, sum
import pandas as pd
import matplotlib.pyplot as plt
import seaborn as sns
sale_df = (sale2019_df.select(
                            col("Order_ID"),
                            col("Month"),
                            col("Price_pro"),
                            col("Quantity_pro"),
                            expr("Price_pro * Quantity_pro").alias("Sales")))
sale_df.show(10)<jupyter_output>+--------+-----+---------+------------+-----+
|Order_ID|Month|Price_pro|Quantity_pro|Sales|
+--------+-----+---------+------------+-----+
|  295665|   12|     1700|           1| 1700|
|  295666|   12|      600|           1|  600|
|  295667|   12|       11|           1|   11|
|  295668|   12|      149|           1|  149|
|  295669|   12|       11|           1|   11|
|  295670|   12|        3|           1|    3|
|  295671|   12|       11|           1|   11|
|  295672|   12|       11|           2|   22|
|  295673|   12|       99|           1|   99|
|  295674|   12|        2|           4|    8|
+--------+-----+---------+------------+-----+
only showing top 10 rows<jupyter_text>thấy rằng ta cần so sánh tháng thì phải xem xét tháng<jupyter_code>sale_df = (sale_df
           .groupBy("Month")
           .agg(round(sum("Sales"),2).alias("Total Sale"))
           .orderBy("Total Sale",ascending=False))
sale_df.show()<jupyter_output>+-----+----------+
|Month|Total Sale|
+-----+----------+
|   12|   4591824|
|   10|   3719205|
|    4|   3374951|
|   11|   3184394|
|    5|   3138287|
|    3|   2794068|
|    7|   2635443|
|    6|   2566187|
|    8|   2234194|
|    2|   2191696|
|    9|   2087435|
|    1|   1805316|
+-----+----------+<jupyter_text>nhìn qua ta thấy ngay được rằng tháng 12 là tháng mà nên sale nhất<jupyter_code>pandas_df = (sale_df.orderBy("Month")).toPandas()
pandas_df
plt.figure(figsize=(10,6))
plt.ticklabel_format(useOffset=False,style="plain")
sns.set_theme(style="whitegrid")
sns.barplot(x="Month",y="Total Sale",data=pandas_df).set_title("total sales per month")
plt.ylabel("Total Sale")
plt.show()<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.23693871787630613, 'fraction_numerical': 0.11776334368822367, 'mean_word_length': 1.647234678624813, 'pattern_counts': {'":': 0, '<': 14, '<?xml version=': 0, '>': 14, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '4', 'dir': 'jupyter-structured-clean-dedup', 'id': '423077', 'license': 'MIT', 'n_tokens_mistral': 1558, 'n_tokens_neox': 1299, 'n_words': 274, 'path': 'Section4/challenge3.ipynb', 'repo_name': 'dotrungkien3210/spark3', 'size': '3541'}