<jupyter_start><jupyter_text>Programming Assignment: Data Science Basics In this programming assignment you need to apply your new `numpy`, `pandas` and `matplotlib` knowledge. You will need to do several [`groupby`](https://pandas.pydata.org/pandas-docs/stable/generated/pandas.DataFrame.groupby.html)s and [`join`](https://pandas.pydata.org/pandas-docs/stable/generated/pandas.DataFrame.join.html)`s to solve the task.<jupyter_code># Import Packages
import pandas as pd
import numpy as np
import os
import matplotlib.pyplot as plt
%matplotlib inline<jupyter_output><empty_output><jupyter_text>Pandas allows you to load the required data sets directly from github:<jupyter_code># Load Data
transactions    = pd.read_csv('https://github.com/NikoStein/pds_data/raw/main/data/sales_train.csv.gz')
items           = pd.read_csv('https://github.com/NikoStein/pds_data/raw/main/data/items.csv')
item_categories = pd.read_csv('https://github.com/NikoStein/pds_data/raw/main/data/item_categories.csv')<jupyter_output><empty_output><jupyter_text>Get to know the dataPrint the **shape** of the loaded dataframes and use [`df.head`](https://pandas.pydata.org/pandas-docs/stable/generated/pandas.DataFrame.head.html) function to print several rows. Examine the features you are given.<jupyter_code># Write your code here<jupyter_output><empty_output><jupyter_text>Maximum total revenue among all the shopsNow use your `pandas` skills to get answers for the following questions. The first question is:What was the maximum total revenue among all the shops in July, 2013? * Hereinafter *revenue* refers to total sales minus value of goods returned.*Hints:** Sometimes items are returned, find such examples in the dataset. * It is handy to split `date` field into [`day`, `month`, `year`] components and use `df.year == 13` and `df.month == 6` in order to select target subset of dates.* You may work with `date` feature as with strings, or you may first convert it to `pd.datetime` type with `pd.to_datetime` function, but do not forget to set correct `format` argument.<jupyter_code># Write your code here
max_revenue = # Save the final value in this variable<jupyter_output><empty_output><jupyter_text>In addition we can find the index (shop id) of the row with the max value in the column using ```idxmax()```<jupyter_code># Write your code here<jupyter_output><empty_output><jupyter_text>Constant priceHow many items are there, such that their price stays constant (to the best of our knowledge) during the whole period of time?* Let's assume, that the items are returned for the same price as they had been sold.<jupyter_code># Write your code here
num_items_constant_price = # Save the final value in this variable<jupyter_output><empty_output><jupyter_text>Variance of sold items per dayWhat was the variance of the number of sold items per day sequence for the shop with `shop_id = 25` in December, 2014?* Do not count the items that were sold but returned back later.* Fill `total_num_items_sold`: An (ordered) array that contains the total number of items sold on each day * Then compute variance of the of `total_num_items_sold`* If there were no sales at a given day, ***do not*** impute missing value with zero, just ignore that day<jupyter_code># Write your code here
total_num_items_sold = # Save the final value in this variable
total_num_items_sold_var = # Save the final value in this variable<jupyter_output><empty_output><jupyter_text>Vizualization of the daily revenueUse `total_num_items_sold` to plot the daily revenue of `shop_id = 25` in December, 2014.* y-label: 'Num items'* x-label: 'Day'* plot-title: 'Daily revenue for shop_id = 25'* use plt.show() to display the plot in the end<jupyter_code># Write your code here

plt.show()<jupyter_output><empty_output><jupyter_text>Final submissionOnly for the submission (do not change the code).<jupyter_code>print(max_revenue)
print(num_items_constant_price)
print(total_num_items_sold)
print(total_num_items_sold_var)<jupyter_output><empty_output><jupyter_text>BonusWhat item category that generated the highest revenue in spring 2014?* Submit the `id` of the category found.    * Here, spring is the period from March to Mai.<jupyter_code># Write your code here
category_id_with_max_revenue = # Save the final value in this variable
print(category_id_with_max_revenue)<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.08692660550458715, 'fraction_numerical': 0.005733944954128441, 'mean_word_length': 5.988782051282051, 'pattern_counts': {'":': 0, '<': 41, '<?xml version=': 0, '>': 41, 'https://': 6, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '10', 'dir': 'jupyter-structured-clean-dedup', 'id': '501659', 'license': 'MIT', 'n_tokens_mistral': 1356, 'n_tokens_neox': 1316, 'n_words': 489, 'path': 'assignments/01_PA_Data Science Basic.ipynb', 'repo_name': 'princessivy/course', 'size': '4360'}