<jupyter_start><jupyter_text>Book Recommendations Dataset This notebook uses the [Book Recommendations](https://www.kaggle.com/arashnic/book-recommendation-dataset) dataset available on Kaggle.   Imports<jupyter_code># TODO: Import any other modules you need
import pandas as pd
import numpy as np
import seaborn as sns
import matplotlib.pyplot as plt
# TODO: Read Books.csv, Ratings.csv and Users.csv into similary named dataframes
data1_df = pd.read_csv("Books.csv")
data1_df.head()
data1_df.shape
data1_df.info
data1_df.isnull().sum()
data2_df  = pd.read_csv("Ratings.csv")

data2_df.head()
data1_df.value_counts('Publisher')
data3_df= pd.read_csv("Users.csv")

data3_df.head()

data3_df.info
data3_df.isnull().sum()<jupyter_output><empty_output><jupyter_text>Exploratory Data Analysis (EDA)  TODO: * Use the `describe` and `info` methods on each of your dataframes. * Check the `dtypes` for the columns in each dataframe.* Check for missing values in each dataframe* Check for any other operations commonly used during EDA and perform it here<jupyter_code>#Descibe methods on each dataframe(data1_df)
data1_df.describe()
#Descibe methods on each dataframe(data2_df)
data2_df.describe()
#Descibe methods on each dataframe(data3_df)
data3_df.describe()
#Use info method on all datatypes.
data1_df.info()
#Use info method on all datatypes.
data2_df.info()
#Use info method on all datatypes.
data3_df.info()
#Check the dtypes for the column in each dataframe
data1_df.dtypes
#Check the dtypes for the column in each dataframe
data2_df.dtypes
#Check the dtypes for the column in each dataframe
data3_df.dtypes
#Check for missing values in each dataframe
data1_df.isnull().sum()
#Check for missing values in each dataframe
data2_df.isnull().sum()
#Check for missing values in each dataframe
data3_df.isnull().sum()
#check shapes of the dataframes.
data1_df.shape
#check shapes of the dataframes.
data2_df.shape
#check shapes of the dataframes.
data3_df.shape<jupyter_output><empty_output><jupyter_text>Fact-Finding You may need to apply the `groupby` method here. Here's a great introduction: [Pandas GroupBy: Your Guide to Grouping Data in Python](https://realpython.com/pandas-groupby/). TODO: * Group the `ratings` dataframe by `User-ID` and check the `average` rating by user.* Group the `ratings` dataframe by `ISBN` and check the `average` rating each book has gotten.* Create a `Series` containing the number of books each user has read* Who are the top 10 most rated authors?* Who are the top 10 highest rated authors?<jupyter_code>#ranking of Users.csv

data3_df.rank()
# 2 rows of rating
data2_df.head(2)
#Group the ratings dataframe  by USER-ID and check the average by user
data2_df.groupby('User-ID')['Book-Rating'].mean().head()
#Group the ratings dataframe  by ISBN and check the average by user
data2_df.groupby('ISBN')['Book-Rating'].mean().head(3)
# Determine number of books read by each user.
books_by_each_user=data2_df.groupby('User-ID')['ISBN'].count()
books_by_each_user
# Top 10 people who have read the most books.
books_by_each_user.sort_values(ascending=False).head(10)<jupyter_output><empty_output><jupyter_text>Engineering  TODO: * Apply the `to_datetime` method to the `Year-of-Publication` `inplace`. * Use the `drop` method to drop all the `Image` columns in the `books` dataframe.* Read this article - [How to Split String Column in Pandas into Multiple Columns](https://www.statology.org/pandas-split-column/) then apply the method to split the `location` column into three columns `city`, `state`, `country`.* Which other `str` methods are available in pandas? Try out one or two others. * Merge the `ratings` and `books` dataframe into one dataframe named `books_and_ratings`* Merge the `books_and_ratings` dataset and the `users` dataframe into a dataframe named `user_preferences`<jupyter_code>data1_df.head()
#Apply the to-datetime method to the year Year-of-publication implace
data1_df['Year-Of-Publication']=pd.to_datetime(data1_df['Year-Of-Publication'].astype(str), errors='coerce')
data1_df
#Use the drop method to drop all the image columns int the book dafaframe
data1_df.drop(columns=['Image-URL-S', 'Image-URL-M', 'Image-URL-L'])
#Apply the method to split the `location` column into three columns `city`, `state`, `country`.
data3_df[['city', 'state', 'country']] = data3_df['Location'].str.split(' ',2,expand=True)
data3_df = data3_df[['city', 'state', 'country']]
data3_df
# Which other 'str' Method are available in pandas? try out one or two
BooksCAP = data1_df['Book-Author'].str.upper().head()
Ratingslower = data1_df['Publisher'].str.islower().head()


print(BooksCAP)
print()
print(Ratingslower)

#Merge the `ratings` and `books` dataframe into one dataframe named `books_and_ratings`
books_and_ratings=pd.merge(data2_df,data1_df, on='ISBN')
books_and_ratings
data3_df= pd.read_csv("Users.csv")
data3_df.head()
#Merge the `books_and_ratings` dataset and the `users` dataframe into a dataframe named `user_preferences`
user_preferences=pd.merge(books_and_ratings,data3_df,on='User-ID')
user_preferences.head()<jupyter_output><empty_output><jupyter_text>Querying and Indexing  Here's a useful article:* [The Unreasonable Effectiveness of Method Chaining in Pandas](https://towardsdatascience.com/the-unreasonable-effectiveness-of-method-chaining-in-pandas-15c2109e3c69)  TODO:* Use the `query` method to return rows from the `user_preferences` dataframe where `User-ID` is equal to `276822`* Repeat the same operation as before but this time return rows where `ISBN` is equal to `0345260317`* Return rows where the `ISBN` is equal to `0345260317` and `Book-Rating` is greater than 0.* Repeat all the operations performed above using the `loc` method rather than `query`* Time the operation of both methods (See [Profiling and Timing Code](https://jakevdp.github.io/PythonDataScienceHandbook/01.07-timing-and-profiling.html)). Which is faster?* Use the `to_dict` method to convert the output of query to dictionary. * Write a function that takes in a `dataframe` and a `user_id` and returns a dictionary of rows with matching user ids. Test your function using the `user_preferences` dataframe and any user id in the data.<jupyter_code>#Use the `query` method to return rows from the `user_preferences` dataframe where `User-ID` is equal to `276822`
user_preferences.query('`User-ID`==276822')
#Use the `query` method to return rows from the `user_preferences` dataframe where  `ISBN` is equal to `0345260317`
user_preferences.query("`ISBN`=='0345260317'")

#Return rows where the `ISBN` is equal to `0345260317` and `Book-Rating` is greater than 0.
user_preferences.query(("`ISBN` == 0345260317 ") and ("`Book-Rating` > 0") )
#Repeat all the operations performed above using the `loc` method rather than `query`

user_preferences.loc[user_preferences['User-ID'] == 276822]
user_preferences.loc[user_preferences.ISBN == '0345260317']
user_preferences.loc[(user_preferences['ISBN'] == '0345260317') &(user_preferences['Book-Rating'] > 0)]



%%timeit
user_preferences.query("`User-ID`==276822")
%%timeit
user_preferences.query("ISBN =='0345260317'")
%%timeit
user_preferences.loc[user_preferences['User-ID'] == 276822]
%%timeit
user_preferences.loc[user_preferences.ISBN=='0345260317']
%%timeit
rate = user_preferences['Book-Rating']
user_preferences.query("(ISBN=='0345260317') and @rate> 0")
%%timeit
user_preferences.loc[(user_preferences['ISBN'] == '0345260317') & (user_preferences['Book-Rating']>0)]
#Use the `to_dict` method to convert the output of query to dictionary.
user_preferences.query("`User-ID`==276822").to_dict()
user_preferences.query("ISBN =='0345260317'").to_dict()
user_preferences.query(("`ISBN` == 0345260317 ") and ("`Book-Rating` > 0") ).to_dict()
#function that takes in a `dataframe` and a `user_id` and returns a dictionary of rows with matching user ids
#Test your function using the `user_preferences` dataframe and any user id in the data.
def dict_pands(dataframe:None, user_id:None)->dict:
    pand = dataframe.loc[dataframe['User-ID']==user_id].to_dict()
    return pand

dict_pands(user_preferences,152435 )<jupyter_output><empty_output><jupyter_text>Hierarchical Indexing Study the [Hierarchical Indexing](https://jakevdp.github.io/PythonDataScienceHandbook/03.05-hierarchical-indexing.html) chapter from Python Data Science Handbook.  TODO* Include 3 or 4 of your observations in the `Comments` below after exploring the dataset<jupyter_code>1. The highest book rating was 10 while the lowest was 0
2.At Number 271359 <NAME>'s  A Guided Tour of Rene Descartes' Meditations o...	 published 2000-01-01 was the last book on the recomendation list.
3. Harlequin with 7535 books is the hihgest publisher on the recomendation list.<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.11123813879044243, 'fraction_numerical': 0.034526123242254486, 'mean_word_length': 6.229752066115703, 'pattern_counts': {'":': 0, '<': 26, '<?xml version=': 0, '>': 32, 'https://': 6, 'lorem ipsum': 0, 'www.': 2, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '6', 'dir': 'jupyter-structured-clean-dedup', 'id': '481488', 'license': 'MIT', 'n_tokens_mistral': 3052, 'n_tokens_neox': 2826, 'n_words': 914, 'path': '3_Introduction_to_Pandas/dara_obademi.ipynb', 'repo_name': 'Jayozone1/cohort7_practicals', 'size': '8759'}