<jupyter_start><jupyter_text>Names (e.g. title, author) are objects, values are ints or floats.<jupyter_code># inspect first values of df
books[:5]
# number of books and columns
books.shape
# check if image urls are working
books["image_url"][:5].to_list()
books.describe()<jupyter_output><empty_output><jupyter_text>There are 10,000 entries for books. Interestingly, the original publication years reach from 2017 to -1750. Latter sounds like an error. The best rating for a book is 4.8 and the worst 2.47. Not too bad. The book with the most ratings has 4.7 million ratings, the book with the fewest ratings only 2,716 ratings.Inspect the ratings. What is their distribution?<jupyter_code>books[books["original_publication_year"] == -1750]["title"]<jupyter_output><empty_output><jupyter_text>Quick search for this title shows that this book really exists and was written a long time ago.<jupyter_code>show_columns_and_types(ratings)
ratings[:4]<jupyter_output>column name; column type;
{'user_id': 'int64', 'book_id': 'int64', 'rating': 'int64'}<jupyter_text>Ratings dataframe contains entries with a user id, the corresponding book id and a rating.<jupyter_code># check if any null/nan values are in the data for the ratings
ratings.isnull().sum()
ratings.describe()<jupyter_output><empty_output><jupyter_text>Ratings are within the range of 1-5.<jupyter_code>len(ratings.groupby(["user_id", "book_id"]).size().reset_index()) == ratings.shape[0]<jupyter_output><empty_output><jupyter_text>No duplicate ratings for a user - book combination.<jupyter_code># inspect the distribution of ratings
sns.displot(data=ratings["rating"], discrete=True)
# group ratings by user
ratings_per_user = ratings.groupby(["user_id"]).count()["rating"].reset_index(name="rating_counts")

print("There are {} users in the dataset.".format(ratings.groupby(["user_id"]).count().shape[0]))
# inspect the distribution of ratings per user
sns.displot(data=ratings_per_user, x="rating_counts", binwidth=10)<jupyter_output><empty_output><jupyter_text>Most of the users have around 100 ratings. Maximum number of ratings is 200.<jupyter_code># inspect the distribution of ratings per book
ratings_per_book = ratings.groupby(["book_id"]).count()["rating"].reset_index(name="rating_counts")
book_hist = sns.displot(data=ratings_per_book, x="rating_counts", binwidth=100)
book_hist.set(xlim=(0, 2000))
plt.show()
# how many books have more than 2000 ratings
ratings_per_book[ratings_per_book["rating_counts"]>2000].count()
# most rated books
ratings_per_book[ratings_per_book["rating_counts"]>2000].sort_values(["rating_counts"], ascending=False)
print("Top 10 books have {} ratings in total.".format(ratings_per_book[ratings_per_book["rating_counts"]>2000].sort_values(["rating_counts"], ascending=False).iloc[:10].sum()["rating_counts"]))
ratings_per_book["rating_counts"].describe()
show_columns_and_types(book_tags)
show_columns_and_types(tags)
book_tags[:4]
tags[:50]<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.12546249579549276, 'fraction_numerical': 0.026908846283215607, 'mean_word_length': 8.29375, 'pattern_counts': {'":': 0, '<': 28, '<?xml version=': 0, '>': 31, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '7', 'dir': 'jupyter-structured-clean-dedup', 'id': '453860', 'license': 'MIT', 'n_tokens_mistral': 1034, 'n_tokens_neox': 969, 'n_words': 247, 'path': 'data_inspection.ipynb', 'repo_name': 'christofranz/book-recommendation-engine', 'size': '2973'}