<jupyter_start><jupyter_text>Content-Based Filtering model  Content-based filtering approaches leverage **description or attributes** from items the user has interacted to recommend similar items. It depends only on the user **previous choices**, making this method robust to **avoid the cold-start problem**. For textual items, like articles, news and books, it is simple to use the article **category** or **raw text** to build **item profiles** and **user profiles**.Suppose I watch a particular genre movie I will be recommended movies w.r.t that specific genre. The Title, Year of Release, Director, Cast are also helpful in identifying similar movie content.  Approach 2: Using Rated Content to RecommendIn this approach contents of the product are already **rated** and based on the **user’s preference**. An **item score** is predicted to products and recommendation can be made. Usually `rating table` (user rating), `item profile` (book genres) are the only material we've got.* `rating table`: user-to-book relationship* `item profile`: attribute-to-book relationship  ![](https://www.analyticsvidhya.com/wp-content/uploads/2015/08/81.png) Then we will create the `user profile` so that we can understand what attribute the users actually prefer.* `user profile`: user-to-attribute relationship  ![](https://www.analyticsvidhya.com/wp-content/uploads/2015/08/91.png) Thus, with the `user profile`, we can get all the item score which is the user preference from `user profile` and `item profile`.  ![](https://www.analyticsvidhya.com/wp-content/uploads/2015/08/121.png) Let's go through the code!<jupyter_code>import pandas as pd
import numpy as np
from tqdm import tqdm
from . import Data_cleaning
#import book csv with description, and more than 10 user ratings
df_n_des = pd.read_csv('books_n_description.csv', index_col=0, names=['isbn', 'title', 'author', 'pub_year', 'publisher', 'category','description'], skiprows=1)
df_n_des.head()
df_n_des.shape
#make a copy to books_wd which categories cell is not null
books_wd = df_n_des[df_n_des['category'].notnull()].copy()

# filter out books with less than 5 characters in categories
books_wd = books_wd[books_wd['category'].map(len) >1]
books_wd.shape<jupyter_output><empty_output><jupyter_text>We have **1,466** books in total  Item profile<jupyter_code>df_item = books_wd[['isbn','category']]
df_item.head()
# one-hot encoding for category
df_genre = pd.get_dummies(df_item['category'])<jupyter_output><empty_output><jupyter_text>let’s treat all articles as having unit weight.  For binary representation, we can perform normalization by dividing the term occurrence by the sqrt of number of attributes in the article.<jupyter_code>#normalized
df_genre_normalized = df_genre.apply(lambda x: x/np.sqrt(df_genre.sum(axis=1)))
#create item profile
df_item = pd.concat([df_item, df_genre_normalized], axis=1)

df_item.drop(columns='category', inplace=True)
df_item.sort_values('isbn', inplace=True)
df_item.set_index('isbn', inplace=True)
df_item.head()
df_item.shape<jupyter_output><empty_output><jupyter_text>In `df_item`, there are **1,466** books and **158** genres. Let's load the merged dataframe of `books`, `ratings`, `users` and get the record which matched the 1,466 books in `df_item`.<jupyter_code>df_merged = get_merged_data_frame(path='../data/')
df_merged.head()
#top users number and their number of ratings given
df_merged.user.value_counts().sort_values(ascending=False)[:10]
df_final = df_merged[df_merged['isbn'].isin(df_item.index)]
#create isbn and category dict
isbn_cat_dict = pd.Series(books_wd['category'].values,index=books_wd['isbn']).to_dict()

#create isbn and description dict
isbn_des_dict = pd.Series(books_wd['description'].values,index=books_wd['isbn']).to_dict()
df_final['category'] = df_final['isbn'].map(isbn_cat_dict)
df_final['description'] = df_final['isbn'].map(isbn_des_dict)
df_final.shape
df_final = df_final[['user','rating','isbn','title','author','publisher','pub_year','category','description']]
df_final.head()<jupyter_output><empty_output><jupyter_text>There are **72,821** ratings of the **1,466** books.<jupyter_code>user_ratings = df_final.user.value_counts()
# Let's find which users have 10 ratings in df_final
user_ratings[user_ratings == 10]<jupyter_output><empty_output><jupyter_text>Rating dataframe<jupyter_code>rating = pd.pivot_table(df_final, values='rating', index=['isbn'], columns = ['user'])
rating.sort_index(axis=1, inplace=True)
rating.head()<jupyter_output><empty_output><jupyter_text>User Profile<jupyter_code># get all the user number
users_no = rating.columns
#create an empty dataframe
df_users = pd.DataFrame(columns = df_item.columns)
df_users
for i in tqdm(range(len(users_no))):
    working_df = df_item.mul(rating.iloc[:,i], axis=0)
    working_df.replace(0, np.NaN, inplace=True)    
    df_users.loc[users_no[i]] = working_df.mean(axis=0)
#user profile
df_users.head()<jupyter_output><empty_output><jupyter_text>IDFLet’s consider how common different terms are among our documents.  The dot product of article vectors and IDF vectors gives us the **weighted scores** of each article.<jupyter_code>document_frequency = df_item.sum()
idf = (len(books_wd)/document_frequency).apply(np.log) #log inverse of DF
#The dot product of article vectors and IDF vectors gives us the weighted scores of each article.
idf_df_item = df_item.mul(idf.values)<jupyter_output><empty_output><jupyter_text>Make Prediction<jupyter_code>#make an empty dataframe
df_predict = pd.DataFrame()
#user predict by tfidf
for i in tqdm(range(len(users_no))):
    working_df = idf_df_item.mul(df_users.iloc[i], axis=1)
    df_predict[users_no[i]] = working_df.sum(axis=1)
#all user score predict of all books
df_predict.head()
def recommender(user_no):
    
    #get all book isbn
    isbn_no = df_predict.index

    #user predicted rating to all books
    user_predicted_rating = df_predict[user_no]

    #combine book rating and book detail
    user_rating_book = pd.concat([user_predicted_rating,books_wd.set_index('isbn')], axis=1)

    #books already read by user
    already_read = df_final[df_final['user'].isin([user_no])]['isbn']

    #recommendation without books being read by user
    all_rec = user_rating_book[~user_rating_book.index.isin(already_read)]

    return all_rec.sort_values(by=[user_no], ascending=False).iloc[0:10]
recommender('242409')
#books Read by user 242409
df_final[df_final['user'].isin(['242409'])].sort_values(by=['rating'], ascending=False)<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.12176046618616776, 'fraction_numerical': 0.014108265603435055, 'mean_word_length': 6.663924794359577, 'pattern_counts': {'":': 0, '<': 37, '<?xml version=': 0, '>': 38, 'https://': 3, 'lorem ipsum': 0, 'www.': 3, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '9', 'dir': 'jupyter-structured-clean-dedup', 'id': '71009', 'license': 'MIT', 'n_tokens_mistral': 2202, 'n_tokens_neox': 2098, 'n_words': 584, 'path': 'content_based_filtering/content_based_recommender_approach2_v2.ipynb', 'repo_name': 'youonf/recommendation_system', 'size': '6521'}