<jupyter_start><jupyter_text>Project: Investigate a TMDb movie data Table of ContentsIntroductionData WranglingExploratory Data AnalysisConclusions  IntroductionThis data set contains information about 10,000 movies collected from The Movie Database (TMDb), including user ratings and revenue. Questions- Which genres are most popular from year to year?- What kinds of properties are associated with movies that have high revenues?<jupyter_code>#importing first all necessary libraries
import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
import seaborn as sns
%matplotlib inline<jupyter_output><empty_output><jupyter_text>Data Wrangling General Properties<jupyter_code># Load your data and print out a few lines
df = pd.read_csv("tmdb-movies.csv")
df.head()
#discover types of each variable and the missings values
df.info()
#number of columns and rows in the data
df.shape
#discover the features of each column 
df.describe()<jupyter_output><empty_output><jupyter_text>Data Cleaning<jupyter_code># First make a copy of my dataset
df_clean = df.copy()<jupyter_output><empty_output><jupyter_text>- Drop unnecessary columns (imdb_id, cast, homepage, tagline, keywords, overview, director)<jupyter_code>df_clean = df_clean.drop(["imdb_id", "cast", "homepage", "tagline", "keywords", "overview", "director"], axis=1)
#drop any nan values
df_clean.dropna(inplace=True)
#reset the index of rows
df_clean.reset_index(drop=True, inplace=True)<jupyter_output><empty_output><jupyter_text>- Drop duplicates rows<jupyter_code>#detect the numbers of duplicated rows
df_clean.duplicated().sum()
#drop any duplicate row
df_clean.drop_duplicates(inplace=True)
#then resst the index of the rows
df_clean.reset_index(drop=True, inplace=True)
df_clean.info()
#discover the charactristics of each variable 
df_clean.hist(figsize=(20,16));<jupyter_output><empty_output><jupyter_text>- Replace values equals 0 in columns ("budget", "revenue") with the mean<jupyter_code>#see first the rows that contain budget and revenue values of 0
df_clean.loc[(df_clean["budget"]==0) & (df_clean["revenue"]==0)]
#replace all values equals zero in budget and revenue columns with the mean value
df_clean["budget"] = df_clean["budget"].replace(0, df["budget"].mean())
df_clean["revenue"] = df_clean["revenue"].replace(0, df["revenue"].mean())
df_clean.loc[(df_clean["budget"]==0) & (df_clean["revenue"]==0)]<jupyter_output><empty_output><jupyter_text>- Convert "release_date" to datetime<jupyter_code>df_clean["release_date"] = pd.to_datetime(df_clean["release_date"])
df_clean.info()<jupyter_output><class 'pandas.core.frame.DataFrame'>
RangeIndex: 9826 entries, 0 to 9825
Data columns (total 14 columns):
 #   Column                Non-Null Count  Dtype         
---  ------                --------------  -----         
 0   id                    9826 non-null   int64         
 1   popularity            9826 non-null   float64       
 2   budget                9826 non-null   float64       
 3   revenue               9826 non-null   float64       
 4   original_title        9826 non-null   object        
 5   runtime               9826 non-null   int64         
 6   genres                9826 non-null   object        
 7   production_companies  9826 non-null   object        
 8   release_date          9826 non-null   datetime64[ns]
 9   vote_count            9826 non-null   int64         
 10  vote_average          9826 non-null   float64       
 11  release_year          9826 non-null   int64         
 12  budget_adj            9826 non-null   float64       
 13  revenue_adj       [...]<jupyter_text>- Deal with "geners" and "production_companies" columns<jupyter_code>df_clean["genres"].value_counts()
df_clean["production_companies"].value_counts()<jupyter_output><empty_output><jupyter_text>- Reduce the values in the rows of "genres" and "production_companies" columns that have multiple values<jupyter_code>#replace the cells that conatins multiple values with teh first value in them
df_clean["genres"] = df_clean["genres"].apply(lambda x: x.split("|")[0])
df_clean["production_companies"] = df_clean["production_companies"].apply(lambda x: x.split("|")[0])
df_clean["production_companies"].value_counts()<jupyter_output><empty_output><jupyter_text>Exploratory Data Analysis Question 1: Which genres are most popular from year to year?<jupyter_code># get first popularity of each genre 
genre_popularity = df_clean.groupby(["genres"])["popularity"].mean()
genre_popularity
plt.subplots(figsize=(30,10))
plt.bar(genre_popularity.index, genre_popularity)
plt.title("Popularity of genres among years")
plt.xlabel("Genres")
plt.ylabel("Popularity");<jupyter_output><empty_output><jupyter_text>From the above gragh we observe that advanture movies is the most popular genre  Question 2: What kinds of properties are associated with movies that have high revenues?<jupyter_code>#start by sorting revenues
sorted_revenues = df_clean.sort_values(by=["revenue"], ascending=False)
sorted_revenues
sorted_revenues.hist(figsize=(20,10));<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.10292374555511656, 'fraction_numerical': 0.025879099170288425, 'mean_word_length': 4.509249183895538, 'pattern_counts': {'":': 0, '<': 45, '<?xml version=': 0, '>': 45, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '11', 'dir': 'jupyter-structured-clean-dedup', 'id': '409819', 'license': 'MIT', 'n_tokens_mistral': 1692, 'n_tokens_neox': 1565, 'n_words': 433, 'path': 'investigate-a-dataset-template.ipynb', 'repo_name': 'YasminaaAyman/Investigate-tmdb_movies-dataset', 'size': '5062'}