<jupyter_start><jupyter_text>Exploring Netflix Data This notebook follows the [CRISP-DM](https://paginas.fe.up.pt/~ec/files_0405/slides/02%20CRISP.pdf) process to analyze the [Netflix Movies and TV Shows on Kaggle.](https://www.kaggle.com/shivamb/netflix-shows)This dataset consists of tv shows and movies available on Netflix as of 2019. The dataset is collected from Flixable which is a third-party Netflix search engine.In 2018, they released an interesting [report](https://flixable.com/netflix-museum/) which shows that the number of TV shows on Netflix has nearly tripled since 2010. The streaming service’s number of movies has decreased by more than 2,000 titles since 2010, while its number of TV shows has nearly tripled. It will be interesting to explore what all other insights can be obtained from the same dataset.Integrating this dataset with other external datasets such as IMDB ratings, rotten tomatoes can also provide many interesting findings.  Business UnderstandingThese are the guiding questions that drive this analysis* In what month of the year, does the most new content arrives on netflix?* What is the average delay between release of a show and its arrival on netflix?* How does the different genres relate to the target audience belonging to different age groups?* Which Country has most content available on netflix?  Data Understanding  Setup<jupyter_code>import matplotlib.pyplot as plt
import pandas as pd
import seaborn as sns

from  matplotlib.ticker import FuncFormatter

from collections import Counter

%matplotlib inline
pd.set_option('display.max_columns', 100)
pd.set_option('display.max_rows', 100)<jupyter_output><empty_output><jupyter_text>As the data is already collected for us by Flixable, and made accessible by Kaggle, we move onto next steps in the CRISP-DM cycle.  Data Reading<jupyter_code>df = pd.read_csv('netflix_titles.csv')
df.head()<jupyter_output><empty_output><jupyter_text>Getting to know Data First we will have a look at the total number of rows and column in the data<jupyter_code>df.shape<jupyter_output><empty_output><jupyter_text>The data types for every column can be found as:<jupyter_code>df.info()<jupyter_output><class 'pandas.core.frame.DataFrame'>
RangeIndex: 7787 entries, 0 to 7786
Data columns (total 12 columns):
 #   Column        Non-Null Count  Dtype 
---  ------        --------------  ----- 
 0   show_id       7787 non-null   object
 1   type          7787 non-null   object
 2   title         7787 non-null   object
 3   director      5398 non-null   object
 4   cast          7069 non-null   object
 5   country       7280 non-null   object
 6   date_added    7777 non-null   object
 7   release_year  7787 non-null   int64 
 8   rating        7780 non-null   object
 9   duration      7787 non-null   object
 10  listed_in     7787 non-null   object
 11  description   7787 non-null   object
dtypes: int64(1), object(11)
memory usage: 730.2+ KB<jupyter_text>Below we can see the total missing values in each of the fields.<jupyter_code>df.isna().sum()<jupyter_output><empty_output><jupyter_text>Data Wrangling First we will convert the string column to datetime and separate out the month and year columns for our analysis.For this analysis, we will remove any rows with missing values for the *date_added* column.<jupyter_code>df.dropna(subset=['date_added'], inplace=True)
df['date_added'] = pd.to_datetime(df['date_added'])
df['added_year']= df['date_added'].dt.year
df['added_month']= df['date_added'].dt.month_name().str.slice(stop=3)

df.head()<jupyter_output><empty_output><jupyter_text>Finding Results  In what month of the year, does the most new content arrives on netflix?<jupyter_code>hue_order = ['Jan', 'Feb', 'Mar', 'Apr', 'May', 'Jun', 'Jul', 'Aug', 'Sep', 'Oct', 'Nov', 'Dec']

def plot_month_wise_data(data, hue=None, hue_order=None):
    """
    Plots Number of media items added per month to netflix
    
    :param data: dataframe
    :param hue: list
    :param hue_order: string
    :return: None
    """
    
    plt.figure(figsize=(8 , 8))
    ax = sns.countplot(data=data, x='added_month', order=hue_order, hue=hue)
    ax.set(xlabel='Months', ylabel='Number of items added', title='Media Added per Month on Netflix')
    plt.show()

plot_month_wise_data(df, hue_order=hue_order)<jupyter_output><empty_output><jupyter_text>The above graph shows the arrival of content each month on netflix. For a better comparison, let's create another graph that displays data for movies and TV shows separately.<jupyter_code>plot_month_wise_data(df, hue_order=hue_order, hue='type')<jupyter_output><empty_output><jupyter_text>From the above graphs, it can be seen that most content gets uploaded to netlflix in December followed by October and then January. This can lead us to conclusion that Netflix adds new data near holiday season.  What is the average delay between release of a show and its arrival on netflix. Let's start with finding the number of shows released each year and the numberof shows added per year in our data.<jupyter_code>release_year = df.release_year.value_counts()
added_year = df.added_year.value_counts()
def create_line_plot_between_added_released_year(rel_year, add_year):
    """
    Plots a line graph between the added and released year data
    :param rel_year: series
    :param add_year: series
    :return: None
    """
    plt.figure(figsize=(15 , 6))
    ax = sns.lineplot(data=rel_year)
    ax.xaxis.set_major_formatter(FuncFormatter(lambda x, _: int(x)))
    sns.lineplot(data=add_year)
    plt.legend(['Release Year', 'Added Year'])
    plt.show()

create_line_plot_between_added_released_year(release_year, added_year)<jupyter_output><empty_output><jupyter_text>As we can see from above graph, Netflix had media that got released in late 1940s, but this graph does not provides any meaningful insight due to the fact of the large difference between oldest data available and release on netflix. Therefore, we take into account only the media added or released during last 2 decades.<jupyter_code>create_line_plot_between_added_released_year(release_year[:20], added_year)<jupyter_output><empty_output><jupyter_text>Since there are outliers in the release data, mean might not be the most accurate measure to use here. We therefore plot the median of difference between movies released and added every year since 2000. In easy words, This graph will tell us the difference of how many years it took on average for a media released in that year to arrive on netflix.<jupyter_code>diff_med  = {}

for year in range(2000, 2021):
    rows = df[df['release_year'] == year]
    dif = rows['added_year'].median() - year
    diff_med[year] = dif

diff_median = pd.Series(diff_med)
diff_median
plt.figure(figsize=(12 , 8))
ax = sns.lineplot(data=diff_median, marker='o')
ax.xaxis.set_major_formatter(FuncFormatter(lambda x, _: int(x)))
ax.set(xlabel='Year of Release', ylabel='Average difference in years', title='Median of difference between release and added date of media')
plt.show()<jupyter_output><empty_output><jupyter_text>How does the different genres relate to the target audience belonging to different age groups?As we can see from the data, that a media might belong to multiple genres. For reference, we print the first 10 members of the `listed_in` parameter. And then we proceed on to finding all the unique genres.<jupyter_code>all_genres = df['listed_in'].unique()
all_genres[:10]
genres_set = set()
for gen in all_genres:
    separate_genres = [word.strip() for word in gen.split(',')]
    genres_set.update(separate_genres)

genres = list(genres_set)
genres<jupyter_output><empty_output><jupyter_text>We then try to find out the total number of shows belonging to each unique genre.<jupyter_code>all_genre_list = df['listed_in'].str.split(',').explode().str.strip().to_list()
genre_counts = Counter(all_genre_list)
genre_counts<jupyter_output><empty_output><jupyter_text>As we can see, this data is unsorted and is difficult to understand in first glance, we sort the genres in the ascending order, and then plot a bar graph for visual representation.<jupyter_code>sorted_genres = dict(sorted(genre_counts.items(), key=lambda item: item[1], reverse=True))
sorted_genres
plt.figure(figsize=(10 , 10))
plt.bar(sorted_genres.keys(), sorted_genres.values())
plt.xticks(rotation='vertical')
plt.xlabel('Genre')
plt.ylabel('Count of Shows')
plt.title('Count of Shows per Genre')
plt.show()<jupyter_output><empty_output><jupyter_text>Since there are total of 42 unique genres, We pick the top 5 most popular genres and plot the various genres the shows belong to.<jupyter_code>top_five = list(sorted_genres.keys())[:5]
top_five
df['rating'].value_counts()
for top in top_five:
    plt.figure(figsize=(4 , 4))
    rating_records = df[df['listed_in'].str.contains(top)]
    ax = sns.countplot(data=rating_records, x='rating')
    ax.set(xlabel='Rating', ylabel='Number of movies/shows', title=top)
    plt.xticks(rotation='vertical')
    plt.show()<jupyter_output><empty_output><jupyter_text>Which Country has most content available on netflix?For this comparison, we being by dropping records that do not have country data available. Furthermore, we also pick the first country where multiple countries are listed for a better graphical approach.Finally, we plot a pie chart of the top 20 countries with percent of data available. We are not including all countries as the graph becomes difficult to understand.<jupyter_code>df['country'].value_counts()
df.dropna(subset=['country'], inplace=True)
df['country'] = df.country.str.split(',').str[0]
plt.figure(figsize=(15 , 8))
ax = sns.countplot(data=df, x='country')
plt.xticks(rotation='vertical')
plt.show()<jupyter_output><empty_output><jupyter_text>As evident from above, the United States is a clear winner. However, there are se many countries that it is very hard to read the graph. For a clearer representation, we take the top 20 countries and display pie chart with the precent of data for each country.<jupyter_code>df['country'].value_counts()[:20].plot.pie(autopct='%1.1f%%',figsize=(10,10));<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.09247927358863009, 'fraction_numerical': 0.017469403868930124, 'mean_word_length': 5.1561360874848114, 'pattern_counts': {'":': 0, '<': 69, '<?xml version=': 0, '>': 69, 'https://': 3, 'lorem ipsum': 0, 'www.': 1, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '17', 'dir': 'jupyter-structured-clean-dedup', 'id': '118008', 'license': 'MIT', 'n_tokens_mistral': 3229, 'n_tokens_neox': 3031, 'n_words': 1095, 'path': 'netflix_analysis.ipynb', 'repo_name': 'Ramshah1/Netflix-Data-Analysis', 'size': '10132'}