<jupyter_start><jupyter_text>Learning Objectives:1. Reading files2. Exploring the read dataframe3. Checking the dataframe info4. Merging the two dataframes into one5. Defining questions for the analysis6. Cleaning Steps<jupyter_code>import numpy as np
import pandas as pd<jupyter_output><empty_output><jupyter_text>Reading the files<jupyter_code>movies_df = pd.read_csv("../data/movies.csv")
credits_df = pd.read_csv("https://raw.githubusercontent.com/harshitcodes/tmdb_movie_data_analysis/master/tmdb-5000-movie-dataset/tmdb_5000_credits.csv")<jupyter_output><empty_output><jupyter_text>Exploring the read dataframe* Look at samples rows* Columns and shape of the dataframe* Check if you can merge the files* Understanding the type of questions that we can answer using the data.* Define the cleaning steps - using pandas* Start looking for answers<jupyter_code>movies_df.head()
movies_df.sample(5)
movies_df.shape
credits_df.head()<jupyter_output><empty_output><jupyter_text>Checking the dataframe info<jupyter_code>movies_df.info()
credits_df.info()
movies_df.head()
credits_df.head()<jupyter_output><empty_output><jupyter_text>Merging the two dataframes<jupyter_code>movies_df = pd.merge(movies_df, credits_df, left_on='id', right_on='movie_id')
movies_df.head()<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.09722222222222222, 'fraction_numerical': 0.011574074074074073, 'mean_word_length': 8.398550724637682, 'pattern_counts': {'":': 0, '<': 21, '<?xml version=': 0, '>': 21, 'https://': 1, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '5', 'dir': 'jupyter-structured-clean-dedup', 'id': '503990', 'license': 'MIT', 'n_tokens_mistral': 467, 'n_tokens_neox': 449, 'n_words': 96, 'path': 'module_4_pandas/exploratory_analysis_part1.ipynb', 'repo_name': 'wiplane/foundations-of-datascience-ml', 'size': '1296'}