<jupyter_start><jupyter_text>Importing the relevant libraries<jupyter_code>import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
import seaborn as sns<jupyter_output><empty_output><jupyter_text>Loading the raw data<jupyter_code>raw_data = pd.read_csv('1.04. Real-life example.csv')
raw_data.shape
raw_data.head(10)
raw_data.tail()<jupyter_output><empty_output><jupyter_text>Preprocessing  Exploring the Descriptive Statistics of the variables<jupyter_code>raw_data.describe(include='all')
raw_data.describe()<jupyter_output><empty_output><jupyter_text>Determining the variables of interest<jupyter_code>raw_data.info()<jupyter_output><class 'pandas.core.frame.DataFrame'>
RangeIndex: 4345 entries, 0 to 4344
Data columns (total 9 columns):
 #   Column        Non-Null Count  Dtype  
---  ------        --------------  -----  
 0   Brand         4345 non-null   object 
 1   Price         4173 non-null   float64
 2   Body          4345 non-null   object 
 3   Mileage       4345 non-null   int64  
 4   EngineV       4195 non-null   float64
 5   Engine Type   4345 non-null   object 
 6   Registration  4345 non-null   object 
 7   Year          4345 non-null   int64  
 8   Model         4345 non-null   object 
dtypes: float64(2), int64(2), object(5)
memory usage: 305.6+ KB<jupyter_text>Dealing with missing values<jupyter_code># checking the null values
raw_data.isnull().sum()<jupyter_output><empty_output><jupyter_text>we have null values in Price and EngineV<jupyter_code># Null values are less then 5% so we can remove them
data_no_missing_value = raw_data.dropna(axis=0)<jupyter_output><empty_output><jupyter_text>Visualizing the Data  Exploring the PDFs<jupyter_code># A great step in the data exploration is to display the probability distribution function (PDF) of a variable
# The PDF will show us how that variable is distributed 
# This makes it very easy to spot anomalies, such as outliers
# The PDF is often the basis on which we decide whether we want to transform a feature
sns.histplot(data_no_missing_value['Price'], kde=True)<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.08544152744630072, 'fraction_numerical': 0.03866348448687351, 'mean_word_length': 4.14987714987715, 'pattern_counts': {'":': 0, '<': 29, '<?xml version=': 0, '>': 29, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '7', 'dir': 'jupyter-structured-clean-dedup', 'id': '270246', 'license': 'Apache-2.0', 'n_tokens_mistral': 739, 'n_tokens_neox': 674, 'n_words': 201, 'path': 'Part 1 - Regression/5. Decision Tree Regression/Preowned_Car_Price_Prediction.ipynb', 'repo_name': 'gopalchettri/Machine_Learning_A_to_Z', 'size': '2095'}