<jupyter_start><jupyter_text>Complete Pandas ( Exploratory Data Analysis )<jupyter_code>import pandas as pd 
import numpy as np
df = pd.read_csv('autoclean.csv')
df.head()
df.drop("Unnamed: 0",axis="columns",inplace=True)
df.head()<jupyter_output><empty_output><jupyter_text>2. Analyzing Individual Feature Patterns using Visualization<jupyter_code>import seaborn as sns
import matplotlib.pyplot as plt
df.dtypes
df.corr()<jupyter_output><empty_output><jupyter_text>Find the correlation between the following columns: bore, stroke,compression-ratio , and horsepower<jupyter_code>df[["bore","stroke","compression-ratio","horsepower"]].corr()<jupyter_output><empty_output><jupyter_text>Continuous numerical variables:Continuous numerical variables are variables that may contain any value within some range. Continuous numerical variables can have the type "int64" or "float64". A great way to visualize these variables is by using scatterplots with fitted lines.In order to start understanding the (linear) relationship between an individual variable and the price. We can do this by using "regplot", which plots the scatterplot plus the fitted regression line for the data.Let's find the scatterplot of "engine-size" and "price"<jupyter_code>df[["engine-size","price"]].head()
# Engine size as potential predictor variable for price
sns.regplot(x="engine-size",y="price",data=df,color='r')
plt.ylim(0,)
df[["engine-size","price"]].corr()
df[["highway-mpg","price"]].head()
sns.regplot(x="highway-mpg",y="price",data=df,color='r')<jupyter_output><empty_output><jupyter_text>Weak Linear Relationship<jupyter_code>sns.regplot(x="peak-rpm",y="price",data=df)
df[["peak-rpm","price"]].corr()
sns.regplot(x="price",y="stroke",data=df)
df[["stroke","price"]].corr()<jupyter_output><empty_output><jupyter_text>Categorical variablesThese are variables that describe a 'characteristic' of a data unit, and are selected from a small group of categories. The categorical variables can have the type "object" or "int64". A good way to visualize categorical variables is by using boxplots.Let's look at the relationship between "body-style" and "price"<jupyter_code>sns.boxplot(x="body-style",y="price",data=df)
sns.boxplot(x="engine-location",y="price",data=df)
sns.boxplot(x="drive-wheels",y="price",data=df)<jupyter_output><empty_output><jupyter_text>3. Descriptive Statistical AnalysisLet's first take a look at the variables by utilizing a description method.The describe function automatically computes basic statistics for all continuous variables. Any NaN values are automatically skipped in these statistics.This will show:    the count of that variable    the mean    the standard deviation (std)     the minimum value    the IQR (Interquartile Range: 25%, 50% and 75%)    the maximum value<jupyter_code>df.describe()
df.describe(include=['object'])
from scipy import stats<jupyter_output><empty_output><jupyter_text>Wheel-base Vs Price<jupyter_code>pearson_coef, p_value = stats.pearsonr(df['wheel-base'],df['price'])
print("The Pearson Coeffient is", pearson_coef,"and the p_value is",p_value)
pearson_coef, P_value= stats.pearsonr(df['horsepower'],df['price'])
print("The Pearson Coefficient is",pearson_coef,"and the p_value is",p_value)
pearson_coef, p_value = stats.pearsonr(df['length'], df['price'])
print("The Pearson Correlation Coefficient is", pearson_coef, " with a P-value of P = ", p_value)  
pearson_coef, p_value = stats.pearsonr(df['width'], df['price'])
print("The Pearson Correlation Coefficient is", pearson_coef, " with a P-value of P =", p_value ) 
pearson_coef , p_value= stats.pearsonr(df['curb-weight'],df['price'],)
print( "The Pearson Correlation Coefficient is", pearson_coef, " with a P-value of P = ", p_value)  
pearson_coef, p_value = stats.pearsonr(df['engine-size'], df['price'])
print("The Pearson Correlation Coefficient is", pearson_coef, " with a P-value of P =", p_value) 
pearson_coef, p_value = stats.pearsonr(df['bore'], df['price'])
print("The Pearsoan Correlation Coefficient is", pearson_coef, " with a P-value of P =  ", p_value ) 
pearson_coef, p_value = stats.pearsonr(df['city-mpg'], df['price'])
print("The Pearson Correlation Coefficient is", pearson_coef, " with a P-value of P = ", p_value)  
pearson_coef, p_value = stats.pearsonr(df['highway-mpg'], df['price'])
print( "The Pearson Correlation Coefficient is", pearson_coef, " with a P-value of P = ", p_value )<jupyter_output>The Pearson Correlation Coefficient is 0.7893464215248446  with a P-value of P =  6.91574669541859e-45
{'fraction_non_alphanumeric': 0.12888986978591924, 'fraction_numerical': 0.011035091591260208, 'mean_word_length': 7.285191956124314, 'pattern_counts': {'":': 0, '<': 32, '<?xml version=': 0, '>': 32, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '8', 'dir': 'jupyter-structured-clean-dedup', 'id': '390535', 'license': 'CC0-1.0', 'n_tokens_mistral': 1558, 'n_tokens_neox': 1451, 'n_words': 404, 'path': 'Data_Analysis/Complete Pandas_ExploratoryDataAnalysis.ipynb', 'repo_name': 'ShivamBhosale/100DaysOfCode_V2', 'size': '4531'}