<jupyter_start><jupyter_text>Mini Project 1 - CS498 Data Science and Analytics Group Members: <NAME> (ashar29),  <NAME> (anunays2), <NAME> (br17)<jupyter_code>#If you need to change the directory, then we use the os package
import os 
os.chdir ('C:\\Users\\Anunay\\Documents\\CS 498\\MP\\MP1\\')
#Import all the necessary packages for this analysis

import pandas as pd
import os
import numpy as np
import matplotlib.pyplot as plt
import datetime
from scipy import stats
import seaborn as sns
%matplotlib inline
# Upload data file from your local drive

from google.colab import files

uploaded = files.upload()
df_diseng = pd.read_csv("mp1_av_disengagements.csv") #pd.read_csv to import the notebook
df_diseng.head() #Displays the top n rows of the dataset
uploaded = files.upload()
df_miles = pd.read_csv('mp1_av_totalmiles.csv')
df_miles.head()<jupyter_output><empty_output><jupyter_text>2. Summarize the following information a. Total number of AV disengagements over the entire duration of available data<jupyter_code>print(f'Total number of AV disengagements over the entire duration of available data are {df_miles["total number of disengagements"].sum(axis = 0)}')<jupyter_output>Total number of AV disengagements over the entire duration of available data are 1024<jupyter_text>b. Number of unique months that have recorded AV disengagements<jupyter_code>df_diseng['month']= df_diseng['Month'].apply(lambda x:x.split('-')[1]) #Split the data so that month and the date are apart
df_diseng['month'].nunique() #Count the unique months now
print(f'Number of unique months that have recorded AV disengagements are {df_diseng["month"].nunique()}')<jupyter_output>Number of unique months that have recorded AV disengagements are 12<jupyter_text>c. List of unique locations of AV disengagements<jupyter_code>print(f'There are {df_diseng["Location"].nunique()} locations namely {list(df_diseng["Location"].unique())[0]} and {list(df_diseng["Location"].unique())[1]}')<jupyter_output>There are 2 locations namely urban-street and highway<jupyter_text>d. Number of unique causes for AV disengagements<jupyter_code>print(f'Number of unique causes for AV disengagements are {df_diseng["Cause"].nunique()}')<jupyter_output>Number of unique causes for AV disengagements are 10<jupyter_text>e) Columns with missing values in the dataset Lets visualize the missing values in the datasets through a missing map Disengagements dataset<jupyter_code>#We create a heatmap which distinguishes the columns with missing values
sns.heatmap(df_diseng.isnull(), cbar=False)
plt.title("Missing map of Disengagement data")<jupyter_output><empty_output><jupyter_text>We can see above that the only column which has missing values is Reaction timeLets see how many of null values are actually there<jupyter_code>print(f'The number of missing values in the reaction time column are {len(df_diseng[df_diseng["ReactionTime"].isnull()])}')<jupyter_output>The number of missing values in the reaction time column are 532<jupyter_text>Total miles dataset<jupyter_code>sns.heatmap(df_miles.isnull(), cbar=False)
plt.title("Missing map of Total miles data")<jupyter_output><empty_output><jupyter_text>We can see above that there are no column which has missing values in this dataset 3) Plot a pie chart for the causes of AV disengagement. Based on the pie-chart, list the top 2 leading causes of disengagement?<jupyter_code>cause = df_diseng["Cause"].value_counts()  #Counts the occurences of different causes of disengagement
cause
plt.pie(cause, labels=cause.keys(),autopct = '%0.1f%%', shadow = True,radius =1) #produces the piechart using matplotlib
plt.title("Causes of AV disengagement")
plt.show()<jupyter_output><empty_output><jupyter_text>From the above pie chart we can see that Unwanted Driver discomfort and Recklessly behaving agent are the two leading causes of disengagements  4. Visualize the trend of disengagement/mile over time with monthly granularity. How would you describe the trend? Are AVs maturing over time?<jupyter_code>df1=df_miles.groupby(['Month']).sum()[['total number of disengagements','miles driven in autonomous mode']]
df1=df1.reset_index(0)
df1['month']= df1['Month'].apply(lambda x:x.split('-')[1]) #Splits the column to get the month and date separately
df1["month"] = pd.to_datetime(df1.month, format='%b', errors='coerce').dt.month #Converts the column into date
df1['year']= df1['Month'].apply(lambda x:x.split('-')[0]) #Splits the column to get the month and date separately

df1['disengagement/mile']=df1['total number of disengagements']/df1['miles driven in autonomous mode']
df1 = df1.sort_values(by=["year","month"],ascending=True)
df1['date']=df1['Month'].apply(lambda x:x.split('-')[1])+"-"+df1['year'].astype(str)
#We leverage matplotlib section to plot a line to understand the trend over months
plt.plot(df1['date'],df1['disengagement/mile'],'#8B008B',alpha=0.5,lw=1, ls='-', marker='o', markersize=8, markerfacecolor="yellow",markeredgecolor='black')
plt.xlabel('Months')
plt.ylabel('Disengagement/Mile')
plt.title('Disengagement/Mile Across Months')
plt.show()
#plt.xticks(['Jan','Feb','Mar','Apr','May','June','July','Aug','Sep','Oct','Nov','Dec'])<jupyter_output><empty_output><jupyter_text>The trend of disengagement/mile decreases over the course of time. We observe occassional peaks in disengagements but overall the trend is declining. Thus we can say that the AVs appear to have matured over time. It has the highest disengagement per mile in the month of September 2014 and lowest being November 2015  TASK 1 1. What do the following distributions signify about samples drawn from it     a. Gaussian distribution     b. Exponential distribution     c. Weibull distribution Regardless of the type of distribution, ths sample distribution depends on the sample size taken from the population. For instance, when we take n=10 from any of these distributions, it looks very disconnected with mean and variance being different from the original distribution. As the sample size increases, the distribution resembles close to the parent distribution and mean of the sample converges towards the mean of the entire population (Central Limit Theorem)References:1. https://amsi.org.au/ESA_Senior_Years/SeniorTopic4/4_md/SeniorTopic4b.htmlcontent_62. https://amsi.org.au/ESA_Senior_Years/SeniorTopic4/4b/4b_2content_7.htmlcontent_8 2. Distribution of Reaction Time<jupyter_code>sns.distplot(df_diseng['ReactionTime'].dropna(), hist = False, kde = True,rug=True,fit=stats.weibull_min,  #Displays the probability density of the distribution which can be above 1
                 kde_kws = {'shade': True, 'linewidth': 3}) 
plt.xlabel('Reaction Time', fontsize=12)
plt.ylabel('Probability Density', fontsize=12)
plt.title('Distribution of Reaction Time', fontsize=14)<jupyter_output><empty_output><jupyter_text>Distribution of Reaction time removing the extreme values(>4) for better visualization<jupyter_code>df_diseng2 = df_diseng[df_diseng['ReactionTime']<=4]

sns.distplot(df_diseng2['ReactionTime'], hist = False, kde = True,rug=True,fit=stats.weibull_min,
                 kde_kws = {'shade': True, 'linewidth': 3})
plt.title('Distribution of Reaction Time ', fontsize=14)
plt.xlabel('Reaction Time', fontsize=12)
plt.ylabel('Probability density', fontsize=12)<jupyter_output><empty_output><jupyter_text>The reaction time follows a weibull distribution. The distribution signifies that most of the drivers having a reaction time of 1-2s post which there is a very less population. References:https://www.itl.nist.gov/div898/handbook/eda/section3/eda3668.htm  3a. Average Reaction for entire duration of the dataset<jupyter_code>df_diseng.groupby('Location').mean()<jupyter_output><empty_output><jupyter_text>4. Perform a hypothesis testing at a 0.05 significance level Assumptions: From the given question, we have the mean of both the populations (AV and Non AV cars)Since no additional information is given, we assume that both the population has constant variance.Null hypothesis :The average reaction time for humans in Non AV cars is same as AV carsAlternative hypothesis : The average reaction time for humans in Non AV cars is different from AV cars<jupyter_code>from scipy import stats
###We assume both the population have constant variance
#se - standard deviation of the population
se = df_diseng['ReactionTime'].std() /np.sqrt(len(df_diseng['ReactionTime']))

Mean_of_NON_AV=1.09
Mean_of_AV=df_diseng['ReactionTime'].mean()
#t=x-u/se
ttest=(Mean_of_NON_AV-Mean_of_AV)/se  
print(round(ttest,3))
# degrees of freedom
df = len(df_diseng['ReactionTime'])-1

from scipy.stats import t
alpha=0.05 #Significance level given in the question
cv = t.ppf(1.0 - alpha, df)
p = (1 - t.cdf(abs(ttest), df)) * 2
if p<0.05:
    print("We reject the Null Hypothesis : The average reaction time for humans in Non AV cars is different as AV cars")
else:
    print("We fail  to reject the Null Hypothesis :The average reaction time for humans in Non AV cars same as AV cars")<jupyter_output>We reject the Null Hypothesis : The average reaction time for humans in Non AV cars is different as AV cars<jupyter_text>5. Plot the probability distribution of disengagements/mile with monthly granularity<jupyter_code>sns.distplot(df1['disengagement/mile'], hist = True, kde = True, fit=stats.norm,
                 kde_kws = {'shade': True, 'linewidth': 3})
plt.title('Distribution of #disengagement/mile ', fontsize=14)
plt.xlabel('#disengagement/mile', fontsize=12)
plt.ylabel('Probability density', fontsize=12)<jupyter_output><empty_output><jupyter_text>The above figure follows more or less a normal distribution (Except that it has a local peak due to data). An additional support to normality are the tails of the distribution which look symmetric unlike a skewed distribution  TASK 2   Assumptions :1. There can be at most one disengagement in a mile2. A day can be either clear or cloudy, but not both. The probability of a day being clear in California is 72% 3. The AV is equally likely to drive on a cloudy day as on a clear day. a.)  The assumption on maximum number of disengagements in a mile allows us to treat the occurrence of a disengagement in a mile as a random variable with a Bernoulli distribution b.) probability of disengagement per mile on a cloudy day.                                                                                 Let ,Based on assumption 2 we have,Probability of a day being clear in california be denoted as P(Clear)=0.72    Probability of a day being clear in california be denoted as P(Cloudy)=0.28    Probability of a disengagement/mile in california be denoted as P(DM)    We have to find : P(DM | cloudy) P(DM | cloudy) = [P(cloudy | DM) * P(DM)] / P(Cloudy) - using bayes theorem  Calculating: P(cloudy | DM) , let it be x<jupyter_code>x = len(df_diseng[df_diseng['Weather']=='cloudy']) /len(df_diseng)
print('Probability: ', x)<jupyter_output>Probability:  0.8154296875<jupyter_text>P(cloudy/DM) = 0.82  Calculating : P(DM) , let it be y<jupyter_code>y=df_miles['total number of disengagements'].sum()/df_miles['miles driven in autonomous mode'].sum()<jupyter_output><empty_output><jupyter_text>Calculating : P(DM) , let it be y<jupyter_code>y=df_miles['total number of disengagements'].sum()/df_miles['miles driven in autonomous mode'].sum()<jupyter_output><empty_output><jupyter_text>P(DM) = 0.002026803687040926  P(DM | cloudy) =[ 0.8154296875 * 0.002026803687040926 ] / 0.28P(DM | cloudy)  = 0.005903 probability of disengagement per mile on a cloudy day is 0.0059  c.) probability of disengagement per mile on a cloudy day. We have to find : P(DM | clear) P(DM | clear) = [P(clear | DM) * P(DM)] / P(Clear) - using bayes theoremCalculating: P(clear | DM) , let it be x<jupyter_code>x = len(df_diseng[df_diseng['Weather']=='clear']) /len(df_diseng)<jupyter_output><empty_output><jupyter_text>P(DM) will be same as part a) P(DM) = 0.002026803687040926P(DM | clear) =[ 0.1845703125 * 0.002026803687040926 ] / 0.72P(DM | clear)= 0.00052 probability of disengagement per mile on a cloudy day is 0.00052  d.) probability of automatic disengagement per mile on a cloudy day. Probability of a automatic disengagement/mile in california be denoted as P(ADM) P(ADM | Cloudy) = [P(Cloudy | ADM) * P(ADM)] / P(Cloudy) Calculating: P(cloudy | ADM) , let it be x<jupyter_code>x=len(df_diseng[(df_diseng['Weather']=='cloudy') & (df_diseng['TypeOfTrigger']=='automatic')]) / len(df_diseng[df_diseng['TypeOfTrigger']=='automatic'])<jupyter_output><empty_output><jupyter_text>Calculating: P(cloudy | ADM) , let it be x P(cloudy/ADM) = 0.8052738336713996  Calculating : P(ADM) , let it be y<jupyter_code>y=df_miles['number of automatic disengagements'].sum()/df_miles['miles driven in autonomous mode'].sum()<jupyter_output><empty_output><jupyter_text>P(ADM)= 0.0009757951344835708  P(ADM | cloudy) =[ 0.8052738336713996 * 0.0009757951344835708 ] / 0.28 P(ADM | cloudy) = 0.002806 probability of automatic disengagement per mile on a cloudy day is 0.0028 probability of automatic disengagement per mile on a clear day.  P(ADM | Clear) = [P(Clear | ADM) * P(ADM)] / P(Clear)  Calculating: P(cloudy/ADM) , let it be x<jupyter_code>x=len(df_diseng[(df_diseng['Weather']=='clear') & (df_diseng['TypeOfTrigger']=='automatic')]) / len(df_diseng[df_diseng['TypeOfTrigger']=='automatic'])<jupyter_output><empty_output><jupyter_text>P(clear | ADM) = 0.1947261663286004   We have P(ADM) calculated above P(ADM)= 0.0009757951344835708   P(ADM | Clear) = [x * 0.0009757951344835708] / 0.72  P(ADM | Clear) = 0.000264  e.) How likely is it that in 12000 miles, there are 150 or more disengagements under cloudy conditions?  The above distribution can be represented as a binomial distribution which can then be approximated to a normal distribution using Central Limit Theorem  The normal distribution will have mean = n * p  and standard deviation = sqrt(np(1-p)) , where n is the number of trials and p is probability of successGiven n =12000 , p= 0.005903  from the data<jupyter_code>import numpy as np
import matplotlib.pyplot as plt
from scipy.stats import norm
import matplotlib.pyplot as plt
import numpy as np
import scipy.stats as stats
import math
import scipy

mu = (12000*0.005903)
variance = (12000*0.005903) * (1-0.005903)
sigma = math.sqrt(variance)
x = np.linspace(mu - 3*sigma, mu + 3*sigma, 100)
plt.plot(x, stats.norm.pdf(x, mu, sigma))
plt.show()

x=150 ##given 
z = ( x - mu ) / sigma
p_values = scipy.stats.norm.sf(abs(z))
p_values<jupyter_output><empty_output><jupyter_text>2.) a.)  Normal distribution represents an approximate representation of the data in the hypotheses test<jupyter_code># DO NOT EXECUTE THIS CELL IN COLAB. LET IT BE GUYS!
# import matplotlib.pyplot as plt
# import matplotlib.image as mpimg
# img=mpimg.imread('Normal_dist.PNG')
# imgplot = plt.imshow(img)<jupyter_output><empty_output><jupyter_text>The application of the normal distribution comes from assessing data points in terms of the standard deviation. We can determine how anomalous a data point is based on how many standard deviations it is from the mean. The normal distribution has the following helpful properties:68% of data is within ± 1 standard deviations from the mean95% of data is within ± 2 standard deviations from the mean99.7% of data is within ± 3 standard deviations from the meanReference -https://towardsdatascience.com/statistical-significance-hypothesis-testing-the-normal-curve-and-p-values-93274fa32687 b.) It is not necessary that rejecting the NULL hypothesis means accepting the alternate hypothesis. It depends on the formulation of the experiment and a relationship may exist between 2 phenonmena that is not identifiedby the experimentFor example, there can be a set of hypothesis such as:H0: u1=u2HA: u1 > u2There is a solution of u1<jupyter_code><jupyter_output><empty_output><jupyter_text>3.) At a 0.05 significance level, test the following hypothesis: The AV has more disengagements (automatic and manual) on cloudy days than clear days. Based on the result of the hypothesis test, what can you conclude about the impact of weather conditions on AV safety? [Hint: Use a Z-test for testing this hypothesis].<jupyter_code>Null Hypothesis : The AV on cloudy days has same disengagement on clear days<jupyter_output><empty_output><jupyter_text>Null Hypothesis : The AV on cloudy days has same disengagement on clear days  Alternative Hypothesis : The AV has more disengagement on cloudy days than clear days<jupyter_code>import math as m

Number_of_cloudy_disengagements=835
Number_of_clear_disengagements=189

Prob_cloudy_day=0.28
Prob_clear_day=(1-0.28)

Mean_cloudy_disengagements= 835*0.28
std_cloudy_disengagements= m.sqrt(835*0.28*0.72)
Mean_clear_disengagements= 189*0.72
std_clear_disengagements= m.sqrt(835*0.72*0.28)

z=(Mean_cloudy_disengagements-Mean_clear_disengagements)/m.sqrt(Mean_cloudy_disengagements**2/835+Mean_clear_disengagements**2/189)
z
from scipy.stats import t
df=Number_of_cloudy_disengagements+Number_of_clear_disengagements-1
alpha=0.05#Significance level given in the question
cv = t.ppf(1.0 - alpha, df)
p = (1 - t.cdf(abs(z), df)) * 2
if p<0.05:
    print("We reject the Null Hypothesis : The AV on cloudy days has same disengagement on clear days")
else:
    print("We fail  to reject the Null Hypothesis : The AV has more disengagement on cloudy days than clear days")<jupyter_output>We reject the Null Hypothesis : The AV on cloudy days has same disengagement on clear days<jupyter_text>4. Conditional probability on reaction timeConditional Probability :  P(Dm | Cloudy)=P(Dm,Cloudy | P(C)<jupyter_code>Prob1= len(df_diseng[(df_diseng['ReactionTime']>0.6) & (df_diseng['TypeOfTrigger']=='automatic') & (df_diseng['Weather']=='cloudy')])/len(df_diseng[(df_diseng['TypeOfTrigger']=='automatic') & (df_diseng['Weather']=='cloudy')])
print("The Probability of disengagement per mile on a cloudy day is {}".format(round(Prob1,2)))<jupyter_output>The Probability of disengagement per mile on a cloudy day is 0.47<jupyter_text>Conditional Probability :  P(Dm | Clear)=P(Dm,Clear)/P(Clear)<jupyter_code>Prob2= len(df_diseng[(df_diseng['ReactionTime']>0.9) & (df_diseng['TypeOfTrigger']=='automatic') & (df_diseng['Weather']=='clear')])/len(df_diseng[(df_diseng['TypeOfTrigger']=='automatic') & (df_diseng['Weather']=='clear')])
print("The Probability of disengagement per mile on a cloudy day is {}".format(round(Prob2,2)))<jupyter_output>The Probability of disengagement per mile on a cloudy day is 0.28<jupyter_text>5. Probability on Accident per mile Theorem of total probability would be P(accident per mile)=P(Dm | Cloudy)*P(Cloudy) + P(Dm | clear)*P(clear)<jupyter_code>Prob3 = Prob1*0.28+Prob2*0.72
print("The Probability of Accident per mile on a cloudy day is {}".format(round(Prob3,2)))<jupyter_output>The Probability of Accident per mile on a cloudy day is 0.34<jupyter_text>6. The probability of a human driver causing a car accident is 2x10-6 [4]. How do AVs compare to human drivers?Justify your conclusion and explain its consequences  The probability of human driver causing a car accident is 2*10^-4 while the probability of AV for the same comes to be 0.34. There is a bigger risk associated with AV's as they can't be trained with all the possible scenarios of accident that happens in the real life.On the other hand, the humans react to any known/unknown signs of danger and their sudden response would to avoid accidents and mitigate the impactTraining the system with lesser data and of low variance can result in high probability of fatalities happening in the road. The system needs to be trained with different conditions like the weather, road and cities  7. KS Test Initially, we need to formulate the hypothesis for this testH0: Both the samples (clear and cloudy) from the same distributionHA: Both the samples (clear and cloudy) doesn't from the same distribution<jupyter_code>from scipy import stats

Reaction_time_cloudy=df_diseng[(df_diseng['Weather']=='cloudy')]['ReactionTime']
Reaction_time_clear=df_diseng[(df_diseng['Weather']=='clear')]['ReactionTime']

stats.ks_2samp(Reaction_time_cloudy, Reaction_time_clear)<jupyter_output><empty_output><jupyter_text>The given significance level is 0.1 (alpha) From the ks test, we get the P value as 0.7Decision rule:If pvalue else we fail to reject the NULL hypothesis or accept the alternate hypothesisHere 0.7>0.1,so we fail to reject the NULL hypothesis i.e. both the samples come from the same distribution We can conclude that the weather be it cloudy or clear doesn't have any impact on the disengagement reaction time. Task 3 1. Modify your pandas data frame to include a ‘Class’ column<jupyter_code>df_diseng.loc[df_diseng['Cause'].isin (["Incorrect behavior prediction of others",
                                        "Recklessly behaving agent",
                                       "Unwanted Driver Discomfort"]),'class'] = 'Controller'
df_diseng.loc[df_diseng['Cause'].isin (["Adverse road surface conditions",
                                        "Emergency Vehicle",
                                       "Position Estimation Failure",
                                       "Incorrect Traffic Light Detection"]),'class'] = 'Perception System'

df_diseng.loc[df_diseng['Cause'].isin (["System Tuning and Calibration",
                                        "Hardware Fault",
                                       "Software Froze"]),'class'] = 'Computer System'<jupyter_output><empty_output><jupyter_text>Split the data randomly into training and testing (80-20 split). Use the fields ‘Location’, ‘Weather’, and ‘TypeOfTrigger’ as features and use ‘Class’ as assigned in the previous question as the label<jupyter_code>from sklearn import preprocessing
#creating labelEncoder
le = preprocessing.LabelEncoder()
# Converting string labels into numbers.
df_diseng['Location_encoded']=le.fit_transform(df_diseng['Location'])
df_diseng['Weather_encoded']=le.fit_transform(df_diseng['Weather'])
df_diseng['TypeOfTrigger_encoded']=le.fit_transform(df_diseng['TypeOfTrigger'])
df_diseng['class_encoded']=le.fit_transform(df_diseng['class'])

from sklearn.model_selection import train_test_split
X=df_diseng[['Location_encoded','Weather_encoded','TypeOfTrigger_encoded']]
y=df_diseng[['class_encoded']]
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.20, random_state=101)<jupyter_output><empty_output><jupyter_text>3. Using the training dataset, create a NB model to identify the cause of disengagement based on the features ‘Location’, ‘Weather’, and ‘TypeOfTrigger’. Show the conditional probability tables from the training dataset<jupyter_code>#Import Gaussian Naive Bayes model
from sklearn.naive_bayes import GaussianNB

#Create a Gaussian Classifier
gnb = GaussianNB()

#Train the model using the training sets
model=gnb.fit(X_train, y_train)

##Making predictions based on the test set
y_pred=model.predict(X_test)<jupyter_output><empty_output><jupyter_text>4.) Calculating the accuracy of the model using the confusion matrix and classification report<jupyter_code>from sklearn.metrics import classification_report,confusion_matrix
cm=confusion_matrix(y_test,y_pred)
cr=classification_report(y_test,y_pred)
print(cr)  ###classification report 
print(cm)  ###confusion matrix
accuracy =cm.diagonal().sum() / cm.sum()
print(f"accuracy of the model with the validation set approach came to be {round(accuracy,4) *100} %")<jupyter_output>accuracy of the model with the validation set approach came to be 75.12 %<jupyter_text>5.) Performing cross validation to get a better estimate of the accuracy<jupyter_code>from sklearn.model_selection import KFold, cross_val_score
k_fold =KFold(n_splits=5, shuffle=False, random_state=42)
clf = GaussianNB()
print(cross_val_score(clf, X, y, cv=k_fold, n_jobs=1))
CV_accuracy=cross_val_score(clf, X, y, cv=k_fold, n_jobs=1)
from math import *
CV_accuracy_average=np.mean(CV_accuracy)*100
print(f'CV_accuracy_average : {CV_accuracy_average}')
random_state=[None , 101 , 42 ,1 ,10 ]
accuracy_list=[]
def cross_validation(random_state,accuracy_list):
    """
    Summary line : This function will do cross validation using 20% data as test set and 80% data as training set
                   using different random seeds 
  
    Parameters: 
    arg1 : random_state : A list containing different random state values
    arg2 : accuracy_list : an empty list which will hold 5 different accuracies for all the models 
  
    Returns: 
    average accuracy of all the models
    
    """
    
    for num in random_state:
        X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.20, random_state=num) 
        from sklearn.naive_bayes import GaussianNB

        #Create a Gaussian Classifier
        gnb = GaussianNB()

        #Train the model using the training sets
        model=gnb.fit(X_train, y_train)
        y_pred=model.predict(X_test)
        from sklearn.metrics import confusion_matrix
        cm=confusion_matrix(y_test,y_pred)
        accuracy =cm.diagonal().sum() / cm.sum()
        accuracy_list.append(accuracy)
    return (np.mean(accuracy_list)*100)    

cross_validation(random_state,accuracy_list)<jupyter_output>/usr/local/lib/python3.6/dist-packages/sklearn/naive_bayes.py:206: DataConversionWarning: A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().
  y = column_or_1d(y, warn=True)
/usr/local/lib/python3.6/dist-packages/sklearn/naive_bayes.py:206: DataConversionWarning: A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().
  y = column_or_1d(y, warn=True)
/usr/local/lib/python3.6/dist-packages/sklearn/naive_bayes.py:206: DataConversionWarning: A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().
  y = column_or_1d(y, warn=True)
/usr/local/lib/python3.6/dist-packages/sklearn/naive_bayes.py:206: DataConversionWarning: A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().
  y = col[...]<jupyter_text>**6.) Is the NB model doing better than chance? Explain.** Yes, the NB model is performing better than chance. The model predictions were nearly 75 %  correct for the test data set when we used the  validation set approach . We further validated this accuracy by performing k-fold cross validation approach (using Sklearn library and manually coding the k-fold CV). In both the cases the minimum average accuracy we achieved was nearly 58% (accuracy range: 58% - 64%). By randomly guessing, we can have at max 50% correct predictions, which is lower than the accuracy achieved by the NB model.   7.) What are the assumptions in NB in the context of this problem? Are the assumptions realistic? Explain.  **Assumption:**Naive Bayes assumes that the features are class conditionally independent (independent given/conditioning on the class) which may not always hold in a real-world problem. Here we have three classes: Perception System, Controller, and Computer System; which we assume to be independent. **For an AV to perform safely, it is important for all three classes to show congruity, and thus are inter-dependent.**For example, disengagement could result due to reckless behaving agent. However, the root cause could be software malfunction, which failed to process input from perception system (position estimation failure) and thus controller behaves in an unwanted manner.We believe that we could have achieved better accuracy had the assumptions been more realistic.   8) Based on your answer to part 7, comment on whether any improvements can be gained in classification accuracy. If yes, how?  Yes, there are dfferent approaches we can use to improve the accuracy of the model some of which are as follows : 1.) As the assumption of the naive bayes may not be applicable in context of this problem , one way to improve the model accuracy is to use a different classfication process like  Linear Discriminant Analysis , Quadratic Discriminant Analysis , Multi class logistic regression. 2.) Bayesian Networks do not assume class independence and hence could have been a better choice for this problem and thus might have produced a higher accuracy.3.)  Ensemble models which inloves combining multiple models to improve the accuracy using bagging and boosting. This ensembling can improve the predictive performance more than any single model.4.) Here we have a quite small dataset ,a general appraoch which can increase the accuracy is collecting more data as training a model on a more diverse dataset which has more variablity can result in a better machine learning model.<jupyter_code><jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.09260606060606061, 'fraction_numerical': 0.0267012987012987, 'mean_word_length': 5.371579876434246, 'pattern_counts': {'":': 0, '<': 142, '<?xml version=': 0, '>': 144, 'https://': 4, 'lorem ipsum': 0, 'www.': 1, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 11, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '37', 'dir': 'jupyter-structured-clean-dedup', 'id': '666920', 'license': 'BSD-3-Clause', 'n_tokens_mistral': 9414, 'n_tokens_neox': 8642, 'n_words': 3126, 'path': 'MP1_02142020.ipynb', 'repo_name': 'anirudh217sharma/fall2019', 'size': '28912'}