<jupyter_start><jupyter_text>ID= Index number for data, so we can drop this as pandas has inbuilt index.SepalLengthCm: Length of the sepal (in cm)SepalWidthCm : Width of the sepal (in cm)PetalLengthCm: Length of the petal (in cm)PetalWidthCm :Width of the petal (in cm)Species: Species name<jupyter_code>iris.drop('Id', axis=1, inplace=True)
#Checking null values
iris.isnull().any()
# chcking numerical values
iris.describe()<jupyter_output><empty_output><jupyter_text>NO zero values, no need of normalization as values are in same range<jupyter_code>#Now we will plot the related features to get the relations
# Sepal featurs

fig = iris[iris.Species=='Iris-setosa'].plot(kind='scatter',x='SepalLengthCm',y='SepalWidthCm',color='orange', label='Setosa')
iris[iris.Species=='Iris-versicolor'].plot(kind='scatter',x='SepalLengthCm',y='SepalWidthCm',color='blue', label='versicolor',ax=fig)
iris[iris.Species=='Iris-virginica'].plot(kind='scatter',x='SepalLengthCm',y='SepalWidthCm',color='green', label='virginica', ax=fig)
fig.set_xlabel("Sepal Length")
fig.set_ylabel("Sepal Width")
fig.set_title("Sepal Length VS Width")
fig=plt.gcf()
fig.set_size_inches(10,6)
plt.show()

#Petal Features

fig = iris[iris.Species=='Iris-setosa'].plot.scatter(x='PetalLengthCm',y='PetalWidthCm',color='orange', label='Setosa')
iris[iris.Species=='Iris-versicolor'].plot.scatter(x='PetalLengthCm',y='PetalWidthCm',color='blue', label='versicolor',ax=fig)
iris[iris.Species=='Iris-virginica'].plot.scatter(x='PetalLengthCm',y='PetalWidthCm',color='green', label='virginica', ax=fig)
fig.set_xlabel("Petal Length")
fig.set_ylabel("Petal Width")
fig.set_title(" Petal Length VS Width")
fig=plt.gcf()
fig.set_size_inches(10,6)
plt.show()<jupyter_output><empty_output><jupyter_text>We can see petal has classified the difference much larger than sepals.<jupyter_code># Now we will see lenght and width variation according to the species

plt.figure(figsize=(15,10))
plt.subplot(2,2,1)
sns.violinplot(x='Species',y='PetalLengthCm',data=iris)
plt.subplot(2,2,2)
sns.violinplot(x='Species',y='PetalWidthCm',data=iris)
plt.subplot(2,2,3)
sns.violinplot(x='Species',y='SepalLengthCm',data=iris)
plt.subplot(2,2,4)
sns.violinplot(x='Species',y='SepalWidthCm',data=iris)<jupyter_output><empty_output><jupyter_text>The violinplot shows density of the length and width in the species.The thinner part denotes that there is less density whereas the fatter part conveys higher density<jupyter_code># We will also check for the correlation between different parameter
plt.figure(figsize=(7,4)) 
sns.heatmap(iris.corr(),annot=True,cmap='cubehelix_r') #draws  heatmap with input as the correlation matrix calculted by(iris.corr())
plt.show()<jupyter_output><empty_output><jupyter_text>So we can see the correlation between the sepal lenght and width is v less .11 but with petals it is v high 0.96  Linear Discriminant Analysis (LDA)<jupyter_code># LDA
x= iris.iloc[:,:4].values
y=  iris.iloc[:,[-1]].values

# # Scaling data
# from sklearn.preprocessing import StandardScaler
# ss= StandardScaler()
# ss.fit_transform(x)



from sklearn.discriminant_analysis import LinearDiscriminantAnalysis as LDA
c= LDA(n_components=2)
x= c.fit_transform(x,y)


x
# x= iris.iloc[:,:4]
# y=  iris.iloc[:,[-1]].values

# we will give y as 1d array in classification models

from sklearn.model_selection import train_test_split
x_train, x_test, y_train, y_test = train_test_split(x, y, test_size = 0.3, random_state = 0)

print(x_train.shape,x_test.shape,y_train.shape,y_test.shape)<jupyter_output>(105, 2) (45, 2) (105, 1) (45, 1)<jupyter_text>Logistic Regression<jupyter_code>from sklearn.linear_model import LogisticRegression

regclass= LogisticRegression(random_state=0)

regclass.fit(x_train,y_train)
ypred= regclass.predict(x_test)
from sklearn.metrics import confusion_matrix, accuracy_score, classification_report
cm= confusion_matrix(y_test,ypred)
cm
acc_score= accuracy_score(y_test,ypred)
acc_score
creport= classification_report(y_test,ypred) 
creport<jupyter_output><empty_output><jupyter_text>K Nearest Neighbours<jupyter_code>#K nearest neighbour
from sklearn.neighbors import KNeighborsClassifier
np.ceil(np.sqrt(len(x_train)))<jupyter_output><empty_output><jupyter_text>In KNN, finding the value of k is not easy. A small value of k means that noise will have a higher influence on the result and a large value make it computationally expensive.Data scientists usually choose as an odd number if the number of classes is 2 andanother simple approach to select k is set k=sqrt(n) or k= n**0.5<jupyter_code>knn= KNeighborsClassifier(n_neighbors=11)<jupyter_output><empty_output><jupyter_text>MAin Parametrs we should know is n_neighbors= No of neighbors required to classify the new objectmetric : string or DistanceMetric object (default = 'minkowski')    the distance metric to use for the tree.  The default metric is    minkowski, and with p=2 is equivalent to the standard Euclidean    metric. See the documentation of the DistanceMetric class for a    list of available metrics.p : integer, optional (default = 2)    Power parameter for the Minkowski metric. When p = 1, this is    equivalent to using manhattan_distance (l1), and euclidean_distance    (l2) for p = 2. For arbitrary p, minkowski_distance (l_p) is used.<jupyter_code>knn.fit(x_train,y_train)

yp= knn.predict(x_test)

acc_score= accuracy_score(yp,y_test)
acc_score
cm= confusion_matrix(y_test,ypred)
cm
# Now checking with diff number for neighbors
score=[]
for i in range(1,11):
    knn= KNeighborsClassifier(n_neighbors=i)
    knn.fit(x_train,y_train)
    score.append(accuracy_score(knn.predict(x_test),y_test))
    
plt.plot(list(range(1,11)),score)  
for i in range(10,25):
    knn= KNeighborsClassifier(n_neighbors=i)
    knn.fit(x_train,y_train)
    score=accuracy_score(knn.predict(x_test),y_test)
    print(i,score)<jupyter_output>10 0.9777777777777777
11 0.9777777777777777
12 0.9777777777777777
13 0.9777777777777777
14 0.9777777777777777
15 0.9777777777777777
16 0.9777777777777777
17 0.9777777777777777
18 0.9777777777777777
19 0.9777777777777777
20 0.9777777777777777
21 0.9777777777777777
22 0.9777777777777777
23 0.9777777777777777
24 0.9777777777777777<jupyter_text>Naive Bayes Theorem Bayes Theorem:         P(A|X)= (P(X|A)*P(A))/P(X)    where P= Probabality    | = is given that or in respect to so P(A|B) means prob of a given by b    A= Classication Variable    X= predicting Features in whose respect we will calculate P(A|X)How its works.?SO as we gave a and b category data so now it will compare P(A|X) and P(B|X)so it means  P(A|X)= (P(X|A)*P(A))/P(X)   Compared   P(B|X)= (P(X|B)*P(B))/P(X)Who so ever will have more probability in respect to X , will be predicted new one for that observation   so by mathematics we can see P(X) denominator in both will be cancelled so main part is only played by Numerators  Why naive?? As we have to make independence prediction like P(A) and P(B). Naive Bayes is the assumption of independent predictors. In real life, it is almost impossible that we get a set of predictors which are completely independent.<jupyter_code>from sklearn.naive_bayes import GaussianNB
NB= GaussianNB()
NB.fit(x_train,y_train)
ynb= NB.predict(x_test)
ascore= accuracy_score(y_test,ynb)
ascore<jupyter_output><empty_output><jupyter_text>WHy Naive Bayes is naive???????????In simple terms, a naive Bayes classifier assumes that the presence (or absence) of a particular feature of a class is unrelated to the presence (or absence) of any other feature, given the class variable. For example, a fruit may be considered to be an apple if it is red, round, and about 4" in diameter. Even if these features depend on each other or upon the existence of the other features, a naive Bayes classifier considers all of these properties to independently contribute to the probability that this fruit is an apple.  Decision Tree Classifier<jupyter_code>from sklearn.tree import DecisionTreeClassifier
dtc= DecisionTreeClassifier(max_depth=3)<jupyter_output><empty_output><jupyter_text>When you will change max_depth, for 2 accuracy is 91% and above that its 97%, so we have to chosse wisely for models<jupyter_code>dtc.fit(x_train,y_train)
ydp= dtc.predict(x_test)
accuracy_score(y_test,ydp)
confusion_matrix(y_test,ydp)<jupyter_output><empty_output><jupyter_text>Random forest classifier<jupyter_code>from sklearn.ensemble import RandomForestClassifier
rfc= RandomForestClassifier()
rfc.fit(x_train,y_train)
yrf= rfc.predict(x_test)
accuracy_score(y_test,yrf)
confusion_matrix(y_test,yrf)<jupyter_output><empty_output><jupyter_text>AdaBoost<jupyter_code>x= iris.iloc[:,:4].values
y=  iris.iloc[:,4].values

from sklearn.model_selection import train_test_split
x_train, x_test, y_train, y_test = train_test_split(x, y, test_size = 0.3, random_state = 0)

print(x_train.shape,x_test.shape,y_train.shape,y_test.shape)


from sklearn.ensemble import AdaBoostClassifier
abc= AdaBoostClassifier(n_estimators=50,learning_rate=0.7,base_estimator=rfc, random_state=1)<jupyter_output><empty_output><jupyter_text>base_estimator : object, optional (default=DecisionTreeClassifier)    The base estimator from which the boosted ensemble is built.    Support for sample weighting is required, as well as proper `classes_`    and `n_classes_` attributes.n_estimators : integer, optional (default=50)    The maximum number of estimators at which boosting is terminated.    In case of perfect fit, the learning procedure is stopped early.learning_rate : float, optional (default=1.)    Learning rate shrinks the contribution of each classifier by    ``learning_rate``. There is a trade-off between ``learning_rate`` and    ``n_estimators``.<jupyter_code>abc.fit(x_train,y_train)
yabc= abc.predict(x_test)
abc.score(x_test,y_test)
from sklearn import metrics
print("Accuracy:",metrics.accuracy_score(y_test, yabc))<jupyter_output>Accuracy: 0.9777777777777777<jupyter_text>SVC<jupyter_code>from sklearn.svm import SVC
#Import scikit-learn metrics module for accuracy calculation
from sklearn import metrics
svc=SVC(probability=True, kernel='linear')

# Create adaboost classifer object
abc =AdaBoostClassifier(n_estimators=50, base_estimator=svc,learning_rate=1)
abc.fit(x_train,y_train)
yas=abc.predict(x_test)
abc.score(x_test,y_test)<jupyter_output><empty_output><jupyter_text>So we can see by using base estimator as SVC accuracy score has changed drastically<jupyter_code># Using Decision Tree as Base
abc =AdaBoostClassifier(n_estimators=50, base_estimator=dtc,learning_rate=1)

abc.fit(x_train,y_train)
yas=abc.predict(x_test)
abc.score(x_test,y_test)
# Using Random Forest as Base
abc =AdaBoostClassifier(n_estimators=50, base_estimator=rfc,learning_rate=1)

abc.fit(x_train,y_train)
yas=abc.predict(x_test)
abc.score(x_test,y_test)<jupyter_output><empty_output><jupyter_text>Gradient Boosting<jupyter_code>from sklearn.ensemble import GradientBoostingClassifier
params = {
    'n_estimators': 50, 
    'max_depth': 1,
    'learning_rate': 1,
    'criterion': 'mse'
}

# gb= GradientBoostingRegressor(**params)
gc= GradientBoostingClassifier(**params)
gc.fit(x_train,y_train)
ygc=abc.predict(x_test)
gc.score(x_test,y_test)<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.10532417729421208, 'fraction_numerical': 0.036564701685543564, 'mean_word_length': 7.1026011560693645, 'pattern_counts': {'":': 0, '<': 70, '<?xml version=': 0, '>': 70, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 16, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '18', 'dir': 'jupyter-structured-clean-dedup', 'id': '421506', 'license': 'MIT', 'n_tokens_mistral': 4316, 'n_tokens_neox': 3977, 'n_words': 1014, 'path': 'Iris_Classification_ADABoost.ipynb', 'repo_name': 'pankush9096/Iris-Data-Classification-model', 'size': '11213'}