<jupyter_start><jupyter_text>TOEFL Score:* The lowest TOEFL Score is 92 and the highest Toefl score is 120. The average is 107.41<jupyter_code>y=np.array([df["TOEFL Score"].min(),df["TOEFL Score"].mean(),df["TOEFL Score"].max()])
x=["Worest","Average","Best"]
plt.bar(x,y)
plt.title("TOEFL Scores")
plt.xlabel("Level")
plt.ylabel("TOEFL Score")
plt.show()<jupyter_output><empty_output><jupyter_text>GRE Score:* This histogram shows the frequency for GRE scores.* There is a density between 310 and 330.Being above this range would be a good feature for a candidate to stand out.<jupyter_code>df["GRE Score"].plot(kind='hist',bins=200,figsize=(6,6))
plt.title("GRE Scores")
plt.xlabel("GRE Score")
plt.ylabel("Frequency")
plt.show()<jupyter_output><empty_output><jupyter_text>CGPA Score for University Ratings:* As the quality of the university increases the CGPA score increases<jupyter_code>plt.scatter(df["University Rating"],df.CGPA)
plt.title("CGPA Scores for University Ratings")
plt.xlabel("University Rating")
plt.ylabel("CGPA")
plt.show()
plt.scatter(df["GRE Score"],df.CGPA)
plt.title("CGPA for GRE Scores")
plt.xlabel("GRE Score")
plt.ylabel("CGPA")
plt.show()<jupyter_output><empty_output><jupyter_text>* Candidates with high GRE scores usually have a high CGPA score<jupyter_code>df[df.CGPA >=8.5].plot(kind='scatter',x='GRE Score',y='TOEFL Score',color='red')
plt.xlabel("GRE Score")
plt.ylabel("TOEFL  SCORE")
plt.title("CGPA >=8.5")
plt.grid(True)
plt.show()<jupyter_output><empty_output><jupyter_text>* Candidates who graduate from good universitiew are more fortunate to be accepted.<jupyter_code>s = df[df["Chance of Admit"] >= 0.75]["University Rating"].value_counts().head(5)
plt.title("University Ratings of Candidates with an 75% acceptance chance")
s.plot(kind='bar',figsize=(20, 10))
plt.xlabel("University Rating")
plt.ylabel("Candidates")
plt.show()<jupyter_output><empty_output><jupyter_text>*Candidates with high CGPA scores usually have a high SOP score<jupyter_code>plt.scatter(df["CGPA"],df.SOP)
plt.xlabel("CGPA")
plt.ylabel("SOP")
plt.title("SOP for CGPA")
plt.show()<jupyter_output><empty_output><jupyter_text>* Candidates with high GRE scores usually have a high SOP score<jupyter_code>plt.scatter(df["GRE Score"],df["SOP"])
plt.xlabel("GRE Score")
plt.ylabel("SOP")
plt.title("SOP for GRE Score")
plt.show()<jupyter_output><empty_output><jupyter_text>REGRESSION ALGORITHMS (SUPERVISED MACHINE LEARNING ALGORITHMS)  Preparing Data for Regression<jupyter_code>#reading the dataset
df=pd.read_csv("Admission_Predict.csv")

#it may be needed in the future
serialNo=df['Serial No.'].values

df.drop(['Serial No.'],axis=1,inplace=True)

df=df.rename(columns = {'Chance of Admit ':'Chance of Admit'})<jupyter_output><empty_output><jupyter_text>train_test_split:* it split the data into random train (80%) and test (20%) subsets.<jupyter_code>y = df["Chance of Admit"].values
x = df.drop(["Chance of Admit"],axis=1)

#separating train (80%) and test (20%) sets
from sklearn.model_selection import train_test_split

x_train,x_test,y_train,y_test=train_test_split(x,y,test_size=0.20,random_state=42)
#normalization
from sklearn.preprocessing import MinMaxScaler
scalerX=MinMaxScaler(feature_range=(0,1))
x_train[x_train.columns]=scalerX.fit_transform(x_train[x_train.columns])
x_test[x_test.columns]=scalerX.transform(x_test[x_test.columns])<jupyter_output>C:\Anaconda3\lib\site-packages\sklearn\preprocessing\data.py:323: DataConversionWarning: Data with input dtype int64, float64 were all converted to float64 by MinMaxScaler.
  return self.partial_fit(X, y)
C:\Anaconda3\lib\site-packages\ipykernel_launcher.py:4: SettingWithCopyWarning: 
A value is trying to be set on a copy of a slice from a DataFrame.
Try using .loc[row_indexer,col_indexer] = value instead

See the caveats in the documentation: http://pandas.pydata.org/pandas-docs/stable/indexing.html#indexing-view-versus-copy
  after removing the cwd from sys.path.
C:\Anaconda3\lib\site-packages\pandas\core\indexing.py:543: SettingWithCopyWarning: 
A value is trying to be set on a copy of a slice from a DataFrame.
Try using .loc[row_indexer,col_indexer] = value instead

See the caveats in the documentation: http://pandas.pydata.org/pandas-docs/stable/indexing.html#indexing-view-versus-copy
  self.obj[item] = s
C:\Anaconda3\lib\site-packages\ipykernel_launcher.py:5: SettingWithCopyWarni[...]<jupyter_text>Linear Regression<jupyter_code>from sklearn.linear_model import LinearRegression
lr=LinearRegression()
lr.fit(x_train,y_train)
y_head_lr=lr.predict(x_test)

print("real value of y_test[1]:"+str(y_test[1]) + "-> the predict:" +str(lr.predict(x_test.iloc[[1],:])))
print("real value of y_test[2]: " + str(y_test[2]) + " -> the predict: " + str(lr.predict(x_test.iloc[[2],:])))

from sklearn.metrics import r2_score
print("r_square score:",r2_score(y_test,y_head_lr))

y_head_lr_train=lr.predict(x_train)
print("r_square score (train dataset):",r2_score(y_train,y_head_lr_train))<jupyter_output>real value of y_test[1]:0.68-> the predict:[0.72368741]
real value of y_test[2]: 0.9 -> the predict: [0.93536809]
r_square score: 0.8212082591486992
r_square score (train dataset): 0.7951946003191086<jupyter_text>Random Forest Regresssion<jupyter_code>from sklearn.ensemble import RandomForestRegressor
rfr=RandomForestRegressor(n_estimators=100,random_state=42)
rfr.fit(x_train,y_train)
y_head_rfr=rfr.predict(x_test)

from sklearn.metrics import r2_score
print("r_square score:",r2_score(y_test,y_head_rfr))
print("real value of y_test[1]:" +str(y_test[1])+"-> the predict:"+str(rfr.predict(x_test.iloc[[1],:])))
print("real value of y_test[2]:" +str(y_test[2])+"-> the predict:"+str(rfr.predict(x_test.iloc[[2],:])))

y_head_rf_train=rfr.predict(x_train)
print("r_square score (train dataset):",r2_score(y_train,y_head_rf_train))<jupyter_output>r_square score: 0.8074111823415694
real value of y_test[1]:0.68-> the predict:[0.7249]
real value of y_test[2]:0.9-> the predict:[0.9407]
r_square score (train dataset): 0.9634880602889714<jupyter_text>Decision Tree Regression<jupyter_code>from sklearn.tree import DecisionTreeRegressor
dtr=DecisionTreeRegressor(random_state=42)
dtr.fit(x_train,y_train)
y_head_dtr=dtr.predict(x_test)

from sklearn.metrics import r2_score
print("r_square score:",r2_score(y_test,y_head_dtr))
print("real value of y_test[1]:" +str(y_test[1])+ "-> the predict" +str(dtr.predict(x_test.iloc[[1],:])))
print("real value of y_test[2]:" +str(y_test[2])+ "-> the predict"+ str(dtr.predict(x_test.iloc[[2],:])))

y_head_dtr_train=dtr.predict(x_train)
print("r_square score (train dataset):",r2_score(y_train,y_head_dtr_train))<jupyter_output>r_square score: 0.6262105228127393
real value of y_test[1]:0.68-> the predict[0.73]
real value of y_test[2]:0.9-> the predict[0.94]
r_square score (train dataset): 1.0<jupyter_text>Comparison of Regression Algorithms* Linear regression and random forest regression algorithms were better than decision tree regression algorithm.<jupyter_code>y = np.array([r2_score(y_test,y_head_lr),r2_score(y_test,y_head_rfr),r2_score(y_test,y_head_dtr)])
x=["LinearRegression","RandomForestReg","DecisionTreeReg."]
plt.bar(x,y)
plt.title("Comparision of Regression Algorithms")
plt.xlabel("Regressor")
plt.ylabel("r2_score")
plt.show()<jupyter_output><empty_output><jupyter_text>* These are the regression estimates for samples with 5 and 50 indexes:<jupyter_code>print("real value of y_test[5]:" +str(y_test[5]) +"-> the predict:" +str(lr.predict(x_test.iloc[[5],:])))
print("real value of y_test[5]:" +str(y_test[5]) +"-> the predict:" +str(rfr.predict(x_test.iloc[[5],:])))
print("real value of y_test[5]:" +str(y_test[5]) +"-> the predict:" +str(dtr.predict(x_test.iloc[[5],:])))
print()
print("real value of y_test[50]:" +str(y_test[50]) +"-> the predict:" +str(lr.predict(x_test.iloc[[50],:])))
print("real value of y_test[50]:" +str(y_test[50]) +"-> the predict:" +str(rfr.predict(x_test.iloc[[50],:])))
print("real value of y_test[50]:" +str(y_test[50]) +"-> the predict:" +str(lr.predict(x_test.iloc[[50],:])))<jupyter_output>real value of y_test[5]:0.94-> the predict:[0.92863016]
real value of y_test[5]:0.94-> the predict:[0.9167]
real value of y_test[5]:0.94-> the predict:[0.91]

real value of y_test[50]:0.86-> the predict:[0.89624998]
real value of y_test[50]:0.86-> the predict:[0.9111]
real value of y_test[50]:0.86-> the predict:[0.89624998]<jupyter_text>This is the estimate and the actual acceptance possibilities made with 3 regression algorithms for test samples with 0, 5, 10, 15, 20, 25, 30, 35, 40, 45, 50, 55, 60, 65, 70, 75 indexes:<jupyter_code>red=plt.scatter(np.arange(0,80,5),y_head_lr[0:80:5],color='red')
green=plt.scatter(np.arange(0,80,5),y_head_rfr[0:80:5],color='green')
blue=plt.scatter(np.arange(0,80,5),y_head_dtr[0:80:5],color="blue")
black=plt.scatter(np.arange(0,80,5),y_test[0:80:5],color="black")
plt.title("Comparision of Regression Algorithms")
plt.xlabel("Index of Candidate")
plt.ylabel("Chance of Admit")
plt.legend((red,green,blue,black),('LR','RFR','DTR','REAL'))
plt.show()<jupyter_output><empty_output><jupyter_text>Comment:* Because most candidates in the data have over 70% chance, many unsuccessful candidates are not well predicted.<jupyter_code>df['Chance of Admit'].plot(kind='hist',bins=200,figsize=(6,6))
plt.title("Chance of Admit")
plt.xlabel("Chance of Admit")
plt.ylabel("Frequency")
plt.show()<jupyter_output><empty_output><jupyter_text>CLASSIFICATION ALGORITHMS (SUPERVISED MACHINE LEARNING ALGORITHMS)  Preparing Data for Classification * If a candidate's Chance of Admit is greater than 80%, the candidate will receive the 1 label.* If a candidate's Chance of Admit is less than or equal to 80%, the candidate will receive the 0 label.<jupyter_code># reading the dataset
df = pd.read_csv("Admission_Predict.csv",sep = ",")

# it may be needed in the future.
serialNo = df["Serial No."].values
df.drop(["Serial No."],axis=1,inplace = True)

df=df.rename(columns = {'Chance of Admit ':'Chance of Admit'})
y = df["Chance of Admit"].values
x = df.drop(["Chance of Admit"],axis=1)

# separating train (80%) and test (%20) sets
from sklearn.model_selection import train_test_split
x_train, x_test,y_train, y_test = train_test_split(x,y,test_size = 0.20,random_state = 42)

#normalization
from sklearn.preprocessing import MinMaxScaler
scalerX=MinMaxScaler(feature_range=(0,1))
x_train[x_train.columns]=scalerX.fit_transform(x_train[x_train.columns])
x_test[x_test.columns]=scalerX.transform(x_test[x_test.columns])

y_train_01=[1 if each >0.8 else 0 for each in y_train]
y_test_01=[1 if each > 0.8 else 0 for each in y_test]


#List to array
y_train_01=np.array(y_train_01)
y_test_01=np.array(y_test_01)<jupyter_output>C:\Anaconda3\lib\site-packages\sklearn\preprocessing\data.py:323: DataConversionWarning: Data with input dtype int64, float64 were all converted to float64 by MinMaxScaler.
  return self.partial_fit(X, y)
C:\Anaconda3\lib\site-packages\ipykernel_launcher.py:4: SettingWithCopyWarning: 
A value is trying to be set on a copy of a slice from a DataFrame.
Try using .loc[row_indexer,col_indexer] = value instead

See the caveats in the documentation: http://pandas.pydata.org/pandas-docs/stable/indexing.html#indexing-view-versus-copy
  after removing the cwd from sys.path.
C:\Anaconda3\lib\site-packages\pandas\core\indexing.py:543: SettingWithCopyWarning: 
A value is trying to be set on a copy of a slice from a DataFrame.
Try using .loc[row_indexer,col_indexer] = value instead

See the caveats in the documentation: http://pandas.pydata.org/pandas-docs/stable/indexing.html#indexing-view-versus-copy
  self.obj[item] = s
C:\Anaconda3\lib\site-packages\ipykernel_launcher.py:5: SettingWithCopyWarni[...]<jupyter_text>Logistic Regression<jupyter_code>from sklearn.linear_model import LogisticRegression
lrc=LogisticRegression()
lrc.fit(x_train,y_train_01)
print("score:",lrc.score(x_test,y_test_01))
print("real value of y_test_01[1]:" +str(y_test_01[1]) + "-> the predict:" +str(lrc.predict(x_test.iloc[[1],:])))
print("real value of y_test_01[2]:"+str(y_test_01[2]) +"->the predict:" +str(lrc.predict(x_test.iloc[[2],:])))

#confusion matrix
from sklearn.metrics import confusion_matrix
cm_lrc=confusion_matrix(y_test_01,lrc.predict(x_test))

#cm visualization
import seaborn as sns
import matplotlib.pyplot as plt
f,ax=plt.subplots(figsize=(5,5))
sns.heatmap(cm_lrc,annot=True,linewidths=0.5,linecolor="red",fmt=".0f",ax=ax)
plt.title("Test for Test Dataset")
plt.xlabel("Predicted y values")
plt.ylabel("Predicted y values")
plt.show()


from sklearn.metrics import precision_score,recall_score
print("precision_score:",precision_score(y_test_01,lrc.predict(x_test)))
print("recall_score:",recall_score(y_test_01,lrc.predict(x_test)))


from sklearn.metrics import f1_score
print("f1_score:",f1_score(y_test_01,lrc.predict(x_test)))<jupyter_output><empty_output><jupyter_text>Test for Train Dataset:<jupyter_code>cm_lrc_train=confusion_matrix(y_train_01,lrc.predict(x_train))
f,ax=plt.subplots(figsize=(5,5))
sns.heatmap(cm_lrc_train,annot=True,linewidths=0.5,linecolor="red",fmt=".0f",ax=ax)
plt.xlabel("predicted y values")
plt.ylabel("real y values")
plt.title("Test for Train Dataset")
plt.show()<jupyter_output><empty_output><jupyter_text>Support Vector Machine<jupyter_code>from sklearn.svm import SVC
svm=SVC(random_state=1)
svm.fit(x_train,y_train_01)
print("score:",svm.score(x_test,y_test_01))
print("real value of y_test_01[1]:"+str(y_test_01[1]))
print("real value of y_test_01[2]:"+str(y_test_01[2]))
#confusion matrix
from sklearn.metrics import confusion_matrix
cm_svm=confusion_matrix(y_test_01,svm.predict(x_test))
#cm visualization
import seaborn as sns
import matplotlib.pyplot as plt
f,ax=plt.subplots(figsize=(5,5))
sns.heatmap(cm_svm,annot=True,linewidths=0.5,linecolor="red",fmt=".0f",ax=ax)
plt.title("Test for Test Dataset")
plt.xlabel("predicted y values")
plt.ylabel("real y values")
plt.show()
from sklearn.metrics import precision_score,recall_score
print("precision_score:",precision_score(y_test_01,svm.predict(x_test)))
print("recall_score:",recall_score(y_test_01,svm.predict(x_test)))


from sklearn.metrics import f1_score
print("f1_score",f1_score(y_test_01,svm.predict(x_test)))<jupyter_output>precision_score: 0.9565217391304348
recall_score: 0.7586206896551724
f1_score 0.8461538461538461<jupyter_text>Test for Train Dataset:<jupyter_code>cm_svm_train=confusion_matrix(y_train_01,svm.predict(x_train))
f,ax=plt.subplots(figsize=(5,5))
sns.heatmap(cm_svm_train,annot=True,linewidth=0.5,linecolor="red",fmt=".0f",ax=ax)
plt.xlabel("predicted y values")
plt.ylabel("real y values")
plt.title("Test for train dataset")
plt.show()<jupyter_output><empty_output><jupyter_text>Gaussian Naive Bayes<jupyter_code>from sklearn.naive_bayes import GaussianNB
nb=GaussianNB()
nb.fit(x_train,y_train_01)
print("score:",nb.score(x_test,y_test_01))
print("real value of y_test_01[1]:"+str(y_test_01[1]))
print("real value of y_test_01[2]:" +str(y_test_01[2]))

#confusian matrix
from sklearn.metrics import confusion_matrix
cm_nb=confusion_matrix(y_test_01,nb.predict(x_test))
# cm visualization
import seaborn as sns
import matplotlib.pyplot as plt
f, ax = plt.subplots(figsize =(5,5))
sns.heatmap(cm_nb,annot = True,linewidths=0.5,linecolor="red",fmt = ".0f",ax=ax)
plt.title("Test for Test Dataset")
plt.xlabel("predicted y  values")
plt.ylabel("real y values")
plt.show()
from sklearn.metrics import precision_score,recall_score
print("precision_score",precision_score(y_test_01,nb.predict(x_test)))
print("recall_score:",recall_score(y_test_01,nb.predict(x_test)))

from sklearn.metrics import f1_score
print("f1_score:",f1_score(y_test_01,nb.predict(x_test)))<jupyter_output>precision_score 0.9333333333333333
recall_score: 0.9655172413793104
f1_score: 0.9491525423728815<jupyter_text>Test for Train Dataset:<jupyter_code>cm_nb_train=confusion_matrix(y_train_01,nb.predict(x_train))
f,ax=plt.subplots(figsize=(5,5))
sns.heatmap(cm_nb_train,annot=True,linewidths=0.5,linecolor="red",fmt="0.0f",ax=ax)
plt.xlabel("predicted y values")
plt.ylabel("predicted y values")
plt.title("Test for Train Dataset")
plt.show()<jupyter_output><empty_output><jupyter_text>Decision Tree Classification<jupyter_code>from sklearn.tree import DecisionTreeClassifier
dtc=DecisionTreeClassifier()
dtc.fit(x_train,y_train_01)
print("score:",dtc.score(x_test,y_test_01))
print("real value of y_test_01[1]:"+str(y_test_01[1]))
print("real value of y_test_01[2]:" +str(y_test_01[2]))
#confusian matrix
from sklearn.metrics import confusion_matrix
cm_dtc=confusion_matrix(y_test_01,nb.predict(x_test))
# cm visualization
import seaborn as sns
import matplotlib.pyplot as plt
f, ax = plt.subplots(figsize =(5,5))
sns.heatmap(cm_dtc,annot = True,linewidths=0.5,linecolor="red",fmt = ".0f",ax=ax)
plt.title("Test for Test Dataset")
plt.xlabel("predicted y  values")
plt.ylabel("real y values")
plt.show()
from sklearn.metrics import precision_score,recall_score
print("precision_score",precision_score(y_test_01,dtc.predict(x_test)))
print("recall_score:",recall_score(y_test_01,dtc.predict(x_test)))

from sklearn.metrics import f1_score
print("f1_score:",f1_score(y_test_01,dtc.predict(x_test)))<jupyter_output>precision_score 0.9285714285714286
recall_score: 0.896551724137931
f1_score: 0.912280701754386<jupyter_text>Test for Train Dataset:<jupyter_code>cm_dtc_train=confusion_matrix(y_train_01,dtc.predict(x_train))
f,ax=plt.subplots(figsize=(5,5))
sns.heatmap(cm_dtc_train,annot=True,linewidths=0.5,linecolor="red",fmt="0.0f",ax=ax)
plt.xlabel("predicted y values")
plt.ylabel("predicted y values")
plt.title("Test for Train Dataset")
plt.show()<jupyter_output><empty_output><jupyter_text>Random Forest Classification<jupyter_code>from sklearn.ensemble import RandomForestClassifier
rfc = RandomForestClassifier(n_estimators = 100,random_state = 1)
rfc.fit(x_train,y_train_01)
print("score: ", rfc.score(x_test,y_test_01))
print("real value of y_test_01[1]: " + str(y_test_01[1]) + " -> the predict: " + str(rfc.predict(x_test.iloc[[1],:])))
print("real value of y_test_01[2]: " + str(y_test_01[2]) + " -> the predict: " + str(rfc.predict(x_test.iloc[[2],:])))

# confusion matrix
from sklearn.metrics import confusion_matrix
cm_rfc = confusion_matrix(y_test_01,rfc.predict(x_test))
# print("y_test_01 == 1 :" + str(len(y_test_01[y_test_01==1]))) # 29
# cm visualization
import seaborn as sns
import matplotlib.pyplot as plt
f, ax = plt.subplots(figsize =(5,5))
sns.heatmap(cm_rfc,annot = True,linewidths=0.5,linecolor="red",fmt = ".0f",ax=ax)
plt.title("Test for Test Dataset")
plt.xlabel("predicted y values")
plt.ylabel("real y values")
plt.show()

from sklearn.metrics import precision_score, recall_score
print("precision_score: ", precision_score(y_test_01,rfc.predict(x_test)))
print("recall_score: ", recall_score(y_test_01,rfc.predict(x_test)))

from sklearn.metrics import f1_score
print("f1_score: ",f1_score(y_test_01,rfc.predict(x_test)))
cm_rfc_train = confusion_matrix(y_train_01,rfc.predict(x_train))
f, ax = plt.subplots(figsize =(5,5))
sns.heatmap(cm_rfc_train,annot = True,linewidths=0.5,linecolor="red",fmt = ".0f",ax=ax)
plt.xlabel("predicted y values")
plt.ylabel("real y values")
plt.title("Test for Train Dataset")
plt.show()<jupyter_output><empty_output><jupyter_text>K Nearest Neighbors Classification<jupyter_code>from sklearn.neighbors import KNeighborsClassifier

#finding k value
scores=[]
for each in range(1,50):
    knn_n=KNeighborsClassifier(n_neighbors=each)
    knn_n.fit(x_train,y_train_01)
    scores.append(knn_n.score(x_test,y_test_01))
plt.plot(range(1,50),scores) 
plt.xlabel("k")
plt.ylabel("accuracy")
plt.show()



knn = KNeighborsClassifier(n_neighbors = 3) # n_neighbors = k
knn.fit(x_train,y_train_01)
print("score of 3 :",knn.score(x_test,y_test_01))
print("real value of y_test_01[1]: " + str(y_test_01[1]) + " -> the predict: " + str(knn.predict(x_test.iloc[[1],:])))
print("real value of y_test_01[2]: " + str(y_test_01[2]) + " -> the predict: " + str(knn.predict(x_test.iloc[[2],:])))

#confusion matrix
from sklearn.metrics import confusion_matrix
cm_knn=confusion_matrix(y_test_01,knn.predict(x_test))

# cm visualization
import seaborn as sns
import matplotlib.pyplot as plt
f, ax = plt.subplots(figsize =(5,5))
sns.heatmap(cm_knn,annot = True,linewidths=0.5,linecolor="red",fmt = ".0f",ax=ax)
plt.title("Test for Test Dataset")
plt.xlabel("predicted y values")
plt.ylabel("real y values")
plt.show()

from sklearn.metrics import precision_score, recall_score
print("precision_score: ", precision_score(y_test_01,knn.predict(x_test)))
print("recall_score: ", recall_score(y_test_01,knn.predict(x_test)))

from sklearn.metrics import f1_score
print("f1_score: ",f1_score(y_test_01,knn.predict(x_test)))<jupyter_output><empty_output><jupyter_text>Comparison of Classification Algorithms * All classification algorithms achieved around 90% success. The most successful one is Gaussian Naive Bayes with 96% score.<jupyter_code>y = np.array([lrc.score(x_test,y_test_01),svm.score(x_test,y_test_01),nb.score(x_test,y_test_01),dtc.score(x_test,y_test_01),rfc.score(x_test,y_test_01),knn.score(x_test,y_test_01)])
x=["LogisticReg.","SVM","GNB","DEC. Tree","Ran.Forest","KNN"]

plt.bar(x,y)
plt.title("Comparison of Classification Algorithms")
plt.xlabel("Classificatin")
plt.ylabel("Score")
plt.show()<jupyter_output><empty_output><jupyter_text>CLUSTERING ALGORITHMS (UNSUPERVISED MACHINE LEARNING ALGORITHMS)  Preparing Data for Clustering<jupyter_code>df = pd.read_csv("Admission_Predict.csv",sep = ",")
df=df.rename(columns = {'Chance of Admit ':'ChanceOfAdmit'})
serial = df["Serial No."]
df.drop(["Serial No."],axis=1,inplace = True)
df = (df- np.min(df))/(np.max(df)-np.min(df))
y = df.ChanceOfAdmit 
x = df.drop(["ChanceOfAdmit"],axis=1)<jupyter_output><empty_output><jupyter_text>Principal Component Analysis<jupyter_code>#for data visualization
from sklearn.decomposition import PCA
pca=PCA(n_components=1,whiten=True) #whitten =normalize
pca.fit(x)
x_pca=pca.transform(x)
x_pca=x_pca.reshape(400,)
dictionary={"x":x_pca,"y":y}
data=pd.DataFrame(dictionary)
print("data")
print(data.head())
print("\ndf:")
print(df.head())<jupyter_output>data
          x         y
0 -1.377553  0.920635
1 -0.919033  0.666667
2 -0.320986  0.603175
3 -0.520960  0.730159
4  1.137743  0.492063

df:
   GRE Score  TOEFL Score  University Rating    SOP   LOR       CGPA  \
0       0.94     0.928571               0.75  0.875  0.875  0.913462   
1       0.68     0.535714               0.75  0.750  0.875  0.663462   
2       0.52     0.428571               0.50  0.500  0.625  0.384615   
3       0.64     0.642857               0.50  0.625  0.375  0.599359   
4       0.48     0.392857               0.25  0.250  0.500  0.451923   

   Research  ChanceOfAdmit  
0       1.0       0.920635  
1       1.0       0.666667  
2       1.0       0.603175  
3       1.0       0.730159  
4       0.0       0.492063<jupyter_text>K-means Clustering<jupyter_code>df["Serial No."] = serial
from sklearn.cluster import KMeans
wcss = []
for k in range(1,15):
    kmeans = KMeans(n_clusters=k)
    kmeans.fit(x)
    wcss.append(kmeans.inertia_)
plt.plot(range(1,15),wcss)
plt.xlabel("k values")
plt.ylabel("WCSS")
plt.show()

kmeans = KMeans(n_clusters=3)
clusters_knn = kmeans.fit_predict(x)

df["label_kmeans"] = clusters_knn


plt.scatter(df[df.label_kmeans == 0 ]["Serial No."],df[df.label_kmeans == 0].ChanceOfAdmit,color = "red")
plt.scatter(df[df.label_kmeans == 1 ]["Serial No."],df[df.label_kmeans == 1].ChanceOfAdmit,color = "blue")
plt.scatter(df[df.label_kmeans == 2 ]["Serial No."],df[df.label_kmeans == 2].ChanceOfAdmit,color = "green")
plt.title("K-means Clustering")
plt.xlabel("Candidates")
plt.ylabel("Chance of Admit")
plt.show()

df["label_kmeans"] = clusters_knn
plt.scatter(data.x[df.label_kmeans == 0 ],data[df.label_kmeans == 0].y,color = "red")
plt.scatter(data.x[df.label_kmeans == 1 ],data[df.label_kmeans == 1].y,color = "blue")
plt.scatter(data.x[df.label_kmeans == 2 ],data[df.label_kmeans == 2].y,color = "green")
plt.title("K-means Clustering")
plt.xlabel("X")
plt.ylabel("Chance of Admit")
plt.show()<jupyter_output><empty_output><jupyter_text>Hierarchical Clustering<jupyter_code>df["Serial No."] = serial

from scipy.cluster.hierarchy import linkage, dendrogram
merg = linkage(x,method="ward")
dendrogram(merg,leaf_rotation = 90)
plt.xlabel("data points")
plt.ylabel("euclidean distance")
plt.show()

from sklearn.cluster import AgglomerativeClustering
hiyerartical_cluster = AgglomerativeClustering(n_clusters = 3,affinity= "euclidean",linkage = "ward")
clusters_hiyerartical = hiyerartical_cluster.fit_predict(x)

df["label_hiyerartical"] = clusters_hiyerartical

plt.scatter(df[df.label_hiyerartical == 0 ]["Serial No."],df[df.label_hiyerartical == 0].ChanceOfAdmit,color = "red")
plt.scatter(df[df.label_hiyerartical == 1 ]["Serial No."],df[df.label_hiyerartical == 1].ChanceOfAdmit,color = "blue")
plt.scatter(df[df.label_hiyerartical == 2 ]["Serial No."],df[df.label_hiyerartical == 2].ChanceOfAdmit,color = "green")
plt.title("Hierarchical Clustering")
plt.xlabel("Candidates")
plt.ylabel("Chance of Admit")
plt.show()

plt.scatter(data[df.label_hiyerartical == 0 ].x,data.y[df.label_hiyerartical == 0],color = "red")
plt.scatter(data[df.label_hiyerartical == 1 ].x,data.y[df.label_hiyerartical == 1],color = "blue")
plt.scatter(data[df.label_hiyerartical == 2 ].x,data.y[df.label_hiyerartical == 2],color = "green")
plt.title("Hierarchical Clustering")
plt.xlabel("X")
plt.ylabel("Chance of Admit")
plt.show()
print(df.head())<jupyter_output>GRE Score  TOEFL Score  University Rating    SOP   LOR       CGPA  \
0       0.94     0.928571               0.75  0.875  0.875  0.913462   
1       0.68     0.535714               0.75  0.750  0.875  0.663462   
2       0.52     0.428571               0.50  0.500  0.625  0.384615   
3       0.64     0.642857               0.50  0.625  0.375  0.599359   
4       0.48     0.392857               0.25  0.250  0.500  0.451923   

   Research  ChanceOfAdmit  Serial No.  label_kmeans  label_hiyerartical  
0       1.0       0.920635           1             2                   1  
1       1.0       0.666667           2             2                   1  
2       1.0       0.603175           3             1                   2  
3       1.0       0.730159           4             1                   2  
4       0.0       0.492063           5             0                   0<jupyter_text>Comparison of Clustering Algorithms * K-means Clustering and Hierarchical Clustering are similarly.  THE THREE IMPORTANT FEATURES  Correlation between All Columns * The 3 most important features for admission to the Master: CGPA, GRE SCORE, and TOEFL SCORE* The 3 least important features for admission to the Master: Research, LOR, and SOP<jupyter_code>fig,ax=plt.subplots(figsize=(10,10))
sns.heatmap(df.corr(),ax=ax,annot=True,linewidths=0.05,fmt='.2f',cmap="magma")
plt.show()<jupyter_output><empty_output><jupyter_text>The Three Features for Linear Regression * The first results for Linear Regression (7 features): r_square score:  0.821208259148699* The results for Linear Regression now (3 features):                               r_square score:  0.8212241793299223* The two results are very close. If these 3 features (CGPA, GRE SCORE, and TOEFL SCORE) are used instead of all 7 features together, the result is not bad and performance is increased because less calculation is required.<jupyter_code>df = pd.read_csv("Admission_Predict.csv",sep = ",")
df=df.rename(columns = {'Chance of Admit ':'Chance of Admit'})
newDF = pd.DataFrame()
newDF["GRE Score"] = df["GRE Score"]
newDF["TOEFL Score"] = df["TOEFL Score"]
newDF["CGPA"] = df["CGPA"]
newDF["Chance of Admit"] = df["Chance of Admit"]

y_new = df["Chance of Admit"].values
x_new = df.drop(["Chance of Admit"],axis=1)

# separating train (80%) and test (%20) sets
from sklearn.model_selection import train_test_split
x_train_new, x_test_new,y_train_new, y_test_new = train_test_split(x_new,y_new,test_size = 0.20,random_state = 42)

# normalization
from sklearn.preprocessing import MinMaxScaler
scalerX = MinMaxScaler(feature_range=(0, 1))
x_train[x_train.columns] = scalerX.fit_transform(x_train[x_train.columns])
x_test[x_test.columns] = scalerX.transform(x_test[x_test.columns])

from sklearn.linear_model import LinearRegression
lr_new = LinearRegression()
lr_new.fit(x_train_new,y_train_new)
y_head_lr_new = lr_new.predict(x_test_new)

from sklearn.metrics import r2_score
print("r_square score: ", r2_score(y_test_new,y_head_lr_new))<jupyter_output>r_square score:  0.8212241793299226
{'fraction_non_alphanumeric': 0.14512250374655838, 'fraction_numerical': 0.052416965810476425, 'mean_word_length': 6.348015364916773, 'pattern_counts': {'":': 2, '<': 125, '<?xml version=': 0, '>': 160, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 1, 'word_list_counts': {'cursed_substrings.json': 13, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '34', 'dir': 'jupyter-structured-clean-dedup', 'id': '363730', 'license': 'Apache-2.0', 'n_tokens_mistral': 12115, 'n_tokens_neox': 11044, 'n_words': 2036, 'path': 'US_graduate_schools_admission_analysis.ipynb', 'repo_name': 'ranjankumar001/kaggle_Graduate_Admission', 'size': '28693'}