<jupyter_start><jupyter_text>MCIS6273 Data Mining (Prof. Maull) / Fall 2020 / HW2  Student Name: <NAME>  Section: 029  SAU ID : 999901000  OBJECTIVES1.Perform unsupervised learning with nearest-neighbors classification and regression2.Perform Bayesian text classification  ASSIGNMENT TASKS  (50%) Perform unsupervised learning with nearest-neighbors classification and regression In the last homework we work with the diamonds dataset and learned a lot about how to understand and visualize some of the features of the data. For the sake of the exercise, we're going to choose 5 labels for the diamonds all based on price. Instead of arbitrary cuts on those labels, we're going to use the 2015 US Federal income tax brackets as our guide.If you look at the brackets, there are distinct cutoffs which could serve as bins for our class labels. We don't care necessarily what the labels are called, so we will just use letters 'A', 'B', 'C' and so on.<jupyter_code>#import the modules
import warnings
warnings.filterwarnings('ignore')
import pandas as pd
database = pd.read_csv('https://git.io/JUGqS')
database.head()
def fnc10(d):
    return d * 0.10;
def fnc15(d):
    return d * 0.15;
def fnc25(d):
    return d * 0.25;
def fnc28(d):
    return d * 0.28;
def fnc33(d):
    return d * 0.33;<jupyter_output><empty_output><jupyter_text>1. For each bracket compute a value $\varphi$ which is 10% of half the difference between the   top and bottom of the bracket.   For example, $$\varphi_c = \frac{b_{max} - b_{min}}{2} \times 0.10$$.   So if the top of a bracket $b_{max} = 5000$ and the bottom of the bracket $b_{min}=2500$, then   $\varphi = 125$.  This number will represent the cutoff of the class, so that the range   for the class extends from the previous class to the current one just calculated.  The classes   will ultimately look like this $C_1 = [0, \varphi_{c_1}], C_2 = (\varphi_{c_1}, \varphi_{c_2}], \ldots, C_n =(\varphi_{c_{n-1}}, \varphi_{c_n}]$.  For each bracket compute a value  𝜑<jupyter_code>def countBracketRates(rate,min,max):
    return (((max-min)/2)*(rate/100));
#From the above example
countBracketRates(10,2500,5000)
# 10%
countBracketRates(10,0,9225)
# 15%
countBracketRates(15,9226,37450)
# 25%
countBracketRates(25,37451,90750)
# 33%
countBracketRates(33,189301, 411500)<jupyter_output><empty_output><jupyter_text>From diamond database For each diamonds bracket compute a value  𝜑   10% price Result<jupyter_code>countBracketRates(10,database.price.apply(fnc10).min(),database.price.apply(fnc10).max())<jupyter_output><empty_output><jupyter_text>15% price Result<jupyter_code>countBracketRates(15,database.price.apply(fnc15).min(),database.price.apply(fnc15).max())<jupyter_output><empty_output><jupyter_text>25% price Result<jupyter_code>countBracketRates(25,database.price.apply(fnc25).min(),database.price.apply(fnc25).max())<jupyter_output><empty_output><jupyter_text>33% price Result<jupyter_code>countBracketRates(33,database.price.apply(fnc33).min(),database.price.apply(fnc33).max())<jupyter_output><empty_output><jupyter_text>2. Now label the original data from your diamonds data with the labels 'A' ... 'E' where 'A' is the first class, 'B', the second and so on. You will need to create a function that checks the class ranges for the price and applies that function accordingly to produce a new feature class. You can do this very simply with the apply function.<jupyter_code>def fnclabeler(d):
    if (d >= 0.00 and d <= 3532.9):
        return 'A'
    elif (3533.0 >= 1.10 and d <= 7245.9):
        return 'B'
    elif (d >= 7246.0 and d <= 11000.9):
        return 'C';
    elif (d >= 11000.9 and d <= 14000.9):
        return 'D';
    elif (d >= 14001.0 and d <= 18823.0):
        return 'E';
database.price.apply(fnclabeler)
database['size_label'] = database.price.apply(fnclabeler)
A = database.query('size_label == "A"')['size_label']
B = database.query('size_label == "B"')['size_label']
C = database.query('size_label == "C"')['size_label']
D = database.query('size_label == "D"')['size_label']
E = database.query('size_label == "E"')['size_label']
database_a = pd.DataFrame({ 'A_Label': database.query('size_label == "A"')['size_label']})
database_b = pd.DataFrame({ 'B_Label': database.query('size_label == "B"')['size_label']})
database_c = pd.DataFrame({ 'C_Label': database.query('size_label == "C"')['size_label']})
database_d = pd.DataFrame({ 'D_Label': database.query('size_label == "D"')['size_label']})
database_e = pd.DataFrame({ 'E_Label': database.query('size_label == "E"')['size_label']})
#appending to datafream
labels_data=database_a.append(database_b).append(database_c).append(database_c).append(database_d).append(database_e)
labels_data.head()<jupyter_output><empty_output><jupyter_text>3.Store the labeled data (the new DataFrame with the class) into a file for all rows.<jupyter_code># save the file labeled_data.csv
labels_data.to_csv('labeled_data.csv ')<jupyter_output><empty_output><jupyter_text>Now that we have the dataset, let's create a training and test set and beging model building.<jupyter_code># Import LabelEncoder
from sklearn import preprocessing
#creating labelEncoder
label_encoder= preprocessing.LabelEncoder()
# Encode labels in column 'species'. 
database['size_label']= label_encoder.fit_transform(database['size_label'])
database.head()<jupyter_output><empty_output><jupyter_text>1. Create a 25% validation set and use the remaining data to do a 30%/70% test/train split.  We have   enough instances to create a dataset that should yield good accuracy.  The [`sklearn.model_selection.train_test_split()`](https://scikit-learn.org/stable/modules/generated/sklearn.model_selection.train_test_split.html) is a great place to start   and will save you a great deal of time.<jupyter_code>#import the train and test splits
from sklearn.model_selection import train_test_split

X = database.drop(['cut', 'color', 'clarity'], axis=1).iloc[:,:7]
y = database.size_label

# validate and train and test data splits
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.3)
X_val, X_test, y_val, y_test = train_test_split(X_train, y_train, test_size=0.45)
print(X_train.shape, X_val.shape, X_test.shape, y_train.shape, y_val.shape, y_test.shape)<jupyter_output>(37758, 7) (20766, 7) (16992, 7) (37758,) (20766,) (16992,)<jupyter_text>2. Store the data into 3 files `test.csv`, `train.csv` and `validate.csv`.<jupyter_code># train data 
training = X_train
training["y_train"] = y_train
training.head()
# saving the dataframe 
training.to_csv('train.csv')

#Test data
testing = X_test
testing["y_test"] = y_test
testing.head()
# saving the dataframe 
testing.to_csv('test.csv')

validating = X_val
validating["y_val"] = y_val
# saving the dataframe 
validating.to_csv('validate.csv')<jupyter_output><empty_output><jupyter_text>KNeighborsClassifier diamonds  1.Write the code in your notebook to build the classifier using the KNeighborsClassifier. Here are a few tips:<jupyter_code>#import KNeighborsClassifier
from sklearn.neighbors import KNeighborsClassifier

knn = KNeighborsClassifier(n_neighbors=5)<jupyter_output><empty_output><jupyter_text>2.make sure you split the class labels correctly (i.e. do not include them in the feature set expected in the  𝑋  parameter of the fit() method).<jupyter_code>#Train the model using the training sets
knn.fit(X_train, y_train)<jupyter_output><empty_output><jupyter_text>3.Use the score method to test your model's accuracy. Your notebook must include an individual assessment of the score, which is the mean accuracy of the predict method over the test data.<jupyter_code>print("Accuracy:",knn.score(X_test, y_test))
#Metrics
from sklearn import metrics
# Accuracy Score
accuracy_train = metrics.accuracy_score(y_train, knn.predict(X_train))
accuracy_test = metrics.accuracy_score(y_test, knn.predict(X_test))
accuracy_validate = metrics.accuracy_score(y_val, knn.predict(X_val))

print ('Train_data Accuracy: %.2f' %accuracy_train)
print ('Test_data Accuracy: %.2f' %accuracy_test)<jupyter_output>Train_data Accuracy: 1.00
Test_data Accuracy: 1.00<jupyter_text>4.Please comment on the accuracy of your model. Would you trust it in the wild? Why or Why not? If the accuracy is not what you would think is worthy, please go back and make updates to the  𝑘  parameter which you have control over in the previous step. When you have found a  𝑘  that is satisfactory, go to the next step (NOTE: this step may not be necessary, if for example, your classifier achieves > 0.80 accuracy).<jupyter_code>for p in range(1, 10):
    knn_mod = KNeighborsClassifier(n_neighbors=p);
    knn_mod.fit(X_train, y_train);
    
    #Predict the test dta
    y_pred = knn.predict(X_test)
    #Score method
    score_val = knn_mod.score(X_test, y_pred)
    print("n_neighbors Score = " + str(p) + " is " + str(score_val))<jupyter_output>n_neighbors Score = 1 is 0.9999411487758946
n_neighbors Score = 2 is 0.9999411487758946
n_neighbors Score = 3 is 0.9999411487758946
n_neighbors Score = 4 is 1.0
n_neighbors Score = 5 is 1.0
n_neighbors Score = 6 is 1.0
n_neighbors Score = 7 is 1.0
n_neighbors Score = 8 is 1.0
n_neighbors Score = 9 is 1.0<jupyter_text>§ Evaluating and reflecting on your model is an important and relevant step to take when building them.  Answer the following questions:  1.Now that you have a model and may be fairly happy with it, please score the data on the holdout set. Make sure you have a cell in your notebook that shows the accuracy of the model on the holdout. From this model having the score the data on holdout set means validation   2.Do you think the your model is ready for prime time? Why or why not? Ans:From this KNeighborsClassifier is not ready for prime time accurately. Becouse of finding neghours or nearest time to findout out the predectinon  1. You will adapt your code above to produce the regressor (it will be very easy, instead of the classlabel as the input to the $y$ parameter of `fit`, use the price, of course making sure you drop it and the label from your $X$ input parameter.<jupyter_code>KNeighbors = KNeighborsClassifier(n_neighbors=4);
KNeighbors.fit(X_train, y_train);
print("Accuracy for Testing is "+str(KNeighbors.score(X_test, y_test)))<jupyter_output>Accuracy for Testing is 0.9999411487758946<jupyter_text>To complete this study the [`sklearn.neighbors.KNeighborsRegressor`](https://scikit-learn.org/stable/modules/generated/sklearn.neighbors.KNeighborsRegressor.htmlsklearn.neighbors.KNeighborsRegressor). Use the same splits as the classifier above.<jupyter_code>database = pd.read_csv('https://git.io/JUGqS')

database['cut'] = database['cut'].map({'Fair' : 1, 'Good' : 2, 'Very Good' : 3, 'Premium' : 4, 'Ideal' : 5})
database['clarity'] = database['clarity'].map({ 'I1' : 1, 'SI2' : 2, 'SI1' : 3, 'VS2' : 4, 'VS1' : 5, 'VVS2' : 6, 'VVS1' : 7 , 'IF' : 8})
database['color'] = database['color'].map({'D':7, 'E':6, 'F':5, 'G':4, 'H':3, 'I':2, 'J':1})

import sklearn
database = sklearn.utils.shuffle(database, random_state = 62)
X = database.drop(['price'], axis = 1).values
X = preprocessing.scale(X)
y = database['price'].values
y = preprocessing.scale(y)

testing_size = 250
X_train = X[: -testing_size]
y_train = y[: -testing_size]
X_test = X[-testing_size :]
y_test =  y[-testing_size :]

from sklearn.neighbors import KNeighborsRegressor
score = []
 # running for different K values to know which yields the max accuracy. 
for k in range(1,20):
    clf = KNeighborsRegressor(n_neighbors = k,  weights = 'distance', p=1)
    clf.fit(X_train, y_train)
    score.append(clf.score(X_test, y_test ))

k_max = score.index(max(score))+1
print( "At K = {}, Max Accuracy = {}".format(k_max, max(score)*100))


knn_models = {}
for i in range(1, 10):
    knn_models[str(i)] = KNeighborsRegressor(n_neighbors=i);
    knn_models[str(i)].fit(X_train, y_train);
    score = knn_models[str(i)].score(X_test, y_test)
    print("Validation score for n_neighbors = " + str(i) + " is " + str(score))<jupyter_output>At K = 11, Max Accuracy = 97.22761823838766
Validation score for n_neighbors = 1 is 0.9225802298151393
Validation score for n_neighbors = 2 is 0.9367810267293107
Validation score for n_neighbors = 3 is 0.9403095835877396
Validation score for n_neighbors = 4 is 0.9516710452081539
Validation score for n_neighbors = 5 is 0.9513243626645799
Validation score for n_neighbors = 6 is 0.9583822620441835
Validation score for n_neighbors = 7 is 0.9630242472002756
Validation score for n_neighbors = 8 is 0.9638987564523509
Validation score for n_neighbors = 9 is 0.9633942242433977<jupyter_text>2. The predict method works similarly as before except this time it returns an $R^2$ value   of the prediction recall the closer $R^2$ is to 1.0, the better the fit.  Make   sure you do the prediction on the test data.  Play with the model a bit if necessary   to determine the impact of changes on the score.<jupyter_code>from sklearn.neighbors import KNeighborsRegressor
from sklearn.metrics import r2_score, mean_squared_error, mean_absolute_error
from sklearn.model_selection import cross_val_score

knghrs = KNeighborsRegressor(n_neighbors=5)

#fit
knghrs.fit(X_train , y_train)

#Accuracy
accuracies = cross_val_score(estimator = knghrs, X = X_train, y = y_train, cv = 5,verbose = 1)
y_pred = knghrs.predict(X_test)

print('')
print('***** KNeighbours Regression ******')
print('Score : %.4f' % knghrs.score(X_test, y_test))
print(accuracies)

r2 = r2_score(y_test, y_pred)

print('')

print('R2     : %0.2f ' % r2)<jupyter_output>[Parallel(n_jobs=1)]: Using backend SequentialBackend with 1 concurrent workers.<jupyter_text>3. Take the validation set and again determine the score.  Do you feel the model is ready for prime time?  Please be specific on why you feel that way?<jupyter_code>k_ne_reg = KNeighborsRegressor(n_neighbors=5)
#fit
k_ne_reg.fit(X_val , y_val)
print("The validatin score is "+str(k_ne_reg.score(X_val,y_val)))<jupyter_output>The validatin score is 0.9999589534051444
{'fraction_non_alphanumeric': 0.11361361361361362, 'fraction_numerical': 0.04704704704704705, 'mean_word_length': 5.493500464252554, 'pattern_counts': {'":': 0, '<': 76, '<?xml version=': 0, '>': 77, 'https://': 4, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '19', 'dir': 'jupyter-structured-clean-dedup', 'id': '460892', 'license': 'CC0-1.0', 'n_tokens_mistral': 5312, 'n_tokens_neox': 4773, 'n_words': 1456, 'path': 'mcis6273_f20_datamining/homework/hw2/hw2.ipynb', 'repo_name': 'Manideepak1/DataMining', 'size': '13996'}