<jupyter_start><jupyter_text>Data Set Sursă dataset:: A fost folosit data setul numit Dermatology care se poate descarca de la adresa http://archive.ics.uci.edu/ml/datasets/dermatologyArticol relevant:: <NAME>, <NAME>, and <NAME>, "Learning Differential Diagnosis of Eryhemato-Squamous Diseases using Voting Feature Intervals", Aritificial Intelligence in MedicineScurta decriere: Baza de date cuprinde informatii privitoare la conditiile necesare pentru diagnosticarea diferentiata pentru bolile de piele. Cuprinde 366 intrari si 33 atribute ce pot lua valori intre 0 si 3, fiecare dintre acestea indicand severitatea unei conditii medicale. Sunt incluse  6 clase codificate cu cifre intr 1 si 6 dupa cum urmeaza: Cuprinde Clasele:                    psoriasis			        seboreic dermatitis         lichen planus              pityriasis rosea            cronic dermatitis           pityriasis rubra pilaris   Load data<jupyter_code>header =['erythema','scaling','definite borders','itching','koebner phenomenon','polygonal papules',
'follicular papules','oral mucosal involvement','knee and elbow involvement','scalp involvement','family history','melanin incontinence',
'eosinophils in the infiltrate','PNL infiltrate','fibrosis of the papillary dermis','exocytosis','acanthosis','hyperkeratosis',
'parakeratosis','clubbing of the rete ridges','elongation of the rete ridges','thinning of the suprapapillary epidermis','spongiform pustule','munro microabcess','focal hypergranulosis',
'disappearance of the granular layer','vacuolisation and damage of basal layer','spongiosis','saw-tooth appearance of retes','follicular horn plug','perifollicular parakeratosis','inflammatory monoluclear inflitrate',
'band-like infiltrate','Age','Class']

print(len(header))
#Alcatuim o lista de stringuri ce ar putea reprezenta missing data in data setul nostru
missingValues = ["n/a", "na", "--"," ","?"]
data_spam = pd.read_csv("./Dataset/dermatology.data",names=header,sep=',',na_values=missingValues)<jupyter_output>35<jupyter_text>Se observa existenta unor date lipsa,fiecare coloana ar trebui sa aibe 4601 intrari, acest lucru reiese din informatiile despre setul de date<jupyter_code>display(data_spam.info())
#Afisam numarul de missind data in functie de coloana
print (data_spam.isnull().sum())
#Afisam numarul de missind data in total
print (data_spam.isnull().sum().sum())

#data_spam.replace(to_replace=missingValues, value=np.nan, inplace=True)
display(HTML("<h3><b>Email Spam Dataset"))
display(HTML(data_spam[5:27].to_html()))<jupyter_output><empty_output><jupyter_text>Inlocuim datele lipsa<jupyter_code>imputer = SimpleImputer(missing_values=np.nan, strategy='mean')
print(imputer)
imputer.fit(data_spam)
data_spam = pd.DataFrame(data=imputer.transform(data_spam), columns=header)



display(HTML("<i>Dataset extract with values imputed:</i>"))
display(HTML(data_spam[5:20].to_html()))

X = data_spam.values[:, :34]
y = data_spam.values[:, -1]

display(X)
display(np.unique(y))<jupyter_output>SimpleImputer(add_indicator=False, copy=True, fill_value=None,
              missing_values=nan, strategy='mean', verbose=0)<jupyter_text>Print Function<jupyter_code>def print_function(data_set:dict):
    df_print = pd.DataFrame({"Test accuracy for each fold":data_set['test_accuracy'], 
                    "Train accuracy for each fold": data_set['train_accuracy'], 
                    "Average test accuracy %": round(data_set['test_accuracy'].mean() * 100, 4),
                    "Average train accuracy %": round(data_set['train_accuracy'].mean() * 100, 4),
                    "Test F1 score for each fold": data_set['test_f1_macro'],
                    "Train F1 score for each fold": data_set['train_f1_macro'],
                    "Average test F1 score %": round(data_set['test_f1_macro'].mean() * 100, 4),
                    "Average train F1 score %":round(data_set['train_f1_macro'].mean() * 100, 4)
                   })
    display(HTML(df_print.to_html()))<jupyter_output><empty_output><jupyter_text>K-Nearest Neighbors Classifier<jupyter_code># hiperparametri
knn_neighbors = 5
knn_minkowski_p = 3

# scalare date
scaler = MinMaxScaler()
X_scaled = scaler.fit_transform(X)

# implementare KNN
model = KNeighborsClassifier(n_neighbors=knn_neighbors, p=knn_minkowski_p)
model_cv_stats = cross_validate(model, X_scaled, y, cv=5, scoring=('accuracy', 'f1_macro'), return_train_score=True) 

# statistici
display(HTML(f"<h4>5-fold cross validation for {knn_neighbors}-nearest neighbors classification:</h4>"))
print_function(model_cv_stats)<jupyter_output><empty_output><jupyter_text>Decision Tree Classifier<jupyter_code># hiperparametri
dt_criterion = 'gini'#default value
dt_splitter = 'best' #intrebarea care reduce cel mai mult incertitudinea
#gini imputity-cantitatea de incertitudine pe un singur nod, cat de amestecate sunt clasificarile din frunze dupa intrebarea din nod
 
# implementare Decision Tree
model = DecisionTreeClassifier(criterion=dt_criterion, splitter=dt_splitter)
model_dc_stats = cross_validate(model, X, y, cv=5, scoring=('accuracy', 'f1_macro'), return_train_score=True)

# statistici
display(HTML(f"<h4>5-fold cross validation for Decision Trees classification:</h4>"))
print_function(model_dc_stats)<jupyter_output><empty_output><jupyter_text>Random Forest Classifier<jupyter_code># hiperparametri
rfc_n_estimators = 150
rfc_criterion = 'gini'

# implementare Random Forest
model = RandomForestClassifier(n_estimators=rfc_n_estimators, criterion=rfc_criterion)
model_rfc_stats = cross_validate(model, X, y, cv=5, scoring=('accuracy', 'f1_macro'), return_train_score=True)

display(HTML(f"<h4>5-fold cross validation for Random Forest classification</h4>"))
print_function(model_rfc_stats)<jupyter_output><empty_output><jupyter_text>Multilayer Perceptron Classifier<jupyter_code># hiperparametri
mlp_solver = 'adam'
mlp_activation = 'logistic'
mlp_alpha=1e-3
mlp_hidden_layer_sizes = (50,50)
max_iter=1000

# implementare MLP
model = MLPClassifier(solver=mlp_solver, activation=mlp_activation, alpha=mlp_alpha, hidden_layer_sizes=mlp_hidden_layer_sizes, max_iter=max_iter, random_state=0)
model_mpc_stats = cross_validate(model, X, y, cv=5, scoring=('accuracy', 'f1_macro'), return_train_score=True)
print_function(model_mpc_stats)<jupyter_output><empty_output><jupyter_text>Gaussian Naive Bayes Classifier<jupyter_code># implementare GNB
model = GaussianNB()
model_gnb_stats = cross_validate(model, X, y, cv=5, scoring=('accuracy', 'f1_macro'), return_train_score=True)

# statistici
display(HTML(f"<h4>5-fold cross validation for Gaussian NB classification</h4>"))
print_function(model_gnb_stats)<jupyter_output><empty_output><jupyter_text>Optimizarea Hiperparametrilor  K-Nearest Neighbors Classifier<jupyter_code>pipe = Pipeline([('scaler', MinMaxScaler()), ('knn', KNeighborsClassifier())])
parameter_grid = {'knn__n_neighbors': list(range(1, 10)), 'knn__p': list(range(1, 5))}
strat_k_fold = StratifiedKFold(n_splits=5, shuffle=True)
grid_search =GridSearchCV(pipe, param_grid=parameter_grid, scoring='accuracy', cv=4, return_train_score=True)

scores = cross_val_score(grid_search, X, y, cv=strat_k_fold )
print("Scorurile rezultate in urma 5-fold cross validation",scores)
print("Media scorurilor",scores.mean())


grid_search.fit(X, y)
print("Cel mai bun set de parametrii...",grid_search.best_params_)

grid_search = pd.DataFrame(grid_search.cv_results_)
display(HTML(grid_search.to_html()))<jupyter_output>Scorurile rezultate in urma 5-fold cross validation [0.95945946 0.98630137 0.98630137 0.94520548 0.94520548]
Media scorurilor 0.9644946316179193
Cel mai bun set de parametrii... {'knn__n_neighbors': 6, 'knn__p': 1}<jupyter_text>Decision Tree Classifier<jupyter_code>pipe = Pipeline([('dtc', DecisionTreeClassifier())])

parameter_grid = { 'dtc__criterion': ['gini', 'entropy'],'dtc__splitter': ['best', 'random'] }
strat_k_fold = StratifiedKFold(n_splits=5, shuffle=True)
grid_search =GridSearchCV(pipe, param_grid=parameter_grid, scoring='accuracy', cv=4)
strat_k_fold = StratifiedKFold(n_splits=5, shuffle=True)
scores = cross_val_score(grid_search, X, y, cv=strat_k_fold )

print("Scorurile rezultate in urma 5-fold cross validation",scores)
print("Media scorurilor",scores.mean())
grid_search.fit(X, y)
print("Cel mai bun set de parametrii...",grid_search.best_params_)
grid_search = pd.DataFrame(grid_search.cv_results_)
display(HTML(grid_search.to_html()))<jupyter_output>Scorurile rezultate in urma 5-fold cross validation [0.93243243 0.95890411 0.93150685 0.93150685 0.95890411]
Media scorurilor 0.9426508700481303
Cel mai bun set de parametrii... {'dtc__criterion': 'gini', 'dtc__splitter': 'best'}<jupyter_text>Random Forest Classifier<jupyter_code>pipe = Pipeline([('rfc', RandomForestClassifier())])

parameter_grid = { 'rfc__criterion': ['gini', 'entropy'],'rfc__n_estimators': np.linspace(start=1, stop=150, num=25, dtype=int)}
strat_k_fold = StratifiedKFold(n_splits=5, shuffle=True)
grid_search =GridSearchCV(pipe, param_grid=parameter_grid, scoring='accuracy', cv=4)
strat_k_fold = StratifiedKFold(n_splits=5, shuffle=True)
scores = cross_val_score(grid_search, X, y, cv=strat_k_fold )

print("Scorurile rezultate in urma 5-fold cross validation",scores)
print("Media scorurilor",scores.mean())
grid_search.fit(X, y)
print("Cel mai bun set de parametrii...",grid_search.best_params_)
grid_search = pd.DataFrame(grid_search.cv_results_)
display(HTML(grid_search.to_html()))

#Do not run this it can take a while<jupyter_output>Scorurile rezultate in urma 5-fold cross validation [0.95945946 0.97260274 0.98630137 1.         0.97260274]
Media scorurilor 0.9781932617549056
Cel mai bun set de parametrii... {'rfc__criterion': 'gini', 'rfc__n_estimators': 118}<jupyter_text>Gaussian Naive Bayes Classifier<jupyter_code>pipe = Pipeline([('gnb', GaussianNB())])

parameter_grid = { 'gnb__var_smoothing': np.linspace(start=1e-9, stop=1e-2, num=100)}
strat_k_fold = StratifiedKFold(n_splits=5, shuffle=True)
grid_search =GridSearchCV(pipe, param_grid=parameter_grid, scoring='accuracy', cv=4)
strat_k_fold = StratifiedKFold(n_splits=5, shuffle=True)
scores = cross_val_score(grid_search, X, y, cv=strat_k_fold )

print("Scorurile rezultate in urma 5-fold cross validation",scores)
print("Media scorurilor",scores.mean())
grid_search.fit(X, y)
print("Cel mai bun set de parametrii...",grid_search.best_params_)
grid_search = pd.DataFrame(grid_search.cv_results_)
display(HTML(grid_search.to_html()))<jupyter_output>Scorurile rezultate in urma 5-fold cross validation [0.90540541 0.97260274 1.         0.98630137 0.97260274]
Media scorurilor 0.9673824509440948
Cel mai bun set de parametrii... {'gnb__var_smoothing': 0.0004040413636363637}<jupyter_text>Multilayer Perceptron Classifier<jupyter_code>pipe = Pipeline([('mlp', MLPClassifier(max_iter= 1000))])

parameter_grid = { 'mlp__alpha': np.linspace(start=0, stop=1e-1, num=50),
                  'mlp__activation': ['identity', 'logistic', 'tanh', 'relu']
                 }

strat_k_fold = StratifiedKFold(n_splits=5, shuffle=True)
grid_search =GridSearchCV(pipe, param_grid=parameter_grid, scoring='accuracy', cv=4)
strat_k_fold = StratifiedKFold(n_splits=5, shuffle=True)
scores = cross_val_score(grid_search, X, y, cv=strat_k_fold )

print("Scorurile rezultate in urma 5-fold cross validation",scores)
print("Media scorurilor",scores.mean())
grid_search.fit(X, y)
print("Cel mai bun set de parametrii...",grid_search.best_params_)
grid_search = pd.DataFrame(grid_search.cv_results_)
display(HTML(grid_search.to_html()))

#Do not run this it can take a while<jupyter_output>C:\Anaconda3\envs\machine-learning\lib\site-packages\sklearn\neural_network\_multilayer_perceptron.py:573: UserWarning: Training interrupted by user.
  warnings.warn("Training interrupted by user.")
C:\Anaconda3\envs\machine-learning\lib\site-packages\sklearn\neural_network\_multilayer_perceptron.py:573: UserWarning: Training interrupted by user.
  warnings.warn("Training interrupted by user.")
{'fraction_non_alphanumeric': 0.12004001333777926, 'fraction_numerical': 0.033677892630876956, 'mean_word_length': 7.106081081081081, 'pattern_counts': {'":': 8, '<': 65, '<?xml version=': 0, '>': 65, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '14', 'dir': 'jupyter-structured-clean-dedup', 'id': '29594', 'license': 'MIT', 'n_tokens_mistral': 4633, 'n_tokens_neox': 4231, 'n_words': 863, 'path': 'Laborator6/Dermatology.ipynb', 'repo_name': 'MateiSirbu/IDS_Sirbu_Tataru', 'size': '12009'}