<jupyter_start><jupyter_text>**Attribute Information:**   1.  ID number 2.  Diagnosis (M = malignant, B = benign) **Ten real-valued features are computed for each cell nucleus:**1. Radius (mean of distances from center to points on the perimeter) 2. Texture (standard deviation of gray-scale values) 3. Perimeter 4. Area 5. Smoothness (local variation in radius lengths) 6. Compactness (perimeter^2 / area - 1.0) 7. Concavity (severity of concave portions of the contour) 8. Concave points (number of concave portions of the contour) 9. Symmetry 10. Fractal dimension ("coastline approximation" - 1)<jupyter_code>import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
import seaborn as sns
%matplotlib inline
from sklearn.datasets import load_breast_cancer
cancer = load_breast_cancer()
cancer.DESCR<jupyter_output><empty_output><jupyter_text>As we can see above, not much can be done in the current form of the dataset. We need to view the data in a better format.<jupyter_code>df_cancer = pd.DataFrame(np.c_[cancer['data'], cancer['target']], columns = np.append(cancer['feature_names'], ['target']))
df_cancer.head()<jupyter_output><empty_output><jupyter_text>**Dataset info found from sklearn dataset repository**- mean radius = mean of distances from center to points on the perimeter- mean texture = standard deviation of gray-scale values- mean perimeter = mean size of the core tumor- mean area = - mean smoothness = mean of local variation in radius lengths- mean compactness = mean of perimeter^2 / area - 1.0- mean concavity = mean of severity of concave portions of the contour- mean concave points = mean for number of concave portions of the contour- mean symmetry =- mean fractal dimension = mean for "coastline approximation" - 1- radius error = standard error for the mean of distances from center to points on the perimeter- texture error = standard error for standard deviation of gray-scale values- perimeter error =- area error =- smoothness error = standard error for local variation in radius lengths- compactness error = standard error for perimeter^2 / area - 1.0- concavity error = standard error for severity of concave portions of the contour- concave points error = standard error for number of concave portions of the contour- symmetry error =- fractal dimension error = standard error for "coastline approximation" - 1- worst radius = "worst" or largest mean value for mean of distances from center to points on the perimeter- worst texture = "worst" or largest mean value for standard deviation of gray-scale values- worst perimeter =- worst smoothness = "worst" or largest mean value for local variation in radius lengths- worst compactness = "worst" or largest mean value for perimeter^2 / area - 1.0- worst concavity = "worst" or largest mean value for severity of concave portions of the contour- worst concave points = "worst" or largest mean value for number of concave portions of the contour- worst fractal dimension = "worst" or largest mean value for "coastline approximation" - 1<jupyter_code>df_cancer.shape
df_cancer.columns<jupyter_output><empty_output><jupyter_text>The above plots shows the relationship between our features. But the only problem with them is that they do not show us which of the "dots" is Malignant and which is Benign. This issue will be addressed below by using "target" variable as the "hue" for the plots.<jupyter_code>sns.pairplot(df_cancer, hue = 'target', vars = ['mean radius', 'mean texture', 'mean perimeter','mean area','mean smoothness'] )<jupyter_output><empty_output><jupyter_text>**Note:**       1.0 (Orange) = Benign (No Cancer)    0.0 (Blue) = Malignant (Cancer)<jupyter_code>df_cancer['target'].value_counts()<jupyter_output><empty_output><jupyter_text>correlation between features<jupyter_code>plt.figure(figsize=(20,12)) 
sns.heatmap(df_cancer.corr(), annot=True)<jupyter_output><empty_output><jupyter_text>There is a strong correlation between the mean radius and mean perimeter, mean area and mean primeter  Model Training ** var names as below **- "y" = predicted feature (Output). this will be "target".- "X" = remaining columns from df.<jupyter_code>X = df_cancer.drop(['target'], axis = 1) # drop "target" feature for training.
X.head()
y = df_cancer['target']
y.head()
from sklearn.model_selection import train_test_split<jupyter_output><empty_output><jupyter_text>**80:20 ratio for train test splits**<jupyter_code>X_train, X_test, y_train, y_test = train_test_split(X, y, test_size = 0.2, random_state = 20)
print ('Train "X" (input features) shape ', X_train.shape)
print ('Test "X" (input features) shape ', X_test.shape)
print ('Train "y" (output feature) shape ', y_train.shape)
print ('Test "y" (output features) shape ', y_test.shape)
from sklearn.svm import SVC
svc_model = SVC()
svc_model.fit(X_train, y_train)
y_predict = svc_model.predict(X_test)
from sklearn.metrics import classification_report, confusion_matrix
cm = np.array(confusion_matrix(y_test, y_predict, labels=[1,0]))
confusion = pd.DataFrame(cm, index=['is_cancer', 'is_healthy'],
                         columns=['predicted_cancer','predicted_healthy'])
confusion
sns.heatmap(confusion, annot=True)
print(classification_report(y_test, y_predict))<jupyter_output>precision    recall  f1-score   support

        0.0       0.00      0.00      0.00        48
        1.0       0.58      1.00      0.73        66

avg / total       0.34      0.58      0.42       114<jupyter_text>**there are a lot of false positives in this model**<jupyter_code>X_train_min = X_train.min()
X_train_min
X_train_max = X_train.max()
X_train_max
X_train_range = (X_train_max- X_train_min)
X_train_range
X_train_scaled = (X_train - X_train_min)/(X_train_range)
X_train_scaled.head()
X_test_min = X_test.min()
X_test_range = (X_test - X_test_min).max()
X_test_scaled = (X_test - X_test_min)/X_test_range
svc_model = SVC()
svc_model.fit(X_train_scaled, y_train)
y_predict = svc_model.predict(X_test_scaled)
cm = confusion_matrix(y_test, y_predict)<jupyter_output><empty_output><jupyter_text>SVM with Normalized data<jupyter_code>cm = np.array(confusion_matrix(y_test, y_predict, labels=[1,0]))
confusion = pd.DataFrame(cm, index=['is_cancer', 'is_healthy'],
                         columns=['predicted_cancer','predicted_healthy'])
confusion
sns.heatmap(confusion,annot=True,fmt="d")
print(classification_report(y_test,y_predict))<jupyter_output>precision    recall  f1-score   support

        0.0       1.00      0.98      0.99        48
        1.0       0.99      1.00      0.99        66

avg / total       0.99      0.99      0.99       114<jupyter_text>**only 1 false positive**  Further Model Improvement  Gridsearch Model<jupyter_code>param_grid = {'C': [0.1, 1, 10, 100], 'gamma': [1, 0.1, 0.01, 0.001], 'kernel': ['rbf']} 
from sklearn.model_selection import GridSearchCV
grid = GridSearchCV(SVC(),param_grid,refit=True,verbose=4)
grid.fit(X_train_scaled,y_train)<jupyter_output>Fitting 3 folds for each of 16 candidates, totalling 48 fits
[CV] C=0.1, gamma=1, kernel=rbf ......................................
[CV]  C=0.1, gamma=1, kernel=rbf, score=0.9605263157894737, total=   0.0s
[CV] C=0.1, gamma=1, kernel=rbf ......................................
[CV]  C=0.1, gamma=1, kernel=rbf, score=0.9407894736842105, total=   0.0s
[CV] C=0.1, gamma=1, kernel=rbf ......................................
[CV]  C=0.1, gamma=1, kernel=rbf, score=0.9403973509933775, total=   0.0s
[CV] C=0.1, gamma=0.1, kernel=rbf ....................................
[CV]  C=0.1, gamma=0.1, kernel=rbf, score=0.8881578947368421, total=   0.0s
[CV] C=0.1, gamma=0.1, kernel=rbf ....................................
[CV]  C=0.1, gamma=0.1, kernel=rbf, score=0.868421052631579, total=   0.0s
[CV] C=0.1, gamma=0.1, kernel=rbf ....................................
[CV]  C=0.1, gamma=0.1, kernel=rbf, score=0.8741721854304636, total=   0.0s
[CV] C=0.1, gamma=0.01, kernel=rbf .............................[...]<jupyter_text>**Let's print out the "grid" with the best parameter**<jupyter_code>print (grid.best_params_)
print ('\n')
print (grid.best_estimator_)<jupyter_output>{'C': 100, 'gamma': 0.01, 'kernel': 'rbf'}


SVC(C=100, cache_size=200, class_weight=None, coef0=0.0,
  decision_function_shape='ovr', degree=3, gamma=0.01, kernel='rbf',
  max_iter=-1, probability=False, random_state=None, shrinking=True,
  tol=0.001, verbose=False)<jupyter_text>**best parameters are "C" = 100, "gamma" = "0.01" and "kernel" = 'rbf'**<jupyter_code>grid_predictions = grid.predict(X_test_scaled)
cm = np.array(confusion_matrix(y_test, grid_predictions, labels=[1,0]))
confusion = pd.DataFrame(cm, index=['is_cancer', 'is_healthy'],
                         columns=['predicted_cancer','predicted_healthy'])
confusion
sns.heatmap(confusion, annot=True)
print(classification_report(y_test,grid_predictions))<jupyter_output>precision    recall  f1-score   support

        0.0       0.94      1.00      0.97        48
        1.0       1.00      0.95      0.98        66

avg / total       0.98      0.97      0.97       114<jupyter_text>**no FP** --- Reference table of common classification metric terms and definitions|  TERM | DESCRIPTION  ||---|---||**TRUE POSITIVES** | The number of "true" classes correctly predicted to be true by the model.  `TP = Sum of observations predicted to be 1 that are actually 1`The true class in a binary classifier is labeled with 1.||**TRUE NEGATIVES** | The number of "false" classes correctly predicted to be false by the model.  `TN = Sum of observations predicted to be 0 that are actually 0`The false class in a binary classifier is labeled with 0.||**FALSE POSITIVES** | The number of "false" classes incorrectly predicted to be true by the model. This is the measure of **Type I error**. `FP = Sum of observations predicted to be 1 that are actually 0`Remember that the "true" and "false" refer to the veracity of your guess, and the "positive" and "negative" component refer to the guessed label.||**FALSE NEGATIVES** | The number of "true" classes incorrectly predicted to be false by the model. This is the measure of **Type II error.** `FN = Sum of observations predicted to be 0 that are actually 1`||**TOTAL POPULATION** | In the context of the confusion matrix, the sum of the cells.  `total population = tp + tn + fp + fn`||**SUPPORT** | The marginal sum of rows in the confusion matrix, or in other words the total number of observations belonging to a class regardless of prediction. ||**ACCURACY** | The number of correct predictions by the model out of the total number of observations.  `accuracy = (tp + tn) / total_population`||**PRECISION** | The ability of the classifier to avoid labeling a class as a member of another class.  `Precision = True Positives / (True Positives + False Positives)`_A precision score of 1 indicates that the classifier never mistakenly classified the current class as another class.  precision score of 0 would mean that the classifier misclassified every instance of the current class_ ||**RECALL/SENSITIVITY**    | The ability of the classifier to correctly identify the current class. `Recall = True Positives / (True Positives + False Negatives)`A recall of 1 indicates that the classifier correctly predicted all observations of the class.  0 means the classifier predicted all observations of the current class incorrectly.||**SPECIFICITY** | Percent of times the classifier predicted 0 out of all the times the class was 0. `specificity = tn / (tn + fp)`||**FALSE POSITIVE RATE** | Percent of times model predicts 1 when the class is 0. `fpr = fp / (tn + fp)`||**F1-SCORE** | The harmonic mean of the precision and recall. The harmonic mean is used here rather than the more conventional arithmetic mean because the harmonic mean is more appropriate for averaging rates. `F1-Score = 2 * (Precision * Recall) / (Precision + Recall)` _The f1-score's best value is 1 and worst value is 0, like the precision and recall scores. It is a useful metric for taking into account both measures at once._ |<jupyter_code>from sklearn.externals import joblib
pickle_file = "SVM_L1.pkl"
joblib.dump(grid, pickle_file)
def load_clf(path):
    return joblib.load(path)

def load_sub_clf(parent_list, file_list):
    sub_clf_dict = {}
    for parent, file in zip(parent_list, file_list):
        sub_clf_dict[parent] = load_clf(file)
    return sub_clf_dict;
def get_clf_prob(clf, x):
    return clf.predict_proba(x)

def get_sub_clf_prob(x_all,parent_prob, sub_clf_dict, precision_setting):
    result = []
    for prob, x in zip(parent_prob, x_all):
        prob_dict = {}
        for p in prob:
            index = np.where(prob == p)[0][0] + 2
            # start index from 2 as 1 is root
            sub_cat_list = dictC[index]
            if(len(sub_cat_list) == 1):
                prob_dict[sub_cat_list[0]] = round(1 * p, precision_setting)
            else:
                clf = sub_clf_dict[index]
                sub_prob = clf.predict_proba([x])
                sub_cat_arr = clf.classes_
                for sub_cat, sub_p in np.nditer([sub_cat_arr, sub_prob]):
                    prob_dict[int(sub_cat)] = round(sub_p * p, precision_setting)
        result.append(prob_dict)
    return result;
from sklearn.utils import class_weight
def get_norm_class_weight():
    class_weights = class_weight.compute_class_weight('balanced', np.unique(y_train), y_train)
    return class_weights
class_weights = get_norm_class_weight()
print(class_weights)
grid_2 = GridSearchCV(SVC(),param_grid,refit=True,verbose=4)
grid_3 = GridSearchCV(SVC(),param_grid,refit=True,verbose=4)
grid_4 = GridSearchCV(SVC(),param_grid,refit=True,verbose=4)

grid_2.fit(X_train_scaled,y_train)
grid_3.fit(X_train_scaled,y_train)
grid_4.fit(X_train_scaled,y_train)
grid_list = [grid_2, grid_3, grid_4]

prefix = "SVM_L"

for x in range(2,5):
    pickle_file = prefix + str(x) + ".pkl"
    joblib.dump(grid_list[x-2], pickle_file)
file_list = ["SVM_L1.pkl","SVM_L2.pkl","SVM_L3.pkl","SVM_L4.pkl"]
parent_list = [1,2,3,4]
sub_clf_dict = load_sub_clf(parent_list, file_list)
sub_clf_dict
p6 = np.array([0.6,0.6,0.6,0.6])
p7 = np.array([0.7,0.7,0.7,0.7])
p8 = np.array([0.8,0.8,0.8,0.8])
p9 = np.array([0.9,0.9,0.9,0.9])
parent_prob = np.array([p6,p7,p8,p9])
X_all = [X_test_scaled, X_test_scaled, X_test_scaled, X_test_scaled]
precision = 4
dictC = [["schema1"],["schema2"],["schema3"],["schema4"]]
h_result_final = get_sub_clf_prob(X_all,parent_prob, sub_clf_dict, precision)<jupyter_output><empty_output><jupyter_text>compare CNN with Heirarchical-SVM<jupyter_code>import tensorflow as tf

model = tf.keras.models.Sequential([
  tf.keras.layers.Embedding(X_train_scaled.shape, 16),
  tf.keras.layers.GlobalAveragePooling1D(),
  tf.keras.layers.Flatten(),
  tf.keras.layers.Dense(512),
  tf.keras.layers.Activation('relu'),
  tf.keras.layers.Dropout(0.5)])

model = keras.Sequential([
  keras.layers.Embedding(encoder.vocab_size, 16),
  keras.layers.GlobalAveragePooling1D(),
  keras.layers.Dense(1, activation='sigmoid')])

model.summary()
model.fit(X_train_scaled, y_train,
              batch_size=5,
              epochs=3,
              validation_data=(X_test_scaled, y_test),
              shuffle=True)
model.evaluate(X_test_scaled, y_test, verbose=1)<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.1264224983649444, 'fraction_numerical': 0.032701111837802485, 'mean_word_length': 4.078379275988044, 'pattern_counts': {'":': 0, '<': 56, '<?xml version=': 0, '>': 56, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 6, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '15', 'dir': 'jupyter-structured-clean-dedup', 'id': '275929', 'license': 'Apache-2.0', 'n_tokens_mistral': 5349, 'n_tokens_neox': 4967, 'n_words': 1509, 'path': 'heirarchical_svm.ipynb', 'repo_name': 'shan-96/heirarchical_SVM', 'size': '15290'}