<jupyter_start><jupyter_text>DenseNet Model Building Pipeline for 1D Signals with DEMODenseNet121, DenseNet161, DenseNet169, DenseNet201, DenseNet264  Test GPU (Optional)Before Starting, kindly check the available GPU from the Google Server, GPU model and other related information. It might help!<jupyter_code>import torch
print("Is CUDA enabled GPU Available?", torch.cuda.is_available())
print("GPU Number:", torch.cuda.device_count())
print("Current GPU Index:", torch.cuda.current_device())
print("GPU Type:", torch.cuda.get_device_name(device=None))
print("GPU Capability:", torch.cuda.get_device_capability(device=None))
print("Is GPU Initialized yet?", torch.cuda.is_initialized())<jupyter_output>Is CUDA enabled GPU Available? True
GPU Number: 1
Current GPU Index: 0
GPU Type: Tesla T4
GPU Capability: (7, 5)
Is GPU Initialized yet? True<jupyter_text>Connect to Google Drive (Optional for Google COLAB) Copy-Paste the Authorization Code and Mount Google Drive to COLAB<jupyter_code>from google.colab import drive
drive.mount('/content/GDrive')<jupyter_output>Mounted at /content/GDrive<jupyter_text>Move to the Target Directory<jupyter_code>%cd /content<jupyter_output>/content/GDrive/MyDrive/Colab_Notebooks/GitHub/GitHub Files<jupyter_text>List the Files and Folders Located in the Current Directory<jupyter_code>!ls<jupyter_output>DenseNet_1DCNN_Implementation_DEMO.ipynb   MobileNet_1DCNN.py
DenseNet_1DCNN.py			   ResNet_1DCNN.py
Drug_Persistency.gsheet			   ResNeXt_1DCNN.py
EfficientNet_1DCNN.py			   trained_models
Inception_1DCNN_Implementation_DEMO.ipynb  VGG_1DCNN.py
Inception_1DCNN.py			   WeatherHistory.csv
Inception_ResNet_1DCNN.py<jupyter_text>Upload Files from Local Directory (if required)<jupyter_code>from google.colab import files
uploaded = files.upload()<jupyter_output><empty_output><jupyter_text>Import Necessary Libraries<jupyter_code>import numpy as np
import pandas as pd
import seaborn as sns
import tensorflow as tf
import matplotlib.pyplot as plt
from scipy import interp
from itertools import cycle
from sklearn.model_selection import train_test_split
from sklearn.preprocessing import LabelEncoder, OneHotEncoder
from sklearn.metrics import mean_absolute_error, accuracy_score, precision_score, recall_score, f1_score, roc_curve, plot_roc_curve
from sklearn.metrics import confusion_matrix, classification_report, auc, precision_recall_curve, average_precision_score
sns.set_theme(style="whitegrid")
# Import DenseNet Module
from DenseNet_1DCNN import DenseNet<jupyter_output><empty_output><jupyter_text>DEMO: Regression and Classification  Regression  Import and Prepare Dataset Import Dataset from a CSV file using Pandas. This DEMO uses the Weather History dataset from Kaggle.  Source: https://www.kaggle.com/budincsevity/szeged-weather<jupyter_code>dataset = pd.read_csv('WeatherHistory.csv')
print(dataset.shape)
dataset.head(5)<jupyter_output>(96453, 11)<jupyter_text>Have a look at the dataset! The CSV file includes a hourly/daily summary for Szeged, Hungary area, between 2006 and 2016.Data available in the hourly response:1. Time  2. Summary  3. PrecipType  4. Temperature  5. ApparentTemperature  6. Humidity  7. WindSpeed  8. WindBearing  9. Visibility  10. LoudCover  11. Pressure   Check for any existing 'NaN' value in the dataset which might create issues afterwards. We, here, check both overall and column by column.<jupyter_code>print(f'Total NULL values in the Original DataFrame = {dataset.isnull().sum().sum()}')
print(dataset.isnull().sum())<jupyter_output>Total NULL values in the Original DataFrame = 517
Formatted Date                0
Summary                       0
Precip Type                 517
Humidity                      0
Wind Speed (km/h)             0
Wind Bearing (degrees)        0
Visibility (km)               0
Pressure (millibars)          0
Daily Summary                 0
Temperature (C)               0
Apparent Temperature (C)      0
dtype: int64<jupyter_text>There are total 517 NULL or NaN values and all of them are in the 'Precip_Type' Column. Let's print out the unique values in that column.<jupyter_code>dataset['Precip Type'].unique()<jupyter_output><empty_output><jupyter_text>So, it can be seen that there are mainly 2 classes in this column other than the NaN. So, we can replace the NaN cell with 'Unknown' instead of '0' (which is done more commonly).<jupyter_code>dataset['Precip Type'] = dataset['Precip Type'].fillna('Unknown')<jupyter_output><empty_output><jupyter_text>One important realization is that we could easily remove those rows in this case instead of filling up the NULLs, but that would remove some datapoints. For very large datasets, it might not matter. On the contrary, sometimes it might remove important observations. If the dataset is too sparse, then it is better to remove the rows instead of filling. But since in this case, value in only 1 column was missing, it was filled up with a suitable replacement.<jupyter_code>dataset['Precip Type'].unique()<jupyter_output><empty_output><jupyter_text>Now, at first slice the DataTime Frame into month and hour since they can be strong indicators of weather (fairly long term and short term). Years can also show gradual change in weather pattern (especially change in the climate) but that has been ignored in this study. Modify it according to your wish to improve performance.<jupyter_code>dataset['Month'] = (dataset['Formatted Date']).str.slice(start=5, stop=7)
dataset['Hour'] = (dataset['Formatted Date']).str.slice(start=11, stop=13)
dataset.head(5)<jupyter_output><empty_output><jupyter_text>The first DateTime Frame column and the column containing the real temperature for that hour have been dropped. The aim here is to predict "Apparent Temperature" from other weather indicators using Deep Learning based Regression.<jupyter_code>dataset.drop(['Formatted Date', 'Temperature (C)'], axis=1, inplace=True)
dataset.head(5)<jupyter_output><empty_output><jupyter_text>Rearrange the Columns, place the target column at the end for next step<jupyter_code>dataset = dataset[['Month', 'Hour', 'Summary', 'Precip Type', 'Humidity', 
                   'Wind Speed (km/h)', 'Wind Bearing (degrees)', 
                   'Visibility (km)', 'Pressure (millibars)', 'Daily Summary', 
                   'Apparent Temperature (C)']]
dataset.head(5)<jupyter_output><empty_output><jupyter_text>At this stage, check the datatypes of the columns. You can see that 'Month' and 'Hour' columns are object types, convert them to numeric 'int64' (otherwise while creating dummy variables, each month or hour will get a separate column, which is unncessary).<jupyter_code>dataset.info()
dataset['Month'] = dataset['Month'].astype('int64')
dataset['Hour'] = dataset['Hour'].astype('int64')
dataset.info()<jupyter_output><class 'pandas.core.frame.DataFrame'>
RangeIndex: 96453 entries, 0 to 96452
Data columns (total 11 columns):
 #   Column                    Non-Null Count  Dtype  
---  ------                    --------------  -----  
 0   Month                     96453 non-null  int64  
 1   Hour                      96453 non-null  int64  
 2   Summary                   96453 non-null  object 
 3   Precip Type               96453 non-null  object 
 4   Humidity                  96453 non-null  float64
 5   Wind Speed (km/h)         96453 non-null  float64
 6   Wind Bearing (degrees)    96453 non-null  int64  
 7   Visibility (km)           96453 non-null  float64
 8   Pressure (millibars)      96453 non-null  float64
 9   Daily Summary             96453 non-null  object 
 10  Apparent Temperature (C)  96453 non-null  float64
dtypes: float64(5), int64(3), object(3)
memory usage: 8.1+ MB<jupyter_text>Convert Text Data into Dummy Variables for Machine Learning.  It is important since Machine Learning models will not accept non-numerical data directly during training or testing.<jupyter_code>dummy_dataset = pd.DataFrame()
for i in range(0,len(dataset.columns)):
  X = dataset[dataset.columns[i]]
  if type(X[0]) == str:
    Y = pd.get_dummies(X)
    dummy_dataset = pd.concat([dummy_dataset, Y], axis=1)
  else:
    dummy_dataset = pd.concat([dummy_dataset, X], axis=1)
#
dummy_dataset.shape
dummy_dataset.head(5)<jupyter_output><empty_output><jupyter_text>At this stage, you can plot the dataset prior to check any pattern in the dataset or just for illustrative purposes. Convert Pandas DataFrame into NumPy Arrays<jupyter_code>X_Data = dummy_dataset.iloc[:,0:-1].values # All columns except the last are the predicting variables
Y_Data = dummy_dataset.iloc[:,-1].values # Last column (Weight Column in this case) is the label
print(X_Data.shape)
print(Y_Data.shape)<jupyter_output>(96453, 251)
(96453,)<jupyter_text>Train-Test Split<jupyter_code>X_Train, X_Test, Y_Train, Y_Test = train_test_split(X_Data, Y_Data, test_size=0.20, random_state=42)<jupyter_output><empty_output><jupyter_text>The third axis is to show the number of channels, which is 1 in this case. If same labels were appropriate for more than 1 dataset, there would be more than 1 channel.<jupyter_code>X_Train = np.expand_dims(X_Train, axis=2)
X_Test = np.expand_dims(X_Test, axis=2)
print(X_Train.shape, X_Test.shape)
print(Y_Train.shape, Y_Test.shape)<jupyter_output>(77162, 251, 1) (19291, 251, 1)
(77162,) (19291,)<jupyter_text>Build and Train Imported Data using the Regression Model Configurations - Select the Configurations for the Regression Model. Vary them (while following the guidelines) to improve performance.<jupyter_code>"Configurations for DenseNet in Regression Mode"
length = X_Train.shape[1]   # Number of Features (or length of the signal)
model_width = 32            # Number of Filter or Kernel in the Input Layer (Power of 2 to avoid error)
num_channel = 1             # Number of Input Channels
problem_type = 'Regression' # Regression or Classification
output_number = 1           # Number of Outputs in the Regression Mode - 1 input is mapped to a single output<jupyter_output><empty_output><jupyter_text>Build Model<jupyter_code>Regression_Model = DenseNet(length, num_channel, model_width, problem_type=problem_type, output_nums=output_number).DenseNet169() # Build Model
Regression_Model.compile(loss='mae', optimizer='adam', metrics= ['mse']) # Compile Model<jupyter_output><empty_output><jupyter_text>Model_Summary<jupyter_code>Regression_Model.summary() # Summary of the Model<jupyter_output>Model: "model_1"
__________________________________________________________________________________________________
Layer (type)                    Output Shape         Param #     Connected to                     
==================================================================================================
input_2 (InputLayer)            [(None, 251, 1)]     0                                            
__________________________________________________________________________________________________
conv1d_168 (Conv1D)             (None, 126, 32)      256         input_2[0][0]                    
__________________________________________________________________________________________________
batch_normalization_168 (BatchN (None, 126, 32)      128         conv1d_168[0][0]                 
__________________________________________________________________________________________________
activation_168 (Activation)     (None, 126, 32)      0           batch_normalization_168[0][[...]<jupyter_text>Upload Past Weights if available<jupyter_code>Regression_Model.load_weights('Saved_Model.h5') # Load Previously Trained Weights for Transfer Learning<jupyter_output><empty_output><jupyter_text>Train Model for 'n' number of Epochs with Batch size of 'm'<jupyter_code># Early Stopping and Model_Checkpoints are optional parameters
# Early Stopping is to stop the training based on certain condition set by the user
# Model Checkpoint is to save a model in a directory based on certain conditions so that it can be used later for Transfer Learning or avoiding retraining
callbacks = [tf.keras.callbacks.EarlyStopping(monitor='val_loss', patience=30, mode='min'), 
             tf.keras.callbacks.ModelCheckpoint('trained_models/DenseNet169_'+str(model_width)+'_'+str(num_channel)+'_'+str(output_number)+'_'+str(problem_type)+'.h5', 
                                                verbose=1, monitor='val_loss', save_best_only=True, mode='min')]
history = Regression_Model.fit(X_Train, Y_Train, epochs=500, batch_size=128, verbose=1, validation_split=0.2, shuffle=True, callbacks=callbacks)
# Save 'History' of the model for model performance analysis performed later<jupyter_output><empty_output><jupyter_text>Test and Predict<jupyter_code># Preictions from the Test Set from the Trained Model
Predictions = Regression_Model.predict(X_Test, verbose=1)
print(Predictions.shape)<jupyter_output><empty_output><jupyter_text>Error Performance (Mean Sqaured Error or MAE)<jupyter_code># Error of the prediction, one of many evaluation metrics
# Using Mean Absolute Error (MAE) in this case as a sample
Error = mean_absolute_error(Y_Test, Predictions)
print(f"MAE: {Error}")<jupyter_output><empty_output><jupyter_text>Plot Train and Validation Error and Loss<jupyter_code>def history_plot(history):
  # list all dictionaries in history
  print(history.history.keys())
  # summarize history for error
  plt.figure(figsize=(12,10))
  plt.subplot(2,1,1)
  plt.plot(history.history['mse'])
  plt.plot(history.history['val_mse'])
  plt.title('Model Error Performance')
  plt.ylabel('Error')
  plt.xlabel('Epoch')
  plt.legend(['Train', 'Val'], loc='upper right')
  plt.show()
  # summarize history for loss
  plt.figure(figsize=(12,10))
  plt.subplot(2,1,2)
  plt.plot(history.history['loss'])
  plt.plot(history.history['val_loss'])
  plt.title('Model Loss')
  plt.ylabel('Loss')
  plt.xlabel('Epoch')
  plt.legend(['Train', 'Val'], loc='upper right')
  plt.show()
#
history_plot(history)<jupyter_output><empty_output><jupyter_text>Plot Prediction Performance<jupyter_code>def plot_prediction(test_labels, test_predictions):
  plt.figure(figsize=(15,10))
  plt.scatter(test_labels, test_predictions)
  '''Add Trendline'''
  z = np.polyfit(test_labels.ravel(), test_predictions.ravel(), 1)
  p = np.poly1d(z)
  plt.plot(test_labels, p(test_labels))
  plt.text(np.max(test_labels)/3,np.max(test_predictions),f'y = {p[1]:.2f}x+{p[0]:.2f}', fontsize=15)
  plt.title('Ground Truth vs. Prediction Scatter Plot', fontsize=20)
  plt.xlabel('Ground Truth', fontsize=15)
  plt.ylabel('Predictions', fontsize=15)
#
plot_prediction(Y_Test, Predictions)<jupyter_output><empty_output><jupyter_text>Kernel Density Plot<jupyter_code>df = pd.DataFrame(data = {'Ground Truth': Y_Test.ravel(), 'Predictions': Predictions.ravel()})
plt.figure(figsize=(15,10))
sns.set_style('whitegrid')
sns.kdeplot(data=df)
plt.title('Kernel Density Estimation (KDE) Plot for Ground Truth and Predictions', fontsize=20)
plt.xlabel('Magnitude', fontsize=15)
plt.ylabel('Density', fontsize=15)
plt.figure(figsize = (15,10))
df = pd.DataFrame(data = {'Ground Truth': Y_Test.ravel(), 'Predictions': Predictions.ravel()})
ax = sns.violinplot(data=df)
plt.title('Violin Plot for Ground Truth and Predictions', fontsize=20)
plt.ylabel('Magnitude', fontsize=15)<jupyter_output><empty_output><jupyter_text>Both from the Kernel Density Plot or the Violin Plot, it can be seen that not only the MAE is good, but also the predicted values follow the same pattern as the ground truth, which proves the robustness of the model.  Classification  Data Preparation Import the dataset from a CSV file and save it into a Pandas DataFrame<jupyter_code>dataset = pd.read_excel('Drug_Persistency.xlsx', sheet_name = 'Dataset')
dataset.drop(columns=['Ptid'], inplace=True) # Drop Patient ID Column
print(dataset.shape)
dataset.head(10)<jupyter_output><empty_output><jupyter_text>Create a Blank DataFrame<jupyter_code>dummy_dataset = pd.DataFrame()
for i in range(0,len(dataset.columns)):
  X = dataset[dataset.columns[i]]
  if type(X[0]) == str:
    Y = pd.get_dummies(X)
    dummy_dataset = pd.concat([dummy_dataset, Y], axis=1)
  else:
    dummy_dataset = pd.concat([dummy_dataset, X], axis=1)
#
dummy_dataset.head(10)<jupyter_output><empty_output><jupyter_text>Convert the DataFrame to a Numpy Array<jupyter_code>X_Data = dummy_dataset.iloc[:,0:-1].values
Y_Data = dummy_dataset.iloc[:,-1].values
print(X_Data.shape)
print(Y_Data.shape)<jupyter_output>(96453, 251)
(96453,)<jupyter_text>Get Unique Labels<jupyter_code>'''Print out the Classes to track the sequence, will be important in the evaluation section (e.g., Labelling Conf Mat)'''
L_E = LabelEncoder()
L_E.fit_transform(Y_Data)
labels = L_E.classes_
print(f'Classes: {labels}')<jupyter_output>Classes: [-27.71666667 -26.22777778 -25.92222222 ...  38.41666667  38.66111111
  39.34444444]<jupyter_text>Train_Test Split<jupyter_code>X_Train, X_Test, Y_Train_, Y_Test_ = train_test_split(X_Data, Y_Data, test_size=0.20, random_state=42)<jupyter_output><empty_output><jupyter_text>One-Hot-Encoding for the Classification Labels<jupyter_code>def one_hot_encoding(data):
  L_E = LabelEncoder()
  integer_encoded = L_E.fit_transform(data)  
  onehot_encoder = OneHotEncoder(sparse=False)
  integer_encoded = integer_encoded.reshape(len(integer_encoded), 1)
  one_hot_encoded_data = onehot_encoder.fit_transform(integer_encoded)
  return one_hot_encoded_data
Y_Train = one_hot_encoding(Y_Train_.ravel())
Y_Test = one_hot_encoding(Y_Test_.ravel())<jupyter_output><empty_output><jupyter_text>Train and Test Data Shapes<jupyter_code>print(X_Train.shape, X_Test.shape)
print(Y_Train.shape, Y_Test.shape)<jupyter_output>(77162, 251) (19291, 251)
(77162, 8721) (19291, 6405)<jupyter_text>Build and Train Imported Data using the Classification Model Configurations<jupyter_code>"Configurations for the 1D Network in Classification Mode"
length = X_Train.shape[1]       # Number of Features (or length of the signal)
model_width = 32                # Number of Filter or Kernels in the Input Layer (Power of 2 to avoid error)
num_channel = 1                 # Number of Input Channels
problem_type = 'Classification' # Regression or Classification
class_number = Y_Train.shape[1] # Number of Output Class in Classification Mode (>=2)<jupyter_output><empty_output><jupyter_text>Build Model<jupyter_code>Classification_Model = DenseNet(length, num_channel, model_width, problem_type=problem_type, output_nums=class_number).DenseNet169()
if class_number == 2:
  Classification_Model.compile(loss='binary_crossentropy', optimizer='adam', metrics=['mse','accuracy'])
elif class_number > 2:
  Classification_Model.compile(loss='categorical_crossentropy', optimizer='adam', metrics=['mse','accuracy'])<jupyter_output><empty_output><jupyter_text>Model Summary<jupyter_code>Classification_Model.summary()<jupyter_output>Model: "model"
__________________________________________________________________________________________________
Layer (type)                    Output Shape         Param #     Connected to                     
==================================================================================================
input_1 (InputLayer)            [(None, 251, 1)]     0                                            
__________________________________________________________________________________________________
conv1d (Conv1D)                 (None, 126, 32)      256         input_1[0][0]                    
__________________________________________________________________________________________________
batch_normalization (BatchNorma (None, 126, 32)      128         conv1d[0][0]                     
__________________________________________________________________________________________________
activation (Activation)         (None, 126, 32)      0           batch_normalization[0][0]    [...]<jupyter_text>Upload Past Weights (Transfer Learning)<jupyter_code>Classification_Model.load_weights('Saved_Model.h5') # Load Previously Trained Weights for Transfer Learning<jupyter_output><empty_output><jupyter_text>Train Model<jupyter_code># Early Stopping and Model_Checkpoints are optional parameters
# Early Stopping is to stop the training based on certain condition set by the user
# Model Checkpoint is to save a model in a directory based on certain conditions so that it can be used later for Transfer Learning or avoiding retraining
callbacks = [tf.keras.callbacks.EarlyStopping(monitor='val_loss', patience=50, mode='min'), 
             tf.keras.callbacks.ModelCheckpoint('Saved_Model.h5', verbose=1, monitor='val_loss', save_best_only=True, mode='min')]
history = Classification_Model.fit(X_Train, Y_Train, epochs=300, batch_size=128, verbose=1, validation_split=0.2, shuffle=True, callbacks=callbacks)<jupyter_output><empty_output><jupyter_text>Test<jupyter_code># Predictions from the Test Set from the Trained Model
Predictions = Classification_Model.predict(X_Test, verbose=1)
print(Predictions.shape)<jupyter_output><empty_output><jupyter_text>Error Performance<jupyter_code># Error of the prediction, one of many evaluation metrics
# Using Mean Absolute Error (MAE) in this case as a sample
Error = mean_absolute_error(Y_Test, Predictions)
print(f"MAE: {Error}")<jupyter_output><empty_output><jupyter_text>Plot Training History [Metrics] - Requires to Train the Model<jupyter_code>def history_plot(history):
  # list all dictionaries in history
  print(history.history.keys())
  # summarize history for error
  plt.figure(figsize=(12,10))
  plt.subplot(2,1,1)
  plt.plot(history.history['mse'])
  plt.plot(history.history['val_mse'])
  plt.title('Model Error Performance')
  plt.ylabel('Error')
  plt.xlabel('Epoch')
  plt.legend(['Train', 'Val'], loc='upper right')
  plt.show()
  # summarize history for loss
  plt.figure(figsize=(12,10))
  plt.subplot(2,1,2)
  plt.plot(history.history['loss'])
  plt.plot(history.history['val_loss'])
  plt.title('Model Loss')
  plt.ylabel('Loss')
  plt.xlabel('Epoch')
  plt.legend(['Train', 'Val'], loc='upper right')
  plt.show()
#
history_plot(history)<jupyter_output><empty_output><jupyter_text>Convert raw predictions into single-column, integer-based predictions<jupyter_code>prediction_shape = Predictions.shape
prediction_length = prediction_shape[0]
Y_Preds = np.zeros((prediction_length, 1), dtype=int)
#
for i in range(0, prediction_length):
  prediction = Predictions[i]
  x = np.where(prediction == np.max(prediction))
  x = int(x[0])
  Y_Preds[i] = x

Y_Preds.shape<jupyter_output><empty_output><jupyter_text>Print Confusion Matrix<jupyter_code>print('-------------------------------------------')
print('Raw Confusion Matrix')
print(confusion_matrix(Y_Test_, Y_Preds, normalize=None))
print('-------------------------------------------')
print('Normalized Confusion Matrix')
print(confusion_matrix(Y_Test_, Y_Preds, normalize='true'))
print('-------------------------------------------')<jupyter_output><empty_output><jupyter_text>Plot Confusion Matrix<jupyter_code># Use the numeric labels or replace with real-class names. But be careful about their respective sequence
labels = ['Class_0', 'Class_1', 'Class_2', 'Class_3', 'Class_4', 'Class_5', 'Class_6', 'Class_7']
def plot_conf_mat(Ground_Truth_Labels, Predictions):
  confusion_matrix_raw = confusion_matrix(Ground_Truth_Labels, Predictions, normalize=None)
  confusion_matrix_norm = confusion_matrix(Ground_Truth_Labels, Predictions, normalize='true')
  shape = confusion_matrix_raw.shape
  data = np.asarray(confusion_matrix_raw, dtype=int)
  text = np.asarray(confusion_matrix_norm, dtype=float)
  annots = (np.asarray(["{0:.2f} ({1:.0f})".format(text, data) for text, data in zip(text.flatten(), data.flatten())])).reshape(shape[0],shape[1])
  fig = plt.figure(figsize=(20, 10))
  sns.heatmap(confusion_matrix_norm, cmap='YlGnBu', annot=annots, fmt='', xticklabels=labels, yticklabels=labels)
  plt.title('Confusion Matrix', fontsize=25)
  plt.xlabel("Predicted", fontsize=15)
  plt.ylabel("Actual", fontsize=15)
#
plot_conf_mat(Y_Test_, Y_Preds)<jupyter_output><empty_output><jupyter_text>Here it can be seen that risk level 6 and 7 could not be predicted well due to less number of instances.<jupyter_code># Evaluating Overall Metrics - Accuracy, Precision, Recall, f1-Score
Accuracy = accuracy_score(Y_Test_, Y_Preds)
Precision = precision_score(Y_Test_, Y_Preds, average= 'weighted')
Recall = recall_score(Y_Test_, Y_Preds, average= 'weighted')
f1_Score = f1_score(Y_Test_, Y_Preds, average= 'weighted')
print(f'Accuracy = {Accuracy:.3f}')
print(f'Precision = {Precision:.3f}')
print(f'Recall = {Recall:.3f}')
print(f'f1-Score = {f1_Score:.3f}')
print(classification_report(Y_Test_, Y_Preds, target_names=labels, zero_division=0))<jupyter_output><empty_output><jupyter_text>Multiclass Receiver Operating Characteristic(ROC) Curves Plot<jupyter_code>def plot_multiclass_roc(Y_Test, Predictions):
  # Compute ROC curve and Area Under Curve (AUC) for each class
  fpr = dict()
  tpr = dict()
  roc_auc = dict()
  for i in range(class_number):
      fpr[i], tpr[i], _ = roc_curve(Y_Test[:, i], Predictions[:, i])
      roc_auc[i] = auc(fpr[i], tpr[i])

  # Compute micro-average ROC curve and ROC area
  fpr["micro"], tpr["micro"], _ = roc_curve(Y_Test.ravel(), Predictions.ravel())
  roc_auc["micro"] = auc(fpr["micro"], tpr["micro"])

  # First aggregate all false positive rates
  all_fpr = np.unique(np.concatenate([fpr[i] for i in range(class_number)]))
  # Then interpolate all ROC curves at this points
  mean_tpr = np.zeros_like(all_fpr)
  for i in range(class_number):
      mean_tpr += interp(all_fpr, fpr[i], tpr[i])

  # Finally average it and compute AUC
  mean_tpr /= class_number

  fpr["macro"] = all_fpr
  tpr["macro"] = mean_tpr
  roc_auc["macro"] = auc(fpr["macro"], tpr["macro"])

  # Plot all ROC curves
  plt.figure(figsize=(20, 10))
  plt.plot(fpr["micro"], tpr["micro"],
          label='micro-average ROC curve (area = {0:0.2f})'
                ''.format(roc_auc["micro"]),
          color='deeppink', linestyle=':', linewidth=4)

  plt.plot(fpr["macro"], tpr["macro"],
          label='macro-average ROC curve (area = {0:0.2f})'''.format(roc_auc["macro"]),
          color='navy', linestyle=':', linewidth=4)

  for i in range(class_number):
      plt.plot(fpr[i], tpr[i], lw=2,
              label='ROC curve of class {0} (area = {1:0.2f})'''.format(i, roc_auc[i]))

  plt.plot([0, 1], [0, 1], 'k--', lw=2)
  plt.xlim([0.0, 1.0])
  plt.ylim([0.0, 1.05])
  plt.xlabel('False Positive Rate', fontsize=15)
  plt.ylabel('True Positive Rate', fontsize=15)
  plt.title('MultiClass ROC Plot with Respective AUC', fontsize=25)
  plt.legend(loc="lower right")
  plt.show()
#
plot_multiclass_roc(Y_Test, Predictions)
def plot_multiclass_precision_recall_curves(Y_Test, Predictions):
  # For each class
  precision = dict()
  recall = dict()
  average_precision = dict()
  for i in range(class_number):
      precision[i], recall[i], _ = precision_recall_curve(Y_Test[:, i], Predictions[:, i])
      average_precision[i] = average_precision_score(Y_Test[:, i], Predictions[:, i])

  # A "micro-average": quantifying score on all classes jointly
  precision["micro"], recall["micro"], _ = precision_recall_curve(Y_Test.ravel(), Predictions.ravel())
  average_precision["micro"] = average_precision_score(Y_Test, Predictions, average="micro")
  print('Average precision score, micro-averaged over all classes: {0:0.2f}'.format(average_precision["micro"]))

  from itertools import cycle
  # setup plot details
  colors = cycle(['navy', 'turquoise', 'darkorange', 'cornflowerblue', 'teal'])

  plt.figure(figsize=(20, 10))
  f_scores = np.linspace(0.2, 0.8, num=4)
  lines = []
  labels = []
  for f_score in f_scores:
      x = np.linspace(0.01, 1)
      y = f_score * x / (2 * x - f_score)
      l, = plt.plot(x[y >= 0], y[y >= 0], color='gray', alpha=0.2)
      plt.annotate('f1={0:0.1f}'.format(f_score), xy=(0.9, y[45] + 0.02))

  lines.append(l)
  labels.append('iso-f1 curves')
  l, = plt.plot(recall["micro"], precision["micro"], color='gold', lw=2)
  lines.append(l)
  labels.append('micro-average Precision-recall (area = {0:0.2f})'''.format(average_precision["micro"]))

  for i, color in zip(range(class_number), colors):
      l, = plt.plot(recall[i], precision[i], color=color, lw=2)
      lines.append(l)
      labels.append('Precision-recall for class {0} (area = {1:0.2f})'''.format(i, average_precision[i]))

  fig = plt.gcf()
  fig.subplots_adjust(bottom=0.25)
  plt.xlim([0.0, 1.0])
  plt.ylim([0.0, 1.05])
  plt.xlabel('Recall', fontsize=15)
  plt.ylabel('Precision', fontsize=15)
  plt.title('MultiClass Precision-Recall Curves', fontsize=25)
  plt.legend(lines, labels, loc=(0, -.3), prop=dict(size=14))
  plt.show()
#
plot_multiclass_precision_recall_curves(Y_Test, Predictions)<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.12404852108025442, 'fraction_numerical': 0.0258246150637795, 'mean_word_length': 5.142613151152861, 'pattern_counts': {'":': 1, '<': 184, '<?xml version=': 0, '>': 188, 'https://': 1, 'lorem ipsum': 0, 'www.': 1, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 2, 'word_list_counts': {'cursed_substrings.json': 6, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '49', 'dir': 'jupyter-structured-clean-dedup', 'id': '47763', 'license': 'MIT', 'n_tokens_mistral': 9807, 'n_tokens_neox': 9084, 'n_words': 2312, 'path': 'Codes/DenseNet_1DCNN_Implementation_DEMO.ipynb', 'repo_name': 'Sakib1263/DenseNet-Model-Builder-Tensorflow-Keras', 'size': '28771'}