<jupyter_start><jupyter_text>Shuffle the dataset.<jupyter_code>from sklearn.utils import shuffle

processed_data = shuffle(processed_data)
X = processed_data[:,:-1].astype(float)
y = processed_data[:,-1].astype(int)<jupyter_output><empty_output><jupyter_text>Split the data into train and test set. 18% of it preserved for test set, and remaining 72% for train set.<jupyter_code>X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.1)
X_train[0].shape[0]
print(X_train.dtype)
print(y_train)<jupyter_output>[0 0 0 ... 1 1 0]<jupyter_text>Fully connected neural network model with 3 hidden layers:<jupyter_code>import tensorflow as tf 

model = tf.keras.models.Sequential([
  tf.keras.Input(shape=(X_train[0].shape[0],)), #Xtrain[0].shape[0] = 104
  tf.keras.layers.Dense(15, activation='relu'), #represents 1st hidden layer 
  tf.keras.layers.Dropout(0.2), # dropout regularization with dropout probability 20% percent
  tf.keras.layers.Dense(7, activation='relu'), #represents the 2nd hidden layer 
  tf.keras.layers.Dropout(0.2), # dropout regularization with dropout probability 20% percent
  tf.keras.layers.Dense(4, activation='relu'), #represents the 3rd hidden layer 
  tf.keras.layers.Dropout(0.2), # dropout regularization with dropout probability 20% percent
  tf.keras.layers.Dense(1, activation='sigmoid') # sigmoid activation at output since it is binary classification
])
model.compile(optimizer='adam', 
              loss='binary_crossentropy',
              metrics=['accuracy'])<jupyter_output><empty_output><jupyter_text>Train the model with 10 epochs:<jupyter_code>r2 = model.fit(X_train, y_train, epochs=150)<jupyter_output>Epoch 1/150
101/101 [==============================] - 1s 890us/step - loss: 19911.3032 - accuracy: 0.1746
Epoch 2/150
101/101 [==============================] - 0s 910us/step - loss: 2116.1180 - accuracy: 0.5821
Epoch 3/150
101/101 [==============================] - 0s 1ms/step - loss: 746.0180 - accuracy: 0.7824
Epoch 4/150
101/101 [==============================] - 0s 840us/step - loss: 475.4041 - accuracy: 0.8253
Epoch 5/150
101/101 [==============================] - 0s 820us/step - loss: 300.5241 - accuracy: 0.8459
Epoch 6/150
101/101 [==============================] - 0s 980us/step - loss: 245.9800 - accuracy: 0.8524
Epoch 7/150
101/101 [==============================] - 0s 930us/step - loss: 189.8506 - accuracy: 0.8586
Epoch 8/150
101/101 [==============================] - 0s 1ms/step - loss: 187.2163 - accuracy: 0.8644
Epoch 9/150
101/101 [==============================] - 0s 1ms/step - loss: 165.6396 - accuracy: 0.8763
Epoch 10/150
101/101 [==============================] - 0s[...]<jupyter_text>Plot the losses:<jupyter_code>plt.plot(r2.history['loss'], label='loss')
#plt.plot(r2.history['val_loss'], label='val_loss')
plt.legend()<jupyter_output><empty_output><jupyter_text>Plot the accuracies:<jupyter_code>plt.plot(r2.history['accuracy'], label='acc')
#plt.plot(r2.history['val_accuracy'], label='val_acc')
plt.legend()
print(model.evaluate(X_test, y_test))
model.save_weights('./weights/model_full/')<jupyter_output><empty_output><jupyter_text>**Applying this dataset on different models to create benchmarks to compare with Fully Connected model:** *Logistic Regression* **(linear classifier):**<jupyter_code>from sklearn.linear_model import LogisticRegression
model =LogisticRegression()
model.fit(X_train, y_train)
print(model.score(X_train, y_train))
print(model.score(X_test,y_test))


for i in range(X_train[0].shape[0]):
 print("weight for the feature",i,":",model.coef_[0][i])

model.coef_[0].sort()

print("sorted list, positive weights contribute MORE TO DETERMINE potential hypertension case and negative features conribute more to make prediction as NOT hypertension. So we can say 0 valued weights do not contribute to anything:",model.coef_[0])<jupyter_output>weight for the feature 0 : -0.001269072069105969
weight for the feature 1 : -0.001912998093316698
weight for the feature 2 : 0.0021300703994232392
weight for the feature 3 : 0.008678357639568106
weight for the feature 4 : 0.004297028507627697
weight for the feature 5 : -0.013368846962379723
weight for the feature 6 : -0.003938173393702886
weight for the feature 7 : 0.008190261014102238
weight for the feature 8 : -0.006175514618128337
weight for the feature 9 : -0.002315577506692285
weight for the feature 10 : -0.0006153388168114538
weight for the feature 11 : -0.013036858065776865
weight for the feature 12 : -0.005181563397706072
weight for the feature 13 : -4.3683729383293066e-07
weight for the feature 14 : -0.008780364879894267
weight for the feature 15 : -0.01166293116274796
weight for the feature 16 : -0.002427192807723251
weight for the feature 17 : -0.0010810258876364372
weight for the feature 18 : 0.0008874175115078015
weight for the feature 19 : -0.0019423972322216313
weight fo[...]<jupyter_text>*Decision Tree*<jupyter_code>from sklearn.tree import DecisionTreeClassifier
model =DecisionTreeClassifier()
model.fit(X_train, y_train)
print(model.score(X_train, y_train))
print(model.score(X_test,y_test))<jupyter_output>1.0
1.0<jupyter_text>*AdaBoost*<jupyter_code>from sklearn.ensemble import AdaBoostClassifier
model =AdaBoostClassifier()
model.fit(X_train, y_train)
print(model.score(X_train, y_train))
print(model.score(X_test,y_test))
# Visualize the data (tsne is great but slow.)

from sklearn.manifold import TSNE
tsne = TSNE(n_components=2)
transformed = tsne.fit_transform(X_train)

#visualize in the 2d
plt.scatter(transformed[:,0], transformed[:,1], c=y_train) #take first two columns in order to get 2d plot.
plt.show()
transformed = tsne.fit_transform(X_test)

# visualize the clouds in 2-D
plt.scatter(transformed[:,0], transformed[:,1], c=y_test) #take first two columns in order to get 2d plot.
plt.show()<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.1668379582048647, 'fraction_numerical': 0.12504282288454951, 'mean_word_length': 5.934679334916865, 'pattern_counts': {'":': 1, '<': 33, '<?xml version=': 0, '>': 33, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '9', 'dir': 'jupyter-structured-clean-dedup', 'id': '63335', 'license': 'Apache-2.0', 'n_tokens_mistral': 2528, 'n_tokens_neox': 2005, 'n_words': 496, 'path': 'utils/notebooks/model_systolic_diastolic_excluded.ipynb', 'repo_name': 'pazara18/swe-project-group30', 'size': '5838'}