<jupyter_start><jupyter_text>so it's a binary classification task with an even mix of positive and negative examples.<jupyter_code># df.describe()
id = df.pop('id')
# pca = PCA()
# df = pd.DataFrame(pca.fit_transform(df))
# df.describe()
# pca.explained_variance_ratio_
ncols = df.shape[1]<jupyter_output><empty_output><jupyter_text>dense = layers.Dense(64, activation="relu")x = dense(inputs)> The "layer call" action is like drawing an arrow from "inputs" to this layer you created. You're "passing" the inputs to the dense layer, and you get x as the output.[tensorflow docs](https://www.tensorflow.org/guide/keras/functional)<jupyter_code>inputs = keras.Input(shape=(ncols,))
norm = layers.BatchNormalization()
x = norm(inputs)<jupyter_output>2021-11-17 19:10:54.020372: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2021-11-17 19:10:54.125555: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2021-11-17 19:10:54.126236: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2021-11-17 19:10:54.127424: I tensorflow/core/platform/cpu_feature_guard.cc:142] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations:  AVX2 AVX512F FMA
To enable them in other operations, rebuild TensorFlow with the appropriate compil[...]<jupyter_text>skipping assignment of dense layer and going straight to "layer call<jupyter_code>x = layers.Dense(64, activation="relu")(x)
x = layers.Dropout(rate=0.25)(x)
x = layers.Dense(32, activation="relu")(x)
x = layers.BatchNormalization()(x)
x = layers.Dropout(rate=0.25)(x)
x = layers.Dense(16, activation="relu")(x)
outputs = layers.Dense(1)(x)
model = keras.Model(inputs, outputs, name = "cone_NN")
model.summary()
keras.utils.plot_model(model, "cone_NN.png", show_shapes=True)
callback = keras.callbacks.EarlyStopping(
    monitor='val_auc', min_delta=0.001, patience=25,
    mode='max', baseline=None, restore_best_weights=True
)

model.compile(optimizer = keras.optimizers.Adam(),
              loss=keras.losses.BinaryCrossentropy(from_logits=True),
              metrics=[tf.keras.metrics.AUC(curve='ROC', from_logits=True)]
             )<jupyter_output><empty_output><jupyter_text>dont need to scale it, model is using per batch normalization<jupyter_code># pipe = Pipeline([('pca', PCA())])
# pipe.fit(df, y)
history = model.fit(df, y, batch_size = 1000, epochs=1000, callbacks=[callback])
for key in history.history:
    plt.plot(history.history[key])
del df
df = pd.read_csv("../input/tabular-playground-series-nov-2021/test.csv")
id = df.pop('id')
# df = pd.DataFrame(pca.transform(df))
# df = pca.transform(df)
# res = pipe.transform(df)
y_hat = model.predict(df)
y_hat.shape
# y_hat = pd.DataFrame(res)
# y_hat.describe()<jupyter_output><empty_output><jupyter_text>predicts the logit<jupyter_code># y_hat = pd.DataFrame(tf.nn.sigmoid(res))<jupyter_output><empty_output><jupyter_text>nn sigmoid ran and ran<jupyter_code>y_hat_proba = keras.activations.sigmoid(y_hat).numpy()
y_hat_proba = pd.DataFrame(y_hat_proba, index=id)
y_hat_proba.describe()
y_hat_proba.rename(columns = {0: "target"}, inplace=True)
y_hat_proba.to_csv('./submission.csv')
# res['id'] = id
# res.rename(columns = {1: 'target'}, inplace=True)
# cols = ['id','target']
# res.to_csv('./submission.csv', index=False, columns=cols)
# res.head()<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.13574423480083858, 'fraction_numerical': 0.03537735849056604, 'mean_word_length': 6.003669724770642, 'pattern_counts': {'":': 0, '<': 24, '<?xml version=': 0, '>': 25, 'https://': 1, 'lorem ipsum': 0, 'www.': 1, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '6', 'dir': 'jupyter-structured-clean-dedup', 'id': '347387', 'license': 'MIT', 'n_tokens_mistral': 1460, 'n_tokens_neox': 1357, 'n_words': 313, 'path': 'Kaggle/November2021 Kaggle Tabular Playground Competition - deep neural network without pca.ipynb', 'repo_name': 'DU-ds/Misc-Python-Scripts', 'size': '3816'}