<jupyter_start><jupyter_text>Exercise 1 - Deep AutoencoderBuild and train a deep autoencoder network. If possible, try to make your middle layer (embedding) just two dimensional, so that we can visualize the images compressed into a 2D space. Use the code below to extract the embedded representation for any image. For a set of test images, make a 2D scatter plot of the embedding representation and then color by the image class. Do you see any separation between the different MNIST numbers?<jupyter_code>from keras import backend as K
def extract_layer_output(model, layer_num):
    return K.function([model.input] + [K.learning_phase()], [model.layers[layer_num].output])
embedding_layer = extract_layer_output(autoenc, 3) # you'll need to edit this
embeddings = embedding_layer([x_test[1:1000,]])[0]
plt.scatter(embeddings[:,0], embeddings[:,1], c=y_test[1:1000])<jupyter_output><empty_output><jupyter_text>Exercise 2 - AirfoilBuild and train an autoencoder on the UCI Airfoil dataset (https://archive.ics.uci.edu/ml/datasets/airfoil+self-noise) *  Download the data and load it into your python session (either with Numpy or Pandas). You may have to do some data transformations to get things into the proper arrays that Keras expects.* Do some quick exploratory data analsis. What variables are in this dataset? For our purposes, we will *not* consider the "Sound Pressure" variable as a target. Instead, we will proceed with all the varialbes and consider only unsupervised methods. Make a few scatterplots (or scatterplot matrix) to see how the variables relate to each other. This dataset is derived from NASA wind tunnel experiments. Thus, the relationships between these variables are probably described by some rather simple physics equations. It might be the case that even though these data are measured in several dimensions, their variability might span only a low-dimensional manifold. We'll use an autocoder to test this hypothesis and see if we can capture this manifold.   * Build and train an autoencoder. Explore which activation functions seems to be most helpful toward getting the smallest loss. Explore which architectures seem to be more successful than others.* (Optional) If time permits, explore some visualization techniques of your embeddings, as described in Exercise 1.<jupyter_code>import pandas as pd

data = pd.read_csv('airfoil_self_noise.dat.txt', sep='\t', header=None)
from sklearn.model_selection import train_test_split
from sklearn import preprocessing

x = data.values #returns a numpy array
min_max_scaler = preprocessing.MinMaxScaler()
x_scaled = min_max_scaler.fit_transform(x)
data = pd.DataFrame(x_scaled)


Y = data[5]
X = data.drop(5, axis=1)

X_train, X_test, y_train, y_test = train_test_split(X, Y, test_size=0.2, random_state=42)

autoenc = Sequential()
autoenc.add(Dense(32, activation="relu", input_shape=(5,)))
autoenc.add(Dense(5, activation="sigmoid"))
autoenc.add(Dense(1, activation="sigmoid"))

autoenc.compile(optimizer='adadelta', loss='logcosh')

autoenc.fit(X_train, y_train,
                epochs=10,
                batch_size=32,
                shuffle=True,
                validation_data=(X_test, y_test))


from keras.losses import logcosh

score = autoenc.evaluate(X_test, y_test, batch_size=128)
score<jupyter_output>301/301 [==============================] - 0s 13us/step
{'fraction_non_alphanumeric': 0.0864897107068297, 'fraction_numerical': 0.014017297942141366, 'mean_word_length': 5.222634508348794, 'pattern_counts': {'":': 0, '<': 8, '<?xml version=': 0, '>': 8, 'https://': 1, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 1, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '2', 'dir': 'jupyter-structured-clean-dedup', 'id': '532219', 'license': 'MIT', 'n_tokens_mistral': 992, 'n_tokens_neox': 935, 'n_words': 380, 'path': 'Materials/L6 - Unsupervised Methods in DL/autoencoders.ipynb', 'repo_name': 'WuZhuoran/ANLY-590-Neural-Nets-and-Deep-Learning', 'size': '3353'}