<jupyter_start><jupyter_text>Keras TunerA hyperparameter tuner for [Keras](https://keras.io), specifically for `tf.keras` with TensorFlow 2.0.Full documentation and tutorials available on the [Keras Tuner website.](https://keras-team.github.io/keras-tuner/)  A simple helloworld example Different workflows are shown here.<jupyter_code>try:
  # %tensorflow_version only exists in Colab.
  %tensorflow_version 2.x
except Exception:
  pass<jupyter_output>TensorFlow 2.x selected.<jupyter_text>1. Install keras-tuner<jupyter_code>!pip install -U keras-tuner<jupyter_output>Requirement already up-to-date: keras-tuner in /usr/local/lib/python3.6/dist-packages (1.0.1)
Requirement already satisfied, skipping upgrade: tabulate in /usr/local/lib/python3.6/dist-packages (from keras-tuner) (0.8.6)
Requirement already satisfied, skipping upgrade: terminaltables in /usr/local/lib/python3.6/dist-packages (from keras-tuner) (3.1.0)
Requirement already satisfied, skipping upgrade: requests in /tensorflow-2.1.0/python3.6 (from keras-tuner) (2.22.0)
Requirement already satisfied, skipping upgrade: future in /usr/local/lib/python3.6/dist-packages (from keras-tuner) (0.16.0)
Requirement already satisfied, skipping upgrade: scipy in /usr/local/lib/python3.6/dist-packages (from keras-tuner) (1.4.1)
Requirement already satisfied, skipping upgrade: colorama in /usr/local/lib/python3.6/dist-packages (from keras-tuner) (0.4.3)
Requirement already satisfied, skipping upgrade: tqdm in /usr/local/lib/python3.6/dist-packages (from keras-tuner) (4.28.1)
Requirement already satisfie[...]<jupyter_text>2. Import required libraries<jupyter_code>from tensorflow import keras
from tensorflow.keras import layers


from kerastuner.tuners import RandomSearch
from kerastuner.engine.hypermodel import HyperModel
from kerastuner.engine.hyperparameters import HyperParameters<jupyter_output><empty_output><jupyter_text>3. Load dataset<jupyter_code>(x, y), (val_x, val_y) = keras.datasets.mnist.load_data()
x = x.astype('float32') / 255.
val_x = val_x.astype('float32') / 255.

x = x[:10000]
y = y[:10000]<jupyter_output><empty_output><jupyter_text>4. Usage  4.1 Basic usage- We define a `build_model` function- It returns a compiled model- It uses hyperparameters defined on the fly<jupyter_code>def build_model(hp):
    model = keras.Sequential()
    model.add(layers.Flatten(input_shape=(28, 28)))
    for i in range(hp.Int('num_layers', 2, 20)):
        model.add(layers.Dense(units=hp.Int('units_' + str(i), 32, 512, 32),
                               activation='relu'))
    model.add(layers.Dense(10, activation='softmax'))
    model.compile(
        optimizer=keras.optimizers.Adam(
            hp.Choice('learning_rate', [1e-2, 1e-3, 1e-4])),
        loss='sparse_categorical_crossentropy',
        metrics=['accuracy'])
    return model
tuner = RandomSearch(
    build_model,
    objective='val_accuracy',
    max_trials=5,
    executions_per_trial=3,
    directory='test_dir',
    project_name='helloworld')
tuner.search_space_summary()
tuner.search(x=x, y=y, epochs=3, validation_data=(val_x, val_y))
tuner.results_summary()<jupyter_output><empty_output><jupyter_text>4.2 We override the loss and metrics<jupyter_code>tuner = RandomSearch(
    build_model,
    objective='val_accuracy',
    loss=keras.losses.SparseCategoricalCrossentropy(name='my_loss'),
    metrics=['accuracy', 'mse'],
    max_trials=5,
    directory='test_dir',
    project_name='helloworld2')
tuner.search(x=x, y=y, epochs=5, validation_data=(val_x, val_y))<jupyter_output>Train on 10000 samples, validate on 10000 samples
Epoch 1/5
10000/10000 [==============================] - 3s 278us/sample - loss: 2.0697 - accuracy: 0.2453 - mse: 27.1467 - val_loss: 1.6866 - val_accuracy: 0.3332 - val_mse: 27.2620
Epoch 2/5
10000/10000 [==============================] - 2s 214us/sample - loss: 1.3720 - accuracy: 0.4770 - mse: 27.1656 - val_loss: 1.2037 - val_accuracy: 0.5582 - val_mse: 27.2799
Epoch 3/5
10000/10000 [==============================] - 2s 214us/sample - loss: 0.9330 - accuracy: 0.6802 - mse: 27.1813 - val_loss: 0.7973 - val_accuracy: 0.7348 - val_mse: 27.2963
Epoch 4/5
10000/10000 [==============================] - 2s 215us/sample - loss: 0.7074 - accuracy: 0.7675 - mse: 27.1936 - val_loss: 0.6685 - val_accuracy: 0.7836 - val_mse: 27.3030
Epoch 5/5
10000/10000 [==============================] - 2s 214us/sample - loss: 0.5949 - accuracy: 0.8163 - mse: 27.2001 - val_loss: 0.5853 - val_accuracy: 0.8250 - val_mse: 27.3081<jupyter_text>4.3 We define a HyperModel subclass<jupyter_code>class MyHyperModel(HyperModel):

    def __init__(self, img_size, num_classes):
        self.img_size = img_size
        self.num_classes = num_classes

    def build(self, hp):
        model = keras.Sequential()
        model.add(layers.Flatten(input_shape=self.img_size))
        for i in range(hp.Int('num_layers', 2, 20)):
            model.add(layers.Dense(units=hp.Int('units_' + str(i), 32, 512, 32),
                                   activation='relu'))
        model.add(layers.Dense(self.num_classes, activation='softmax'))
        model.compile(
            optimizer=keras.optimizers.Adam(
                hp.Choice('learning_rate', [1e-2, 1e-3, 1e-4])),
            loss='sparse_categorical_crossentropy',
            metrics=['accuracy'])
        return model
tuner = RandomSearch(
    MyHyperModel(img_size=(28, 28), num_classes=10),
    objective='val_accuracy',
    max_trials=5,
    directory='test_dir',
    project_name='helloworld3')
tuner.search(x=x,y=y,epochs=5,validation_data=(val_x, val_y))<jupyter_output>Train on 10000 samples, validate on 10000 samples
Epoch 1/5
10000/10000 [==============================] - 2s 228us/sample - loss: 0.8039 - accuracy: 0.7413 - val_loss: 0.3588 - val_accuracy: 0.8966
Epoch 2/5
10000/10000 [==============================] - 2s 175us/sample - loss: 0.2703 - accuracy: 0.9230 - val_loss: 0.2409 - val_accuracy: 0.9296
Epoch 3/5
10000/10000 [==============================] - 2s 172us/sample - loss: 0.1841 - accuracy: 0.9478 - val_loss: 0.2268 - val_accuracy: 0.9366
Epoch 4/5
10000/10000 [==============================] - 2s 177us/sample - loss: 0.1320 - accuracy: 0.9617 - val_loss: 0.2368 - val_accuracy: 0.9323
Epoch 5/5
10000/10000 [==============================] - 2s 179us/sample - loss: 0.1171 - accuracy: 0.9671 - val_loss: 0.1935 - val_accuracy: 0.9456<jupyter_text>4.4 We restrict the search space- This means that default values are being used for params that are left out<jupyter_code>hp = HyperParameters()
hp.Choice('learning_rate', [1e-1, 1e-3])
tuner = RandomSearch(
    build_model,
    max_trials=5,
    hyperparameters=hp,
    tune_new_entries=False,
    objective='val_accuracy',
    directory='test_dir',
    project_name='helloworld4')
tuner.search(x=x, y=y, epochs=5, validation_data=(val_x, val_y))<jupyter_output>Train on 10000 samples, validate on 10000 samples
Epoch 1/5
10000/10000 [==============================] - 2s 186us/sample - loss: 0.7432 - accuracy: 0.7824 - val_loss: 0.3444 - val_accuracy: 0.9004
Epoch 2/5
10000/10000 [==============================] - 2s 154us/sample - loss: 0.2879 - accuracy: 0.9201 - val_loss: 0.2798 - val_accuracy: 0.9168
Epoch 3/5
10000/10000 [==============================] - 2s 152us/sample - loss: 0.2253 - accuracy: 0.9345 - val_loss: 0.2641 - val_accuracy: 0.9221
Epoch 4/5
10000/10000 [==============================] - 2s 153us/sample - loss: 0.1907 - accuracy: 0.9441 - val_loss: 0.2408 - val_accuracy: 0.9268
Epoch 5/5
10000/10000 [==============================] - 2s 155us/sample - loss: 0.1661 - accuracy: 0.9506 - val_loss: 0.2321 - val_accuracy: 0.9297<jupyter_text>4.5 We override specific parameters with fixed values that aren't the default<jupyter_code>tuner = RandomSearch(
    build_model,
    max_trials=5,
    hyperparameters=hp,
    tune_new_entries=True,
    objective='val_accuracy',
    directory='test_dir',
    project_name='helloworld5')
tuner.search(x=x, y=y, epochs=5, validation_data=(val_x, val_y))<jupyter_output>Train on 10000 samples, validate on 10000 samples
Epoch 1/5
10000/10000 [==============================] - 3s 293us/sample - loss: 1.4778 - accuracy: 0.3929 - val_loss: 1.0899 - val_accuracy: 0.6163
Epoch 2/5
10000/10000 [==============================] - 2s 215us/sample - loss: 0.7366 - accuracy: 0.7648 - val_loss: 0.5028 - val_accuracy: 0.8756
Epoch 3/5
10000/10000 [==============================] - 2s 216us/sample - loss: 0.3756 - accuracy: 0.9086 - val_loss: 0.3552 - val_accuracy: 0.9120
Epoch 4/5
10000/10000 [==============================] - 2s 214us/sample - loss: 0.2867 - accuracy: 0.9295 - val_loss: 0.2886 - val_accuracy: 0.9272
Epoch 5/5
10000/10000 [==============================] - 2s 217us/sample - loss: 0.2472 - accuracy: 0.9416 - val_loss: 0.2710 - val_accuracy: 0.9350<jupyter_text>4.6  We reparameterize the search space- This means that we override the distribution of specific hyperparameters<jupyter_code>hp = HyperParameters()
hp.Choice('learning_rate', [1e-1, 1e-3])
tuner = RandomSearch(
    build_model,
    max_trials=5,
    hyperparameters=hp,
    tune_new_entries=True,
    objective='val_accuracy',
    directory='test_dir',
    project_name='helloworld6')
tuner.search(x=x, y=y, epochs=5, validation_data=(val_x, val_y))<jupyter_output>Train on 10000 samples, validate on 10000 samples
Epoch 1/5
10000/10000 [==============================] - 2s 213us/sample - loss: 0.6607 - accuracy: 0.7969 - val_loss: 0.2812 - val_accuracy: 0.9173
Epoch 2/5
10000/10000 [==============================] - 2s 179us/sample - loss: 0.2166 - accuracy: 0.9376 - val_loss: 0.2485 - val_accuracy: 0.9278
Epoch 3/5
10000/10000 [==============================] - 2s 178us/sample - loss: 0.1466 - accuracy: 0.9561 - val_loss: 0.1753 - val_accuracy: 0.9492
Epoch 4/5
10000/10000 [==============================] - 2s 172us/sample - loss: 0.1013 - accuracy: 0.9707 - val_loss: 0.1829 - val_accuracy: 0.9483
Epoch 5/5
10000/10000 [==============================] - 2s 171us/sample - loss: 0.0800 - accuracy: 0.9756 - val_loss: 0.2322 - val_accuracy: 0.9412<jupyter_text>4.7 We predefine the search space- No unregistered parameters are allowed in `build`<jupyter_code>hp = HyperParameters()
hp.Choice('learning_rate', [1e-1, 1e-3])
hp.Int('num_layers', 2, 20)
def build_model(hp):
    model = keras.Sequential()
    model.add(layers.Flatten(input_shape=(28, 28)))
    for i in range(hp.get('num_layers')):
        model.add(layers.Dense(32,
                               activation='relu'))
    model.add(layers.Dense(10, activation='softmax'))
    model.compile(
        optimizer=keras.optimizers.Adam(hp.get('learning_rate')),
        loss='sparse_categorical_crossentropy',
        metrics=['accuracy'])
    return model
tuner = RandomSearch(
    build_model,
    max_trials=5,
    hyperparameters=hp,
    allow_new_entries=False,
    objective='val_accuracy',
    directory='test_dir',
    project_name='helloworld7')
tuner.search(x=x, y=y, epochs=5, validation_data=(val_x, val_y))<jupyter_output>Train on 10000 samples, validate on 10000 samples
Epoch 1/5
10000/10000 [==============================] - 3s 293us/sample - loss: 2.9968 - accuracy: 0.1037 - val_loss: 2.3239 - val_accuracy: 0.0980
Epoch 2/5
10000/10000 [==============================] - 2s 215us/sample - loss: 2.3146 - accuracy: 0.0995 - val_loss: 2.3064 - val_accuracy: 0.1010
Epoch 3/5
10000/10000 [==============================] - 2s 233us/sample - loss: 2.3144 - accuracy: 0.1051 - val_loss: 2.3056 - val_accuracy: 0.1135
Epoch 4/5
10000/10000 [==============================] - 2s 220us/sample - loss: 2.3142 - accuracy: 0.0996 - val_loss: 2.3086 - val_accuracy: 0.0980
Epoch 5/5
10000/10000 [==============================] - 2s 212us/sample - loss: 2.3121 - accuracy: 0.1056 - val_loss: 2.3100 - val_accuracy: 0.1135
{'fraction_non_alphanumeric': 0.19885261115329453, 'fraction_numerical': 0.11608875390196574, 'mean_word_length': 4.22663139329806, 'pattern_counts': {'":': 0, '<': 37, '<?xml version=': 0, '>': 37, 'https://': 2, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 1, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '11', 'dir': 'jupyter-structured-clean-dedup', 'id': '141147', 'license': 'Apache-2.0', 'n_tokens_mistral': 5177, 'n_tokens_neox': 4163, 'n_words': 910, 'path': 'examples/helloworld_keras_tuner.ipynb', 'repo_name': 'mrm8488/keras-tuner', 'size': '11853'}