<jupyter_start><jupyter_text>IntroductionThis notebook is inspired by [Keras example](https://keras.io/examples/generative/lstm_character_level_text_generation/) on generating text from Nietzsche's writings with a character-level LSTM. As our dataset, we used 597 poems by [<NAME>](https://www.kaggle.com/jenlooper/597-poems-by-emily-dickinson).  Setup<jupyter_code>from tensorflow import keras
from tensorflow.keras import layers

import numpy as np
import random
import io<jupyter_output><empty_output><jupyter_text>Prepare the data<jupyter_code>path = keras.utils.get_file(
    "emily-together.txt", origin="https://raw.githubusercontent.com/katarinagresova/M7DataSP_2020/master/assignment_07/data/emily-together.txt"
)
with io.open(path, encoding="utf-8") as f:
    text = f.read().lower()
print("Corpus length:", len(text))

chars = sorted(list(set(text)))
print("Total chars:", len(chars))
char_indices = dict((c, i) for i, c in enumerate(chars))
indices_char = dict((i, c) for i, c in enumerate(chars))

# cut the text in semi-redundant sequences of maxlen characters
maxlen = 20
step = 3
sentences = []
next_chars = []
for i in range(0, len(text) - maxlen, step):
    sentences.append(text[i : i + maxlen])
    next_chars.append(text[i + maxlen])
print("Number of sequences:", len(sentences))

x = np.zeros((len(sentences), maxlen, len(chars)), dtype=np.bool)
y = np.zeros((len(sentences), len(chars)), dtype=np.bool)
for i, sentence in enumerate(sentences):
    for t, char in enumerate(sentence):
        x[i, t, char_indices[char]] = 1
    y[i, char_indices[next_chars[i]]] = 1<jupyter_output>Downloading data from https://raw.githubusercontent.com/katarinagresova/M7DataSP_2020/main/assignment_07/data/emily-together.txt
188416/184901 [==============================] - 0s 0us/step
Corpus length: 183924
Total chars: 42
Number of sequences: 61302<jupyter_text>Build the model: a single LSTM layer<jupyter_code>model = keras.Sequential(
    [
        keras.Input(shape=(maxlen, len(chars))),
        layers.LSTM(128, return_sequences=True),
        layers.LSTM(128, return_sequences=False),
        layers.Dense(len(chars), activation="softmax"),
    ]
)
optimizer = keras.optimizers.RMSprop(learning_rate=0.01)
model.compile(loss="categorical_crossentropy", optimizer=optimizer)<jupyter_output><empty_output><jupyter_text>Prepare the text sampling function<jupyter_code>def sample(preds, temperature=1.0):
    # helper function to sample an index from a probability array
    preds = np.asarray(preds).astype("float64")
    preds = np.log(preds) / temperature
    exp_preds = np.exp(preds)
    preds = exp_preds / np.sum(exp_preds)
    probas = np.random.multinomial(1, preds, 1)
    return np.argmax(probas)<jupyter_output><empty_output><jupyter_text>Train the model<jupyter_code>epochs = 40
batch_size = 128

model.fit(x, y, batch_size=batch_size, epochs=epochs)
for diversity in [0.2, 0.5, 1.0, 1.2]:
  print("...Diversity:", diversity)

  generated = ""
  #sentence = text[start_index : start_index + maxlen]
  sentence = 'sunshine'
  pad_len = maxlen - len(sentence)
  if pad_len < 0:
    sentence = sentence[:maxlen]
  elif pad_len > 0:
      sentence = " " * (pad_len - 1) + sentence + "."

  print('...Generating with seed: "' + sentence + '"')

  for i in range(200):
      x_pred = np.zeros((1, maxlen, len(chars)))
      for t, char in enumerate(sentence):
          x_pred[0, t, char_indices[char]] = 1.0
      preds = model.predict(x_pred, verbose=0)[0]
      next_index = sample(preds, diversity)
      next_char = indices_char[next_index]
      sentence = sentence[1:] + next_char
      generated += next_char



  print("...Generated: ", generated)
  print()<jupyter_output>...Diversity: 0.2
...Generating with seed: "           sunshine."
{'fraction_non_alphanumeric': 0.14361560923812053, 'fraction_numerical': 0.025749933634191664, 'mean_word_length': 4.5988112927191676, 'pattern_counts': {'":': 0, '<': 21, '<?xml version=': 0, '>': 21, 'https://': 4, 'lorem ipsum': 0, 'www.': 1, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 1, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '5', 'dir': 'jupyter-structured-clean-dedup', 'id': '387385', 'license': 'MIT', 'n_tokens_mistral': 1397, 'n_tokens_neox': 1299, 'n_words': 284, 'path': 'assignment_07/PoetryCharByChar.ipynb', 'repo_name': 'katarinagresova/M7DataSP_2020', 'size': '3776'}