<jupyter_start><jupyter_text><NAME> Emitted Tirelessly (P.O.E.T)A notebook that uses a shallow LSTM neural network trained on amateur poems found online to generate new poems.Steps to have P.O.E.T generate a poem:1. Run each of the start up cells at the top of the notebook by selecting the cell and pressing either the play button on the screen or SHIFT+ENTER2. Pick which of the 2 different ways you want P.O.E.T to write a poem: * Probabalistic (random results) * Augmented (you help pick the words) In addition to unstemmed words, P.O.E.T. also uses the following 3 tokens when writing a poem:* starttoken   -- identifies the start of a poem. Often come in pairs b/c of how training was performed. * returntoken  -- identifies the end of a line in the poem ('\n').* endtoken2    -- identifies the end of a poem.  Caveats: This project is still a work in progress. The current P.O.E.T. model is a shallow first prototype trained on a relatively small amount of data.<jupyter_code>#start up cell -- import necessary python functions and libraries
import os
import pickle
import numpy as np
from IPython.display import clear_output
import gzip

import AiPoems
#start up cell -- import necessary metadata for model
with open('tokenizer.pickle', 'rb') as handle:
    tokenizer = pickle.load(handle)
with gzip.GzipFile('predictors.npy.gz', "r") as f:
    predictors=np.load(f)
with gzip.GzipFile('label.npy.gz', "r") as f:
    label=np.load(f)
total_words=len(label[0])
max_sequence_len=len(predictors[0])+1
filename='word_model_love_poems_composite_100.h5'
#start up cell -- initialize model
model = AiPoems.initialize_model(predictors, label, max_sequence_len, 
                 total_words, device='/cpu:0')
model=AiPoems.load_model(model, filename)<jupyter_output><empty_output><jupyter_text>Probability based poemP.O.E.T. will write you a "poem." The program selects the next word from a probability distribution based on the previous words in the poem.Change the maxWords variable to change the maximum number of words P.O.E.T. will include in the poem before ending the poem. P.O.E.T. may end the poem early, if an end-poem-token ('endtoken2') is selected while writing.<jupyter_code>maxWords=150
seed_text='starttoken'
AiPoem=AiPoems.generate_text_random(model, tokenizer, 
                                      maxWords, max_sequence_len, 
                                      seed_text=seed_text, top_n=30)
print(AiPoem.replace('starttoken','').replace('returntoken','\n').split('endtoken2')[0])<jupyter_output><empty_output><jupyter_text>AI augmented writingHelp P.O.E.T. out! The AI will use its probablistic model to generate several lines of "poetry." Select the one you think is best and P.O.E.T will use your selection to generate options for the next line.Enter a non-integer (ex: 'end') to end early.Change choices variable to control number of lines for P.O.E.T to provide as options.<jupyter_code>choices=5
AiPoem=AiPoems.augmented_line_writing(model, tokenizer, max_sequence_len, choices=choices)
print(AiPoem.replace('starttoken','').replace('returntoken','\n '))<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.09184328837508028, 'fraction_numerical': 0.007064868336544637, 'mean_word_length': 4.398613518197574, 'pattern_counts': {'":': 0, '<': 14, '<?xml version=': 0, '>': 14, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 2, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '3', 'dir': 'jupyter-structured-clean-dedup', 'id': '189356', 'license': 'Apache-2.0', 'n_tokens_mistral': 975, 'n_tokens_neox': 941, 'n_words': 353, 'path': 'PoemGeneration.ipynb', 'repo_name': 'keklarup/POET_AiPoetry', 'size': '3120'}