<jupyter_start><jupyter_text>Text Generator<jupyter_code>%tensorflow_version 1.x
!pip install -q gpt-2-simple
import gpt_2_simple as gpt2<jupyter_output><empty_output><jupyter_text>There are three released sizes of GPT-2:
- 124M (default): the "small" model, 500MB on disk.
- 355M: the "medium" model, 1.5GB on disk.
- 774M: the "large" model
- 1558M: the "extra large"

The large model cannot currently be finetuned with Colaboratory but can be used to generate text from a pretrained model.

The extra large is the true model. It will not work if a K80/P4 GPU is attached to the notebook and as with the large model, it cannot be finetuned.

The larger the model, the greater the knowledge but the longer the processing time. 

Use *model_name* to cahnge the base model.<jupyter_code># Check GPU
!nvidia-smi
gpt2.download_gpt2(model_name="124M")
from google.colab import drive
drive.mount('/content/drive')
txt = '/content/drive/MyDrive/data/Text/Down To The Sunless Sea.txt'
sess = gpt2.start_tf_sess()

gpt2.finetune(sess,
              dataset=txt,
              model_name='124M',
              steps=1000,
              restore_from='fresh',
              run_name='run1',
              print_every=10,
              sample_every=200,
              save_every=500
              )
gpt2.copy_checkpoint_to_gdrive(run_name='run1')
gpt2.copy_checkpoint_from_gdrive(run_name='run1')
# sess = gpt2.start_tf_sess()
# gpt2.load_gpt2(sess, run_name='run1')<jupyter_output><empty_output><jupyter_text>After training the model or loading a retrained model from checkpoint, we can generate text. *generate* generates a single text from the loaded model.<jupyter_code>gpt2.generate(sess, run_name='run1')<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.09174311926605505, 'fraction_numerical': 0.0338302752293578, 'mean_word_length': 3.780821917808219, 'pattern_counts': {'":': 0, '<': 13, '<?xml version=': 0, '>': 13, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 1, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '3', 'dir': 'jupyter-structured-clean-dedup', 'id': '498952', 'license': 'MIT', 'n_tokens_mistral': 618, 'n_tokens_neox': 587, 'n_words': 169, 'path': 'Text_Generator.ipynb', 'repo_name': 'antnewman/nlp-textgenerator-gpt2-notebook', 'size': '1744'}