<jupyter_start><jupyter_text>Sample Steam Reviews with GPT-2Code inspired from https://github.com/woctezuma/sample-steam-reviews-with-gpt-2  Setting the GPT-2 model Install the Python packageReference: https://github.com/minimaxir/gpt-2-simple<jupyter_code>!pip install gpt_2_simple<jupyter_output>Collecting gpt_2_simple
  Downloading https://files.pythonhosted.org/packages/bc/7d/1ea4c2a54ecdda5e57e45686e5cdf1ccc45809841ab50c89bc63638c5553/gpt_2_simple-0.5.tar.gz
Requirement already satisfied: regex in /usr/local/lib/python3.6/dist-packages (from gpt_2_simple) (2018.1.10)
Requirement already satisfied: requests in /usr/local/lib/python3.6/dist-packages (from gpt_2_simple) (2.21.0)
Requirement already satisfied: tqdm in /usr/local/lib/python3.6/dist-packages (from gpt_2_simple) (4.28.1)
Requirement already satisfied: numpy in /usr/local/lib/python3.6/dist-packages (from gpt_2_simple) (1.16.3)
Collecting toposort (from gpt_2_simple)
  Downloading https://files.pythonhosted.org/packages/e9/8a/321cd8ea5f4a22a06e3ba30ef31ec33bea11a3443eeb1d89807640ee6ed4/toposort-1.5-py2.py3-none-any.whl
Requirement already satisfied: urllib3<1.25,>=1.21.1 in /usr/local/lib/python3.6/dist-packages (from requests->gpt_2_simple) (1.24.3)
Requirement already satisfied: idna<2.9,>=2.5 in /usr/local/lib/p[...]<jupyter_text>Import<jupyter_code>import gpt_2_simple as gpt2
from datetime import datetime
from google.colab import files<jupyter_output><empty_output><jupyter_text>Downloading GPT-2 Choose between `117M` and `345M` models<jupyter_code># model_name = '117M'
model_name = '345M'<jupyter_output><empty_output><jupyter_text>Download the pre-trained model<jupyter_code>gpt2.download_gpt2(model_name=model_name)<jupyter_output>Fetching checkpoint: 1.00kit [00:00, 257kit/s]                                                      
Fetching encoder.json: 1.04Mit [00:00, 42.4Mit/s]                                                   
Fetching hparams.json: 1.00kit [00:00, 300kit/s]                                                    
Fetching model.ckpt.data-00000-of-00001: 1.42Git [00:22, 62.7Mit/s]                                 
Fetching model.ckpt.index: 11.0kit [00:00, 2.14Mit/s]                                               
Fetching model.ckpt.meta: 927kit [00:00, 39.0Mit/s]                                                 
Fetching vocab.bpe: 457kit [00:00, 30.9Mit/s]<jupyter_text>Uploading a Text File to be Trained to Colaboratory  Either get the data by yourself<jupyter_code>!curl -O https://raw.githubusercontent.com/woctezuma/sample-steam-reviews-with-gpt-2/master/export_review_data.py
!curl -O https://raw.githubusercontent.com/woctezuma/sample-steam-reviews-with-gpt-2/master/requirements.txt
!pip install -r requirements.txt
app_id = 203770 # Artifact: 583950

# num_days = 28*3 # slightly less than 3 months
num_days = -1 # if negative, then no time limit
from export_review_data import apply_workflow_for_app_id

apply_workflow_for_app_id(app_id,
                          num_days=num_days)<jupyter_output>[appID = 203770] expected #reviews = 24077
Number of queries 150 reached. Cooldown: 310 seconds
#reviews = 24071
Filtering out reviews which were not written in english.
#reviews = 24071
Filtering out reviews with strictly fewer than 150 characters.
#reviews = 11011
[review n°49661536] https://steamcommunity.com/profiles/76561198173075417/recommended/203770/
░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░
░░░░░░████░░████░░░████░░░░░░
░░░░░░░████░░████░░░░░░░░░░░░░
░░░░░░████░░████░░░████░░░░░░
░░░░░░░████▄▄████░░░████░░░░░░
░░░░░░██████████░░░████░░░░░░
░░░░░░░████▀▀████░░░████░░░░░░
░░░░░░░████░░████░░░████░░░░░░
░░░░░░████░░████░░░████░░░░░░
░░░░░░░████░░████░░░████░░░░░░
░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░
_______________$
 ___________$$$$$_________________________$$
 ________$$$$$$$$$$$______________________$$
 ______$$$$$$$$$$$$$$$____________________$$
 _____$$$$$$$$$$$$$$$$____________________$$
 ____$$$$$$$$$$$$$$$$$$$__________________$$
 ___$$$$$$$$$$$$$$$$$$$$$_________________$$
 __$$$$$$$$$$$$[...]<jupyter_text>Or get a data snapshot from meCurrently only possible for Artifact, as an example, because the recommended way is to run the code above for the game of your choice instead.<jupyter_code>!mkdir -p data/

## Either Artifact (only the recent English reviews):
# !curl -O https://raw.githubusercontent.com/woctezuma/sample-steam-reviews-with-gpt-2/master/data/with_delimiters/583950.txt
# !mv 583950.txt data/

## Or Cr<NAME> (all the English reviews):
# !curl -O https://raw.githubusercontent.com/wiki/woctezuma/sample-steam-reviews-with-gpt-2/data/with_delimiters/203770.txt
# !mv 203770.txt data/<jupyter_output><empty_output><jupyter_text>Finetune GPT-2<jupyter_code>file_name = 'data/' + str(app_id) + '.txt'

run_name = model_name + '_reviews_' + str(app_id)
sess = gpt2.start_tf_sess()

gpt2.finetune(sess,
              run_name=run_name,
              dataset=file_name,
              model_name=model_name,
              steps=1000,
              restore_from='fresh', # change to 'latest' to resume training
              print_every=10,       # how many steps between printing progress
              sample_every=200,     # how many steps to print a demo sample
              save_every=500        # how many steps between saving checkpoint              
              )<jupyter_output>WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/tensorflow/python/framework/op_def_library.py:263: colocate_with (from tensorflow.python.framework.ops) is deprecated and will be removed in a future version.
Instructions for updating:
Colocations handled automatically by placer.
WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/gpt_2_simple/src/sample.py:71: to_float (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Use tf.cast instead.
WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/gpt_2_simple/src/sample.py:77: multinomial (from tensorflow.python.ops.random_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Use tf.random.categorical instead.
WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/tensorflow/python/ops/math_ops.py:3066: to_int32 (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future ver[...]<jupyter_text>Save a Trained Model Checkpoint<jupyter_code># gpt2.mount_gdrive()
# gpt2.copy_checkpoint_to_gdrive(run_name=run_name)<jupyter_output><empty_output><jupyter_text>Load a Trained Model Checkpoint<jupyter_code># gpt2.mount_gdrive()
# gpt2.copy_checkpoint_from_gdrive(run_name=run_name)<jupyter_output><empty_output><jupyter_text>Generate Text From The Trained Model<jupyter_code>temperature=1.0 # Default is 0.7, but you may want to increase the temperature, especially if your dataset is small, to avoid copying text.
top_k = 40      # Default: 0   ; Recommended: 40  ; useless parameter if top_p > 0.0
top_p = 0.9     # Default: 0.0 ; Recommended: 0.9 ; no need for top_k if top_p > 0.0
num_samples = 3
num_batches = 3 # Unique to GPT-2, you can pass a batch_size to generate multiple samples in parallel, giving a massive speedup.
gen_texts_A = gpt2.generate(sess,
              run_name=run_name,
              nsamples=num_samples,
              batch_size=num_batches,              
              temperature=temperature,
              top_k=top_k,
              top_p=top_p,
              truncate='<|endoftext|>',                            
              return_as_list=True)

print('\n\n--- SEPARATOR ---\n\n'.join(gen_texts_A))
gen_texts_B = gpt2.generate(sess,
              run_name=run_name,
              nsamples=num_samples,
              batch_size=num_batches,
              temperature=temperature,
              top_k=top_k,
              top_p=top_p,                            
              prefix='<|startoftext|>I love',
              truncate='<|endoftext|>',
              return_as_list=True)

print('\n\n--- SEPARATOR ---\n\n'.join(gen_texts_B))
gen_texts_C = gpt2.generate(sess,
              run_name=run_name,
              nsamples=num_samples,
              batch_size=num_batches,
              temperature=temperature,
              top_k=top_k,
              top_p=top_p,                            
              prefix='<|startoftext|>I hate',
              truncate='<|endoftext|>',
              return_as_list=True)

print('\n\n--- SEPARATOR ---\n\n'.join(gen_texts_C))
gen_texts_D = gpt2.generate(sess,
              run_name=run_name,
              nsamples=num_samples,
              batch_size=num_batches,
              temperature=temperature,
              top_k=top_k,
              top_p=top_p,                            
              prefix='<|startoftext|>Please',
              truncate='<|endoftext|>',
              return_as_list=True)

print('\n\n--- SEPARATOR ---\n\n'.join(gen_texts_D))
gen_texts_E = gpt2.generate(sess,
              run_name=run_name,
              nsamples=num_samples,
              batch_size=num_batches,
              temperature=temperature,
              top_k=top_k,
              top_p=top_p,                            
              prefix='<|startoftext|>This game has near infinite replay value',
              truncate='<|endoftext|>',
              return_as_list=True)

print('\n\n--- SEPARATOR ---\n\n'.join(gen_texts_E))<jupyter_output><|startoftext|>This game has near infinite replay value, as long as you can stay with it for a few months before your character dies, your future becomes uncertain. As a new character you'll keep trying to keep your marriage healthy, while the first ruler you play as makes mistakes and dukes are weak or evil. There is so much to keep track of and watch the relationships between your characters, their children and surrounding dukes. It's a unique and interesting take on the whole medieval (and indeed medieval history) world.
If you find your love of historical re-playability, political intrigue and history soothing, this game may just be for you!
9.8/10 is recommended to anyone who enjoys using their imagination and imagination to rule through tough times.
But don't let the silly, discordant, and complex nature of the game scare you away. You'll have a blast in the magic of the role playing and backstabbing that is CK2.

--- SEPARATOR ---

<|startoftext|>This game has near infinite repl[...]<jupyter_text>Copy the Generated Text to Google Drive<jupyter_code>temperature_suffixe = '_temperature_' + str(temperature)
if top_p > 0.0:
  file_name_suffixe = temperature_suffixe + '_top_p_' + str(top_p)
elif top_k > 0:
  file_name_suffixe = temperature_suffixe + '_top_k_' + str(top_k)
else:
  file_name_suffixe = temperature_suffixe
output_file_name = 'output_' + str(app_id) + file_name_suffixe + '.md'

print(output_file_name)
with open(output_file_name, 'w') as f:
  
  f.write('## Game\n\n')
  f.write('[<img alt="game name" src="https://steamcdn-a.akamaihd.net/steam/apps/{}/header.jpg" width="150">](https://store.steampowered.com/app/{})\n\n'.format(app_id, app_id))
  
  f.write('## Reviews generated unconditionally\n\n')
  for (i, gen_text) in enumerate(gen_texts_A):
    f.write('{}.\n\n'.format(i+1))
    f.write('> {}\n\n'.format(gen_text))
    
  f.write('## Reviews starting with I love\n\n')
  for (i, gen_text) in enumerate(gen_texts_B):
    f.write('{}.\n\n'.format(i+1))
    f.write('> {}\n\n'.format(gen_text))
    
  f.write('## Reviews starting with I hate\n\n')    
  for (i, gen_text) in enumerate(gen_texts_C):
    f.write('{}.\n\n'.format(i+1))
    f.write('> {}\n\n'.format(gen_text))
  
  f.write('## Reviews starting with Please\n\n')  
  for (i, gen_text) in enumerate(gen_texts_D):
    f.write('{}.\n\n'.format(i+1))
    f.write('> {}\n\n'.format(gen_text))
  
  f.write('## Reviews starting with This game has near infinite replay value\n\n')  
  for (i, gen_text) in enumerate(gen_texts_E):
    f.write('{}.\n\n'.format(i+1))
    f.write('> {}\n\n'.format(gen_text))
   
gpt2.mount_gdrive()
import shutil

shutil.copyfile(output_file_name, '/content/drive/My Drive/' + output_file_name)<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.1415288406154859, 'fraction_numerical': 0.03546449436353164, 'mean_word_length': 3.322190611664296, 'pattern_counts': {'":': 0, '<': 55, '<?xml version=': 0, '>': 65, 'https://': 11, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 1, 'word_list_counts': {'cursed_substrings.json': 15, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '11', 'dir': 'jupyter-structured-clean-dedup', 'id': '410692', 'license': 'MIT', 'n_tokens_mistral': 4554, 'n_tokens_neox': 4408, 'n_words': 866, 'path': 'gpt_2_for_reviews.ipynb', 'repo_name': 'woctezuma/sample-steam-reviews-with-gpt-2', 'size': '12162'}