<jupyter_start><jupyter_text>GPT-2 Colab Clone git<jupyter_code>!git clone https://github.com/amarten1/gpt-2
!pip install tensorflow==1.15
import tensorflow
print(tensorflow.__version__)
!pip install fire
!pip install regex<jupyter_output>Collecting fire
[?25l  Downloading https://files.pythonhosted.org/packages/34/a7/0e22e70778aca01a52b9c899d9c145c6396d7b613719cd63db97ffa13f2f/fire-0.3.1.tar.gz (81kB)

[K     |████                            | 10kB 27.1MB/s eta 0:00:01
[K     |████████                        | 20kB 33.7MB/s eta 0:00:01
[K     |████████████▏                   | 30kB 15.5MB/s eta 0:00:01
[K     |████████████████▏               | 40kB 11.5MB/s eta 0:00:01
[K     |████████████████████▏           | 51kB 8.8MB/s eta 0:00:01
[K     |████████████████████████▎       | 61kB 10.2MB/s eta 0:00:01
[K     |████████████████████████████▎   | 71kB 9.5MB/s eta 0:00:01
[K     |████████████████████████████████| 81kB 6.4MB/s 
[?25hRequirement already satisfied: six in /usr/local/lib/python3.6/dist-packages (from fire) (1.15.0)
Requirement already satisfied: termcolor in /usr/local/lib/python3.6/dist-packages (from fire) (1.1.0)
Building wheels for collected packages: fire
  Building wheel for fire (setup.py) ... [...]<jupyter_text>Models available*   1558M*   774M*   355M*   124M<jupyter_code>#model parameters
MODEL_NAME='355M'

#seed none for ramdom, set for repeatability
SEED=None

#number of samples to return and batch size batch must be divisible by nsamples
NSAMPLES=3
BATCH_SIZE=1

#length of returned text, None default
LENGTH=1024

#temperature is randomness 0 => repetative, higher more random, default 1
TEMPERATURE=1

#top k is numberof words considered in each step, 0 is special case for no restricion
TOP_K=40

TOP_P=1
MODELS_DIR='models'
!python gpt-2/download_model.py {MODEL_NAME}<jupyter_output>Fetching checkpoint:   0%|                                              | 0.00/77.0 [00:00<?, ?it/s]
Fetching checkpoint: 1.00kit [00:00, 1.24Mit/s]                                                     

Fetching encoder.json:   0%|                                           | 0.00/1.04M [00:00<?, ?it/s]
Fetching encoder.json: 1.04Mit [00:00, 65.0Mit/s]                                                   
Fetching hparams.json: 1.00kit [00:00, 1.27Mit/s]                                                   
Fetching model.ckpt.data-00000-of-00001: 1.42Git [00:17, 82.9Mit/s]                                 
Fetching model.ckpt.index: 11.0kit [00:00, 9.42Mit/s]                                               
Fetching model.ckpt.meta: 927kit [00:00, 52.9Mit/s]                                                 
Fetching vocab.bpe: 457kit [00:00, 50.9Mit/s]<jupyter_text>To run with input<jupyter_code>!python gpt-2/src/interactive_conditional_samples.py {MODEL_NAME} {SEED} {NSAMPLES} {BATCH_SIZE} \
  {LENGTH} {TEMPERATURE} {TOP_K} {TOP_P} {MODELS_DIR}<jupyter_output>2020-11-24 15:59:59.915568: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library libcudart.so.10.1
Traceback (most recent call last):
  File "gpt-2/src/interactive_conditional_samples.py", line 9, in <module>
    import model, sample, encoder
  File "/content/gpt-2/src/model.py", line 3, in <module>
    from tensorflow.contrib.training import HParams
ModuleNotFoundError: No module named 'tensorflow.contrib'<jupyter_text>To run without input<jupyter_code>!python gpt-2/src/generate_unconditional_samples.py {MODEL_NAME} {SEED} {NSAMPLES} {BATCH_SIZE} \
  {LENGTH} {TEMPERATURE} {TOP_K} {TOP_P} {MODELS_DIR}<jupyter_output>WARNING:tensorflow:From gpt-2/src/generate_unconditional_samples.py:54: The name tf.Session is deprecated. Please use tf.compat.v1.Session instead.

2020-11-24 16:18:07.303124: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcuda.so.1
2020-11-24 16:18:07.335946: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:983] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2020-11-24 16:18:07.336515: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1618] Found device 0 with properties: 
name: Tesla T4 major: 7 minor: 5 memoryClockRate(GHz): 1.59
pciBusID: 0000:00:04.0
2020-11-24 16:18:07.336775: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.0
2020-11-24 16:18:07.337761: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10[...]
{'fraction_non_alphanumeric': 0.1427637314734089, 'fraction_numerical': 0.0963382737576286, 'mean_word_length': 3.075488454706927, 'pattern_counts': {'":': 0, '<': 17, '<?xml version=': 0, '>': 16, 'https://': 2, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 16, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '4', 'dir': 'jupyter-structured-clean-dedup', 'id': '386280', 'license': 'MIT', 'n_tokens_mistral': 2020, 'n_tokens_neox': 1652, 'n_words': 357, 'path': 'gpt_2.ipynb', 'repo_name': 'amarten1/gpt-2', 'size': '4588'}