<jupyter_start><jupyter_text>Author : <NAME> Email : <EMAIL> Portfolio : https://imsanjoykb.github.io/<jupyter_code>!pip install transformers<jupyter_output>Collecting transformers
  Downloading transformers-4.10.2-py3-none-any.whl (2.8 MB)
[K     |████████████████████████████████| 2.8 MB 5.3 MB/s 
[?25hRequirement already satisfied: regex!=2019.12.17 in /usr/local/lib/python3.7/dist-packages (from transformers) (2019.12.20)
Requirement already satisfied: numpy>=1.17 in /usr/local/lib/python3.7/dist-packages (from transformers) (1.19.5)
Requirement already satisfied: requests in /usr/local/lib/python3.7/dist-packages (from transformers) (2.23.0)
Collecting sacremoses
  Downloading sacremoses-0.0.45-py3-none-any.whl (895 kB)
[K     |████████████████████████████████| 895 kB 54.5 MB/s 
[?25hRequirement already satisfied: tqdm>=4.27 in /usr/local/lib/python3.7/dist-packages (from transformers) (4.62.0)
Requirement already satisfied: filelock in /usr/local/lib/python3.7/dist-packages (from transformers) (3.0.12)
Requirement already satisfied: importlib-metadata in /usr/local/lib/python3.7/dist-packages (from transformers) (4.6.4)
Collecting[...]<jupyter_text>Import Necessary Libraries<jupyter_code>import numpy as np
import pandas as pd
import seaborn as sns
import urllib
import re
import os
import tensorflow as tf
from transformers import GPT2LMHeadModel, GPT2Tokenizer<jupyter_output><empty_output><jupyter_text>Import Tokenizer<jupyter_code>## GPT-2 is a transformers model pretrained on a very large corpus of English data
## pretrained on the raw texts only, with no humans labelling them with autometic Process
## generating texts from a prompt

tokenizer = GPT2Tokenizer.from_pretrained("gpt2-large")<jupyter_output><empty_output><jupyter_text>Import Model<jupyter_code>## GPT-2 stack of transformer decoders which makes it so powerful in generating meaningful texts

model = GPT2LMHeadModel.from_pretrained("gpt2-large", pad_token_id=tokenizer.eos_token_id)<jupyter_output><empty_output><jupyter_text>Set The Topic<jupyter_code>## Let’s take an example test text like

sentence = '<NAME>'<jupyter_output><empty_output><jupyter_text>Encode the input i.e. topic<jupyter_code>## Tokenize this input text for GPT input i.e. converting text to integer indices

input_ids = tokenizer.encode(sentence, return_tensors='pt')

input_ids
## max_lenth-Number of Words in the Article
## num_beams-Number of different combination of words that can be chained together
## no_repeat_ngram_size-No of words that be combined together and repeated, example: ['benefits of sleeping' can be repeated 2 times but not more ]
## generate text until the output length (which includes the context length) reaches 50

output = model.generate(
    input_ids, 
    max_length=30, 
    num_beams=5, 
    no_repeat_ngram_size=2, 
    early_stopping=True
    )
## Tensor Output
output

## This is our tokenized text, if want to decode these indices values, can do so by decode method provided by the same tokenizer.
print(tokenizer.decode(output[0], skip_special_tokens=True))<jupyter_output><NAME> match-winning goal in the Champions League final against Atletico Madrid.

Ronaldo scored the winner in extra-time to<jupyter_text>Save the output in a variable<jupyter_code>text_x = tokenizer.decode(output[0], skip_special_tokens=True)
text = tokenizer.decode(output[0], skip_special_tokens=True).join(text_x.split(".")[:-1]) + "."
text<jupyter_output><empty_output><jupyter_text>Install & Import Gradio<jupyter_code>!pip install -q gradio
import gradio as gr
def generate_text(sentence):
    input_ids = tokenizer.encode(sentence, return_tensors='pt')
    beam_output = model.generate(input_ids, max_length=30, num_beams=5, no_repeat_ngram_size=2, early_stopping=True)
    output = tokenizer.decode(beam_output[0], skip_special_tokens=True, clean_up_tokenization_spaces=True)
    return ".".join(output.split(".")[:-1]) + "."<jupyter_output><empty_output><jupyter_text>Creating Gradio Interface<jupyter_code>output_text = gr.outputs.Textbox()
gr.Interface(generate_text,"textbox", output_text, title="Text Generation",
             description="Generate Text and Comment With AI System").launch()<jupyter_output>Colab notebook detected. To show errors in colab notebook, set `debug=True` in `launch()`
This share link will expire in 24 hours. If you need a permanent link, visit: https://gradio.app/introducing-hosted (NEW!)
Running on External URL: https://56632.gradio.app
Interface loading below...<jupyter_text>Save Output as CSV File<jupyter_code>from google.colab import files
with open('generatefile.csv','w') as f:
  f.write(text)<jupyter_output><empty_output><jupyter_text>Generate Pickle File<jupyter_code>import pickle
pickle.dump(model,open('nlp_model.pkl','wb'))
pickle.dump(text,open('projmodel.pkl','wb'))<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.12007427274602847, 'fraction_numerical': 0.023519702909015887, 'mean_word_length': 6.25748502994012, 'pattern_counts': {'":': 0, '<': 46, '<?xml version=': 0, '>': 48, 'https://': 3, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 2, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '11', 'dir': 'jupyter-structured-clean-dedup', 'id': '466863', 'license': 'MIT', 'n_tokens_mistral': 1713, 'n_tokens_neox': 1604, 'n_words': 407, 'path': 'Text_Generation.ipynb', 'repo_name': 'imsanjoykb/Text-Generation', 'size': '4897'}