<jupyter_start><jupyter_text>**Paraphrase text using transformers**Data Professor, http://youtube.com/dataprofessorNotes and References:PEGASUS is an acronym for Pre-training with Extracted Gap-sentences for Abstractive SUmmarization Sequence-to-sequence modelsPEGASUS model used herein is from Huggingface's transformers libraryPEGASUS model from Google ResearchRead the original paper PEGASUS: Pre-training with Extracted Gap-sentences for Abstractive Summarization<jupyter_code>! pip install sentence-splitter
! pip install transformers
! pip install SentencePiece
# https://huggingface.co/tuner007/pegasus_paraphrase

import torch
from transformers import PegasusForConditionalGeneration, PegasusTokenizer

model_name = 'tuner007/pegasus_paraphrase'
torch_device = 'cuda' if torch.cuda.is_available() else 'cpu'
tokenizer = PegasusTokenizer.from_pretrained(model_name)
model = PegasusForConditionalGeneration.from_pretrained(model_name).to(torch_device)

def get_response(input_text,num_return_sequences):
  batch = tokenizer.prepare_seq2seq_batch([input_text],truncation=True,padding='longest',max_length=60, return_tensors="pt").to(torch_device)
  translated = model.generate(**batch,max_length=60,num_beams=10, num_return_sequences=num_return_sequences, temperature=1.5)
  tgt_text = tokenizer.batch_decode(translated, skip_special_tokens=True)
  return tgt_text
text = "Where can I get most delicious pizza in Tempe?"
get_response(text, 5)
text = "What is the best way to learn data science?"
get_response(text, 10)
# Paragraph of text
context = "In this task, you will be shown a Persian passage and a question, and you have to determine whether the question is answerable based on the passage or not. If the question is answerable, choose the \"True\" label, and if not select \"False\""
print(context)
# Takes the input paragraph and splits it into a list of sentences
from sentence_splitter import SentenceSplitter, split_text_into_sentences

splitter = SentenceSplitter(language='en')

sentence_list = splitter.split(context)
sentence_list
# Do a for loop to iterate through the list of sentences and paraphrase each sentence in the iteration
paraphrase = []

for i in sentence_list:
  a = get_response(i,10)
  paraphrase.append(a)
# This is the paraphrased text
paraphrase
paraphrase2 = [' '.join(x) for x in paraphrase]
paraphrase2
# Combines the above list into a paragraph
paraphrase3 = [' '.join(x for x in paraphrase2) ]
paraphrased_text = str(paraphrase3).strip('[]').strip("'")
paraphrased_text
# Comparison of the original (context variable) and the paraphrased version (paraphrase3 variable)

print(context)
print(paraphrased_text)<jupyter_output>In this video, I will be showing you how to build a stock price web application in Python using the Streamlit and yfinance library. The app will be able to retrieve company information as well as the stock price data for S and P 500 companies. All of this in less than 50 lines of code.
In this video, I will show you how to use the Streamlit and yfinance libraries to build a stock price web application. The stock price data for S and P 500 companies will be retrieved by the app. This is in less than 50 lines of code.
{'fraction_non_alphanumeric': 0.06800376057662175, 'fraction_numerical': 0.011281729865246004, 'mean_word_length': 6.1730337078651685, 'pattern_counts': {'":': 0, '<': 4, '<?xml version=': 0, '>': 4, 'https://': 1, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '1', 'dir': 'jupyter-structured-clean-dedup', 'id': '652247', 'license': 'Apache-2.0', 'n_tokens_mistral': 988, 'n_tokens_neox': 915, 'n_words': 352, 'path': 'dataset3_paraphrase_kiran.ipynb', 'repo_name': 'mouryateja95/CSE576HW', 'size': '3191'}