Model: Hakulani/t5-end2end-questions-generation
Ruuning on RTX3090 

Description: A T5-based model for end-to-end question generation, trained on a large corpus of text data.

Intended Use: This model is intended for generating questions based on a given input text. It can be used for various natural language processing tasks, such as text summarization, translation, and information retrieval.

Input: The model takes a text input in the form of a string.

Output: The model generates a list of questions related to the input text.

Training Data: The model was trained on a large corpus of text data from various sources, including news articles, web pages, and academic papers.

we're going to learn to fine-tune the 🤗 t5 model to generate questions without providing answers and use Weight and Biases for measurements and logs.

Dataset 🛢️
As dataset we use SQuAD v1.1: Stanford Question Answering Dataset is a reading comprehension dataset, consisting of questions posed by crowdworkers on a set of Wikipedia articles, where the answer to every question is a segment of text, or span, from the corresponding reading passage, or the question might be unanswerable.

Requirements
This is not an introduction to Hugging Face Transformer library, it's a hands-on on how to fine tune t5 for this specific task.
If you're not familiar with Hugging Face, you can watch the HF Course on Transformer models (it's free) here
🏗️ This notebook is a work in progress, some elements (check todo at the end) will change.
Sources 📚
Transformer-based End-to-End Question Generation's Paper
Patil Suraj's work on question generation


Parameters:
_name_or_path: t5-base
architectures: T5ForConditionalGeneration
d_ff: 3072
d_kv: 64
d_model: 768
decoder_start_token_id: 0
dense_act_fn: relu
dropout_rate: 0.1
eos_token_id: 1
feed_forward_proj: relu
initializer_factor: 1.0
is_encoder_decoder: true
is_gated_act: false
layer_norm_epsilon: 1e-06
model_type: t5
n_positions: 512
num_decoder_layers: 12
num_heads: 12
num_layers: 12
output_past: true
pad_token_id: 0
relative_attention_max_distance: 128
relative_attention_num_buckets: 32
task_specific_params: {"summarization": {"early_stopping": true, "length_penalty": 2.0, "max_length": 200, "min_length": 30, "no_repeat_ngram_size": 3, "num_beams": 4, "prefix": "summarize: "}, "translation_en_to_de": {"early_stopping": true, "max_length": 300, "num_beams": 4, "prefix": "translate English to German: "}, "translation_en_to_fr": {"early_stopping": true, "max_length": 300, "num_beams": 4, "prefix": "translate English to French: "}, "translation_en_to_ro": {"early_stopping": true, "max_length": 300, "num_beams": 4, "prefix": "translate English to Romanian: "}}
torch_dtype: float32
transformers_version: 4.24.0
use_cache: true
vocab_size: 32101

Hyperparameters:

early_stopping: true
length_penalty: 2.0
max_length: 200
min_length: 30
no_repeat_ngram_size: 3
num_beams: 4
prefix: "summarize: " for summarization, "translate English to German: " for translation_en_to_de, "translate English to French: " for translation_en_to_fr, "translate English to Romanian: " for translation_en_to_ro


text = "Forrest Gump is a 1994 American comedy-drama film directed by Robert Zemeckis and written by Eric Roth. \
It is based on the 1986 novel of the same name by Winston Groom and stars Tom Hanks, Robin Wright, Gary Sinise, \
Mykelti Williamson and Sally Field. The story depicts several decades in the life of Forrest Gump (Hanks), \
a slow-witted but kind-hearted man from Alabama who witnesses and unwittingly influences several defining \
historical events in the 20th century United States. The film differs substantially from the novel."
```

checkpoint = "t5-base"
model = T5ForConditionalGeneration.from_pretrained(checkpoint)
tokenizer = T5TokenizerFast.from_pretrained(checkpoint)
tokenizer.sep_token = '<sep>'
tokenizer.add_tokens(['<sep>'])
model.resize_token_embeddings(len(tokenizer))
# Check the sep_token_id to verify that it was added to the tokenizer
tokenizer.sep_token_id
from transformers import T5ForConditionalGeneration, T5TokenizerFast

hfmodel = T5ForConditionalGeneration.from_pretrained("Hakulani/t5-end2end-questions-generation")
def hf_run_model(input_string, **generator_args):
  generator_args = {
  "max_length": 256,
  "num_beams": 4,
  "length_penalty": 1.5,
  "no_repeat_ngram_size": 3,
  "early_stopping": True,
  }
  input_string = "generate questions: " + input_string + " </s>"
  input_ids = tokenizer.encode(input_string, return_tensors="pt")
  res = hfmodel.generate(input_ids, **generator_args)
  output = tokenizer.batch_decode(res, skip_special_tokens=True)
  output = [item.split("<sep>") for item in output]
  return output


text = "Forrest Gump is a 1994 American comedy-drama film directed by Robert Zemeckis and written by Eric Roth. \
It is based on the 1986 novel of the same name by Winston Groom and stars Tom Hanks, Robin Wright, Gary Sinise, \
Mykelti Williamson and Sally Field. The story depicts several decades in the life of Forrest Gump (Hanks), \
a slow-witted but kind-hearted man from Alabama who witnesses and unwittingly influences several defining \
historical events in the 20th century United States. The film differs substantially from the novel."

hf_run_model(text)

[['Who directed the 1994 film Forrest Gump?',
  ' Who wrote the film for the film?',
  ' What is the film based on?',
  ' Which actor stars in the movie?',
  ' In what year was the novel of the same name written?',
  '']]
```
{"language": ["en"], "datasets": ["tgsc/squad-pt-v1.1"]}
transformers pytorch t5 text2text-generation en dataset:tgsc/squad-pt-v1.1 autotrain_compatible endpoints_compatible text-generation-inference region:us
TAGS
#transformers #pytorch #t5 #text2text-generation #en #dataset-tgsc/squad-pt-v1.1 #autotrain_compatible #endpoints_compatible #text-generation-inference #region-us 

Model: Hakulani/t5-end2end-questions-generation
Ruuning on RTX3090 

Description: A T5-based model for end-to-end question generation, trained on a large corpus of text data.

Intended Use: This model is intended for generating questions based on a given input text. It can be used for various natural language processing tasks, such as text summarization, translation, and information retrieval.

Input: The model takes a text input in the form of a string.

Output: The model generates a list of questions related to the input text.

Training Data: The model was trained on a large corpus of text data from various sources, including news articles, web pages, and academic papers.

we're going to learn to fine-tune the  t5 model to generate questions without providing answers and use Weight and Biases for measurements and logs.

Dataset ️
As dataset we use SQuAD v1.1: Stanford Question Answering Dataset is a reading comprehension dataset, consisting of questions posed by crowdworkers on a set of Wikipedia articles, where the answer to every question is a segment of text, or span, from the corresponding reading passage, or the question might be unanswerable.

Requirements
This is not an introduction to Hugging Face Transformer library, it's a hands-on on how to fine tune t5 for this specific task.
If you're not familiar with Hugging Face, you can watch the HF Course on Transformer models (it's free) here
️ This notebook is a work in progress, some elements (check todo at the end) will change.
Sources 
Transformer-based End-to-End Question Generation's Paper
Patil Suraj's work on question generation


Parameters:
_name_or_path: t5-base
architectures: T5ForConditionalGeneration
d_ff: 3072
d_kv: 64
d_model: 768
decoder_start_token_id: 0
dense_act_fn: relu
dropout_rate: 0.1
eos_token_id: 1
feed_forward_proj: relu
initializer_factor: 1.0
is_encoder_decoder: true
is_gated_act: false
layer_norm_epsilon: 1e-06
model_type: t5
n_positions: 512
num_decoder_layers: 12
num_heads: 12
num_layers: 12
output_past: true
pad_token_id: 0
relative_attention_max_distance: 128
relative_attention_num_buckets: 32
task_specific_params: {"summarization": {"early_stopping": true, "length_penalty": 2.0, "max_length": 200, "min_length": 30, "no_repeat_ngram_size": 3, "num_beams": 4, "prefix": "summarize: "}, "translation_en_to_de": {"early_stopping": true, "max_length": 300, "num_beams": 4, "prefix": "translate English to German: "}, "translation_en_to_fr": {"early_stopping": true, "max_length": 300, "num_beams": 4, "prefix": "translate English to French: "}, "translation_en_to_ro": {"early_stopping": true, "max_length": 300, "num_beams": 4, "prefix": "translate English to Romanian: "}}
torch_dtype: float32
transformers_version: 4.24.0
use_cache: true
vocab_size: 32101

Hyperparameters:

early_stopping: true
length_penalty: 2.0
max_length: 200
min_length: 30
no_repeat_ngram_size: 3
num_beams: 4
prefix: "summarize: " for summarization, "translate English to German: " for translation_en_to_de, "translate English to French: " for translation_en_to_fr, "translate English to Romanian: " for translation_en_to_ro


text = "Forrest Gump is a 1994 American comedy-drama film directed by Robert Zemeckis and written by Eric Roth. \
It is based on the 1986 novel of the same name by Winston Groom and stars Tom Hanks, Robin Wright, Gary Sinise, \
Mykelti Williamson and Sally Field. The story depicts several decades in the life of Forrest Gump (Hanks), \
a slow-witted but kind-hearted man from Alabama who witnesses and unwittingly influences several defining \
historical events in the 20th century United States. The film differs substantially from the novel."

TAGS
#transformers #pytorch #t5 #text2text-generation #en #dataset-tgsc/squad-pt-v1.1 #autotrain_compatible #endpoints_compatible #text-generation-inference #region-us 
