guy-smiley/flan-t5-small-samsum-3

Model Card for Model ID

A version of google/flan-t5-small, fine-tuned on the samsum dataset.

Model Details

Model Description

• Developed by: guy-smiley
• Model type: Language model
• Language(s) (NLP): English
• Finetuned from model: flan-t5-small

Uses

Chat and dialogue summarization

Bias, Risks, and Limitations

"Language models, including Flan-T5, can potentially be used for language generation in a harmful way, according to Rae et al. (2021). Flan-T5 should not be used directly in any application, without a prior assessment of safety and fairness concerns specific to the application."

Training Details

Training Data

samsum

Training Procedure

• Trained with Seq2Se2Trainer
• GitHub repo:

Training Hyperparameters

• learning_rate: 0.00005
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 1

Training Results

• epoch: 1
  • train_loss: 1.83195
  • eval
    • eval_loss: 1.67304
    • eval_rouge1: 42.8081
    • eval_rouge2: 18.6456
    • eval_rougeL: 35.4345
    • eval_rougeLsum: 39.1534