---
library_name: transformers
datasets:
- samsum
---

# Model Card for Model ID

A version of google/flan-t5-small, fine-tuned on the samsum dataset.



## Model Details

### Model Description

- **Developed by:** [guy-smiley](https://huggingface.co/guy-smiley)
- **Model type:** Language model
- **Language(s) (NLP):** English
- **Finetuned from model:** [flan-t5-small](https://huggingface.co/google/flan-t5-small)

## Uses

Chat and dialogue summarization

## Bias, Risks, and Limitations

"Language models, including Flan-T5, can potentially be used for language generation in a harmful way, according to Rae et al. (2021). Flan-T5 should not be used directly in any application, without a prior assessment of safety and fairness concerns specific to the application."

## Training Details

### Training Data

[samsum](https://huggingface.co/datasets/samsum)

### Training Procedure 

* Trained with [Seq2Se2Trainer](https://huggingface.co/docs/transformers/v4.38.2/en/main_classes/trainer#transformers.Seq2SeqTrainer)
* GitHub repo:  

#### Training Hyperparameters

* learning_rate: 0.00005
* train_batch_size: 8
* eval_batch_size: 8
* seed: 42
* optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
* lr_scheduler_type: linear
* num_epochs: 1

#### Training Results
* epoch: 1
    * train_loss: 1.83195
    * eval
        * eval_loss: 1.67304
        * eval_rouge1: 42.8081
        * eval_rouge2: 18.6456
        * eval_rougeL: 35.4345
        * eval_rougeLsum: 39.1534