Edit model card

flan-t5-base-finetuned-govReport-3072

This model is a fine-tuned version of google/flan-t5-base on the gov_report_summarization_dataset dataset. It achieves the following results on the evaluation set:

  • Loss: nan
  • Rouge1: 0.042
  • Rouge2: 0.0216
  • Rougel: 0.0379
  • Rougelsum: 0.0406

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 3e-05
  • train_batch_size: 4
  • eval_batch_size: 4
  • seed: 42
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: linear
  • num_epochs: 10
  • mixed_precision_training: Native AMP

Training results

Training Loss Epoch Step Validation Loss Rouge1 Rouge2 Rougel Rougelsum
0.0 1.0 250 nan 0.042 0.0216 0.0379 0.0406
0.0 2.0 500 nan 0.042 0.0216 0.0379 0.0406
0.0 3.0 750 nan 0.042 0.0216 0.0379 0.0406
0.0 4.0 1000 nan 0.042 0.0216 0.0379 0.0406
0.0 5.0 1250 nan 0.042 0.0216 0.0379 0.0406
0.0 6.0 1500 nan 0.042 0.0216 0.0379 0.0406
0.0 7.0 1750 nan 0.042 0.0216 0.0379 0.0406
0.0 8.0 2000 nan 0.042 0.0216 0.0379 0.0406
0.0 9.0 2250 nan 0.042 0.0216 0.0379 0.0406
0.0 10.0 2500 nan 0.042 0.0216 0.0379 0.0406

Framework versions

  • PEFT 0.8.2
  • Transformers 4.37.0
  • Pytorch 2.1.2
  • Datasets 2.1.0
  • Tokenizers 0.15.1
Downloads last month
8
Inference Examples
Inference API (serverless) does not yet support peft models for this pipeline type.

Model tree for RMWeerasinghe/flan-t5-base-finetuned-govReport-3072

Adapter
(127)
this model