metadata

license: apache-2.0
base_model: google/flan-t5-small
tags:
  - generated_from_trainer
model-index:
  - name: text-correction-en-small
    results: []

text-correction-en-small

This model is a fine-tuned version of google/flan-t5-small on the small portion of this dataset.

Model description

This model is fine-tuned on the task of text correction: it fixes English text with the wrong grammar, punctuation and order of words. Significantly larger model that supports Russian language as well can be found here.

Training and evaluation data

Around 10% of the data from this dataset was used.

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 5e-05
train_batch_size: 4
eval_batch_size: 4
seed: 42
gradient_accumulation_steps: 2
total_train_batch_size: 8
optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
lr_scheduler_type: linear
lr_scheduler_warmup_steps: 100
num_epochs: 1

Framework versions

Transformers 4.41.0
Pytorch 2.2.2
Datasets 2.19.1
Tokenizers 0.19.1