---
license: apache-2.0
base_model: t5-base
tags:
- generated_from_trainer
datasets:
- glue
metrics:
- accuracy
model-index:
- name: t5-base_sst2_dense
  results:
  - task:
      name: Text Classification
      type: text-classification
    dataset:
      name: glue
      type: glue
      config: sst2
      split: validation
      args: sst2
    metrics:
    - name: Accuracy
      type: accuracy
      value: 0.9231651376146789
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# t5-base_sst2_dense

This model is a fine-tuned version of [t5-base](https://huggingface.co/t5-base) on the glue dataset.
It achieves the following results on the evaluation set:
- Loss: 0.2156
- Accuracy: 0.9232

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 5e-05
- train_batch_size: 32
- eval_batch_size: 64
- seed: 42
- gradient_accumulation_steps: 2
- total_train_batch_size: 64
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_steps: 200
- num_epochs: 5

### Training results

| Training Loss | Epoch | Step | Validation Loss | Accuracy |
|:-------------:|:-----:|:----:|:---------------:|:--------:|
| 0.6905        | 0.01  | 10   | 0.7366          | 0.5080   |
| 0.684         | 0.02  | 20   | 0.7306          | 0.5069   |
| 0.7013        | 0.03  | 30   | 0.7228          | 0.5080   |
| 0.6954        | 0.04  | 40   | 0.7114          | 0.5046   |
| 0.6893        | 0.05  | 50   | 0.7026          | 0.5034   |
| 0.6888        | 0.06  | 60   | 0.6912          | 0.5023   |
| 0.6814        | 0.07  | 70   | 0.6848          | 0.5034   |
| 0.679         | 0.08  | 80   | 0.6745          | 0.5206   |
| 0.6616        | 0.09  | 90   | 0.6685          | 0.5252   |
| 0.6604        | 0.1   | 100  | 0.6580          | 0.5378   |
| 0.6524        | 0.1   | 110  | 0.6378          | 0.6525   |
| 0.6344        | 0.11  | 120  | 0.6128          | 0.7271   |
| 0.5915        | 0.12  | 130  | 0.5672          | 0.8016   |
| 0.562         | 0.13  | 140  | 0.4903          | 0.8578   |
| 0.4653        | 0.14  | 150  | 0.3825          | 0.8796   |
| 0.3632        | 0.15  | 160  | 0.2811          | 0.8991   |
| 0.2754        | 0.16  | 170  | 0.3029          | 0.8933   |
| 0.2298        | 0.17  | 180  | 0.3001          | 0.8991   |
| 0.2819        | 0.18  | 190  | 0.2636          | 0.9083   |
| 0.2532        | 0.19  | 200  | 0.2321          | 0.9128   |
| 0.2512        | 0.2   | 210  | 0.2286          | 0.9186   |
| 0.2149        | 0.21  | 220  | 0.2424          | 0.9128   |
| 0.2466        | 0.22  | 230  | 0.2505          | 0.9140   |
| 0.1853        | 0.23  | 240  | 0.2178          | 0.9186   |
| 0.2279        | 0.24  | 250  | 0.2152          | 0.9186   |
| 0.219         | 0.25  | 260  | 0.2188          | 0.9197   |
| 0.2144        | 0.26  | 270  | 0.2179          | 0.9209   |
| 0.1507        | 0.27  | 280  | 0.2185          | 0.9186   |
| 0.1801        | 0.28  | 290  | 0.2473          | 0.9243   |
| 0.1735        | 0.29  | 300  | 0.2402          | 0.9128   |
| 0.1437        | 0.29  | 310  | 0.2436          | 0.9255   |
| 0.2221        | 0.3   | 320  | 0.2209          | 0.9163   |
| 0.1611        | 0.31  | 330  | 0.2101          | 0.9232   |
| 0.1813        | 0.32  | 340  | 0.2291          | 0.9174   |
| 0.1871        | 0.33  | 350  | 0.2386          | 0.9174   |
| 0.2126        | 0.34  | 360  | 0.2225          | 0.9197   |
| 0.2023        | 0.35  | 370  | 0.2116          | 0.9232   |
| 0.127         | 0.36  | 380  | 0.2155          | 0.9232   |
| 0.2769        | 0.37  | 390  | 0.2149          | 0.9243   |
| 0.1457        | 0.38  | 400  | 0.2166          | 0.9232   |
| 0.2129        | 0.39  | 410  | 0.2271          | 0.9232   |
| 0.1652        | 0.4   | 420  | 0.2308          | 0.9220   |
| 0.1783        | 0.41  | 430  | 0.2400          | 0.9278   |
| 0.1305        | 0.42  | 440  | 0.2404          | 0.9232   |
| 0.2595        | 0.43  | 450  | 0.2389          | 0.9209   |
| 0.1901        | 0.44  | 460  | 0.2102          | 0.9266   |
| 0.1993        | 0.45  | 470  | 0.2129          | 0.9255   |
| 0.147         | 0.46  | 480  | 0.2208          | 0.9232   |
| 0.1801        | 0.47  | 490  | 0.2143          | 0.9255   |
| 0.1716        | 0.48  | 500  | 0.2416          | 0.9209   |
| 0.1281        | 0.48  | 510  | 0.2152          | 0.9232   |
| 0.1837        | 0.49  | 520  | 0.2112          | 0.9243   |
| 0.1681        | 0.5   | 530  | 0.2178          | 0.9232   |
| 0.1408        | 0.51  | 540  | 0.2127          | 0.9243   |
| 0.1229        | 0.52  | 550  | 0.3322          | 0.9278   |
| 0.1304        | 0.53  | 560  | 0.3586          | 0.9209   |
| 0.1905        | 0.54  | 570  | 0.3354          | 0.9243   |
| 0.147         | 0.55  | 580  | 0.3431          | 0.9278   |
| 0.1538        | 0.56  | 590  | 0.3444          | 0.9232   |
| 0.1504        | 0.57  | 600  | 0.2196          | 0.9266   |
| 0.1628        | 0.58  | 610  | 0.3452          | 0.9163   |
| 0.1387        | 0.59  | 620  | 0.3282          | 0.9278   |
| 0.2104        | 0.6   | 630  | 0.2132          | 0.9243   |
| 0.1482        | 0.61  | 640  | 0.2154          | 0.9243   |
| 0.217         | 0.62  | 650  | 0.3472          | 0.9197   |
| 0.1692        | 0.63  | 660  | 0.2063          | 0.9243   |
| 0.175         | 0.64  | 670  | 0.2019          | 0.9278   |
| 0.1473        | 0.65  | 680  | 0.1957          | 0.9266   |
| 0.1154        | 0.66  | 690  | 0.2020          | 0.9255   |
| 0.1369        | 0.67  | 700  | 0.2087          | 0.9266   |
| 0.1262        | 0.67  | 710  | 0.3224          | 0.9289   |
| 0.2111        | 0.68  | 720  | 0.3325          | 0.9243   |
| 0.1349        | 0.69  | 730  | 0.3285          | 0.9289   |
| 0.1814        | 0.7   | 740  | 0.3324          | 0.9266   |
| 0.1217        | 0.71  | 750  | 0.3212          | 0.9243   |
| 0.173         | 0.72  | 760  | 0.2176          | 0.9220   |
| 0.1441        | 0.73  | 770  | 0.2130          | 0.9232   |
| 0.1706        | 0.74  | 780  | 0.2136          | 0.9220   |
| 0.1411        | 0.75  | 790  | 0.2101          | 0.9220   |
| 0.1051        | 0.76  | 800  | 0.2078          | 0.9243   |
| 0.115         | 0.77  | 810  | 0.2160          | 0.9266   |
| 0.2031        | 0.78  | 820  | 0.2162          | 0.9209   |
| 0.12          | 0.79  | 830  | 0.2059          | 0.9255   |
| 0.176         | 0.8   | 840  | 0.2100          | 0.9255   |
| 0.1306        | 0.81  | 850  | 0.4307          | 0.9243   |
| 0.1359        | 0.82  | 860  | 0.4397          | 0.9289   |
| 0.1921        | 0.83  | 870  | 0.5446          | 0.9278   |
| 0.1772        | 0.84  | 880  | 0.5423          | 0.9266   |
| 0.1771        | 0.85  | 890  | 0.4273          | 0.9266   |
| 0.1965        | 0.86  | 900  | 0.3224          | 0.9243   |
| 0.1227        | 0.86  | 910  | 0.2131          | 0.9278   |
| 0.2046        | 0.87  | 920  | 0.3130          | 0.9278   |
| 0.1061        | 0.88  | 930  | 0.3180          | 0.9289   |
| 0.1364        | 0.89  | 940  | 0.5501          | 0.9186   |
| 0.1213        | 0.9   | 950  | 0.4400          | 0.9220   |
| 0.1611        | 0.91  | 960  | 0.4364          | 0.9255   |
| 0.1632        | 0.92  | 970  | 0.4475          | 0.9220   |
| 0.1617        | 0.93  | 980  | 0.5758          | 0.9209   |
| 0.1478        | 0.94  | 990  | 0.2143          | 0.9220   |
| 0.1314        | 0.95  | 1000 | 0.2156          | 0.9232   |
| 0.1814        | 0.96  | 1010 | 0.2191          | 0.9220   |
| 0.1669        | 0.97  | 1020 | 0.2129          | 0.9243   |
| 0.1206        | 0.98  | 1030 | 0.2119          | 0.9220   |
| 0.1852        | 0.99  | 1040 | 0.2104          | 0.9209   |
| 0.1381        | 1.0   | 1050 | 0.1999          | 0.9255   |
| 0.135         | 1.01  | 1060 | 0.2090          | 0.9243   |
| 0.1253        | 1.02  | 1070 | 0.4486          | 0.9209   |
| 0.1244        | 1.03  | 1080 | 0.4319          | 0.9197   |
| 0.1772        | 1.04  | 1090 | 0.4248          | 0.9243   |
| 0.1264        | 1.05  | 1100 | 0.3090          | 0.9289   |
| 0.6928        | 1.05  | 1110 | 0.3174          | 0.9278   |
| 0.0908        | 1.06  | 1120 | 0.4359          | 0.9266   |
| 0.1286        | 1.07  | 1130 | 0.4302          | 0.9312   |
| 0.0953        | 1.08  | 1140 | 0.5397          | 0.9289   |
| 0.1091        | 1.09  | 1150 | 0.5455          | 0.9255   |
| 0.1546        | 1.1   | 1160 | 0.4261          | 0.9300   |


### Framework versions

- Transformers 4.34.0
- Pytorch 2.0.1+cu118
- Datasets 2.14.5
- Tokenizers 0.14.1