README.md

metadata

language:
  - nl
datasets:
  - yhavinga/mc4_nl_cleaned
tags:
  - seq2seq
  - lm-head
license: apache-2.0
inference: false

Work in progress. Jan 2022

This model is a re-training of the original t5-base-dutch model that was trained during the summer 2021 HuggingFace Flax/Jax community week. These two models are now the same. Eval accuracy 0.70 and loss 1,38.

NB! Consider using yhavinga/t5-v1.1-base-dutch-cased that has an eval accuracy of 0,78 and loss 0,96.

These models need to be finetuned, therefore the inference widget on the right has been turned off.

A collection of Dutch T5 models

• Many thanks to the Google TPU Research Cloud for providing access to a TPU cluster!
• Continuation of work started during the Hugging Face community week, organized by HuggingFace and TPU usage sponsored by Google, for the project Pre-train T5 from scratch in Dutch.
• Using improved training script - no more exceptions during training, so no restarting required.
• All models trained with tensorboard metrics.
• Thanks to @gsarti for creating the t5-flax-gcp repository!