region:us
This dataset repository contains the [TFRecord](https://www.tensorflow.org/tutorials/load_data/tfrecord) shards of the [WikText (v1) dataset](https://huggingface.co/datasets/wikitext.

We used [this script](https://github.com/huggingface/transformers/blob/main/examples/tensorflow/language-modeling-tpu/prepare_tfrecord_shards.py) to prepare these TFRecord shards.

For more details on how these TFRecord shards should be used, refer to the following tutorial: [Training a masked language model end-to-end from scratch on TPUs
](https://github.com/huggingface/transformers/tree/main/examples/tensorflow/language-modeling-tpu).
