task_categories:automatic-speech-recognition language:en license:cc0-1.0 region:us
A large-scale multilingual speech corpus for representation learning, semi-supervised learning and interpretation.
@inproceedings{wang-etal-2021-voxpopuli,
    title = "{V}ox{P}opuli: A Large-Scale Multilingual Speech Corpus for Representation Learning, 
    Semi-Supervised Learning and Interpretation",
    author = "Wang, Changhan  and
      Riviere, Morgane  and
      Lee, Ann  and
      Wu, Anne  and
      Talnikar, Chaitanya  and
      Haziza, Daniel  and
      Williamson, Mary  and
      Pino, Juan  and
      Dupoux, Emmanuel",
    booktitle = "Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics 
    and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)",
    month = aug,
    year = "2021",
    publisher = "Association for Computational Linguistics",
    url = "https://aclanthology.org/2021.acl-long.80",
    doi = "10.18653/v1/2021.acl-long.80",
    pages = "993--1003",
}
---
license: cc0-1.0
task_categories:
- automatic-speech-recognition
language:
- en
-pretty_name: VoxPopuli
---

# Distil Whisper: VoxPopuli With Timestamps

This is a variant of the [VoxPopuli](https://huggingface.co/datasets/facebook/voxpopuli) dataset, augmented to return the pseudo-labelled Whisper 
Transcriptions alongside the original dataset elements. The pseudo-labelled transcriptions were generated by 
labelling the input audio data with the Whisper [large-v2](https://huggingface.co/openai/whisper-large-v2)
model with *greedy* sampling and timestamp prediction. For information on how the original dataset was curated, refer to the original 
[dataset card](https://huggingface.co/datasets/facebook/voxpopuli).

## Standalone Usage

First, install the latest version of the 🤗 Datasets package:

```bash
pip install --upgrade pip
pip install --upgrade datasets[audio]
```

The dataset can be downloaded and pre-processed on disk using the [`load_dataset`](https://huggingface.co/docs/datasets/v2.14.5/en/package_reference/loading_methods#datasets.load_dataset) 
function:

```python
from datasets import load_dataset

dataset = load_dataset("distil-whisper/voxpopuli", "en")
# take the first sample of the validation set
sample = dataset["validation"][0]
```

It can also be streamed directly from the Hub using Datasets' [streaming mode](https://huggingface.co/blog/audio-datasets#streaming-mode-the-silver-bullet).
Loading a dataset in streaming mode loads individual samples of the dataset at a time, rather than downloading the entire 
dataset to disk:

```python
from datasets import load_dataset

dataset = load_dataset("distil-whisper/voxpopuli", "en", streaming=True)
# take the first sample of the validation set
sample = next(iter(dataset["validation"]))
```

## Distil Whisper Usage

To use this dataset to reproduce a Distil Whisper training run, refer to the instructions on the 
[Distil Whisper repository](https://github.com/huggingface/distil-whisper#training).

## License

This dataset is licensed under cc0-1.0.

