task_categories:automatic-speech-recognition annotations_creators:crowdsourced annotations_creators:expert-generated annotations_creators:other annotations_creators:machine-generated language_creators:crowdsourced language_creators:expert-generated language_creators:other multilinguality:monolingual size_categories:1K<n<10K source_datasets:original source_datasets:extended|librispeech_asr source_datasets:extended|common_voice language:pl license:cc-by-sa-4.0 benchmark polish asr speech doi:10.57967/hf/1068 region:us
BIGOS (Benchmark Intended Grouping of Open Speech) dataset goal is to simplify access to the openly available Polish speech corpora and
enable systematic benchmarking of open and commercial Polish ASR systems.
@InProceedings{huggingface:dataset,
title = {A great new dataset},
author={huggingface, Inc.
},
year={2020}
}
---
annotations_creators:
- crowdsourced
- expert-generated
- other
- machine-generated
language:
- pl
language_creators:
- crowdsourced
- expert-generated
- other
license:
- cc-by-sa-4.0
multilinguality:
- monolingual
pretty_name: pl-asr-bigos
size_categories:
- 1K<n<10K
source_datasets:
- original
- extended|librispeech_asr
- extended|common_voice
tags:
- benchmark
- polish
- asr
- speech
task_categories:
- automatic-speech-recognition
task_ids: []
extra_gated_prompt: |-
    Original datasets used for curation of BIGOS have specific terms of usage that must be understood and agreed to before use. Below are the links to the license terms and datasets the specific license type applies to:
    * [Creative Commons 0](https://creativecommons.org/share-your-work/public-domain/cc0) which applies to [Common Voice](https://huggingface.co/datasets/mozilla-foundation/common_voice_13_0)
    * [Creative Commons By Attribution Share Alike 4.0](https://creativecommons.org/licenses/by-sa/4.0/), which applies to [Clarin Cyfry](https://clarin-pl.eu/dspace/handle/11321/317), [Azon acoustic speech resources corpus](https://zasobynauki.pl/zasoby/korpus-nagran-probek-mowy-do-celow-budowy-modeli-akustycznych-dla-automatycznego-rozpoznawania-mowy,53293/).
    * [Creative Commons By Attribution 3.0](https://creativecommons.org/licenses/by/3.0/), which applies to [CLARIN Mobile database](https://clarin-pl.eu/dspace/handle/11321/237), [CLARIN Studio database](https://clarin-pl.eu/dspace/handle/11321/236), [PELCRA Spelling and Numbers Voice Database](http://pelcra.pl/new/snuv) and [FLEURS dataset](https://huggingface.co/datasets/google/fleurs)
    * [Creative Commons By Attribution 4.0](https://creativecommons.org/licenses/by/4.0/), which applies to [Multilingual Librispeech](https://huggingface.co/datasets/facebook/multilingual_librispeech) and [Poly AI Minds 14](https://huggingface.co/datasets/PolyAI/minds14)
    * [Proprietiary License of Munich AI Labs dataset](https://www.caito.de/2019/01/03/the-m-ailabs-speech-dataset)
    * Public domain mark, which applies to [PWR datasets](https://www.ii.pwr.edu.pl/~sas/ASR/)
    To use selected dataset, you also need to fill in the access forms on the specific datasets pages:
    * Common Voice: https://huggingface.co/datasets/mozilla-foundation/common_voice_13_0

extra_gated_fields:
 I hereby confirm that I have read and accepted the license terms of datasets comprising BIGOS corpora: checkbox
 I hereby confirm that I have registered on the original Common Voice page and agree to not attempt to determine the identity of speakers in the Common Voice dataset: checkbox
---

# Dataset Card for Polish ASR BIGOS corpora

## Table of Contents
- [Table of Contents](#table-of-contents)
- [Dataset Description](#dataset-description)
  - [Dataset Summary](#dataset-summary)
  - [Supported Tasks and Leaderboards](#supported-tasks-and-leaderboards)
  - [Languages](#languages)
- [Dataset Structure](#dataset-structure)
  - [Data Instances](#data-instances)
  - [Data Fields](#data-fields)
  - [Data Splits](#data-splits)
- [Dataset Creation](#dataset-creation)
  - [Curation Rationale](#curation-rationale)
  - [Source Data](#source-data)
  - [Annotations](#annotations)
  - [Personal and Sensitive Information](#personal-and-sensitive-information)
- [Considerations for Using the Data](#considerations-for-using-the-data)
  - [Social Impact of Dataset](#social-impact-of-dataset)
  - [Discussion of Biases](#discussion-of-biases)
  - [Other Known Limitations](#other-known-limitations)
- [Additional Information](#additional-information)
  - [Dataset Curators](#dataset-curators)
  - [Licensing Information](#licensing-information)
  - [Citation Information](#citation-information)
  - [Contributions](#contributions)

## Dataset Description

- **Homepage:https://huggingface.co/datasets/michaljunczyk/pl-asr-bigos**
- **Repository: https://github.com/goodmike31/pl-asr-bigos-tools**
- **Paper:https://www.researchgate.net/publication/370983845_BIGOS_-_Benchmark_Intended_Grouping_of_Open_Speech_Corpora_for_Polish_Automatic_Speech_Recognition**
- **Leaderboard:https://huggingface.co/spaces/michaljunczyk/pl-asr-bigos-benchmark/**
- **Point of Contact:michal.junczyk@amu.edu.pl**

### Dataset Summary

The BIGOS (Benchmark Intended Grouping of Open Speech) corpora aims at simplifying the access and use of publicly available ASR speech datasets for Polish.<br>
The initial release consist of test split with 1900 recordings and original transcriptions extracted from 10 publicly available datasets.

### Supported Tasks and Leaderboards
The leaderboard with benchmark of publicly available ASR systems supporting Polish is [under construction](https://huggingface.co/spaces/michaljunczyk/pl-asr-bigos-benchmark/).<br>
Evaluation results of 3 commercial and 5 freely available can be found in the [paper](https://www.researchgate.net/publication/370983845_BIGOS_-_Benchmark_Intended_Grouping_of_Open_Speech_Corpora_for_Polish_Automatic_Speech_Recognition).

### Languages
Polish

## Dataset Structure
Dataset consists audio recordings in WAV format and corresponding metadata.<br>
Audio and metadata can be used in raw format (TSV) or via hugging face datasets library.


### Data Instances
1900 audio files with original transcriptions are available in "test" split.<br>
This consitutes 1.6% of the total available transcribed speech in 10 source datasets considered in the initial release.

### Data Fields
Available fields:
* file_id - file identifier
* dataset_id - source dataset identifier
* audio - binary representation of audio file
* ref_original - original transcription of audio file
* hyp_whisper_cloud - ASR hypothesis (output) from Whisper Cloud system
* hyp_google_default - ASR hypothesis (output) from Google ASR system, default model
* hyp_azure_default - ASR hypothesis (output) from Azure ASR system, default model
* hyp_whisper_tiny - ASR hypothesis (output) from Whisper tiny model
* hyp_whisper_base - ASR hypothesis (output) from Whisper base model
* hyp_whisper_small - ASR hypothesis (output) from Whisper small model
* hyp_whisper_medium - ASR hypothesis (output) from Whisper medium model
* hyp_whisper_large - ASR hypothesis (output) from Whisper large (V2) model
<br><br>

Fields to be added in the next release:
* ref_spoken - manual transcription in a spoken format (without normalization)
* ref_written - manual transcription in a written format (with normalization)


### Data Splits
Initial release contains only "test" split.<br>
"Dev" and "train" splits will be added in the next release.

## Dataset Creation

### Curation Rationale
[Polish ASR Speech Data Catalog](https://github.com/goodmike31/pl-asr-speech-data-survey) was used to identify suitable datasets which can be repurposed and included in the BIGOS corpora.<br>
The following mandatory criteria were considered:
* Dataset must be downloadable.
* The license must allow for free, noncommercial use.
* Transcriptions must be available and align with the recordings.
* The sampling rate of audio recordings must be at least 8 kHz.
* Audio encoding using a minimum of 16 bits per sample.

### Source Data
10 datasets that meet the criteria were chosen as sources for the BIGOS dataset.
* The Common Voice dataset (mozilla-common-voice-19)
* The Multilingual LibriSpeech (MLS) dataset (fair-mls-20)
* The Clarin Studio Corpus (clarin-pjatk-studio-15)
* The Clarin Mobile Corpus (clarin-pjatk-mobile-15)
* The Jerzy Sas PWR datasets from Politechnika Wrocławska (pwr-viu-unk, pwr-shortwords-unk, pwr-maleset-unk). More info [here](https://www.ii.pwr.edu.pl/)
* The Munich-AI Labs Speech corpus (mailabs-19)
* The AZON Read and Spontaneous Speech Corpora (pwr-azon-spont-20, pwr-azon-read-20) More info [here](https://zasobynauki.pl/zasoby/korpus-nagran-probek-mowy-do-celow-budowy-modeli-akustycznych-dla-automatycznego-rozpoznawania-mowy)

#### Initial Data Collection and Normalization
Source text and audio files were extracted and encoded in a unified format.<br>
Dataset-specific transcription norms are preserved, including punctuation and casing. <br>
To strike a balance in the evaluation dataset and to facilitate the comparison of Word Error Rate (WER) scores across multiple datasets, 200 samples are randomly selected from each corpus. <br>
The only exception is ’pwr-azon-spont-20’, which contains significantly longer recordings and utterances, therefore only 100 samples are selected. <br>
#### Who are the source language producers?
1. Clarin corpora - Polish Japanese Academy of Technology
2. Common Voice - Mozilla foundation
3. Multlingual librispeech - Facebook AI research lab
4. Jerzy Sas and AZON datasets - Politechnika Wrocławska

Please refer to the [paper](https://www.researchgate.net/publication/370983845_BIGOS_-_Benchmark_Intended_Grouping_of_Open_Speech_Corpora_for_Polish_Automatic_Speech_Recognition) for more details.

### Annotations

#### Annotation process

Current release contains original transcriptions.
Manual transcriptions are planned for subsequent releases.

#### Who are the annotators?
Depends on the source dataset.

### Personal and Sensitive Information
This corpus does not contain PII or Sensitive Information.
All IDs pf speakers are anonymized.

## Considerations for Using the Data

### Social Impact of Dataset
To be updated.
### Discussion of Biases
To be updated.

### Other Known Limitations
The dataset in the initial release contains only a subset of recordings from original datasets.

## Additional Information

### Dataset Curators
Original authors of the source datasets - please refer to [source-data](#source-data) for details.

Michał Junczyk (michal.junczyk@amu.edu.pl) - curator of BIGOS corpora.

### Licensing Information
The BIGOS corpora is available under [Creative Commons By Attribution Share Alike 4.0 license.](https://creativecommons.org/licenses/by-sa/4.0/)

Original datasets used for curation of BIGOS have specific terms of usage that must be understood and agreed to before use. Below are the links to the license terms and datasets the specific license type applies to:
* [Creative Commons 0](https://creativecommons.org/share-your-work/public-domain/cc0) which applies to [Common Voice](https://huggingface.co/datasets/mozilla-foundation/common_voice_13_0)
* [Creative Commons By Attribution Share Alike 4.0](https://creativecommons.org/licenses/by-sa/4.0/), which applies to [Clarin Cyfry](https://clarin-pl.eu/dspace/handle/11321/317), [Azon acoustic speech resources corpus](https://zasobynauki.pl/zasoby/korpus-nagran-probek-mowy-do-celow-budowy-modeli-akustycznych-dla-automatycznego-rozpoznawania-mowy,53293/).
* [Creative Commons By Attribution 3.0](https://creativecommons.org/licenses/by/3.0/), which applies to [CLARIN Mobile database](https://clarin-pl.eu/dspace/handle/11321/237), [CLARIN Studio database](https://clarin-pl.eu/dspace/handle/11321/236), [PELCRA Spelling and Numbers Voice Database](http://pelcra.pl/new/snuv) and [FLEURS dataset](https://huggingface.co/datasets/google/fleurs)
* [Creative Commons By Attribution 4.0](https://creativecommons.org/licenses/by/4.0/), which applies to [Multilingual Librispeech](https://huggingface.co/datasets/facebook/multilingual_librispeech) and [Poly AI Minds 14](https://huggingface.co/datasets/PolyAI/minds14)
* [Proprietiary License of Munich AI Labs dataset](https://www.caito.de/2019/01/03/the-m-ailabs-speech-dataset)
* Public domain mark, which applies to [PWR datasets](https://www.ii.pwr.edu.pl/~sas/ASR/)

### Citation Information
TODO

### Contributions

Thanks to [@goodmike31](https://github.com/goodmike31) for adding this dataset.