task_categories:automatic-speech-recognition annotations_creators:expert-generated language_creators:other multilinguality:monolingual size_categories:10K<n<100K source_datasets:original language:es license:cc-by-sa-4.0 ciempiess spanish mexican spanish ciempiess project ciempiess-unam project region:us
CIEMPIESS BALANCE is a Radio Corpus designed to create acoustic models for automatic speech recognition. It is "balance" because it is designed to balance the CIEMPIESS LIGHT, which means that if both corpora are combined, one will get a gender balanced corpus.
@misc{carlosmenaciempiessbalance2018,
      title={CIEMPIESS BALANCE CORPUS: Audio and Transcripts of Mexican Spanish Broadcast Conversations.}, 
      ldc_catalog_no={LDC2018S11},
      DOI={https://doi.org/10.35111/rfmw-n126},
      author={Hernandez Mena, Carlos Daniel},
      journal={Linguistic Data Consortium, Philadelphia},
      year={2018},
      url={https://catalog.ldc.upenn.edu/LDC2018S11},
}
---
annotations_creators:
- expert-generated
language:
- es
language_creators:
- other
license:
- cc-by-sa-4.0
multilinguality:
- monolingual
pretty_name: 'CIEMPIESS BALANCE CORPUS: Audio and Transcripts of Mexican Spanish Broadcast Conversations.'
size_categories:
- 10K<n<100K
source_datasets:
- original
tags:
- ciempiess
- spanish
- mexican spanish
- ciempiess project
- ciempiess-unam project
task_categories:
- automatic-speech-recognition
task_ids: []
---

# Dataset Card for ciempiess_balance
## Table of Contents
- [Dataset Description](#dataset-description)
  - [Dataset Summary](#dataset-summary)
  - [Supported Tasks](#supported-tasks-and-leaderboards)
  - [Languages](#languages)
- [Dataset Structure](#dataset-structure)
  - [Data Instances](#data-instances)
  - [Data Fields](#data-fields)
  - [Data Splits](#data-splits)
- [Dataset Creation](#dataset-creation)
  - [Curation Rationale](#curation-rationale)
  - [Source Data](#source-data)
  - [Annotations](#annotations)
  - [Personal and Sensitive Information](#personal-and-sensitive-information)
- [Considerations for Using the Data](#considerations-for-using-the-data)
  - [Social Impact of Dataset](#social-impact-of-dataset)
  - [Discussion of Biases](#discussion-of-biases)
  - [Other Known Limitations](#other-known-limitations)
- [Additional Information](#additional-information)
  - [Dataset Curators](#dataset-curators)
  - [Licensing Information](#licensing-information)
  - [Citation Information](#citation-information)
  - [Contributions](#contributions)
  
## Dataset Description
- **Homepage:** [CIEMPIESS-UNAM Project](https://ciempiess.org/)
- **Repository:** [CIEMPIESS BALANCE at LDC](https://catalog.ldc.upenn.edu/LDC2018S11)
- **Point of Contact:** [Carlos Mena](mailto:carlos.mena@ciempiess.org)

### Dataset Summary

The CIEMPIESS BALANCE Corpus is designed to match with the [CIEMPIESS LIGHT](https://huggingface.co/datasets/ciempiess/ciempiess_light) Corpus [(LDC2017S23)](https://catalog.ldc.upenn.edu/LDC2017S23). So, "Balance" means that if the CIEMPIESS BALANCE is combined with the CIEMPIESS LIGHT, one will get a gender balanced corpus. To appreciate this, one need to know that the CIEMPIESS LIGHT is by itself, a gender unbalanced corpus of approximately 25% of female speakers and 75% of male speakers. So, the CIEMPIESS BALANCE is a gender unbalanced corpus with approximately 25% of male speakers and 75% of female speakers. 

Furthermore, the match between the two datasets is more profound than just the number of the speakers. In both corpus speakers are numbered as: F_01, M_01, F_02, M_02, etc. So, the relation between the speakers is that the speech of F_01 in CIEMPIES LIGHT has an approximate amount of time as the speech of M_01 in the CIEMPIESS BALANCE.

The consequence of this speaker-to-speaker match is that the CIEMPIESS BALANCE has a size of 18 hours and 20 minutes against the 18 hours and 25 minutes of the CIEMPIESS LIGHT. It is a very good match between them!

CIEMPIESS is the acronym for:

"Corpus de Investigación en Español de México del Posgrado de Ingeniería Eléctrica y Servicio Social".

### Example Usage
The CIEMPIESS BALANCE contains only the train split:
```python
from datasets import load_dataset
ciempiess_balance = load_dataset("ciempiess/ciempiess_balance")
```
It is also valid to do:
```python
from datasets import load_dataset
ciempiess_balance = load_dataset("ciempiess/ciempiess_balance",split="train")
```

### Supported Tasks
automatic-speech-recognition: The dataset can be used to test a model for Automatic Speech Recognition (ASR). The model is presented with an audio file and asked to transcribe the audio file to written text. The most common evaluation metric is the word error rate (WER).

### Languages
The language of the corpus is Spanish with the accent of Central Mexico.

## Dataset Structure

### Data Instances
```python
{
  'audio_id': 'CMPB_F_41_01CAR_00011', 
  'audio': {
    'path': '/home/carlos/.cache/HuggingFace/datasets/downloads/extracted/6564823bd50fe590ce15086c22ddf7efe2302a8f988f12469f61940f2b88c051/train/female/F_41/CMPB_F_41_01CAR_00011.flac', 
    'array': array([0.00283813, 0.00442505, 0.00720215, ..., 0.00543213, 0.00570679,
       0.00952148], dtype=float32), 'sampling_rate': 16000
  }, 
  'speaker_id': 'F_41', 
  'gender': 'female', 
  'duration': 7.519000053405762, 
  'normalized_text': 'entonces mira oye pasa esto y tú así de ay pues déjame leer porque ni sé no así pasa porque pues'
}
```

### Data Fields
* `audio_id` (string) - id of audio segment
* `audio` (datasets.Audio) - a dictionary containing the path to the audio, the decoded audio array, and the sampling rate. In non-streaming mode (default), the path points to the locally extracted audio. In streaming mode, the path is the relative path of an audio inside its archive (as files are not downloaded and extracted locally).
* `speaker_id` (string) - id of speaker
* `gender` (string) - gender of speaker (male or female)
* `duration` (float32) - duration of the audio file in seconds.
* `normalized_text` (string) - normalized audio segment transcription

### Data Splits

The corpus counts just with the train split which has a total of 8555 speech files from 53 female speakers and 34 male speakers with a total duration of 18 hours and 20 minutes.

## Dataset Creation

### Curation Rationale

The CIEMPIESS BALANCE (CB) Corpus has the following characteristics:

* The CB has a total of 8555 audio files of 53 female speakers and 34 male speakers. It has a total duration of 18 hours and 20 minutes.

* The total number of audio files that come from male speakers is 2447 with a total duration of 5 hours and 40 minutes. The total number of audio files that come from female speakers is 6108 with a total duration of 12 hours and 40 minutes.

* Every audio file in the CB has a duration between 5 and 10 seconds approximately.

* Speakers in the CB and the CIEMPIESS LIGHT (CL) are different persons. In fact, speakers in the CB are not present in any other CIEMPIESS dataset.

* The CL is slightly bigger (18 hours / 25 minutes) than the CB (18 hours / 20 minutes).

* Data in CB is classified by gender and also by speaker, so one can easily select audios from a particular set of speakers to do experiments.

* Audio files in the CL and the CB are all of the same type. In both, speakers talk about legal and lawyer issues. They also talk about things related to the [UNAM University](https://www.unam.mx/) and the ["Facultad de Derecho de la UNAM"](https://www.derecho.unam.mx/).

* As in the CL, transcriptions in the CB were made by humans.

* Audio files in the CB are distributed in a 16khz@16bit mono format.

### Source Data

#### Initial Data Collection and Normalization

The CIEMPIESS BALANCE is a Radio Corpus designed to train acoustic models of automatic speech recognition and it is made out of recordings of spontaneous conversations in Spanish between a radio moderator and his guests. Most of the speech in these conversations has the accent of Central Mexico.

All the recordings that constitute the CIEMPIESS BALANCE come from [RADIO-IUS](https://www.derecho.unam.mx/cultura-juridica/radio.php), a radio station belonging to [UNAM](https://www.unam.mx/). Recordings were donated by Lic. Cesar Gabriel Alanis Merchand and Mtro. Ricardo Rojas Arevalo from the [Facultad de Derecho de la UNAM](https://www.derecho.unam.mx/) with the condition that they have to be used for academic and research purposes only.

### Annotations

#### Annotation process

The annotation process is at follows:

* 1. A whole podcast is manually segmented keeping just the portions containing good quality speech.
* 2. A second pass os segmentation is performed; this time to separate speakers and put them in different folders.
* 3. The resulting speech files between 5 and 10 seconds are transcribed by students from different departments (computing, engineering, linguistics). Most of them are native speakers but not with a particular training as transcribers.

#### Who are the annotators?

The CIEMPIESS BALANCE Corpus was created by the social service program ["Desarrollo de Tecnologías del Habla"](http://profesores.fi-b.unam.mx/carlos_mena/servicio.html) of the ["Facultad de Ingeniería"](https://www.ingenieria.unam.mx/) (FI) in the ["Universidad Nacional Autónoma de México"](https://www.unam.mx/) (UNAM) between 2016 and 2018 by Carlos Daniel Hernández Mena, head of the program.

### Personal and Sensitive Information

The dataset could contain names revealing the identity of some speakers; on the other side, the recordings come from publicly available podcasts, so, there is not a real intent of the participants to be anonymized. Anyway, you agree to not attempt to determine the identity of speakers in this dataset.

## Considerations for Using the Data

### Social Impact of Dataset

This dataset is valuable because it contains spontaneous speech.

### Discussion of Biases

The dataset is not gender balanced. It is comprised of 53 female speakers and 34 male speakers and the vocabulary is limited to legal issues.

### Other Known Limitations

"CIEMPIESS BALANCE CORPUS" by Carlos Daniel Hernández Mena is licensed under a [Creative Commons Attribution-ShareAlike 4.0 International (CC BY-SA 4.0)](https://creativecommons.org/licenses/by-sa/4.0/) License with the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

### Dataset Curators

The dataset was collected by students belonging to the social service program ["Desarrollo de Tecnologías del Habla"](http://profesores.fi-b.unam.mx/carlos_mena/servicio.html). It was curated by [Carlos Daniel Hernández Mena](https://huggingface.co/carlosdanielhernandezmena) in 2018.

### Licensing Information
[CC-BY-SA-4.0](https://creativecommons.org/licenses/by-sa/4.0/)

### Citation Information
```
@misc{carlosmenaciempiessbalance2018,
      title={CIEMPIESS BALANCE CORPUS: Audio and Transcripts of Mexican Spanish Broadcast Conversations.}, 
      ldc_catalog_no={LDC2018S11},
      DOI={https://doi.org/10.35111/rfmw-n126},
      author={Hernandez Mena, Carlos Daniel},
      journal={Linguistic Data Consortium, Philadelphia},
      year={2018},
      url={https://catalog.ldc.upenn.edu/LDC2018S11},
}
```
### Contributions

The authors want to thank to Alejandro V. Mena, Elena Vera and Angélica Gutiérrez for their support to the social service program: "Desarrollo de Tecnologías del Habla." We also thank to the social service students for all the hard work.

