task_categories:token-classification size_categories:n<1K language:es license:mit region:us
---
dataset_info:
  features:
  - name: text
    dtype: string
  - name: annotation
    list:
    - name: end
      dtype: int64
    - name: label
      dtype: string
    - name: start
      dtype: int64
  - name: id
    dtype: string
  splits:
  - name: train
    num_bytes: 43389.8358778626
    num_examples: 209
  - name: test
    num_bytes: 11003.164122137405
    num_examples: 53
  download_size: 42448
  dataset_size: 54393
task_categories:
- token-classification
language:
- es
size_categories:
- n<1K
license: mit
---

# Dataset Card for "podcasts-ner-es"

## Table of Contents
- [Table of Contents](#table-of-contents)
- [Dataset Description](#dataset-description)
  - [Dataset Summary](#dataset-summary)
  - [Supported Tasks and Leaderboards](#supported-tasks-and-leaderboards)
  - [Languages](#languages)
- [Dataset Structure](#dataset-structure)
  - [Data Instances](#data-instances)
  - [Data Fields](#data-fields)
  - [Data Splits](#data-splits)
- [Dataset Creation](#dataset-creation)
  - [Curation Rationale](#curation-rationale)
  - [Source Data](#source-data)
  - [Annotations](#annotations)
  - [Personal and Sensitive Information](#personal-and-sensitive-information)
- [Considerations for Using the Data](#considerations-for-using-the-data)
  - [Social Impact of Dataset](#social-impact-of-dataset)
  - [Discussion of Biases](#discussion-of-biases)
  - [Other Known Limitations](#other-known-limitations)
- [Additional Information](#additional-information)
  - [Dataset Curators](#dataset-curators)
  - [Licensing Information](#licensing-information)
  - [Citation Information](#citation-information)
  - [Contributions](#contributions)
- [Team members](#team-members)

## Dataset Description

- **Homepage:**
- **Repository:**
- **Paper:**
- **Leaderboard:**
- **Point of Contact:**

### Dataset Summary

This dataset comprises of small text snippets extracted from the "Deforme Semanal" podcast,
accompanied by annotations that identify the presence of a predetermined set of entities.
The purpose of this dataset is to facilitate Named Entity Recognition (NER) tasks.
The dataset was created to aid in the identification of entities such as famous people, books, or films in podcasts.
The transcription of the audio was first done, followed by annotation with GPT-3 and curation with Argilla.
The dataset is in Spanish, covering mostly topics such as love, feminism, and art, which are the main themes of the podcast.

### Supported Tasks and Leaderboards

Named Entity Recognition

### Languages

The dataset is in Spanish and the language used is primarily informal. 
It is important to note that the language may include aggressive or offensive content.

## Dataset Structure

### Data Instances

```
{
"text":"Tengo 39 años, pues, ya veré cuándo yo quiero dejar de comer ternera, está mal, porque hay sobre explotación y todo esto, muy mal."
"annotation": [ { "end": 13, "label": "DATES", "start": 6 } ]
"id": "53c4748e-dbd2-4cf5-946f-d134b0bf6155"
}
```

### Data Fields

`text`: Snippet of text of no more than 512 characters extracted from a podcast episode.

`id`: Unique identification number for each instance in the dataset.

`annotation`: List of dictonary-like format with the following fields:

  - `end`: end character of the entity ocurrence in the text.
  - `start`: start character of the entity ocurrence in the text.
  - `label`: label for the entity from the predefined set of entities. The label of the entities is one of:
    'people', 'products', 'books', 'animals', 'organizations', 'topics', 'dates', 'places', 'artista', 'objects','songs', and 'films'.

### Data Splits

The dataset was shuffled and split using the `train_test_split` function from the Hugging Face datasets library. 
The split was made with a train size of 0.8 and a seed of 42.

## Dataset Creation

### Curation Rationale

We created this dataset with the aim of making the information from our favorite podcasts more accessible, as retrieving information from audio formats can be challenging.
We chose to focus on the Named Entity Recognition (NER) task as it was relatively easy to annotate and validate.

### Source Data

#### Initial Data Collection and Normalization

We collected the data from a playlist on YouTube containing approximately 15 episodes of the "Deforme Semanal" podcast.
You can find the playlist at this [link](https://www.youtube.com/playlist?list=PLLbN7SMQhMVZoXhtQ00AyebQE_-ttDrs9).

We then transcribed the audio stream using OpenAI's Whisper (medium size) and split the resulting text files
into chunks of less than 512 characters.

### Annotations

#### Annotation process

To annotate the texts, we used OpenAI's API and GPT-3, with the following prompt:
```
Perform named entity recognition in Spanish. The classes are books, films, video games, songs, places, dates, topics, organizations, and people. The output should follow the format:
[{'class': 'people', 'text': 'name of the person'}, {'class': 'books', 'start': 'name of the book'}]

Sentence:
```
Finally, to ensure the quality of the dataset, we validated the annotations using Argilla by checking that the tokens were classified
correctly.


## Considerations for Using the Data

### Discussion of Biases

The dataset was obtained from the "Deforme Semanal" podcast, which primarily focuses on art, feminism, and culture.
As a result, the data is directly related to the topics and individuals discussed in these contexts. Additionally,
the language used in the podcast is informal and can be aggressive or offensive at times, which may be reflected in the dataset.

Although we attempted to minimize these biases during the validation process, their effectiveness is likely limited.

### Other Known Limitations

One issue that we have encountered with the token/entity data is that there can be some ambiguity in terms of their distinctions.
In some cases, it may not be clear how to differentiate between two tokens or entities, which can impact the accuracy
and effectiveness of models trained on this data.

Furthermore, the dataset size is relatively small, which can pose a challenge when it comes to training machine learning models.
With a limited amount of data, it can be difficult to capture the full range of variations and patterns in the data,
and overfitting can become a concern. This is especially true when dealing with complex models that require a large
amount of data to train effectively.

## Team members 

[David Mora](https://huggingface.co/DavidFM43)

[Sergio Perez](https://huggingface.co/sergiopperez)

[Albeto Fernandez](https://huggingface.co/AlbertoFH98)
