region:us
---
configs:
- config_name: default
  data_files:
  - split: train
    path: data/train-*
  - split: val
    path: data/val-*
  - split: test
    path: data/test-*
dataset_info:
  features:
  - name: text
    dtype: string
  - name: summary
    dtype: string
  - name: text_len
    dtype: int64
  - name: summary_len
    dtype: int64
  splits:
  - name: train
    num_bytes: 118935809
    num_examples: 75219
  - name: val
    num_bytes: 6551332
    num_examples: 4178
  - name: test
    num_bytes: 6670392
    num_examples: 4178
  download_size: 81334629
  dataset_size: 132157533
---
# Dataset Card for "nordjylland-news-summarization"

## Dataset Description

- **Point of Contact:** [Oliver Kinch](mailto:oliver.kinch@alexandra.dk)
- **Size of dataset:** 148 MB

### Dataset Summary

This dataset consists of pairs containing text and corresponding summaries extracted from the Danish newspaper [TV2 Nord](https://www.tv2nord.dk/). 

### Supported Tasks and Leaderboards

Summarization is the intended task for this dataset. No leaderboard is active at this point.

### Languages

The dataset is available in Danish (`da`).

## Dataset Structure

An example from the dataset looks as follows.

```
{
  "text": "some text",
  "summary": "some summary",
  "text_len": <number of chars in text>,
  "summary_len": <number of chars in summary>
}
```

### Data Fields

- `text`: a `string` feature.
- `summary`: a `string` feature.
- `text_len`: an `int64` feature.
- `summary_len`: an `int64` feature.

### Dataset Statistics

#### Number of samples

- Train: 75219
- Val: 4178
- Test: 4178

#### Text Length Distribution

- Minimum length: 21
- Maximum length: 35164

![image/png](https://cdn-uploads.huggingface.co/production/uploads/61e0713ac50610f535ed2c88/YBO73NHfW5Ufh0svopGbc.png)

#### Summary Length Distribution

- Minimum length: 12
- Maximum length: 499

![image/png](https://cdn-uploads.huggingface.co/production/uploads/61e0713ac50610f535ed2c88/tSLeODADes_r-V7sED2tH.png)


## Potential Dataset Issues

Within the dataset, there are 181 instances where the length of the summary exceeds the length of the corresponding text.


## Dataset Creation

### Curation Rationale

There are not many large-scale summarization datasets in Danish.

### Source Data

The dataset has been collected through the TV2 Nord API, which can be accessed [here](https://developer.bazo.dk/#876ab6f9-e057-43e3-897a-1563de34397e).

## Additional Information

### Dataset Curators

[Oliver Kinch](https://huggingface.co/oliverkinch) from the [The Alexandra
Institute](https://alexandra.dk/)

### Licensing Information

The dataset is licensed under the [CC0
license](https://creativecommons.org/share-your-work/public-domain/cc0/).