task_categories:summarization annotations_creators:crowdsourced language_creators:crowdsourced multilinguality:multilingual size_categories:10K<n<100K size_categories:1K<n<10K source_datasets:original language:ar language:cs language:de language:en language:es language:fr language:hi language:id language:it language:ja language:ko language:nl language:pt language:ru language:th language:tr language:vi language:zh license:cc-by-3.0 arxiv:2010.03093 region:us
WikiLingua is a large-scale multilingual dataset for the evaluation of
cross-lingual abstractive summarization systems. The dataset includes ~770k
article and summary pairs in 18 languages from WikiHow. The gold-standard
article-summary alignments across languages was done by aligning the images
that are used to describe each how-to step in an article.
@inproceedings{ladhak-etal-2020-wikilingua,
    title = "{W}iki{L}ingua: A New Benchmark Dataset for Cross-Lingual Abstractive Summarization",
    author = "Ladhak, Faisal  and
      Durmus, Esin  and
      Cardie, Claire  and
      McKeown, Kathleen",
    booktitle = "Findings of the Association for Computational Linguistics: EMNLP 2020",
    month = nov,
    year = "2020",
    address = "Online",
    publisher = "Association for Computational Linguistics",
    url = "https://aclanthology.org/2020.findings-emnlp.360",
    doi = "10.18653/v1/2020.findings-emnlp.360",
    pages = "4034--4048",
}
---
annotations_creators:
- crowdsourced
language_creators:
- crowdsourced
language:
- ar
- cs
- de
- en
- es
- fr
- hi
- id
- it
- ja
- ko
- nl
- pt
- ru
- th
- tr
- vi
- zh
license:
- cc-by-3.0
multilinguality:
- multilingual
size_categories:
- 10K<n<100K
- 1K<n<10K
source_datasets:
- original
task_categories:
- summarization
task_ids: []
paperswithcode_id: wikilingua
pretty_name: WikiLingua
dataset_info:
- config_name: arabic
  features:
  - name: url
    dtype: string
  - name: article
    sequence:
    - name: section_name
      dtype: string
    - name: document
      dtype: string
    - name: summary
      dtype: string
    - name: english_url
      dtype: string
    - name: english_section_name
      dtype: string
  splits:
  - name: train
    num_bytes: 119116119
    num_examples: 9995
  download_size: 119358890
  dataset_size: 119116119
- config_name: chinese
  features:
  - name: url
    dtype: string
  - name: article
    sequence:
    - name: section_name
      dtype: string
    - name: document
      dtype: string
    - name: summary
      dtype: string
    - name: english_url
      dtype: string
    - name: english_section_name
      dtype: string
  splits:
  - name: train
    num_bytes: 41170689
    num_examples: 6541
  download_size: 41345464
  dataset_size: 41170689
- config_name: czech
  features:
  - name: url
    dtype: string
  - name: article
    sequence:
    - name: section_name
      dtype: string
    - name: document
      dtype: string
    - name: summary
      dtype: string
    - name: english_url
      dtype: string
    - name: english_section_name
      dtype: string
  splits:
  - name: train
    num_bytes: 20816390
    num_examples: 2520
  download_size: 20894511
  dataset_size: 20816390
- config_name: dutch
  features:
  - name: url
    dtype: string
  - name: article
    sequence:
    - name: section_name
      dtype: string
    - name: document
      dtype: string
    - name: summary
      dtype: string
    - name: english_url
      dtype: string
    - name: english_section_name
      dtype: string
  splits:
  - name: train
    num_bytes: 87258040
    num_examples: 10862
  download_size: 87533442
  dataset_size: 87258040
- config_name: english
  features:
  - name: url
    dtype: string
  - name: article
    sequence:
    - name: section_name
      dtype: string
    - name: document
      dtype: string
    - name: summary
      dtype: string
  splits:
  - name: train
    num_bytes: 333700114
    num_examples: 57945
  download_size: 338036185
  dataset_size: 333700114
- config_name: french
  features:
  - name: url
    dtype: string
  - name: article
    sequence:
    - name: section_name
      dtype: string
    - name: document
      dtype: string
    - name: summary
      dtype: string
    - name: english_url
      dtype: string
    - name: english_section_name
      dtype: string
  splits:
  - name: train
    num_bytes: 197550376
    num_examples: 21690
  download_size: 198114157
  dataset_size: 197550376
- config_name: german
  features:
  - name: url
    dtype: string
  - name: article
    sequence:
    - name: section_name
      dtype: string
    - name: document
      dtype: string
    - name: summary
      dtype: string
    - name: english_url
      dtype: string
    - name: english_section_name
      dtype: string
  splits:
  - name: train
    num_bytes: 168674340
    num_examples: 20103
  download_size: 169195050
  dataset_size: 168674340
- config_name: hindi
  features:
  - name: url
    dtype: string
  - name: article
    sequence:
    - name: section_name
      dtype: string
    - name: document
      dtype: string
    - name: summary
      dtype: string
    - name: english_url
      dtype: string
    - name: english_section_name
      dtype: string
  splits:
  - name: train
    num_bytes: 63785051
    num_examples: 3402
  download_size: 63874759
  dataset_size: 63785051
- config_name: indonesian
  features:
  - name: url
    dtype: string
  - name: article
    sequence:
    - name: section_name
      dtype: string
    - name: document
      dtype: string
    - name: summary
      dtype: string
    - name: english_url
      dtype: string
    - name: english_section_name
      dtype: string
  splits:
  - name: train
    num_bytes: 136408861
    num_examples: 16308
  download_size: 136833587
  dataset_size: 136408861
- config_name: italian
  features:
  - name: url
    dtype: string
  - name: article
    sequence:
    - name: section_name
      dtype: string
    - name: document
      dtype: string
    - name: summary
      dtype: string
    - name: english_url
      dtype: string
    - name: english_section_name
      dtype: string
  splits:
  - name: train
    num_bytes: 138119527
    num_examples: 17673
  download_size: 138578956
  dataset_size: 138119527
- config_name: japanese
  features:
  - name: url
    dtype: string
  - name: article
    sequence:
    - name: section_name
      dtype: string
    - name: document
      dtype: string
    - name: summary
      dtype: string
    - name: english_url
      dtype: string
    - name: english_section_name
      dtype: string
  splits:
  - name: train
    num_bytes: 40145031
    num_examples: 4372
  download_size: 40259570
  dataset_size: 40145031
- config_name: korean
  features:
  - name: url
    dtype: string
  - name: article
    sequence:
    - name: section_name
      dtype: string
    - name: document
      dtype: string
    - name: summary
      dtype: string
    - name: english_url
      dtype: string
    - name: english_section_name
      dtype: string
  splits:
  - name: train
    num_bytes: 38647614
    num_examples: 4111
  download_size: 38748961
  dataset_size: 38647614
- config_name: portuguese
  features:
  - name: url
    dtype: string
  - name: article
    sequence:
    - name: section_name
      dtype: string
    - name: document
      dtype: string
    - name: summary
      dtype: string
    - name: english_url
      dtype: string
    - name: english_section_name
      dtype: string
  splits:
  - name: train
    num_bytes: 204270845
    num_examples: 28143
  download_size: 204997686
  dataset_size: 204270845
- config_name: russian
  features:
  - name: url
    dtype: string
  - name: article
    sequence:
    - name: section_name
      dtype: string
    - name: document
      dtype: string
    - name: summary
      dtype: string
    - name: english_url
      dtype: string
    - name: english_section_name
      dtype: string
  splits:
  - name: train
    num_bytes: 241924032
    num_examples: 18143
  download_size: 242377242
  dataset_size: 241924032
- config_name: spanish
  features:
  - name: url
    dtype: string
  - name: article
    sequence:
    - name: section_name
      dtype: string
    - name: document
      dtype: string
    - name: summary
      dtype: string
    - name: english_url
      dtype: string
    - name: english_section_name
      dtype: string
  splits:
  - name: train
    num_bytes: 314618618
    num_examples: 38795
  download_size: 315609530
  dataset_size: 314618618
- config_name: thai
  features:
  - name: url
    dtype: string
  - name: article
    sequence:
    - name: section_name
      dtype: string
    - name: document
      dtype: string
    - name: summary
      dtype: string
    - name: english_url
      dtype: string
    - name: english_section_name
      dtype: string
  splits:
  - name: train
    num_bytes: 86982851
    num_examples: 5093
  download_size: 87104200
  dataset_size: 86982851
- config_name: turkish
  features:
  - name: url
    dtype: string
  - name: article
    sequence:
    - name: section_name
      dtype: string
    - name: document
      dtype: string
    - name: summary
      dtype: string
    - name: english_url
      dtype: string
    - name: english_section_name
      dtype: string
  splits:
  - name: train
    num_bytes: 11371821
    num_examples: 1512
  download_size: 11405793
  dataset_size: 11371821
- config_name: vietnamese
  features:
  - name: url
    dtype: string
  - name: article
    sequence:
    - name: section_name
      dtype: string
    - name: document
      dtype: string
    - name: summary
      dtype: string
    - name: english_url
      dtype: string
    - name: english_section_name
      dtype: string
  splits:
  - name: train
    num_bytes: 69868788
    num_examples: 6616
  download_size: 70024093
  dataset_size: 69868788
config_names:
- arabic
- chinese
- czech
- dutch
- english
- french
- german
- hindi
- indonesian
- italian
- japanese
- korean
- portuguese
- russian
- spanish
- thai
- turkish
- vietnamese
---
# Dataset Card for "wiki_lingua"

## Table of Contents
- [Dataset Description](#dataset-description)
  - [Dataset Summary](#dataset-summary)
  - [Supported Tasks and Leaderboards](#supported-tasks-and-leaderboards)
  - [Languages](#languages)
- [Dataset Structure](#dataset-structure)
  - [Data Instances](#data-instances)
  - [Data Fields](#data-fields)
  - [Data Splits](#data-splits)
- [Dataset Creation](#dataset-creation)
  - [Curation Rationale](#curation-rationale)
  - [Source Data](#source-data)
  - [Annotations](#annotations)
  - [Personal and Sensitive Information](#personal-and-sensitive-information)
- [Considerations for Using the Data](#considerations-for-using-the-data)
  - [Social Impact of Dataset](#social-impact-of-dataset)
  - [Discussion of Biases](#discussion-of-biases)
  - [Other Known Limitations](#other-known-limitations)
- [Additional Information](#additional-information)
  - [Dataset Curators](#dataset-curators)
  - [Licensing Information](#licensing-information)
  - [Citation Information](#citation-information)
  - [Contributions](#contributions)

## Dataset Description

- **Repository:** [URL](https://github.com/esdurmus/Wikilingua)
- **Paper:** [WikiLingua: A Multilingual Abstractive Summarization Dataset](https://arxiv.org/abs/2010.03093)

### Dataset Summary

We introduce WikiLingua, a large-scale, multilingual dataset for the evaluation of cross-lingual abstractive summarization systems. We extract article and summary pairs in 18 languages from WikiHow, a high quality, collaborative resource of how-to guides on a diverse set of topics written by human authors. We create gold-standard article-summary alignments across languages by aligning the images that are used to describe each how-to step in an article.

### Supported Tasks and Leaderboards

[More Information Needed]

### Languages

The table below shows number of article-summary pairs with a parallel article-summary pair in English. 
______________________________
| Language    | Num. parallel |
| ----------- | --------------|
| English     |   141,457     |
| Spanish     |   113,215     |
| Portuguese  |    81,695     |
| French      |    63,692     |
| German      |    58,375     |
| Russian     |    52,928     |
| Italian     |    50,968     |
| Indonesian  |    47,511     |
| Dutch       |    31,270     |
| Arabic      |    29,229     |
| Vietnamese  |    19,600     |
| Chinese     |    18,887     |
| Thai        |    14,770     |
| Japanese    |    12,669     |
| Korean      |    12,189     |
| Hindi       |     9,929     |
| Czech       |     7,200     |
| Turkish     |     4,503     |


## Dataset Structure

### Data Instances

```
{
    'article': {
        'document': ['make sure that the area is a safe place, especially if you plan on walking home at night.  It’s always a good idea to practice the buddy system.  Have a friend meet up and walk with you. Research the bus, train, or streetcar routes available in your area to find safe and affordable travel to your destination.  Make sure you check the schedule for your outgoing and return travel.  Some public transportation will cease to run late at night.  Be sure if you take public transportation to the venue that you will also be able to get home late at night. Check the routes.  Even if some public transit is still running late at night, the routing may change.  Some may run express past many of the stops, or not travel all the way to the ends.  Be sure that your stop will still be available when you need it for your return trip. If you are taking public transit in a vulnerable state after drinking, it is always a good idea to travel in groups.  Having friends available is a good way to stay safe and make sure that you reach your destination. This is more expensive option than a taxi or ride share service, but could be a fun and fancy way to stay safe and ensure that you will have a ride home. Plan this service in advance with a scheduled time to pick you up from your home and the venue.  You want to be sure that the service will still be available when you need to get home. This may be easy in a large city, but taxis may be less frequent in smaller towns.  This is especially true late at night, so this is a less reliable option than scheduling a ride in advance.  Have a friend accompany you and help you flag a cab to make sure you are able to get one. Set up a plan to call a friend when you get home to make sure that you made it safely to your destination. If there are no taxis readily available call a local service to send a car to pick you up. You can share a ride with your friends, or other people using the app at the same moment.  If you are in a vulnerable state it is best to share the ride with your friends to make sure you get home safe. You can request the car to yourself rather than sharing rides with strangers. If you travel home on your own or are the last of your group to be dropped off, make plans to call a friend when you get home so they know you made it safely to your destination. There may be a designated driver service in your area which can chauffeur your group.  Make reservations with them in advance and keep their contact information handy while you are drinking.',
            "Designating a driver is a very popular tactic to avoid drinking and driving.  It is important to plan in advance, because your brain function will slow down and your decision making skills will be impaired once you start drinking. Decide before you begin drinking that you will not drive.  Figure out who will be getting you home before you leave. Make sure this person is responsible and keep them in your sight while you are drinking.  Have their contact information handy in case you can’t find them when you are ready to leave.  Choose a friend who doesn’t drink alcohol.  You likely have someone in your friend group who doesn’t drink.  This person is the most likely to remain sober. Decide on one person who will remain sober.  You can take turns within your friend group, alternating who will be the designated driver on each occasion.  Be sure that the designated driver actually remains sober.  The person who has drank the least is still not sober. If you don’t have your car with you, you can guarantee that you won’t make the choice to drive it home. If you are drinking at your home.  Give your keys to a responsible friend to ensure that you don't choose to drive somewhere after you have been drinking. It may be tempting to stay longer or leave with someone else.  Stick to the plan you made in advance and only leave with your sober, designated driver.  Keep the phone number of your driver handy in case you can't find them when you are ready to leave. If your designated driver drinks alcohol, find alternate transportation to get home.",
            'If you have been drinking at all you are at least on the spectrum of drunkenness.  You could be showing signs of impairment and slower brain function including lack of motor skills and slower reaction time, leading to the inability to operate a motor vehicle.  Some of these signs could be:  Poor balance or stumbling.  Difficulty speaking clearly and slurred words.  Abnormal behavior leading to you doing things you wouldn’t normally do if you were sober. As soon as you notice that you are showing signs of impairment, give your keys to a friend, the host or the bartender to ensure that you won’t drive until you are sober.  Make sure to only give them your car key.  Hold onto your house keys. If your friend, the host or the bartender are advising you not to drive, you are likely too drunk.  Listen to their advice and acknowledge that they are trying to help you. Bystander intervention is common when it comes to drinking and driving.  Many people will be willing to step in, take your keys and help you get home safely.  If no one if offering to help, you may need to ask.  Take a ride from a sober friend.  It is best to get in a car with someone you trust when you are in this vulnerable state. Allow the host or bartender to call a cab or car service to take you home. If you are having a difficult time finding a safe way to get home, find a place to stay which does not involve you driving.  Ask the host of the party if there is a place you can sleep.  Give them your keys and ask that they keep them in a safe place until the morning. Stay with a friend if they live nearby and are on their way home. Find a hotel within walking distance.  Call them to book a room, or have a friend help you secure one.  Ask the friend if they will walk you to the hotel and make sure you get checked in safely. There are people in your life who care about you and want to be sure that you are safe.  It may seem scary or embarrassing to call your parents or your siblings if you are too drunk to drive, but they will be glad you did.  Your safety is the most important. You may need your phone to call someone for a ride or get help from a friend.  Be sure to charge your phone before you leave the house.  It is also a good idea to bring a charger with you in case your battery dies before the end of the night or you end up staying where you are and need to get home the next morning. You may also want to invest in a portable battery charger for your phone should there not be a power outlet available.  Make sure it is fully charged before you leave your house.  Keep it handy in your pocket or your bag throughout the night.'
        ],
        'section_name': ['Finding Other Transportation',
            'Designating a Driver',
            'Staying Safe'
        ],
        'summary': ['Walk to the venue where you will be drinking if it is close enough. Take public transit. Show up in style by hiring a limo or black car service. Flag a taxi cab for a convenient option to get where you’re going. Request a rideshare service like Uber or Lyft using an app on your phone. Reserve a designated driver service.',
            'Plan in advance. Assign a designated driver. Leave your car at home. Leave the venue with your designated driver.',
            'Pay attention to your body. Give up your keys. Listen to other people. Accept help. Stay where you are. Have an emergency back-up plan. Make sure that your phone is charged.'
        ]
    },
    'url': 'https://www.wikihow.com/Avoid-Drinking-and-Driving'
}
```
### Data Fields

- `url`: WikiHow URL of the article
- `article`: A dictionary containing `section_name`, `document` and `summary`
  - `section_name`: List of section headings in an article
  - `document`: List of documents, one for each section in the `section_name` list
  - `summary`: List of summarized document

### Data Splits

|            |   train |
|:-----------|--------:|
| arabic     |    9995 |
| chinese    |    6541 |
| czech      |    2520 |
| dutch      |   10862 |
| english    |   57945 |
| french     |   21690 |
| german     |   20103 |
| hindi      |    3402 |
| indonesian |   16308 |
| italian    |   17673 |
| japanese   |    4372 |
| korean     |    4111 |
| portuguese |   28143 |
| russian    |   18143 |
| spanish    |    6616 |
| thai       |    5093 |
| turkish    |    1512 |
| vietnamese |    6616 |

## Dataset Creation

### Curation Rationale

[More Information Needed]

### Source Data

[More Information Needed]

#### Initial Data Collection and Normalization

[More Information Needed]

#### Who are the source language producers?

[More Information Needed]

### Annotations

[More Information Needed]

#### Annotation process

[More Information Needed]

#### Who are the annotators?

[More Information Needed]

### Personal and Sensitive Information

[More Information Needed]

## Considerations for Using the Data

### Social Impact of Dataset

[More Information Needed]

### Discussion of Biases

[More Information Needed]

### Other Known Limitations

[More Information Needed]

## Additional Information

### Dataset Curators

[More Information Needed]

### Licensing Information

- Article provided by wikiHow https://www.wikihow.com/Main-Page, a wiki building the world's largest, highest quality how-to manual. Please edit this article and find author credits at wikiHow.com. Content on wikiHow can be shared under a [Creative Commons license](http://creativecommons.org/licenses/by-nc-sa/3.0/).
- Refer to [this webpage](https://www.wikihow.com/wikiHow:Attribution) for the specific attribution guidelines.
- also see https://gem-benchmark.com/data_cards/WikiLingua

### Citation Information

```bibtex
@inproceedings{ladhak-etal-2020-wikilingua,
    title = "{W}iki{L}ingua: A New Benchmark Dataset for Cross-Lingual Abstractive Summarization",
    author = "Ladhak, Faisal  and
      Durmus, Esin  and
      Cardie, Claire  and
      McKeown, Kathleen",
    booktitle = "Findings of the Association for Computational Linguistics: EMNLP 2020",
    month = nov,
    year = "2020",
    address = "Online",
    publisher = "Association for Computational Linguistics",
    url = "https://aclanthology.org/2020.findings-emnlp.360",
    doi = "10.18653/v1/2020.findings-emnlp.360",
    pages = "4034--4048",
}
```

### Contributions

Thanks to [@katnoria](https://github.com/katnoria) for adding this dataset.