region:us
Distilled CNN/DailyMail non-anonymized summarization dataset.
There are two features:
  - article: text of news article, used as the document to be summarized
  - highlights: joined text of highlights with <s> and </s> around each
    highlight, which is the target summary

The pseudo labels are generated by running 
    1. facebook/bart-large-cnn on the CNN/DailyMail dataset, or
    2. sshleifer/pegasus-cnn-ft-v2 on the CNN/DailyMail dataset.

The files used here is directly downloaded from 
https://github.com/huggingface/transformers/blob/main/examples/research_projects/seq2seq-distillation/precomputed_pseudo_labels.md.
# Distilled CNN/DailyMail Dataset

This folder contains the distilled data and dataset loading script to build a dataset on top of it.

- `cnn_bart_pl` is downloaded from [Saved Pseudo-Labels](https://github.com/huggingface/transformers/blob/main/examples/research_projects/seq2seq-distillation/precomputed_pseudo_labels.md), which is generated by facebook/bart-large-cnn, this corresponds to version "1.0.0". It contains train/validataion/test splits.
- `pegasus_cnn_cnn_pls` is also downloaded from [Saved Pseudo-Labels](https://github.com/huggingface/transformers/blob/main/examples/research_projects/seq2seq-distillation/precomputed_pseudo_labels.md). It is generated by sshleifer/pegasus-cnn-ft-v2, and it corresponds to version "2.0.0". It only includes the train split.


## Updates
- 03/16/2023
    1. Remove "(CNN)" in the beginning of articles.