task_categories:summarization language:en license:bsd-3-clause airoboros mixture of experts summarization arxiv:2105.08209 region:us
---
license: bsd-3-clause
task_categories:
- summarization
language:
- en
tags:
- airoboros
- mixture of experts
- summarization
pretty_name: Airoboros Summarization Expert
---
This is my adaptation and cleaned version of the Booksum dataset to work with Airoboros by Jon Durbin
[huggingface](https://huggingface.co/jondurbin)

I created this dataset for the purposes of improving the LLM capabilities with summarization. It's a core feature that I feel many applications rely on, yet we're still relying on older Longformer, RoBERTa, or BART solutions. 

This dataset has been altered from the original as follows:
- Cleaned up bad formatting, extra quotes at the beginning of summaries, extra line breaks, and references to pictures
- Tried to push everything as close to 4096 tokens in total for larger contexts
- Added instruction formatting that requires Airoboros to only use knowledge contained in the CONTEXT
- Added instruction formatting for requesting specific word counts

## Example short instruction from the dataset
```
You are an extremely careful writing assistant. Don't make up answers if you don't know.
USER:  
BEGININPUT  
BEGINCONTEXT  
IT was a sight that some people remembered better even than their own sorrows--the sight in that grey clear morning, when the fatal cart with the two young women in it was descried by the waiting watching multitude, cleaving its way towards the hideous symbol of a deliberately inflicted sudden death.  
All Stoniton had heard of Dinah Morris, the young Methodist woman who had brought the obstinate criminal to confess, and there was as much eagerness to see her as to see the wretched Hetty.  
But Dinah was hardly conscious of the multitude. When Hetty had caught sight of the vast crowd in the distance, she had clutched Dinah convulsively.  
"Close your eyes, Hetty," Dinah said, "and let us pray without ceasing to God."  
And in a low voice, as the cart went slowly along through the midst of the gazing crowd, she poured forth her soul with the wrestling intensity of a last pleading, for the trembling creature that clung to her and clutched her as the only visible sign of love and pity.  
Dinah did not know that the crowd was silent, gazing at her with a sort of awe--she did not even know how near they were to the fatal spot, when the cart stopped, and she shrank appalled at a loud shout hideous to her ear, like a vast yell of demons. Hetty's shriek mingled with the sound, and they clasped each other in mutual horror.  
But it was not a shout of execration--not a yell of exultant cruelty.  
It was a shout of sudden excitement at the appearance of a horseman cleaving the crowd at full gallop. The horse is hot and distressed, but answers to the desperate spurring; the rider looks as if his eyes were glazed by madness, and he saw nothing but what was unseen by others. See, he has something in his hand--he is holding it up as if it were a signal.  
The Sheriff knows him: it is Arthur Donnithorne, carrying in his hand a hard-won release from death.  
ENDCONTEXT  
ENDINPUT  
BEGININSTRUCTION  
Summarize the text in around 70 words.  
ENDINSTRUCTION  
ASSISTANT: Dinah rides out to the gallows with Hetty. At the sight of the crowd, Hetty clings to Dinah. They pray together and keep their eyes closed. The crowd is silent and stares and Dinah in awe. As they arrive at the gallows, a huge cry goes up from the crowd because a man has arrived on horseback. Captain Donnithorne arrives, and he has with him a stay of execution.  
```

# Original Dataset README follows

---

license:
- bsd-3-clause
train-eval-index:
- config: kmfoda--booksum
  task: summarization
  task_id: summarization
  splits:
    eval_split: test
  col_mapping:
    chapter: text
    summary_text: target
---

# BOOKSUM: A Collection of Datasets for Long-form Narrative Summarization
Authors: [Wojciech Kryściński](https://twitter.com/iam_wkr), [Nazneen Rajani](https://twitter.com/nazneenrajani), [Divyansh Agarwal](https://twitter.com/jigsaw2212), [Caiming Xiong](https://twitter.com/caimingxiong), [Dragomir Radev](http://www.cs.yale.edu/homes/radev/)

## Introduction
The majority of available text summarization datasets include short-form source documents that lack long-range causal and temporal dependencies, and often contain strong layout and stylistic biases. 
While relevant, such datasets will offer limited challenges for future generations of text summarization systems.
We address these issues by introducing BookSum, a collection of datasets for long-form narrative summarization.
Our dataset covers source documents from the literature domain, such as novels, plays and stories, and includes highly abstractive, human written summaries on three levels of granularity of increasing difficulty: paragraph-, chapter-, and book-level.
The domain and structure of our dataset poses a unique set of challenges for summarization systems, which include: processing very long documents, non-trivial causal and temporal dependencies, and rich discourse structures.
To facilitate future work, we trained and evaluated multiple extractive and abstractive summarization models as baselines for our dataset.

## Links

- [paper](https://arxiv.org/abs/2105.08209) by SalesForce Research
- [GitHub repo](https://github.com/salesforce/booksum)

## Table of Contents

1. [Citation](#citation)
2. [Legal Note](#legal-note)
3. [License](#license)


## Citation
```
@article{kryscinski2021booksum,
      title={BookSum: A Collection of Datasets for Long-form Narrative Summarization}, 
      author={Wojciech Kry{\'s}ci{\'n}ski and Nazneen Rajani and Divyansh Agarwal and Caiming Xiong and Dragomir Radev},
      year={2021},
      eprint={2105.08209},
      archivePrefix={arXiv},
      primaryClass={cs.CL}
}
```

## Legal Note
By downloading or using the resources, including any code or scripts, shared in this code
repository, you hereby agree to the following terms, and your use of the resources is conditioned
on and subject to these terms.
1. You may only use the scripts shared in this code repository for research purposes. You
may not use or allow others to use the scripts for any other purposes and other uses are
expressly prohibited.
2. You will comply with all terms and conditions, and are responsible for obtaining all
rights, related to the services you access and the data you collect.
3. We do not make any representations or warranties whatsoever regarding the sources from
which data is collected. Furthermore, we are not liable for any damage, loss or expense of
any kind arising from or relating to your use of the resources shared in this code
repository or the data collected, regardless of whether such liability is based in tort,
contract or otherwise.

## License
The code is released under the **BSD-3 License** (see `LICENSE.txt` for details).