task_categories:text-generation task_categories:fill-mask language:en license:cc-by-sa-3.0 language-modeling masked-language-modeling region:us
---
license: cc-by-sa-3.0
language:
- en
task_categories:
- text-generation
- fill-mask
tags:
- language-modeling
- masked-language-modeling
pretty_name: LongContextQA
configs:
- config_name: default
  default: true
  data_files:
  - split: train
    path:
    - "project-gutenberg.jsonl"
---
Long context QA with the following augmentations

- Smart augmentation (changes the answer to the question and in the context)
- Changes the data around the answer within the chunk
  - Random noise
  - Random chunks of information
- Lots of varied lengths
- A few different prompt formats (aimed towards RWKV)