task_categories:question-answering size_categories:100K<n<1M language:en region:us
---
task_categories:
- question-answering
language:
- en
size_categories:
- 100K<n<1M
---

# Dataset Card for CoQA_Chat

## Dataset Description

A data set for training LLMs for in-context or Document Question-Answering conversations.

 - Point of Contact: info@alderley.ai 

### Dataset Summary

This dataset is an amended version of the CoQA dataset, with the question responses amended to be more conversational in nature, with a greater emphasis on returning contextually relervant infomration with the answer.

CoQA is a large-scale dataset for building Conversational Question Answering systems. The goal of the CoQA challenge is to measure the ability of machines to understand a text passage and answer a series of interconnected questions that appear in a conversation. CoQA is pronounced as coca .

https://stanfordnlp.github.io/coqa/

### Supported Tasks

In context and Document Question-Answerining

### Languages

English Only

## Dataset Structure

We provide both csv and jsonl files. 


### Data Fields

The csv and jsonl datasets has the following attributes:   
- id: Matches the original CoQA id (string)   
- local_order : Int associated with the order of the questions for user/assistant chat conversations. (integer)  
- context: Matches the original CoQA context (string)  
- question: Matches the original CoQA question (string)  
- answer: Conversational answer to question. (evolution of original CoQA answer) (string)  


### Data Splits

The original training and validation dataset have been combined into a single data splie.

## Dataset Creation

### Curation Rationale

This data set is specifically to support the training of large language models for in-context question-answering or document question-answering converstations. Small Instruct and Chat trained LLMs struggle with this task and have a tendency to ignore the provided context when generating an output. This data set is designed to  support the training of small LLMs that excel at this task.

### Source Data

#### Initial Data Collection and Normalization

CoQA
https://huggingface.co/datasets/coqa
https://stanfordnlp.github.io/coqa/

This new answer data set was generated from the original CoQA data set over several days by querying gpt-3.5-turbo with the following prompt...

```
system_intel = """In the dataset provided to you, there are several questions with two corresponding reference text for the answer. Each item in this dataset has an ID, a question, and two reference text answers. Your task is to use this information to create a concise and conversationally natural answer.
When writing your response, incorporate the essential elements from the question, reference text and answer, avoiding the use of pronouns. Instead, use the specific name or title of the entity being referred to. If a question can be answered with 'yes' or 'no', begin with that before providing a brief explanation.
Do not introduce new information, but do make sure that your response can stand on its own, even without the original question for context. However, strive to keep your answers succinct and avoid excessive context.
Each of your answers should be returned as a valid JSON object, with the keys "id" and "answer" surrounded by double quotes (""). If you need to use quotes within your answer, use single quotes ('') to keep the JSON formatting correct.
Here are a few examples:
    For [28960 'What is the official name of Brunei?' /n 'Brunei, officially the Nation of Brunei'   ‘Nation of Brunei’], output: {"id" : 28960, "answer" : "The official name of Brunei is the Nation of Brunei."}.
    For [28961, 'Where is it geographically?' /n 'sovereign state located on the north coast of the island of Borneo in Southeast Asia'   ‘Southeast Asia], output: {"id" :28961, "answer": "Brunei is located on the north coast of Borneo in Southeast Asia."}.
    For [28962, 'What body of water is it by?'/n 'Apart from its coastline with the South China Sea'   ‘South China Sea’], output: {"id": 28962, "answer": "Brunei is by the South China Sea."}.
    For [28963, 'When did Sultan Bolkaih rule?' /n 'Sultan Bolkiah (reigned 1485–1528'   ‘1485-1528’], output: {"id": 28963, "answer" : "Sultan Bolkaih ruled from 1485 to 1528."}.
    For [28964, 'What modern day areas did he rule over?' /n'including modern-day Sarawak and Sabah'  ‘Sarawak and Sabah’], output: {"id": 28964, "answer" : "Sultan Bolkaih ruled over modern-day Sarawak and Sabah."}.
Strive to keep your answers conversational, direct and to the point. Include only as much context as necessary for the answer to make sense on its own, do not use pronouns.
"""
prompt = f"Here is the dataset: {item_list}"

```

## Considerations for Using the Data

### Discussion of Biases

Data is in English only  
There are limited number of responses that reflect true negative responses, where there is insuffiecent information in the context to answer the question.   
In addition these are not uniformly answered with a specific token set, rather the answer will mention its unknown but will also include question contextual information  
e.g. As an answer to `"Where did he go afterwards?"` The answer may say `"It's unknown where he went afterwards."`


### Other Known Limitations

The nature of the questions in this data set are that they reflect a back and forth converstaion between a user and an assistant, and the user questions include the use of dummy pronouns and pronouns that require prior information from either the previous question or responses.   
Consequently for best use, it may be that for training, several questions that reflect a back and forth between the user and the agent are included in the training data before the final question is posed. This will encourage the model to learn about pronoun use in questioning.   
Heres an example:  
User: 'what is the official name of the Vat?'  
Assistant: 'The official name of the Vat is Vatican Apostolic Library, although it is more commonly called the Vatican Library or simply the Vat.'  
User: 'where is it?'  
Assistant: 'The Vatican Library is located in Vatican City and is the library of the Holy See.'  
User : 'how many printed books does it contain?'  
Assistant :   

### Contributions

Alderley.ai