task_categories:conversational task_categories:text-generation size_categories:10K<n<100K language:en license:cc-by-sa-4.0 region:us
---
license: cc-by-sa-4.0
task_categories:
- conversational
- text-generation
language:
- en
pretty_name: Llama Functions
size_categories:
- 10K<n<100K
---
# Dataset Card for Dataset Name

## Dataset Description

- **Homepage:** https://marclove.com
- **Repository:** https://huggingface.co/datasets/marclove/llama_functions

### Dataset Summary

‼️ This dataset is still in a beta state. Its contents, and likely its format, will change. If you need to depend on it in its current state, please create your own fork and provide attribution to this original repository. ‼️

Llama Functions is a synthetic dataset generated from a mix of manual curation of OpenAPI endpoints and prompting of OpenAI models. It is further mixed with chat completions from the Guanaco subset of the OASST1 chat dialogue dataset. It is a total of 18,000 rows, 9,000 rows from the synthetic dataset of function calls and 9,000 rows from the Guanaco dataset.

The dataset is mixed with Guanaco in order to maintain accuracy and helpfulness when calling a function is not the appropriate response. I plan to remove the Guanaco portion of the dataset and instead provide fine-tuning recommendations, guidelines for use, more detailed information regarding limitations, and eval stats of 7B, 13B, and 70B models.

There is no existing evaluation benchmark to measure the accuracy of function calls, which makes it hard during training to identify when we've maximized the balance of function calling accuracy and chat model performance. I'm working on a custom HF eval for this purpose, but until then I have chosen to mix the two datasets in equal parts to get a proxy of performance for both tasks in the eval & test stats during fine-tuning.

### Languages

English primarily, though since it has been mixed with the multilingual Guanaco dataset, other languages are included.

## Dataset Structure

### Data Fields

| Field | Description |
|-------|-------------|
| `input` |A prompt in Llama-2 Chat format, including an appropriate system instruction and chat history. |
| `output` | The expected completion. |

### Data Splits

There are currently no splits, but future versions will likely have train, eval, and test splits.

## Dataset Creation

### Curation Rationale

In an effort to enable tool-using chat agents and autonomous agents, I developed this synthetic dataset to bring [OpenAI-style function calling](https://openai.com/blog/function-calling-and-other-api-updates#function-calling) to the Llama family and to fully open source models.

### Source Data
The data was sourced by prompting OpenAI models to generate function calls of:

1. Real OpenAPI endpoints collected and filtered from the web
2. Manually written (but artificial) OpenAPI endpoints, and
3. Prompted iterations of 1 & 2.

Prompted iterations were generated by ChatGPT-4 (July 20, 2023 version). Generated function calls and their natural language counterparts were generated by iterative prompting of `gpt-3.5-turbo-0301`. A blog post detailing the generation process will be published in the next few days.

OpenAI's TOS give me ownership of this synthetic dataset. I am licensing it under [Creative Commons' Attribution-ShareAlike 4.0 International (CC BY-SA 4.0) license](https://creativecommons.org/licenses/by-sa/4.0/). I have used the dataset to fine tune a research-only model, [marclove/llama-2-7b-chat-functions](https://huggingface.co/marclove/llama-2-7b-chat-functions), per OpenAI TOS. You are responsible for determining whether you can use the dataset for your particular use case. I take no responsibility and make no guarantees beyond licensing my own rights under the designated CC license.

#### Who are the source language producers?

- Marc Love
- Prompting of ChatGPT-4 & API calls to gpt-3.5-turbo-0301

### Personal and Sensitive Information

None.

## Considerations for Using the Data

### Social Impact of Dataset

Unknown, beyond those of the [Guanaco subset of the OASST1 dataset](https://huggingface.co/datasets/timdettmers/openassistant-guanaco/viewer/timdettmers--openassistant-guanaco/).

### Discussion of Biases

Unknown, beyond those of the [Guanaco subset of the OASST1 dataset](https://huggingface.co/datasets/timdettmers/openassistant-guanaco/viewer/timdettmers--openassistant-guanaco/).

### Other Known Limitations

Fine-tuning on this dataset can lead to hallucinated function calls. This is more pronounced in smaller models. 

## Additional Information

### Dataset Curators

Marc Love

### Licensing Information

[Creative Commons' Attribution-ShareAlike 4.0 International (CC BY-SA 4.0) license](https://creativecommons.org/licenses/by-sa/4.0/). Please note that the synthetic data portion of the dataset was generated using OpenAI models, which may or may not impact your ability to use the dataset, depending on your use case.

### Citation Information

If you use this dataset, please cite:

```
@misc{LlamaFunctions,
  title = {LlamaFunctions: An Open Dataset of Structured API Calls From Natural Language Prompts},
  author = {Marc Love},
  year = {2023},
  publisher = {HuggingFace},
  journal = {HuggingFace repository},
  howpublished = {\url{https://https://huggingface.co/marclove/llama_functions},
}
```