APIGen: Automated Pipeline for Generating Verifiable and Diverse Function-Calling Datasets
Abstract
The advancement of function-calling agent models requires diverse, reliable, and high-quality datasets. This paper presents APIGen, an automated data generation pipeline designed to synthesize verifiable high-quality datasets for function-calling applications. We leverage APIGen and collect 3,673 executable APIs across 21 different categories to generate diverse function-calling datasets in a scalable and structured manner. Each data in our dataset is verified through three hierarchical stages: format checking, actual function executions, and semantic verification, ensuring its reliability and correctness. We demonstrate that models trained with our curated datasets, even with only 7B parameters, can achieve state-of-the-art performance on the Berkeley Function-Calling Benchmark, outperforming multiple GPT-4 models. Moreover, our 1B model achieves exceptional performance, surpassing GPT-3.5-Turbo and Claude-3 Haiku. We release a dataset containing 60,000 high-quality entries, aiming to advance the field of function-calling agent domains. The dataset is available on Huggingface: https://huggingface.co/datasets/Salesforce/xlam-function-calling-60k and the project homepage: https://apigen-pipeline.github.io/
Community
- APIGen is a sophisticated high-quality data synthesis pipeline designed for function-calling and code-based agents.
- We have released a collection of 60,000 verified and diverse datasets, available at Hugging Face Datasets.
- Our results demonstrate that high-quality synthetic data can enable smaller models (7B and 1B) to achieve performance comparable to that of GPT-4 and GPT-3.5, respectively.
- Models are coming soon.
![](https://cdn-uploads.huggingface.co/production/uploads/638e971f2cc490759fe9210e/JtKy0ny7TcKbTNx97K2CV.jpeg)
Models citing this paper 8
Browse 8 models citing this paperDatasets citing this paper 2
Spaces citing this paper 0
No Space linking this paper