license:cc-by-4.0 code region:us
---
license: cc-by-4.0
tags:
- code
---
# Dataset Card for Pipeline2Code

## Dataset origin
[Code4ML: a Large-scale Dataset of annotated Machine Learning Code](https://zenodo.org/record/7733823)

## Dataset Summary

This dataset is designed for the iterative generation of Machine Learning (ML) code based on high-level ML pipeline descriptions.
It consists of code snippets extracted from Kaggle kernels, organized as Jupyter Notebook snippets. 
Each kernel includes a set of prompts and completions. 
The initial prompt contains an <SOS> token, meta-information about the task the notebook aims 
to solve, and the semantic type of the code snippet. The final prompt of each kernel consists of the semantic 
type of the code snippet followed by an <EOS> token. Each prompt is associated with a code snippet completion. 
Subsequent prompts include previously generated completions and the semantic type of the snippet.