@inproceedings{
wen2023grounding,
title={Grounding Code Generation with Input-Output Specifications},
author={Yeming Wen and Pengcheng Yin and Kensen Shi and Henryk Michalewski and Swarat Chaudhuri and Alex Polozov},
booktitle={NeurIPS 2023 Workshop on Instruction Tuning and Instruction Following},
year={2023},
url={https://openreview.net/forum?id=wq4OaU8tfE}
}
Large language models (LLMs) have demonstrated significant potential in code generation. However, the code generated by these models occasionally deviates from the user's intended outcome, resulting in executable but incorrect code. To mitigate this issue, we propose Gift4Code, a novel approach for the instruction fine-tuning of LLMs specifically tailored for code generation. Our method leverages synthetic data produced by the LLM itself and utilizes execution-derived feedback as a key learning signal. This feedback, in the form of program input-output specifications, is provided to the LLM to facilitate fine-tuning. We evaluated our approach on two challenging data science benchmarks, Arcade and DS-1000. Our results suggest that the method enhances the LLM's alignment with user intentions, reducing the incidence of executable but incorrect outputs.
Grounding Code Generation with Input-Output Specifications
Yeming Wen Pengcheng Yin Kensen Shi Henryk Michalewski Swarat Chaudhuri Alex Polozov