While large language models (LLMs)  are increasingly being used for program synthesis, they lack the global view needed to develop useful abstractions; they generally predict programs one at a time, often repeating the same functionality.
Generating redundant code from scratch is both inefficient and error-prone.
To address this, we proposeRefactoring forGeneralizableAbstractionLearning (ReGAL) , a gradient-free method for learning a library ofreusablefunctions via coderefactorization, i.e. restructuring code without changing its execution output.ReGALlearns from a small set of existing programs, iteratively verifying and refining its abstractions via execution.
We find that the shared function libraries discovered byReGALmake programseasier to predictacross diverse domains.
On three datasets (LOGO graphics generation, Date reasoning, and TextCraft, a Minecraft-based text-game) , both open-source and proprietary LLMs improve in accuracy when predicting programs withReGALfunctions.
For CodeLlama-13B,ReGALresults in absolute accuracy increases ofon graphics,on date understanding, andon TextCraft, outperforming GPT-3.5 in two of three domains.
Our analysis revealsReGAL’s abstractions encapsulate frequently-used subroutines as well as environment dynamics.111Code available athttps://github.com/esteng/regal_program_learning