We introducephi-1, a new large language model for code, with significantly smaller size than competing models:phi-1is a Transformer-based model withB parameters, trained fordays onA100s, using a selection of “textbook quality” data from the web (B tokens)  and synthetically generated textbooks and exercises with GPT-3.5 (B tokens) . Despite this small scale,phi-1attainspass@1accuracyon HumanEval andon MBPP. It also displays surprising emergent properties compared tophi-1-base, our modelbeforeour finetuning stage on a dataset of coding exercises, andphi-1-small, a smaller model with 350M parameters trained with the same pipeline asphi-1that still achieveson HumanEval.