The striking progress of AI in the last few years can be largely attributed to major efforts throughout the world towards _scaling-up_ to ever-larger models and datasets. Large Language Models (LLMs) have steadily increased in size from a mere billion parameters just five years ago (GPT-2 had 1.5 billion parameters [RWC\({}^{+}\)19]) to trillion parameters today. The impetus for this effort originates in the seemingly predictable improvement one obtains by training large models, the so-called _scaling laws_[KMH\({}^{+}\)20, HBM\({}^{+}\)22, MRB\({}^{+}\)23]. However these laws assume a "fixed" data source. This assumption is now significantly disrupted by the existence of frontier LLMs themselves, which allow us to interact with data in novel ways. In our previous works on the phi models [GZA\({}^{+}\)23, LBE\({}^{+}\)23, JBA\({}^{+}\)23] it was shown that a combination of LLM-based filtering of web data, and LLM-created synthetic data, enable performance in smaller language models that were typically seen only in much larger models. For example our previous model trained on this data recipe, **phi-2** (2.7B parameters), matched the performance of models 25 times larger trained on regular data. In this report we present a new model, **phi-3-mini** (3.8B parameters), trained for 3.3T tokens on larger and more advanced versions of the datasets used in **phi-2**. With its small size, **phi-3-mini** can easily be inferenced locally on a modern phone (see Figure 1), yet it achieves a quality that seems on-par with models such as Miktral 8x7B [JSR\({}^{+}\)24] and GPT-3.5.