To test our data on larger size of models, we also trained **phi-3-medium**, a model with 14B parameters using the same tokenizer and architecture of **phi-3-mini**, and trained on the same data for slightly more epochs (4.8T tokens total as for **phi-3-small**). The model has 40 heads and 40 layers, with embedding dimension 5120. We observe that some benchmarks improve much less from 7B to 14B than they do from 3.8B to 7B, perhaps indicating that our data mixture needs further work to be in the "data optimal regime" for 14B parameters model. We are still actively investigating some of those benchmarks (including a regression on HumanEval), hence the numbers for **phi-3-medium** should be considered as a "preview".