Training Methodology.We follow the sequence of works initiated in "Textbooks Are All You Need" [GZA\({}^{+}\)23], which utilize high quality training data to improve the performance of small language models and deviate from the standard _scaling-laws_. In this work we show that such method allows to reach the level of highly capable models such as GPT-3.5 or Mistral with only 3.8B total parameters (while Mistral has 45B total parameters for example). Our training data of consists of heavily filtered web data (according to the "educational level") from various open internet sources, as well as synthetic LLM-generated data. Pre-training is performed in two disjoint and sequential phases; phase-1 comprises mostly of web sources aimed at teaching the model general knowledge and language understanding. Phase-2 merges even more heavily filtered webdata (a subset used in Phase-1) with some synthetic data that teach the model logical reasoning and various niche skills.