Overall, our approach to studying knowledge capacity scaling laws offers a flexible and **more accurate playground** compared to traditional methods that evaluate language models trained on internet data against real-world benchmarks. This accuracy is partly due to the synthetic nature of our dataset, which eliminates concerns about benchmark contamination that could compromise the validity of real-world benchmark results. In this paper, we've conducted a thorough comparison across different model architectures and types of knowledge. While we haven't explored various quantization methods, this represents a promising direction for future research. We've also investigated the impact of junk data and proposed mitigation strategies. We believe the insights gained from this principled exploration can assist practitioners in making informed decisions about model selection, training data preparation, and further theoretical research into LLMs.