**Knowledge extraction.** The "2bit/param" result is not about word-by-word memorization. Even better, such knowledge is also flexibly extractable (e.g., via fine-tuning using QAs like "What is Anya Forger's birthday?") [3] and thus can be further manipulated in downstream tasks (such as comparing the birthdates of two people, or performing calculations on the retrieved knowledge, etc.) [4]. This is because our \(\mathsf{bioS}(N)\) data is knowledge-augmented: the English biographies have sufficient text diversities [3]. We also verify in Appendix A.2 that such knowledge is extractable.