parameters, our \(\mathsf{bioS}(N)\) data utilizes only \(3275\) tokens (after applying GPT2's tokenizer), reducing the effective embedding layer size to \(3275\times(64h)\). This adjustment explains why, for \(\mathsf{bioS}\) data, GPT2small, typically known to have \(124\)M parameters, is counted as having only \(88\)M parameters in this paper.