The **phi-3-mini** model is a transformer decoder architecture [VSP\({}^{+}\)17], with default context length \(4K\). We also introduce a long context version via LongRope [DZZ\({}^{+}\)24] that extends the context length to \(128K\), called **phi-3-mini-128K**.