Update README.md
#4
by
cyente
- opened
README.md
CHANGED
@@ -27,7 +27,7 @@ Qwen2.5-Coder is the latest series of Code-Specific Qwen large language models (
|
|
27 |
|
28 |
**This repo contains the 1.5B Qwen2.5-Coder model**, which has the following features:
|
29 |
- Type: Causal Language Models
|
30 |
-
- Training Stage: Pretraining
|
31 |
- Architecture: transformers with RoPE, SwiGLU, RMSNorm, Attention QKV bias and tied word embeddings
|
32 |
- Number of Parameters: 1.54B
|
33 |
- Number of Paramaters (Non-Embedding): 1.31B
|
|
|
27 |
|
28 |
**This repo contains the 1.5B Qwen2.5-Coder model**, which has the following features:
|
29 |
- Type: Causal Language Models
|
30 |
+
- Training Stage: Pretraining
|
31 |
- Architecture: transformers with RoPE, SwiGLU, RMSNorm, Attention QKV bias and tied word embeddings
|
32 |
- Number of Parameters: 1.54B
|
33 |
- Number of Paramaters (Non-Embedding): 1.31B
|