BA-Zephyria-39b [EXPERIMENTAL]

Model Information

Base Model: unsloth/Mistral-Small-Instruct-2409

Strategy: Balanced Approach

Total Layers: 55

Duplication Start: Layer 19 (34.5% of model)

Duplicated Layers: 23 (41.8% of model)

Unique Final Layers: 14 (25.5% of model)

Model Characteristics

• Models down_proj and o_proj layers have been nulled and will require healing
• Combines benefits of early and mid duplication strategies
• Balanced between unique initial layers, duplicated middle layers, and unique final layers
• Versatile approach suitable for a wide range of tasks
• Provides substantial unique layers at the end for task-specific adaptations

Configuration Visualization

[    Unique    ][    Duplicated    ][    Unique    ]
0 ----------- 18 19 ------------ 41 42 ---------- 54
     34.5%           41.8%            23.7%