metadata
base_model: microsoft/Phi-3-mini-4k-instruct
datasets:
- AlignmentLab-AI/alpaca-cot-collection
language:
- en
library_name: peft
license: apache-2.0
pipeline_tag: text-generation
Xenith-3B
Xenith-3B is a fine-tuned language model based on the microsoft/Phi-3-mini-4k-instruct model. It has been specifically trained on the AlignmentLab-AI/alpaca-cot-collection dataset, which focuses on chain-of-thought reasoning and instruction following.
Model Overview
- Model Name: Xenith-3B
- Base Model: microsoft/Phi-3-mini-4k-instruct
- Fine-Tuned On: AlignmentLab-AI/alpaca-cot-collection
- Model Size: 3 Billion parameters
- Architecture: Transformer-based LLM
Training Details
- Objective: Fine-tune the base model to enhance its performance on tasks requiring complex reasoning and multi-step problem-solving.
- Training Duration: 10 epochs
- Batch Size: 8
- Learning Rate: 3e-5
- Optimizer: AdamW
- Hardware Used: 2x NVIDIA L4 GPUs
Performance
Xenith-3B excels in tasks that require:
- Chain-of-thought reasoning
- Instruction following
- Contextual understanding
- Complex problem-solving
- The model has shown significant improvements in these areas compared to the base model.