Edit model card

Starcoder-2-chat

image/jpeg

Starcoder-2-chat is an instruction fine-tuned of bigcode/starcoder2-7b using the glaiveai/glaive-code-assistant-v2 dataset using LoRA.

πŸ† Evaluation results

Thanks to Muhammad Bin Usman for running evals on Starcoder2-chat.

HUMANEVAL

0.3231707317073171

HUMANEVALPLUS

0.25609756097560976

INSTRUCT-HUMANEVAL

0.3231707317073171

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 5e-7
  • train_batch_size: 2
  • eval_batch_size: Not specified
  • seed: Not specified
  • gradient_accumulation_steps: 8
  • total_train_batch_size: Not specified
  • optimizer: PagedAdamW with 32-bit precision
  • lr_scheduler_type: Cosine
  • lr_scheduler_warmup_steps: 100
  • training_epoch: 1

Framework versions

  • Transformers 4.39.0.dev0
  • Peft 0.9.1.dev0
  • Datasets 2.18.0
  • torch 2.2.0
  • accelerate 0.27.2
Downloads last month
328
Safetensors
Model size
7.17B params
Tensor type
F32
Β·
Inference Examples
This model does not have enough activity to be deployed to Inference API (serverless) yet. Increase its social visibility and check back later, or deploy to Inference Endpoints (dedicated) instead.

Model tree for abideen/starcoder2-chat

Finetuned
(11)
this model

Collection including abideen/starcoder2-chat