File size: 1,268 Bytes
59c69b6 29cd99c c464d9c 59c69b6 477e7d5 c464d9c 477e7d5 59c69b6 477e7d5 29cd99c |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 |
---
library_name: transformers
license: bigcode-openrail-m
tags:
- code
---
# Model Card for StarCoderBase1B-Racket-SelfInstruct
Each commit to this repository has a checkpoint (one per epoch) for a fine-tuned
StarCoderBase-1B. The dataset for fine-tuning is a Racket self-instruction
dataset. As shown in Evaluation below, self-instruction was not effective, and
this model is barely any better than StarCoderBase-1B.
## Finetuning Dataset and Hyperparameters
- Dataset: https://huggingface.co/datasets/nuprl/MultiPL-T-racket-selfinstruct
- Base Model: https://huggingface.co/bigcode/starcoderbase-1b
- Batch size: 8
- Epochs: 7
- Learning rate: 3e-05
- Warmup steps: 100
- Sequence length: 2048
## Evaluation
The results on MultiPL-HumanEval-Racket are as follows:
```
Dataset,Pass@k,Estimate,NumProblems,MinCompletions,MaxCompletions
humaneval-rkt-checkpoint_1494-0.2-reworded,1,7.70,161,50,50
humaneval-rkt-checkpoint_1992-0.2-reworded,1,6.86,161,50,50
humaneval-rkt-checkpoint_2490-0.2-reworded,1,6.82,161,50,50
humaneval-rkt-checkpoint_2988-0.2-reworded,1,6.91,161,50,50
humaneval-rkt-checkpoint_498-0.2-reworded,1,6.19,161,50,50
humaneval-rkt-checkpoint_6973-0.2-reworded,1,6.53,161,50,50
humaneval-rkt-checkpoint_996-0.2-reworded,1,7.08,161,50,50
``` |