jrc commited on
Commit
d5fac40
1 Parent(s): 4e2781a

Update README.md

Browse files

# Llama3 CoEdit

This is a Llama3 8B based model trained using [torchtune](https://pytorch.org/torchtune) on the `grammarly/coedit` dataset.

### Training details

The exact training script (`lora_finetune_distributed`) and config (`8B_lora.yaml`) are both included in this repository. Specifically, in order to add the dataset, I added the following lines to the config:

```
dataset:
_component_: torchtune.datasets.instruct_dataset
source: grammarly/coedit
template: GrammarErrorCorrectionTemplate
column_map: {"sentence": "src", "output": "tgt"}
train_on_input: False
split: train
```

### Evaluation results

Files changed (1) hide show
  1. README.md +1 -1
README.md CHANGED
@@ -8,5 +8,5 @@ metrics:
8
  - accuracy
9
  tags:
10
  - torchtune
11
- - grammar
12
  ---
 
8
  - accuracy
9
  tags:
10
  - torchtune
11
+ - grammar-correction
12
  ---