philschmid
/

gemma-7b-dolly-chatml

Generated from Trainer

Model card Files Files and versions Metrics Training metrics Community

philschmid HF staff commited on Feb 29

Commit

7c9f7d5

•

1 Parent(s): 0c1fecf

Update trl-lora.py

Files changed (1) hide show

trl-lora.py +1 -1

trl-lora.py CHANGED Viewed

@@ -64,7 +64,7 @@ trainer = SFTTrainer(
     tokenizer=tokenizer,
     packing=True,
     dataset_kwargs={
-        "add_special_tokens": True, # make sure we add <bos> and <eos> tokens
         "append_concat_token": False, # make sure to not add additional tokens when packing
     }
 )

     tokenizer=tokenizer,
     packing=True,
     dataset_kwargs={
+        "add_special_tokens": False, # <bos> and <eos> should be part of the dataset.
         "append_concat_token": False, # make sure to not add additional tokens when packing
     }
 )