mylesgoose commited on
Commit
4eb7274
1 Parent(s): a75702a

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +4 -3
README.md CHANGED
@@ -390,8 +390,9 @@ Dapper and dignified
390
  Country cottage charm<|eot_id|>
391
 
392
  and so the model learns to do this in its first output:
 
393
  <|eot_id|><|start_header_id|>assistant<|end_header_id|>
394
- which naturally messes up the training. can you please put a new line character after the eot_id or prior to the start header id in the chat template: so that the format is like so :
395
  <|begin_of_text|><|start_header_id|>user<|end_header_id|>
396
 
397
  <|image|>If I had to write a haiku for this one, it would be: <|eot_id|>
@@ -411,8 +412,8 @@ If I had to write a haiku for this one, it would be:<|eot_id|>#notice that this
411
  <|start_header_id|>assistant<|end_header_id|>
412
 
413
  ['A rabbit on a sunny day']
414
- this is an example of the 3.1 models chat template. i have not examined your one yet however i have examined the output of it above. to prevent the model learning that eot comes first there need to be a clearer distinction made with a \n
415
-
416
 
417
  Llama 3.2 Version Release Date: September 25, 2024
418
 
 
390
  Country cottage charm<|eot_id|>
391
 
392
  and so the model learns to do this in its first output:
393
+
394
  <|eot_id|><|start_header_id|>assistant<|end_header_id|>
395
+ which naturally messes up the training. so i put a new line character after the eot_id or prior to the start header id in the chat template: so that the format is like so :
396
  <|begin_of_text|><|start_header_id|>user<|end_header_id|>
397
 
398
  <|image|>If I had to write a haiku for this one, it would be: <|eot_id|>
 
412
  <|start_header_id|>assistant<|end_header_id|>
413
 
414
  ['A rabbit on a sunny day']
415
+ this is an example of the 3.1 models chat template.. to prevent the model learning that eot comes first there need to be a clearer distinction made with a \n
416
+ which Is done in this template
417
 
418
  Llama 3.2 Version Release Date: September 25, 2024
419