Junrulu
/

Llama-3-8B-Instruct-Iterative-SamPO

Text Generation

text-generation-inference

Inference Endpoints

Model card Files Files and versions Community

Junrulu commited on Jun 14

Commit

b9e22c6

•

1 Parent(s): 35869f7

Update README.md

Files changed (1) hide show

README.md +2 -2

README.md CHANGED Viewed

@@ -52,8 +52,8 @@ The model is trained to use the following format:
 The following hyperparameters were used during DPO/SamPO training:
 - DPO beta: 0.1
-- learning_rate: 4e-7 * sqrt(Num of Nodes)
-- total_train_batch_size: 128 * Num of Nodes
 - optimizer: AdamW with beta1 0.9, beta2 0.999 and epsilon 1e-8
 - lr_scheduler_type: linear
 - lr_scheduler_warmup_ratio: 0.1

 The following hyperparameters were used during DPO/SamPO training:
 - DPO beta: 0.1
+- learning_rate: 4e-7
+- total_train_batch_size: 128
 - optimizer: AdamW with beta1 0.9, beta2 0.999 and epsilon 1e-8
 - lr_scheduler_type: linear
 - lr_scheduler_warmup_ratio: 0.1