Linaqruf commited on
Commit
b2ea1b4
1 Parent(s): fa0a3e5

fix some human error in defining training config

Browse files
Files changed (1) hide show
  1. README.md +4 -4
README.md CHANGED
@@ -217,16 +217,16 @@ These are the key hyperparameters used during training:
217
  | Feature | Pretraining | Finetuning |
218
  |-------------------------------|----------------------------|---------------------------------|
219
  | **Hardware** | 2x H100 80GB PCIe | 1x A100 80GB PCIe |
220
- | **Batch Size** | 64 | 48 |
221
  | **Gradient Accumulation Steps** | 2 | 1 |
222
  | **Noise Offset** | None | 0.0357 |
223
  | **Epochs** | 10 | 10 |
224
  | **UNet Learning Rate** | 5e-6 | 3.75e-6 |
225
  | **Text Encoder Learning Rate** | 2.5e-6 | None |
226
- | **Optimizer** | AdamW8bit | Adafactor |
227
- | **Optimizer Args** | Weight Decay: 0.1, Betas: (0.9, 0.99) | Scale Parameter: False, Relative Step: False, Warmup Init: False |
228
  | **Scheduler** | Constant with Warmups | Constant with Warmups |
229
- | **Warmup Steps** | 0.5% | 0.5% |
230
 
231
  ## License
232
 
 
217
  | Feature | Pretraining | Finetuning |
218
  |-------------------------------|----------------------------|---------------------------------|
219
  | **Hardware** | 2x H100 80GB PCIe | 1x A100 80GB PCIe |
220
+ | **Batch Size** | 32 | 48 |
221
  | **Gradient Accumulation Steps** | 2 | 1 |
222
  | **Noise Offset** | None | 0.0357 |
223
  | **Epochs** | 10 | 10 |
224
  | **UNet Learning Rate** | 5e-6 | 3.75e-6 |
225
  | **Text Encoder Learning Rate** | 2.5e-6 | None |
226
+ | **Optimizer** | Adafactor | Adafactor |
227
+ | **Optimizer Args** | Scale Parameter: False, Relative Step: False, Warmup Init: False (0.9, 0.99) | Scale Parameter: False, Relative Step: False, Warmup Init: False |
228
  | **Scheduler** | Constant with Warmups | Constant with Warmups |
229
+ | **Warmup Steps** | 0.05% | 0.05% |
230
 
231
  ## License
232