Update README.md
Browse files
README.md
CHANGED
@@ -75,20 +75,6 @@ As you can see, my model performs better in maximum case :D
|
|
75 |
|
76 |
## Training procedure
|
77 |
|
78 |
-
#### Model hyperparameters
|
79 |
-
```python
|
80 |
-
model = Wav2Vec2ForCTC.from_pretrained(
|
81 |
-
model_name_or_path if not last_checkpoint else last_checkpoint,
|
82 |
-
# hp-mehrdad: Hyperparams of 'm3hrdadfi/wav2vec2-large-xlsr-persian-v3'
|
83 |
-
attention_dropout = 0.05316,
|
84 |
-
hidden_dropout = 0.01941,
|
85 |
-
feat_proj_dropout = 0.01249,
|
86 |
-
mask_time_prob = 0.04529,
|
87 |
-
layerdrop = 0.01377,
|
88 |
-
ctc_loss_reduction = 'mean',
|
89 |
-
ctc_zero_infinity = True,
|
90 |
-
)
|
91 |
-
```
|
92 |
#### Training hyperparameters
|
93 |
|
94 |
The following hyperparameters were used during training:
|
@@ -133,7 +119,7 @@ The following hyperparameters were used during training:
|
|
133 |
Several models with differet hyperparameters were trained. The following figures show the training process for three of them.
|
134 |
![wer](wandb-wer.png)
|
135 |
![loss](wandb-loss.png)
|
136 |
-
'20_2000_1e-5_hp-mehrdad' is the current model and it's
|
137 |
```python
|
138 |
model = Wav2Vec2ForCTC.from_pretrained(
|
139 |
model_name_or_path if not last_checkpoint else last_checkpoint,
|
@@ -146,8 +132,6 @@ model = Wav2Vec2ForCTC.from_pretrained(
|
|
146 |
ctc_loss_reduction = 'mean',
|
147 |
ctc_zero_infinity = True,
|
148 |
)
|
149 |
-
|
150 |
-
learning_rate = 1e-5
|
151 |
```
|
152 |
The hyperparameters of '19_2000_1e-5_hp-base' are:
|
153 |
```python
|
@@ -162,8 +146,6 @@ model = Wav2Vec2ForCTC.from_pretrained(
|
|
162 |
ctc_loss_reduction = 'mean',
|
163 |
ctc_zero_infinity = True,
|
164 |
)
|
165 |
-
|
166 |
-
learing_rate = 1e-5
|
167 |
```
|
168 |
|
169 |
And the hyperparameters of '22_2000_1e-5_hp-masoud' are:
|
@@ -179,9 +161,8 @@ model = Wav2Vec2ForCTC.from_pretrained(
|
|
179 |
ctc_loss_reduction = 'mean',
|
180 |
ctc_zero_infinity = True,
|
181 |
)
|
182 |
-
|
183 |
-
learning_rate = 1e-5
|
184 |
```
|
|
|
185 |
As you can see this model performs better with WER metric on validation(evaluation) set.
|
186 |
|
187 |
#### Framework versions
|
|
|
75 |
|
76 |
## Training procedure
|
77 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
78 |
#### Training hyperparameters
|
79 |
|
80 |
The following hyperparameters were used during training:
|
|
|
119 |
Several models with differet hyperparameters were trained. The following figures show the training process for three of them.
|
120 |
![wer](wandb-wer.png)
|
121 |
![loss](wandb-loss.png)
|
122 |
+
'20_2000_1e-5_hp-mehrdad' is the current model and it's hyperparameters are:
|
123 |
```python
|
124 |
model = Wav2Vec2ForCTC.from_pretrained(
|
125 |
model_name_or_path if not last_checkpoint else last_checkpoint,
|
|
|
132 |
ctc_loss_reduction = 'mean',
|
133 |
ctc_zero_infinity = True,
|
134 |
)
|
|
|
|
|
135 |
```
|
136 |
The hyperparameters of '19_2000_1e-5_hp-base' are:
|
137 |
```python
|
|
|
146 |
ctc_loss_reduction = 'mean',
|
147 |
ctc_zero_infinity = True,
|
148 |
)
|
|
|
|
|
149 |
```
|
150 |
|
151 |
And the hyperparameters of '22_2000_1e-5_hp-masoud' are:
|
|
|
161 |
ctc_loss_reduction = 'mean',
|
162 |
ctc_zero_infinity = True,
|
163 |
)
|
|
|
|
|
164 |
```
|
165 |
+
Learning rate is 1e-5 for all three models.
|
166 |
As you can see this model performs better with WER metric on validation(evaluation) set.
|
167 |
|
168 |
#### Framework versions
|