Update README.md
Browse files
README.md
CHANGED
|
@@ -209,13 +209,6 @@ An experiment investigating transfer learning capabilities by fine-tuning models
|
|
| 209 |
|
| 210 |
This model is a fine-tuned version of [pszemraj/long-t5-tglobal-base-16384-book-summary](https://huggingface.co/pszemraj/long-t5-tglobal-base-16384-book-summary) on the `pszemraj/scientific_lay_summarisation-elife-norm` dataset for two epochs.
|
| 211 |
|
| 212 |
-
It achieves the following results on the evaluation set:
|
| 213 |
-
- Loss: 2.3994
|
| 214 |
-
- Rouge1: 34.2428
|
| 215 |
-
- Rouge2: 4.3644
|
| 216 |
-
- Rougel: 12.5332
|
| 217 |
-
- Rougelsum: 30.6965
|
| 218 |
-
- Gen Len: 294.0249
|
| 219 |
|
| 220 |
|
| 221 |
## Usage
|
|
@@ -245,6 +238,18 @@ print(summary)
|
|
| 245 |
|
| 246 |
## Training procedure
|
| 247 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 248 |
### Training hyperparameters
|
| 249 |
|
| 250 |
The following hyperparameters were used during training:
|
|
|
|
| 209 |
|
| 210 |
This model is a fine-tuned version of [pszemraj/long-t5-tglobal-base-16384-book-summary](https://huggingface.co/pszemraj/long-t5-tglobal-base-16384-book-summary) on the `pszemraj/scientific_lay_summarisation-elife-norm` dataset for two epochs.
|
| 211 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 212 |
|
| 213 |
|
| 214 |
## Usage
|
|
|
|
| 238 |
|
| 239 |
## Training procedure
|
| 240 |
|
| 241 |
+
> Note: this model was trained at a lower LR & not till "absolute convergence" with the intention of retaining some of the properties learned from the initial fine-tuning on `booksum`
|
| 242 |
+
|
| 243 |
+
### Results
|
| 244 |
+
|
| 245 |
+
It achieves the following results on the evaluation set:
|
| 246 |
+
- Loss: 2.3994
|
| 247 |
+
- Rouge1: 34.2428
|
| 248 |
+
- Rouge2: 4.3644
|
| 249 |
+
- Rougel: 12.5332
|
| 250 |
+
- Rougelsum: 30.6965
|
| 251 |
+
- Gen Len: 294.0249
|
| 252 |
+
|
| 253 |
### Training hyperparameters
|
| 254 |
|
| 255 |
The following hyperparameters were used during training:
|