AmelieSchreiber
commited on
Commit
•
508e6f9
1
Parent(s):
499f3ab
Update README.md
Browse files
README.md
CHANGED
@@ -14,7 +14,9 @@ tags:
|
|
14 |
|
15 |
These are the checkpoints for the first ever QLoRA for ESM-2! They haven't been checked for overfitting yet, so use with caution!
|
16 |
You can load and use them similarly to the LoRA models. This is the smallest `esm2_t6_8M_UR50D` model, so the metrics aren't great.
|
17 |
-
Scaling to larger models for better metrics is in progress. These checkpoints were trained using
|
|
|
|
|
18 |
|
19 |
## QLoRA Info
|
20 |
|
|
|
14 |
|
15 |
These are the checkpoints for the first ever QLoRA for ESM-2! They haven't been checked for overfitting yet, so use with caution!
|
16 |
You can load and use them similarly to the LoRA models. This is the smallest `esm2_t6_8M_UR50D` model, so the metrics aren't great.
|
17 |
+
Scaling to larger models for better metrics is in progress. These checkpoints were trained using
|
18 |
+
[the 600K dataset](https://huggingface.co/datasets/AmelieSchreiber/600K_data). To replicate the training of QLoRA for ESM-2 models,
|
19 |
+
you can use the `conda-environment.yml` file.
|
20 |
|
21 |
## QLoRA Info
|
22 |
|