AmelieSchreiber
commited on
Commit
•
28bad80
1
Parent(s):
86ae9fd
Update README.md
Browse files
README.md
CHANGED
@@ -6,7 +6,7 @@ license: mit
|
|
6 |
|
7 |
This model is the ESM-2 model [esm2_t12_35M_UR50D](https://huggingface.co/facebook/esm2_t12_35M_UR50D) finetuned with QLoRA on
|
8 |
[this dataset](https://huggingface.co/datasets/AmelieSchreiber/2600K_binding_sites) of 2.6M protein sequences with binding and active
|
9 |
-
site annotations. The model and dataset size were scaled in a one-to-one way (following the Chinchilla paper) up from the smaller
|
10 |
QLoRA adaptations of the `esm2_t6_8M_UR50D` models which were trained on 600K proteins. Since this model is 4.375 times larger, a dataset
|
11 |
approximately 4.375 times larger is needed if Chinchilla scaling laws hold for QLoRA finetuning of protein language models. Determining if
|
12 |
such scaling laws also hold is part of this project, so checking for improvements in performance metrics over a period of 3 epochs, as well
|
|
|
6 |
|
7 |
This model is the ESM-2 model [esm2_t12_35M_UR50D](https://huggingface.co/facebook/esm2_t12_35M_UR50D) finetuned with QLoRA on
|
8 |
[this dataset](https://huggingface.co/datasets/AmelieSchreiber/2600K_binding_sites) of 2.6M protein sequences with binding and active
|
9 |
+
site annotations from UniProt. The model and dataset size were scaled in a one-to-one way (following the Chinchilla paper) up from the smaller
|
10 |
QLoRA adaptations of the `esm2_t6_8M_UR50D` models which were trained on 600K proteins. Since this model is 4.375 times larger, a dataset
|
11 |
approximately 4.375 times larger is needed if Chinchilla scaling laws hold for QLoRA finetuning of protein language models. Determining if
|
12 |
such scaling laws also hold is part of this project, so checking for improvements in performance metrics over a period of 3 epochs, as well
|