Update README.md

README.md

@@ -10,7 +10,16 @@ tags:
 - biology
 ---
 
+# ESM-2 QLoRA
+
 These are the checkpoints for the first ever QLoRA for ESM-2! They haven't been checked for overfitting yet, so use with caution!
 You can load and use them similarly to the LoRA models. This is the smallest `esm2_t6_8M_UR50D` model, so the metrics aren't great. 
 Scaling to larger models for better metrics is in progress. 
 
+## QLoRA Info
+
+Note, we are only training 0.58% of the parameters, using only the query, key, and value weight matrices. 
+
+```
+trainable params: 23682 || all params: 4075265 || trainable%: 0.5811155838945443
+```