esm2_t12_35M-lora-binding-sites_2024-04-25_17-02-01

This model is a fine-tuned version of facebook/esm2_t12_35M_UR50D on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.3783
• Accuracy: 0.8613

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0005701568055793089
• train_batch_size: 64
• eval_batch_size: 64
• seed: 8893
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: cosine
• num_epochs: 30

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy
0.6443	1.0	24	0.6716	0.5977
0.6838	2.0	48	0.6690	0.5977
0.5906	3.0	72	0.6440	0.5977
0.585	4.0	96	0.5897	0.7012
0.6142	5.0	120	0.5522	0.6992
0.61	6.0	144	0.5307	0.7148
0.5131	7.0	168	0.5134	0.75
0.365	8.0	192	0.4437	0.8105
0.4282	9.0	216	0.4010	0.8379
0.3102	10.0	240	0.3643	0.8516
0.41	11.0	264	0.3624	0.8652
0.317	12.0	288	0.3877	0.8145
0.2332	13.0	312	0.3383	0.8672
0.2698	14.0	336	0.3389	0.8672
0.1326	15.0	360	0.3548	0.8535
0.1187	16.0	384	0.3313	0.8770
0.1597	17.0	408	0.3447	0.8652
0.1886	18.0	432	0.3543	0.8672
0.1883	19.0	456	0.3396	0.875
0.1156	20.0	480	0.3510	0.8652
0.2604	21.0	504	0.3529	0.875
0.1104	22.0	528	0.3654	0.8711
0.1723	23.0	552	0.3631	0.8691
0.1516	24.0	576	0.3719	0.8633
0.0731	25.0	600	0.3745	0.8672
0.1381	26.0	624	0.3765	0.8691
0.291	27.0	648	0.3760	0.8613
0.0924	28.0	672	0.3789	0.8633
0.2029	29.0	696	0.3783	0.8594
0.242	30.0	720	0.3783	0.8613

Framework versions

• PEFT 0.10.0
• Transformers 4.39.3
• Pytorch 2.2.1
• Datasets 2.16.1
• Tokenizers 0.15.2