fine_tuned_gpt2_clm-model

This model is a fine-tuned version of gpt2 on the eli5 dataset. It achieves the following results on the evaluation set:

• Loss: 3.3066

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 15

Training results

Training Loss	Epoch	Step	Validation Loss
No log	1.0	142	3.3422
No log	2.0	284	3.3226
No log	3.0	426	3.3148
3.4352	4.0	568	3.3095
3.4352	5.0	710	3.3074
3.4352	6.0	852	3.3066
3.4352	7.0	994	3.3046
3.3068	8.0	1136	3.3049
3.3068	9.0	1278	3.3048
3.3068	10.0	1420	3.3050
3.2433	11.0	1562	3.3062
3.2433	12.0	1704	3.3059
3.2433	13.0	1846	3.3062
3.2433	14.0	1988	3.3065
3.2113	15.0	2130	3.3066

Inference:

• prompt = "dna phosphorylation is the process of"

• generated Text: dna phosphorylation is the process of forming the deoxygenated product. For example, in a protein phosphorylation inhibitor, it occurs to deoxygenate the phosphorylated protein by binding a phosphate molecule and preventing it from being destroyed by a nonenzymatic process.

In a phosphorylation inhibitor like dna, the product is phosphorylated by the phosphocreatine, a phosphorylated phosphocreatine molecule that can bind to other phosphocreatine molecules that bind to phosphocreatine. This interaction helps to separate the phosphocreatine molecule that is phosphorylated from the phosphocreatine-phosphocreatine-phosphocreatine-phosphocreatine-glucose molecule that is phosphocreatine-phosphocreatine-glucose-phosphocreatine-phosphocreatine-glucose.

In anoxidase inhibitors like dna, they are a bit more specific, more specific, and have a more complicated interaction with the phosphocreatine molecule that can bind to phosphocreatine molecules.

I would argue that both dna-and phosphocreatine-phosphocreatine-glucose will not be able to bind to phosphocreatine because the phosphocreatine-phosphocreatine-phosphocreatine-glucose-phosphocreatine molecule that was phosphocreatine-phosphocreatine-phosphocreatine-phosphocreatine-phosphocreatine-phosphocreatine-phosphocreatine-glucose, is phosphocreatine.

That is, dna-and phosphocreatine-glucose will be able to bind to phosphocreatine because the phosphocreatine molecule that was phosphocreatine-glucose will not be phosphocreatine because the phosphocreatine-phosphocreatine-glucose molecule that was phosphocreatine-phosphocreatine-phosphocreatine-phosphocreatine-glucose, is phosphocreatine.

Edit: Added: The final point is that it can't bind phosphocreatine because that phosphocreatine molecule (a phosphocreatine-phosphocreatine-phosphocreatine-phosphocreatine molecule) can not be phosphoc

Evaluation metric:

Perplexity: 27.29

GPU:

• CUDA Version: 12.1
• 4x Tesla T4

Framework versions

• Transformers 4.34.0
• Pytorch 2.1.0+cu121
• Datasets 2.14.5
• Tokenizers 0.14.1