MIDICausalFinetuning4

This model is a fine-tuned version of on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.0222

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 500

Training results

Training Loss	Epoch	Step	Validation Loss
No log	1.0	1	4.7564
No log	2.0	2	4.2796
No log	3.0	3	3.9949
No log	4.0	4	3.7963
No log	5.0	5	3.6377
No log	6.0	6	3.5060
No log	7.0	7	3.3965
No log	8.0	8	3.2999
No log	9.0	9	3.2098
No log	10.0	10	3.1233
No log	11.0	11	3.0391
No log	12.0	12	2.9529
No log	13.0	13	2.8613
No log	14.0	14	2.7629
No log	15.0	15	2.6630
No log	16.0	16	2.5652
No log	17.0	17	2.4777
No log	18.0	18	2.4026
No log	19.0	19	2.3401
No log	20.0	20	2.2842
No log	21.0	21	2.2316
No log	22.0	22	2.1806
No log	23.0	23	2.1304
No log	24.0	24	2.0802
No log	25.0	25	2.0289
No log	26.0	26	1.9770
No log	27.0	27	1.9237
No log	28.0	28	1.8690
No log	29.0	29	1.8143
No log	30.0	30	1.7591
No log	31.0	31	1.7008
No log	32.0	32	1.6392
No log	33.0	33	1.5777
No log	34.0	34	1.5191
No log	35.0	35	1.4578
No log	36.0	36	1.3959
No log	37.0	37	1.3435
No log	38.0	38	1.2901
No log	39.0	39	1.2289
No log	40.0	40	1.1688
No log	41.0	41	1.1256
No log	42.0	42	1.0843
No log	43.0	43	1.0364
No log	44.0	44	0.9923
No log	45.0	45	0.9577
No log	46.0	46	0.9238
No log	47.0	47	0.8884
No log	48.0	48	0.8536
No log	49.0	49	0.8185
No log	50.0	50	0.7881
No log	51.0	51	0.7595
No log	52.0	52	0.7320
No log	53.0	53	0.7073
No log	54.0	54	0.6851
No log	55.0	55	0.6594
No log	56.0	56	0.6342
No log	57.0	57	0.6123
No log	58.0	58	0.5941
No log	59.0	59	0.5731
No log	60.0	60	0.5497
No log	61.0	61	0.5318
No log	62.0	62	0.5164
No log	63.0	63	0.4998
No log	64.0	64	0.4823
No log	65.0	65	0.4658
No log	66.0	66	0.4524
No log	67.0	67	0.4396
No log	68.0	68	0.4273
No log	69.0	69	0.4139
No log	70.0	70	0.4016
No log	71.0	71	0.3905
No log	72.0	72	0.3796
No log	73.0	73	0.3703
No log	74.0	74	0.3607
No log	75.0	75	0.3503
No log	76.0	76	0.3403
No log	77.0	77	0.3316
No log	78.0	78	0.3226
No log	79.0	79	0.3131
No log	80.0	80	0.3053
No log	81.0	81	0.2981
No log	82.0	82	0.2909
No log	83.0	83	0.2849
No log	84.0	84	0.2810
No log	85.0	85	0.2748
No log	86.0	86	0.2674
No log	87.0	87	0.2595
No log	88.0	88	0.2523
No log	89.0	89	0.2461
No log	90.0	90	0.2407
No log	91.0	91	0.2358
No log	92.0	92	0.2298
No log	93.0	93	0.2245
No log	94.0	94	0.2207
No log	95.0	95	0.2171
No log	96.0	96	0.2133
No log	97.0	97	0.2097
No log	98.0	98	0.2066
No log	99.0	99	0.2025
No log	100.0	100	0.1974
No log	101.0	101	0.1926
No log	102.0	102	0.1886
No log	103.0	103	0.1853
No log	104.0	104	0.1818
No log	105.0	105	0.1788
No log	106.0	106	0.1756
No log	107.0	107	0.1718
No log	108.0	108	0.1680
No log	109.0	109	0.1643
No log	110.0	110	0.1619
No log	111.0	111	0.1594
No log	112.0	112	0.1563
No log	113.0	113	0.1535
No log	114.0	114	0.1506
No log	115.0	115	0.1468
No log	116.0	116	0.1431
No log	117.0	117	0.1401
No log	118.0	118	0.1382
No log	119.0	119	0.1365
No log	120.0	120	0.1345
No log	121.0	121	0.1325
No log	122.0	122	0.1303
No log	123.0	123	0.1277
No log	124.0	124	0.1250
No log	125.0	125	0.1223
No log	126.0	126	0.1189
No log	127.0	127	0.1156
No log	128.0	128	0.1127
No log	129.0	129	0.1102
No log	130.0	130	0.1077
No log	131.0	131	0.1054
No log	132.0	132	0.1038
No log	133.0	133	0.1030
No log	134.0	134	0.1023
No log	135.0	135	0.1007
No log	136.0	136	0.0984
No log	137.0	137	0.0958
No log	138.0	138	0.0938
No log	139.0	139	0.0933
No log	140.0	140	0.0916
No log	141.0	141	0.0892
No log	142.0	142	0.0870
No log	143.0	143	0.0851
No log	144.0	144	0.0835
No log	145.0	145	0.0822
No log	146.0	146	0.0809
No log	147.0	147	0.0805
No log	148.0	148	0.0799
No log	149.0	149	0.0789
No log	150.0	150	0.0770
No log	151.0	151	0.0751
No log	152.0	152	0.0740
No log	153.0	153	0.0737
No log	154.0	154	0.0731
No log	155.0	155	0.0719
No log	156.0	156	0.0702
No log	157.0	157	0.0686
No log	158.0	158	0.0677
No log	159.0	159	0.0674
No log	160.0	160	0.0672
No log	161.0	161	0.0669
No log	162.0	162	0.0663
No log	163.0	163	0.0654
No log	164.0	164	0.0644
No log	165.0	165	0.0634
No log	166.0	166	0.0620
No log	167.0	167	0.0606
No log	168.0	168	0.0595
No log	169.0	169	0.0589
No log	170.0	170	0.0583
No log	171.0	171	0.0578
No log	172.0	172	0.0573
No log	173.0	173	0.0572
No log	174.0	174	0.0574
No log	175.0	175	0.0572
No log	176.0	176	0.0566
No log	177.0	177	0.0556
No log	178.0	178	0.0545
No log	179.0	179	0.0535
No log	180.0	180	0.0529
No log	181.0	181	0.0526
No log	182.0	182	0.0521
No log	183.0	183	0.0515
No log	184.0	184	0.0510
No log	185.0	185	0.0507
No log	186.0	186	0.0507
No log	187.0	187	0.0505
No log	188.0	188	0.0499
No log	189.0	189	0.0492
No log	190.0	190	0.0486
No log	191.0	191	0.0480
No log	192.0	192	0.0476
No log	193.0	193	0.0471
No log	194.0	194	0.0468
No log	195.0	195	0.0464
No log	196.0	196	0.0461
No log	197.0	197	0.0457
No log	198.0	198	0.0453
No log	199.0	199	0.0449
No log	200.0	200	0.0445
No log	201.0	201	0.0442
No log	202.0	202	0.0439
No log	203.0	203	0.0437
No log	204.0	204	0.0434
No log	205.0	205	0.0430
No log	206.0	206	0.0426
No log	207.0	207	0.0423
No log	208.0	208	0.0419
No log	209.0	209	0.0415
No log	210.0	210	0.0412
No log	211.0	211	0.0409
No log	212.0	212	0.0406
No log	213.0	213	0.0403
No log	214.0	214	0.0402
No log	215.0	215	0.0400
No log	216.0	216	0.0398
No log	217.0	217	0.0395
No log	218.0	218	0.0392
No log	219.0	219	0.0389
No log	220.0	220	0.0387
No log	221.0	221	0.0385
No log	222.0	222	0.0384
No log	223.0	223	0.0382
No log	224.0	224	0.0380
No log	225.0	225	0.0376
No log	226.0	226	0.0373
No log	227.0	227	0.0369
No log	228.0	228	0.0366
No log	229.0	229	0.0364
No log	230.0	230	0.0362
No log	231.0	231	0.0361
No log	232.0	232	0.0359
No log	233.0	233	0.0358
No log	234.0	234	0.0357
No log	235.0	235	0.0355
No log	236.0	236	0.0353
No log	237.0	237	0.0351
No log	238.0	238	0.0349
No log	239.0	239	0.0347
No log	240.0	240	0.0346
No log	241.0	241	0.0345
No log	242.0	242	0.0343
No log	243.0	243	0.0341
No log	244.0	244	0.0340
No log	245.0	245	0.0338
No log	246.0	246	0.0337
No log	247.0	247	0.0336
No log	248.0	248	0.0336
No log	249.0	249	0.0335
No log	250.0	250	0.0334
No log	251.0	251	0.0333
No log	252.0	252	0.0331
No log	253.0	253	0.0329
No log	254.0	254	0.0327
No log	255.0	255	0.0324
No log	256.0	256	0.0322
No log	257.0	257	0.0320
No log	258.0	258	0.0319
No log	259.0	259	0.0318
No log	260.0	260	0.0317
No log	261.0	261	0.0316
No log	262.0	262	0.0315
No log	263.0	263	0.0315
No log	264.0	264	0.0314
No log	265.0	265	0.0313
No log	266.0	266	0.0313
No log	267.0	267	0.0311
No log	268.0	268	0.0310
No log	269.0	269	0.0309
No log	270.0	270	0.0307
No log	271.0	271	0.0306
No log	272.0	272	0.0305
No log	273.0	273	0.0303
No log	274.0	274	0.0302
No log	275.0	275	0.0301
No log	276.0	276	0.0300
No log	277.0	277	0.0299
No log	278.0	278	0.0298
No log	279.0	279	0.0298
No log	280.0	280	0.0297
No log	281.0	281	0.0297
No log	282.0	282	0.0296
No log	283.0	283	0.0296
No log	284.0	284	0.0295
No log	285.0	285	0.0294
No log	286.0	286	0.0293
No log	287.0	287	0.0291
No log	288.0	288	0.0290
No log	289.0	289	0.0289
No log	290.0	290	0.0288
No log	291.0	291	0.0287
No log	292.0	292	0.0286
No log	293.0	293	0.0286
No log	294.0	294	0.0285
No log	295.0	295	0.0284
No log	296.0	296	0.0284
No log	297.0	297	0.0283
No log	298.0	298	0.0282
No log	299.0	299	0.0281
No log	300.0	300	0.0281
No log	301.0	301	0.0280
No log	302.0	302	0.0279
No log	303.0	303	0.0278
No log	304.0	304	0.0277
No log	305.0	305	0.0276
No log	306.0	306	0.0276
No log	307.0	307	0.0275
No log	308.0	308	0.0274
No log	309.0	309	0.0274
No log	310.0	310	0.0273
No log	311.0	311	0.0272
No log	312.0	312	0.0272
No log	313.0	313	0.0271
No log	314.0	314	0.0270
No log	315.0	315	0.0270
No log	316.0	316	0.0269
No log	317.0	317	0.0269
No log	318.0	318	0.0268
No log	319.0	319	0.0268
No log	320.0	320	0.0267
No log	321.0	321	0.0267
No log	322.0	322	0.0266
No log	323.0	323	0.0266
No log	324.0	324	0.0265
No log	325.0	325	0.0264
No log	326.0	326	0.0264
No log	327.0	327	0.0263
No log	328.0	328	0.0262
No log	329.0	329	0.0262
No log	330.0	330	0.0261
No log	331.0	331	0.0260
No log	332.0	332	0.0260
No log	333.0	333	0.0259
No log	334.0	334	0.0259
No log	335.0	335	0.0259
No log	336.0	336	0.0258
No log	337.0	337	0.0258
No log	338.0	338	0.0257
No log	339.0	339	0.0256
No log	340.0	340	0.0256
No log	341.0	341	0.0256
No log	342.0	342	0.0255
No log	343.0	343	0.0255
No log	344.0	344	0.0254
No log	345.0	345	0.0254
No log	346.0	346	0.0254
No log	347.0	347	0.0253
No log	348.0	348	0.0253
No log	349.0	349	0.0253
No log	350.0	350	0.0252
No log	351.0	351	0.0252
No log	352.0	352	0.0252
No log	353.0	353	0.0252
No log	354.0	354	0.0251
No log	355.0	355	0.0251
No log	356.0	356	0.0251
No log	357.0	357	0.0250
No log	358.0	358	0.0250
No log	359.0	359	0.0249
No log	360.0	360	0.0249
No log	361.0	361	0.0248
No log	362.0	362	0.0248
No log	363.0	363	0.0247
No log	364.0	364	0.0247
No log	365.0	365	0.0246
No log	366.0	366	0.0246
No log	367.0	367	0.0245
No log	368.0	368	0.0245
No log	369.0	369	0.0244
No log	370.0	370	0.0244
No log	371.0	371	0.0244
No log	372.0	372	0.0243
No log	373.0	373	0.0243
No log	374.0	374	0.0242
No log	375.0	375	0.0242
No log	376.0	376	0.0241
No log	377.0	377	0.0241
No log	378.0	378	0.0241
No log	379.0	379	0.0241
No log	380.0	380	0.0240
No log	381.0	381	0.0240
No log	382.0	382	0.0240
No log	383.0	383	0.0240
No log	384.0	384	0.0239
No log	385.0	385	0.0239
No log	386.0	386	0.0239
No log	387.0	387	0.0238
No log	388.0	388	0.0238
No log	389.0	389	0.0238
No log	390.0	390	0.0238
No log	391.0	391	0.0237
No log	392.0	392	0.0237
No log	393.0	393	0.0237
No log	394.0	394	0.0236
No log	395.0	395	0.0236
No log	396.0	396	0.0235
No log	397.0	397	0.0235
No log	398.0	398	0.0235
No log	399.0	399	0.0235
No log	400.0	400	0.0234
No log	401.0	401	0.0234
No log	402.0	402	0.0234
No log	403.0	403	0.0234
No log	404.0	404	0.0234
No log	405.0	405	0.0233
No log	406.0	406	0.0233
No log	407.0	407	0.0233
No log	408.0	408	0.0233
No log	409.0	409	0.0233
No log	410.0	410	0.0233
No log	411.0	411	0.0232
No log	412.0	412	0.0232
No log	413.0	413	0.0232
No log	414.0	414	0.0232
No log	415.0	415	0.0232
No log	416.0	416	0.0231
No log	417.0	417	0.0231
No log	418.0	418	0.0231
No log	419.0	419	0.0231
No log	420.0	420	0.0231
No log	421.0	421	0.0230
No log	422.0	422	0.0230
No log	423.0	423	0.0230
No log	424.0	424	0.0230
No log	425.0	425	0.0229
No log	426.0	426	0.0229
No log	427.0	427	0.0229
No log	428.0	428	0.0228
No log	429.0	429	0.0228
No log	430.0	430	0.0228
No log	431.0	431	0.0228
No log	432.0	432	0.0228
No log	433.0	433	0.0227
No log	434.0	434	0.0227
No log	435.0	435	0.0227
No log	436.0	436	0.0227
No log	437.0	437	0.0227
No log	438.0	438	0.0227
No log	439.0	439	0.0226
No log	440.0	440	0.0226
No log	441.0	441	0.0226
No log	442.0	442	0.0226
No log	443.0	443	0.0226
No log	444.0	444	0.0226
No log	445.0	445	0.0226
No log	446.0	446	0.0226
No log	447.0	447	0.0225
No log	448.0	448	0.0225
No log	449.0	449	0.0225
No log	450.0	450	0.0225
No log	451.0	451	0.0225
No log	452.0	452	0.0225
No log	453.0	453	0.0225
No log	454.0	454	0.0224
No log	455.0	455	0.0224
No log	456.0	456	0.0224
No log	457.0	457	0.0224
No log	458.0	458	0.0224
No log	459.0	459	0.0224
No log	460.0	460	0.0224
No log	461.0	461	0.0224
No log	462.0	462	0.0223
No log	463.0	463	0.0223
No log	464.0	464	0.0223
No log	465.0	465	0.0223
No log	466.0	466	0.0223
No log	467.0	467	0.0223
No log	468.0	468	0.0223
No log	469.0	469	0.0223
No log	470.0	470	0.0223
No log	471.0	471	0.0223
No log	472.0	472	0.0223
No log	473.0	473	0.0223
No log	474.0	474	0.0223
No log	475.0	475	0.0222
No log	476.0	476	0.0222
No log	477.0	477	0.0222
No log	478.0	478	0.0222
No log	479.0	479	0.0222
No log	480.0	480	0.0222
No log	481.0	481	0.0222
No log	482.0	482	0.0222
No log	483.0	483	0.0222
No log	484.0	484	0.0222
No log	485.0	485	0.0222
No log	486.0	486	0.0222
No log	487.0	487	0.0222
No log	488.0	488	0.0222
No log	489.0	489	0.0222
No log	490.0	490	0.0222
No log	491.0	491	0.0222
No log	492.0	492	0.0222
No log	493.0	493	0.0222
No log	494.0	494	0.0222
No log	495.0	495	0.0222
No log	496.0	496	0.0222
No log	497.0	497	0.0222
No log	498.0	498	0.0222
No log	499.0	499	0.0222
0.3551	500.0	500	0.0222

Framework versions

• Transformers 4.41.2
• Pytorch 2.3.0+cu121
• Datasets 2.19.1
• Tokenizers 0.19.1