Tirthankar/mms_kas_speed1LM

mms_kas_speed1

This model is a fine-tuned version of facebook/mms-1b-all on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.8096
• Wer: 0.5141

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.001
• train_batch_size: 16
• eval_batch_size: 8
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 100
• num_epochs: 30

Training results

Training Loss	Epoch	Step	Validation Loss	Wer
6.8005	0.12	100	1.8647	1.0536
1.8539	0.23	200	1.5445	0.9513
1.6826	0.35	300	1.5297	0.9316
1.6042	0.46	400	1.3865	0.9056
1.5353	0.58	500	1.3953	0.9043
1.5256	0.69	600	1.4043	0.9020
1.5491	0.81	700	1.3509	0.8694
1.475	0.93	800	1.4780	0.9107
1.4637	1.04	900	1.2694	0.8848
1.4401	1.16	1000	1.4087	0.8711
1.4125	1.27	1100	1.3147	0.8519
1.3864	1.39	1200	1.1807	0.8231
1.3294	1.5	1300	1.1504	0.8144
1.3606	1.62	1400	1.2630	0.8449
1.3254	1.74	1500	1.3070	0.8340
1.349	1.85	1600	1.1008	0.7980
1.3251	1.97	1700	1.1150	0.7854
1.3539	2.08	1800	1.0956	0.7943
1.266	2.2	1900	1.1196	0.7870
1.2787	2.31	2000	1.1659	0.7994
1.2902	2.43	2100	1.1099	0.7827
1.2761	2.55	2200	1.1361	0.8044
1.2559	2.66	2300	1.1497	0.8049
1.2294	2.78	2400	1.2398	0.8053
1.2438	2.89	2500	1.1234	0.7849
1.2736	3.01	2600	1.0433	0.7618
1.2337	3.12	2700	1.0905	0.7703
1.2079	3.24	2800	1.3420	0.8411
1.209	3.36	2900	1.0911	0.7879
1.2158	3.47	3000	1.2058	0.8023
1.176	3.59	3100	1.1623	0.7880
1.1775	3.7	3200	1.0644	0.7419
1.2212	3.82	3300	1.0549	0.7605
1.1774	3.94	3400	1.1500	0.7675
1.1046	4.05	3500	0.9748	0.7184
1.1979	4.17	3600	1.0100	0.7255
1.1544	4.28	3700	1.0290	0.7275
1.1558	4.4	3800	1.0180	0.7314
1.1593	4.51	3900	1.0051	0.7097
1.1415	4.63	4000	1.0115	0.7353
1.1399	4.75	4100	1.0527	0.7241
1.1297	4.86	4200	1.0546	0.7317
1.1137	4.98	4300	1.0818	0.7462
1.1948	5.09	4400	1.0758	0.7360
1.0913	5.21	4500	1.0204	0.7210
1.1632	5.32	4600	0.9644	0.7132
1.1216	5.44	4700	0.9569	0.6822
1.0728	5.56	4800	0.9980	0.7230
1.1097	5.67	4900	0.9867	0.7014
1.072	5.79	5000	0.9946	0.6972
1.1009	5.9	5100	0.9339	0.6818
1.1313	6.02	5200	0.9417	0.6842
1.0768	6.13	5300	0.9828	0.7195
1.0997	6.25	5400	1.0191	0.7258
1.0839	6.37	5500	1.0013	0.7045
1.0997	6.48	5600	0.9832	0.7159
1.0854	6.6	5700	1.0778	0.7397
1.0398	6.71	5800	1.0442	0.7268
1.0398	6.83	5900	1.0284	0.6932
1.0773	6.94	6000	1.1135	0.7522
1.0676	7.06	6100	0.9657	0.6816
1.0227	7.18	6200	0.9636	0.6695
1.0415	7.29	6300	0.9700	0.6709
1.0438	7.41	6400	0.9603	0.6662
1.0452	7.52	6500	0.9563	0.6674
1.0295	7.64	6600	0.9782	0.6633
1.0722	7.75	6700	0.9988	0.6752
0.9848	7.87	6800	0.9744	0.6897
1.0332	7.99	6900	0.9118	0.6485
1.0041	8.1	7000	0.8834	0.6329
1.0168	8.22	7100	0.9263	0.6365
1.0368	8.33	7200	1.0263	0.6867
1.0407	8.45	7300	1.0120	0.7029
1.0175	8.56	7400	0.8795	0.6295
1.0289	8.68	7500	0.8969	0.6294
1.018	8.8	7600	0.9635	0.6718
1.005	8.91	7700	0.9609	0.6625
1.0355	9.03	7800	0.8945	0.6302
0.9918	9.14	7900	0.8980	0.6427
1.0118	9.26	8000	0.8830	0.6211
1.0235	9.38	8100	0.8767	0.6207
0.9781	9.49	8200	0.8673	0.6104
0.9999	9.61	8300	0.9355	0.6280
0.9523	9.72	8400	0.8717	0.6121
0.9823	9.84	8500	0.8792	0.6220
1.0153	9.95	8600	0.9116	0.6311
1.0141	10.07	8700	0.8710	0.6157
0.9347	10.19	8800	0.9062	0.6315
0.9759	10.3	8900	0.8952	0.6227
0.9917	10.42	9000	0.8938	0.6283
0.9994	10.53	9100	0.8733	0.6225
0.9571	10.65	9200	0.9060	0.6364
0.9428	10.76	9300	0.8709	0.6237
0.9431	10.88	9400	0.8321	0.5943
0.8845	11.0	9500	0.8420	0.6032
0.9799	11.11	9600	0.8888	0.6028
0.977	11.23	9700	0.8922	0.6046
0.9392	11.34	9800	0.8611	0.5955
0.9547	11.46	9900	0.8472	0.5885
0.9546	11.57	10000	0.8656	0.5942
0.9166	11.69	10100	0.8665	0.5987
0.9515	11.81	10200	0.8541	0.6064
0.9418	11.92	10300	0.8384	0.5919
0.9039	12.04	10400	0.8492	0.5828
0.8965	12.15	10500	0.8454	0.5875
0.9085	12.27	10600	0.8676	0.6012
0.9113	12.38	10700	0.8536	0.5983
0.9243	12.5	10800	0.8816	0.5968
0.9469	12.62	10900	0.8526	0.5965
0.9149	12.73	11000	0.8378	0.5937
0.9198	12.85	11100	0.8462	0.5990
0.9557	12.96	11200	0.8405	0.5935
0.9775	13.08	11300	0.8657	0.5948
0.874	13.19	11400	0.8501	0.5864
0.9158	13.31	11500	0.8703	0.5879
0.8855	13.43	11600	0.8297	0.5895
0.9415	13.54	11700	0.8645	0.5887
0.8593	13.66	11800	0.8784	0.5928
0.9216	13.77	11900	0.8388	0.5816
0.9196	13.89	12000	0.8077	0.5743
0.9172	14.0	12100	0.8880	0.5897
0.9014	14.12	12200	0.8789	0.5974
0.8785	14.24	12300	0.8454	0.5726
0.8721	14.35	12400	0.8427	0.5672
0.8966	14.47	12500	0.8278	0.5709
0.8975	14.58	12600	0.8523	0.5813
0.8921	14.7	12700	0.8126	0.5697
0.8766	14.81	12800	0.8205	0.5665
0.8852	14.93	12900	0.8418	0.5640
0.8276	15.05	13000	0.8332	0.5785
0.851	15.16	13100	0.8144	0.5731
0.8916	15.28	13200	0.8452	0.5632
0.8623	15.39	13300	0.8398	0.5682
0.8932	15.51	13400	0.8249	0.5667
0.8442	15.62	13500	0.8300	0.5646
0.8592	15.74	13600	0.8153	0.5584
0.9012	15.86	13700	0.8109	0.5651
0.8537	15.97	13800	0.8101	0.5677
0.8812	16.09	13900	0.8057	0.5597
0.853	16.2	14000	0.8124	0.5645
0.8691	16.32	14100	0.8086	0.5621
0.844	16.44	14200	0.8074	0.5550
0.8612	16.55	14300	0.8361	0.5654
0.8315	16.67	14400	0.8216	0.5582
0.8665	16.78	14500	0.8307	0.5596
0.8487	16.9	14600	0.7991	0.5577
0.8567	17.01	14700	0.8181	0.5535
0.8288	17.13	14800	0.8308	0.5552
0.8199	17.25	14900	0.8383	0.5639
0.8264	17.36	15000	0.8355	0.5626
0.8374	17.48	15100	0.8925	0.5725
0.8549	17.59	15200	0.8190	0.5649
0.8164	17.71	15300	0.8422	0.5585
0.8575	17.82	15400	0.8195	0.5498
0.8553	17.94	15500	0.8355	0.5610
0.8234	18.06	15600	0.8214	0.5470
0.8293	18.17	15700	0.8215	0.5511
0.7996	18.29	15800	0.8075	0.5461
0.8468	18.4	15900	0.8182	0.5487
0.8138	18.52	16000	0.8309	0.5627
0.805	18.63	16100	0.8103	0.5575
0.8329	18.75	16200	0.8094	0.5402
0.8483	18.87	16300	0.8116	0.5428
0.8222	18.98	16400	0.8336	0.5413
0.8294	19.1	16500	0.8040	0.5419
0.8043	19.21	16600	0.7930	0.5427
0.8216	19.33	16700	0.8451	0.5574
0.7831	19.44	16800	0.8462	0.5546
0.8069	19.56	16900	0.8230	0.5481
0.8022	19.68	17000	0.7943	0.5441
0.8143	19.79	17100	0.8110	0.5406
0.8018	19.91	17200	0.8033	0.5366
0.7918	20.02	17300	0.8030	0.5344
0.8177	20.14	17400	0.8017	0.5377
0.7763	20.25	17500	0.8152	0.5411
0.8226	20.37	17600	0.8176	0.5403
0.7929	20.49	17700	0.8153	0.5406
0.7727	20.6	17800	0.8128	0.5378
0.8095	20.72	17900	0.8041	0.5493
0.7799	20.83	18000	0.8276	0.5411
0.8088	20.95	18100	0.8295	0.5426
0.7682	21.06	18200	0.8031	0.5349
0.7972	21.18	18300	0.8072	0.5269
0.7694	21.3	18400	0.8043	0.5270
0.7826	21.41	18500	0.8324	0.5343
0.7667	21.53	18600	0.8143	0.5316
0.7569	21.64	18700	0.8142	0.5347
0.7939	21.76	18800	0.8043	0.5338
0.7685	21.88	18900	0.8080	0.5408
0.7667	21.99	19000	0.8021	0.5308
0.7993	22.11	19100	0.8081	0.5393
0.7205	22.22	19200	0.8173	0.5408
0.7751	22.34	19300	0.8017	0.5267
0.7477	22.45	19400	0.8166	0.5382
0.7769	22.57	19500	0.8138	0.5341
0.7766	22.69	19600	0.8235	0.5349
0.7494	22.8	19700	0.8135	0.5304
0.8126	22.92	19800	0.8116	0.5317
0.7985	23.03	19900	0.8099	0.5303
0.7698	23.15	20000	0.8009	0.5323
0.7719	23.26	20100	0.8241	0.5411
0.7761	23.38	20200	0.8154	0.5289
0.7523	23.5	20300	0.7987	0.5285
0.7292	23.61	20400	0.7981	0.5255
0.7497	23.73	20500	0.8062	0.5180
0.7469	23.84	20600	0.7998	0.5287
0.7592	23.96	20700	0.8060	0.5265
0.7454	24.07	20800	0.8077	0.5296
0.7512	24.19	20900	0.8025	0.5277
0.7107	24.31	21000	0.8019	0.5284
0.7251	24.42	21100	0.7989	0.5248
0.7594	24.54	21200	0.8122	0.5249
0.7689	24.65	21300	0.8044	0.5225
0.7655	24.77	21400	0.8296	0.5247
0.7278	24.88	21500	0.8119	0.5245
0.7731	25.0	21600	0.7953	0.5222
0.7447	25.12	21700	0.8010	0.5208
0.7226	25.23	21800	0.8155	0.5212
0.7278	25.35	21900	0.8084	0.5229
0.7221	25.46	22000	0.8268	0.5277
0.739	25.58	22100	0.8054	0.5233
0.7657	25.69	22200	0.8004	0.5192
0.7624	25.81	22300	0.8081	0.5215
0.7264	25.93	22400	0.8069	0.5210
0.7596	26.04	22500	0.8084	0.5225
0.706	26.16	22600	0.8108	0.5195
0.7472	26.27	22700	0.8026	0.5159
0.7441	26.39	22800	0.8052	0.5158
0.7447	26.5	22900	0.8117	0.5185
0.6842	26.62	23000	0.7987	0.5139
0.7491	26.74	23100	0.7985	0.5140
0.7017	26.85	23200	0.8118	0.5158
0.7251	26.97	23300	0.8076	0.5172
0.7659	27.08	23400	0.8078	0.5160
0.7246	27.2	23500	0.8105	0.5159
0.7258	27.31	23600	0.8139	0.5183
0.7133	27.43	23700	0.8158	0.5150
0.6811	27.55	23800	0.8186	0.5145
0.7248	27.66	23900	0.7984	0.5108
0.7335	27.78	24000	0.8076	0.5162
0.6924	27.89	24100	0.8034	0.5129
0.7464	28.01	24200	0.8088	0.5131
0.7253	28.12	24300	0.8072	0.5119
0.7401	28.24	24400	0.8094	0.5125
0.7092	28.36	24500	0.8070	0.5153
0.7352	28.47	24600	0.8053	0.5128
0.7121	28.59	24700	0.8034	0.5139
0.6904	28.7	24800	0.8108	0.5136
0.7099	28.82	24900	0.8095	0.5141
0.6814	28.94	25000	0.8127	0.5167
0.6657	29.05	25100	0.8089	0.5139
0.721	29.17	25200	0.8117	0.5163
0.6886	29.28	25300	0.8120	0.5154
0.6974	29.4	25400	0.8087	0.5143
0.7067	29.51	25500	0.8102	0.5162
0.7311	29.63	25600	0.8119	0.5157
0.697	29.75	25700	0.8097	0.5145
0.7126	29.86	25800	0.8098	0.5139
0.7021	29.98	25900	0.8096	0.5141

Framework versions

• Transformers 4.34.0.dev0
• Pytorch 2.1.0.dev20230523+cu117
• Datasets 2.14.5
• Tokenizers 0.13.2