layoutlm-doclaynet

This model is a fine-tuned version of microsoft/layoutlm-base-uncased on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.0004
• Footer: {'precision': 0.9992401215805471, 'recall': 0.9984813971146546, 'f1': 0.9988606152677554, 'number': 1317}
• Header: {'precision': 0.9988290398126464, 'recall': 0.9976608187134502, 'f1': 0.9982445874780573, 'number': 855}
• Able: {'precision': 0.9987046632124352, 'recall': 1.0, 'f1': 0.9993519118600129, 'number': 771}
• Aption: {'precision': 0.99375, 'recall': 0.9921996879875195, 'f1': 0.9929742388758782, 'number': 641}
• Ext: {'precision': 0.9927302100161551, 'recall': 0.9935327405012127, 'f1': 0.9931313131313131, 'number': 2474}
• Icture: {'precision': 0.9975216852540273, 'recall': 0.9975216852540273, 'f1': 0.9975216852540273, 'number': 807}
• Itle: {'precision': 0.9568965517241379, 'recall': 0.9736842105263158, 'f1': 0.9652173913043478, 'number': 114}
• Ootnote: {'precision': 1.0, 'recall': 0.9824561403508771, 'f1': 0.9911504424778761, 'number': 57}
• Ormula: {'precision': 0.9819819819819819, 'recall': 0.990909090909091, 'f1': 0.9864253393665158, 'number': 110}
• Overall Precision: 0.9952
• Overall Recall: 0.9955
• Overall F1: 0.9954
• Overall Accuracy: 0.9999

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 3e-05
• train_batch_size: 16
• eval_batch_size: 8
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 15

Training results

Training Loss	Epoch	Step	Validation Loss	Footer	Header	Able	Aption	Ext	Icture	Itle	Ootnote	Ormula	Overall Precision	Overall Recall	Overall F1	Overall Accuracy
0.4275	1.0	307	0.2122	{'precision': 0.6709050112191474, 'recall': 0.6810933940774487, 'f1': 0.6759608138658629, 'number': 1317}	{'precision': 0.7034482758620689, 'recall': 0.7157894736842105, 'f1': 0.7095652173913042, 'number': 855}	{'precision': 0.43611705924339755, 'recall': 0.7924773022049286, 'f1': 0.5626151012891344, 'number': 771}	{'precision': 0.28087649402390436, 'recall': 0.43993759750390016, 'f1': 0.34285714285714286, 'number': 641}	{'precision': 0.33685839566675263, 'recall': 0.5278900565885206, 'f1': 0.41127381514722083, 'number': 2474}	{'precision': 0.49547920433996384, 'recall': 0.6790582403965304, 'f1': 0.5729221118661788, 'number': 807}	{'precision': 0.09734513274336283, 'recall': 0.09649122807017543, 'f1': 0.09691629955947136, 'number': 114}	{'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 57}	{'precision': 0.10673234811165845, 'recall': 0.5909090909090909, 'f1': 0.1808066759388039, 'number': 110}	0.4199	0.6062	0.4961	0.9351
0.2047	2.0	614	0.1510	{'precision': 0.8542568542568543, 'recall': 0.8990129081245254, 'f1': 0.8760636330003699, 'number': 1317}	{'precision': 0.8981132075471698, 'recall': 0.8350877192982457, 'f1': 0.8654545454545455, 'number': 855}	{'precision': 0.6210418794688458, 'recall': 0.788586251621271, 'f1': 0.694857142857143, 'number': 771}	{'precision': 0.37422771403353927, 'recall': 0.6614664586583463, 'f1': 0.47801578354002255, 'number': 641}	{'precision': 0.43380583043594545, 'recall': 0.6556184316895716, 'f1': 0.5221310156124256, 'number': 2474}	{'precision': 0.46895544192841493, 'recall': 0.7955390334572491, 'f1': 0.5900735294117647, 'number': 807}	{'precision': 0.22818791946308725, 'recall': 0.2982456140350877, 'f1': 0.2585551330798479, 'number': 114}	{'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 57}	{'precision': 0.11032531824611033, 'recall': 0.7090909090909091, 'f1': 0.19094247246022034, 'number': 110}	0.5173	0.7425	0.6098	0.9506
0.1157	3.0	921	0.0578	{'precision': 0.9427710843373494, 'recall': 0.9506454062262718, 'f1': 0.9466918714555767, 'number': 1317}	{'precision': 0.9485294117647058, 'recall': 0.9052631578947369, 'f1': 0.9263913824057451, 'number': 855}	{'precision': 0.7569296375266524, 'recall': 0.920881971465629, 'f1': 0.8308952603861907, 'number': 771}	{'precision': 0.6173913043478261, 'recall': 0.7753510140405616, 'f1': 0.6874135546334715, 'number': 641}	{'precision': 0.679323438039351, 'recall': 0.7954729183508489, 'f1': 0.732824427480916, 'number': 2474}	{'precision': 0.812039312039312, 'recall': 0.8190830235439901, 'f1': 0.8155459592843924, 'number': 807}	{'precision': 0.3493150684931507, 'recall': 0.4473684210526316, 'f1': 0.3923076923076923, 'number': 114}	{'precision': 0.125, 'recall': 0.14035087719298245, 'f1': 0.1322314049586777, 'number': 57}	{'precision': 0.4380952380952381, 'recall': 0.8363636363636363, 'f1': 0.5750000000000001, 'number': 110}	0.7499	0.8414	0.7931	0.9845
0.0672	4.0	1228	0.0574	{'precision': 0.9497413155949741, 'recall': 0.9757023538344722, 'f1': 0.9625468164794007, 'number': 1317}	{'precision': 0.96, 'recall': 0.9543859649122807, 'f1': 0.9571847507331378, 'number': 855}	{'precision': 0.8123543123543123, 'recall': 0.9040207522697795, 'f1': 0.8557397176181706, 'number': 771}	{'precision': 0.5407035175879397, 'recall': 0.8393135725429017, 'f1': 0.6577017114914424, 'number': 641}	{'precision': 0.6827350427350427, 'recall': 0.807194826192401, 'f1': 0.7397666234487869, 'number': 2474}	{'precision': 0.6983511154219205, 'recall': 0.8921933085501859, 'f1': 0.7834602829162134, 'number': 807}	{'precision': 0.5446428571428571, 'recall': 0.5350877192982456, 'f1': 0.5398230088495576, 'number': 114}	{'precision': 0.25675675675675674, 'recall': 0.3333333333333333, 'f1': 0.2900763358778626, 'number': 57}	{'precision': 0.6289308176100629, 'recall': 0.9090909090909091, 'f1': 0.7434944237918215, 'number': 110}	0.7458	0.8722	0.8041	0.9830
0.0379	5.0	1535	0.0258	{'precision': 0.9848369977255497, 'recall': 0.9863325740318907, 'f1': 0.9855842185128985, 'number': 1317}	{'precision': 0.9811764705882353, 'recall': 0.9754385964912281, 'f1': 0.9782991202346041, 'number': 855}	{'precision': 0.9011264080100125, 'recall': 0.933852140077821, 'f1': 0.9171974522292994, 'number': 771}	{'precision': 0.7589403973509934, 'recall': 0.8939157566302652, 'f1': 0.820916905444126, 'number': 641}	{'precision': 0.8442001516300227, 'recall': 0.9001616814874697, 'f1': 0.871283255086072, 'number': 2474}	{'precision': 0.8192771084337349, 'recall': 0.9268897149938042, 'f1': 0.8697674418604652, 'number': 807}	{'precision': 0.6782608695652174, 'recall': 0.6842105263157895, 'f1': 0.6812227074235808, 'number': 114}	{'precision': 0.574468085106383, 'recall': 0.47368421052631576, 'f1': 0.5192307692307692, 'number': 57}	{'precision': 0.9464285714285714, 'recall': 0.9636363636363636, 'f1': 0.9549549549549549, 'number': 110}	0.8760	0.9253	0.9000	0.9921
0.0256	6.0	1842	0.0111	{'precision': 0.9811746987951807, 'recall': 0.9893697798025817, 'f1': 0.9852551984877127, 'number': 1317}	{'precision': 0.9893992932862191, 'recall': 0.9824561403508771, 'f1': 0.9859154929577465, 'number': 855}	{'precision': 0.9606099110546379, 'recall': 0.980544747081712, 'f1': 0.9704749679075738, 'number': 771}	{'precision': 0.8571428571428571, 'recall': 0.9173166926677067, 'f1': 0.8862094951017331, 'number': 641}	{'precision': 0.8938223938223938, 'recall': 0.9357316087308003, 'f1': 0.9142969984202212, 'number': 2474}	{'precision': 0.9262899262899262, 'recall': 0.9343246592317225, 'f1': 0.9302899444787168, 'number': 807}	{'precision': 0.7666666666666667, 'recall': 0.8070175438596491, 'f1': 0.7863247863247863, 'number': 114}	{'precision': 0.41935483870967744, 'recall': 0.45614035087719296, 'f1': 0.43697478991596644, 'number': 57}	{'precision': 0.9473684210526315, 'recall': 0.9818181818181818, 'f1': 0.9642857142857142, 'number': 110}	0.9227	0.9491	0.9357	0.9969
0.0134	7.0	2149	0.0093	{'precision': 0.9879245283018868, 'recall': 0.9939255884586181, 'f1': 0.9909159727479183, 'number': 1317}	{'precision': 0.9655963302752294, 'recall': 0.9847953216374269, 'f1': 0.97510133178923, 'number': 855}	{'precision': 0.9569620253164557, 'recall': 0.980544747081712, 'f1': 0.968609865470852, 'number': 771}	{'precision': 0.8998505231689088, 'recall': 0.9391575663026521, 'f1': 0.9190839694656487, 'number': 641}	{'precision': 0.9224103977944073, 'recall': 0.946645109135004, 'f1': 0.9343706363455017, 'number': 2474}	{'precision': 0.9261904761904762, 'recall': 0.9640644361833953, 'f1': 0.9447480267152397, 'number': 807}	{'precision': 0.7016129032258065, 'recall': 0.7631578947368421, 'f1': 0.73109243697479, 'number': 114}	{'precision': 0.7169811320754716, 'recall': 0.6666666666666666, 'f1': 0.6909090909090909, 'number': 57}	{'precision': 0.9818181818181818, 'recall': 0.9818181818181818, 'f1': 0.9818181818181818, 'number': 110}	0.9372	0.9603	0.9486	0.9976
0.0096	8.0	2456	0.0056	{'precision': 0.9946848899012908, 'recall': 0.9946848899012908, 'f1': 0.9946848899012908, 'number': 1317}	{'precision': 0.9964830011723329, 'recall': 0.9941520467836257, 'f1': 0.9953161592505855, 'number': 855}	{'precision': 0.9871134020618557, 'recall': 0.993514915693904, 'f1': 0.9903038138332256, 'number': 771}	{'precision': 0.9357798165137615, 'recall': 0.9547581903276131, 'f1': 0.9451737451737452, 'number': 641}	{'precision': 0.9389131297104324, 'recall': 0.9567502021018593, 'f1': 0.9477477477477478, 'number': 2474}	{'precision': 0.9541062801932367, 'recall': 0.9789343246592317, 'f1': 0.9663608562691132, 'number': 807}	{'precision': 0.8032786885245902, 'recall': 0.8596491228070176, 'f1': 0.8305084745762712, 'number': 114}	{'precision': 0.620253164556962, 'recall': 0.8596491228070176, 'f1': 0.7205882352941176, 'number': 57}	{'precision': 0.956140350877193, 'recall': 0.990909090909091, 'f1': 0.9732142857142858, 'number': 110}	0.9569	0.9727	0.9647	0.9986
0.0052	9.0	2763	0.0039	{'precision': 0.9931766489764974, 'recall': 0.9946848899012908, 'f1': 0.9939301972685887, 'number': 1317}	{'precision': 0.9976608187134502, 'recall': 0.9976608187134502, 'f1': 0.9976608187134502, 'number': 855}	{'precision': 0.9935233160621761, 'recall': 0.9948119325551232, 'f1': 0.9941672067401167, 'number': 771}	{'precision': 0.9493087557603687, 'recall': 0.9641185647425897, 'f1': 0.956656346749226, 'number': 641}	{'precision': 0.9686872741870735, 'recall': 0.9753435731608731, 'f1': 0.9720040281973816, 'number': 2474}	{'precision': 0.9633699633699634, 'recall': 0.9776951672862454, 'f1': 0.970479704797048, 'number': 807}	{'precision': 0.847457627118644, 'recall': 0.8771929824561403, 'f1': 0.8620689655172413, 'number': 114}	{'precision': 0.9285714285714286, 'recall': 0.9122807017543859, 'f1': 0.9203539823008849, 'number': 57}	{'precision': 0.990909090909091, 'recall': 0.990909090909091, 'f1': 0.990909090909091, 'number': 110}	0.9750	0.9811	0.9780	0.9990
0.0042	10.0	3070	0.0043	{'precision': 0.9977238239757208, 'recall': 0.9984813971146546, 'f1': 0.9981024667931689, 'number': 1317}	{'precision': 0.9976553341148886, 'recall': 0.9953216374269006, 'f1': 0.996487119437939, 'number': 855}	{'precision': 0.982051282051282, 'recall': 0.993514915693904, 'f1': 0.9877498388136685, 'number': 771}	{'precision': 0.9521604938271605, 'recall': 0.9625585023400937, 'f1': 0.9573312645461599, 'number': 641}	{'precision': 0.9668265387689848, 'recall': 0.9777687954729184, 'f1': 0.972266881028939, 'number': 2474}	{'precision': 0.9672727272727273, 'recall': 0.9888475836431226, 'f1': 0.9779411764705883, 'number': 807}	{'precision': 0.8760330578512396, 'recall': 0.9298245614035088, 'f1': 0.902127659574468, 'number': 114}	{'precision': 0.8412698412698413, 'recall': 0.9298245614035088, 'f1': 0.8833333333333334, 'number': 57}	{'precision': 0.990909090909091, 'recall': 0.990909090909091, 'f1': 0.990909090909091, 'number': 110}	0.9742	0.9843	0.9793	0.9988
0.0023	11.0	3377	0.0017	{'precision': 0.9977220956719818, 'recall': 0.9977220956719818, 'f1': 0.9977220956719818, 'number': 1317}	{'precision': 0.9988304093567252, 'recall': 0.9988304093567252, 'f1': 0.9988304093567252, 'number': 855}	{'precision': 0.9871630295250321, 'recall': 0.9974059662775616, 'f1': 0.9922580645161291, 'number': 771}	{'precision': 0.9720930232558139, 'recall': 0.9781591263650546, 'f1': 0.9751166407465008, 'number': 641}	{'precision': 0.9819277108433735, 'recall': 0.9882780921584479, 'f1': 0.9850926672038678, 'number': 2474}	{'precision': 0.9889025893958077, 'recall': 0.9938042131350682, 'f1': 0.9913473423980222, 'number': 807}	{'precision': 0.9391304347826087, 'recall': 0.9473684210526315, 'f1': 0.9432314410480349, 'number': 114}	{'precision': 0.9642857142857143, 'recall': 0.9473684210526315, 'f1': 0.9557522123893805, 'number': 57}	{'precision': 0.9818181818181818, 'recall': 0.9818181818181818, 'f1': 0.9818181818181818, 'number': 110}	0.9865	0.9909	0.9887	0.9995
0.0019	12.0	3684	0.0011	{'precision': 0.9992395437262357, 'recall': 0.9977220956719818, 'f1': 0.9984802431610942, 'number': 1317}	{'precision': 0.9988304093567252, 'recall': 0.9988304093567252, 'f1': 0.9988304093567252, 'number': 855}	{'precision': 0.9871630295250321, 'recall': 0.9974059662775616, 'f1': 0.9922580645161291, 'number': 771}	{'precision': 0.9875195007800313, 'recall': 0.9875195007800313, 'f1': 0.9875195007800313, 'number': 641}	{'precision': 0.9847205468435867, 'recall': 0.9898949070331448, 'f1': 0.9873009473896394, 'number': 2474}	{'precision': 0.9889434889434889, 'recall': 0.9975216852540273, 'f1': 0.9932140653917335, 'number': 807}	{'precision': 0.9487179487179487, 'recall': 0.9736842105263158, 'f1': 0.9610389610389611, 'number': 114}	{'precision': 0.9821428571428571, 'recall': 0.9649122807017544, 'f1': 0.9734513274336283, 'number': 57}	{'precision': 0.9316239316239316, 'recall': 0.990909090909091, 'f1': 0.960352422907489, 'number': 110}	0.9886	0.9934	0.9910	0.9996
0.0012	13.0	3991	0.0008	{'precision': 0.9992401215805471, 'recall': 0.9984813971146546, 'f1': 0.9988606152677554, 'number': 1317}	{'precision': 0.9988290398126464, 'recall': 0.9976608187134502, 'f1': 0.9982445874780573, 'number': 855}	{'precision': 0.9974093264248705, 'recall': 0.9987029831387808, 'f1': 0.9980557355800389, 'number': 771}	{'precision': 0.9875583203732504, 'recall': 0.9906396255850234, 'f1': 0.9890965732087228, 'number': 641}	{'precision': 0.9871071716357775, 'recall': 0.9902991107518189, 'f1': 0.9887005649717514, 'number': 2474}	{'precision': 0.9938271604938271, 'recall': 0.9975216852540273, 'f1': 0.9956709956709957, 'number': 807}	{'precision': 0.9823008849557522, 'recall': 0.9736842105263158, 'f1': 0.9779735682819383, 'number': 114}	{'precision': 0.9818181818181818, 'recall': 0.9473684210526315, 'f1': 0.9642857142857142, 'number': 57}	{'precision': 0.9646017699115044, 'recall': 0.990909090909091, 'f1': 0.9775784753363229, 'number': 110}	0.9922	0.9938	0.9930	0.9998
0.0011	14.0	4298	0.0006	{'precision': 0.9992406985573272, 'recall': 0.9992406985573272, 'f1': 0.9992406985573272, 'number': 1317}	{'precision': 0.9988290398126464, 'recall': 0.9976608187134502, 'f1': 0.9982445874780573, 'number': 855}	{'precision': 0.9974126778783958, 'recall': 1.0, 'f1': 0.9987046632124352, 'number': 771}	{'precision': 0.9890965732087228, 'recall': 0.9906396255850234, 'f1': 0.9898674980514419, 'number': 641}	{'precision': 0.9879178413209827, 'recall': 0.9915117219078415, 'f1': 0.98971151906395, 'number': 2474}	{'precision': 0.9975216852540273, 'recall': 0.9975216852540273, 'f1': 0.9975216852540273, 'number': 807}	{'precision': 0.940677966101695, 'recall': 0.9736842105263158, 'f1': 0.956896551724138, 'number': 114}	{'precision': 0.9642857142857143, 'recall': 0.9473684210526315, 'f1': 0.9557522123893805, 'number': 57}	{'precision': 0.9732142857142857, 'recall': 0.990909090909091, 'f1': 0.9819819819819819, 'number': 110}	0.9923	0.9945	0.9934	0.9998
0.0008	15.0	4605	0.0004	{'precision': 0.9992401215805471, 'recall': 0.9984813971146546, 'f1': 0.9988606152677554, 'number': 1317}	{'precision': 0.9988290398126464, 'recall': 0.9976608187134502, 'f1': 0.9982445874780573, 'number': 855}	{'precision': 0.9987046632124352, 'recall': 1.0, 'f1': 0.9993519118600129, 'number': 771}	{'precision': 0.99375, 'recall': 0.9921996879875195, 'f1': 0.9929742388758782, 'number': 641}	{'precision': 0.9927302100161551, 'recall': 0.9935327405012127, 'f1': 0.9931313131313131, 'number': 2474}	{'precision': 0.9975216852540273, 'recall': 0.9975216852540273, 'f1': 0.9975216852540273, 'number': 807}	{'precision': 0.9568965517241379, 'recall': 0.9736842105263158, 'f1': 0.9652173913043478, 'number': 114}	{'precision': 1.0, 'recall': 0.9824561403508771, 'f1': 0.9911504424778761, 'number': 57}	{'precision': 0.9819819819819819, 'recall': 0.990909090909091, 'f1': 0.9864253393665158, 'number': 110}	0.9952	0.9955	0.9954	0.9999

Framework versions

• Transformers 4.26.0
• Pytorch 1.12.1
• Datasets 2.9.0
• Tokenizers 0.13.2