layoutlm-funsd

This model is a fine-tuned version of microsoft/layoutlm-base-uncased on the funsd dataset. It achieves the following results on the evaluation set:

• Loss: 0.6771
• Answer: {'precision': 0.7107258938244854, 'recall': 0.8108776266996292, 'f1': 0.7575057736720554, 'number': 809}
• Header: {'precision': 0.3543307086614173, 'recall': 0.37815126050420167, 'f1': 0.3658536585365853, 'number': 119}
• Question: {'precision': 0.7716814159292036, 'recall': 0.8187793427230047, 'f1': 0.7945330296127562, 'number': 1065}
• Overall Precision: 0.7216
• Overall Recall: 0.7893
• Overall F1: 0.7539
• Overall Accuracy: 0.8139

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 3e-05
• train_batch_size: 16
• eval_batch_size: 8
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 15
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Answer	Header	Question	Overall Precision	Overall Recall	Overall F1	Overall Accuracy
1.8027	1.0	10	1.5884	{'precision': 0.01997780244173141, 'recall': 0.022249690976514216, 'f1': 0.02105263157894737, 'number': 809}	{'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119}	{'precision': 0.18858307849133538, 'recall': 0.17370892018779344, 'f1': 0.18084066471163246, 'number': 1065}	0.1079	0.1019	0.1048	0.3753
1.4071	2.0	20	1.2076	{'precision': 0.23890339425587467, 'recall': 0.22620519159456118, 'f1': 0.23238095238095238, 'number': 809}	{'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119}	{'precision': 0.41302791696492486, 'recall': 0.5417840375586854, 'f1': 0.4687246141348498, 'number': 1065}	0.3512	0.3813	0.3656	0.5772
1.0593	3.0	30	0.9154	{'precision': 0.4750542299349241, 'recall': 0.5414091470951793, 'f1': 0.5060658578856152, 'number': 809}	{'precision': 0.11363636363636363, 'recall': 0.04201680672268908, 'f1': 0.06134969325153375, 'number': 119}	{'precision': 0.5922493681550126, 'recall': 0.6600938967136151, 'f1': 0.6243339253996447, 'number': 1065}	0.5323	0.5750	0.5528	0.7136
0.802	4.0	40	0.7552	{'precision': 0.5981404958677686, 'recall': 0.715698393077874, 'f1': 0.6516601012943164, 'number': 809}	{'precision': 0.20253164556962025, 'recall': 0.13445378151260504, 'f1': 0.1616161616161616, 'number': 119}	{'precision': 0.6680707666385847, 'recall': 0.7446009389671362, 'f1': 0.7042628774422734, 'number': 1065}	0.6213	0.6964	0.6567	0.7659
0.6561	5.0	50	0.7030	{'precision': 0.6381856540084389, 'recall': 0.7478368355995055, 'f1': 0.6886738759248718, 'number': 809}	{'precision': 0.3, 'recall': 0.226890756302521, 'f1': 0.25837320574162675, 'number': 119}	{'precision': 0.6780766096169519, 'recall': 0.7812206572769953, 'f1': 0.7260034904013962, 'number': 1065}	0.6464	0.7346	0.6876	0.7889
0.5591	6.0	60	0.6842	{'precision': 0.6502100840336135, 'recall': 0.765142150803461, 'f1': 0.7030096536059057, 'number': 809}	{'precision': 0.3132530120481928, 'recall': 0.2184873949579832, 'f1': 0.25742574257425743, 'number': 119}	{'precision': 0.7165820642978004, 'recall': 0.7953051643192488, 'f1': 0.7538940809968847, 'number': 1065}	0.6730	0.7486	0.7088	0.7942
0.4858	7.0	70	0.6508	{'precision': 0.6569948186528497, 'recall': 0.7836835599505563, 'f1': 0.7147688838782412, 'number': 809}	{'precision': 0.34210526315789475, 'recall': 0.3277310924369748, 'f1': 0.33476394849785407, 'number': 119}	{'precision': 0.7205503009458297, 'recall': 0.7868544600938967, 'f1': 0.7522441651705565, 'number': 1065}	0.6740	0.7582	0.7136	0.8063
0.431	8.0	80	0.6674	{'precision': 0.6578140960163432, 'recall': 0.796044499381953, 'f1': 0.7203579418344519, 'number': 809}	{'precision': 0.35964912280701755, 'recall': 0.3445378151260504, 'f1': 0.351931330472103, 'number': 119}	{'precision': 0.7482517482517482, 'recall': 0.8037558685446009, 'f1': 0.775011317338162, 'number': 1065}	0.6889	0.7732	0.7286	0.7969
0.3878	9.0	90	0.6526	{'precision': 0.6787564766839378, 'recall': 0.8096415327564895, 'f1': 0.7384441939120632, 'number': 809}	{'precision': 0.336283185840708, 'recall': 0.31932773109243695, 'f1': 0.32758620689655166, 'number': 119}	{'precision': 0.7586206896551724, 'recall': 0.7849765258215963, 'f1': 0.7715736040609138, 'number': 1065}	0.7014	0.7672	0.7328	0.8073
0.3744	10.0	100	0.6519	{'precision': 0.6854410201912858, 'recall': 0.7972805933250927, 'f1': 0.7371428571428571, 'number': 809}	{'precision': 0.3130434782608696, 'recall': 0.3025210084033613, 'f1': 0.3076923076923077, 'number': 119}	{'precision': 0.7611940298507462, 'recall': 0.8140845070422535, 'f1': 0.7867513611615246, 'number': 1065}	0.7052	0.7767	0.7393	0.8120
0.3161	11.0	110	0.6696	{'precision': 0.6948257655755016, 'recall': 0.8133498145859085, 'f1': 0.7494305239179954, 'number': 809}	{'precision': 0.3283582089552239, 'recall': 0.3697478991596639, 'f1': 0.34782608695652173, 'number': 119}	{'precision': 0.7604166666666666, 'recall': 0.8225352112676056, 'f1': 0.7902571041948578, 'number': 1065}	0.7067	0.7918	0.7468	0.8060
0.3039	12.0	120	0.6656	{'precision': 0.7007534983853606, 'recall': 0.8046971569839307, 'f1': 0.7491369390103566, 'number': 809}	{'precision': 0.3524590163934426, 'recall': 0.36134453781512604, 'f1': 0.35684647302904565, 'number': 119}	{'precision': 0.7695769576957696, 'recall': 0.8028169014084507, 'f1': 0.7858455882352942, 'number': 1065}	0.7165	0.7772	0.7456	0.8131
0.2877	13.0	130	0.6742	{'precision': 0.6927138331573389, 'recall': 0.8108776266996292, 'f1': 0.7471526195899771, 'number': 809}	{'precision': 0.32592592592592595, 'recall': 0.3697478991596639, 'f1': 0.3464566929133859, 'number': 119}	{'precision': 0.7651715039577837, 'recall': 0.8169014084507042, 'f1': 0.7901907356948229, 'number': 1065}	0.7075	0.7878	0.7455	0.8109
0.2681	14.0	140	0.6743	{'precision': 0.7128927410617552, 'recall': 0.8133498145859085, 'f1': 0.7598152424942264, 'number': 809}	{'precision': 0.36220472440944884, 'recall': 0.3865546218487395, 'f1': 0.37398373983739847, 'number': 119}	{'precision': 0.7734513274336283, 'recall': 0.8206572769953052, 'f1': 0.7963553530751709, 'number': 1065}	0.7239	0.7918	0.7563	0.8148
0.2609	15.0	150	0.6771	{'precision': 0.7107258938244854, 'recall': 0.8108776266996292, 'f1': 0.7575057736720554, 'number': 809}	{'precision': 0.3543307086614173, 'recall': 0.37815126050420167, 'f1': 0.3658536585365853, 'number': 119}	{'precision': 0.7716814159292036, 'recall': 0.8187793427230047, 'f1': 0.7945330296127562, 'number': 1065}	0.7216	0.7893	0.7539	0.8139

Framework versions

• Transformers 4.40.1
• Pytorch 2.3.0+cu121
• Datasets 2.19.0
• Tokenizers 0.19.1