lilt-en-funsd

This model is a fine-tuned version of SCUT-DLVCLab/lilt-roberta-en-base on the funsd-layoutlmv3 dataset. It achieves the following results on the evaluation set:

• Loss: 1.6853
• Answer: {'precision': 0.8719723183391004, 'recall': 0.9253365973072215, 'f1': 0.8978622327790974, 'number': 817}
• Header: {'precision': 0.6224489795918368, 'recall': 0.5126050420168067, 'f1': 0.5622119815668203, 'number': 119}
• Question: {'precision': 0.908411214953271, 'recall': 0.9025069637883009, 'f1': 0.9054494643688868, 'number': 1077}
• Overall Precision: 0.8791
• Overall Recall: 0.8887
• Overall F1: 0.8839
• Overall Accuracy: 0.8067

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• training_steps: 2500
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Answer	Header	Question	Overall Precision	Overall Recall	Overall F1	Overall Accuracy
0.4358	10.53	200	1.0235	{'precision': 0.8292964244521338, 'recall': 0.8800489596083231, 'f1': 0.8539192399049881, 'number': 817}	{'precision': 0.4657534246575342, 'recall': 0.5714285714285714, 'f1': 0.5132075471698114, 'number': 119}	{'precision': 0.8694852941176471, 'recall': 0.8783658310120706, 'f1': 0.8739030023094689, 'number': 1077}	0.8248	0.8609	0.8425	0.7868
0.0513	21.05	400	1.2438	{'precision': 0.8439306358381503, 'recall': 0.8935128518971848, 'f1': 0.8680142687277052, 'number': 817}	{'precision': 0.7045454545454546, 'recall': 0.5210084033613446, 'f1': 0.5990338164251209, 'number': 119}	{'precision': 0.8937893789378938, 'recall': 0.9220055710306406, 'f1': 0.9076782449725778, 'number': 1077}	0.8648	0.8867	0.8756	0.8066
0.0139	31.58	600	1.3473	{'precision': 0.8443935926773455, 'recall': 0.9033047735618115, 'f1': 0.872856298048492, 'number': 817}	{'precision': 0.6111111111111112, 'recall': 0.5546218487394958, 'f1': 0.5814977973568282, 'number': 119}	{'precision': 0.8945945945945946, 'recall': 0.9220055710306406, 'f1': 0.9080932784636488, 'number': 1077}	0.8590	0.8927	0.8755	0.8101
0.0087	42.11	800	1.3432	{'precision': 0.8778718258766627, 'recall': 0.8886168910648715, 'f1': 0.8832116788321168, 'number': 817}	{'precision': 0.5813953488372093, 'recall': 0.6302521008403361, 'f1': 0.6048387096774193, 'number': 119}	{'precision': 0.9113573407202216, 'recall': 0.9164345403899722, 'f1': 0.9138888888888889, 'number': 1077}	0.8769	0.8882	0.8825	0.8161
0.0039	52.63	1000	1.5068	{'precision': 0.8678362573099415, 'recall': 0.9082007343941249, 'f1': 0.8875598086124402, 'number': 817}	{'precision': 0.5564516129032258, 'recall': 0.5798319327731093, 'f1': 0.5679012345679013, 'number': 119}	{'precision': 0.8998144712430427, 'recall': 0.9006499535747446, 'f1': 0.9002320185614848, 'number': 1077}	0.8658	0.8847	0.8752	0.8028
0.0028	63.16	1200	1.5721	{'precision': 0.8624277456647399, 'recall': 0.9130966952264382, 'f1': 0.8870392390011891, 'number': 817}	{'precision': 0.6288659793814433, 'recall': 0.5126050420168067, 'f1': 0.5648148148148148, 'number': 119}	{'precision': 0.9085714285714286, 'recall': 0.8857938718662952, 'f1': 0.8970380818053596, 'number': 1077}	0.8752	0.8748	0.8750	0.8145
0.0027	73.68	1400	1.5657	{'precision': 0.8695150115473441, 'recall': 0.9216646266829865, 'f1': 0.8948306595365418, 'number': 817}	{'precision': 0.6310679611650486, 'recall': 0.5462184873949579, 'f1': 0.5855855855855856, 'number': 119}	{'precision': 0.8944494995450409, 'recall': 0.9127205199628597, 'f1': 0.9034926470588234, 'number': 1077}	0.8709	0.8947	0.8826	0.8130
0.0012	84.21	1600	1.6853	{'precision': 0.8719723183391004, 'recall': 0.9253365973072215, 'f1': 0.8978622327790974, 'number': 817}	{'precision': 0.6224489795918368, 'recall': 0.5126050420168067, 'f1': 0.5622119815668203, 'number': 119}	{'precision': 0.908411214953271, 'recall': 0.9025069637883009, 'f1': 0.9054494643688868, 'number': 1077}	0.8791	0.8887	0.8839	0.8067
0.0007	94.74	1800	1.6321	{'precision': 0.8642117376294591, 'recall': 0.9192166462668299, 'f1': 0.8908659549228943, 'number': 817}	{'precision': 0.5964912280701754, 'recall': 0.5714285714285714, 'f1': 0.5836909871244635, 'number': 119}	{'precision': 0.9101964452759589, 'recall': 0.903435468895079, 'f1': 0.9068033550792172, 'number': 1077}	0.8733	0.8902	0.8817	0.8045
0.0004	105.26	2000	1.7732	{'precision': 0.8535469107551488, 'recall': 0.9130966952264382, 'f1': 0.8823181549379067, 'number': 817}	{'precision': 0.5752212389380531, 'recall': 0.5462184873949579, 'f1': 0.5603448275862069, 'number': 119}	{'precision': 0.8991825613079019, 'recall': 0.9192200557103064, 'f1': 0.909090909090909, 'number': 1077}	0.8625	0.8947	0.8783	0.7991
0.0003	115.79	2200	1.7988	{'precision': 0.8785714285714286, 'recall': 0.9033047735618115, 'f1': 0.8907664453832227, 'number': 817}	{'precision': 0.6190476190476191, 'recall': 0.5462184873949579, 'f1': 0.5803571428571429, 'number': 119}	{'precision': 0.8940639269406393, 'recall': 0.9090064995357474, 'f1': 0.9014732965009209, 'number': 1077}	0.8735	0.8852	0.8793	0.7950
0.0003	126.32	2400	1.8038	{'precision': 0.8584686774941995, 'recall': 0.9057527539779682, 'f1': 0.8814770696843359, 'number': 817}	{'precision': 0.63, 'recall': 0.5294117647058824, 'f1': 0.5753424657534247, 'number': 119}	{'precision': 0.8943533697632058, 'recall': 0.9117920148560817, 'f1': 0.9029885057471265, 'number': 1077}	0.8665	0.8867	0.8765	0.7953

Framework versions

• Transformers 4.35.0
• Pytorch 2.1.0+cu118
• Datasets 2.14.6
• Tokenizers 0.14.1