layoutlm-funsd

This model is a fine-tuned version of microsoft/layoutlm-base-uncased on the funsd dataset. It achieves the following results on the evaluation set:

• Loss: 0.8174
• Answer: {'precision': 0.7233333333333334, 'recall': 0.8046971569839307, 'f1': 0.7618490345231129, 'number': 809}
• Header: {'precision': 0.35766423357664234, 'recall': 0.4117647058823529, 'f1': 0.3828125, 'number': 119}
• Question: {'precision': 0.7904085257548845, 'recall': 0.8356807511737089, 'f1': 0.8124144226380648, 'number': 1065}
• Overall Precision: 0.7351
• Overall Recall: 0.7978
• Overall F1: 0.7652
• Overall Accuracy: 0.8019

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 3e-05
• train_batch_size: 16
• eval_batch_size: 8
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 20

Training results

Training Loss	Epoch	Step	Validation Loss	Answer	Header	Question	Overall Precision	Overall Recall	Overall F1	Overall Accuracy
1.3435	1.0	10	1.1455	{'precision': 0.29554655870445345, 'recall': 0.27070457354758964, 'f1': 0.2825806451612903, 'number': 809}	{'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119}	{'precision': 0.43828125, 'recall': 0.5267605633802817, 'f1': 0.47846481876332625, 'number': 1065}	0.3858	0.3914	0.3885	0.6180
0.9706	2.0	20	0.8933	{'precision': 0.5545454545454546, 'recall': 0.6786155747836835, 'f1': 0.6103390772651472, 'number': 809}	{'precision': 0.08695652173913043, 'recall': 0.03361344537815126, 'f1': 0.048484848484848485, 'number': 119}	{'precision': 0.6115916955017301, 'recall': 0.6638497652582159, 'f1': 0.6366501575866726, 'number': 1065}	0.5748	0.6322	0.6022	0.7308
0.7426	3.0	30	0.7478	{'precision': 0.6294058408862034, 'recall': 0.7725587144622992, 'f1': 0.6936736958934517, 'number': 809}	{'precision': 0.1891891891891892, 'recall': 0.11764705882352941, 'f1': 0.1450777202072539, 'number': 119}	{'precision': 0.6858333333333333, 'recall': 0.7727699530516432, 'f1': 0.726710816777042, 'number': 1065}	0.6449	0.7336	0.6864	0.7770
0.6123	4.0	40	0.6950	{'precision': 0.6286266924564797, 'recall': 0.8034610630407911, 'f1': 0.705371676614216, 'number': 809}	{'precision': 0.19387755102040816, 'recall': 0.15966386554621848, 'f1': 0.17511520737327188, 'number': 119}	{'precision': 0.6943268416596104, 'recall': 0.7699530516431925, 'f1': 0.730186999109528, 'number': 1065}	0.6438	0.7471	0.6916	0.7886
0.5267	5.0	50	0.6804	{'precision': 0.6574172892209178, 'recall': 0.761433868974042, 'f1': 0.7056128293241695, 'number': 809}	{'precision': 0.21818181818181817, 'recall': 0.20168067226890757, 'f1': 0.2096069868995633, 'number': 119}	{'precision': 0.7246496290189612, 'recall': 0.8253521126760563, 'f1': 0.771729587357331, 'number': 1065}	0.6721	0.7622	0.7143	0.8013
0.4587	6.0	60	0.6701	{'precision': 0.670490093847758, 'recall': 0.7948084054388134, 'f1': 0.7273755656108597, 'number': 809}	{'precision': 0.2108843537414966, 'recall': 0.2605042016806723, 'f1': 0.2330827067669173, 'number': 119}	{'precision': 0.7309602649006622, 'recall': 0.8291079812206573, 'f1': 0.7769467663880335, 'number': 1065}	0.6729	0.7812	0.7230	0.7977
0.3981	7.0	70	0.6637	{'precision': 0.7029063509149623, 'recall': 0.8071693448702101, 'f1': 0.7514384349827388, 'number': 809}	{'precision': 0.2698412698412698, 'recall': 0.2857142857142857, 'f1': 0.27755102040816326, 'number': 119}	{'precision': 0.7621483375959079, 'recall': 0.8394366197183099, 'f1': 0.7989276139410187, 'number': 1065}	0.7096	0.7933	0.7491	0.8062
0.3608	8.0	80	0.6778	{'precision': 0.7083333333333334, 'recall': 0.7985166872682324, 'f1': 0.7507263219058687, 'number': 809}	{'precision': 0.25874125874125875, 'recall': 0.31092436974789917, 'f1': 0.2824427480916031, 'number': 119}	{'precision': 0.7633851468048359, 'recall': 0.8300469483568075, 'f1': 0.7953216374269007, 'number': 1065}	0.7081	0.7863	0.7451	0.8003
0.311	9.0	90	0.6931	{'precision': 0.6991247264770241, 'recall': 0.7898640296662547, 'f1': 0.7417295414973882, 'number': 809}	{'precision': 0.2835820895522388, 'recall': 0.31932773109243695, 'f1': 0.30039525691699603, 'number': 119}	{'precision': 0.7606244579358196, 'recall': 0.8234741784037559, 'f1': 0.7908025247971145, 'number': 1065}	0.7060	0.7797	0.7411	0.8055
0.276	10.0	100	0.7144	{'precision': 0.7298787210584344, 'recall': 0.8182941903584673, 'f1': 0.7715617715617716, 'number': 809}	{'precision': 0.3103448275862069, 'recall': 0.37815126050420167, 'f1': 0.34090909090909094, 'number': 119}	{'precision': 0.7814159292035399, 'recall': 0.8291079812206573, 'f1': 0.8045558086560365, 'number': 1065}	0.7287	0.7978	0.7617	0.8062
0.2393	11.0	110	0.7342	{'precision': 0.7155555555555555, 'recall': 0.796044499381953, 'f1': 0.7536571094207138, 'number': 809}	{'precision': 0.296551724137931, 'recall': 0.36134453781512604, 'f1': 0.32575757575757575, 'number': 119}	{'precision': 0.774869109947644, 'recall': 0.8338028169014085, 'f1': 0.8032564450474899, 'number': 1065}	0.7188	0.7903	0.7529	0.8042
0.2227	12.0	120	0.7539	{'precision': 0.7054945054945055, 'recall': 0.7935723114956736, 'f1': 0.7469458987783596, 'number': 809}	{'precision': 0.33884297520661155, 'recall': 0.3445378151260504, 'f1': 0.3416666666666667, 'number': 119}	{'precision': 0.7686440677966102, 'recall': 0.8516431924882629, 'f1': 0.8080178173719377, 'number': 1065}	0.7191	0.7978	0.7564	0.8006
0.2119	13.0	130	0.7774	{'precision': 0.7263736263736263, 'recall': 0.8170580964153276, 'f1': 0.7690517742873763, 'number': 809}	{'precision': 0.28125, 'recall': 0.37815126050420167, 'f1': 0.3225806451612903, 'number': 119}	{'precision': 0.7714033539276258, 'recall': 0.8206572769953052, 'f1': 0.7952684258416743, 'number': 1065}	0.7172	0.7928	0.7531	0.7952
0.1882	14.0	140	0.7688	{'precision': 0.7270668176670442, 'recall': 0.7935723114956736, 'f1': 0.7588652482269503, 'number': 809}	{'precision': 0.3384615384615385, 'recall': 0.3697478991596639, 'f1': 0.35341365461847385, 'number': 119}	{'precision': 0.7883597883597884, 'recall': 0.8394366197183099, 'f1': 0.8130968622100955, 'number': 1065}	0.7359	0.7928	0.7633	0.8024
0.1767	15.0	150	0.7717	{'precision': 0.7244785949506037, 'recall': 0.8158220024721878, 'f1': 0.7674418604651163, 'number': 809}	{'precision': 0.3548387096774194, 'recall': 0.3697478991596639, 'f1': 0.36213991769547327, 'number': 119}	{'precision': 0.789612676056338, 'recall': 0.8422535211267606, 'f1': 0.8150840527033166, 'number': 1065}	0.7374	0.8033	0.7690	0.8020
0.1703	16.0	160	0.7943	{'precision': 0.7231638418079096, 'recall': 0.7911001236093943, 'f1': 0.755608028335301, 'number': 809}	{'precision': 0.36231884057971014, 'recall': 0.42016806722689076, 'f1': 0.38910505836575876, 'number': 119}	{'precision': 0.79185119574845, 'recall': 0.8394366197183099, 'f1': 0.8149498632634458, 'number': 1065}	0.7361	0.7948	0.7643	0.8017
0.1643	17.0	170	0.8087	{'precision': 0.7207207207207207, 'recall': 0.7911001236093943, 'f1': 0.7542722451384797, 'number': 809}	{'precision': 0.33098591549295775, 'recall': 0.3949579831932773, 'f1': 0.3601532567049809, 'number': 119}	{'precision': 0.7932263814616756, 'recall': 0.8356807511737089, 'f1': 0.8139003200731596, 'number': 1065}	0.7328	0.7913	0.7609	0.7990
0.1443	18.0	180	0.8170	{'precision': 0.7230419977298524, 'recall': 0.7873918417799752, 'f1': 0.7538461538461538, 'number': 809}	{'precision': 0.36231884057971014, 'recall': 0.42016806722689076, 'f1': 0.38910505836575876, 'number': 119}	{'precision': 0.7898936170212766, 'recall': 0.8366197183098592, 'f1': 0.8125854993160054, 'number': 1065}	0.7350	0.7918	0.7623	0.7994
0.148	19.0	190	0.8169	{'precision': 0.7245240761478163, 'recall': 0.799752781211372, 'f1': 0.7602820211515863, 'number': 809}	{'precision': 0.35766423357664234, 'recall': 0.4117647058823529, 'f1': 0.3828125, 'number': 119}	{'precision': 0.792149866190901, 'recall': 0.8338028169014085, 'f1': 0.8124428179322964, 'number': 1065}	0.7364	0.7948	0.7645	0.8015
0.1441	20.0	200	0.8174	{'precision': 0.7233333333333334, 'recall': 0.8046971569839307, 'f1': 0.7618490345231129, 'number': 809}	{'precision': 0.35766423357664234, 'recall': 0.4117647058823529, 'f1': 0.3828125, 'number': 119}	{'precision': 0.7904085257548845, 'recall': 0.8356807511737089, 'f1': 0.8124144226380648, 'number': 1065}	0.7351	0.7978	0.7652	0.8019

Framework versions

• Transformers 4.34.0
• Pytorch 2.0.1+cu118
• Datasets 2.14.5
• Tokenizers 0.14.1