metadata
license: mit
base_model: microsoft/layoutlm-base-uncased
tags:
- generated_from_trainer
datasets:
- funsd
model-index:
- name: layoutlm-funsd
results: []
layoutlm-funsd
This model is a fine-tuned version of microsoft/layoutlm-base-uncased on the funsd dataset. It achieves the following results on the evaluation set:
- Loss: 1.1050
- Answer: {'precision': 0.37133808392715756, 'recall': 0.5797280593325093, 'f1': 0.45270270270270274, 'number': 809}
- Header: {'precision': 0.32926829268292684, 'recall': 0.226890756302521, 'f1': 0.26865671641791045, 'number': 119}
- Question: {'precision': 0.49682539682539684, 'recall': 0.5877934272300469, 'f1': 0.538494623655914, 'number': 1065}
- Overall Precision: 0.4307
- Overall Recall: 0.5630
- Overall F1: 0.4880
- Overall Accuracy: 0.6093
Model description
More information needed
Intended uses & limitations
More information needed
Training and evaluation data
More information needed
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 3e-05
- train_batch_size: 16
- eval_batch_size: 8
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 15
- mixed_precision_training: Native AMP
Training results
| Training Loss | Epoch | Step | Validation Loss | Answer | Header | Question | Overall Precision | Overall Recall | Overall F1 | Overall Accuracy |
|---|---|---|---|---|---|---|---|---|---|---|
| 1.8038 | 1.0 | 10 | 1.5073 | {'precision': 0.06441476826394343, 'recall': 0.10135970333745364, 'f1': 0.07877041306436118, 'number': 809} | {'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119} | {'precision': 0.24326241134751772, 'recall': 0.3220657276995305, 'f1': 0.2771717171717171, 'number': 1065} | 0.1584 | 0.2132 | 0.1818 | 0.3843 |
| 1.4521 | 2.0 | 20 | 1.3396 | {'precision': 0.20421753607103219, 'recall': 0.45488257107540175, 'f1': 0.28188433550363845, 'number': 809} | {'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119} | {'precision': 0.2649350649350649, 'recall': 0.38309859154929576, 'f1': 0.31324376199616116, 'number': 1065} | 0.2321 | 0.3894 | 0.2909 | 0.4184 |
| 1.278 | 3.0 | 30 | 1.2050 | {'precision': 0.2645794966236955, 'recall': 0.5327564894932015, 'f1': 0.3535684987694832, 'number': 809} | {'precision': 0.12903225806451613, 'recall': 0.06722689075630252, 'f1': 0.08839779005524862, 'number': 119} | {'precision': 0.34989503149055284, 'recall': 0.4694835680751174, 'f1': 0.400962309542903, 'number': 1065} | 0.3010 | 0.4711 | 0.3673 | 0.4760 |
| 1.1503 | 4.0 | 40 | 1.1044 | {'precision': 0.28089080459770116, 'recall': 0.48331273176761436, 'f1': 0.3552930486142663, 'number': 809} | {'precision': 0.2391304347826087, 'recall': 0.18487394957983194, 'f1': 0.2085308056872038, 'number': 119} | {'precision': 0.4, 'recall': 0.5295774647887324, 'f1': 0.45575757575757575, 'number': 1065} | 0.3376 | 0.4902 | 0.3998 | 0.5630 |
| 1.07 | 5.0 | 50 | 1.1546 | {'precision': 0.30014025245441794, 'recall': 0.5290482076637825, 'f1': 0.38299776286353465, 'number': 809} | {'precision': 0.3188405797101449, 'recall': 0.18487394957983194, 'f1': 0.23404255319148937, 'number': 119} | {'precision': 0.4058373870743572, 'recall': 0.5483568075117371, 'f1': 0.4664536741214057, 'number': 1065} | 0.3524 | 0.5188 | 0.4197 | 0.5383 |
| 0.9914 | 6.0 | 60 | 1.0507 | {'precision': 0.3119065010956903, 'recall': 0.5278121137206427, 'f1': 0.3921028466483012, 'number': 809} | {'precision': 0.2345679012345679, 'recall': 0.15966386554621848, 'f1': 0.18999999999999997, 'number': 119} | {'precision': 0.4122938530734633, 'recall': 0.5164319248826291, 'f1': 0.45852438516048355, 'number': 1065} | 0.3578 | 0.4997 | 0.4170 | 0.6002 |
| 0.9373 | 7.0 | 70 | 1.0652 | {'precision': 0.3710691823899371, 'recall': 0.43757725587144625, 'f1': 0.4015882019285309, 'number': 809} | {'precision': 0.25510204081632654, 'recall': 0.21008403361344538, 'f1': 0.23041474654377883, 'number': 119} | {'precision': 0.4739583333333333, 'recall': 0.5981220657276995, 'f1': 0.5288501452885015, 'number': 1065} | 0.4240 | 0.5098 | 0.4630 | 0.6006 |
| 0.8833 | 8.0 | 80 | 1.0389 | {'precision': 0.3351605324980423, 'recall': 0.5290482076637825, 'f1': 0.4103547459252157, 'number': 809} | {'precision': 0.375, 'recall': 0.20168067226890757, 'f1': 0.2622950819672132, 'number': 119} | {'precision': 0.44528301886792454, 'recall': 0.5539906103286385, 'f1': 0.49372384937238495, 'number': 1065} | 0.3908 | 0.5228 | 0.4473 | 0.6143 |
| 0.8029 | 9.0 | 90 | 1.0520 | {'precision': 0.3685612788632327, 'recall': 0.5129789864029666, 'f1': 0.4289405684754522, 'number': 809} | {'precision': 0.28695652173913044, 'recall': 0.2773109243697479, 'f1': 0.2820512820512821, 'number': 119} | {'precision': 0.4902874902874903, 'recall': 0.5924882629107981, 'f1': 0.5365646258503401, 'number': 1065} | 0.4268 | 0.5414 | 0.4773 | 0.6023 |
| 0.7658 | 10.0 | 100 | 1.0764 | {'precision': 0.3386511965192168, 'recall': 0.5772558714462299, 'f1': 0.42687385740402195, 'number': 809} | {'precision': 0.3709677419354839, 'recall': 0.19327731092436976, 'f1': 0.2541436464088398, 'number': 119} | {'precision': 0.4847986852917009, 'recall': 0.5539906103286385, 'f1': 0.5170902716914987, 'number': 1065} | 0.4063 | 0.5419 | 0.4644 | 0.6066 |
| 0.7112 | 11.0 | 110 | 1.0675 | {'precision': 0.3728963684676705, 'recall': 0.5203955500618047, 'f1': 0.43446852425180593, 'number': 809} | {'precision': 0.3333333333333333, 'recall': 0.21008403361344538, 'f1': 0.2577319587628866, 'number': 119} | {'precision': 0.4918032786885246, 'recall': 0.5915492957746479, 'f1': 0.5370843989769821, 'number': 1065} | 0.4330 | 0.5399 | 0.4806 | 0.6124 |
| 0.6875 | 12.0 | 120 | 1.1100 | {'precision': 0.37746256895193064, 'recall': 0.5920889987639061, 'f1': 0.46102021174205965, 'number': 809} | {'precision': 0.33783783783783783, 'recall': 0.21008403361344538, 'f1': 0.25906735751295334, 'number': 119} | {'precision': 0.514554794520548, 'recall': 0.564319248826291, 'f1': 0.5382892969099866, 'number': 1065} | 0.4401 | 0.5544 | 0.4907 | 0.6102 |
| 0.6571 | 13.0 | 130 | 1.0804 | {'precision': 0.36231884057971014, 'recall': 0.5253399258343634, 'f1': 0.4288597376387487, 'number': 809} | {'precision': 0.313953488372093, 'recall': 0.226890756302521, 'f1': 0.2634146341463415, 'number': 119} | {'precision': 0.46940244780417567, 'recall': 0.612206572769953, 'f1': 0.5313773431132844, 'number': 1065} | 0.4169 | 0.5539 | 0.4758 | 0.6141 |
| 0.6564 | 14.0 | 140 | 1.0934 | {'precision': 0.37943262411347517, 'recall': 0.5290482076637825, 'f1': 0.44192049561177077, 'number': 809} | {'precision': 0.37662337662337664, 'recall': 0.24369747899159663, 'f1': 0.29591836734693877, 'number': 119} | {'precision': 0.49803613511390415, 'recall': 0.5953051643192488, 'f1': 0.542343883661249, 'number': 1065} | 0.4403 | 0.5474 | 0.4880 | 0.6215 |
| 0.6558 | 15.0 | 150 | 1.1050 | {'precision': 0.37133808392715756, 'recall': 0.5797280593325093, 'f1': 0.45270270270270274, 'number': 809} | {'precision': 0.32926829268292684, 'recall': 0.226890756302521, 'f1': 0.26865671641791045, 'number': 119} | {'precision': 0.49682539682539684, 'recall': 0.5877934272300469, 'f1': 0.538494623655914, 'number': 1065} | 0.4307 | 0.5630 | 0.4880 | 0.6093 |
Framework versions
- Transformers 4.36.2
- Pytorch 2.1.2+cu118
- Datasets 2.18.0
- Tokenizers 0.15.0