|
--- |
|
license: mit |
|
base_model: microsoft/layoutlm-base-uncased |
|
tags: |
|
- generated_from_trainer |
|
datasets: |
|
- funsd |
|
model-index: |
|
- name: layoutlm-funsd |
|
results: [] |
|
--- |
|
|
|
<!-- This model card has been generated automatically according to the information the Trainer had access to. You |
|
should probably proofread and complete it, then remove this comment. --> |
|
|
|
# layoutlm-funsd |
|
|
|
This model is a fine-tuned version of [microsoft/layoutlm-base-uncased](https://huggingface.co/microsoft/layoutlm-base-uncased) on the funsd dataset. |
|
It achieves the following results on the evaluation set: |
|
- Loss: 1.1516 |
|
- Answer: {'precision': 0.38400702987697716, 'recall': 0.5401730531520396, 'f1': 0.44889573703133023, 'number': 809} |
|
- Header: {'precision': 0.3218390804597701, 'recall': 0.23529411764705882, 'f1': 0.27184466019417475, 'number': 119} |
|
- Question: {'precision': 0.5132192846034215, 'recall': 0.6197183098591549, 'f1': 0.5614632071458954, 'number': 1065} |
|
- Overall Precision: 0.4480 |
|
- Overall Recall: 0.5645 |
|
- Overall F1: 0.4996 |
|
- Overall Accuracy: 0.6209 |
|
|
|
## Model description |
|
|
|
More information needed |
|
|
|
## Intended uses & limitations |
|
|
|
More information needed |
|
|
|
## Training and evaluation data |
|
|
|
More information needed |
|
|
|
## Training procedure |
|
|
|
### Training hyperparameters |
|
|
|
The following hyperparameters were used during training: |
|
- learning_rate: 3e-05 |
|
- train_batch_size: 16 |
|
- eval_batch_size: 8 |
|
- seed: 42 |
|
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08 |
|
- lr_scheduler_type: linear |
|
- num_epochs: 15 |
|
|
|
### Training results |
|
|
|
| Training Loss | Epoch | Step | Validation Loss | Answer | Header | Question | Overall Precision | Overall Recall | Overall F1 | Overall Accuracy | |
|
|:-------------:|:-----:|:----:|:---------------:|:-----------------------------------------------------------------------------------------------------------:|:-----------------------------------------------------------------------------------------------------------:|:-----------------------------------------------------------------------------------------------------------:|:-----------------:|:--------------:|:----------:|:----------------:| |
|
| 1.7219 | 1.0 | 10 | 1.5555 | {'precision': 0.04431137724550898, 'recall': 0.04573547589616811, 'f1': 0.04501216545012165, 'number': 809} | {'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119} | {'precision': 0.26320501342882724, 'recall': 0.27605633802816903, 'f1': 0.2694775435380385, 'number': 1065} | 0.1696 | 0.1661 | 0.1678 | 0.3589 | |
|
| 1.4917 | 2.0 | 20 | 1.3323 | {'precision': 0.17622377622377622, 'recall': 0.311495673671199, 'f1': 0.2251004912907548, 'number': 809} | {'precision': 0.1, 'recall': 0.008403361344537815, 'f1': 0.015503875968992248, 'number': 119} | {'precision': 0.29382407985028075, 'recall': 0.4422535211267606, 'f1': 0.3530734632683658, 'number': 1065} | 0.2379 | 0.3633 | 0.2875 | 0.4413 | |
|
| 1.2799 | 3.0 | 30 | 1.2482 | {'precision': 0.24236517218973358, 'recall': 0.4610630407911001, 'f1': 0.317717206132879, 'number': 809} | {'precision': 0.273972602739726, 'recall': 0.16806722689075632, 'f1': 0.20833333333333331, 'number': 119} | {'precision': 0.35655737704918034, 'recall': 0.4084507042253521, 'f1': 0.38074398249452956, 'number': 1065} | 0.2924 | 0.4155 | 0.3432 | 0.4580 | |
|
| 1.1477 | 4.0 | 40 | 1.1758 | {'precision': 0.2900516795865633, 'recall': 0.5550061804697157, 'f1': 0.38099278744166315, 'number': 809} | {'precision': 0.3559322033898305, 'recall': 0.17647058823529413, 'f1': 0.2359550561797753, 'number': 119} | {'precision': 0.4393939393939394, 'recall': 0.49014084507042255, 'f1': 0.46338215712383485, 'number': 1065} | 0.3549 | 0.4977 | 0.4144 | 0.5219 | |
|
| 1.0484 | 5.0 | 50 | 1.0885 | {'precision': 0.3271441202475685, 'recall': 0.4573547589616811, 'f1': 0.3814432989690722, 'number': 809} | {'precision': 0.2826086956521739, 'recall': 0.2184873949579832, 'f1': 0.24644549763033172, 'number': 119} | {'precision': 0.4808, 'recall': 0.564319248826291, 'f1': 0.5192224622030237, 'number': 1065} | 0.4032 | 0.5003 | 0.4465 | 0.5827 | |
|
| 0.9672 | 6.0 | 60 | 1.0745 | {'precision': 0.30431309904153353, 'recall': 0.47095179233621753, 'f1': 0.36972343522561857, 'number': 809} | {'precision': 0.34782608695652173, 'recall': 0.20168067226890757, 'f1': 0.25531914893617025, 'number': 119} | {'precision': 0.43936243936243935, 'recall': 0.5953051643192488, 'f1': 0.5055821371610846, 'number': 1065} | 0.3759 | 0.5213 | 0.4368 | 0.5916 | |
|
| 0.8787 | 7.0 | 70 | 1.1863 | {'precision': 0.3697033898305085, 'recall': 0.43139678615574784, 'f1': 0.3981745579007416, 'number': 809} | {'precision': 0.25, 'recall': 0.2184873949579832, 'f1': 0.23318385650224216, 'number': 119} | {'precision': 0.4801556420233463, 'recall': 0.5793427230046948, 'f1': 0.5251063829787234, 'number': 1065} | 0.4252 | 0.4977 | 0.4586 | 0.5870 | |
|
| 0.8501 | 8.0 | 80 | 1.1043 | {'precision': 0.31553860819828405, 'recall': 0.40914709517923364, 'f1': 0.3562970936490851, 'number': 809} | {'precision': 0.3484848484848485, 'recall': 0.19327731092436976, 'f1': 0.24864864864864866, 'number': 119} | {'precision': 0.41997593261131166, 'recall': 0.6553990610328638, 'f1': 0.5119178584525119, 'number': 1065} | 0.3788 | 0.5278 | 0.4411 | 0.5878 | |
|
| 0.805 | 9.0 | 90 | 1.0872 | {'precision': 0.3356828193832599, 'recall': 0.47095179233621753, 'f1': 0.39197530864197533, 'number': 809} | {'precision': 0.32894736842105265, 'recall': 0.21008403361344538, 'f1': 0.25641025641025644, 'number': 119} | {'precision': 0.45454545454545453, 'recall': 0.6197183098591549, 'f1': 0.5244338498212157, 'number': 1065} | 0.4003 | 0.5349 | 0.4579 | 0.6053 | |
|
| 0.7686 | 10.0 | 100 | 1.1006 | {'precision': 0.35418427726120033, 'recall': 0.5179233621755254, 'f1': 0.42068273092369474, 'number': 809} | {'precision': 0.3333333333333333, 'recall': 0.2184873949579832, 'f1': 0.2639593908629441, 'number': 119} | {'precision': 0.49634443541835904, 'recall': 0.5737089201877934, 'f1': 0.5322299651567944, 'number': 1065} | 0.4238 | 0.5299 | 0.4709 | 0.6028 | |
|
| 0.7078 | 11.0 | 110 | 1.1631 | {'precision': 0.38475665748393023, 'recall': 0.5179233621755254, 'f1': 0.4415173867228662, 'number': 809} | {'precision': 0.28846153846153844, 'recall': 0.25210084033613445, 'f1': 0.26905829596412556, 'number': 119} | {'precision': 0.520764119601329, 'recall': 0.5887323943661972, 'f1': 0.5526663728514765, 'number': 1065} | 0.4489 | 0.5399 | 0.4902 | 0.6064 | |
|
| 0.7162 | 12.0 | 120 | 1.1517 | {'precision': 0.36400817995910023, 'recall': 0.4400494437577256, 'f1': 0.3984331281477337, 'number': 809} | {'precision': 0.28421052631578947, 'recall': 0.226890756302521, 'f1': 0.25233644859813087, 'number': 119} | {'precision': 0.4661458333333333, 'recall': 0.672300469483568, 'f1': 0.550557477893118, 'number': 1065} | 0.4212 | 0.5514 | 0.4776 | 0.6014 | |
|
| 0.6912 | 13.0 | 130 | 1.2013 | {'precision': 0.3880718954248366, 'recall': 0.5871446229913473, 'f1': 0.4672897196261682, 'number': 809} | {'precision': 0.3888888888888889, 'recall': 0.23529411764705882, 'f1': 0.2931937172774869, 'number': 119} | {'precision': 0.5526552655265526, 'recall': 0.5765258215962441, 'f1': 0.5643382352941176, 'number': 1065} | 0.4641 | 0.5605 | 0.5077 | 0.6082 | |
|
| 0.664 | 14.0 | 140 | 1.1337 | {'precision': 0.37344028520499106, 'recall': 0.5179233621755254, 'f1': 0.4339720352149145, 'number': 809} | {'precision': 0.3218390804597701, 'recall': 0.23529411764705882, 'f1': 0.27184466019417475, 'number': 119} | {'precision': 0.5037650602409639, 'recall': 0.6281690140845071, 'f1': 0.5591307981613038, 'number': 1065} | 0.4399 | 0.5600 | 0.4927 | 0.6142 | |
|
| 0.6496 | 15.0 | 150 | 1.1516 | {'precision': 0.38400702987697716, 'recall': 0.5401730531520396, 'f1': 0.44889573703133023, 'number': 809} | {'precision': 0.3218390804597701, 'recall': 0.23529411764705882, 'f1': 0.27184466019417475, 'number': 119} | {'precision': 0.5132192846034215, 'recall': 0.6197183098591549, 'f1': 0.5614632071458954, 'number': 1065} | 0.4480 | 0.5645 | 0.4996 | 0.6209 | |
|
|
|
|
|
### Framework versions |
|
|
|
- Transformers 4.37.2 |
|
- Pytorch 2.1.0+cu121 |
|
- Datasets 2.17.1 |
|
- Tokenizers 0.15.2 |
|
|
