File size: 9,354 Bytes
a166494 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 |
---
license: mit
base_model: microsoft/layoutlm-base-uncased
tags:
- generated_from_trainer
datasets:
- funsd
model-index:
- name: layoutlm-funsd
results: []
---
<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->
# layoutlm-funsd
This model is a fine-tuned version of [microsoft/layoutlm-base-uncased](https://huggingface.co/microsoft/layoutlm-base-uncased) on the funsd dataset.
It achieves the following results on the evaluation set:
- Loss: 1.0745
- Answer: {'precision': 0.3554006968641115, 'recall': 0.5043263288009888, 'f1': 0.41696474195196725, 'number': 809}
- Header: {'precision': 0.3411764705882353, 'recall': 0.24369747899159663, 'f1': 0.28431372549019607, 'number': 119}
- Question: {'precision': 0.4910979228486647, 'recall': 0.6215962441314554, 'f1': 0.5486945710733527, 'number': 1065}
- Overall Precision: 0.4258
- Overall Recall: 0.5514
- Overall F1: 0.4805
- Overall Accuracy: 0.6117
## Model description
More information needed
## Intended uses & limitations
More information needed
## Training and evaluation data
More information needed
## Training procedure
### Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 3e-05
- train_batch_size: 16
- eval_batch_size: 8
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 15
- mixed_precision_training: Native AMP
### Training results
| Training Loss | Epoch | Step | Validation Loss | Answer | Header | Question | Overall Precision | Overall Recall | Overall F1 | Overall Accuracy |
|:-------------:|:-----:|:----:|:---------------:|:------------------------------------------------------------------------------------------------------------:|:------------------------------------------------------------------------------------------------------------:|:-----------------------------------------------------------------------------------------------------------:|:-----------------:|:--------------:|:----------:|:----------------:|
| 1.7729 | 1.0 | 10 | 1.5447 | {'precision': 0.04415584415584416, 'recall': 0.042027194066749075, 'f1': 0.04306523115896137, 'number': 809} | {'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119} | {'precision': 0.22091584158415842, 'recall': 0.3352112676056338, 'f1': 0.26631853785900783, 'number': 1065} | 0.1639 | 0.1962 | 0.1786 | 0.3568 |
| 1.4627 | 2.0 | 20 | 1.3779 | {'precision': 0.12121212121212122, 'recall': 0.2373300370828183, 'f1': 0.16046803175929797, 'number': 809} | {'precision': 0.04081632653061224, 'recall': 0.01680672268907563, 'f1': 0.023809523809523808, 'number': 119} | {'precision': 0.23460246360582307, 'recall': 0.39342723004694835, 'f1': 0.293931953700456, 'number': 1065} | 0.1793 | 0.3076 | 0.2265 | 0.4108 |
| 1.2914 | 3.0 | 30 | 1.2345 | {'precision': 0.15814696485623003, 'recall': 0.24474660074165636, 'f1': 0.19213973799126638, 'number': 809} | {'precision': 0.15789473684210525, 'recall': 0.12605042016806722, 'f1': 0.14018691588785046, 'number': 119} | {'precision': 0.31094527363184077, 'recall': 0.5868544600938967, 'f1': 0.4065040650406504, 'number': 1065} | 0.2496 | 0.4205 | 0.3133 | 0.4524 |
| 1.1698 | 4.0 | 40 | 1.1615 | {'precision': 0.2040990606319385, 'recall': 0.2954264524103832, 'f1': 0.2414141414141414, 'number': 809} | {'precision': 0.19387755102040816, 'recall': 0.15966386554621848, 'f1': 0.17511520737327188, 'number': 119} | {'precision': 0.351129363449692, 'recall': 0.6422535211267606, 'f1': 0.4540325257218719, 'number': 1065} | 0.2928 | 0.4727 | 0.3616 | 0.4925 |
| 1.096 | 5.0 | 50 | 1.1141 | {'precision': 0.22423802612481858, 'recall': 0.3819530284301607, 'f1': 0.28257887517146774, 'number': 809} | {'precision': 0.2682926829268293, 'recall': 0.18487394957983194, 'f1': 0.21890547263681595, 'number': 119} | {'precision': 0.3757159221076747, 'recall': 0.615962441314554, 'f1': 0.4667378157239417, 'number': 1065} | 0.3079 | 0.4952 | 0.3797 | 0.5360 |
| 1.0157 | 6.0 | 60 | 1.0480 | {'precision': 0.27807900852052675, 'recall': 0.4437577255871446, 'f1': 0.34190476190476193, 'number': 809} | {'precision': 0.3013698630136986, 'recall': 0.18487394957983194, 'f1': 0.22916666666666669, 'number': 119} | {'precision': 0.45481049562682213, 'recall': 0.5859154929577465, 'f1': 0.512105047189167, 'number': 1065} | 0.3673 | 0.5043 | 0.4250 | 0.5881 |
| 0.9412 | 7.0 | 70 | 1.0314 | {'precision': 0.29177057356608477, 'recall': 0.4338689740420272, 'f1': 0.34890656063618286, 'number': 809} | {'precision': 0.2926829268292683, 'recall': 0.20168067226890757, 'f1': 0.23880597014925373, 'number': 119} | {'precision': 0.45625451916124365, 'recall': 0.5924882629107981, 'f1': 0.5155228758169934, 'number': 1065} | 0.3771 | 0.5048 | 0.4317 | 0.5961 |
| 0.8828 | 8.0 | 80 | 1.0804 | {'precision': 0.3174061433447099, 'recall': 0.45982694684796044, 'f1': 0.37556789500252397, 'number': 809} | {'precision': 0.2828282828282828, 'recall': 0.23529411764705882, 'f1': 0.25688073394495414, 'number': 119} | {'precision': 0.46117804551539493, 'recall': 0.6469483568075117, 'f1': 0.5384915982805784, 'number': 1065} | 0.3939 | 0.5464 | 0.4578 | 0.5872 |
| 0.8304 | 9.0 | 90 | 1.0436 | {'precision': 0.3404255319148936, 'recall': 0.49443757725587145, 'f1': 0.40322580645161293, 'number': 809} | {'precision': 0.36363636363636365, 'recall': 0.23529411764705882, 'f1': 0.2857142857142857, 'number': 119} | {'precision': 0.4878765613519471, 'recall': 0.6234741784037559, 'f1': 0.5474031327287716, 'number': 1065} | 0.4179 | 0.5479 | 0.4742 | 0.6095 |
| 0.814 | 10.0 | 100 | 1.0871 | {'precision': 0.3464391691394659, 'recall': 0.5772558714462299, 'f1': 0.4330088085303662, 'number': 809} | {'precision': 0.4166666666666667, 'recall': 0.25210084033613445, 'f1': 0.31413612565445026, 'number': 119} | {'precision': 0.5084294587400178, 'recall': 0.5380281690140845, 'f1': 0.5228102189781022, 'number': 1065} | 0.4201 | 0.5369 | 0.4714 | 0.5989 |
| 0.7273 | 11.0 | 110 | 1.0650 | {'precision': 0.3483348334833483, 'recall': 0.4783683559950556, 'f1': 0.40312499999999996, 'number': 809} | {'precision': 0.30434782608695654, 'recall': 0.23529411764705882, 'f1': 0.2654028436018957, 'number': 119} | {'precision': 0.4900953778429934, 'recall': 0.6272300469483568, 'f1': 0.5502471169686985, 'number': 1065} | 0.4221 | 0.5434 | 0.4751 | 0.6139 |
| 0.7257 | 12.0 | 120 | 1.1221 | {'precision': 0.34212629896083135, 'recall': 0.5290482076637825, 'f1': 0.41553398058252433, 'number': 809} | {'precision': 0.38666666666666666, 'recall': 0.24369747899159663, 'f1': 0.29896907216494845, 'number': 119} | {'precision': 0.48787878787878786, 'recall': 0.6046948356807512, 'f1': 0.5400419287211741, 'number': 1065} | 0.4161 | 0.5524 | 0.4747 | 0.6032 |
| 0.694 | 13.0 | 130 | 1.0688 | {'precision': 0.3702451394759087, 'recall': 0.5414091470951793, 'f1': 0.43975903614457834, 'number': 809} | {'precision': 0.345679012345679, 'recall': 0.23529411764705882, 'f1': 0.27999999999999997, 'number': 119} | {'precision': 0.5052041633306645, 'recall': 0.5924882629107981, 'f1': 0.5453759723422645, 'number': 1065} | 0.4365 | 0.5504 | 0.4869 | 0.6148 |
| 0.6617 | 14.0 | 140 | 1.0465 | {'precision': 0.3598901098901099, 'recall': 0.4857849196538937, 'f1': 0.41346659652814305, 'number': 809} | {'precision': 0.3411764705882353, 'recall': 0.24369747899159663, 'f1': 0.28431372549019607, 'number': 119} | {'precision': 0.48916184971098264, 'recall': 0.6356807511737089, 'f1': 0.5528787260106166, 'number': 1065} | 0.4291 | 0.5514 | 0.4827 | 0.6191 |
| 0.6536 | 15.0 | 150 | 1.0745 | {'precision': 0.3554006968641115, 'recall': 0.5043263288009888, 'f1': 0.41696474195196725, 'number': 809} | {'precision': 0.3411764705882353, 'recall': 0.24369747899159663, 'f1': 0.28431372549019607, 'number': 119} | {'precision': 0.4910979228486647, 'recall': 0.6215962441314554, 'f1': 0.5486945710733527, 'number': 1065} | 0.4258 | 0.5514 | 0.4805 | 0.6117 |
### Framework versions
- Transformers 4.38.2
- Pytorch 2.1.0+cu121
- Datasets 2.18.0
- Tokenizers 0.15.2
|