|
--- |
|
tags: |
|
- generated_from_trainer |
|
datasets: |
|
- funsd |
|
model-index: |
|
- name: ananth-docai1 |
|
results: [] |
|
--- |
|
|
|
<!-- This model card has been generated automatically according to the information the Trainer had access to. You |
|
should probably proofread and complete it, then remove this comment. --> |
|
|
|
# ananth-docai1 |
|
|
|
This model is a fine-tuned version of [microsoft/layoutlm-base-uncased](https://huggingface.co/microsoft/layoutlm-base-uncased) on the funsd dataset. |
|
It achieves the following results on the evaluation set: |
|
- Loss: 0.7024 |
|
- Answer: {'precision': 0.7113513513513513, 'recall': 0.8133498145859085, 'f1': 0.7589388696655133, 'number': 809} |
|
- Header: {'precision': 0.30952380952380953, 'recall': 0.3277310924369748, 'f1': 0.31836734693877555, 'number': 119} |
|
- Question: {'precision': 0.7811387900355872, 'recall': 0.8244131455399061, 'f1': 0.8021927820922796, 'number': 1065} |
|
- Overall Precision: 0.7241 |
|
- Overall Recall: 0.7903 |
|
- Overall F1: 0.7558 |
|
- Overall Accuracy: 0.8106 |
|
|
|
## Model description |
|
|
|
More information needed |
|
|
|
## Intended uses & limitations |
|
|
|
More information needed |
|
|
|
## Training and evaluation data |
|
|
|
More information needed |
|
|
|
## Training procedure |
|
|
|
### Training hyperparameters |
|
|
|
The following hyperparameters were used during training: |
|
- learning_rate: 3e-05 |
|
- train_batch_size: 16 |
|
- eval_batch_size: 8 |
|
- seed: 42 |
|
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08 |
|
- lr_scheduler_type: linear |
|
- num_epochs: 15 |
|
- mixed_precision_training: Native AMP |
|
|
|
### Training results |
|
|
|
| Training Loss | Epoch | Step | Validation Loss | Answer | Header | Question | Overall Precision | Overall Recall | Overall F1 | Overall Accuracy | |
|
|:-------------:|:-----:|:----:|:---------------:|:-------------------------------------------------------------------------------------------------------------:|:------------------------------------------------------------------------------------------------------------:|:-----------------------------------------------------------------------------------------------------------:|:-----------------:|:--------------:|:----------:|:----------------:| |
|
| 1.7944 | 1.0 | 10 | 1.6233 | {'precision': 0.01929260450160772, 'recall': 0.014833127317676144, 'f1': 0.016771488469601678, 'number': 809} | {'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119} | {'precision': 0.27685325264750377, 'recall': 0.17183098591549295, 'f1': 0.2120509849362688, 'number': 1065} | 0.1520 | 0.0978 | 0.1190 | 0.3505 | |
|
| 1.5001 | 2.0 | 20 | 1.2971 | {'precision': 0.11125, 'recall': 0.1100123609394314, 'f1': 0.11062771908017402, 'number': 809} | {'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119} | {'precision': 0.4044059795436664, 'recall': 0.48262910798122066, 'f1': 0.4400684931506849, 'number': 1065} | 0.2912 | 0.3026 | 0.2968 | 0.5348 | |
|
| 1.136 | 3.0 | 30 | 0.9852 | {'precision': 0.4911699779249448, 'recall': 0.5500618046971569, 'f1': 0.5189504373177842, 'number': 809} | {'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119} | {'precision': 0.6086587436332768, 'recall': 0.6732394366197183, 'f1': 0.6393223361569326, 'number': 1065} | 0.5562 | 0.5830 | 0.5693 | 0.6941 | |
|
| 0.8567 | 4.0 | 40 | 0.8143 | {'precision': 0.627744510978044, 'recall': 0.7775030902348579, 'f1': 0.6946438431805633, 'number': 809} | {'precision': 0.06666666666666667, 'recall': 0.01680672268907563, 'f1': 0.026845637583892617, 'number': 119} | {'precision': 0.6987179487179487, 'recall': 0.7164319248826291, 'f1': 0.7074640704682429, 'number': 1065} | 0.6563 | 0.6994 | 0.6772 | 0.7467 | |
|
| 0.6998 | 5.0 | 50 | 0.7133 | {'precision': 0.6534859521331946, 'recall': 0.7762669962917181, 'f1': 0.7096045197740113, 'number': 809} | {'precision': 0.2, 'recall': 0.11764705882352941, 'f1': 0.14814814814814817, 'number': 119} | {'precision': 0.7243532560214094, 'recall': 0.7624413145539906, 'f1': 0.7429094236047575, 'number': 1065} | 0.6757 | 0.7296 | 0.7016 | 0.7781 | |
|
| 0.5886 | 6.0 | 60 | 0.6775 | {'precision': 0.648406374501992, 'recall': 0.8046971569839307, 'f1': 0.7181467181467182, 'number': 809} | {'precision': 0.25806451612903225, 'recall': 0.13445378151260504, 'f1': 0.17679558011049723, 'number': 119} | {'precision': 0.712947189097104, 'recall': 0.7859154929577464, 'f1': 0.7476552032157214, 'number': 1065} | 0.6714 | 0.7546 | 0.7106 | 0.7890 | |
|
| 0.5185 | 7.0 | 70 | 0.6770 | {'precision': 0.6755888650963597, 'recall': 0.7799752781211372, 'f1': 0.7240390131956398, 'number': 809} | {'precision': 0.2079207920792079, 'recall': 0.17647058823529413, 'f1': 0.19090909090909092, 'number': 119} | {'precision': 0.7341337907375644, 'recall': 0.8037558685446009, 'f1': 0.7673688928731511, 'number': 1065} | 0.6851 | 0.7566 | 0.7191 | 0.7955 | |
|
| 0.4672 | 8.0 | 80 | 0.6729 | {'precision': 0.683083511777302, 'recall': 0.788627935723115, 'f1': 0.7320711417096959, 'number': 809} | {'precision': 0.23300970873786409, 'recall': 0.20168067226890757, 'f1': 0.21621621621621623, 'number': 119} | {'precision': 0.747431506849315, 'recall': 0.819718309859155, 'f1': 0.7819077474249888, 'number': 1065} | 0.6961 | 0.7702 | 0.7313 | 0.8007 | |
|
| 0.4188 | 9.0 | 90 | 0.6664 | {'precision': 0.6888888888888889, 'recall': 0.8046971569839307, 'f1': 0.74230330672748, 'number': 809} | {'precision': 0.2727272727272727, 'recall': 0.25210084033613445, 'f1': 0.26200873362445415, 'number': 119} | {'precision': 0.7708703374777975, 'recall': 0.8150234741784037, 'f1': 0.792332268370607, 'number': 1065} | 0.7102 | 0.7772 | 0.7422 | 0.8045 | |
|
| 0.3724 | 10.0 | 100 | 0.6845 | {'precision': 0.6928721174004193, 'recall': 0.8170580964153276, 'f1': 0.7498581962563812, 'number': 809} | {'precision': 0.33, 'recall': 0.2773109243697479, 'f1': 0.30136986301369867, 'number': 119} | {'precision': 0.7818343722172751, 'recall': 0.8244131455399061, 'f1': 0.8025594149908593, 'number': 1065} | 0.7221 | 0.7888 | 0.7540 | 0.8047 | |
|
| 0.3402 | 11.0 | 110 | 0.6830 | {'precision': 0.7118093174431203, 'recall': 0.8121137206427689, 'f1': 0.7586605080831409, 'number': 809} | {'precision': 0.3090909090909091, 'recall': 0.2857142857142857, 'f1': 0.296943231441048, 'number': 119} | {'precision': 0.787422497785651, 'recall': 0.8347417840375587, 'f1': 0.8103919781221514, 'number': 1065} | 0.7308 | 0.7928 | 0.7605 | 0.8129 | |
|
| 0.3219 | 12.0 | 120 | 0.6944 | {'precision': 0.7179203539823009, 'recall': 0.8022249690976514, 'f1': 0.7577349678925861, 'number': 809} | {'precision': 0.3220338983050847, 'recall': 0.31932773109243695, 'f1': 0.32067510548523204, 'number': 119} | {'precision': 0.781882145998241, 'recall': 0.8347417840375587, 'f1': 0.8074477747502271, 'number': 1065} | 0.7300 | 0.7908 | 0.7592 | 0.8097 | |
|
| 0.3004 | 13.0 | 130 | 0.6978 | {'precision': 0.7147540983606557, 'recall': 0.8084054388133498, 'f1': 0.7587006960556845, 'number': 809} | {'precision': 0.33043478260869563, 'recall': 0.31932773109243695, 'f1': 0.32478632478632474, 'number': 119} | {'precision': 0.7890974084003575, 'recall': 0.8291079812206573, 'f1': 0.8086080586080587, 'number': 1065} | 0.7329 | 0.7903 | 0.7605 | 0.8144 | |
|
| 0.2942 | 14.0 | 140 | 0.7001 | {'precision': 0.7145945945945946, 'recall': 0.8170580964153276, 'f1': 0.7623990772779701, 'number': 809} | {'precision': 0.30708661417322836, 'recall': 0.3277310924369748, 'f1': 0.3170731707317073, 'number': 119} | {'precision': 0.7820284697508897, 'recall': 0.8253521126760563, 'f1': 0.8031064412973961, 'number': 1065} | 0.7256 | 0.7923 | 0.7575 | 0.8108 | |
|
| 0.2853 | 15.0 | 150 | 0.7024 | {'precision': 0.7113513513513513, 'recall': 0.8133498145859085, 'f1': 0.7589388696655133, 'number': 809} | {'precision': 0.30952380952380953, 'recall': 0.3277310924369748, 'f1': 0.31836734693877555, 'number': 119} | {'precision': 0.7811387900355872, 'recall': 0.8244131455399061, 'f1': 0.8021927820922796, 'number': 1065} | 0.7241 | 0.7903 | 0.7558 | 0.8106 | |
|
|
|
|
|
### Framework versions |
|
|
|
- Transformers 4.25.1 |
|
- Pytorch 1.13.1+cu117 |
|
- Datasets 2.7.1 |
|
- Tokenizers 0.13.2 |
|
|
