metadata
tags:
- generated_from_trainer
datasets:
- funsd
model-index:
- name: ananth-docai1
results: []
ananth-docai1
This model is a fine-tuned version of microsoft/layoutlm-base-uncased on the funsd dataset. It achieves the following results on the evaluation set:
- Loss: 0.7024
- Answer: {'precision': 0.7113513513513513, 'recall': 0.8133498145859085, 'f1': 0.7589388696655133, 'number': 809}
- Header: {'precision': 0.30952380952380953, 'recall': 0.3277310924369748, 'f1': 0.31836734693877555, 'number': 119}
- Question: {'precision': 0.7811387900355872, 'recall': 0.8244131455399061, 'f1': 0.8021927820922796, 'number': 1065}
- Overall Precision: 0.7241
- Overall Recall: 0.7903
- Overall F1: 0.7558
- Overall Accuracy: 0.8106
Model description
More information needed
Intended uses & limitations
More information needed
Training and evaluation data
More information needed
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 3e-05
- train_batch_size: 16
- eval_batch_size: 8
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 15
- mixed_precision_training: Native AMP
Training results
| Training Loss | Epoch | Step | Validation Loss | Answer | Header | Question | Overall Precision | Overall Recall | Overall F1 | Overall Accuracy |
|---|---|---|---|---|---|---|---|---|---|---|
| 1.7944 | 1.0 | 10 | 1.6233 | {'precision': 0.01929260450160772, 'recall': 0.014833127317676144, 'f1': 0.016771488469601678, 'number': 809} | {'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119} | {'precision': 0.27685325264750377, 'recall': 0.17183098591549295, 'f1': 0.2120509849362688, 'number': 1065} | 0.1520 | 0.0978 | 0.1190 | 0.3505 |
| 1.5001 | 2.0 | 20 | 1.2971 | {'precision': 0.11125, 'recall': 0.1100123609394314, 'f1': 0.11062771908017402, 'number': 809} | {'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119} | {'precision': 0.4044059795436664, 'recall': 0.48262910798122066, 'f1': 0.4400684931506849, 'number': 1065} | 0.2912 | 0.3026 | 0.2968 | 0.5348 |
| 1.136 | 3.0 | 30 | 0.9852 | {'precision': 0.4911699779249448, 'recall': 0.5500618046971569, 'f1': 0.5189504373177842, 'number': 809} | {'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119} | {'precision': 0.6086587436332768, 'recall': 0.6732394366197183, 'f1': 0.6393223361569326, 'number': 1065} | 0.5562 | 0.5830 | 0.5693 | 0.6941 |
| 0.8567 | 4.0 | 40 | 0.8143 | {'precision': 0.627744510978044, 'recall': 0.7775030902348579, 'f1': 0.6946438431805633, 'number': 809} | {'precision': 0.06666666666666667, 'recall': 0.01680672268907563, 'f1': 0.026845637583892617, 'number': 119} | {'precision': 0.6987179487179487, 'recall': 0.7164319248826291, 'f1': 0.7074640704682429, 'number': 1065} | 0.6563 | 0.6994 | 0.6772 | 0.7467 |
| 0.6998 | 5.0 | 50 | 0.7133 | {'precision': 0.6534859521331946, 'recall': 0.7762669962917181, 'f1': 0.7096045197740113, 'number': 809} | {'precision': 0.2, 'recall': 0.11764705882352941, 'f1': 0.14814814814814817, 'number': 119} | {'precision': 0.7243532560214094, 'recall': 0.7624413145539906, 'f1': 0.7429094236047575, 'number': 1065} | 0.6757 | 0.7296 | 0.7016 | 0.7781 |
| 0.5886 | 6.0 | 60 | 0.6775 | {'precision': 0.648406374501992, 'recall': 0.8046971569839307, 'f1': 0.7181467181467182, 'number': 809} | {'precision': 0.25806451612903225, 'recall': 0.13445378151260504, 'f1': 0.17679558011049723, 'number': 119} | {'precision': 0.712947189097104, 'recall': 0.7859154929577464, 'f1': 0.7476552032157214, 'number': 1065} | 0.6714 | 0.7546 | 0.7106 | 0.7890 |
| 0.5185 | 7.0 | 70 | 0.6770 | {'precision': 0.6755888650963597, 'recall': 0.7799752781211372, 'f1': 0.7240390131956398, 'number': 809} | {'precision': 0.2079207920792079, 'recall': 0.17647058823529413, 'f1': 0.19090909090909092, 'number': 119} | {'precision': 0.7341337907375644, 'recall': 0.8037558685446009, 'f1': 0.7673688928731511, 'number': 1065} | 0.6851 | 0.7566 | 0.7191 | 0.7955 |
| 0.4672 | 8.0 | 80 | 0.6729 | {'precision': 0.683083511777302, 'recall': 0.788627935723115, 'f1': 0.7320711417096959, 'number': 809} | {'precision': 0.23300970873786409, 'recall': 0.20168067226890757, 'f1': 0.21621621621621623, 'number': 119} | {'precision': 0.747431506849315, 'recall': 0.819718309859155, 'f1': 0.7819077474249888, 'number': 1065} | 0.6961 | 0.7702 | 0.7313 | 0.8007 |
| 0.4188 | 9.0 | 90 | 0.6664 | {'precision': 0.6888888888888889, 'recall': 0.8046971569839307, 'f1': 0.74230330672748, 'number': 809} | {'precision': 0.2727272727272727, 'recall': 0.25210084033613445, 'f1': 0.26200873362445415, 'number': 119} | {'precision': 0.7708703374777975, 'recall': 0.8150234741784037, 'f1': 0.792332268370607, 'number': 1065} | 0.7102 | 0.7772 | 0.7422 | 0.8045 |
| 0.3724 | 10.0 | 100 | 0.6845 | {'precision': 0.6928721174004193, 'recall': 0.8170580964153276, 'f1': 0.7498581962563812, 'number': 809} | {'precision': 0.33, 'recall': 0.2773109243697479, 'f1': 0.30136986301369867, 'number': 119} | {'precision': 0.7818343722172751, 'recall': 0.8244131455399061, 'f1': 0.8025594149908593, 'number': 1065} | 0.7221 | 0.7888 | 0.7540 | 0.8047 |
| 0.3402 | 11.0 | 110 | 0.6830 | {'precision': 0.7118093174431203, 'recall': 0.8121137206427689, 'f1': 0.7586605080831409, 'number': 809} | {'precision': 0.3090909090909091, 'recall': 0.2857142857142857, 'f1': 0.296943231441048, 'number': 119} | {'precision': 0.787422497785651, 'recall': 0.8347417840375587, 'f1': 0.8103919781221514, 'number': 1065} | 0.7308 | 0.7928 | 0.7605 | 0.8129 |
| 0.3219 | 12.0 | 120 | 0.6944 | {'precision': 0.7179203539823009, 'recall': 0.8022249690976514, 'f1': 0.7577349678925861, 'number': 809} | {'precision': 0.3220338983050847, 'recall': 0.31932773109243695, 'f1': 0.32067510548523204, 'number': 119} | {'precision': 0.781882145998241, 'recall': 0.8347417840375587, 'f1': 0.8074477747502271, 'number': 1065} | 0.7300 | 0.7908 | 0.7592 | 0.8097 |
| 0.3004 | 13.0 | 130 | 0.6978 | {'precision': 0.7147540983606557, 'recall': 0.8084054388133498, 'f1': 0.7587006960556845, 'number': 809} | {'precision': 0.33043478260869563, 'recall': 0.31932773109243695, 'f1': 0.32478632478632474, 'number': 119} | {'precision': 0.7890974084003575, 'recall': 0.8291079812206573, 'f1': 0.8086080586080587, 'number': 1065} | 0.7329 | 0.7903 | 0.7605 | 0.8144 |
| 0.2942 | 14.0 | 140 | 0.7001 | {'precision': 0.7145945945945946, 'recall': 0.8170580964153276, 'f1': 0.7623990772779701, 'number': 809} | {'precision': 0.30708661417322836, 'recall': 0.3277310924369748, 'f1': 0.3170731707317073, 'number': 119} | {'precision': 0.7820284697508897, 'recall': 0.8253521126760563, 'f1': 0.8031064412973961, 'number': 1065} | 0.7256 | 0.7923 | 0.7575 | 0.8108 |
| 0.2853 | 15.0 | 150 | 0.7024 | {'precision': 0.7113513513513513, 'recall': 0.8133498145859085, 'f1': 0.7589388696655133, 'number': 809} | {'precision': 0.30952380952380953, 'recall': 0.3277310924369748, 'f1': 0.31836734693877555, 'number': 119} | {'precision': 0.7811387900355872, 'recall': 0.8244131455399061, 'f1': 0.8021927820922796, 'number': 1065} | 0.7241 | 0.7903 | 0.7558 | 0.8106 |
Framework versions
- Transformers 4.25.1
- Pytorch 1.13.1+cu117
- Datasets 2.7.1
- Tokenizers 0.13.2