End of training

README.md

@@ -0,0 +1,79 @@
+---
+base_model: microsoft/layoutlm-base-uncased
+tags:
+- generated_from_trainer
+datasets:
+- funsd
+model-index:
+- name: layoutlm-funsd
+  results: []
+---
+
+<!-- This model card has been generated automatically according to the information the Trainer had access to. You
+should probably proofread and complete it, then remove this comment. -->
+
+# layoutlm-funsd
+
+This model is a fine-tuned version of [microsoft/layoutlm-base-uncased](https://huggingface.co/microsoft/layoutlm-base-uncased) on the funsd dataset.
+It achieves the following results on the evaluation set:
+- Loss: 0.6726
+- Answer: {'precision': 0.71960569550931, 'recall': 0.8121137206427689, 'f1': 0.7630662020905924, 'number': 809}
+- Header: {'precision': 0.3442622950819672, 'recall': 0.35294117647058826, 'f1': 0.3485477178423237, 'number': 119}
+- Question: {'precision': 0.773936170212766, 'recall': 0.819718309859155, 'f1': 0.796169630642955, 'number': 1065}
+- Overall Precision: 0.7268
+- Overall Recall: 0.7888
+- Overall F1: 0.7565
+- Overall Accuracy: 0.8038
+
+## Model description
+
+More information needed
+
+## Intended uses & limitations
+
+More information needed
+
+## Training and evaluation data
+
+More information needed
+
+## Training procedure
+
+### Training hyperparameters
+
+The following hyperparameters were used during training:
+- learning_rate: 3e-05
+- train_batch_size: 16
+- eval_batch_size: 8
+- seed: 42
+- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
+- lr_scheduler_type: linear
+- num_epochs: 15
+
+### Training results
+
+| Training Loss | Epoch | Step | Validation Loss | Answer                                                                                                      | Header                                                                                                       | Question                                                                                                    | Overall Precision | Overall Recall | Overall F1 | Overall Accuracy |
+|:-------------:|:-----:|:----:|:---------------:|:-----------------------------------------------------------------------------------------------------------:|:------------------------------------------------------------------------------------------------------------:|:-----------------------------------------------------------------------------------------------------------:|:-----------------:|:--------------:|:----------:|:----------------:|
+| 1.7892        | 1.0   | 10   | 1.5673          | {'precision': 0.016726403823178016, 'recall': 0.0173053152039555, 'f1': 0.01701093560145808, 'number': 809} | {'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119}                                                  | {'precision': 0.2742155525238745, 'recall': 0.18873239436619718, 'f1': 0.22358175750834258, 'number': 1065} | 0.1369            | 0.1079         | 0.1207     | 0.3817           |
+| 1.4288        | 2.0   | 20   | 1.2189          | {'precision': 0.21368421052631578, 'recall': 0.25092707045735474, 'f1': 0.23081296191017622, 'number': 809} | {'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119}                                                  | {'precision': 0.40669856459330145, 'recall': 0.6384976525821596, 'f1': 0.49689440993788825, 'number': 1065} | 0.3368            | 0.4431         | 0.3827     | 0.6054           |
+| 1.0674        | 3.0   | 30   | 0.9170          | {'precision': 0.48810754912099275, 'recall': 0.5834363411619283, 'f1': 0.5315315315315315, 'number': 809}   | {'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119}                                                  | {'precision': 0.5930232558139535, 'recall': 0.7183098591549296, 'f1': 0.6496815286624205, 'number': 1065}   | 0.5447            | 0.6207         | 0.5802     | 0.7133           |
+| 0.8227        | 4.0   | 40   | 0.7915          | {'precision': 0.5774518790100825, 'recall': 0.7787391841779975, 'f1': 0.6631578947368422, 'number': 809}    | {'precision': 0.08695652173913043, 'recall': 0.03361344537815126, 'f1': 0.048484848484848485, 'number': 119} | {'precision': 0.6742616033755274, 'recall': 0.7502347417840376, 'f1': 0.7102222222222223, 'number': 1065}   | 0.6171            | 0.7190         | 0.6642     | 0.7554           |
+| 0.6799        | 5.0   | 50   | 0.7317          | {'precision': 0.6394485683987274, 'recall': 0.7453646477132262, 'f1': 0.6883561643835616, 'number': 809}    | {'precision': 0.13636363636363635, 'recall': 0.07563025210084033, 'f1': 0.09729729729729729, 'number': 119}  | {'precision': 0.7052810902896082, 'recall': 0.7774647887323943, 'f1': 0.7396158999553373, 'number': 1065}   | 0.6596            | 0.7225         | 0.6897     | 0.7768           |
+| 0.5807        | 6.0   | 60   | 0.6756          | {'precision': 0.6624338624338625, 'recall': 0.7737948084054388, 'f1': 0.7137970353477766, 'number': 809}    | {'precision': 0.1744186046511628, 'recall': 0.12605042016806722, 'f1': 0.14634146341463414, 'number': 119}   | {'precision': 0.7015748031496063, 'recall': 0.8366197183098592, 'f1': 0.7631691648822269, 'number': 1065}   | 0.6658            | 0.7687         | 0.7136     | 0.7978           |
+| 0.5033        | 7.0   | 70   | 0.6534          | {'precision': 0.6901408450704225, 'recall': 0.7873918417799752, 'f1': 0.7355658198614318, 'number': 809}    | {'precision': 0.21, 'recall': 0.17647058823529413, 'f1': 0.19178082191780824, 'number': 119}                 | {'precision': 0.7219917012448133, 'recall': 0.8169014084507042, 'f1': 0.7665198237885463, 'number': 1065}   | 0.6858            | 0.7667         | 0.7240     | 0.8036           |
+| 0.4565        | 8.0   | 80   | 0.6414          | {'precision': 0.6953713670613563, 'recall': 0.7985166872682324, 'f1': 0.7433831990794016, 'number': 809}    | {'precision': 0.3055555555555556, 'recall': 0.2773109243697479, 'f1': 0.2907488986784141, 'number': 119}     | {'precision': 0.7297748123436196, 'recall': 0.8215962441314554, 'f1': 0.7729681978798587, 'number': 1065}   | 0.6950            | 0.7797         | 0.7349     | 0.8047           |
+| 0.3992        | 9.0   | 90   | 0.6539          | {'precision': 0.6824742268041237, 'recall': 0.8182941903584673, 'f1': 0.7442383361439011, 'number': 809}    | {'precision': 0.25663716814159293, 'recall': 0.24369747899159663, 'f1': 0.25, 'number': 119}                 | {'precision': 0.7504317789291882, 'recall': 0.815962441314554, 'f1': 0.7818263607737291, 'number': 1065}    | 0.6961            | 0.7827         | 0.7369     | 0.7994           |
+| 0.3623        | 10.0  | 100  | 0.6492          | {'precision': 0.710239651416122, 'recall': 0.8059332509270705, 'f1': 0.755066589461494, 'number': 809}      | {'precision': 0.34710743801652894, 'recall': 0.35294117647058826, 'f1': 0.35000000000000003, 'number': 119}  | {'precision': 0.7538200339558574, 'recall': 0.8338028169014085, 'f1': 0.7917967008470798, 'number': 1065}   | 0.7136            | 0.7938         | 0.7515     | 0.8082           |
+| 0.3282        | 11.0  | 110  | 0.6552          | {'precision': 0.7079261672095548, 'recall': 0.8059332509270705, 'f1': 0.753757225433526, 'number': 809}     | {'precision': 0.35537190082644626, 'recall': 0.36134453781512604, 'f1': 0.3583333333333333, 'number': 119}   | {'precision': 0.7664618086040387, 'recall': 0.819718309859155, 'f1': 0.7921960072595282, 'number': 1065}    | 0.7189            | 0.7868         | 0.7513     | 0.8090           |
+| 0.3131        | 12.0  | 120  | 0.6544          | {'precision': 0.7166123778501629, 'recall': 0.8158220024721878, 'f1': 0.7630057803468208, 'number': 809}    | {'precision': 0.35398230088495575, 'recall': 0.33613445378151263, 'f1': 0.3448275862068966, 'number': 119}   | {'precision': 0.7619461337966985, 'recall': 0.8234741784037559, 'f1': 0.7915162454873647, 'number': 1065}   | 0.7217            | 0.7913         | 0.7549     | 0.8091           |
+| 0.2983        | 13.0  | 130  | 0.6732          | {'precision': 0.721058434399118, 'recall': 0.8084054388133498, 'f1': 0.7622377622377622, 'number': 809}     | {'precision': 0.3629032258064516, 'recall': 0.37815126050420167, 'f1': 0.37037037037037035, 'number': 119}   | {'precision': 0.7795698924731183, 'recall': 0.8169014084507042, 'f1': 0.797799174690509, 'number': 1065}    | 0.7308            | 0.7873         | 0.7580     | 0.8035           |
+| 0.2751        | 14.0  | 140  | 0.6745          | {'precision': 0.7142857142857143, 'recall': 0.8158220024721878, 'f1': 0.7616849394114252, 'number': 809}    | {'precision': 0.3416666666666667, 'recall': 0.3445378151260504, 'f1': 0.34309623430962344, 'number': 119}    | {'precision': 0.7723649247121346, 'recall': 0.8187793427230047, 'f1': 0.7948951686417502, 'number': 1065}   | 0.7239            | 0.7893         | 0.7552     | 0.8041           |
+| 0.2697        | 15.0  | 150  | 0.6726          | {'precision': 0.71960569550931, 'recall': 0.8121137206427689, 'f1': 0.7630662020905924, 'number': 809}      | {'precision': 0.3442622950819672, 'recall': 0.35294117647058826, 'f1': 0.3485477178423237, 'number': 119}    | {'precision': 0.773936170212766, 'recall': 0.819718309859155, 'f1': 0.796169630642955, 'number': 1065}      | 0.7268            | 0.7888         | 0.7565     | 0.8038           |
+
+
+### Framework versions
+
+- Transformers 4.34.0
+- Pytorch 2.0.1+cu118
+- Datasets 2.14.5
+- Tokenizers 0.14.1

added_tokens.json

@@ -0,0 +1,7 @@
+{
+  "[CLS]": 101,
+  "[MASK]": 103,
+  "[PAD]": 0,
+  "[SEP]": 102,
+  "[UNK]": 100
+}

preprocessor_config.json

@@ -0,0 +1,14 @@
+{
+  "apply_ocr": true,
+  "do_resize": true,
+  "feature_extractor_type": "LayoutLMv2FeatureExtractor",
+  "image_processor_type": "LayoutLMv2ImageProcessor",
+  "ocr_lang": null,
+  "processor_class": "LayoutLMv2Processor",
+  "resample": 2,
+  "size": {
+    "height": 224,
+    "width": 224
+  },
+  "tesseract_config": ""
+}

special_tokens_map.json

@@ -0,0 +1,7 @@
+{
+  "cls_token": "[CLS]",
+  "mask_token": "[MASK]",
+  "pad_token": "[PAD]",
+  "sep_token": "[SEP]",
+  "unk_token": "[UNK]"
+}

tokenizer_config.json

@@ -0,0 +1,80 @@
+{
+  "added_tokens_decoder": {
+    "0": {
+      "content": "[PAD]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "100": {
+      "content": "[UNK]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "101": {
+      "content": "[CLS]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "102": {
+      "content": "[SEP]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "103": {
+      "content": "[MASK]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    }
+  },
+  "additional_special_tokens": [],
+  "apply_ocr": false,
+  "clean_up_tokenization_spaces": true,
+  "cls_token": "[CLS]",
+  "cls_token_box": [
+    0,
+    0,
+    0,
+    0
+  ],
+  "do_basic_tokenize": true,
+  "do_lower_case": true,
+  "mask_token": "[MASK]",
+  "model_max_length": 512,
+  "never_split": null,
+  "only_label_first_subword": true,
+  "pad_token": "[PAD]",
+  "pad_token_box": [
+    0,
+    0,
+    0,
+    0
+  ],
+  "pad_token_label": -100,
+  "processor_class": "LayoutLMv2Processor",
+  "sep_token": "[SEP]",
+  "sep_token_box": [
+    1000,
+    1000,
+    1000,
+    1000
+  ],
+  "strip_accents": null,
+  "tokenize_chinese_chars": true,
+  "tokenizer_class": "LayoutLMv2Tokenizer",
+  "unk_token": "[UNK]"
+}