Model save

Browse files

Files changed (5) hide show

README.md +124 -0
all_results.json +74 -0
eval_results.json +68 -0
train_results.json +9 -0
trainer_state.json +656 -0

README.md ADDED Viewed

	@@ -0,0 +1,124 @@

+---
+license: mit
+base_model: microsoft/deberta-v3-base
+tags:
+- generated_from_trainer
+model-index:
+- name: deberta-v3-base_finetuned_ai4privacy_v2
+  results: []
+---
+<!-- This model card has been generated automatically according to the information the Trainer had access to. You
+should probably proofread and complete it, then remove this comment. -->
+# deberta-v3-base_finetuned_ai4privacy_v2
+This model is a fine-tuned version of [microsoft/deberta-v3-base](https://huggingface.co/microsoft/deberta-v3-base) on the None dataset.
+It achieves the following results on the evaluation set:
+- Loss: 0.0424
+- Overall Precision: 0.9550
+- Overall Recall: 0.9680
+- Overall F1: 0.9614
+- Overall Accuracy: 0.9818
+- Accountname F1: 0.9965
+- Accountnumber F1: 1.0
+- Age F1: 0.9676
+- Amount F1: 0.9897
+- Bic F1: 0.9868
+- Bitcoinaddress F1: 0.9926
+- Buildingnumber F1: 0.9818
+- City F1: 0.9968
+- Companyname F1: 0.9979
+- County F1: 1.0
+- Creditcardcvv F1: 1.0
+- Creditcardissuer F1: 0.9964
+- Creditcardnumber F1: 0.9631
+- Currency F1: 0.8896
+- Currencycode F1: 0.9949
+- Currencyname F1: 0.7481
+- Currencysymbol F1: 0.9984
+- Date F1: 0.9609
+- Dob F1: 0.9160
+- Email F1: 1.0
+- Ethereumaddress F1: 1.0
+- Eyecolor F1: 0.9608
+- Firstname F1: 0.9908
+- Gender F1: 0.9988
+- Height F1: 0.9925
+- Iban F1: 0.9974
+- Ip F1: 0.2177
+- Ipv4 F1: 0.8145
+- Ipv6 F1: 0.7835
+- Jobarea F1: 0.9831
+- Jobtitle F1: 0.9992
+- Jobtype F1: 0.9899
+- Lastname F1: 0.9643
+- Litecoinaddress F1: 0.9815
+- Mac F1: 1.0
+- Maskednumber F1: 0.9590
+- Middlename F1: 0.4375
+- Nearbygpscoordinate F1: 1.0
+- Ordinaldirection F1: 0.4
+- Password F1: 0.9960
+- Phoneimei F1: 0.9961
+- Phonenumber F1: 0.9978
+- Pin F1: 0.9814
+- Prefix F1: 0.9807
+- Secondaryaddress F1: 0.9989
+- Sex F1: 0.9783
+- Ssn F1: 0.9950
+- State F1: 0.9923
+- Street F1: 0.9968
+- Time F1: 0.9862
+- Url F1: 1.0
+- Useragent F1: 0.9975
+- Username F1: 1.0
+- Vehiclevin F1: 1.0
+- Vehiclevrm F1: 0.9929
+- Zipcode F1: 0.9943
+## Model description
+More information needed
+## Intended uses & limitations
+More information needed
+## Training and evaluation data
+More information needed
+## Training procedure
+### Training hyperparameters
+The following hyperparameters were used during training:
+- learning_rate: 5e-05
+- train_batch_size: 8
+- eval_batch_size: 8
+- seed: 42
+- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
+- lr_scheduler_type: cosine_with_restarts
+- lr_scheduler_warmup_ratio: 0.2
+- num_epochs: 7
+### Training results
+| Training Loss | Epoch | Step | Validation Loss | Overall Precision | Overall Recall | Overall F1 | Overall Accuracy | Accountname F1 | Accountnumber F1 | Age F1 | Amount F1 | Bic F1 | Bitcoinaddress F1 | Buildingnumber F1 | City F1 | Companyname F1 | County F1 | Creditcardcvv F1 | Creditcardissuer F1 | Creditcardnumber F1 | Currency F1 | Currencycode F1 | Currencyname F1 | Currencysymbol F1 | Date F1 | Dob F1 | Email F1 | Ethereumaddress F1 | Eyecolor F1 | Firstname F1 | Gender F1 | Height F1 | Iban F1 | Ip F1  | Ipv4 F1 | Ipv6 F1 | Jobarea F1 | Jobtitle F1 | Jobtype F1 | Lastname F1 | Litecoinaddress F1 | Mac F1 | Maskednumber F1 | Middlename F1 | Nearbygpscoordinate F1 | Ordinaldirection F1 | Password F1 | Phoneimei F1 | Phonenumber F1 | Pin F1 | Prefix F1 | Secondaryaddress F1 | Sex F1 | Ssn F1 | State F1 | Street F1 | Time F1 | Url F1 | Useragent F1 | Username F1 | Vehiclevin F1 | Vehiclevrm F1 | Zipcode F1 |
+|:-------------:|:-----:|:----:|:---------------:|:-----------------:|:--------------:|:----------:|:----------------:|:--------------:|:----------------:|:------:|:---------:|:------:|:-----------------:|:-----------------:|:-------:|:--------------:|:---------:|:----------------:|:-------------------:|:-------------------:|:-----------:|:---------------:|:---------------:|:-----------------:|:-------:|:------:|:--------:|:------------------:|:-----------:|:------------:|:---------:|:---------:|:-------:|:------:|:-------:|:-------:|:----------:|:-----------:|:----------:|:-----------:|:------------------:|:------:|:---------------:|:-------------:|:----------------------:|:-------------------:|:-----------:|:------------:|:--------------:|:------:|:---------:|:-------------------:|:------:|:------:|:--------:|:---------:|:-------:|:------:|:------------:|:-----------:|:-------------:|:-------------:|:----------:|
+| 0.2542        | 1.0   | 1070 | 0.1744          | 0.7933            | 0.8151         | 0.8040     | 0.9469           | 0.9734         | 0.9011           | 0.8465 | 0.8807    | 0.5502 | 0.7683            | 0.8006            | 0.6202  | 0.8908         | 0.8431    | 0.0              | 0.8100              | 0.6975              | 0.5052      | 0.0             | 0.0             | 0.7156            | 0.8292  | 0.3902 | 0.9907   | 0.9810             | 0.4808      | 0.8015       | 0.8076    | 0.9744    | 0.9529  | 0.0    | 0.8335  | 0.7879  | 0.0        | 0.9365      | 0.0        | 0.0         | 0.0                | 0.9355 | 0.0890          | 0.0           | 1.0                    | 0.0                 | 0.9670      | 0.9882       | 0.9706         | 0.7341 | 0.8914    | 0.9112              | 0.0851 | 0.9072 | 0.6932   | 0.7065    | 0.9543  | 0.9939 | 0.9667       | 0.8542      | 0.9642        | 0.0184        | 0.7480     |
+| 0.1494        | 2.0   | 2140 | 0.1213          | 0.8470            | 0.8747         | 0.8606     | 0.9590           | 0.9832         | 0.8908           | 0.9126 | 0.5531    | 0.9419 | 0.7910            | 0.7723            | 0.8919  | 0.9175         | 0.9560    | 0.0              | 0.9858              | 0.8649              | 0.6121      | 0.8738          | 0.0             | 0.9287            | 0.8676  | 0.5886 | 0.9882   | 0.9873             | 0.8319      | 0.9381       | 0.8815    | 0.9744    | 0.9580  | 0.0    | 0.8388  | 0.8040  | 0.0        | 0.9355      | 0.6122     | 0.5918      | 0.1882             | 0.9908 | 0.7311          | 0.0           | 0.9967                 | 0.0                 | 0.9440      | 0.9738       | 0.9257         | 0.7566 | 0.9015    | 0.9467              | 0.8776 | 0.9010 | 0.6753   | 0.9297    | 0.9021  | 0.9836 | 0.9637       | 0.9585      | 0.9451        | 0.9417        | 0.6969     |
+| 0.1008        | 3.0   | 3210 | 0.0728          | 0.9038            | 0.9222         | 0.9129     | 0.9708           | 0.9773         | 0.9091           | 0.9144 | 0.8619    | 0.9610 | 0.9546            | 0.9023            | 0.9711  | 0.9906         | 0.8943    | 0.3889           | 0.9680              | 0.8853              | 0.7540      | 0.9239          | 0.0633          | 0.9665            | 0.8906  | 0.6993 | 0.9987   | 0.9905             | 0.9804      | 0.9640       | 0.9763    | 0.7631    | 0.9724  | 0.0    | 0.8374  | 0.8081  | 0.6087     | 0.9882      | 0.8649     | 0.5698      | 0.8708             | 0.9931 | 0.8410          | 0.0           | 1.0                    | 0.0                 | 0.9858      | 0.9921       | 0.9903         | 0.8889 | 0.9173    | 0.9897              | 0.9263 | 0.9042 | 0.8811   | 0.9503    | 0.9836  | 0.9965 | 0.9692       | 0.9847      | 0.9584        | 0.9763        | 0.9171     |
+| 0.0793        | 4.0   | 4280 | 0.0675          | 0.9356            | 0.9446         | 0.9401     | 0.9742           | 0.9894         | 0.9810           | 0.9608 | 0.9825    | 0.9539 | 0.9633            | 0.9435            | 0.9881  | 0.9793         | 0.9894    | 0.8462           | 0.9893              | 0.9303              | 0.8391      | 0.9694          | 0.4124          | 0.9922            | 0.8978  | 0.7260 | 0.9975   | 0.9968             | 0.9804      | 0.9757       | 0.9834    | 0.9925    | 0.9949  | 0.0    | 0.8325  | 0.8106  | 0.8333     | 0.9899      | 0.9495     | 0.7368      | 0.9086             | 1.0    | 0.9003          | 0.0           | 1.0                    | 0.0                 | 0.9919      | 0.9934       | 0.9913         | 0.9556 | 0.9338    | 0.9943              | 0.9474 | 0.9888 | 0.9788   | 0.9915    | 0.9699  | 0.9948 | 0.9778       | 0.9898      | 0.9752        | 0.9835        | 0.9559     |
+| 0.0656        | 5.0   | 5350 | 0.0469          | 0.9524            | 0.9588         | 0.9556     | 0.9780           | 0.9965         | 0.9965           | 0.9626 | 0.9866    | 0.9770 | 0.9789            | 0.9657            | 0.9935  | 0.9958         | 0.9970    | 0.9655           | 0.9964              | 0.9432              | 0.8342      | 0.9744          | 0.6977          | 0.9938            | 0.9382  | 0.8596 | 1.0      | 1.0                | 0.9608      | 0.9822       | 0.9941    | 0.9925    | 0.9949  | 0.1360 | 0.8390  | 0.8003  | 0.9333     | 0.9992      | 0.98       | 0.9157      | 0.9570             | 1.0    | 0.9361          | 0.0           | 1.0                    | 0.4                 | 0.9828      | 0.9921       | 0.9978         | 0.9814 | 0.9652    | 0.9989              | 0.9574 | 0.9938 | 0.9923   | 0.9957    | 0.9766  | 0.9991 | 0.9975       | 0.9983      | 0.9944        | 0.9976        | 0.9765     |
+| 0.057         | 6.0   | 6420 | 0.0448          | 0.9595            | 0.9662         | 0.9628     | 0.9803           | 0.9965         | 1.0              | 0.9582 | 0.9897    | 0.9836 | 0.9851            | 0.9833            | 0.9978  | 0.9979         | 0.9985    | 1.0              | 0.9964              | 0.9581              | 0.8956      | 0.9949          | 0.7294          | 0.9984            | 0.9524  | 0.8848 | 1.0      | 1.0                | 0.9608      | 0.9886       | 0.9988    | 0.9925    | 0.9974  | 0.1663 | 0.8377  | 0.8195  | 0.9123     | 0.9992      | 0.9608     | 0.9556      | 0.9563             | 1.0    | 0.9527          | 0.2857        | 1.0                    | 0.4                 | 0.9960      | 0.9961       | 0.9978         | 0.9852 | 0.9798    | 0.9989              | 0.9783 | 0.9975 | 0.9942   | 0.9979    | 0.9807  | 1.0    | 0.9975       | 1.0         | 0.9808        | 0.9929        | 0.9932     |
+| 0.0455        | 7.0   | 7490 | 0.0424          | 0.9550            | 0.9680         | 0.9614     | 0.9818           | 0.9965         | 1.0              | 0.9676 | 0.9897    | 0.9868 | 0.9926            | 0.9818            | 0.9968  | 0.9979         | 1.0       | 1.0              | 0.9964              | 0.9631              | 0.8896      | 0.9949          | 0.7481          | 0.9984            | 0.9609  | 0.9160 | 1.0      | 1.0                | 0.9608      | 0.9908       | 0.9988    | 0.9925    | 0.9974  | 0.2177 | 0.8145  | 0.7835  | 0.9831     | 0.9992      | 0.9899     | 0.9643      | 0.9815             | 1.0    | 0.9590          | 0.4375        | 1.0                    | 0.4                 | 0.9960      | 0.9961       | 0.9978         | 0.9814 | 0.9807    | 0.9989              | 0.9783 | 0.9950 | 0.9923   | 0.9968    | 0.9862  | 1.0    | 0.9975       | 1.0         | 1.0           | 0.9929        | 0.9943     |
+### Framework versions
+- Transformers 4.35.2
+- Pytorch 2.1.0+cu118
+- Datasets 2.15.0
+- Tokenizers 0.15.0

all_results.json ADDED Viewed

	@@ -0,0 +1,74 @@

+{
+    "epoch": 7.0,
+    "eval_ACCOUNTNAME_f1": 0.9964539007092198,
+    "eval_ACCOUNTNUMBER_f1": 1.0,
+    "eval_AGE_f1": 0.9676375404530744,
+    "eval_AMOUNT_f1": 0.9897330595482546,
+    "eval_BIC_f1": 0.9868421052631579,
+    "eval_BITCOINADDRESS_f1": 0.9925611052072264,
+    "eval_BUILDINGNUMBER_f1": 0.9818181818181818,
+    "eval_CITY_f1": 0.9967567567567569,
+    "eval_COMPANYNAME_f1": 0.997907949790795,
+    "eval_COUNTY_f1": 1.0,
+    "eval_CREDITCARDCVV_f1": 1.0,
+    "eval_CREDITCARDISSUER_f1": 0.9964412811387899,
+    "eval_CREDITCARDNUMBER_f1": 0.963110667996012,
+    "eval_CURRENCYCODE_f1": 0.9948717948717948,
+    "eval_CURRENCYNAME_f1": 0.7481481481481482,
+    "eval_CURRENCYSYMBOL_f1": 0.9984301412872841,
+    "eval_CURRENCY_f1": 0.8896103896103896,
+    "eval_DATE_f1": 0.9608621667612025,
+    "eval_DOB_f1": 0.9159763313609468,
+    "eval_EMAIL_f1": 1.0,
+    "eval_ETHEREUMADDRESS_f1": 1.0,
+    "eval_EYECOLOR_f1": 0.9607843137254902,
+    "eval_FIRSTNAME_f1": 0.9907935116177115,
+    "eval_GENDER_f1": 0.998819362455726,
+    "eval_HEIGHT_f1": 0.9925373134328358,
+    "eval_IBAN_f1": 0.9974424552429667,
+    "eval_IPV4_f1": 0.8145094806265457,
+    "eval_IPV6_f1": 0.7834890965732086,
+    "eval_IP_f1": 0.21772939346811815,
+    "eval_JOBAREA_f1": 0.983050847457627,
+    "eval_JOBTITLE_f1": 0.9991575400168492,
+    "eval_JOBTYPE_f1": 0.98989898989899,
+    "eval_LASTNAME_f1": 0.9642857142857142,
+    "eval_LITECOINADDRESS_f1": 0.9815303430079155,
+    "eval_MAC_f1": 1.0,
+    "eval_MASKEDNUMBER_f1": 0.958974358974359,
+    "eval_MIDDLENAME_f1": 0.43750000000000006,
+    "eval_NEARBYGPSCOORDINATE_f1": 1.0,
+    "eval_ORDINALDIRECTION_f1": 0.4,
+    "eval_PASSWORD_f1": 0.9959514170040485,
+    "eval_PHONEIMEI_f1": 0.9960578186596584,
+    "eval_PHONENUMBER_f1": 0.9978260869565218,
+    "eval_PIN_f1": 0.9814126394052044,
+    "eval_PREFIX_f1": 0.980688497061293,
+    "eval_SECONDARYADDRESS_f1": 0.9988545246277205,
+    "eval_SEX_f1": 0.9782608695652174,
+    "eval_SSN_f1": 0.995049504950495,
+    "eval_STATE_f1": 0.992337164750958,
+    "eval_STREET_f1": 0.9968119022316684,
+    "eval_TIME_f1": 0.9861878453038674,
+    "eval_URL_f1": 1.0,
+    "eval_USERAGENT_f1": 0.9975308641975309,
+    "eval_USERNAME_f1": 1.0,
+    "eval_VEHICLEVIN_f1": 1.0,
+    "eval_VEHICLEVRM_f1": 0.9929078014184397,
+    "eval_ZIPCODE_f1": 0.994311717861206,
+    "eval_loss": 0.04238386079668999,
+    "eval_overall_accuracy": 0.9817632402056937,
+    "eval_overall_f1": 0.961428430502121,
+    "eval_overall_precision": 0.9549752584390769,
+    "eval_overall_recall": 0.9679694095446194,
+    "eval_runtime": 51.5335,
+    "eval_samples": 8557,
+    "eval_samples_per_second": 166.047,
+    "eval_steps_per_second": 20.763,
+    "total_flos": 2378184631534176.0,
+    "train_loss": 0.19348664430177737,
+    "train_runtime": 1425.9685,
+    "train_samples": 34228,
+    "train_samples_per_second": 42.006,
+    "train_steps_per_second": 5.253
+}

eval_results.json ADDED Viewed

	@@ -0,0 +1,68 @@

+{
+    "epoch": 7.0,
+    "eval_ACCOUNTNAME_f1": 0.9964539007092198,
+    "eval_ACCOUNTNUMBER_f1": 1.0,
+    "eval_AGE_f1": 0.9676375404530744,
+    "eval_AMOUNT_f1": 0.9897330595482546,
+    "eval_BIC_f1": 0.9868421052631579,
+    "eval_BITCOINADDRESS_f1": 0.9925611052072264,
+    "eval_BUILDINGNUMBER_f1": 0.9818181818181818,
+    "eval_CITY_f1": 0.9967567567567569,
+    "eval_COMPANYNAME_f1": 0.997907949790795,
+    "eval_COUNTY_f1": 1.0,
+    "eval_CREDITCARDCVV_f1": 1.0,
+    "eval_CREDITCARDISSUER_f1": 0.9964412811387899,
+    "eval_CREDITCARDNUMBER_f1": 0.963110667996012,
+    "eval_CURRENCYCODE_f1": 0.9948717948717948,
+    "eval_CURRENCYNAME_f1": 0.7481481481481482,
+    "eval_CURRENCYSYMBOL_f1": 0.9984301412872841,
+    "eval_CURRENCY_f1": 0.8896103896103896,
+    "eval_DATE_f1": 0.9608621667612025,
+    "eval_DOB_f1": 0.9159763313609468,
+    "eval_EMAIL_f1": 1.0,
+    "eval_ETHEREUMADDRESS_f1": 1.0,
+    "eval_EYECOLOR_f1": 0.9607843137254902,
+    "eval_FIRSTNAME_f1": 0.9907935116177115,
+    "eval_GENDER_f1": 0.998819362455726,
+    "eval_HEIGHT_f1": 0.9925373134328358,
+    "eval_IBAN_f1": 0.9974424552429667,
+    "eval_IPV4_f1": 0.8145094806265457,
+    "eval_IPV6_f1": 0.7834890965732086,
+    "eval_IP_f1": 0.21772939346811815,
+    "eval_JOBAREA_f1": 0.983050847457627,
+    "eval_JOBTITLE_f1": 0.9991575400168492,
+    "eval_JOBTYPE_f1": 0.98989898989899,
+    "eval_LASTNAME_f1": 0.9642857142857142,
+    "eval_LITECOINADDRESS_f1": 0.9815303430079155,
+    "eval_MAC_f1": 1.0,
+    "eval_MASKEDNUMBER_f1": 0.958974358974359,
+    "eval_MIDDLENAME_f1": 0.43750000000000006,
+    "eval_NEARBYGPSCOORDINATE_f1": 1.0,
+    "eval_ORDINALDIRECTION_f1": 0.4,
+    "eval_PASSWORD_f1": 0.9959514170040485,
+    "eval_PHONEIMEI_f1": 0.9960578186596584,
+    "eval_PHONENUMBER_f1": 0.9978260869565218,
+    "eval_PIN_f1": 0.9814126394052044,
+    "eval_PREFIX_f1": 0.980688497061293,
+    "eval_SECONDARYADDRESS_f1": 0.9988545246277205,
+    "eval_SEX_f1": 0.9782608695652174,
+    "eval_SSN_f1": 0.995049504950495,
+    "eval_STATE_f1": 0.992337164750958,
+    "eval_STREET_f1": 0.9968119022316684,
+    "eval_TIME_f1": 0.9861878453038674,
+    "eval_URL_f1": 1.0,
+    "eval_USERAGENT_f1": 0.9975308641975309,
+    "eval_USERNAME_f1": 1.0,
+    "eval_VEHICLEVIN_f1": 1.0,
+    "eval_VEHICLEVRM_f1": 0.9929078014184397,
+    "eval_ZIPCODE_f1": 0.994311717861206,
+    "eval_loss": 0.04238386079668999,
+    "eval_overall_accuracy": 0.9817632402056937,
+    "eval_overall_f1": 0.961428430502121,
+    "eval_overall_precision": 0.9549752584390769,
+    "eval_overall_recall": 0.9679694095446194,
+    "eval_runtime": 51.5335,
+    "eval_samples": 8557,
+    "eval_samples_per_second": 166.047,
+    "eval_steps_per_second": 20.763
+}

train_results.json ADDED Viewed

	@@ -0,0 +1,9 @@

+{
+    "epoch": 7.0,
+    "total_flos": 2378184631534176.0,
+    "train_loss": 0.19348664430177737,
+    "train_runtime": 1425.9685,
+    "train_samples": 34228,
+    "train_samples_per_second": 42.006,
+    "train_steps_per_second": 5.253
+}

trainer_state.json ADDED Viewed

	@@ -0,0 +1,656 @@

+{
+  "best_metric": 0.04238386079668999,
+  "best_model_checkpoint": "deberta-v3-base_finetuned_ai4privacy_v2/checkpoint-7490",
+  "epoch": 7.0,
+  "eval_steps": 500,
+  "global_step": 7490,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "epoch": 0.47,
+      "learning_rate": 1.6688918558077437e-05,
+      "loss": 1.5449,
+      "step": 500
+    },
+    {
+      "epoch": 0.93,
+      "learning_rate": 3.3377837116154874e-05,
+      "loss": 0.2542,
+      "step": 1000
+    },
+    {
+      "epoch": 1.0,
+      "eval_ACCOUNTNAME_f1": 0.973404255319149,
+      "eval_ACCOUNTNUMBER_f1": 0.9011177987962167,
+      "eval_AGE_f1": 0.8465116279069768,
+      "eval_AMOUNT_f1": 0.8807339449541284,
+      "eval_BIC_f1": 0.550185873605948,
+      "eval_BITCOINADDRESS_f1": 0.7682819383259912,
+      "eval_BUILDINGNUMBER_f1": 0.8006279434850864,
+      "eval_CITY_f1": 0.6201743462017434,
+      "eval_COMPANYNAME_f1": 0.8907563025210083,
+      "eval_COUNTY_f1": 0.8430769230769231,
+      "eval_CREDITCARDCVV_f1": 0.0,
+      "eval_CREDITCARDISSUER_f1": 0.8100358422939068,
+      "eval_CREDITCARDNUMBER_f1": 0.697503671071953,
+      "eval_CURRENCYCODE_f1": 0.0,
+      "eval_CURRENCYNAME_f1": 0.0,
+      "eval_CURRENCYSYMBOL_f1": 0.7155635062611806,
+      "eval_CURRENCY_f1": 0.5051546391752578,
+      "eval_DATE_f1": 0.8291727851199217,
+      "eval_DOB_f1": 0.3902439024390244,
+      "eval_EMAIL_f1": 0.9906658369632855,
+      "eval_ETHEREUMADDRESS_f1": 0.9810126582278481,
+      "eval_EYECOLOR_f1": 0.4807692307692308,
+      "eval_FIRSTNAME_f1": 0.801470588235294,
+      "eval_GENDER_f1": 0.807556080283353,
+      "eval_HEIGHT_f1": 0.9743589743589743,
+      "eval_IBAN_f1": 0.9528662420382167,
+      "eval_IPV4_f1": 0.8334687246141349,
+      "eval_IPV6_f1": 0.7878787878787878,
+      "eval_IP_f1": 0.0,
+      "eval_JOBAREA_f1": 0.0,
+      "eval_JOBTITLE_f1": 0.9364548494983278,
+      "eval_JOBTYPE_f1": 0.0,
+      "eval_LASTNAME_f1": 0.0,
+      "eval_LITECOINADDRESS_f1": 0.0,
+      "eval_MAC_f1": 0.9354838709677419,
+      "eval_MASKEDNUMBER_f1": 0.08899297423887587,
+      "eval_MIDDLENAME_f1": 0.0,
+      "eval_NEARBYGPSCOORDINATE_f1": 1.0,
+      "eval_ORDINALDIRECTION_f1": 0.0,
+      "eval_PASSWORD_f1": 0.966966966966967,
+      "eval_PHONEIMEI_f1": 0.9882044560943644,
+      "eval_PHONENUMBER_f1": 0.9706202393906419,
+      "eval_PIN_f1": 0.7340823970037453,
+      "eval_PREFIX_f1": 0.8913560666137984,
+      "eval_SECONDARYADDRESS_f1": 0.9111617312072893,
+      "eval_SEX_f1": 0.0851063829787234,
+      "eval_SSN_f1": 0.9072164948453607,
+      "eval_STATE_f1": 0.6931818181818182,
+      "eval_STREET_f1": 0.7064935064935065,
+      "eval_TIME_f1": 0.9543010752688172,
+      "eval_URL_f1": 0.993939393939394,
+      "eval_USERAGENT_f1": 0.966707768187423,
+      "eval_USERNAME_f1": 0.8541846419327007,
+      "eval_VEHICLEVIN_f1": 0.9641873278236913,
+      "eval_VEHICLEVRM_f1": 0.018433179723502304,
+      "eval_ZIPCODE_f1": 0.7480403135498319,
+      "eval_loss": 0.1744486540555954,
+      "eval_overall_accuracy": 0.9468748343317606,
+      "eval_overall_f1": 0.8040267470056581,
+      "eval_overall_precision": 0.7932817786370228,
+      "eval_overall_recall": 0.8150667924715698,
+      "eval_runtime": 60.657,
+      "eval_samples_per_second": 141.072,
+      "eval_steps_per_second": 17.64,
+      "step": 1070
+    },
+    {
+      "epoch": 1.4,
+      "learning_rate": 4.999998625559004e-05,
+      "loss": 0.1683,
+      "step": 1500
+    },
+    {
+      "epoch": 1.87,
+      "learning_rate": 4.913907550717881e-05,
+      "loss": 0.1494,
+      "step": 2000
+    },
+    {
+      "epoch": 2.0,
+      "eval_ACCOUNTNAME_f1": 0.9831709477413639,
+      "eval_ACCOUNTNUMBER_f1": 0.8908227848101267,
+      "eval_AGE_f1": 0.9126213592233009,
+      "eval_AMOUNT_f1": 0.5531019978969506,
+      "eval_BIC_f1": 0.9419354838709677,
+      "eval_BITCOINADDRESS_f1": 0.7910189982728842,
+      "eval_BUILDINGNUMBER_f1": 0.7723132969034608,
+      "eval_CITY_f1": 0.8918617614269788,
+      "eval_COMPANYNAME_f1": 0.9174503657262278,
+      "eval_COUNTY_f1": 0.9559939301972686,
+      "eval_CREDITCARDCVV_f1": 0.0,
+      "eval_CREDITCARDISSUER_f1": 0.9858156028368794,
+      "eval_CREDITCARDNUMBER_f1": 0.8648648648648647,
+      "eval_CURRENCYCODE_f1": 0.8737864077669903,
+      "eval_CURRENCYNAME_f1": 0.0,
+      "eval_CURRENCYSYMBOL_f1": 0.9286798179059179,
+      "eval_CURRENCY_f1": 0.6120689655172413,
+      "eval_DATE_f1": 0.8675564681724846,
+      "eval_DOB_f1": 0.5885885885885886,
+      "eval_EMAIL_f1": 0.9881619937694703,
+      "eval_ETHEREUMADDRESS_f1": 0.9873015873015873,
+      "eval_EYECOLOR_f1": 0.8318584070796459,
+      "eval_FIRSTNAME_f1": 0.9380606578385307,
+      "eval_GENDER_f1": 0.8815060908084165,
+      "eval_HEIGHT_f1": 0.9743589743589743,
+      "eval_IBAN_f1": 0.9579617834394905,
+      "eval_IPV4_f1": 0.8387622149837133,
+      "eval_IPV6_f1": 0.8040435458786935,
+      "eval_IP_f1": 0.0,
+      "eval_JOBAREA_f1": 0.0,
+      "eval_JOBTITLE_f1": 0.935483870967742,
+      "eval_JOBTYPE_f1": 0.6122448979591837,
+      "eval_LASTNAME_f1": 0.5918367346938777,
+      "eval_LITECOINADDRESS_f1": 0.18823529411764708,
+      "eval_MAC_f1": 0.9907834101382489,
+      "eval_MASKEDNUMBER_f1": 0.7311258278145696,
+      "eval_MIDDLENAME_f1": 0.0,
+      "eval_NEARBYGPSCOORDINATE_f1": 0.9966996699669967,
+      "eval_ORDINALDIRECTION_f1": 0.0,
+      "eval_PASSWORD_f1": 0.9440488301119023,
+      "eval_PHONEIMEI_f1": 0.9738219895287957,
+      "eval_PHONENUMBER_f1": 0.9257375381485249,
+      "eval_PIN_f1": 0.7565982404692082,
+      "eval_PREFIX_f1": 0.9014740108611327,
+      "eval_SECONDARYADDRESS_f1": 0.9466515323496028,
+      "eval_SEX_f1": 0.8775510204081634,
+      "eval_SSN_f1": 0.9010152284263959,
+      "eval_STATE_f1": 0.6753246753246753,
+      "eval_STREET_f1": 0.9296956977964324,
+      "eval_TIME_f1": 0.9020771513353116,
+      "eval_URL_f1": 0.9835782195332756,
+      "eval_USERAGENT_f1": 0.9636803874092009,
+      "eval_USERNAME_f1": 0.9585097375105841,
+      "eval_VEHICLEVIN_f1": 0.945054945054945,
+      "eval_VEHICLEVRM_f1": 0.9417249417249417,
+      "eval_ZIPCODE_f1": 0.6969353007945516,
+      "eval_loss": 0.12132773548364639,
+      "eval_overall_accuracy": 0.9590415098340667,
+      "eval_overall_f1": 0.8606190906647774,
+      "eval_overall_precision": 0.8470231797633933,
+      "eval_overall_recall": 0.8746585886676268,
+      "eval_runtime": 52.6965,
+      "eval_samples_per_second": 162.383,
+      "eval_steps_per_second": 20.305,
+      "step": 2140
+    },
+    {
+      "epoch": 2.34,
+      "learning_rate": 4.662875629851768e-05,
+      "loss": 0.1178,
+      "step": 2500
+    },
+    {
+      "epoch": 2.8,
+      "learning_rate": 4.264055722602958e-05,
+      "loss": 0.1008,
+      "step": 3000
+    },
+    {
+      "epoch": 3.0,
+      "eval_ACCOUNTNAME_f1": 0.9773123909249563,
+      "eval_ACCOUNTNUMBER_f1": 0.9090909090909091,
+      "eval_AGE_f1": 0.91437802907916,
+      "eval_AMOUNT_f1": 0.8619329388560159,
+      "eval_BIC_f1": 0.9610389610389611,
+      "eval_BITCOINADDRESS_f1": 0.9546391752577319,
+      "eval_BUILDINGNUMBER_f1": 0.9023255813953489,
+      "eval_CITY_f1": 0.9710610932475885,
+      "eval_COMPANYNAME_f1": 0.9905956112852664,
+      "eval_COUNTY_f1": 0.8943089430894309,
+      "eval_CREDITCARDCVV_f1": 0.3888888888888889,
+      "eval_CREDITCARDISSUER_f1": 0.9679715302491102,
+      "eval_CREDITCARDNUMBER_f1": 0.8852772466539198,
+      "eval_CURRENCYCODE_f1": 0.9238578680203046,
+      "eval_CURRENCYNAME_f1": 0.06329113924050632,
+      "eval_CURRENCYSYMBOL_f1": 0.9664634146341463,
+      "eval_CURRENCY_f1": 0.753968253968254,
+      "eval_DATE_f1": 0.890649762282092,
+      "eval_DOB_f1": 0.6993006993006993,
+      "eval_EMAIL_f1": 0.9987484355444305,
+      "eval_ETHEREUMADDRESS_f1": 0.9904761904761905,
+      "eval_EYECOLOR_f1": 0.9803921568627451,
+      "eval_FIRSTNAME_f1": 0.9639913232104121,
+      "eval_GENDER_f1": 0.976303317535545,
+      "eval_HEIGHT_f1": 0.7630522088353413,
+      "eval_IBAN_f1": 0.9723618090452263,
+      "eval_IPV4_f1": 0.8373983739837398,
+      "eval_IPV6_f1": 0.8080808080808081,
+      "eval_IP_f1": 0.0,
+      "eval_JOBAREA_f1": 0.608695652173913,
+      "eval_JOBTITLE_f1": 0.9882154882154882,
+      "eval_JOBTYPE_f1": 0.8648648648648648,
+      "eval_LASTNAME_f1": 0.5698113207547171,
+      "eval_LITECOINADDRESS_f1": 0.8707865168539326,
+      "eval_MAC_f1": 0.9930715935334872,
+      "eval_MASKEDNUMBER_f1": 0.8409703504043126,
+      "eval_MIDDLENAME_f1": 0.0,
+      "eval_NEARBYGPSCOORDINATE_f1": 1.0,
+      "eval_ORDINALDIRECTION_f1": 0.0,
+      "eval_PASSWORD_f1": 0.9858299595141701,
+      "eval_PHONEIMEI_f1": 0.9921052631578947,
+      "eval_PHONENUMBER_f1": 0.9902702702702703,
+      "eval_PIN_f1": 0.888888888888889,
+      "eval_PREFIX_f1": 0.9172576832151301,
+      "eval_SECONDARYADDRESS_f1": 0.9896907216494845,
+      "eval_SEX_f1": 0.9263157894736842,
+      "eval_SSN_f1": 0.9041713641488163,
+      "eval_STATE_f1": 0.8811188811188813,
+      "eval_STREET_f1": 0.9502645502645503,
+      "eval_TIME_f1": 0.9835616438356163,
+      "eval_URL_f1": 0.9965397923875432,
+      "eval_USERAGENT_f1": 0.969249692496925,
+      "eval_USERNAME_f1": 0.9847457627118644,
+      "eval_VEHICLEVIN_f1": 0.9584487534626039,
+      "eval_VEHICLEVRM_f1": 0.976303317535545,
+      "eval_ZIPCODE_f1": 0.9171270718232043,
+      "eval_loss": 0.07277020812034607,
+      "eval_overall_accuracy": 0.9708370884800933,
+      "eval_overall_f1": 0.9128895880444401,
+      "eval_overall_precision": 0.9037815739524018,
+      "eval_overall_recall": 0.9221830461339823,
+      "eval_runtime": 52.7758,
+      "eval_samples_per_second": 162.139,
+      "eval_steps_per_second": 20.274,
+      "step": 3210
+    },
+    {
+      "epoch": 3.27,
+      "learning_rate": 3.744698952525733e-05,
+      "loss": 0.0855,
+      "step": 3500
+    },
+    {
+      "epoch": 3.74,
+      "learning_rate": 3.140292654261333e-05,
+      "loss": 0.0793,
+      "step": 4000
+    },
+    {
+      "epoch": 4.0,
+      "eval_ACCOUNTNAME_f1": 0.9894366197183099,
+      "eval_ACCOUNTNUMBER_f1": 0.9810017271157168,
+      "eval_AGE_f1": 0.9607843137254902,
+      "eval_AMOUNT_f1": 0.9824922760041195,
+      "eval_BIC_f1": 0.9539473684210527,
+      "eval_BITCOINADDRESS_f1": 0.9632738719832109,
+      "eval_BUILDINGNUMBER_f1": 0.9435114503816794,
+      "eval_CITY_f1": 0.9880823401950162,
+      "eval_COMPANYNAME_f1": 0.9793388429752066,
+      "eval_COUNTY_f1": 0.9894419306184011,
+      "eval_CREDITCARDCVV_f1": 0.8461538461538461,
+      "eval_CREDITCARDISSUER_f1": 0.9893238434163701,
+      "eval_CREDITCARDNUMBER_f1": 0.9303238469087342,
+      "eval_CURRENCYCODE_f1": 0.9693877551020408,
+      "eval_CURRENCYNAME_f1": 0.41237113402061853,
+      "eval_CURRENCYSYMBOL_f1": 0.9921996879875196,
+      "eval_CURRENCY_f1": 0.839080459770115,
+      "eval_DATE_f1": 0.8977635782747604,
+      "eval_DOB_f1": 0.7260458839406208,
+      "eval_EMAIL_f1": 0.9974968710888611,
+      "eval_ETHEREUMADDRESS_f1": 0.9968253968253968,
+      "eval_EYECOLOR_f1": 0.9803921568627451,
+      "eval_FIRSTNAME_f1": 0.9756944444444445,
+      "eval_GENDER_f1": 0.9833729216152018,
+      "eval_HEIGHT_f1": 0.9925373134328358,
+      "eval_IBAN_f1": 0.9948849104859335,
+      "eval_IPV4_f1": 0.8325203252032521,
+      "eval_IPV6_f1": 0.81055900621118,
+      "eval_IP_f1": 0.0,
+      "eval_JOBAREA_f1": 0.8333333333333334,
+      "eval_JOBTITLE_f1": 0.9899159663865545,
+      "eval_JOBTYPE_f1": 0.9494949494949495,
+      "eval_LASTNAME_f1": 0.7368421052631577,
+      "eval_LITECOINADDRESS_f1": 0.9086021505376344,
+      "eval_MAC_f1": 1.0,
+      "eval_MASKEDNUMBER_f1": 0.9002624671916011,
+      "eval_MIDDLENAME_f1": 0.0,
+      "eval_NEARBYGPSCOORDINATE_f1": 1.0,
+      "eval_ORDINALDIRECTION_f1": 0.0,
+      "eval_PASSWORD_f1": 0.9919028340080972,
+      "eval_PHONEIMEI_f1": 0.9934469200524245,
+      "eval_PHONENUMBER_f1": 0.9913232104121475,
+      "eval_PIN_f1": 0.9555555555555556,
+      "eval_PREFIX_f1": 0.9337748344370861,
+      "eval_SECONDARYADDRESS_f1": 0.9942726231386025,
+      "eval_SEX_f1": 0.9473684210526316,
+      "eval_SSN_f1": 0.9888475836431226,
+      "eval_STATE_f1": 0.9788053949903661,
+      "eval_STREET_f1": 0.9915074309978769,
+      "eval_TIME_f1": 0.9699453551912569,
+      "eval_URL_f1": 0.9948096885813148,
+      "eval_USERAGENT_f1": 0.9777777777777777,
+      "eval_USERNAME_f1": 0.989778534923339,
+      "eval_VEHICLEVIN_f1": 0.9752066115702478,
+      "eval_VEHICLEVRM_f1": 0.9835294117647059,
+      "eval_ZIPCODE_f1": 0.9559322033898306,
+      "eval_loss": 0.06745261698961258,
+      "eval_overall_accuracy": 0.9741716588029475,
+      "eval_overall_f1": 0.940074628709813,
+      "eval_overall_precision": 0.935612395474668,
+      "eval_overall_recall": 0.9445796295376669,
+      "eval_runtime": 53.5857,
+      "eval_samples_per_second": 159.688,
+      "eval_steps_per_second": 19.968,
+      "step": 4280
+    },
+    {
+      "epoch": 4.21,
+      "learning_rate": 2.4921355453803287e-05,
+      "loss": 0.0717,
+      "step": 4500
+    },
+    {
+      "epoch": 4.67,
+      "learning_rate": 1.8445158099368897e-05,
+      "loss": 0.0656,
+      "step": 5000
+    },
+    {
+      "epoch": 5.0,
+      "eval_ACCOUNTNAME_f1": 0.9964539007092198,
+      "eval_ACCOUNTNUMBER_f1": 0.9965277777777778,
+      "eval_AGE_f1": 0.9626016260162603,
+      "eval_AMOUNT_f1": 0.9866117404737385,
+      "eval_BIC_f1": 0.9770491803278688,
+      "eval_BITCOINADDRESS_f1": 0.9789029535864978,
+      "eval_BUILDINGNUMBER_f1": 0.9657320872274143,
+      "eval_CITY_f1": 0.9935064935064936,
+      "eval_COMPANYNAME_f1": 0.99581589958159,
+      "eval_COUNTY_f1": 0.9969788519637462,
+      "eval_CREDITCARDCVV_f1": 0.9655172413793104,
+      "eval_CREDITCARDISSUER_f1": 0.9964412811387899,
+      "eval_CREDITCARDNUMBER_f1": 0.9431704885343968,
+      "eval_CURRENCYCODE_f1": 0.9743589743589743,
+      "eval_CURRENCYNAME_f1": 0.6976744186046512,
+      "eval_CURRENCYSYMBOL_f1": 0.99375,
+      "eval_CURRENCY_f1": 0.8341880341880342,
+      "eval_DATE_f1": 0.9382303839732888,
+      "eval_DOB_f1": 0.8596059113300493,
+      "eval_EMAIL_f1": 1.0,
+      "eval_ETHEREUMADDRESS_f1": 1.0,
+      "eval_EYECOLOR_f1": 0.9607843137254902,
+      "eval_FIRSTNAME_f1": 0.9821506312581629,
+      "eval_GENDER_f1": 0.9940968122786304,
+      "eval_HEIGHT_f1": 0.9925373134328358,
+      "eval_IBAN_f1": 0.9948849104859335,
+      "eval_IPV4_f1": 0.8390243902439024,
+      "eval_IPV6_f1": 0.8003207698476343,
+      "eval_IP_f1": 0.13599999999999998,
+      "eval_JOBAREA_f1": 0.9333333333333333,
+      "eval_JOBTITLE_f1": 0.9991575400168492,
+      "eval_JOBTYPE_f1": 0.98,
+      "eval_LASTNAME_f1": 0.9157175398633257,
+      "eval_LITECOINADDRESS_f1": 0.956989247311828,
+      "eval_MAC_f1": 1.0,
+      "eval_MASKEDNUMBER_f1": 0.9360613810741688,
+      "eval_MIDDLENAME_f1": 0.0,
+      "eval_NEARBYGPSCOORDINATE_f1": 1.0,
+      "eval_ORDINALDIRECTION_f1": 0.4,
+      "eval_PASSWORD_f1": 0.9828456104944501,
+      "eval_PHONEIMEI_f1": 0.9921259842519685,
+      "eval_PHONENUMBER_f1": 0.9978260869565218,
+      "eval_PIN_f1": 0.9814126394052044,
+      "eval_PREFIX_f1": 0.9651741293532339,
+      "eval_SECONDARYADDRESS_f1": 0.9988545246277205,
+      "eval_SEX_f1": 0.9574468085106383,
+      "eval_SSN_f1": 0.9938042131350682,
+      "eval_STATE_f1": 0.9923076923076923,
+      "eval_STREET_f1": 0.9957446808510638,
+      "eval_TIME_f1": 0.9766162310866574,
+      "eval_URL_f1": 0.9991341991341991,
+      "eval_USERAGENT_f1": 0.9975308641975309,
+      "eval_USERNAME_f1": 0.9982964224872232,
+      "eval_VEHICLEVIN_f1": 0.9944444444444445,
+      "eval_VEHICLEVRM_f1": 0.9976359338061466,
+      "eval_ZIPCODE_f1": 0.9765363128491621,
+      "eval_loss": 0.04688200727105141,
+      "eval_overall_accuracy": 0.9779966071144569,
+      "eval_overall_f1": 0.95558140110366,
+      "eval_overall_precision": 0.9523527670908553,
+      "eval_overall_recall": 0.9588320007945573,
+      "eval_runtime": 53.4519,
+      "eval_samples_per_second": 160.088,
+      "eval_steps_per_second": 20.018,
+      "step": 5350
+    },
+    {
+      "epoch": 5.14,
+      "learning_rate": 1.2416849135825289e-05,
+      "loss": 0.063,
+      "step": 5500
+    },
+    {
+      "epoch": 5.61,
+      "learning_rate": 7.248339276261382e-06,
+      "loss": 0.057,
+      "step": 6000
+    },
+    {
+      "epoch": 6.0,
+      "eval_ACCOUNTNAME_f1": 0.9964539007092198,
+      "eval_ACCOUNTNUMBER_f1": 1.0,
+      "eval_AGE_f1": 0.9581993569131834,
+      "eval_AMOUNT_f1": 0.9897330595482546,
+      "eval_BIC_f1": 0.9836065573770492,
+      "eval_BITCOINADDRESS_f1": 0.9850746268656716,
+      "eval_BUILDINGNUMBER_f1": 0.9833080424886191,
+      "eval_CITY_f1": 0.9978401727861772,
+      "eval_COMPANYNAME_f1": 0.997907949790795,
+      "eval_COUNTY_f1": 0.9984871406959154,
+      "eval_CREDITCARDCVV_f1": 1.0,
+      "eval_CREDITCARDISSUER_f1": 0.9964412811387899,
+      "eval_CREDITCARDNUMBER_f1": 0.9580838323353293,
+      "eval_CURRENCYCODE_f1": 0.9948717948717948,
+      "eval_CURRENCYNAME_f1": 0.7294117647058823,
+      "eval_CURRENCYSYMBOL_f1": 0.9984301412872841,
+      "eval_CURRENCY_f1": 0.8955696202531646,
+      "eval_DATE_f1": 0.9524342473419138,
+      "eval_DOB_f1": 0.8848484848484849,
+      "eval_EMAIL_f1": 1.0,
+      "eval_ETHEREUMADDRESS_f1": 1.0,
+      "eval_EYECOLOR_f1": 0.9607843137254902,
+      "eval_FIRSTNAME_f1": 0.98861646234676,
+      "eval_GENDER_f1": 0.998819362455726,
+      "eval_HEIGHT_f1": 0.9925373134328358,
+      "eval_IBAN_f1": 0.9974424552429667,
+      "eval_IPV4_f1": 0.8377483443708609,
+      "eval_IPV6_f1": 0.8195429472025216,
+      "eval_IP_f1": 0.16633663366336635,
+      "eval_JOBAREA_f1": 0.912280701754386,
+      "eval_JOBTITLE_f1": 0.9991575400168492,
+      "eval_JOBTYPE_f1": 0.9607843137254902,
+      "eval_LASTNAME_f1": 0.9555555555555556,
+      "eval_LITECOINADDRESS_f1": 0.9562982005141388,
+      "eval_MAC_f1": 1.0,
+      "eval_MASKEDNUMBER_f1": 0.9527458492975734,
+      "eval_MIDDLENAME_f1": 0.2857142857142857,
+      "eval_NEARBYGPSCOORDINATE_f1": 1.0,
+      "eval_ORDINALDIRECTION_f1": 0.4,
+      "eval_PASSWORD_f1": 0.9959514170040485,
+      "eval_PHONEIMEI_f1": 0.9960578186596584,
+      "eval_PHONENUMBER_f1": 0.9978260869565218,
+      "eval_PIN_f1": 0.9851851851851852,
+      "eval_PREFIX_f1": 0.9797979797979799,
+      "eval_SECONDARYADDRESS_f1": 0.9988545246277205,
+      "eval_SEX_f1": 0.9782608695652174,
+      "eval_SSN_f1": 0.9975247524752475,
+      "eval_STATE_f1": 0.9942418426103647,
+      "eval_STREET_f1": 0.997872340425532,
+      "eval_TIME_f1": 0.9807162534435262,
+      "eval_URL_f1": 1.0,
+      "eval_USERAGENT_f1": 0.9975308641975309,
+      "eval_USERNAME_f1": 1.0,
+      "eval_VEHICLEVIN_f1": 0.9808219178082193,
+      "eval_VEHICLEVRM_f1": 0.9929078014184397,
+      "eval_ZIPCODE_f1": 0.9931818181818183,
+      "eval_loss": 0.044823143631219864,
+      "eval_overall_accuracy": 0.9802602979377617,
+      "eval_overall_f1": 0.962836500395883,
+      "eval_overall_precision": 0.9594654568765718,
+      "eval_overall_recall": 0.9662313154889011,
+      "eval_runtime": 52.7618,
+      "eval_samples_per_second": 162.182,
+      "eval_steps_per_second": 20.28,
+      "step": 6420
+    },
+    {
+      "epoch": 6.07,
+      "learning_rate": 3.2927896799234857e-06,
+      "loss": 0.0525,
+      "step": 6500
+    },
+    {
+      "epoch": 6.54,
+      "learning_rate": 8.204806633547002e-07,
+      "loss": 0.0455,
+      "step": 7000
+    },
+    {
+      "epoch": 7.0,
+      "eval_ACCOUNTNAME_f1": 0.9964539007092198,
+      "eval_ACCOUNTNUMBER_f1": 1.0,
+      "eval_AGE_f1": 0.9676375404530744,
+      "eval_AMOUNT_f1": 0.9897330595482546,
+      "eval_BIC_f1": 0.9868421052631579,
+      "eval_BITCOINADDRESS_f1": 0.9925611052072264,
+      "eval_BUILDINGNUMBER_f1": 0.9818181818181818,
+      "eval_CITY_f1": 0.9967567567567569,
+      "eval_COMPANYNAME_f1": 0.997907949790795,
+      "eval_COUNTY_f1": 1.0,
+      "eval_CREDITCARDCVV_f1": 1.0,
+      "eval_CREDITCARDISSUER_f1": 0.9964412811387899,
+      "eval_CREDITCARDNUMBER_f1": 0.963110667996012,
+      "eval_CURRENCYCODE_f1": 0.9948717948717948,
+      "eval_CURRENCYNAME_f1": 0.7481481481481482,
+      "eval_CURRENCYSYMBOL_f1": 0.9984301412872841,
+      "eval_CURRENCY_f1": 0.8896103896103896,
+      "eval_DATE_f1": 0.9608621667612025,
+      "eval_DOB_f1": 0.9159763313609468,
+      "eval_EMAIL_f1": 1.0,
+      "eval_ETHEREUMADDRESS_f1": 1.0,
+      "eval_EYECOLOR_f1": 0.9607843137254902,
+      "eval_FIRSTNAME_f1": 0.9907935116177115,
+      "eval_GENDER_f1": 0.998819362455726,
+      "eval_HEIGHT_f1": 0.9925373134328358,
+      "eval_IBAN_f1": 0.9974424552429667,
+      "eval_IPV4_f1": 0.8145094806265457,
+      "eval_IPV6_f1": 0.7834890965732086,
+      "eval_IP_f1": 0.21772939346811815,
+      "eval_JOBAREA_f1": 0.983050847457627,
+      "eval_JOBTITLE_f1": 0.9991575400168492,
+      "eval_JOBTYPE_f1": 0.98989898989899,
+      "eval_LASTNAME_f1": 0.9642857142857142,
+      "eval_LITECOINADDRESS_f1": 0.9815303430079155,
+      "eval_MAC_f1": 1.0,
+      "eval_MASKEDNUMBER_f1": 0.958974358974359,
+      "eval_MIDDLENAME_f1": 0.43750000000000006,
+      "eval_NEARBYGPSCOORDINATE_f1": 1.0,
+      "eval_ORDINALDIRECTION_f1": 0.4,
+      "eval_PASSWORD_f1": 0.9959514170040485,
+      "eval_PHONEIMEI_f1": 0.9960578186596584,
+      "eval_PHONENUMBER_f1": 0.9978260869565218,
+      "eval_PIN_f1": 0.9814126394052044,
+      "eval_PREFIX_f1": 0.980688497061293,
+      "eval_SECONDARYADDRESS_f1": 0.9988545246277205,
+      "eval_SEX_f1": 0.9782608695652174,
+      "eval_SSN_f1": 0.995049504950495,
+      "eval_STATE_f1": 0.992337164750958,
+      "eval_STREET_f1": 0.9968119022316684,
+      "eval_TIME_f1": 0.9861878453038674,
+      "eval_URL_f1": 1.0,
+      "eval_USERAGENT_f1": 0.9975308641975309,
+      "eval_USERNAME_f1": 1.0,
+      "eval_VEHICLEVIN_f1": 1.0,
+      "eval_VEHICLEVRM_f1": 0.9929078014184397,
+      "eval_ZIPCODE_f1": 0.994311717861206,
+      "eval_loss": 0.04238386079668999,
+      "eval_overall_accuracy": 0.9817632402056937,
+      "eval_overall_f1": 0.961428430502121,
+      "eval_overall_precision": 0.9549752584390769,
+      "eval_overall_recall": 0.9679694095446194,
+      "eval_runtime": 51.5385,
+      "eval_samples_per_second": 166.031,
+      "eval_steps_per_second": 20.761,
+      "step": 7490
+    },
+    {
+      "epoch": 7.0,
+      "step": 7490,
+      "total_flos": 2378184631534176.0,
+      "train_loss": 0.19348664430177737,
+      "train_runtime": 1425.9685,
+      "train_samples_per_second": 42.006,
+      "train_steps_per_second": 5.253
+    },
+    {
+      "epoch": 7.0,
+      "eval_ACCOUNTNAME_f1": 0.9964539007092198,
+      "eval_ACCOUNTNUMBER_f1": 1.0,
+      "eval_AGE_f1": 0.9676375404530744,
+      "eval_AMOUNT_f1": 0.9897330595482546,
+      "eval_BIC_f1": 0.9868421052631579,
+      "eval_BITCOINADDRESS_f1": 0.9925611052072264,
+      "eval_BUILDINGNUMBER_f1": 0.9818181818181818,
+      "eval_CITY_f1": 0.9967567567567569,
+      "eval_COMPANYNAME_f1": 0.997907949790795,
+      "eval_COUNTY_f1": 1.0,
+      "eval_CREDITCARDCVV_f1": 1.0,
+      "eval_CREDITCARDISSUER_f1": 0.9964412811387899,
+      "eval_CREDITCARDNUMBER_f1": 0.963110667996012,
+      "eval_CURRENCYCODE_f1": 0.9948717948717948,
+      "eval_CURRENCYNAME_f1": 0.7481481481481482,
+      "eval_CURRENCYSYMBOL_f1": 0.9984301412872841,
+      "eval_CURRENCY_f1": 0.8896103896103896,
+      "eval_DATE_f1": 0.9608621667612025,
+      "eval_DOB_f1": 0.9159763313609468,
+      "eval_EMAIL_f1": 1.0,
+      "eval_ETHEREUMADDRESS_f1": 1.0,
+      "eval_EYECOLOR_f1": 0.9607843137254902,
+      "eval_FIRSTNAME_f1": 0.9907935116177115,
+      "eval_GENDER_f1": 0.998819362455726,
+      "eval_HEIGHT_f1": 0.9925373134328358,
+      "eval_IBAN_f1": 0.9974424552429667,
+      "eval_IPV4_f1": 0.8145094806265457,
+      "eval_IPV6_f1": 0.7834890965732086,
+      "eval_IP_f1": 0.21772939346811815,
+      "eval_JOBAREA_f1": 0.983050847457627,
+      "eval_JOBTITLE_f1": 0.9991575400168492,
+      "eval_JOBTYPE_f1": 0.98989898989899,
+      "eval_LASTNAME_f1": 0.9642857142857142,
+      "eval_LITECOINADDRESS_f1": 0.9815303430079155,
+      "eval_MAC_f1": 1.0,
+      "eval_MASKEDNUMBER_f1": 0.958974358974359,
+      "eval_MIDDLENAME_f1": 0.43750000000000006,
+      "eval_NEARBYGPSCOORDINATE_f1": 1.0,
+      "eval_ORDINALDIRECTION_f1": 0.4,
+      "eval_PASSWORD_f1": 0.9959514170040485,
+      "eval_PHONEIMEI_f1": 0.9960578186596584,
+      "eval_PHONENUMBER_f1": 0.9978260869565218,
+      "eval_PIN_f1": 0.9814126394052044,
+      "eval_PREFIX_f1": 0.980688497061293,
+      "eval_SECONDARYADDRESS_f1": 0.9988545246277205,
+      "eval_SEX_f1": 0.9782608695652174,
+      "eval_SSN_f1": 0.995049504950495,
+      "eval_STATE_f1": 0.992337164750958,
+      "eval_STREET_f1": 0.9968119022316684,
+      "eval_TIME_f1": 0.9861878453038674,
+      "eval_URL_f1": 1.0,
+      "eval_USERAGENT_f1": 0.9975308641975309,
+      "eval_USERNAME_f1": 1.0,
+      "eval_VEHICLEVIN_f1": 1.0,
+      "eval_VEHICLEVRM_f1": 0.9929078014184397,
+      "eval_ZIPCODE_f1": 0.994311717861206,
+      "eval_loss": 0.04238386079668999,
+      "eval_overall_accuracy": 0.9817632402056937,
+      "eval_overall_f1": 0.961428430502121,
+      "eval_overall_precision": 0.9549752584390769,
+      "eval_overall_recall": 0.9679694095446194,
+      "eval_runtime": 51.5335,
+      "eval_samples_per_second": 166.047,
+      "eval_steps_per_second": 20.763,
+      "step": 7490
+    }
+  ],
+  "logging_steps": 500,
+  "max_steps": 7490,
+  "num_train_epochs": 7,
+  "save_steps": 500,
+  "total_flos": 2378184631534176.0,
+  "trial_name": null,
+  "trial_params": null
+}