
<!-- This model card has been generated automatically according to the information Keras had access to. You should
probably proofread and complete it, then remove this comment. -->

# Bert_1e-05

This model is a fine-tuned version of [](https://huggingface.co/) on an unknown dataset.
It achieves the following results on the evaluation set:
- Train Loss: 0.3364
- Train Prediction Logits Accuracy: 0.0842
- Train Seq Relationship Logits Accuracy: 0.9769
- Validation Loss: 2.5552
- Validation Prediction Logits Accuracy: 0.0845
- Validation Seq Relationship Logits Accuracy: 0.5286
- Train Lr: 8.163814e-06
- Epoch: 201

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- optimizer: {'name': 'Adam', 'learning_rate': 8.147356e-06, 'decay': 0.0, 'beta_1': 0.9, 'beta_2': 0.999, 'epsilon': 1e-07, 'amsgrad': False}
- training_precision: float32

### Training results

| Train Loss | Train Prediction Logits Accuracy | Train Seq Relationship Logits Accuracy | Validation Loss | Validation Prediction Logits Accuracy | Validation Seq Relationship Logits Accuracy | Train Lr      | Epoch |
|:----------:|:--------------------------------:|:--------------------------------------:|:---------------:|:-------------------------------------:|:-------------------------------------------:|:-------------:|:-----:|
| 2.2104     | 0.0706                           | 0.4976                                 | 1.4331          | 0.0733                                | 0.5065                                      | 1e-05         | 0     |
| 1.3307     | 0.0759                           | 0.5009                                 | 1.1897          | 0.0789                                | 0.4935                                      | 9.9999e-06    | 1     |
| 1.1413     | 0.0805                           | 0.4980                                 | 1.0788          | 0.0819                                | 0.4940                                      | 9.9997e-06    | 2     |
| 1.0732     | 0.0820                           | 0.4993                                 | 1.0470          | 0.0825                                | 0.4935                                      | 9.9993995e-06 | 3     |
| 1.0487     | 0.0824                           | 0.5018                                 | 1.0386          | 0.0827                                | 0.4935                                      | 9.998999e-06  | 4     |
| 1.0327     | 0.0828                           | 0.5015                                 | 1.0190          | 0.0835                                | 0.4935                                      | 9.998499e-06  | 5     |
| 1.0215     | 0.0831                           | 0.5024                                 | 1.0091          | 0.0834                                | 0.4935                                      | 9.997899e-06  | 6     |
| 1.0138     | 0.0833                           | 0.5009                                 | 1.0045          | 0.0834                                | 0.4933                                      | 9.997199e-06  | 7     |
| 1.0086     | 0.0833                           | 0.5019                                 | 0.9944          | 0.0839                                | 0.5066                                      | 9.9964e-06    | 8     |
| 1.0050     | 0.0834                           | 0.5020                                 | 1.0004          | 0.0835                                | 0.5068                                      | 9.9955005e-06 | 9     |
| 1.0010     | 0.0835                           | 0.5032                                 | 0.9928          | 0.0836                                | 0.5051                                      | 9.994501e-06  | 10    |
| 0.9988     | 0.0836                           | 0.5052                                 | 0.9941          | 0.0835                                | 0.5076                                      | 9.993401e-06  | 11    |
| 0.9961     | 0.0835                           | 0.5073                                 | 0.9953          | 0.0834                                | 0.5079                                      | 9.992203e-06  | 12    |
| 0.9937     | 0.0837                           | 0.5112                                 | 0.9897          | 0.0839                                | 0.5072                                      | 9.990904e-06  | 13    |
| 0.9913     | 0.0837                           | 0.5157                                 | 0.9894          | 0.0839                                | 0.5099                                      | 9.989505e-06  | 14    |
| 0.9894     | 0.0838                           | 0.5197                                 | 0.9845          | 0.0842                                | 0.5269                                      | 9.988007e-06  | 15    |
| 0.9859     | 0.0838                           | 0.5243                                 | 0.9839          | 0.0836                                | 0.5332                                      | 9.986409e-06  | 16    |
| 0.9844     | 0.0838                           | 0.5285                                 | 0.9831          | 0.0841                                | 0.5352                                      | 9.984712e-06  | 17    |
| 0.9831     | 0.0838                           | 0.5345                                 | 0.9805          | 0.0840                                | 0.5192                                      | 9.982915e-06  | 18    |
| 0.9803     | 0.0839                           | 0.5406                                 | 0.9832          | 0.0840                                | 0.5302                                      | 9.981019e-06  | 19    |
| 0.9788     | 0.0838                           | 0.5416                                 | 0.9771          | 0.0842                                | 0.5245                                      | 9.979023e-06  | 20    |
| 0.9770     | 0.0838                           | 0.5439                                 | 0.9796          | 0.0835                                | 0.5340                                      | 9.976928e-06  | 21    |
| 0.9748     | 0.0839                           | 0.5481                                 | 0.9830          | 0.0842                                | 0.5341                                      | 9.974733e-06  | 22    |
| 0.9742     | 0.0839                           | 0.5514                                 | 0.9824          | 0.0839                                | 0.5342                                      | 9.972439e-06  | 23    |
| 0.9704     | 0.0840                           | 0.5537                                 | 0.9820          | 0.0840                                | 0.5251                                      | 9.970046e-06  | 24    |
| 0.9699     | 0.0840                           | 0.5578                                 | 0.9830          | 0.0839                                | 0.5385                                      | 9.967554e-06  | 25    |
| 0.9676     | 0.0839                           | 0.5634                                 | 0.9804          | 0.0838                                | 0.5254                                      | 9.964963e-06  | 26    |
| 0.9642     | 0.0839                           | 0.5666                                 | 0.9781          | 0.0841                                | 0.5343                                      | 9.962274e-06  | 27    |
| 0.9632     | 0.0839                           | 0.5711                                 | 0.9857          | 0.0842                                | 0.5263                                      | 9.959485e-06  | 28    |
| 0.9608     | 0.0839                           | 0.5742                                 | 0.9755          | 0.0839                                | 0.5340                                      | 9.956598e-06  | 29    |
| 0.9575     | 0.0840                           | 0.5788                                 | 0.9743          | 0.0844                                | 0.5249                                      | 9.953612e-06  | 30    |
| 0.9559     | 0.0839                           | 0.5817                                 | 0.9737          | 0.0838                                | 0.5338                                      | 9.950527e-06  | 31    |
| 0.9538     | 0.0839                           | 0.5887                                 | 0.9855          | 0.0843                                | 0.5275                                      | 9.947344e-06  | 32    |
| 0.9500     | 0.0839                           | 0.5940                                 | 0.9828          | 0.0844                                | 0.5302                                      | 9.944062e-06  | 33    |
| 0.9438     | 0.0840                           | 0.5993                                 | 0.9853          | 0.0839                                | 0.5353                                      | 9.9406825e-06 | 34    |
| 0.9419     | 0.0839                           | 0.6045                                 | 0.9898          | 0.0843                                | 0.5312                                      | 9.937205e-06  | 35    |
| 0.9376     | 0.0841                           | 0.6097                                 | 0.9919          | 0.0840                                | 0.5331                                      | 9.9336285e-06 | 36    |
| 0.9344     | 0.0841                           | 0.6148                                 | 0.9969          | 0.0838                                | 0.5329                                      | 9.929954e-06  | 37    |
| 0.9282     | 0.0840                           | 0.6228                                 | 0.9952          | 0.0839                                | 0.5313                                      | 9.926182e-06  | 38    |
| 0.9235     | 0.0841                           | 0.6279                                 | 1.0027          | 0.0838                                | 0.5317                                      | 9.9223125e-06 | 39    |
| 0.9210     | 0.0840                           | 0.6326                                 | 1.0027          | 0.0842                                | 0.5318                                      | 9.918345e-06  | 40    |
| 0.9132     | 0.0841                           | 0.6395                                 | 1.0130          | 0.0845                                | 0.5322                                      | 9.914281e-06  | 41    |
| 0.9077     | 0.0840                           | 0.6465                                 | 1.0166          | 0.0840                                | 0.5329                                      | 9.910119e-06  | 42    |
| 0.9026     | 0.0840                           | 0.6540                                 | 1.0206          | 0.0844                                | 0.5293                                      | 9.90586e-06   | 43    |
| 0.8962     | 0.0839                           | 0.6603                                 | 1.0319          | 0.0844                                | 0.5240                                      | 9.901503e-06  | 44    |
| 0.8890     | 0.0840                           | 0.6655                                 | 1.0389          | 0.0840                                | 0.5331                                      | 9.8970495e-06 | 45    |
| 0.8816     | 0.0840                           | 0.6733                                 | 1.0503          | 0.0839                                | 0.5287                                      | 9.892499e-06  | 46    |
| 0.8745     | 0.0840                           | 0.6798                                 | 1.0499          | 0.0837                                | 0.5284                                      | 9.887852e-06  | 47    |
| 0.8687     | 0.0840                           | 0.6860                                 | 1.0681          | 0.0842                                | 0.5312                                      | 9.883108e-06  | 48    |
| 0.8606     | 0.0840                           | 0.6926                                 | 1.0608          | 0.0844                                | 0.5207                                      | 9.8782675e-06 | 49    |
| 0.8504     | 0.0840                           | 0.7009                                 | 1.0815          | 0.0842                                | 0.5200                                      | 9.873331e-06  | 50    |
| 0.8432     | 0.0840                           | 0.7062                                 | 1.1080          | 0.0837                                | 0.5272                                      | 9.868298e-06  | 51    |
| 0.8357     | 0.0840                           | 0.7130                                 | 1.0939          | 0.0836                                | 0.5274                                      | 9.863169e-06  | 52    |
| 0.8267     | 0.0840                           | 0.7200                                 | 1.1253          | 0.0840                                | 0.5288                                      | 9.857944e-06  | 53    |
| 0.8178     | 0.0840                           | 0.7260                                 | 1.1478          | 0.0841                                | 0.5201                                      | 9.852623e-06  | 54    |
| 0.8080     | 0.0840                           | 0.7334                                 | 1.1286          | 0.0838                                | 0.5252                                      | 9.847207e-06  | 55    |
| 0.8007     | 0.0840                           | 0.7402                                 | 1.1341          | 0.0839                                | 0.5304                                      | 9.841696e-06  | 56    |
| 0.7909     | 0.0840                           | 0.7474                                 | 1.1668          | 0.0844                                | 0.5261                                      | 9.83609e-06   | 57    |
| 0.7818     | 0.0840                           | 0.7524                                 | 1.1632          | 0.0840                                | 0.5305                                      | 9.830388e-06  | 58    |
| 0.7726     | 0.0840                           | 0.7588                                 | 1.1894          | 0.0837                                | 0.5302                                      | 9.824592e-06  | 59    |
| 0.7614     | 0.0840                           | 0.7663                                 | 1.2324          | 0.0844                                | 0.5256                                      | 9.818701e-06  | 60    |
| 0.7526     | 0.0841                           | 0.7718                                 | 1.2456          | 0.0840                                | 0.5273                                      | 9.812716e-06  | 61    |
| 0.7444     | 0.0841                           | 0.7781                                 | 1.2365          | 0.0843                                | 0.5357                                      | 9.806636e-06  | 62    |
| 0.7337     | 0.0840                           | 0.7832                                 | 1.2556          | 0.0841                                | 0.5222                                      | 9.800461e-06  | 63    |
| 0.7236     | 0.0841                           | 0.7895                                 | 1.2543          | 0.0842                                | 0.5242                                      | 9.794193e-06  | 64    |
| 0.7174     | 0.0839                           | 0.7944                                 | 1.2844          | 0.0841                                | 0.5172                                      | 9.787831e-06  | 65    |
| 0.7078     | 0.0840                           | 0.8004                                 | 1.2781          | 0.0839                                | 0.5323                                      | 9.781375e-06  | 66    |
| 0.7006     | 0.0840                           | 0.8043                                 | 1.3261          | 0.0839                                | 0.5216                                      | 9.774826e-06  | 67    |
| 0.6886     | 0.0840                           | 0.8101                                 | 1.3130          | 0.0843                                | 0.5192                                      | 9.768184e-06  | 68    |
| 0.6805     | 0.0840                           | 0.8152                                 | 1.3506          | 0.0845                                | 0.5225                                      | 9.761448e-06  | 69    |
| 0.6716     | 0.0841                           | 0.8213                                 | 1.3442          | 0.0841                                | 0.5206                                      | 9.75462e-06   | 70    |
| 0.6637     | 0.0840                           | 0.8259                                 | 1.3656          | 0.0843                                | 0.5212                                      | 9.747699e-06  | 71    |
| 0.6552     | 0.0841                           | 0.8298                                 | 1.3985          | 0.0842                                | 0.5221                                      | 9.7406855e-06 | 72    |
| 0.6470     | 0.0840                           | 0.8346                                 | 1.3939          | 0.0838                                | 0.5289                                      | 9.73358e-06   | 73    |
| 0.6399     | 0.0840                           | 0.8382                                 | 1.4051          | 0.0845                                | 0.5305                                      | 9.726382e-06  | 74    |
| 0.6297     | 0.0840                           | 0.8436                                 | 1.4613          | 0.0840                                | 0.5238                                      | 9.719092e-06  | 75    |
| 0.6253     | 0.0840                           | 0.8463                                 | 1.4475          | 0.0839                                | 0.5273                                      | 9.711712e-06  | 76    |
| 0.6173     | 0.0840                           | 0.8511                                 | 1.4491          | 0.0842                                | 0.5275                                      | 9.704239e-06  | 77    |
| 0.6109     | 0.0840                           | 0.8547                                 | 1.4864          | 0.0842                                | 0.5215                                      | 9.696676e-06  | 78    |
| 0.6031     | 0.0842                           | 0.8577                                 | 1.5231          | 0.0838                                | 0.5272                                      | 9.689022e-06  | 79    |
| 0.5925     | 0.0841                           | 0.8626                                 | 1.5257          | 0.0840                                | 0.5263                                      | 9.681276e-06  | 80    |
| 0.5887     | 0.0841                           | 0.8662                                 | 1.5597          | 0.0838                                | 0.5238                                      | 9.673441e-06  | 81    |
| 0.5826     | 0.0839                           | 0.8688                                 | 1.5773          | 0.0842                                | 0.5276                                      | 9.665516e-06  | 82    |
| 0.5751     | 0.0841                           | 0.8720                                 | 1.5350          | 0.0842                                | 0.5278                                      | 9.6575e-06    | 83    |
| 0.5669     | 0.0840                           | 0.8762                                 | 1.5761          | 0.0838                                | 0.5260                                      | 9.649395e-06  | 84    |
| 0.5613     | 0.0840                           | 0.8801                                 | 1.5982          | 0.0839                                | 0.5319                                      | 9.6411995e-06 | 85    |
| 0.5579     | 0.0839                           | 0.8820                                 | 1.6631          | 0.0842                                | 0.5297                                      | 9.632915e-06  | 86    |
| 0.5504     | 0.0840                           | 0.8854                                 | 1.6899          | 0.0843                                | 0.5193                                      | 9.624541e-06  | 87    |
| 0.5440     | 0.0841                           | 0.8888                                 | 1.6789          | 0.0838                                | 0.5253                                      | 9.616079e-06  | 88    |
| 0.5411     | 0.0841                           | 0.8897                                 | 1.6283          | 0.0842                                | 0.5241                                      | 9.607528e-06  | 89    |
| 0.5346     | 0.0841                           | 0.8925                                 | 1.7220          | 0.0838                                | 0.5279                                      | 9.598889e-06  | 90    |
| 0.5279     | 0.0840                           | 0.8953                                 | 1.7215          | 0.0846                                | 0.5245                                      | 9.590162e-06  | 91    |
| 0.5275     | 0.0840                           | 0.8967                                 | 1.7656          | 0.0841                                | 0.5242                                      | 9.5813475e-06 | 92    |
| 0.5214     | 0.0841                           | 0.8995                                 | 1.7701          | 0.0843                                | 0.5269                                      | 9.572445e-06  | 93    |
| 0.5151     | 0.0840                           | 0.9014                                 | 1.7710          | 0.0841                                | 0.5194                                      | 9.563456e-06  | 94    |
| 0.5109     | 0.0840                           | 0.9035                                 | 1.7668          | 0.0845                                | 0.5207                                      | 9.554379e-06  | 95    |
| 0.5060     | 0.0840                           | 0.9054                                 | 1.7678          | 0.0839                                | 0.5250                                      | 9.545216e-06  | 96    |
| 0.5051     | 0.0840                           | 0.9073                                 | 1.8488          | 0.0843                                | 0.5217                                      | 9.5359665e-06 | 97    |
| 0.4997     | 0.0841                           | 0.9085                                 | 1.7930          | 0.0838                                | 0.5275                                      | 9.5266305e-06 | 98    |
| 0.4935     | 0.0841                           | 0.9110                                 | 1.8310          | 0.0840                                | 0.5244                                      | 9.517208e-06  | 99    |
| 0.4890     | 0.0840                           | 0.9148                                 | 1.8634          | 0.0838                                | 0.5291                                      | 9.5077e-06    | 100   |
| 0.4842     | 0.0841                           | 0.9154                                 | 1.9333          | 0.0839                                | 0.5289                                      | 9.498107e-06  | 101   |
| 0.4799     | 0.0840                           | 0.9178                                 | 1.8706          | 0.0846                                | 0.5268                                      | 9.488429e-06  | 102   |
| 0.4802     | 0.0840                           | 0.9183                                 | 1.9054          | 0.0845                                | 0.5260                                      | 9.478666e-06  | 103   |
| 0.4736     | 0.0840                           | 0.9214                                 | 1.9271          | 0.0834                                | 0.5231                                      | 9.468818e-06  | 104   |
| 0.4689     | 0.0840                           | 0.9225                                 | 1.9448          | 0.0845                                | 0.5174                                      | 9.458886e-06  | 105   |
| 0.4693     | 0.0840                           | 0.9232                                 | 1.8939          | 0.0844                                | 0.5200                                      | 9.4488705e-06 | 106   |
| 0.4654     | 0.0841                           | 0.9251                                 | 1.9664          | 0.0843                                | 0.5293                                      | 9.438771e-06  | 107   |
| 0.4615     | 0.0841                           | 0.9266                                 | 1.9269          | 0.0839                                | 0.5206                                      | 9.428588e-06  | 108   |
| 0.4591     | 0.0840                           | 0.9277                                 | 1.9660          | 0.0839                                | 0.5226                                      | 9.4183215e-06 | 109   |
| 0.4564     | 0.0842                           | 0.9284                                 | 1.9834          | 0.0839                                | 0.5216                                      | 9.407972e-06  | 110   |
| 0.4533     | 0.0840                           | 0.9297                                 | 2.0348          | 0.0839                                | 0.5234                                      | 9.397541e-06  | 111   |
| 0.4494     | 0.0841                           | 0.9310                                 | 1.9011          | 0.0843                                | 0.5289                                      | 9.387028e-06  | 112   |
| 0.4481     | 0.0840                           | 0.9326                                 | 1.9886          | 0.0846                                | 0.5301                                      | 9.376432e-06  | 113   |
| 0.4436     | 0.0841                           | 0.9343                                 | 1.9610          | 0.0839                                | 0.5257                                      | 9.365755e-06  | 114   |
| 0.4416     | 0.0841                           | 0.9349                                 | 2.0094          | 0.0841                                | 0.5302                                      | 9.354997e-06  | 115   |
| 0.4382     | 0.0840                           | 0.9368                                 | 2.0537          | 0.0841                                | 0.5257                                      | 9.344158e-06  | 116   |
| 0.4379     | 0.0841                           | 0.9364                                 | 2.0767          | 0.0840                                | 0.5210                                      | 9.333237e-06  | 117   |
| 0.4363     | 0.0840                           | 0.9380                                 | 2.0032          | 0.0844                                | 0.5194                                      | 9.322237e-06  | 118   |
| 0.4312     | 0.0841                           | 0.9389                                 | 2.0951          | 0.0845                                | 0.5262                                      | 9.311157e-06  | 119   |
| 0.4276     | 0.0841                           | 0.9400                                 | 2.0397          | 0.0844                                | 0.5263                                      | 9.299996e-06  | 120   |
| 0.4273     | 0.0841                           | 0.9405                                 | 2.1198          | 0.0843                                | 0.5226                                      | 9.288757e-06  | 121   |
| 0.4243     | 0.0841                           | 0.9419                                 | 2.1345          | 0.0839                                | 0.5247                                      | 9.277438e-06  | 122   |
| 0.4222     | 0.0839                           | 0.9429                                 | 2.1052          | 0.0844                                | 0.5177                                      | 9.266041e-06  | 123   |
| 0.4210     | 0.0841                           | 0.9439                                 | 2.0947          | 0.0837                                | 0.5288                                      | 9.254565e-06  | 124   |
| 0.4166     | 0.0841                           | 0.9450                                 | 2.0941          | 0.0839                                | 0.5177                                      | 9.243012e-06  | 125   |
| 0.4168     | 0.0840                           | 0.9456                                 | 2.1108          | 0.0835                                | 0.5292                                      | 9.23138e-06   | 126   |
| 0.4148     | 0.0841                           | 0.9462                                 | 2.1886          | 0.0840                                | 0.5209                                      | 9.2196715e-06 | 127   |
| 0.4126     | 0.0840                           | 0.9474                                 | 2.2154          | 0.0842                                | 0.5231                                      | 9.207885e-06  | 128   |
| 0.4112     | 0.0840                           | 0.9477                                 | 2.2194          | 0.0841                                | 0.5188                                      | 9.196023e-06  | 129   |
| 0.4094     | 0.0841                           | 0.9483                                 | 2.2040          | 0.0839                                | 0.5224                                      | 9.184084e-06  | 130   |
| 0.4069     | 0.0840                           | 0.9495                                 | 2.2368          | 0.0839                                | 0.5292                                      | 9.1720685e-06 | 131   |
| 0.4062     | 0.0841                           | 0.9503                                 | 2.2022          | 0.0839                                | 0.5267                                      | 9.159978e-06  | 132   |
| 0.4052     | 0.0841                           | 0.9499                                 | 2.2235          | 0.0842                                | 0.5214                                      | 9.147811e-06  | 133   |
| 0.4017     | 0.0841                           | 0.9518                                 | 2.2446          | 0.0840                                | 0.5228                                      | 9.13557e-06   | 134   |
| 0.4006     | 0.0840                           | 0.9520                                 | 2.2686          | 0.0840                                | 0.5182                                      | 9.123253e-06  | 135   |
| 0.4001     | 0.0841                           | 0.9518                                 | 2.2369          | 0.0839                                | 0.5193                                      | 9.110862e-06  | 136   |
| 0.3981     | 0.0841                           | 0.9531                                 | 2.3013          | 0.0842                                | 0.5230                                      | 9.098398e-06  | 137   |
| 0.3945     | 0.0841                           | 0.9542                                 | 2.2672          | 0.0842                                | 0.5201                                      | 9.085859e-06  | 138   |
| 0.3934     | 0.0841                           | 0.9548                                 | 2.2886          | 0.0840                                | 0.5239                                      | 9.073247e-06  | 139   |
| 0.3926     | 0.0842                           | 0.9546                                 | 2.2500          | 0.0839                                | 0.5279                                      | 9.060563e-06  | 140   |
| 0.3937     | 0.0842                           | 0.9551                                 | 2.2164          | 0.0840                                | 0.5297                                      | 9.047805e-06  | 141   |
| 0.3891     | 0.0841                           | 0.9566                                 | 2.2879          | 0.0842                                | 0.5184                                      | 9.034976e-06  | 142   |
| 0.3868     | 0.0841                           | 0.9574                                 | 2.2954          | 0.0843                                | 0.5205                                      | 9.022075e-06  | 143   |
| 0.3883     | 0.0841                           | 0.9571                                 | 2.2583          | 0.0839                                | 0.5308                                      | 9.009102e-06  | 144   |
| 0.3885     | 0.0840                           | 0.9568                                 | 2.2577          | 0.0843                                | 0.5248                                      | 8.996058e-06  | 145   |
| 0.3848     | 0.0842                           | 0.9579                                 | 2.2797          | 0.0842                                | 0.5212                                      | 8.982943e-06  | 146   |
| 0.3825     | 0.0841                           | 0.9592                                 | 2.2335          | 0.0842                                | 0.5244                                      | 8.969757e-06  | 147   |
| 0.3836     | 0.0841                           | 0.9589                                 | 2.2234          | 0.0839                                | 0.5172                                      | 8.956501e-06  | 148   |
| 0.3811     | 0.0841                           | 0.9601                                 | 2.2173          | 0.0842                                | 0.5226                                      | 8.943176e-06  | 149   |
| 0.3806     | 0.0840                           | 0.9600                                 | 2.3137          | 0.0844                                | 0.5231                                      | 8.929781e-06  | 150   |
| 0.3783     | 0.0842                           | 0.9608                                 | 2.3174          | 0.0842                                | 0.5216                                      | 8.916318e-06  | 151   |
| 0.3776     | 0.0841                           | 0.9612                                 | 2.3097          | 0.0837                                | 0.5284                                      | 8.902785e-06  | 152   |
| 0.3770     | 0.0842                           | 0.9615                                 | 2.2495          | 0.0843                                | 0.5315                                      | 8.889185e-06  | 153   |
| 0.3769     | 0.0840                           | 0.9619                                 | 2.3789          | 0.0843                                | 0.5229                                      | 8.875517e-06  | 154   |
| 0.3764     | 0.0842                           | 0.9615                                 | 2.2922          | 0.0840                                | 0.5270                                      | 8.861781e-06  | 155   |
| 0.3721     | 0.0842                           | 0.9640                                 | 2.3240          | 0.0839                                | 0.5219                                      | 8.847978e-06  | 156   |
| 0.3723     | 0.0841                           | 0.9631                                 | 2.3228          | 0.0841                                | 0.5218                                      | 8.8341085e-06 | 157   |
| 0.3713     | 0.0841                           | 0.9638                                 | 2.3836          | 0.0843                                | 0.5319                                      | 8.820172e-06  | 158   |
| 0.3710     | 0.0841                           | 0.9640                                 | 2.5066          | 0.0839                                | 0.5179                                      | 8.806171e-06  | 159   |
| 0.3699     | 0.0841                           | 0.9645                                 | 2.3737          | 0.0840                                | 0.5202                                      | 8.7921035e-06 | 160   |
| 0.3690     | 0.0841                           | 0.9648                                 | 2.3973          | 0.0841                                | 0.5288                                      | 8.777971e-06  | 161   |
| 0.3665     | 0.0842                           | 0.9660                                 | 2.3892          | 0.0845                                | 0.5234                                      | 8.763774e-06  | 162   |
| 0.3669     | 0.0842                           | 0.9658                                 | 2.3211          | 0.0839                                | 0.5332                                      | 8.749512e-06  | 163   |
| 0.3657     | 0.0842                           | 0.9658                                 | 2.3416          | 0.0841                                | 0.5285                                      | 8.735186e-06  | 164   |
| 0.3623     | 0.0842                           | 0.9674                                 | 2.3794          | 0.0843                                | 0.5251                                      | 8.720797e-06  | 165   |
| 0.3641     | 0.0840                           | 0.9664                                 | 2.4313          | 0.0839                                | 0.5276                                      | 8.7063445e-06 | 166   |
| 0.3620     | 0.0840                           | 0.9680                                 | 2.5050          | 0.0845                                | 0.5307                                      | 8.691829e-06  | 167   |
| 0.3596     | 0.0841                           | 0.9681                                 | 2.3744          | 0.0844                                | 0.5297                                      | 8.677252e-06  | 168   |
| 0.3586     | 0.0842                           | 0.9690                                 | 2.5406          | 0.0841                                | 0.5213                                      | 8.662611e-06  | 169   |
| 0.3609     | 0.0842                           | 0.9674                                 | 2.4508          | 0.0839                                | 0.5251                                      | 8.64791e-06   | 170   |
| 0.3594     | 0.0842                           | 0.9684                                 | 2.4312          | 0.0842                                | 0.5245                                      | 8.633147e-06  | 171   |
| 0.3569     | 0.0843                           | 0.9693                                 | 2.3835          | 0.0838                                | 0.5219                                      | 8.618324e-06  | 172   |
| 0.3541     | 0.0842                           | 0.9695                                 | 2.4546          | 0.0841                                | 0.5165                                      | 8.60344e-06   | 173   |
| 0.3557     | 0.0842                           | 0.9698                                 | 2.3964          | 0.0845                                | 0.5236                                      | 8.588496e-06  | 174   |
| 0.3551     | 0.0842                           | 0.9705                                 | 2.4822          | 0.0840                                | 0.5236                                      | 8.573492e-06  | 175   |
| 0.3549     | 0.0842                           | 0.9701                                 | 2.3762          | 0.0842                                | 0.5278                                      | 8.558429e-06  | 176   |
| 0.3545     | 0.0841                           | 0.9709                                 | 2.4731          | 0.0837                                | 0.5254                                      | 8.543308e-06  | 177   |
| 0.3511     | 0.0841                           | 0.9711                                 | 2.5143          | 0.0842                                | 0.5241                                      | 8.528127e-06  | 178   |
| 0.3518     | 0.0841                           | 0.9704                                 | 2.4483          | 0.0842                                | 0.5277                                      | 8.5128895e-06 | 179   |
| 0.3504     | 0.0842                           | 0.9715                                 | 2.4476          | 0.0838                                | 0.5205                                      | 8.497594e-06  | 180   |
| 0.3494     | 0.0841                           | 0.9724                                 | 2.5549          | 0.0838                                | 0.5212                                      | 8.48224e-06   | 181   |
| 0.3513     | 0.0842                           | 0.9719                                 | 2.5223          | 0.0842                                | 0.5212                                      | 8.466831e-06  | 182   |
| 0.3491     | 0.0842                           | 0.9725                                 | 2.5944          | 0.0844                                | 0.5212                                      | 8.451365e-06  | 183   |
| 0.3466     | 0.0842                           | 0.9732                                 | 2.5346          | 0.0843                                | 0.5214                                      | 8.435843e-06  | 184   |
| 0.3496     | 0.0841                           | 0.9721                                 | 2.5568          | 0.0841                                | 0.5211                                      | 8.420265e-06  | 185   |
| 0.3454     | 0.0841                           | 0.9734                                 | 2.5095          | 0.0842                                | 0.5274                                      | 8.404632e-06  | 186   |
| 0.3464     | 0.0841                           | 0.9730                                 | 2.5433          | 0.0838                                | 0.5241                                      | 8.388945e-06  | 187   |
| 0.3475     | 0.0841                           | 0.9731                                 | 2.4579          | 0.0840                                | 0.5200                                      | 8.373204e-06  | 188   |
| 0.3453     | 0.0842                           | 0.9735                                 | 2.5508          | 0.0840                                | 0.5303                                      | 8.357409e-06  | 189   |
| 0.3435     | 0.0842                           | 0.9740                                 | 2.5990          | 0.0842                                | 0.5210                                      | 8.34156e-06   | 190   |
| 0.3428     | 0.0842                           | 0.9745                                 | 2.5726          | 0.0842                                | 0.5218                                      | 8.325658e-06  | 191   |
| 0.3414     | 0.0842                           | 0.9748                                 | 2.5876          | 0.0841                                | 0.5204                                      | 8.309704e-06  | 192   |
| 0.3428     | 0.0842                           | 0.9749                                 | 2.5714          | 0.0840                                | 0.5193                                      | 8.293697e-06  | 193   |
| 0.3417     | 0.0842                           | 0.9745                                 | 2.5984          | 0.0840                                | 0.5217                                      | 8.277638e-06  | 194   |
| 0.3428     | 0.0842                           | 0.9750                                 | 2.4623          | 0.0843                                | 0.5210                                      | 8.261528e-06  | 195   |
| 0.3408     | 0.0842                           | 0.9753                                 | 2.6072          | 0.0845                                | 0.5226                                      | 8.245367e-06  | 196   |
| 0.3412     | 0.0842                           | 0.9751                                 | 2.5461          | 0.0845                                | 0.5186                                      | 8.229155e-06  | 197   |
| 0.3381     | 0.0843                           | 0.9767                                 | 2.6322          | 0.0847                                | 0.5253                                      | 8.212894e-06  | 198   |
| 0.3389     | 0.0841                           | 0.9761                                 | 2.6758          | 0.0840                                | 0.5244                                      | 8.196583e-06  | 199   |
| 0.3384     | 0.0842                           | 0.9763                                 | 2.6191          | 0.0841                                | 0.5199                                      | 8.180223e-06  | 200   |
| 0.3364     | 0.0842                           | 0.9769                                 | 2.5552          | 0.0845                                | 0.5286                                      | 8.163814e-06  | 201   |


### Framework versions

- Transformers 4.30.0.dev0
- TensorFlow 2.9.1
- Datasets 2.8.0
- Tokenizers 0.13.2

{"tags": ["generated_from_keras_callback"], "model-index": [{"name": "Bert_1e-05", "results": []}]}
transformers tf bert pretraining generated_from_keras_callback endpoints_compatible region:us
TAGS
#transformers #tf #bert #pretraining #generated_from_keras_callback #endpoints_compatible #region-us 

Bert\_1e-05
===========


This model is a fine-tuned version of [](URL on an unknown dataset.
It achieves the following results on the evaluation set:


* Train Loss: 0.3364
* Train Prediction Logits Accuracy: 0.0842
* Train Seq Relationship Logits Accuracy: 0.9769
* Validation Loss: 2.5552
* Validation Prediction Logits Accuracy: 0.0845
* Validation Seq Relationship Logits Accuracy: 0.5286
* Train Lr: 8.163814e-06
* Epoch: 201


Model description
-----------------


More information needed


Intended uses & limitations
---------------------------


More information needed


Training and evaluation data
----------------------------


More information needed


Training procedure
------------------


### Training hyperparameters


The following hyperparameters were used during training:


* optimizer: {'name': 'Adam', 'learning\_rate': 8.147356e-06, 'decay': 0.0, 'beta\_1': 0.9, 'beta\_2': 0.999, 'epsilon': 1e-07, 'amsgrad': False}
* training\_precision: float32


### Training results



### Framework versions


* Transformers 4.30.0.dev0
* TensorFlow 2.9.1
* Datasets 2.8.0
* Tokenizers 0.13.2



### Training hyperparameters


The following hyperparameters were used during training:


* optimizer: {'name': 'Adam', 'learning\_rate': 8.147356e-06, 'decay': 0.0, 'beta\_1': 0.9, 'beta\_2': 0.999, 'epsilon': 1e-07, 'amsgrad': False}
* training\_precision: float32 ### Training results ### Framework versions


* Transformers 4.30.0.dev0
* TensorFlow 2.9.1
* Datasets 2.8.0
* Tokenizers 0.13.2
TAGS
#transformers #tf #bert #pretraining #generated_from_keras_callback #endpoints_compatible #region-us 
 ### Training hyperparameters


The following hyperparameters were used during training:


* optimizer: {'name': 'Adam', 'learning\_rate': 8.147356e-06, 'decay': 0.0, 'beta\_1': 0.9, 'beta\_2': 0.999, 'epsilon': 1e-07, 'amsgrad': False}
* training\_precision: float32 ### Training results ### Framework versions


* Transformers 4.30.0.dev0
* TensorFlow 2.9.1
* Datasets 2.8.0
* Tokenizers 0.13.2