legal_deberta / README.md

update model card README.md

ad4b9f3 3 months ago

23.1 kB

	---
	license: mit
	base_model: microsoft/deberta-v3-base
	tags:
	- generated_from_trainer
	model-index:
	- name: legal_deberta
	results: []
	---

	<!-- This model card has been generated automatically according to the information the Trainer had access to. You
	should probably proofread and complete it, then remove this comment. -->

	# legal_deberta

	This model is a fine-tuned version of [microsoft/deberta-v3-base](https://huggingface.co/microsoft/deberta-v3-base) on an unknown dataset.
	It achieves the following results on the evaluation set:
	- Loss: 0.3745
	- Law Precision: 0.8118
	- Law Recall: 0.92
	- Law F1: 0.8625
	- Law Number: 75
	- Violated by Precision: 0.8434
	- Violated by Recall: 0.9333
	- Violated by F1: 0.8861
	- Violated by Number: 75
	- Violated on Precision: 0.7532
	- Violated on Recall: 0.7733
	- Violated on F1: 0.7632
	- Violated on Number: 75
	- Violation Precision: 0.5768
	- Violation Recall: 0.6705
	- Violation F1: 0.6201
	- Violation Number: 616
	- Overall Precision: 0.6348
	- Overall Recall: 0.7253
	- Overall F1: 0.6770
	- Overall Accuracy: 0.9495

	## Model description

	More information needed

	## Intended uses & limitations

	More information needed

	## Training and evaluation data

	More information needed

	## Training procedure

	### Training hyperparameters

	The following hyperparameters were used during training:
	- learning_rate: 5e-05
	- train_batch_size: 16
	- eval_batch_size: 16
	- seed: 42
	- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
	- lr_scheduler_type: linear
	- lr_scheduler_warmup_steps: 500
	- num_epochs: 50

	### Training results

	\| Training Loss \| Epoch \| Step \| Validation Loss \| Law Precision \| Law Recall \| Law F1 \| Law Number \| Violated by Precision \| Violated by Recall \| Violated by F1 \| Violated by Number \| Violated on Precision \| Violated on Recall \| Violated on F1 \| Violated on Number \| Violation Precision \| Violation Recall \| Violation F1 \| Violation Number \| Overall Precision \| Overall Recall \| Overall F1 \| Overall Accuracy \|
	\|:-------------:\|:-----:\|:----:\|:---------------:\|:-------------:\|:----------:\|:------:\|:----------:\|:---------------------:\|:------------------:\|:--------------:\|:------------------:\|:---------------------:\|:------------------:\|:--------------:\|:------------------:\|:-------------------:\|:----------------:\|:------------:\|:----------------:\|:-----------------:\|:--------------:\|:----------:\|:----------------:\|
	\| No log \| 1.0 \| 45 \| 0.8353 \| 0.0 \| 0.0 \| 0.0 \| 75 \| 0.0 \| 0.0 \| 0.0 \| 75 \| 0.0 \| 0.0 \| 0.0 \| 75 \| 0.0 \| 0.0 \| 0.0 \| 616 \| 0.0 \| 0.0 \| 0.0 \| 0.8060 \|
	\| No log \| 2.0 \| 90 \| 0.3620 \| 0.0 \| 0.0 \| 0.0 \| 75 \| 0.0 \| 0.0 \| 0.0 \| 75 \| 0.0 \| 0.0 \| 0.0 \| 75 \| 0.1980 \| 0.2873 \| 0.2344 \| 616 \| 0.1928 \| 0.2105 \| 0.2013 \| 0.8923 \|
	\| No log \| 3.0 \| 135 \| 0.2408 \| 0.6582 \| 0.6933 \| 0.6753 \| 75 \| 0.6923 \| 0.12 \| 0.2045 \| 75 \| 0.0 \| 0.0 \| 0.0 \| 75 \| 0.3602 \| 0.4432 \| 0.3974 \| 616 \| 0.3920 \| 0.3971 \| 0.3946 \| 0.9174 \|
	\| No log \| 4.0 \| 180 \| 0.2258 \| 0.7229 \| 0.8 \| 0.7595 \| 75 \| 0.8788 \| 0.7733 \| 0.8227 \| 75 \| 0.3889 \| 0.4667 \| 0.4242 \| 75 \| 0.4976 \| 0.5049 \| 0.5012 \| 616 \| 0.5370 \| 0.5517 \| 0.5443 \| 0.9346 \|
	\| No log \| 5.0 \| 225 \| 0.1876 \| 0.6915 \| 0.8667 \| 0.7692 \| 75 \| 0.8406 \| 0.7733 \| 0.8056 \| 75 \| 0.4409 \| 0.5467 \| 0.4881 \| 75 \| 0.4716 \| 0.5942 \| 0.5259 \| 616 \| 0.5136 \| 0.6302 \| 0.5659 \| 0.9439 \|
	\| No log \| 6.0 \| 270 \| 0.2125 \| 0.6635 \| 0.92 \| 0.7709 \| 75 \| 0.7976 \| 0.8933 \| 0.8428 \| 75 \| 0.4842 \| 0.6133 \| 0.5412 \| 75 \| 0.4460 \| 0.5763 \| 0.5028 \| 616 \| 0.4977 \| 0.6385 \| 0.5594 \| 0.9429 \|
	\| No log \| 7.0 \| 315 \| 0.2972 \| 0.6091 \| 0.8933 \| 0.7243 \| 75 \| 0.7674 \| 0.88 \| 0.8199 \| 75 \| 0.5517 \| 0.64 \| 0.5926 \| 75 \| 0.4647 \| 0.5455 \| 0.5019 \| 616 \| 0.5139 \| 0.6147 \| 0.5598 \| 0.9397 \|
	\| No log \| 8.0 \| 360 \| 0.2361 \| 0.8333 \| 0.8667 \| 0.8497 \| 75 \| 0.8831 \| 0.9067 \| 0.8947 \| 75 \| 0.5889 \| 0.7067 \| 0.6424 \| 75 \| 0.5013 \| 0.6169 \| 0.5531 \| 616 \| 0.5643 \| 0.6730 \| 0.6139 \| 0.9472 \|
	\| No log \| 9.0 \| 405 \| 0.2956 \| 0.5156 \| 0.88 \| 0.6502 \| 75 \| 0.7263 \| 0.92 \| 0.8118 \| 75 \| 0.5096 \| 0.7067 \| 0.5922 \| 75 \| 0.5086 \| 0.5731 \| 0.5389 \| 616 \| 0.5299 \| 0.6433 \| 0.5811 \| 0.9391 \|
	\| No log \| 10.0 \| 450 \| 0.2722 \| 0.7097 \| 0.88 \| 0.7857 \| 75 \| 0.9079 \| 0.92 \| 0.9139 \| 75 \| 0.6049 \| 0.6533 \| 0.6282 \| 75 \| 0.5157 \| 0.6396 \| 0.5710 \| 616 \| 0.5700 \| 0.6873 \| 0.6232 \| 0.9431 \|
	\| No log \| 11.0 \| 495 \| 0.3221 \| 0.7234 \| 0.9067 \| 0.8047 \| 75 \| 0.8919 \| 0.88 \| 0.8859 \| 75 \| 0.7105 \| 0.72 \| 0.7152 \| 75 \| 0.5030 \| 0.5422 \| 0.5219 \| 616 \| 0.5749 \| 0.6207 \| 0.5969 \| 0.9374 \|
	\| 0.271 \| 12.0 \| 540 \| 0.3475 \| 0.6538 \| 0.9067 \| 0.7598 \| 75 \| 0.6542 \| 0.9333 \| 0.7692 \| 75 \| 0.4831 \| 0.76 \| 0.5907 \| 75 \| 0.5141 \| 0.5341 \| 0.5239 \| 616 \| 0.5408 \| 0.6231 \| 0.5790 \| 0.9344 \|
	\| 0.271 \| 13.0 \| 585 \| 0.2667 \| 0.7128 \| 0.8933 \| 0.7929 \| 75 \| 0.6 \| 0.92 \| 0.7263 \| 75 \| 0.5824 \| 0.7067 \| 0.6386 \| 75 \| 0.5530 \| 0.6526 \| 0.5987 \| 616 \| 0.5755 \| 0.7027 \| 0.6328 \| 0.9484 \|
	\| 0.271 \| 14.0 \| 630 \| 0.2868 \| 0.7113 \| 0.92 \| 0.8023 \| 75 \| 0.8642 \| 0.9333 \| 0.8974 \| 75 \| 0.6706 \| 0.76 \| 0.7125 \| 75 \| 0.5393 \| 0.6347 \| 0.5831 \| 616 \| 0.5941 \| 0.6980 \| 0.6419 \| 0.9508 \|
	\| 0.271 \| 15.0 \| 675 \| 0.3035 \| 0.7419 \| 0.92 \| 0.8214 \| 75 \| 0.8537 \| 0.9333 \| 0.8917 \| 75 \| 0.6471 \| 0.7333 \| 0.6875 \| 75 \| 0.5681 \| 0.6769 \| 0.6178 \| 616 \| 0.6147 \| 0.7265 \| 0.6659 \| 0.9475 \|
	\| 0.271 \| 16.0 \| 720 \| 0.2746 \| 0.7053 \| 0.8933 \| 0.7882 \| 75 \| 0.8861 \| 0.9333 \| 0.9091 \| 75 \| 0.6875 \| 0.7333 \| 0.7097 \| 75 \| 0.5654 \| 0.6737 \| 0.6148 \| 616 \| 0.6144 \| 0.7218 \| 0.6638 \| 0.9523 \|
	\| 0.271 \| 17.0 \| 765 \| 0.2846 \| 0.7907 \| 0.9067 \| 0.8447 \| 75 \| 0.9221 \| 0.9467 \| 0.9342 \| 75 \| 0.6709 \| 0.7067 \| 0.6883 \| 75 \| 0.5904 \| 0.6623 \| 0.6243 \| 616 \| 0.6431 \| 0.7134 \| 0.6764 \| 0.9506 \|
	\| 0.271 \| 18.0 \| 810 \| 0.3004 \| 0.75 \| 0.92 \| 0.8263 \| 75 \| 0.8434 \| 0.9333 \| 0.8861 \| 75 \| 0.6747 \| 0.7467 \| 0.7089 \| 75 \| 0.5501 \| 0.6769 \| 0.6070 \| 616 \| 0.6024 \| 0.7277 \| 0.6591 \| 0.9464 \|
	\| 0.271 \| 19.0 \| 855 \| 0.3235 \| 0.6832 \| 0.92 \| 0.7841 \| 75 \| 0.8214 \| 0.92 \| 0.8679 \| 75 \| 0.6667 \| 0.7733 \| 0.7160 \| 75 \| 0.5451 \| 0.6867 \| 0.6078 \| 616 \| 0.5906 \| 0.7360 \| 0.6554 \| 0.9482 \|
	\| 0.271 \| 20.0 \| 900 \| 0.3274 \| 0.7391 \| 0.9067 \| 0.8144 \| 75 \| 0.8861 \| 0.9333 \| 0.9091 \| 75 \| 0.6835 \| 0.72 \| 0.7013 \| 75 \| 0.5679 \| 0.6786 \| 0.6183 \| 616 \| 0.6187 \| 0.7253 \| 0.6678 \| 0.9474 \|
	\| 0.271 \| 21.0 \| 945 \| 0.3756 \| 0.7882 \| 0.8933 \| 0.8375 \| 75 \| 0.8537 \| 0.9333 \| 0.8917 \| 75 \| 0.6962 \| 0.7333 \| 0.7143 \| 75 \| 0.6046 \| 0.6429 \| 0.6231 \| 616 \| 0.6526 \| 0.6992 \| 0.6751 \| 0.9468 \|
	\| 0.271 \| 22.0 \| 990 \| 0.3511 \| 0.8072 \| 0.8933 \| 0.8481 \| 75 \| 0.8974 \| 0.9333 \| 0.9150 \| 75 \| 0.7215 \| 0.76 \| 0.7403 \| 75 \| 0.5813 \| 0.6445 \| 0.6112 \| 616 \| 0.6403 \| 0.7027 \| 0.6701 \| 0.9460 \|
	\| 0.007 \| 23.0 \| 1035 \| 0.3187 \| 0.8375 \| 0.8933 \| 0.8645 \| 75 \| 0.9079 \| 0.92 \| 0.9139 \| 75 \| 0.7237 \| 0.7333 \| 0.7285 \| 75 \| 0.5816 \| 0.6656 \| 0.6207 \| 616 \| 0.6414 \| 0.7146 \| 0.6760 \| 0.9507 \|
	\| 0.007 \| 24.0 \| 1080 \| 0.3383 \| 0.8171 \| 0.8933 \| 0.8535 \| 75 \| 0.8118 \| 0.92 \| 0.8625 \| 75 \| 0.6747 \| 0.7467 \| 0.7089 \| 75 \| 0.5727 \| 0.6331 \| 0.6014 \| 616 \| 0.6251 \| 0.6920 \| 0.6569 \| 0.9473 \|
	\| 0.007 \| 25.0 \| 1125 \| 0.3231 \| 0.7952 \| 0.88 \| 0.8354 \| 75 \| 0.8434 \| 0.9333 \| 0.8861 \| 75 \| 0.6667 \| 0.72 \| 0.6923 \| 75 \| 0.5524 \| 0.6851 \| 0.6116 \| 616 \| 0.6053 \| 0.7277 \| 0.6609 \| 0.9483 \|
	\| 0.007 \| 26.0 \| 1170 \| 0.3099 \| 0.7033 \| 0.8533 \| 0.7711 \| 75 \| 0.8642 \| 0.9333 \| 0.8974 \| 75 \| 0.7143 \| 0.7333 \| 0.7237 \| 75 \| 0.5722 \| 0.6753 \| 0.6195 \| 616 \| 0.6199 \| 0.7194 \| 0.6659 \| 0.9509 \|
	\| 0.007 \| 27.0 \| 1215 \| 0.3202 \| 0.7701 \| 0.8933 \| 0.8272 \| 75 \| 0.8961 \| 0.92 \| 0.9079 \| 75 \| 0.6582 \| 0.6933 \| 0.6753 \| 75 \| 0.5977 \| 0.6705 \| 0.6320 \| 616 \| 0.6435 \| 0.7146 \| 0.6772 \| 0.9509 \|
	\| 0.007 \| 28.0 \| 1260 \| 0.3381 \| 0.7263 \| 0.92 \| 0.8118 \| 75 \| 0.7931 \| 0.92 \| 0.8519 \| 75 \| 0.7 \| 0.7467 \| 0.7226 \| 75 \| 0.5909 \| 0.6753 \| 0.6303 \| 616 \| 0.6315 \| 0.7253 \| 0.6752 \| 0.9496 \|
	\| 0.007 \| 29.0 \| 1305 \| 0.3413 \| 0.7841 \| 0.92 \| 0.8466 \| 75 \| 0.7955 \| 0.9333 \| 0.8589 \| 75 \| 0.6463 \| 0.7067 \| 0.6752 \| 75 \| 0.5895 \| 0.6737 \| 0.6288 \| 616 \| 0.6310 \| 0.7218 \| 0.6733 \| 0.95 \|
	\| 0.007 \| 30.0 \| 1350 \| 0.3427 \| 0.8023 \| 0.92 \| 0.8571 \| 75 \| 0.8961 \| 0.92 \| 0.9079 \| 75 \| 0.6962 \| 0.7333 \| 0.7143 \| 75 \| 0.5974 \| 0.6721 \| 0.6325 \| 616 \| 0.6492 \| 0.7218 \| 0.6836 \| 0.9508 \|
	\| 0.007 \| 31.0 \| 1395 \| 0.3473 \| 0.8023 \| 0.92 \| 0.8571 \| 75 \| 0.8861 \| 0.9333 \| 0.9091 \| 75 \| 0.7143 \| 0.7333 \| 0.7237 \| 75 \| 0.5738 \| 0.6623 \| 0.6149 \| 616 \| 0.6317 \| 0.7158 \| 0.6711 \| 0.9493 \|
	\| 0.007 \| 32.0 \| 1440 \| 0.3531 \| 0.7188 \| 0.92 \| 0.8070 \| 75 \| 0.8140 \| 0.9333 \| 0.8696 \| 75 \| 0.7195 \| 0.7867 \| 0.7516 \| 75 \| 0.5740 \| 0.6672 \| 0.6171 \| 616 \| 0.6214 \| 0.7241 \| 0.6689 \| 0.9495 \|
	\| 0.007 \| 33.0 \| 1485 \| 0.3556 \| 0.7263 \| 0.92 \| 0.8118 \| 75 \| 0.8140 \| 0.9333 \| 0.8696 \| 75 \| 0.7108 \| 0.7867 \| 0.7468 \| 75 \| 0.5783 \| 0.6656 \| 0.6189 \| 616 \| 0.6249 \| 0.7229 \| 0.6703 \| 0.9492 \|
	\| 0.0009 \| 34.0 \| 1530 \| 0.3569 \| 0.7667 \| 0.92 \| 0.8364 \| 75 \| 0.8140 \| 0.9333 \| 0.8696 \| 75 \| 0.7215 \| 0.76 \| 0.7403 \| 75 \| 0.5783 \| 0.6656 \| 0.6189 \| 616 \| 0.6286 \| 0.7206 \| 0.6715 \| 0.9493 \|
	\| 0.0009 \| 35.0 \| 1575 \| 0.3630 \| 0.7841 \| 0.92 \| 0.8466 \| 75 \| 0.8140 \| 0.9333 \| 0.8696 \| 75 \| 0.7 \| 0.7467 \| 0.7226 \| 75 \| 0.5838 \| 0.6672 \| 0.6227 \| 616 \| 0.6326 \| 0.7206 \| 0.6737 \| 0.9489 \|
	\| 0.0009 \| 36.0 \| 1620 \| 0.3624 \| 0.7667 \| 0.92 \| 0.8364 \| 75 \| 0.8140 \| 0.9333 \| 0.8696 \| 75 \| 0.7179 \| 0.7467 \| 0.7320 \| 75 \| 0.5782 \| 0.6721 \| 0.6216 \| 616 \| 0.6278 \| 0.7241 \| 0.6726 \| 0.9493 \|
	\| 0.0009 \| 37.0 \| 1665 \| 0.3614 \| 0.7667 \| 0.92 \| 0.8364 \| 75 \| 0.8140 \| 0.9333 \| 0.8696 \| 75 \| 0.7403 \| 0.76 \| 0.75 \| 75 \| 0.5744 \| 0.6705 \| 0.6187 \| 616 \| 0.6265 \| 0.7241 \| 0.6718 \| 0.9493 \|
	\| 0.0009 \| 38.0 \| 1710 \| 0.3630 \| 0.7841 \| 0.92 \| 0.8466 \| 75 \| 0.8235 \| 0.9333 \| 0.8750 \| 75 \| 0.7662 \| 0.7867 \| 0.7763 \| 75 \| 0.5842 \| 0.6705 \| 0.6243 \| 616 \| 0.6385 \| 0.7265 \| 0.6796 \| 0.9496 \|
	\| 0.0009 \| 39.0 \| 1755 \| 0.3645 \| 0.8118 \| 0.92 \| 0.8625 \| 75 \| 0.8235 \| 0.9333 \| 0.8750 \| 75 \| 0.7662 \| 0.7867 \| 0.7763 \| 75 \| 0.5842 \| 0.6705 \| 0.6243 \| 616 \| 0.6405 \| 0.7265 \| 0.6808 \| 0.9499 \|
	\| 0.0009 \| 40.0 \| 1800 \| 0.3670 \| 0.8313 \| 0.92 \| 0.8734 \| 75 \| 0.8333 \| 0.9333 \| 0.8805 \| 75 \| 0.7632 \| 0.7733 \| 0.7682 \| 75 \| 0.5784 \| 0.6705 \| 0.6211 \| 616 \| 0.6374 \| 0.7253 \| 0.6785 \| 0.9499 \|
	\| 0.0009 \| 41.0 \| 1845 \| 0.3693 \| 0.8214 \| 0.92 \| 0.8679 \| 75 \| 0.8333 \| 0.9333 \| 0.8805 \| 75 \| 0.7632 \| 0.7733 \| 0.7682 \| 75 \| 0.5831 \| 0.6721 \| 0.6244 \| 616 \| 0.6405 \| 0.7265 \| 0.6808 \| 0.9497 \|
	\| 0.0009 \| 42.0 \| 1890 \| 0.3727 \| 0.7931 \| 0.92 \| 0.8519 \| 75 \| 0.8235 \| 0.9333 \| 0.8750 \| 75 \| 0.7436 \| 0.7733 \| 0.7582 \| 75 \| 0.5946 \| 0.6737 \| 0.6317 \| 616 \| 0.6456 \| 0.7277 \| 0.6842 \| 0.9491 \|
	\| 0.0009 \| 43.0 \| 1935 \| 0.3739 \| 0.8023 \| 0.92 \| 0.8571 \| 75 \| 0.8235 \| 0.9333 \| 0.8750 \| 75 \| 0.7160 \| 0.7733 \| 0.7436 \| 75 \| 0.5968 \| 0.6753 \| 0.6337 \| 616 \| 0.6459 \| 0.7289 \| 0.6849 \| 0.9490 \|
	\| 0.0009 \| 44.0 \| 1980 \| 0.3750 \| 0.8023 \| 0.92 \| 0.8571 \| 75 \| 0.8235 \| 0.9333 \| 0.8750 \| 75 \| 0.7 \| 0.7467 \| 0.7226 \| 75 \| 0.5968 \| 0.6753 \| 0.6337 \| 616 \| 0.6445 \| 0.7265 \| 0.6831 \| 0.9490 \|
	\| 0.0001 \| 45.0 \| 2025 \| 0.3753 \| 0.8023 \| 0.92 \| 0.8571 \| 75 \| 0.8235 \| 0.9333 \| 0.8750 \| 75 \| 0.7160 \| 0.7733 \| 0.7436 \| 75 \| 0.5932 \| 0.6769 \| 0.6323 \| 616 \| 0.6429 \| 0.7301 \| 0.6837 \| 0.9490 \|
	\| 0.0001 \| 46.0 \| 2070 \| 0.3755 \| 0.8023 \| 0.92 \| 0.8571 \| 75 \| 0.8235 \| 0.9333 \| 0.8750 \| 75 \| 0.7342 \| 0.7733 \| 0.7532 \| 75 \| 0.5912 \| 0.6786 \| 0.6319 \| 616 \| 0.6426 \| 0.7313 \| 0.6841 \| 0.9492 \|
	\| 0.0001 \| 47.0 \| 2115 \| 0.3759 \| 0.8023 \| 0.92 \| 0.8571 \| 75 \| 0.8235 \| 0.9333 \| 0.8750 \| 75 \| 0.7342 \| 0.7733 \| 0.7532 \| 75 \| 0.5896 \| 0.6786 \| 0.6309 \| 616 \| 0.6413 \| 0.7313 \| 0.6833 \| 0.9492 \|
	\| 0.0001 \| 48.0 \| 2160 \| 0.3742 \| 0.8118 \| 0.92 \| 0.8625 \| 75 \| 0.8434 \| 0.9333 \| 0.8861 \| 75 \| 0.7532 \| 0.7733 \| 0.7632 \| 75 \| 0.5770 \| 0.6688 \| 0.6195 \| 616 \| 0.6350 \| 0.7241 \| 0.6767 \| 0.9493 \|
	\| 0.0001 \| 49.0 \| 2205 \| 0.3744 \| 0.8118 \| 0.92 \| 0.8625 \| 75 \| 0.8434 \| 0.9333 \| 0.8861 \| 75 \| 0.7532 \| 0.7733 \| 0.7632 \| 75 \| 0.5760 \| 0.6705 \| 0.6197 \| 616 \| 0.6341 \| 0.7253 \| 0.6767 \| 0.9495 \|
	\| 0.0001 \| 50.0 \| 2250 \| 0.3745 \| 0.8118 \| 0.92 \| 0.8625 \| 75 \| 0.8434 \| 0.9333 \| 0.8861 \| 75 \| 0.7532 \| 0.7733 \| 0.7632 \| 75 \| 0.5768 \| 0.6705 \| 0.6201 \| 616 \| 0.6348 \| 0.7253 \| 0.6770 \| 0.9495 \|


	### Framework versions

	- Transformers 4.31.0
	- Pytorch 2.4.0+cu121
	- Datasets 2.15.0
	- Tokenizers 0.13.3