
# Korean GPT Bot Sentiment Classification (ko-gpt-bot-sc)

### Method
 - Promt-Tuning/Prefix-tuning/Soft Embedding
 - Parameters
| Parameters | No.                 |
|------------|---------------------|
| All        | 6173039616 (100.0%) |
| Trainable  | 6537216 (0.1%)      |
| Freezed    | 6166502400 (99.9%)  |

<img src="https://huggingface.co/anhdungitvn/ko-gpt-bot-sc-7b/resolve/main/metrics/prompt_tuning.png"  width="800">


### Model
```
LAYER NAME                   	        #PARAMS	     RATIO	 MEM(MB)
--model:                     	  6,177,233,921	   100.00%	23552.28
  --learned_embedding:       	      6,537,216	     0.11%	   24.94
  --transformer:             	  5,906,391,041	    95.62%	22519.09
    --wte                    
      --weight:              	    264,241,152	     4.28%	 1008.00
    --h:                     	  5,642,141,697	    91.34%	21511.06
      --0:                   	    205,549,569	     3.33%	  772.11
        --ln_1:              	          8,192	     0.00%	    0.03
        --attn:              	     71,303,169	     1.15%	  260.00
        --mlp:               	    134,238,208	     2.17%	  512.08
      --1(partially shared): 	    201,355,264	     3.26%	  768.11
        --ln_1:              	          8,192	     0.00%	    0.03
        --attn(shared):      	     67,108,864	     1.09%	  256.00
        --mlp:               	    134,238,208	     2.17%	  512.08
      --2(partially shared): 	    201,355,264	     3.26%	  768.11
        --ln_1:              	          8,192	     0.00%	    0.03
        --attn(shared):      	     67,108,864	     1.09%	  256.00
        --mlp:               	    134,238,208	     2.17%	  512.08
      --3(partially shared): 	    201,355,264	     3.26%	  768.11
        --ln_1:              	          8,192	     0.00%	    0.03
        --attn(shared):      	     67,108,864	     1.09%	  256.00
        --mlp:               	    134,238,208	     2.17%	  512.08
      --4(partially shared): 	    201,355,264	     3.26%	  768.11
        --ln_1:              	          8,192	     0.00%	    0.03
        --attn(shared):      	     67,108,864	     1.09%	  256.00
        --mlp:               	    134,238,208	     2.17%	  512.08
      --5(partially shared): 	    201,355,264	     3.26%	  768.11
        --ln_1:              	          8,192	     0.00%	    0.03
        --attn(shared):      	     67,108,864	     1.09%	  256.00
        --mlp:               	    134,238,208	     2.17%	  512.08
      --6(partially shared): 	    201,355,264	     3.26%	  768.11
        --ln_1:              	          8,192	     0.00%	    0.03
        --attn(shared):      	     67,108,864	     1.09%	  256.00
        --mlp:               	    134,238,208	     2.17%	  512.08
      --7(partially shared): 	    201,355,264	     3.26%	  768.11
        --ln_1:              	          8,192	     0.00%	    0.03
        --attn(shared):      	     67,108,864	     1.09%	  256.00
        --mlp:               	    134,238,208	     2.17%	  512.08
      --8(partially shared): 	    201,355,264	     3.26%	  768.11
        --ln_1:              	          8,192	     0.00%	    0.03
        --attn(shared):      	     67,108,864	     1.09%	  256.00
        --mlp:               	    134,238,208	     2.17%	  512.08
      --9(partially shared): 	    201,355,264	     3.26%	  768.11
        --ln_1:              	          8,192	     0.00%	    0.03
        --attn(shared):      	     67,108,864	     1.09%	  256.00
        --mlp:               	    134,238,208	     2.17%	  512.08
      --10(partially shared):	    201,355,264	     3.26%	  768.11
        --ln_1:              	          8,192	     0.00%	    0.03
        --attn(shared):      	     67,108,864	     1.09%	  256.00
        --mlp:               	    134,238,208	     2.17%	  512.08
      --11(partially shared):	    201,355,264	     3.26%	  768.11
        --ln_1:              	          8,192	     0.00%	    0.03
        --attn(shared):      	     67,108,864	     1.09%	  256.00
        --mlp:               	    134,238,208	     2.17%	  512.08
      --12(partially shared):	    201,355,264	     3.26%	  768.11
        --ln_1:              	          8,192	     0.00%	    0.03
        --attn(shared):      	     67,108,864	     1.09%	  256.00
        --mlp:               	    134,238,208	     2.17%	  512.08
      --13(partially shared):	    201,355,264	     3.26%	  768.11
        --ln_1:              	          8,192	     0.00%	    0.03
        --attn(shared):      	     67,108,864	     1.09%	  256.00
        --mlp:               	    134,238,208	     2.17%	  512.08
      --14(partially shared):	    201,355,264	     3.26%	  768.11
        --ln_1:              	          8,192	     0.00%	    0.03
        --attn(shared):      	     67,108,864	     1.09%	  256.00
        --mlp:               	    134,238,208	     2.17%	  512.08
      --15(partially shared):	    201,355,264	     3.26%	  768.11
        --ln_1:              	          8,192	     0.00%	    0.03
        --attn(shared):      	     67,108,864	     1.09%	  256.00
        --mlp:               	    134,238,208	     2.17%	  512.08
      --16(partially shared):	    201,355,264	     3.26%	  768.11
        --ln_1:              	          8,192	     0.00%	    0.03
        --attn(shared):      	     67,108,864	     1.09%	  256.00
        --mlp:               	    134,238,208	     2.17%	  512.08
      --17(partially shared):	    201,355,264	     3.26%	  768.11
        --ln_1:              	          8,192	     0.00%	    0.03
        --attn(shared):      	     67,108,864	     1.09%	  256.00
        --mlp:               	    134,238,208	     2.17%	  512.08
      --18(partially shared):	    201,355,264	     3.26%	  768.11
        --ln_1:              	          8,192	     0.00%	    0.03
        --attn(shared):      	     67,108,864	     1.09%	  256.00
        --mlp:               	    134,238,208	     2.17%	  512.08
      --19(partially shared):	    201,355,264	     3.26%	  768.11
        --ln_1:              	          8,192	     0.00%	    0.03
        --attn(shared):      	     67,108,864	     1.09%	  256.00
        --mlp:               	    134,238,208	     2.17%	  512.08
      --20(partially shared):	    201,355,264	     3.26%	  768.11
        --ln_1:              	          8,192	     0.00%	    0.03
        --attn(shared):      	     67,108,864	     1.09%	  256.00
        --mlp:               	    134,238,208	     2.17%	  512.08
      --21(partially shared):	    201,355,264	     3.26%	  768.11
        --ln_1:              	          8,192	     0.00%	    0.03
        --attn(shared):      	     67,108,864	     1.09%	  256.00
        --mlp:               	    134,238,208	     2.17%	  512.08
      --22(partially shared):	    201,355,264	     3.26%	  768.11
        --ln_1:              	          8,192	     0.00%	    0.03
        --attn(shared):      	     67,108,864	     1.09%	  256.00
        --mlp:               	    134,238,208	     2.17%	  512.08
      --23(partially shared):	    201,355,264	     3.26%	  768.11
        --ln_1:              	          8,192	     0.00%	    0.03
        --attn(shared):      	     67,108,864	     1.09%	  256.00
        --mlp:               	    134,238,208	     2.17%	  512.08
      --24(partially shared):	    201,355,264	     3.26%	  768.11
        --ln_1:              	          8,192	     0.00%	    0.03
        --attn(shared):      	     67,108,864	     1.09%	  256.00
        --mlp:               	    134,238,208	     2.17%	  512.08
      --25(partially shared):	    201,355,264	     3.26%	  768.11
        --ln_1:              	          8,192	     0.00%	    0.03
        --attn(shared):      	     67,108,864	     1.09%	  256.00
        --mlp:               	    134,238,208	     2.17%	  512.08
      --26(partially shared):	    201,355,264	     3.26%	  768.11
        --ln_1:              	          8,192	     0.00%	    0.03
        --attn(shared):      	     67,108,864	     1.09%	  256.00
        --mlp:               	    134,238,208	     2.17%	  512.08
      --27(partially shared):	    201,355,264	     3.26%	  768.11
        --ln_1:              	          8,192	     0.00%	    0.03
        --attn(shared):      	     67,108,864	     1.09%	  256.00
        --mlp:               	    134,238,208	     2.17%	  512.08
    --ln_f:                  	          8,192	     0.00%	    0.03
      --weight:              	          4,096	     0.00%	    0.02
      --bias:                	          4,096	     0.00%	    0.02
  --lm_head:                 	    264,305,664	     4.28%	 1008.25
    --weight:                	    264,241,152	     4.28%	 1008.00
    --bias:                  	         64,512	     0.00%	    0.25
```

### Metrics

| Metric | Value  |
|--------|--------|
| step   | 520    |
| loss   | 3.1814 |

|              | precision | recall | f1-score | support |
|--------------|-----------|--------|----------|---------|
| 긍정         | 0.92549   | 0.944  | 0.934653 | 500     |
| 부정         | 0.942857  | 0.924  | 0.933333 | 500     |
| accuracy     | 0.934     | 0.934  | 0.934    | 0.934   |
| macro avg    | 0.934174  | 0.934  | 0.933993 | 1000    |
| weighted avg | 0.934174  | 0.934  | 0.933993 | 1000    |


<img src="https://huggingface.co/anhdungitvn/ko-gpt-bot-sc/resolve/main/metrics/loss.png"  width="800">
<img src="https://huggingface.co/anhdungitvn/ko-gpt-bot-sc/resolve/main/metrics/labels_preds.png"  width="800">
<img src="https://huggingface.co/anhdungitvn/ko-gpt-bot-sc/resolve/main/metrics/test_dataset_preds-01.png"  width="800">



### References
 - Prompt Tuning: <a href="https://arxiv.org/abs/2104.08691" download>**The Power of Scale for Parameter-Efficient Prompt Tuning**</a>
 - Prompt Tuning v2: <a href="https://arxiv.org/abs/2110.07602" download>**P-Tuning v2: Prompt Tuning Can Be Comparable to Fine-tuning Universally Across Scales and Tasks**</a>
{"language": ["ko"], "license": "other", "datasets": ["nsmc"]}
ko dataset:nsmc arxiv:2104.08691 arxiv:2110.07602 license:other region:us
TAGS
#ko #dataset-nsmc #arxiv-2104.08691 #arxiv-2110.07602 #license-other #region-us 

Korean GPT Bot Sentiment Classification (ko-gpt-bot-sc)
=======================================================


### Method


* Promt-Tuning/Prefix-tuning/Soft Embedding
* Parameters
| Parameters | No. |
|------------|---------------------|
| All | 6173039616 (100.0%) |
| Trainable | 6537216 (0.1%) |
| Freezed | 6166502400 (99.9%) |


<img src="URL width="800">


### Model


### Metrics




<img src="URL width="800">
<img src="URL width="800">
<img src="URL width="800">


### References


* Prompt Tuning: <a href="URL download>The Power of Scale for Parameter-Efficient Prompt Tuning
* Prompt Tuning v2: <a href="URL download>P-Tuning v2: Prompt Tuning Can Be Comparable to Fine-tuning Universally Across Scales and Tasks



### Method


* Promt-Tuning/Prefix-tuning/Soft Embedding
* Parameters
| Parameters | No. |
|------------|---------------------|
| All | 6173039616 (100.0%) |
| Trainable | 6537216 (0.1%) |
| Freezed | 6166502400 (99.9%) |


<img src="URL width="800"> ### Model ### Metrics




<img src="URL width="800">
<img src="URL width="800">
<img src="URL width="800"> ### References


* Prompt Tuning: <a href="URL download>The Power of Scale for Parameter-Efficient Prompt Tuning
* Prompt Tuning v2: <a href="URL download>P-Tuning v2: Prompt Tuning Can Be Comparable to Fine-tuning Universally Across Scales and Tasks
TAGS
#ko #dataset-nsmc #arxiv-2104.08691 #arxiv-2110.07602 #license-other #region-us 
 ### Method


* Promt-Tuning/Prefix-tuning/Soft Embedding
* Parameters
| Parameters | No. |
|------------|---------------------|
| All | 6173039616 (100.0%) |
| Trainable | 6537216 (0.1%) |
| Freezed | 6166502400 (99.9%) |


<img src="URL width="800"> ### Model ### Metrics




<img src="URL width="800">
<img src="URL width="800">
<img src="URL width="800"> ### References


* Prompt Tuning: <a href="URL download>The Power of Scale for Parameter-Efficient Prompt Tuning
* Prompt Tuning v2: <a href="URL download>P-Tuning v2: Prompt Tuning Can Be Comparable to Fine-tuning Universally Across Scales and Tasks