ko-answerable / README.md
YooSungHyun
init
3b5ca91
metadata
language:
  - ko
metrics:
  - accuracy
pipeline_tag: text-classification
model-index:
  - name: ko-answerable
    results:
      - task:
          type: text-classification
          name: text-classification
        metrics:
          - type: eval_accuracy
            value: 0.892
            name: eval_accuracy
            verified: false
          - type: test_accuracy
            value: 0.837
            name: test_accuracy
            verified: false

ko-answerable: Passage์™€ Question์ด ๋‹ต๋ณ€์„ ํ•  ์ˆ˜ ์žˆ๋Š”๊ฐ€?์˜ 2์ง„ ๋ถ„๋ฅ˜

Model Details

SelfCheckGPT์˜ Answerable model์— ๊ฐ๋ช…๋ฐ›์•„ ์ œ์ž‘ํ•˜๊ฒŒ ๋˜์—ˆ์Šต๋‹ˆ๋‹ค. (https://arxiv.org/abs/2303.08896)

monologg/kobigbird-bert-base ๋ชจ๋ธ์„ ์‚ฌ์šฉํ•˜์—ฌ BigBirdForSequenceClassification ์œผ๋กœ Fine-Tune ๋˜์—ˆ์Šต๋‹ˆ๋‹ค

Max Seq Len: 4096

Input Text Style: <BOS>Question<SEP>Title<SEP>Passage<EOS>

Return: 1: ์‘๋‹ต ์—†์Œ, 0: ์‘๋‹ต ๊ฐ€๋Šฅ (sigmoid score ์‚ฌ์šฉ ๊ฐ€๋Šฅ)

์‚ฌ์šฉ๋œ ๋ฐ์ดํ„ฐ์…‹ (ํ•ด๋‹น ๋ฐ์ดํ„ฐ์…‹์—์„œ 'is_impossible'์„ ๊ธฐ์ค€์œผ๋กœ 50:50์œผ๋กœ ๋žœ๋ค ์ถ”์ถœ(0,1 ๋น„์ค‘์ด ๋งž๋„๋ก))

  1. KLUE
  2. AIHub-๋„์„œ์ž๋ฃŒ ๊ธฐ๊ณ„๋…ํ•ด
  3. AIHub-๋‰ด์Šค ๊ธฐ์‚ฌ ๊ธฐ๊ณ„๋…ํ•ด ๋ฐ์ดํ„ฐ
  4. AIHub-ํ–‰์ • ๋ฌธ์„œ ๋Œ€์ƒ ๊ธฐ๊ณ„๋…ํ•ด ๋ฐ์ดํ„ฐ
  5. ํ‘œ๊ธฐ๋ฐ˜ ์งˆ์˜์‘๋‹ต ๋ฐ์ดํ„ฐ (๋งคํŠœ๋‹˜์—๊ฒŒ ๊ฐœ์ธ์ ์œผ๋กœ ๋ฐ›์Œ)

AIHub-๊ธฐ๊ณ„๋…ํ•ด ๋ฐ์ดํ„ฐ๋„ ์กด์žฌํ•˜๋‚˜, ๋ฐ์ดํ„ฐ ์ „์ฒ˜๋ฆฌํ•˜๊ธฐ ๊ตฌ์กฐ๊ฐ€ ์ข€ ๋ณต์žกํ•˜๊ฒŒ ๋‹ฌ๋ผ์„œ ์ œ์™ธํ•จ.

์˜ˆ์ธก ์‹œ๊ฐ„: ๊ฑด๋‹น ํ‰๊ท  0.05์ดˆ ์ด๋‚ด (RTX 3090 ์‚ฌ์šฉ)

์‚ฌ์šฉ GPU MEM: About 20GB (Seq๊ฐ€ ๊ธธ๋ฉด ๋งŽ์ด ๋จน์Œ)