README.md

metadata

library_name: transformers
license: mit
datasets:
  - Jinyan1/COLING_2025_MGT_en
language:
  - en
metrics:
  - f1
base_model:
  - microsoft/deberta-v3-base

The solution for COLING 2025 Workshop on Detecting AI Generated Content (DAIGenC)

The model that won first place in the competition in the task-1A.

Get Usage

import torch
from transformers import AutoTokenizer, DebertaV2ForSequenceClassification

class MLayerDebertaV2ForSequenceClassification(
    DebertaV2ForSequenceClassification
):
    def __init__(self, config, **kwargs):
        super().__init__(config)
        self.classifier = torch.nn.Sequential(
            torch.nn.Linear(config.hidden_size, 512),
            torch.nn.GELU(),
            torch.nn.Linear(512, 256),
            torch.nn.GELU(),
            torch.nn.Dropout(0.5),
            torch.nn.Linear(256, 2)
        )

tokenizer = AutoTokenizer.from_pretrained(
    "OU-Advacheck/deberta-v3-base-daigenc-mgt1a"
)
model = MLayerDebertaV2ForSequenceClassification.from_pretrained(
    "OU-Advacheck/deberta-v3-base-daigenc-mgt1a"
)
model.eval()

inputs = tokenizer(
    ['Hello, Thanks for sharing your health concern with us. I have gone through your query and here are your answers: 1. If you have regular cycles, there is no further need to use any medication to regulate cycles. 2. Establishment of regular ovulation and timing of intercourse properly is necessary. 3. If you want to conceive quickly, you have to get further evaluation and plan management. Hope this helps.',
     'He might have small intestinal TB rather than stomach TB. Amoebas also involves small intestine/some part of large intestine. If he has taken medicines for both diseases in form of a Complete Course, he should be fine. U can go for an oral+iv contrast CT scan of him. Now, the diagnosis of a lax cardiac can be confirmed by an upper GI endoscopy with manometry (if available). Lax cardiac may cause acidity with reflux.',
     'In Micronesia during the early 1900s, there was a unique culture known as the Yap. What sets the Yap apart from other cultures is their unusual form of currency - enormous limestone disks called Rai Stones. These massive stones are not easily movable, with the largest one weighing four tons and measuring 12 feet in diameter. Instead of physically exchanging the stones, the Yap people simply keep track of ownership through an intricate system. A fascinating tale revolves around some sailors who attempted to transport a Rai Stone across the ocean but encountered difficulties along the way, resulting in the stone falling into the sea.'],
    max_length=512,
    truncation=True,
    padding="max_length",
    return_tensors="pt"
)

torch.softmax(
    model(**inputs)[0], dim=1
).detach().cpu()[:, 1].tolist()

Quality

Quality on the delayed sample of competition organizers.

Main Score (F1 Macro)	Auxiliary Score (F1 Micro)
0.8307	0.8311

Citation

If you use that results in your research, please cite our paper:

@misc{gritsai2024advacheckgenaidetectiontask,
      title={Advacheck at GenAI Detection Task 1: AI Detection Powered by Domain-Aware Multi-Tasking}, 
      author={German Gritsai and Anastasia Voznyuk and Ildar Khabutdinov and Andrey Grabovoy},
      year={2024},
      eprint={2411.11736},
      archivePrefix={arXiv},
      primaryClass={cs.CL},
      url={https://arxiv.org/abs/2411.11736}, 
}