TITLE: Assessing the Effectiveness of GPT-3 in Detecting False Political Statements: A Case Study on the LIAR Dataset 

ABSTRACT: The detection of political fake statements is crucial for maintaining
information integrity and preventing the spread of misinformation in society.
Historically, state-of-the-art machine learning models employed various methods
for detecting deceptive statements. These methods include the use of metadata
(W. Wang et al., 2018), n-grams analysis (Singh et al., 2021), and linguistic
(Wu et al., 2022) and stylometric (Islam et al., 2020) features. Recent
advancements in large language models, such as GPT-3 (Brown et al., 2020) have
achieved state-of-the-art performance on a wide range of tasks. In this study,
we conducted experiments with GPT-3 on the LIAR dataset (W. Wang et al., 2018)
and achieved higher accuracy than state-of-the-art models without using any
additional meta or linguistic features. Additionally, we experimented with
zero-shot learning using a carefully designed prompt and achieved near
state-of-the-art performance. An advantage of this approach is that the model
provided evidence for its decision, which adds transparency to the model's
decision-making and offers a chance for users to verify the validity of the
evidence provided.
The detection of political fake statements is crucial for maintaining
information integrity and preventing the spread of misinformation in society.
Historically, state-of-the-art machine learning models employed various methods
for detecting deceptive statements. These methods include the use of metadata
(W. Wang et al., 2018), n-grams analysis (Singh et al., 2021), and linguistic
(Wu et al., 2022) and stylometric (Islam et al., 2020) features. Recent
advancements in large language models, such as GPT-3 (Brown et al., 2020) have
achieved state-of-the-art performance on a wide range of tasks. In this study,
we conducted experiments with GPT-3 on the LIAR dataset (W. Wang et al., 2018)
and achieved higher accuracy than state-of-the-art models without using any
additional meta or linguistic features. Additionally, we experimented with
zero-shot learning using a carefully designed prompt and achieved near
state-of-the-art performance. An advantage of this approach is that the model
provided evidence for its decision, which adds transparency to the model's
decision-making and offers a chance for users to verify the validity of the
evidence provided.
Assessing the Effectiveness of GPT-3 in Detecting False Political Statements: A Case Study on the LIAR Dataset
http://arxiv.org/abs/2306.08190v1