5-9 juin 2023 PARIS (France)
Towards a Robust Detection of Language Model-Generated Text: Is ChatGPT that easy to detect?
Wissam Antoun  1@  , Virginie Mouilleron  1@  , Benoît Sagot  2@  , Djamé Seddah  2, *@  
1 : Automatic Language Modelling and ANAlysis & Computational Humanities
Inria de Paris
2 : Automatic Language Modelling and ANAlysis & Computational Humanities
Inria de Paris
* : Auteur correspondant

Recent advances in natural language processing (NLP) have led to the development of large language models (LLMs) such as ChatGPT. This paper proposes a methodology for developing and evaluating ChatGPT detectors for French text, with a focus on investigating their robustness on out-of-domain data and against common attack schemes. The proposed method involves translating an English dataset into French and training a classifier on the translated data. Results show that the detectors can effectively detect ChatGPT-generated text, with a degree of robustness against basic attack techniques in in-domain settings. However, vulnerabilities are evident in out-of-domain contexts, highlighting the challenge of detecting adversarial text. The study emphasizes caution when applying in-domain testing results to a wider variety of content. We provide our translated datasets and models as open-source resources.

Personnes connectées : 1 Vie privée