OpenAI Launches Classifier to Identify Text Generated by ChatGPT

OpenAI has trained a classifier to distinguish between text written by humans and text written by AI systems like ChatGPT. The classifier has limitations and is not fully reliable, however, it is intended to provide additional support for determining the source of a piece of text and inform mitigation for false claims that AI-generated text was written by a human. The classifier has been made publicly available for feedback, with the goal of improving future methods for detecting AI-generated text.

The classifier is a language model that has been fine-tuned on a dataset of human-written text and AI-written text on the same topic. The classifier’s reliability typically improves as the length of the input text increases and is more reliable on recent AI systems. However, it is not recommended as the primary decision-making tool and has limitations such as unreliability on short texts, incorrect labeling of human-written text as AI-written, and limited performance in other languages.

In recognition of the important role that identifying AI-written text plays in the education sector, OpenAI has developed a preliminary resource for educators to understand the uses and limitations of the ChatGPT AI-generated text classifier. The organization is also engaging with educators to learn about their experiences and discuss the capabilities and limitations of ChatGPT.

OpenAI says the classifier can distinguish between text written by a human and text written by AIs from a variety of providers. The main purpose of this classifier is to help mitigate false claims that AI-generated text was written by a human, such as running automated misinformation campaigns, using AI tools for academic dishonesty, and positioning an AI chatbot as a human.

The classifier has undergone evaluations on a “challenge set” of English texts and it has achieved 26% accuracy in correctly identifying AI-written text as “likely AI-written.” However, it also has a false positive rate of 9%, where human-written text is incorrectly labeled as AI-written. Our classifier is more reliable for longer texts, and it has significantly improved compared to our previous classifier when it comes to texts from recent AI systems.

  • Requires a minimum of 1,000 characters, which is approximately 150 – 250 words.
  • Isn’t always accurate; it can mislabel both AI-generated and human-written text.
  • AI-generated text can be edited easily to evade the classifier.
  • The classifier is likely to get things wrong on text written by children and on text not in English because it was primarily trained on English content written by adults.

Discover more from Techspace Africa

Subscribe to get the latest posts sent to your email.

spot_img
follow-us-on-google-news-banner

Top Stories

More from this stream

Discover more from Techspace Africa

Subscribe now to keep reading and get access to the full archive.

Continue reading