Article

Testing of detection tools for AI-generated text

Weber-Wulff et al., International Journal for Educational Integrity · link.springer.com

A test of fourteen detectors concluding they are neither accurate nor reliable: all below 80% accuracy, only five above 70%, with false positives and false negatives.

Open on link.springer.com ↗

The question it answers

  1. How can I tell whether text was written by AI?

Read with it

  1. OpenAI's own detector, withdrawn on 20 July 2023 for low accuracy: 26% of AI text caught, 9% of human text wrongly flagged. openai.com Article New AI classifier for indicating AI-written text OpenAI
  2. Seven detectors flagged 61.22% of TOEFL essays by non-native writers as AI-generated while judging US students' essays almost... hai.stanford.edu Article AI-detectors biased against non-native English writers Stanford HAI
  3. The paper behind the Stanford finding, with the data on how detectors penalise plainer writing. arxiv.org Article GPT detectors are biased against non-native English writers Liang et al. (Stanford)
  4. Turnitin's own figures: under 1% false positives for documents with 20% or more AI writing, about 4% at the sentence level. turnitin.com Article Understanding the false positive rate for sentences of our AI writing detection capability Turnitin
  5. The free plan includes a plagiarism check and an AI content detector, each up to 1,000 words. quetext.com Tool Quetext pricing Quetext
  6. How Google watermarks text generated by the Gemini app so it can be identified later, which covers Gemini's text and nothing else. deepmind.google Article Google SynthID Google DeepMind
eChai Partner Brands The partnership →