New Tool Can Tell If Something Is AI

A new study found an AI detector developed by the University of Kansas can detect AI-generated content in academic papers with a 99% accuracy rate, one of the only detectors on the market specifically geared toward academic writing.

Genuine human error? Perhaps, but at least not artificial intelligence.

According to a report published Wednesday in the journal Cell Reports Physical Science, researchers created a tool that can prove AI detection in academic papers with 99% accuracy.

The team of researchers selected 64 perspectives (a type of article) and used them to make 128 articles using ChatGPT, which was then used to train the AI detector.

The model had a 100% accuracy rate of identifying human-created articles from AI-generated ones, and a 92% accuracy rate for identifying specific paragraphs within the text.

According to a survey by Study.com, 89% of college students have admitted to using ChatGPT to help with assignments, while 34% of educators believe the software should be banned, though 66% support students having access.

"Right now, there are some pretty glaring problems with AI writing," lead author Heather Desaire said in a statement. "One of the biggest problems is that it assembles text from many sources and there isn't any kind of accuracy check—it's kind of like the game Two Truths and a Lie."

AI detectors have not proven to be 100% accurate. A University of California at Davis student alleges she was falsely accused by her university of cheating with AI. After uploading a paper for one of her classes, she received an email from her professor claiming a portion was flagged in the program Turnitin as being AI-generated. Her case was immediately forwarded to the Office of Student Support and Judicial Affairs, which handles discipline for academic misconduct. The student pleaded her case and ultimately won, using time stamps to prove she wrote the paper. This wasn't the only time educators falsely labeled assignments as AI-generated. A Texas A&M commerce professor attempted to flunk over half of his senior class after using ChatGPT to test whether the students used the chatbot to write their papers. He copied and pasted the papers into ChatGPT and asked if it wrote them, to which it replied yes. This caused their diplomas to be withheld by the university, though the professor offered the opportunity to redo the assignment. However, the university confirmed to Insider no students were failed or barred from graduating.

OpenAI's chatbot, ChatGPT, was opened to the public in November 2022, and in less than a week surpassed the million-users mark, with people using it for things like creating code and writing essays. The AI's intelligence led to several schools either indefinitely or temporarily banning the software, including New York Public Schools, Seattle Public Schools and the Los Angeles Unified School District.

There are already many existing programs and services that promise to identify AI-written content.

ChatGPT In Schools: Here's Where It's Banned—And How It Could Potentially Help Students (Forbes)

Here's What To Know About OpenAI's ChatGPT—What It's Disrupting And How To Use It (Forbes)