Two days ago, we just reported that Stanford University has developed a model called DetectGPT, which can detect whether a text paragraph is generated by a given large language model (LLM). Stanford University will use DetectGPT to deal with students The phenomenon of cheating and paper generation using AI tools like ChatGPT.

Only a few days later, OpenAI, the developer of ChatGPT, decided to release a free network tool AI Text Classifier (you need to log in to the OpenAI account to use it). The purpose of this tool is the same as DetectGPT, and it is also to help users identify whether a certain text is Written by a human or a machine.

The use of this tool is also very simple. The user only needs to copy a piece of text into the text box, and the system will judge the possibility of the text being generated by the AI ​​system. The judgment result is from “very unlikely – very likely” There are 5 levels in total.

AI Text Classifier can not only detect text generated by ChatGPT, but also detect text generated by other similar AI tools. However, according to OpenAI, the accuracy of the tool does not seem to be very high at present.

The AI ​​Text Classifier correctly identified 26 percent of AI-generated text as “probably written by AI,” while it incorrectly labeled human-written text as written by AI 9 percent of the time.

Therefore, Jan Leike, the head of OpenAI, also emphasized that this tool is not perfect, it will produce both false positives and false negatives, and users should not rely solely on this tool to determine the author of the text.

In addition to being less precise, OpenAI says the tool is better at analyzing English-language text consisting of more than 1,000 characters. AI Text Classifier cannot reliably evaluate shorter text fragments or those written in other languages. Also, the tool cannot accurately detect AI-generated computer code.

Judging from the above results, in order to avoid “unjust and wrongly decided cases”, AI Text Classifier is not yet suitable for teachers to use. Of course, OpenAI is aware of concerns about AI-generated text in education and elsewhere, and they are working on other ways to help people distinguish AI-generated text from human-authored text.