←back to thread

Do AI detectors work? Students face false cheating accusations

(www.bloomberg.com)
427 points JumpCrisscross | 1 comments | 20 Oct 24 17:26 UTC | HN request time: 0s | source
Show context
k__ ◴[21 Oct 24 08:08 UTC] No.41901794[source]▶
I'm a professional writer and test AI and AI detectors ever other month.

Plagiarism detectors kinda work, but you can always use one to locate plagiarized sections and fix them yourself.

I have a plagiarism rate under 5%, usually coming from the use of well known phrases.

An AI usually has over 10%.

Obviously that doesn't help in an academic context when people mark their citations.

The perplexity checks don't work, as humans seem to vary highly in that regard. Some of my own text has less perplexity as a comparable AI text.

replies(1): >>41903899 #
1. tropdrop ◴[21 Oct 24 13:22 UTC] No.41903899[source]▶
FWIW Turnitin does treat things like quotes and footnotes a little differently on the academic side – on the instructor end, it simply gives you an estimation of the amount of text it found that has appeared somewhere else. Citations usually account for about 5-10% "potentially plagiarized" but anything below 10% is treated as fine by the software. You can always go check each of the sections and see if it's a quote or not; if you have a paper that consists of more than 10% quotes it's not a good paper anyway and should be revised.

I did have a very interesting case once of a student who copied and pasted someone's Master's thesis for sections of her paper, but also listed that thesis in the citations... it remains up to the jury (not me) to decide whether she just didn't understand what plagiarism was. I would not have known if Turnitin didn't mark it as 30% plagiarized.

Disclaimer: Someone more senior than I was in charge of the decision to use this software, but it was interesting to see it in action