←back to thread

Do AI detectors work? Students face false cheating accusations

(www.bloomberg.com)
427 points JumpCrisscross | 2 comments | 20 Oct 24 17:26 UTC | HN request time: 0s | source
Show context
mrweasel ◴[21 Oct 24 08:22 UTC] No.41901883[source]▶
The part that annoys me is that students apparently have no right to be told why the AI flagged their work. For any process where an computer is allowed to judge people, where should be a rule in place that demands that the algorithm be able explains EXACTLY why it flagged this person.

Now this would effectively kill off the current AI powered solution, because they have no way of explaining, or even understanding, why a paper may be plagiarized or not, but I'm okay with that.

replies(8): >>41902108 #>>41902131 #>>41902463 #>>41902522 #>>41902919 #>>41905044 #>>41905842 #>>41907688 #
viraptor ◴[21 Oct 24 10:00 UTC] No.41902463[source]▶
> kill off the current AI powered solution, because they have no way of explaining

That's not correct. Some solution look at perplexity for specific models, some will look at ngram frequencies, and similar approaches. Almost all of those can produce a heatmap of "what looks suspicious". I wouldn't expect any of the detection systems to be like black boxes relying on LLM over the whole text.

replies(2): >>41902900 #>>41904060 #
1. mrweasel ◴[21 Oct 24 11:13 UTC] No.41902900[source]▶
Sorry if this is "moving the goal post", but I wouldn't call looking at ngram frequencies for AI. Producing a heatmap doesn't tell you why something is suspicious, but it's obviously better than telling you nothing.

In any case, if you where to use LLMs, or other black box solutions, you'd have to yank those out, if you where met with a requirement to explain why something is suspicious.

replies(1): >>41907817 #
2. viraptor ◴[21 Oct 24 19:49 UTC] No.41907817[source]▶
It's literally the explanation. The only identification we have now is "this local part is often used by an AI model" and "this global structure is often used by an AI model". There's nothing more fancy about it. The heatmap would literally just point out "this part is suspiciously unlikely" - that's the explanation because that's the classification systems use.