←back to thread

Do AI detectors work? Students face false cheating accusations

(www.bloomberg.com)
427 points JumpCrisscross | 1 comments | 20 Oct 24 17:26 UTC | HN request time: 0.001s | source
Show context
lwhi ◴[21 Oct 24 08:16 UTC] No.41901852[source]▶
It is no longer effective to solely use a written essay to measure how deeply a student comprehends a subject.

AI is here to stay; new methods should be used to assess student performance.

I remember being told at school, that we weren't allowed to use calculators in exams. The line provided by teachers was that we could never rely on having a calculator when we need it most—obviously there's irony associated with having 'calculators' in our pockets 24/7 now.

We need to accept that the world has changed; I only hope that we get to decide how society responds to that change together .. rather than have it forced upon us.

replies(26): >>41902001 #>>41902004 #>>41902006 #>>41902027 #>>41902041 #>>41902094 #>>41902144 #>>41902281 #>>41902432 #>>41902446 #>>41902471 #>>41902612 #>>41902683 #>>41902805 #>>41902892 #>>41903019 #>>41903144 #>>41903279 #>>41903529 #>>41903547 #>>41903572 #>>41903881 #>>41904424 #>>41904494 #>>41904546 #>>41905807 #
torginus ◴[21 Oct 24 09:55 UTC] No.41902432[source]▶
Just because a method of assessment became easily spoofable doesn't mean we should give up on it. Imagine if in the era before HTTPS we just said that the internet won't be really viable because it's impossible to communicate securely on it.

I still feel like AI detectors would work well if we have access to the exact model, output probabilities of tokens, We can just take a bit of given text, and calculate the cumulative probability that the AI would complete it exactly like that.

replies(1): >>41902486 #
nick3443 ◴[21 Oct 24 10:04 UTC] No.41902486[source]▶
Probability is not an acceptable way to determine a student's future. They may have learned from the AI and remember some of the exact phrasing, and learned writing/language cues from it as well.
replies(1): >>41903714 #
1. alias_neo ◴[21 Oct 24 13:01 UTC] No.41903714[source]▶
Agreed. I'm not a good writer, tending to stick to a somewhat abrupt, point-making structure almost better suited for bullet pointing. I've taken tips from other HN users on how to improve, but I have no doubt that had I been going through university these days, I'd probably be flagged too.