←back to thread

Do AI detectors work? Students face false cheating accusations

(www.bloomberg.com)
427 points JumpCrisscross | 2 comments | 20 Oct 24 17:26 UTC | HN request time: 0.002s | source
Show context
lwhi ◴[21 Oct 24 08:16 UTC] No.41901852[source]▶
It is no longer effective to solely use a written essay to measure how deeply a student comprehends a subject.

AI is here to stay; new methods should be used to assess student performance.

I remember being told at school, that we weren't allowed to use calculators in exams. The line provided by teachers was that we could never rely on having a calculator when we need it most—obviously there's irony associated with having 'calculators' in our pockets 24/7 now.

We need to accept that the world has changed; I only hope that we get to decide how society responds to that change together .. rather than have it forced upon us.

replies(26): >>41902001 #>>41902004 #>>41902006 #>>41902027 #>>41902041 #>>41902094 #>>41902144 #>>41902281 #>>41902432 #>>41902446 #>>41902471 #>>41902612 #>>41902683 #>>41902805 #>>41902892 #>>41903019 #>>41903144 #>>41903279 #>>41903529 #>>41903547 #>>41903572 #>>41903881 #>>41904424 #>>41904494 #>>41904546 #>>41905807 #
gklitz ◴[21 Oct 24 10:58 UTC] No.41902805[source]▶
Written assay evaluation is not and has never been an effective evaluation. It was always a cost saving measure because allocating 30min face to face time with each individual student for each class is such a gigantic cost for the institution that they cannot even imagine doing it. Think about that the next time you look at your student debt, it couldn’t even buy you 30min time per class individually with the teacher to evaluate your performance. Instead you had to waste more time on a written assignment so they could offload grading to a minimum wage assistent.
replies(10): >>41902890 #>>41903033 #>>41903100 #>>41903238 #>>41903325 #>>41903978 #>>41903995 #>>41904389 #>>41905332 #>>41905671 #
ninalanyon ◴[21 Oct 24 11:47 UTC] No.41903100[source]▶
When I studied physics at Exeter University they still used the tutorial system and finals. Tutorials were held fortnightly; the tutorial groups were typically three or four students. There was no obligation to turn up to lectures or even tutorials. You just had to pass the end of year exams to be allowed to continue to the final. The class of degree that was awarded depended on the open note final exam and the report of the final year project. That report had to be defended orally. Previous years exam papers were available for study as well but the variety of questions that could be asked was so vast that it was rare that any questions were repeated in the finals.

It seems to me that this is pretty much immune to plagiarism as well as being much better for the student.

replies(5): >>41903545 #>>41903679 #>>41903789 #>>41904819 #>>41907979 #
noodlesUK ◴[21 Oct 24 14:51 UTC] No.41904819[source]▶
Fellow UK person - the style of exam that you describe is pretty hard to cheat unless you can find another person to go in your place. I think various institutions have tried digital invigilation but have had little success (and I think this is just a bad idea anyway).

However, you also mentioned a final project. You’d be shocked how much commissioning exists where people have their projects produced for them. I’m not talking an overly helpful study group, I mean straight up essay mills. Tools like ChatGPT make the bar for commissioning lower and cheaper. I don’t know how you can combat this and still have long-term projects like dissertations.

replies(2): >>41905021 #>>41907599 #
1. ninalanyon ◴[21 Oct 24 19:32 UTC] No.41907599[source]▶
My final year project was a 120 page report of measurements of electron spin resonance together with the design of the experimental apparatus. I had to defend the design, conclusions (which I have long forgotten, it was in 1977), and justify the methods and calculation all orally to two academics.

I doubt that anyone could have produced a plausible report without actually doing the work. And to defend it one would have to understand the underlying physics and the work that was done. Plus I think my supervisor and the other two students who worked with me on the project would have remarked on my absence from the laboratory if I had simply bought the paper!

You can still have long term projects and dissertations so long as the degree is awarded for the defence of the dissertation rather than the dissertation itself; that is the student must demonstrate in a viva that they understand everything in the dissertation rather than merely regurgitate it.

replies(1): >>41908926 #
2. noodlesUK ◴[21 Oct 24 21:54 UTC] No.41908926[source]▶
I think that in your case you've correctly observed that it would be nearly impossible to commission or otherwise fake your particular dissertation/project because of its experimental nature, and that you were called to a viva.

There are certainly similar projects being completed by students every year, and doubtless those students are not cheaters, but for each dissertation like yours, there are probably 10 or more projects that are not collaborative and have no artefacts or supporting evidence other than a written report. Such projects are fairly easy to commission. For a reasonable price (potentially thousands of dollars) you can pay a poor research student in the same field as you to churn out a mid-tier dissertation. This can be detected with a viva, but the academics need to be very confident before accusing someone of cheating. More often than not, you can get away with it and just get a not great grade.

I think that in general the natural/formal sciences don't suffer nearly as much as social science and humanities do, simply because exams and labs tend to highlight irregularities, and cheaters are less likely to be drawn into "hard" fields. However, it still exists in every field.