Do AI detectors work? Students face false cheating accusations

AI is here to stay; new methods should be used to assess student performance.

I remember being told at school, that we weren't allowed to use calculators in exams. The line provided by teachers was that we could never rely on having a calculator when we need it most—obviously there's irony associated with having 'calculators' in our pockets 24/7 now.

We need to accept that the world has changed; I only hope that we get to decide how society responds to that change together .. rather than have it forced upon us.

That basically never happens and the outcome is the result of some sort of struggle. Usually just a peaceful one in the courts and legislatures and markets, but a struggle nonetheless.

> new methods should be used to assess student performance.

Such as? We need an answer now because students are being assessed now.

Return to the old "viva voce" exam? Still used for PhDs. But that doesn't scale at all. Perhaps we're going to have to accept that and aggressively ration higher education by the limited amount of time available for human-to-human evaluations.

Personally I think all this is unpredictable and destabilizing. If the AI advocates are right, which I don't think they are, they're going to eradicate most of the white collar jobs and academic specialties for which those people are being trained and evaluated.

My current best guess, is to hand the student stuff that was written by an LLM, and challenge them to find and correct its mistakes.

That's going to be what they do in their careers, unless the LLMs get so good they don't need to, in which case https://xkcd.com/810/ applies.

> Personally I think all this is unpredictable and destabilizing. If the AI advocates are right, which I don't think they are, they're going to eradicate most of the white collar jobs and academic specialties for which those people are being trained and evaluated.

Yup.

I hope the e/acc types are wrong, we're not ready.

Finding errors in a text is a useful exercise, but clearly a huge step down in terms of cognitive challenge from producing a high quality text from scratch. This isn't so much an alternative as it is just giving up on giving students intellectually challenging work.

> That's going to be what they do in their careers

I think this objection is not relevant. Calculators made pen-and-paper arithmetic on large numbers obsolete, but it turns out that the skills you build as a child doing pen-and-paper arithmetic are useful once you move on to more complex mathematics (that is, you learn the skill of executing a procedure on abstract symbols). Pen-and-paper arithmetic may be obsolete as a tool, but learning it is still useful. It's not easy to identify which "useless" skills are still useful as to learn as cognitive training, but I feel pretty confident that writing is one of them.