Do AI detectors work? Students face false cheating accusations

FWIW, I'm a consultant for a large University hospital, and Dutch. My PhD thesis, years ago, got the remark: "Should have checked with a native speaker."

So, now I use ChatGPT to check my English. I just write what I want to write than ask it to make my text more "More concise, business-like and not so American" (yeah the thing is by default as ultra enthusiastic as an American waiter). And 9 out of 10 times it says what I want to say but better than I wrote myself, and in much less words and better English.

I don't think it took less time to write my report, but it is much much better than I could have made alone.

AI detector may go off (or it goes on? of is it of? Idk, perhaps I should ask Chat ;)), but it is about as useful as a spell-check detector.

It's a Large Language Model, you should just is like that, it is not a Large Fact Model. But if you're a teacher you should be a good bullshit detector, right?

If I'm every checking some student's report, you may get this feedback: For god's sake, check the language with ChatGPT, but for God's sake check the fact in some other way.

I completely agree. LLMs are incredibly useful for improving the flow and structure of an argument, not just for non-native speakers, but even for native English speakers.

Making texts more accessible through clear language and well-structured arguments is a valuable service to the reader, and I applaud anyone who leverages LLMs to achieve that. I do the same myself.

When I was a junior in high school, the Advanced English teacher was also the AP English teacher. All the juniors had to write a term paper, and she had the seniors in the AP class give our papers' first draft a once over and give notes.

Both classes got a lesson, from either end, essentially for free (for the teacher). And it really helped. The next year I got to do the same. Of note was that this was back in the day when computers were relatively rare and typing was a skill that was specially taught, so most of the papers were written longhand for the first draft.

It's long been said that if you really want to learn a subject you should teach it. This sort of give-and-take works well, and it is more or less how the rest of society works. Using AI for this would be quite similar, but I think having another human is better. An AI will never stop you in the hall and say "dude, your paper, I got totally lost in the middle section, what the hell," but sometimes that's quite helpful.

>It's a Large Language Model, you should just is like that, it is not a Large Fact Model.

Not by design, but the training corpus necessarily includes a lot of "facts" (claims made by whoever wrote the original text). A model that is trying to output nonfiction on a specific topic, is likely to encounter relatively more models of claims that either actually were incidentally true, or at least have the same general form as true claims without an obvious "tell".

Of course, every now and then it goes off the rails and "hallucinates". Bad luck for the student who doesn't verify the output when this happens (which is probably a lot of students, since part of the motivation to cheat is not knowing the material well enough to do such verification properly).

Yes it's a valuable service but we should also be aware that it puts more and more weight on written language and less weight on spoken language. Being able to write clearly is one thing, but being able to converse verbally with another individual is another entirely, and both have value.

With students, historically we have always assumed that written communication was the more challenging skill and our tests were arranged thusly. But we're in a new place now where the inability to verbally converse is a real hurdle to overcome. Maybe we should rethink how we teach and test.