Do AI detectors work? Students face false cheating accusations

When I studied physics at Exeter University they still used the tutorial system and finals. Tutorials were held fortnightly; the tutorial groups were typically three or four students. There was no obligation to turn up to lectures or even tutorials. You just had to pass the end of year exams to be allowed to continue to the final. The class of degree that was awarded depended on the open note final exam and the report of the final year project. That report had to be defended orally. Previous years exam papers were available for study as well but the variety of questions that could be asked was so vast that it was rare that any questions were repeated in the finals.

It seems to me that this is pretty much immune to plagiarism as well as being much better for the student.

I agree. There are small question about bias (gender, race) etc in these oral systems, but I think they are resolvable and much better than written essays (which are now written by AI).

What about those of us who can explain our ideas and thinking clearly and in great detail in writing but would struggle to even prove we've heard of the topic orally?

the teacher knows you either way so the bias would be there for the written exam as well

Well, you’d do badly.

Of course the current setup works badly for those who explain it much better by speaking.

I also studied Physics there!

Yeah, the General Problems exam was a nightmare, I think the professors competed each year to come up with the toughest questions. Getting 50% was an excellent score.

It did force you to learn all the material though, especially as at the end of 3-4 years you may have forgotten some of it, like Optics or whatever. It was pretty hardcore though, especially compared to my friends studying other subjects.

In a written exam they can cover the names - give you a random number as you enter the room and you write that on the paper, and but your name and number on a different paper. You also need to type everything out on a computer with spell check. (and even then if you write bucket or pail will identify you but it is unlikely any professor knows you well enough to tell those)

When you audition for a symphony you perform behind a curtain and are required to wear soft slippers (so they can't tell if you are a wearing high heals - female).

We can probably use voice changers so the examiner cannot tell who you are by your voice, but those tend to be fatiguing.

you could ask for reasonable accommodations - e.g. if you have a recognised medical condition, or even just going through a rough time - e.g. ask to be allowed to write down your answer while they wait.

In real life you need to be able to communicate written, in formal talks, and in informal discussions.

Those of you who severely lack any of the three will be penalized. Just like someone who can discuss a topic orally but could not write it up would be penalized.

I'm not going to sugar coat it and it may sound harsh, but I doubt this is ever truly an issue outside of the minute edge cases.

Yes, there are people who have trouble with public speaking to a debilitating degree, but it would be excessively rare for someone to not at the very least in a one on one with their professor/teacher be able to be so badly affected as to not seem they've even heard of a topic or at least be able to prove they've worked on it to a certain degree.

I would be immediately skeptical of any student who claims they are completely unable to explain their knowledge unless they are allowed to work in complete isolation with nobody to monitor they aren't cheating in some way.

These systems exist in no small part to train that ability, which is crucial to making it in the upper reaches of business and politics. The approach is probably also good for teaching the material, but training in speaking and arguing is more than just a side-effect of it—it’s part of the point.

Lots of elite prep schools in the US use a similar system, for similar reasons.

(not the CP, but went to a university with a tutorial-style system)

I think the hard answer is that to some extent you just have to learn to. I mean, you could sit silently in supervisions if you really insisted, but to participate properly you just needed to build the confidence.

Is it fun? No, but it's a pretty accurate reflection of life after school: nobody in the real world gives you points for "couldn't say the right thing at the right time, but was thinking it"

At my uni, you could prepare a written answer. The professor would read your written answer and ask follow-up questions.

Fellow UK person - the style of exam that you describe is pretty hard to cheat unless you can find another person to go in your place. I think various institutions have tried digital invigilation but have had little success (and I think this is just a bad idea anyway).

However, you also mentioned a final project. You’d be shocked how much commissioning exists where people have their projects produced for them. I’m not talking an overly helpful study group, I mean straight up essay mills. Tools like ChatGPT make the bar for commissioning lower and cheaper. I don’t know how you can combat this and still have long-term projects like dissertations.

Had a good friend who tutored college students and a rich middle-eastern student paid him to do a lot of his work for him.

Even with extensive notes and prep-time in a one-on-one?

Can you communicate it in real-time through writing? Maybe that's an accommodation that could be done?

This is the kind of opinion that should be common sense but is highly controversial in the modern educational climate, for whatever reason. Probably the whole, "You can't judge a fish by its ability to climb a tree" quote being misapplied constantly.

My final year project was a 120 page report of measurements of electron spin resonance together with the design of the experimental apparatus. I had to defend the design, conclusions (which I have long forgotten, it was in 1977), and justify the methods and calculation all orally to two academics.

I doubt that anyone could have produced a plausible report without actually doing the work. And to defend it one would have to understand the underlying physics and the work that was done. Plus I think my supervisor and the other two students who worked with me on the project would have remarked on my absence from the laboratory if I had simply bought the paper!

You can still have long term projects and dissertations so long as the degree is awarded for the defence of the dissertation rather than the dissertation itself; that is the student must demonstrate in a viva that they understand everything in the dissertation rather than merely regurgitate it.

That won't work in a tutorial system, the student will be quickly discovered to know nothing about the subject. And in open note finals, as in the Exeter Uni. Physics department of the 1970s, regurgitation of course material was of very limited utility because you were never asked for that kind of response. The quantum mechanics final didn't ask a single question that had been directly answered during lectures, it asked us to extend what we had learnt. That exam was what I think Americans might call a 'white knuckle ride'. Open note finals really sort those who understood the subject from those who thought they could just look up the answers, the invigilators spent a lot of time shushing people searching through rucksacks full of notes.

Many years later I took a course in C# at a university in Norway and that was not merely open note but also open book (you could take the set book in). Again that gives the exam author the possibility to really discover who knows what.

I doubt that your rich middle-western student would have passed either of these

Not too dissimilar for me at Birmingham, we had tutorials ~weekly. There were weekly problem sheets that counted for 10% of the grade though.

Similar re: exams, they were available but sticking rigidly to them didn’t help much.

If you can't trust a professor to professionally and impartially grade someone's work, the system would likely collapse. This is not to say that there hasn't been cases where professors have been shown to be biased, there has. But the premise of universities is to give professors some autonomy in the way they teach and evaluate students.

True but I think there's still an element of falsifiability to a teacher's evaluation of an essay that doesn't exist in an oral exam or interactive discussion. An essay is an artifact and if a teacher is giving student A worse grades than student B, a third party can look at that artifact to see whether it's remotely reasonable. A 1:1 discussion or an oral defense is much more subjective.

Not saying this is a fatal flaw, but there is a bit of a tradeoff there.

No system is 100% the question is are we good enough. As a white male I haven't seen many problems - but also because I'm in the group least likely to see one.

I think that in your case you've correctly observed that it would be nearly impossible to commission or otherwise fake your particular dissertation/project because of its experimental nature, and that you were called to a viva.

There are certainly similar projects being completed by students every year, and doubtless those students are not cheaters, but for each dissertation like yours, there are probably 10 or more projects that are not collaborative and have no artefacts or supporting evidence other than a written report. Such projects are fairly easy to commission. For a reasonable price (potentially thousands of dollars) you can pay a poor research student in the same field as you to churn out a mid-tier dissertation. This can be detected with a viva, but the academics need to be very confident before accusing someone of cheating. More often than not, you can get away with it and just get a not great grade.

I think that in general the natural/formal sciences don't suffer nearly as much as social science and humanities do, simply because exams and labs tend to highlight irregularities, and cheaters are less likely to be drawn into "hard" fields. However, it still exists in every field.