Do AI detectors work? Students face false cheating accusations

Seems like the easy fix here is move all evaluation in-class. Are schools really that reliant on internet/computer based assignments? Actually, this could be a great opportunity to dial back unnecessary and wasteful edu-tech creep.

Out of class evaluations doesn't mean electronic. It could be problem sets, essays, longer-form things like projects. All of these things are difficult to do in a limited time window.

These limited time-window assessments are also (a) artificial (don't always reflect how the person might use their knowledge later) (b) stressful (some people work better/worse with a clock ticking) and (c) subject to more variability due to the time pressure (what if you're a bit sick, or have had a bad day or are just tired during the time window?).

The only longterm solution that makes sense is to allow students to use AI tools and to require a log provided by the AI tool to be provided. Adjust the assignment accordingly and use custom system prompts for the AI tools so that the students are both learning about the underlying subject and also learning how to effectively use AI tools.

That overall would be the right thing. Homework is such a weird concept when you think about it. Especially if you get graded on the correctness. There is no step between the teacher explaining and you validating whether you understood the material.

Teacher explains material, you get homework about the material and are graded on it.

It shouldn't be like that. If the work (i.e. the exercises) are important to grasp the material, they should be done in class.

Also removes the need of hiring tutors.

It could also be hybrid, with an out-of-class and an in-class components. There could even be multiple steps, with in-class components aimed at both verifying authorship and providing feedback in an iterative process.

AI makes it impossible to rely on out-of-class assignments to evaluate the kids' knowledge. How we respond to that is unclear, but relying on cheating detectors is not going to work.

> If the work (i.e. the exercises) are important to grasp the material, they should be done in class.

I'd like to offer what I've come to realize about the concept of homework. There are two main benefits to it: [1] it could help drill in what you learned during the lecture and [2] it could be the "boring" prep work that would allow teachers to deliver maximum value in the classroom experience.

Learning simply can't be confined in the classroom. GP suggestion would be, in my view, detrimental for students.

[1] can be done in class but I don't think it should be. A lot of students already lack the motivation to learn the material by themselves and hence need the space to make mistakes and wrap their heads around the concept. A good instructor can explain any topic (calculus, loops and recursion, human anatomy) well and make the demonstration look effortless. It doesn't mean, however, that the students have fully mastered the concept after watching someone do it really well. You only start to learn it once you've fluffed through all the pitfalls at least mostly on your own.

[2] can't be done in class, obviously. You want your piano teacher to teach you rhythm and musical phrasing, hence you better come to class already having mastered notation and the keyboard and with the requisite digital dexterity to perform. You want your coach to focus on the technical aspects of your game, focus on drilling you tactics; you don't want him having to pace you through conditioning exercises---that would be a waste of his expertise. We can better discuss Hamlet if we've all read the material and have a basic idea of the plot and the characters' motivations.

That said, it might make sense to simply not grade homeworks. After all, it's the space for students to fail. Unfortunately, if it weren't graded, a lot of students will just skip it.

Ultimately, it's a question of behavior, motivation, and incentives. I agree that the current system, even pre-AI, could only barely live up to ideals [1] and [2] but I don't have any better system in mind either, unfortunately.

Yep. The solutions which actually benefit education are never expensive, but require higher quality teachers with less centralized control:

- placing less emphasis on numerical grades to disincentive cheating (hard to measure success) - open response written questions (harder to teach, harder to grade) - reading books (hard to determine if students actually did it) - proof based math (hard to teach)

Instead we keep imagining more absurd surveillance systems “what if we can track student eyes to make sure they actually read the paragraph”

Moving everything in class seems like a good idea in theory. But in practice, kids need more time than 50 minutes of class time (assuming no lecture) to work on problems. Sometimes you will get stuck on 1 homework question for hours. If a student is actively working on something, yanking them away from their curiosity seems like the wrong thing to do.

On the other hand, kids do blindly use the hell out of ChatGPT. It's a hard call: teach to the cheaters or teach to the good kids?

I've landed on making take-home assignments worth little and making exams worth most of their grade. I'm considering making homework worth nothing and having their grade be only 2 in-class exams. Hopefully that removes the incentive to cheat. If you don't do homework, then you don't get practice, and you fail the two exams.

(Even with homework worth little, I still get copy-pasted ChatGPT answers on homework by some students... the ones that did poorly on the exams...)

totally agree. More time spent questionning the students about their work would make AI detection useless...

but somehow, we don't trust teacher anymore. Those in power want to check that the teacher actually makes his job so they want to see wome written, reviewable proof... So the grades are there both to control the student and the teacher. WWW (What a wonderful world).

> If you don't do homework, then you don't get practice, and you fail the two exams.

I'd be cautious about that, because it means the kids with undiagnosed ADHD who are functionally incapable of studying without enforced assignments will just completely crash and burn without absorbing any of the material at all.

Or, at least, that's what happened to me in the one and only pre-college class I ever had where "all work is self-study and only the tests count" was the rule.

> you don't want him having to pace you through conditioning exercises---that would be a waste of his expertise

I fundamentally disagree - I vividly remember, many times during homework in maths for example, I realised that I am stuck and so don’t understand something explained earlier, and I need to ask someone. For me, my parents were able to help. But later in Highschool, when you get to differential equations - they no longer can. And obviously if your parents are poorly educated they can’t rather.

Second point, there is no feedback loop this way - a teacher should see how difficult is his homework, how much time students spend on it, and why they are struggling. Marking a piece of paper does not do it. There was wild inconsistency between teachers for how much homework they would set and how long they thought it would take students.

Lastly, the school + homework should be able to accommodate tag the required learning within 1 working day. It is anyway a form of childcare while parents work

That's a non-starter for most schools.

There are more students than ever, and lots of schools now offer remote programs, or just remote options in general for students, to accommodate for the increased demand.

There's little political will to revert to the old ways, as it would drive up the costs. You need more space and you need more workers.

Online classes exist?

> I've landed on making take-home assignments worth little and making exams worth most of their grade.

I feel like this is almost exactly moving all evaluation into the class. If "little" becomes nothing, it is exactly that.

I feel this was always the best strategy. In college, how much homework assignments were worth was an easy way to evaluate how bad the teacher was and how lightweight the class was going to be. My best professors dared you not to do your homework, and would congratulate you if you could pass their exams without having done it.

The very best ones didn't even want you to turn it in, they'd only assign problems that had answers in the back of the book. Why put you through a entire compile cycle of turning it in, having a TA go over it, and getting it back when you were supposed to be onto the next thing? Better and cheaper to find out you're wrong quickly.

> Homework is such a weird concept when you think about it.

It’s not when you reframe it in Puritanical terms. Keep the children busy for 12 hours per day: If they get some practice on their courses, great, but busy, quiet children won’t fall in with the devil.

I wish I could get a refund on all the wasted childhood I spent doing useless homework on subjects I have not used since. No, it didn’t make me “a well-rounded person,” it just detracted from the time I could spend learning about computers—a subject my school could not teach me.

  I'm considering making homework worth nothing and having their grade be only 2 in-class exams.

When I did A levels and my first undergraduate degree (in the UK) that's how it worked. The only measurements used to calculate my A level grades and degree class were:

- Proctored exams at the end of 2 years of study (the last 2 years of high school)

- Proctored exams at the end of 2 years of study (the last 2 years of university)

I completed college with unmanaged ADHD (diagnosed 10 years later; worst result my psych had ever seen on the TOVA lol).

My second and third semesters went exactly as you described for courses where I was exposed to new things and wasn't just repeating high school - mainly because I had no training or coping mechanisms for learning under that type of pedagogy.

After getting my ass kicked in exams and failing a class for the first time in my life, I finally grokked that optional homework assignments were the professor's way of communicating learning milestones to us, and that even though the professor said they weren't graded (unless you asked), you still had to do them or you wouldn't learn the material well enough to pass the exam.

Still had a few bad grades because of the shit foundation I built for myself, but I brought a 2.2 GPA up to a 3.3 by the end.

The point is that it takes is exposure to that style of teaching before it can really be effective.

Minor quibble here: If a student gets stuck on a single homework problem for hours, they're probably hopelessly lost and would benefit from being interrupted. That or the problem is way too broad to be mere homework.