←back to thread

Do AI detectors work? Students face false cheating accusations

(www.bloomberg.com)
427 points JumpCrisscross | 1 comments | 20 Oct 24 17:26 UTC | HN request time: 0s | source
Show context
BubbleRings ◴[21 Oct 24 13:21 UTC] No.41903891[source]▶
Let's talk about the actual one page extract of her essay, which can be seen in the article, it is the second image.

My take is, if she used AI to generate that, she didn't use a very good one. I don't think ChatGPT would make the grammar and clarity mistakes that you see in the image text.

I see this:

"should be exposed to many of these forms and models to strengthen understanding" - much better as "should be exposed to as many of these forms and models as possible to strengthen their understanding"

"it is mentioned that students should have experiencing understanding the..." - plainly wrong, better would be "it is mentioned that students should have experience understanding the..."

"time with initial gird models" -> "time with initial grid models"

And there are other lines that could be improved.

My opinion is, the only solution to this problem is to allow AI detectors to flag work, but that when a work is flagged, that flagging just triggers a face to face meeting between the student and the professor, where the student is required to show through discussion of the work that they understand it well enough to have written it.

However! Often the professor is too busy, or isn't smart enough to review the writing of the student carefully enough to determine whether the student really wrote it. What to do? Why of course: invent AI systems that are really good at interviewing students well enough to tell if they really wrote a piece of work. Yeah you laugh but it will happen some day soon enough.

replies(1): >>41903974 #
ToucanLoucan ◴[21 Oct 24 13:31 UTC] No.41903974[source]▶
Honestly I think you could set AI entirely to the side here, it seems increasingly a cultural meme (and an unfortunately accurate one) that kids can't read or write. And not just on social media either, I've seen this crop up in official training and my professional experience matches it too. The vast majority of people in the United States just write really, really poorly, and the average reading level sits at an utterly pathetic sixth grade.

I don't wanna trot out "think of the children" bullshit here but it's hard for me to not notice that this trend has been happening since smartphones became normal and schools have increasingly become utterly toothless with regard to enforcing standards in education, i.e. "you need to know this shit in order to move to the next grade up." Nobody does that anymore. Just fudge the scores with extra credit or make-up assignments and send them up the chain to be a different teacher's problem next year.

> My opinion is, the only solution to this problem is to allow AI detectors to flag work, but that when a work is flagged, that flagging just triggers a face to face meeting between the student and the professor, where the student is required to show through discussion of the work that they understand it well enough to have written it.

You said it yourself in the subsequent paragraph, but if professors had this much time and energy to teach, their kids wouldn't be writing like deprecated GPT instances in the first place. We need to empower teachers and schools to fail children so they can be taught and experience consequences for lack of performance, and learn to do better. They have no reason to try because no one will hold them accountable, personally or systemically. We just let them fail and keep failing until they turn into failures of adults living in their parents basements playing Elden Ring all day and getting mad at each other over trivial bullshit on social media.

replies(2): >>41904167 #>>41904941 #
1. acdha ◴[21 Oct 24 15:01 UTC] No.41904941[source]▶
> I don't wanna trot out "think of the children" bullshit here but it's hard for me to not notice that this trend has been happening since smartphones became normal and schools have increasingly become utterly toothless with regard to enforcing standards in education, i.e. "you need to know this shit in order to move to the next grade up." Nobody does that anymore. Just fudge the scores with extra credit or make-up assignments and send them up the chain to be a different teacher's problem next year.

I agree in general that we need to have higher standards but that complaint predates smartphones by decades. One of the big challenges here is that consistency was all over the place historically but we have better measurements now and higher expectations for students, and some of the cases where students were allowed to slide were misguided but well-intended attempts to mitigate other problems – for example, many standardized tests had issues with testing things like social norms or English fluency more then subject matter literacy so there’s a temptation to make them less binding when it should be paired with things like improving tests or providing ESP classes so, say, a recent immigrant’s math score isn’t held down by their ability to read story problems.

One other thing I’d keep in mind is that this is heavily politicized and there are massive business conflicts of interest, so it’s important to remember that the situation is not as dire as some people would have you believe. For example, PISA math scores are used to claim Americans are way behind but that’s heavily skewed by socioeconomic status and tracking in some other countries – when you start adjusting for that, the story becomes less that American students as a whole are behind but rather that our affluent kids are okay but we need to better support poor kids.