←back to thread

Do AI detectors work? Students face false cheating accusations

(www.bloomberg.com)
427 points JumpCrisscross | 5 comments | 20 Oct 24 17:26 UTC | HN request time: 0.204s | source
1. greyadept ◴[21 Oct 24 06:25 UTC] No.41901193[source]▶
I'd be really interested to run AI detectors on essays from years before the ChatGPT era, just to see if anything gets flagged.
replies(2): >>41901204 #>>41901363 #
2. woernsn ◴[21 Oct 24 06:49 UTC] No.41901363[source]▶
Yes, 3 out of 500 essays were flagged as 100% AI generated. There is a paragraph in the linked article about it.
replies(2): >>41901540 #>>41901630 #
3. greyadept ◴[21 Oct 24 07:21 UTC] No.41901540[source]▶
And another 9 flagged as partially AI.
4. _pdp_ ◴[21 Oct 24 07:36 UTC] No.41901630[source]▶
This study is not very good frankly. Before ChatGPT there was Davinci and other model families which ChatGPT (what became GPT 3.5) was ultimately based on and they are the predecessors of today's most capable models. They should test it on work that is at least 10 to 15 years old to avoid this problem.
replies(1): >>41901824 #
5. kolinko ◴[21 Oct 24 08:12 UTC] No.41901824{3}[source]▶
What? 10 years ago we wouldn’t dream of what’s happening now.

Models before 2017-2018 (first gpt/bert) didn’t produce any decent text, and before gpt2/gpt3 (2020) you wouldn’t get an essay-grade text.

So you need to go back only 4-5 years to be certain an essay didn’t use AI.