←back to thread

MIT asks arXiv to withdraw preprint of paper on AI and scientific discovery

(economics.mit.edu)
386 points carabiner | 7 comments | 16 May 25 15:09 UTC | HN request time: 1.344s | source | bottom
Show context
intoamplitudes ◴[16 May 25 16:46 UTC] No.44007496[source]▶
First impressions:

1. The data in most of the plots (see the appendix) look fake. Real life data does not look that clean.

2. In May of 2022, 6 months before chatGPT put genAI in the spotlight, how does a second-year PhD student manage to convince a large materials lab firm to conduct an experiment with over 1,000 of its employees? What was the model used? It only says GANs+diffusion. Most of the technical details are just high-level general explanations of what these concepts are, nothing specific.

"Following a short pilot program, the lab began a large-scale rollout of the model in May of 2022." Anyone who has worked at a large company knows -- this just does not happen.

replies(8): >>44007628 #>>44007719 #>>44007830 #>>44008308 #>>44009207 #>>44009339 #>>44009549 #>>44012142 #
pixl97 ◴[16 May 25 16:59 UTC] No.44007628[source]▶
>The data in most of the plots (see the appendix) look fak

Could a Benford's Law analysis apply here to detect that?

replies(1): >>44007841 #
1. constantcrying ◴[16 May 25 17:20 UTC] No.44007841[source]▶
How would you apply it, why would it be applicable?
replies(1): >>44010826 #
2. tough ◴[16 May 25 23:50 UTC] No.44010826[source]▶
Fake data is usually too clean
replies(1): >>44013365 #
3. constantcrying ◴[17 May 25 10:44 UTC] No.44013365[source]▶
And? How is that at all a relevant observation?
replies(2): >>44015471 #>>44017835 #
4. tough ◴[17 May 25 16:42 UTC] No.44015471{3}[source]▶
Anyone looking at inly the data objectively should br able to comento terms that is distribution is unnatural, as it turns out to fake organic isnt as easy
replies(1): >>44015620 #
5. constantcrying ◴[17 May 25 17:07 UTC] No.44015620{4}[source]▶
This was about Benford's law.
replies(1): >>44016789 #
6. tough ◴[17 May 25 20:31 UTC] No.44016789{5}[source]▶
Sorry for replying to the wrong thread, should have just been a general reply indeed
7. pixl97 ◴[17 May 25 23:42 UTC] No.44017835{3}[source]▶
And what exactly do you think Benfords law is?