←back to thread

MIT asks arXiv to withdraw preprint of paper on AI and scientific discovery

(economics.mit.edu)
333 points carabiner | 1 comments | 16 May 25 15:09 UTC | HN request time: 0.21s | source
Show context
rdtsc ◴[16 May 25 22:17 UTC] No.44010258[source]▶
> The paper was championed by MIT economists Daron Acemoglu, who won the 2024 economics Nobel, and David Autor. The two said they were approached in January by a computer scientist with experience in materials science who questioned how the technology worked, and how a lab that he wasn’t aware of had experienced gains in innovation. Unable to resolve those concerns, they brought it to the attention of MIT, which began conducting a review.

So the PhD student might have been kicked out. But what about the people who "championed it". If they worked with the student, surely they might have figured out the mythical lab full of 1000s material scientists might not exist, it might exist but they never actually used any AI tool.

replies(3): >>44010581 #>>44011603 #>>44013783 #
raphman ◴[16 May 25 23:04 UTC] No.44010581[source]▶
Apparently, none of the 21 people mentioned in the acknowledgments questioned the source of the dataset. One of them also wrote a quite popular Twitter thread about the research. When notified of the recent events, he curtly replied that "It indeed seems like the data used in the paper is unreliable." [no need to mention them by name, I think]
replies(2): >>44011154 #>>44013305 #
a_bonobo ◴[17 May 25 00:48 UTC] No.44011154[source]▶
This happens again and again in research - I'm just reminded of the stem cell scandal around Obakata. Before the fraud was uncovered: dozens of senior researchers supporting the research, using the glory for their own gains. After the fraud is exposed: nobody wants to have been involved, it was only that one junior person.

It doesn't excuse the fraud of the junior person but it makes you think how many senior-level people out there are riding similar fraudulent waves, doing zero checks on their own junior people's work.

replies(1): >>44011389 #
1. tanewishly ◴[17 May 25 01:38 UTC] No.44011389[source]▶
Most of the processes surrounding and supporting science are not robust against a dedicated adversary seeking to exploit the system. This is nothing new - Newton ordered and then wrote the anonymous report commissioned by (iirc) the Royal Society to decide who invented calculus, him or Leibniz.

Basically, science is quite vulnerable to malicious exploiters. Part of this is because society isn't funding science anywhere near sufficiently to do a priori in-depth checks. You claim you got data on hundreds of measurable thingies in a certain way (from surveying people to scanning the web to whatever)? If it's not blatantly obviously a lie, it'll probably be accepted. Which is inevitable: at one point, you're going to have to accept the data as genuine. If there's no obvious red flags, you'd only waste time on further checking data - you'd need to do a real deep dive (expensive time-wise) to come up with circumstantial evidence that may still be explainable in a benign manner. For scientists, it is almost always more profitable to spend such time investments on furthering their own scientific efforts.

So yes, there are various ways in which someone willing, dedicated and sufficiently skilled can "Nigerian-Prince" the scientific process. Thankfully, the skill to do so typically requires intimate knowledge of the scientific process and how to conduct research -- this cheating is not easily accessible to outside bullshitters (yet).