MIT asks arXiv to withdraw preprint of paper on AI and scientific discovery

First impressions:

1. The data in most of the plots (see the appendix) look fake. Real life data does not look that clean.

2. In May of 2022, 6 months before chatGPT put genAI in the spotlight, how does a second-year PhD student manage to convince a large materials lab firm to conduct an experiment with over 1,000 of its employees? What was the model used? It only says GANs+diffusion. Most of the technical details are just high-level general explanations of what these concepts are, nothing specific.

"Following a short pilot program, the lab began a large-scale rollout of the model in May of 2022." Anyone who has worked at a large company knows -- this just does not happen.

FWIW, in the q&a after a talk, he claims that it was a GNN (graph neural network), not a GAN.

(In this q&a, the audience does not really question the validity of the research.)

https://doi.org/10.52843/cassyni.n74lq7

Oh interesting. I haven't talked to any recent graduates but I would expect an MIT PhD student to be more articulate and not say "like" every other word.

There was a question at the end that made him a little uncomfortable:

[1:00:20]

   Q: Did you use academic labs only or did you use private labs?

   A: (uncomfortable pause) Oh private, yeah, so like all corporate, yeah...

   Q: So, no academic labs?

   A: I think it's a good question (scratches head uncomfortably, seemingly trying to hide), what this would look like in an academic setting, cause like, ... the goals are driven by what product we're going make ... academia is all, like "we're looking around trying to create cool stuff"...

My 8 year-old is more articulated than this person. Perhaps they are just nervous, I'll give them that I guess.

Don't confuse a polished TED talk from a practiced speaker with a seminar from any random person in academia. I'm sad to admit I've given talks (many recorded) that are much worse than this.

These seminar-style talks in particular have a strong Goodheart bias: academic scientists judge each other on the papers they write, but the highest honors usually come in the form of invited talks. The result is that everyone is scrambling to have their students give talks.

In larger scientific collaborations it can get a bit perverse: you want to get everyone together for discussions, but the institutes will only pay for travel if you give their students a 20 minute talk. You'll often have conferences where everyone crams into a room and listens to back-to-back 20 minute lectures for a week straight (sometimes split into multiple "parallel sessions"), and the only real questions are from a few senior people.

It's a net positive, of course: there's still some time around the lectures and even in 2025 there's no good replacement for face-to-face interaction. But I often wish more people could just go to conferences to discuss their work rather than "giving a talk" on it.

> Don't confuse a polished TED talk from a practiced speaker with a seminar from any random person in academia.

I don’t expect a TED talk but we’re still talking about MIT here. I’ve seen 8 year olds more articulated. I guess where I am from being called in front of the class and having to present or talk about the homework reading is common, so perhaps why it’s seen as exotic in US to be able tie words together without saying “like” after every other word, or slump and touch the hair every 10 seconds.