The paper had an HN thread a few months ago
replies(2):
1. The data in most of the plots (see the appendix) look fake. Real life data does not look that clean.
2. In May of 2022, 6 months before chatGPT put genAI in the spotlight, how does a second-year PhD student manage to convince a large materials lab firm to conduct an experiment with over 1,000 of its employees? What was the model used? It only says GANs+diffusion. Most of the technical details are just high-level general explanations of what these concepts are, nothing specific.
"Following a short pilot program, the lab began a large-scale rollout of the model in May of 2022." Anyone who has worked at a large company knows -- this just does not happen.
https://en.wikipedia.org/wiki/When_Contact_Changes_Minds
https://citeseerx.ist.psu.edu/document?repid=rep1&type=pdf&d...
> As we examined the study’s data in planning our own studies, two features surprised us: voters’ survey responses exhibit much higher test-retest reliabilities than we have observed in any other panel survey data, and the response and reinterview rates of the panel survey were significantly higher than we expected.
> The firm also denied having the capabilities to perform many aspects of the recruitment procedures described in LaCour and Green (2014).
The data quality for that would need to be unimaginably high.
% gunzip -c arXiv-2412.17866v1.tar.gz | tar xOf - main.tex | grep '\bI have\b'
To summarize, I have established three facts. First, AI substantially increases the average rate of materials discovery. Second, it disproportionately benefits researchers with high initial productivity. Third, this heterogeneity is driven almost entirely by differences in judgment. To understand the mechanisms behind these results, I investigate the dynamics of human-AI collaboration in science.
\item Compared to other methods I have used, the AI tool generates potential materials that are more likely to possess desirable properties.
\item The AI tool generates potential materials with physical structures that are more distinct than those produced by other methods I have used.
% gunzip -c arXiv-2412.17866v1.tar.gz | tar xOf - main.tex | grep '\b I \b' | wc
25 1858 12791
%Artificial Intelligence, Scientific Discovery, and Product Innovation [pdf] - https://news.ycombinator.com/item?id=42115310 - Nov 2024 (47 comments)
(Since press release titles about negative news tend to studiously avoid saying anything, we tend to classify them in the "misleading" bucket of https://news.ycombinator.com/newsguidelines.html, which justifies rewriting them.)
> The paper was championed by MIT economists Daron Acemoglu, who won the 2024 economics Nobel, and David Autor. The two said they were approached in January by a computer scientist with experience in materials science who questioned how the technology worked, and how a lab that he wasn’t aware of had experienced gains in innovation. Unable to resolve those concerns, they brought it to the attention of MIT, which began conducting a review.
That’s not a Nobel Prize.
Lay low for a year, work on some start-up-ish looking project, then use his middle name to get hired at one of the many AI startups? (only half joking)...
Between this and the subtle reference to “former second-year PhD student” it makes sense that they’d have to make a public statement.
They do a good job of toeing the required line of privacy while also giving enough information to see what’s going on.
I wonder if the author thought they could leave the paper up and ride it into a new position while telling a story about voluntarily choosing to leave MIT. They probably didn’t expect MIT to make a public statement about the paper and turn it into a far bigger news story than it would have been if the author quietly retracted it.
> Nobel accuses the awarding institution of misusing his family's name
From Alfred Nobel’s great grandnephew (I’m not even sure what that looks like on a family tree), to spare anyone else looking it up.
They may have thought they could jump into an industry job, including the paper and all of its good press coverage on their resume. Only the author can retract an arXiv paper, not their academic institution. It wouldn't be hard to come up with a story that they decided to leave the academic world and go into industry early.
MIT coming out and calling for the paper's retraction certainly hampers that plan. They could leave it up and hope that some future employer is so enamored with their resume that nobody does a Google search about it, but eventually one of their coworkers is going to notice.
AFAICT your take exists entirely to delegitimize economics as a science. Very childish and frustrating.
He probably will never be someone of significance, but he also will probably be able to have a standard middle class life.
Yawn.
Distant relation of man who used his fortune making explosives to give a prize to prominent academic unhappy, complains. The foundation got to make the decision, was given the name. This is "old man yells at cloud" level of discourse. This distant relation has less of a right to say how the name gets to be used than the foundation created by the man.
(In this q&a, the audience does not really question the validity of the research.)
> AFAICT your take exists entirely to delegitimize economics as a science. Very childish and frustrating.
You know, real sciences don't need shiny medallions to make them legitimate. I'd say your comment delegitimizes economics more than the GP's.
So the idea that it should be a "peace prize" or contribute to the world as a whole is entirely lost in this definition. Which is why I find the Sveriges Riksbank memorial prize so unctuous.
Edit: this comment was only partially serious, not meant as legal advice to MIT.
It sounds like "we don't like it and won't tell you why, we're hiding behind MIT policy and vague notions of privacy".
MIT should just demonstrate in a paper what the shortcomings are and print it, adding it to the citation tree of the original.
Looking very briefly at the paper and speculating wildly, I could imagine that the company who were subject of it - or their staff - might not appreciate it and have put pressure on MIT??
Solid amount of Streisand Effect going on here -- lots of attention has been bought to the paper (and that is everything after all!).
Nice to include the giants we stand on as implied coauthors.
Whether MIT is right or wrong, the arrogance displayed is staggering. The only thing more shocking is that obviously this behavior works for them and they are used to people obeying them without question because they are MIT.
Furthermore, if the author could demonstrate to arXiv that the request was fraudulent, the paper would be reinstated. The narrative would also switch to people being angry at MIT for impersonating a student to do something.
The apparent issue is that the data appears to have been entirely fabricated and is a lie. The author appears to simply be a fraud
(Incidentally, I don't think misplaced trust in preprints is much of an academic issue, people that are experts in their field can easily judge quality for themselves. It's laypeople taking them at face value that's the problem.)
I think MIT is trying to protect its reputation as a would-be place of fraud-free research, unlike Harvard.
FERPA is federal law. It is quite likely that MIT is legally bound to not release some pieces of evidence which are crucial in this case (hypothetically, for example: that the student's educational record is inconsistent with claims made in the paper).
[0] E.g. arxiv/0812.0848: "This paper has been withdrawn by the author due to a crucial definition error of Triebel space".
Emails are not people. You can impersonate a person, but you can't impersonate an email. If I own a company and I issue the email dick.less@privateequity.com but then have to fire him... using this email address to transfer company assets back to someone who can be responsible for them isn't fraud (for that purpose, at least). How is this not the same issue?
I miss google search's Cache. As with the seminar, several other hits on MIT pages have been removed. I'm reminded of a PBS News Hour story, on free fusion energy from water in your basement (yes, really), which was memory holed shortly after. The next-ish night they seemed rather put out, protesting they had verified the story... with "a scientist".
That cassyni talk link... I've seen a lot of MIT talks (a favorite mind candy), and though Sloan talks were underrepresented, that looked... more than a little odd. MIT Q&A norms are diverse, from the subtle question you won't appreciate if you haven't already spotted the fatal flaw, to bluntness leaving the speaker in tears. I wonder if there's a seminar tape.
Although not explicitly stated, i read previous comments as using dick.less@privateequity.com to cancel his personal Netflix account. (Let's say that privateequity.com allowed personal usage of company email.)
I see a difference between accessing an email account and impersonating the previous account holder.
*Note that this creates a new version lacking any download which also becomes the default but any previous ones are still available.
Edit: Since the paper has been cited, others may still need to reference the paper to determine if it materially affects a paper citing it. If the paper is removed it’s just a void.
So the PhD student might have been kicked out. But what about the people who "championed it". If they worked with the student, surely they might have figured out the mythical lab full of 1000s material scientists might not exist, it might exist but they never actually used any AI tool.
That high-level Apple employee was probably a manager and oversaw hiring people.
I would tell myself every day, "I wouldn't hire me."
It's not self-defeating.
It's not being a victim.
I wouldn't let it stop me from trying.
It's being accurate about what kind of company you'd want to build yourself, and the internal state of a lot of hiring managers. And with a true model of the world you can make better decisions.
[0]: https://info.arxiv.org/help/withdraw.html: "Articles that have been announced and made public cannot be completely removed. A withdrawal creates a new version of the paper marked as withdrawn."
There was a question at the end that made him a little uncomfortable:
[1:00:20]
Q: Did you use academic labs only or did you use private labs?
A: (uncomfortable pause) Oh private, yeah, so like all corporate, yeah...
Q: So, no academic labs?
A: I think it's a good question (scratches head uncomfortably, seemingly trying to hide), what this would look like in an academic setting, cause like, ... the goals are driven by what product we're going make ... academia is all, like "we're looking around trying to create cool stuff"...
I strongly recommend people not investigate this unless they think 4chan is quaint. As in, if the reason you are not using X is because of the outrage at Elon and the "typical user" of X, then maybe use xcancel instead.
[1]: https://info.arxiv.org/help/withdraw.html#:~:text=Previous%2...
[2]: https://economics.mit.edu/news/assuring-accurate-research-re...
just post a correction notice on arXiv. let others decide if there is merit to it or not.
silencing is so anti-science. shame on MIT.