←back to thread

MIT asks arXiv to withdraw preprint of paper on AI and scientific discovery

(economics.mit.edu)
284 points carabiner | 2 comments | 16 May 25 15:09 UTC | HN request time: 0.412s | source
Show context
tokai ◴[16 May 25 19:29 UTC] No.44009006[source]▶
Impressively the paper seems to have been cited 50 times already. I don't mind much if its taken down or not, but with the old guard publishers you can at least get a redaction notice or comment about the issues with a paper embedded in the publication. If you find this paper cited somewhere and follow it to the source at arxiv, you will never be made aware of the disputes surrounding the research. Preprint servers has somewhat of a weakness here.
replies(2): >>44009954 #>>44011687 #
1. forgotpwd16 ◴[16 May 25 21:29 UTC] No.44009954[source]▶
A weakness that goes hand-in-hand with the lack of peer review. There's moderation but shouldn't be considered equivalent to it. Trusting the study means trusting the author or reviewing the paper yourself. If a withdraw happens, either the author comments on why they did it[0] or, similarly to previous, you've to search it yourself.

[0] E.g. arxiv/0812.0848: "This paper has been withdrawn by the author due to a crucial definition error of Triebel space".

replies(1): >>44011134 #
2. Majromax ◴[17 May 25 00:45 UTC] No.44011134[source]▶
> A weakness that goes hand-in-hand with the lack of peer review

Peer review is not well equipped to catch fraud and deliberate deception. A half-competent fraud will result in data that looks reasonable at first glance, and peer reviewers aren't in the business of trying to replicate studies and results.

Instead, peer review is better at catching papers that either have internal quality problems (e.g. proofs or arguments that don't prove what they claim to prove) or are missing links to a crucial part of the literature (e.g. claiming an already-known result as novel). Here, the value of peer review is more ambiguous. It certainly improves the quality of the paper, but it also delays its publication by a few months.

The machine learning literature gets around this by having almost everything available in preprint with peer-reviewed conferences acting as post-facto gatekeepers, but that just reintroduces the problem of non-peer-reviewed research being seen and cited.