←back to thread

504 points puttycat | 1 comments | | HN request time: 0.191s | source
Show context
MarkusQ ◴[] No.46182578[source]
This is as much a failing of "peer review" as anything. Importantly, it is an intrinsic failure, which won't go away even if LLMs were to go away completely.

Peer review doesn't catch errors.

Acting as if it does, and thus assuming the fact of publication (and where it was published) are indicators of veracity is simply unfounded. We need to go back to the food fight system where everyone publishes whatever they want, their colleagues and other adversaries try their best to shred them, and the winners are the ones that stand up to the maelstrom. It's messy, but it forces critics to put forth their arguments rather than quietly gatekeeping, passing what they approve of, suppressing what they don't.

replies(5): >>46182677 #>>46182703 #>>46182872 #>>46183274 #>>46183453 #
watwut ◴[] No.46182872[source]
Peer review was never supposed to check every single detail and every single citation. They are not proof readers. They are not even really supposed to agree or disagree with your results. They should check the soundness of a method, general structure of a paper, that sort of thing. They do catch some errors, but the expectation is not to do another independent study or something.

Passed peer review is the first basic bar that has to be cleared. It was never supposed to be all there is to the science.

replies(2): >>46183577 #>>46200873 #
1. MarkusQ ◴[] No.46200873[source]
Agreed. But too often it's treated as a golden ticket confirmation of veracity, giving the process more epistemological authority than it warrants.