Reproducibility project fails to validate dozens of biomedical studies

Is there any path forward to fixing the current reproducibility crisis in science? Individuals can do better, but that won't solve a problem at this scale. Could we make systemic changes to how papers are validated and approved for publication in major journals?

Yeah "individuals do better" is never the answer -- you've got to structure incentives, of course.

I don't think you want to slow down publication (and probably peer review and prestiage journals are useless/obsolete in era of internet); it's already crazy slow.

So let's see: you want people to incentivize two things (1) no false claims in original research (2) to have people try to reproduce claims.

So here's a humble proposal for a funding source (say...the govt): set aside a pot of money specifically for people to try to reproduce research; let this be a valid career path. Your goal should try to be getting research validated by repro before OTHER research starts to build on those premises (avoiding having the whole field go off on wild goose chases like happened w/ Alzheheimer's). And then, when results DON'T repro, blackball the original researchers from funding. (With whatever sort of due process is needed to make this reasonable.)

I think it'd sort things out.

Punishing researchers who make mistakes or get unlucky due to noise in the data is a recipe for disaster, just like in other fields. The ideal amount of fraud and false claims in research is not zero, because the policing effort it would take to accomplish this goal would destroy all other forms of value. I can't emphasize enough how bad an idea blackballing researchers for publishing irreproducible results would be.

We have money to fund direct reproducibility studies (this one is an example), and indirect replication by applying othogonal methods to similar research topics can be more powerful than direct replication.