Reproducibility project fails to validate dozens of biomedical studies

This doesn’t really surprise me at all. It’s an unrelated field, but part of the reason I got completely disillusioned with research to the point I switched out of a program with a thesis was because I started noticing reproducibility problems in published work. My field is CS/CE, generally papers reference publicly available datasets and can be easily replicated… except I kept finding papers with results I couldn’t recreate. It’s possible I made mistakes (what does a college student know, after all), but usually there were other systemic problems on top of reproducibility. A secondary trait I would often notice is a complete exclusion of [easily intuited] counter-facts because they cut into the paper’s claim.

To my mind there is a nasty pressure that exists for some professions/careers, where publishing becomes essential. Because it’s essential, standards are relaxed and barriers lowered, leading to the lower quality work being published. Publishing isn’t done in response to genuine discovery or innovation, it’s done because boxes need to be checked. Publishers won’t change because they benefit from this system, authors won’t change because they’re bound to the system.

This same post appears at the top of every single HN story on reproducibility. “I was a student in [totally unrelated field] and found reproducibility to be difficult. I didn’t investigate it deeply and ultimately I left the field, not because I was unsuccessful, of course, but because I understood deeply despite my own extremely limited experience in the area that all of the science was deeply flawed if not false.”

Imagine the guy who got a FAANG job and made it nine weeks in before washing out, informing you how the entire industry doesn’t know how to write code. Maybe they’re right and the industry doesn’t know how to write code! But I want to hear it from the person who actually made a career, not the intern who made it through part of a summer.