←back to thread

247 points rntn | 7 comments | | HN request time: 0.011s | source | bottom
Show context
N_A_T_E ◴[] No.43795965[source]
Is there any path forward to fixing the current reproducibility crisis in science? Individuals can do better, but that won't solve a problem at this scale. Could we make systemic changes to how papers are validated and approved for publication in major journals?
replies(12): >>43796160 #>>43796211 #>>43796313 #>>43796358 #>>43796415 #>>43796725 #>>43796906 #>>43796908 #>>43796955 #>>43797084 #>>43797605 #>>43797627 #
_aavaa_ ◴[] No.43796725[source]
Pre-registration is a pretty big one: essential you outline your research plan (what you’re looking for, how you will analyze the data, what bars you are setting for significance, etc.) before you do any research. You plan is reviewed and accepted (or denied), often by both funding agency and journal you want to submit to, before they know the results.

Then you perform the experiment exactly* how you said you would based on the pre-registration, and you get to publish your results whether they are positive or negative.

* Changes are allowed, but must be explicitly called out and a valid reason given.

https://en.wikipedia.org/wiki/Preregistration_(science)

replies(3): >>43798108 #>>43798882 #>>43803663 #
1. poincaredisk ◴[] No.43798882[source]
Wow, I didn't think it's possible, but it sounds like a great way to make research boring :).
replies(1): >>43799643 #
2. _aavaa_ ◴[] No.43799643[source]
What is boring about this? You get the guarantee of publishing your work, even if you get a negative result.

I take it you don’t do research. Cause boring is nothing compared to wasting month of time and money only to get a negative result that nobody will publish.

replies(2): >>43799898 #>>43799899 #
3. ◴[] No.43799898[source]
4. poincaredisk ◴[] No.43799899[source]
I'm not in the academy, but I do R&D, I published several times, and that's not how I work at all.

I have a broad and open-ended focus, I work as usual on the things I find interesting, then sometimes I see a thing that looks interesting and decide to investigate, then sometimes my initial tests give good results, but more often then they don't, but they give me an idea to do something completely different, and some iterations later I have a result.

I imagine that depends on a field of research. IT is cheap, but I imagine a physicist who wants to do an experiment must secure a funding first, because otherwise it's impossible to do anything. And it requires one to commit to a single topic of research.

replies(1): >>43800035 #
5. _aavaa_ ◴[] No.43800035{3}[source]
> but more often then they don't

That part is true in all fields. And one of the things that pre-registration enables is the publishing of those negative results.

Otherwise, once you're done the research and got the negative result nobody wants to publish it (unless it’s very flashy). Without being able to publish negative results, and therefore read about them, each researcher must conduct an experiment already known, if only in private, to not work.

replies(1): >>43802474 #
6. Scarblac ◴[] No.43802474{4}[source]
So he has to take time to write up those experiments he was doing as exploration, and nobody will read them because who wants to spend time reading about failed experiments.

People will still want to do their own exploring to get a feel for a problem.

replies(1): >>43803222 #
7. _aavaa_ ◴[] No.43803222{5}[source]
> who wants to spend time reading about a failed experiment.

A negative result is *not* a failed experiment. Mapping out the negative space is very valuable if the hypotheses being tested are worthwhile ones.