“Most promising signs yet” of alien life on a planet beyond our Solar System

I'm not convinced about the methods. It looks a lot like p-hacking to me: they have a highly specific hypothesis drawn from a large universe—that dozen or so molecules (§3.1) in their infrared spectrum model they're fitting experimental data against. I don't buy the way they created that hypothesis. The put a handful of highly specific biosignature gases into it, things that were proposed by exobiology theory papers. One very specific hypothesis out of many, and a low likelihood one. And that's the hypothesis they get some borderline ~3σ signals for? Really?

edit: Any chance someone might have the charity to explain why my criticism is so far off-base, according to the HN consensus?

So if I understand correctly, this[1] is what they did?

[1] https://xkcd.com/882/

I'm going to double-down on my stubborn, unpopular opinion. This is my best attempt at explaining my criticism:

- Alien metabolites are a low-prior probability hypothesis. Dimethyl sulfide is a long-postulated biosignature with no natural source, so, it's low-prior

- The paper's model fits Webb data—a handful of photons—against no more than 20 candidate molecules, combined across all of their atmospheric models. Many of those gases are drawn from that low-prior "alien metabolite" class

- There's a much larger class of strongly infrared-absorbing gases, that can naturally occur in planetary atmospheres. Beyond those included in the 20 candidates. These (should!) have higher prior probability of occurring in Webb data than alien metabolites. (This class is so large and complicated, there's major spectral features in our own solar system's gas planets we haven't characterized yet)

- If you were to fit Webb data against that expanded class, those alternative hypotheses, you'd get a large number of 3-sigma detections by pure chance.

- The Webb data is too weak to distinguish between these. With only a few bits of information, you can distinguish between only a small set of alternative hypotheses

- This paper elevates the alien-metabolite hypothesis very highly, and that is why when it has a spurious statistical detection, it happens to be an alien metabolite detection. Because that hypothesis is overrepresented in their model

- The root problem is that since there's only a trickle of real data from this exoplanet, from Webb, it's unlikely one can infer anything super interesting from those few bits

Don't be bothered by the down votes. HN consensus is not something worth pursuing. Your criticism is valid, it's just that it runs against what HN readers want to believe in this instance. Readers here like to think they're motivated by reason and intelligence and whatnot, but that is laughable - examples of logical fallacies and assertions of fact rocketing to the top comments abound. Overconfidence and readiness to accept bold claims is a more dangerous cultural dysfunction than the lack of seriousness and ubiquitous monetization that plagues other platforms.

In any case this study will likely go on the pile of papers judged by time to be an overreach of conclusions and a dead end.

False positives are acceptable if the goal is to generate leads to follow up on. If the detection was due to chance then it won't hold up to further measurement. There's few enough hits that we don't need to worry about being more rigorous (and potentially introducing false negatives) at an earlier stage.

Given the context, a publication seems appropriate. A high profile similar example is when neutrinos supposedly broke the light speed barrier. If the mass media misrepresents things that's hardly the fault of the scientists.