My 16-month theanine self-experiment

This is great. The author defines their own metrics, is doing their own A/B tests and publishes their interpretation plus the raw data. Imagine a world where all health blogging was like that.

Personally, I have not published any results yet, but I have been doing this type of experiments for 4 years now. And collected 48874 data points so far. I built a simple system to do it in Vim:

https://www.gibney.org/a_syntax_for_self-tracking

I also built a bunch of tooling to analyze the data.

I think that mankind could greatly benefit from more people doing randomized studies on their own. Especially if we find a way to collectively interpret the data.

So I really applaud the author for conducting this and especially for providing the raw data.

Reading through the article and the comments here on HN, I wish there was more focus on the interpretation of the experiment. Pretty much all comments here seem to be anecdotal.

Let's look at the author's interpretation. Personally, I find that part a bit short.

They calculated 4 p-values and write:

    Technically, I did find two significant results.

I wonder what "Technically" means here. Are there "significant results" that are "better" than just "technically significant results"?

Then they continue:

    Of course, I don’t think this
    means I’ve proven theanine is harmful.

So what does it mean? What was the goal of collecting the data? What would the interpretation have been if the data would show a significant positive effect of Theanine?

It's great that they offer the raw data. I look forward to taking a look at it later today.

Citizens science used to be much more common and there were several publications initially built around the concept. Unfortunately, the idea has all but been obliterated by inadequate public education and the chronic lack of time that now embattles the middle classes.

fortunately* citzen science was replaced by solid methods... this is a long fancy self reported post on a metric that is easily figured out by biomarkers (stress levels). what's the consequences if the author decided to make some claim about a substance?

I think you are ignoring the fact that citizen science was alive and very well up into the 1980s. It was never a replacement for professional science, but rather citizen scientists often observed things that were deemed worth of follow-up by academic researchers.

Citizens scientists rarely post “facts” but rather interesting avenues for research or further investigation. Part of being an educated citizen scientist is to understand the limitations of your knowledge, data, and methods.

Quacks and cranks, on the other hand, are always making grand new “discoveries” lol.

Anyone who has a decent education can make observations apply the scientific method. I say this coming from a family of actual scientists from molecular biology to particle physics, who will tell you the same, and also give credit to the multitude of citizen scientists who have done just that.

As for myself, I’ll stick to engineering.

>I think you are ignoring the fact that citizen science was alive and very well up into the 1980s.

Some version of that still exists among people with a naturalist bent recording observations of reptiles, birds, plants, etc. But yeah, we don't really have backyard chemists analyzing things anymore.