Most active commenters
  • mg(3)

←back to thread

My 16-month theanine self-experiment

(dynomight.net)
817 points dynm | 15 comments | 09 Mar 25 03:08 UTC | HN request time: 0.638s | source | bottom
Show context
mg ◴[09 Mar 25 08:26 UTC] No.43307263[source]▶
This is great. The author defines their own metrics, is doing their own A/B tests and publishes their interpretation plus the raw data. Imagine a world where all health blogging was like that.

Personally, I have not published any results yet, but I have been doing this type of experiments for 4 years now. And collected 48874 data points so far. I built a simple system to do it in Vim:

https://www.gibney.org/a_syntax_for_self-tracking

I also built a bunch of tooling to analyze the data.

I think that mankind could greatly benefit from more people doing randomized studies on their own. Especially if we find a way to collectively interpret the data.

So I really applaud the author for conducting this and especially for providing the raw data.

Reading through the article and the comments here on HN, I wish there was more focus on the interpretation of the experiment. Pretty much all comments here seem to be anecdotal.

Let's look at the author's interpretation. Personally, I find that part a bit short.

They calculated 4 p-values and write:

    Technically, I did find two significant results.
I wonder what "Technically" means here. Are there "significant results" that are "better" than just "technically significant results"?

Then they continue:

    Of course, I don’t think this
    means I’ve proven theanine is harmful.
So what does it mean? What was the goal of collecting the data? What would the interpretation have been if the data would show a significant positive effect of Theanine?

It's great that they offer the raw data. I look forward to taking a look at it later today.

replies(14): >>43307304 #>>43307775 #>>43307806 #>>43307937 #>>43308201 #>>43308318 #>>43308320 #>>43308521 #>>43308854 #>>43309271 #>>43310099 #>>43320433 #>>43333903 #>>43380374 #
1. kortilla ◴[09 Mar 25 10:19 UTC] No.43307806[source]▶
Well the issue is that an experiment with 1 person can’t prove much because you can’t have a control group.

Too much other stuff is changing in a single persons life that could account for all observed side effects.

You also have latent side effect issues. A person could smoke for 10 years, not smoke for another 10, and then conclude that smoking doesn’t cause cancer. Then they get lung cancer 20 years later.

Excellent data and statistics is not sufficient for a good experiment

replies(3): >>43307871 #>>43309673 #>>43316721 #
2. mg ◴[09 Mar 25 10:41 UTC] No.43307871[source]▶
An experiment of 1 person can very well produce useful data.

It depends on the setup of the experiment.

Imagine an experiment where a person's thumb gets randomly hit with either a hammer or a feather once per day. And they then subjectively rate the experience. After 1000 days of collecting data, I doubt that we would wrongly come to the conclusion that the hammer treatment leads to the nicer outcome.

The setup of the Theanine experiment which is the basis of this thread looks good on first sight. I have the feeling that the interpretation could use more thought though.

replies(8): >>43307925 #>>43308004 #>>43308014 #>>43308106 #>>43308117 #>>43308394 #>>43309398 #>>43322506 #
3. ◴[09 Mar 25 10:54 UTC] No.43307925[source]▶
4. tomalbrc ◴[09 Mar 25 11:09 UTC] No.43308004[source]▶
What a weird take, in 99.999% of cases you don’t have such a black/white contrast
replies(1): >>43308048 #
5. ◴[09 Mar 25 11:11 UTC] No.43308014[source]▶
6. mg ◴[09 Mar 25 11:17 UTC] No.43308048{3}[source]▶
Sure. But even when you add noise to the described experiment, you get useful data.

That is the point I am making: Experiments of a single person can be useful.

The critics of single person experiments usually come up with examples vastly different than the Theanine experiment described here. With long term experiments which are only conducted once. But the Theanine experiment was looking for a short term effect and can be conducted many times. The hammer experiment I made up would be an extreme example of this type of experiment which leads itself well to be conducted by a single person.

What I am trying to point out is that if you are a skeptic, it would be better to try and find weaknesses in the experiment at hand. Not making up completely different experiments.

7. pbhjpbhj ◴[09 Mar 25 11:28 UTC] No.43308106[source]▶
Then you find {imagining scenarios} that the bruising causes some effect that 'inoculates' against heart disease, or the feather carries a pathogen that later induces dementia??
8. chiefalchemist ◴[09 Mar 25 11:30 UTC] No.43308117[source]▶
Yes and no. The issue with a self-administered experiment is that becomes part of the experiment. Does the self-administration and the associated thoughts and beliefs affect the results? It’s not like the experimentor can issue a placebo to themselves.
replies(1): >>43309099 #
9. chkgk ◴[09 Mar 25 12:10 UTC] No.43308394[source]▶
A problem I see is that you can never be sure that it also works for other people. Just because one person reacts to a specific treatment does not mean that people on average react to the treatment in the same way. That’s the problem of having a sample size of 1. In other words: We cannot say much about whether the effect generalizes to a larger population from knowing that it has an affect on just one person.
replies(1): >>43309149 #
10. dublinben ◴[09 Mar 25 13:58 UTC] No.43309099{3}[source]▶
>It’s not like the experimentor can issue a placebo to themselves.

TFA describes a protocol for doing just that. The author randomly selected between the treatment dose and placebo. They didn't reveal the choice until after the effect should be complete so they could record the results.

11. yimby2001 ◴[09 Mar 25 14:08 UTC] No.43309149{3}[source]▶
It’s very possible that some things can be good for one person and not for another. Should we only ever have drugs that can help everyone?
12. VerdisQuo5678 ◴[09 Mar 25 14:41 UTC] No.43309398[source]▶
i hope your 1 person isn't a masochist
13. Zak ◴[09 Mar 25 15:09 UTC] No.43309673[source]▶
The experiment answers a useful question: does theanine have a strong, acute anti-stress effect on the experimenter?

Effect size is the key element of this question. If the substance was alprazolam (Xanax) instead of theanine, we would almost certainly see a strong effect here. The same would be true for heroin, ethanol, or cocaine.

It's not trying to test whether there might be a strong effect for most people or whether there are any side effects. Other experiments have been done with theanine seeking to answer those questions; the answers appear to be no and no.

14. eru ◴[10 Mar 25 03:24 UTC] No.43316721[source]▶
> Too much other stuff is changing in a single persons life that could account for all observed side effects.

> You also have latent side effect issues. A person could smoke for 10 years, not smoke for another 10, and then conclude that smoking doesn’t cause cancer. Then they get lung cancer 20 years later.

But that's not the actual experiment here. Your criticism is invalid. The author randomised every day which supplement to take.

That's pretty close to a blocked design.

15. Suppafly ◴[10 Mar 25 16:57 UTC] No.43322506[source]▶
A similar, actually useful, experiment is the one where a guy cracked the knuckles of only one hand for several years and found no ill effects from doing so compared to the other hand with non-cracked knuckles.