The effect of deactivating Facebook and Instagram on users' emotional state

> People who deactivated Facebook for the six weeks before the election reported a 0.060 standard deviation improvement in an index of happiness, depression, and anxiety, relative to controls who deactivated for just the first of those six weeks. People who deactivated Instagram for those six weeks reported a 0.041 standard deviation improvement relative to controls.

Can anyone translate? Random web search find suggests multiplying by 37 to get a percentage, which sounds very questionable, but even then these improvements seem negligible.

This doesn't really line up with my lived experience. Getting myself out of shitty platforms and community spaces improved my mental state significantly (although the damage that's been done remains).

From the paper PDF (https://www.nber.org/system/files/working_papers/w33697/w336...):

> We estimate that users in the Facebook deactivation group reported a 0.060 standard deviation improvement in an index of happiness, anxiety, and depression, relative to control users. The effect is statistically distinguishable from zero at the p < 0.01 level, both when considered individually and after adjusting for multiple hypothesis testing along with the full set of political outcomes considered in Allcott et al. (2024). Non-preregistered subgroup analyses suggest larger effects of Facebook on people over 35, undecided voters, and people without a college degree.

> We estimate that users in the Instagram deactivation group reported a 0.041 standard deviation improvement in the emotional state index relative to control. The effect is statistically distinguishable from zero at the p = 0.016 level when considered individually, and at the p = 0.14 level after adjusting for multiple hypothesis testing along with the outcomes in Allcott et al. (2024). The latter estimate does not meet our pre-registered p = 0.05 significance threshold. Substitution analyses imply this improvement is achieved without shifts to offline activities. Non-preregistered subgroup analyses suggest larger effects of Instagram on women aged 1824.

Perhaps it wasn't clear what I meant. When I said significantly, I meant it in the colloquial sense, not in the statistical significance sense.

I was looking for a more digestable figure describing the extent of improvements, not whether the study found them confidently distinguishable (which I just assumed they did based on the wording, good to know they didn't for Instagram).

A 0.060 standard deviation improvement is super small. If the average person rates their happiness/anxiety/depression score at, say, 50 out of 100, and the standard deviation (how spread out people’s scores are) is around 10 points, then 0.060 SD = 0.6 points. So quitting Facebook gave an average person a ~1% bump in mood score. Instagram was even smaller: ~0.4 points, or 0.8%.

It's real, but barely noticeable for most people—unless you're in a more affected subgroup (e.g. undecided voters or younger women). Your experience feeling way better likely means you were an outlier (in a good way).

It means that there is a statistically significant improvement, but that improvement is tiny, and will not make you happier than your peers all by itself (assuming a standard peer group of 200 people - you'd likely swap places with 1 or 2 people).

Of course, this study only considered normative people, not marginalized or those who were experiencing active harm from exposure to social media - your personal results may vary and it's important to remember that science is imperfect and social sciences are doubly so.

If going off Facebook improves your life - you do you.

If I understand you, just read the paper for its analysis and interpretation of those numbers.

Alternatively, you'll want to grasp the meaning of "standard deviation" (you're right that you can't multiply all standard deviations by a number and get a percentage - and a percentage of what?), and then find the "index of happiness, depression, and anxiety" they use and grasp its meaning.

Even if it's statistically significant, it's a laughably tiny effect.

Like have one nice meal or a one walk in the woods 2 months ago and rate your mood today kind of effect size.

0.06 std deviation is not anything to write home about and really doubtfully real, given the general quality of psychological science.

Perhaps, how much better of a day would you have if you found a dollar on the ground.

On what scale? What do 'points' on the scale mean? Without knowing those things, we can't say what 6 or 60 points mean.

This is what I was interested in, thank you!

I'm not sure you understood me. I want to specifically avoid doing all that, to save time and effort.

So, ELI5 level.

People who use Facebook also may feel depression, from very strong to none at all. In the middle of this interval there's the "expected value" point, sort of an average level of feeling depressed. This point is at an equal distance from the "most depressed users" group, and from the "not depressed at all" group. Let's call this distance of depression strength a "standard deviation".

Now, the users who stopped using Facebook became slightly less depressed, by 6% of that "standard deviation" range. If you buy a small coke at a McDonald's, then take one sip, you make it about 6% smaller. It's not unnoticeable (you've made that refreshing sip), but about 15 more such sips still remain!

In other words, there is an effect which can definitely be noticed ("statistically significant"), but it's not a big deal either.

The best thing you can do is compare it to another study, since turning 0.06 standard deviations into a percentage of happiness isn’t going to be that telling.

In general, 0.2 is considered a small effect. So 0.06 is quite small — likely not a practically noticeable change in well-being. But impressive to me when I compare it to effect sizes of therapy interventions which can lie around 0.3 for 12 weeks.

Quote:

> “50 randomized controlled trials that were published in 51 articles between 1998 and August 2018. We found standardized mean differences of Hedges’ g = 0.34 for subjective well-being, Hedges’ g = 0.39 for psychological well-being, indicating small to moderate effects, and Hedges’ g = 0.29 for depression, and Hedges’ g = 0.35 for anxiety and stress, indicating small effects.”

(Source: The efficacy of multi-component positive psychology interventions, 2019 — https://www.researchgate.net/publication/331028589_The_Effic...)

On the contrary, reporting changes relative to the standard deviation of a control group frees you from scales and their meanings, because it relates the observed change to the normal spread of scores before the intervention. In this way, you don't need to know the scale and its meaning to know if a change is big or small, and from a statistical perspective, that's (almost) all you need to find if a change is significant or due to random chance. Of course, looking back at the original scale and its meaning can help interpreting the meaning of the results in other ways

As far as I can tell, the algorithm can really harm people during times of mental illness/stress/anxiety. Part of it is that it is like a feedback loop.

When we lost our pet and my wife was very upset for a while, the algo kept showing her more and more content associated with pet loss. It got to the point that some random content pushed to her social media was upsetting her daily.

I can imagine someone experiencing depression, suicidal thoughts, etc can easily be pushed over the edge by the algorithmic feedback loop.

This is a very useful insight, thank you. Wouldn't have occurred to me to check something like that.

In a way this perfectly captures my experiences too, despite my struggles revolving around a different topic, and sometimes it wouldn't even be algorithmically inflicted, but self-inflicted.

I'd keep coming across, and sometimes seeking out, threads with political content. But beyond that, I'd keep stumbling upon or even seeking out people who are being (in my view) inciteful or misleading. This would then piss me off, and I'd start to spiral. Naturally, these are not the kind of people who'd be posting in good faith, adding even more fuel to the fire when I engaged with them and their replies would eventually come about, which of course I'd "helpfully" get a notification for.

>Getting myself out of shitty platforms and community spaces improved my mental state significantly

True. I've experienced it too

Multiply by .37 to get PERCENTILE change, not percent.

If you were average happiness, and you improve that by 1 stdev, you are now happier than 87% of your peers (when you were at 50%ile before). 0.6 stdev improvement would be vs 72% of your peers.

So to put it colloquially, if you have 7 friends, and you were in the middle of them (4th happiest), by quitting Facebook you are now happier than all but 1 one them.

Multiply by .37 to get PERCENTILE ranking change, not percent. If you were average happiness, and you improve that by 1 stdev, you are now happier than 87% of your peers (when you were at 50%ile before). 0.6 stdev improvement would be vs 72% of your peers.

So to put it colloquially, if you have 4 friends, and you were in the middle of them (3rd happiest aka happier than 2 of them), by quitting Facebook you are now happier than all but 1 one them (aka happier than 3 of them).

AKA for every 4 friends you have you can jump ahead of 1 of them in the happiness race by quitting facebook.

Standard deviation helps, but you still need to know: standard deviation of what? It's no different than saying someone scored 78% - 78% of what? What is it in the denominator? Also, different scales can represent the same thing differently.

Secondly, the impact of the difference isn't known - you don't know the curve representing the relationship of score to impact. In some contexts a little change is meaningless - the curve is flat; in others the curve is steep and it can be transformational. And impacts only sometimes scale linearly with performance or score, of course.

Without that knowledge, standard deviation means nothing beyond how unusual, in the given population, the subject's performance is.

Alas, I don't know a faster way. The question asked, iiuc, demonstrates a lack of understanding of standard deviation. That's fine; none of us know everything. But without that we can't intepret the results, and also necessary is understanding what the scale represents. Thus the fastest solution seems to be reading the author's interpretation rather than trying to do it yourself.

Caveat: I'm not very smart

So, if 0.3 is 12 weeks of therapy, then 0.06 is ~2.5 weeks of therapy (0.3/0.06 = 2.4), assuming you pick any 2.5 random weeks of the 12 week course.

Yes, I'm sure the first session is the most important and then a logarithmic curve of blah blah blah.

Essentially, deleting FB is not much, but it's not nothing either.