←back to thread

The effect of deactivating Facebook and Instagram on users' emotional state

(www.nber.org)
506 points imakwana | 2 comments | 21 Apr 25 04:24 UTC | HN request time: 0s | source
Show context
perching_aix ◴[21 Apr 25 05:11 UTC] No.43748660[source]▶
> People who deactivated Facebook for the six weeks before the election reported a 0.060 standard deviation improvement in an index of happiness, depression, and anxiety, relative to controls who deactivated for just the first of those six weeks. People who deactivated Instagram for those six weeks reported a 0.041 standard deviation improvement relative to controls.

Can anyone translate? Random web search find suggests multiplying by 37 to get a percentage, which sounds very questionable, but even then these improvements seem negligible.

This doesn't really line up with my lived experience. Getting myself out of shitty platforms and community spaces improved my mental state significantly (although the damage that's been done remains).

replies(5): >>43748693 #>>43748786 #>>43748791 #>>43748803 #>>43749682 #
SamvitJ ◴[21 Apr 25 05:20 UTC] No.43748693[source]▶
From the paper PDF (https://www.nber.org/system/files/working_papers/w33697/w336...):

> We estimate that users in the Facebook deactivation group reported a 0.060 standard deviation improvement in an index of happiness, anxiety, and depression, relative to control users. The effect is statistically distinguishable from zero at the p < 0.01 level, both when considered individually and after adjusting for multiple hypothesis testing along with the full set of political outcomes considered in Allcott et al. (2024). Non-preregistered subgroup analyses suggest larger effects of Facebook on people over 35, undecided voters, and people without a college degree.

> We estimate that users in the Instagram deactivation group reported a 0.041 standard deviation improvement in the emotional state index relative to control. The effect is statistically distinguishable from zero at the p = 0.016 level when considered individually, and at the p = 0.14 level after adjusting for multiple hypothesis testing along with the outcomes in Allcott et al. (2024). The latter estimate does not meet our pre-registered p = 0.05 significance threshold. Substitution analyses imply this improvement is achieved without shifts to offline activities. Non-preregistered subgroup analyses suggest larger effects of Instagram on women aged 1824.

replies(2): >>43748710 #>>43748877 #
perching_aix ◴[21 Apr 25 05:25 UTC] No.43748710[source]▶
Perhaps it wasn't clear what I meant. When I said significantly, I meant it in the colloquial sense, not in the statistical significance sense.

I was looking for a more digestable figure describing the extent of improvements, not whether the study found them confidently distinguishable (which I just assumed they did based on the wording, good to know they didn't for Instagram).

replies(5): >>43748777 #>>43748809 #>>43748915 #>>43754142 #>>43754150 #
kacesensitive ◴[21 Apr 25 05:44 UTC] No.43748777{3}[source]▶
A 0.060 standard deviation improvement is super small. If the average person rates their happiness/anxiety/depression score at, say, 50 out of 100, and the standard deviation (how spread out people’s scores are) is around 10 points, then 0.060 SD = 0.6 points. So quitting Facebook gave an average person a ~1% bump in mood score. Instagram was even smaller: ~0.4 points, or 0.8%.

It's real, but barely noticeable for most people—unless you're in a more affected subgroup (e.g. undecided voters or younger women). Your experience feeling way better likely means you were an outlier (in a good way).

replies(2): >>43748804 #>>43748805 #
mmooss ◴[21 Apr 25 05:50 UTC] No.43748804{4}[source]▶
On what scale? What do 'points' on the scale mean? Without knowing those things, we can't say what 6 or 60 points mean.
replies(1): >>43748989 #
1. blackbear_ ◴[21 Apr 25 06:36 UTC] No.43748989{5}[source]▶
On the contrary, reporting changes relative to the standard deviation of a control group frees you from scales and their meanings, because it relates the observed change to the normal spread of scores before the intervention. In this way, you don't need to know the scale and its meaning to know if a change is big or small, and from a statistical perspective, that's (almost) all you need to find if a change is significant or due to random chance. Of course, looking back at the original scale and its meaning can help interpreting the meaning of the results in other ways
replies(1): >>43754394 #
2. mmooss ◴[21 Apr 25 17:34 UTC] No.43754394[source]▶
Standard deviation helps, but you still need to know: standard deviation of what? It's no different than saying someone scored 78% - 78% of what? What is it in the denominator? Also, different scales can represent the same thing differently.

Secondly, the impact of the difference isn't known - you don't know the curve representing the relationship of score to impact. In some contexts a little change is meaningless - the curve is flat; in others the curve is steep and it can be transformational. And impacts only sometimes scale linearly with performance or score, of course.

Without that knowledge, standard deviation means nothing beyond how unusual, in the given population, the subject's performance is.