www.technotheoria.org/p/caffeine-works-in-some-people
1 correction found
aka 16 IQ points
The cited meta-analysis did not measure IQ, and its effect sizes are standardized mean differences from tasks like attention, reaction time, and driving performance—not IQ-point changes.
Full reasoning
This is an invalid conversion.
The cited paper is a meta-analysis of attention tests, executive function, reaction time, information-processing tasks, and driving measures after sleep loss—not IQ tests. Its reported Hedges’ g values are standardized mean differences (SMDs) for those outcomes.
Methodologically, an SMD is a unitless effect size. As the Cochrane Handbook explains, the standardized mean difference is used when studies measure outcomes in different ways, and the overall effect is reported in units of standard deviations rather than in the original units of any measurement scale. That means you cannot simply relabel an SMD from heterogeneous cognitive and driving tasks as “IQ points.”
So even if one computes some average standardized effect from the paper, calling it “16 IQ points” is incorrect because the study did not estimate IQ change at all.
2 sources
- Effects of acute caffeine consumption following sleep loss on cognitive, physical, occupational and driving performance: A systematic review and meta-analysis
45 publications providing 327 effect estimates (EEs) were included... Caffeine improved response time and accuracy on attention tests, improved executive function, improved reaction time and response time on information processing tasks, and enhanced lateral and longitudinal measures of vehicular control on driving tests.
- Chapter 6: Choosing effect measures and computing estimates of effect | Cochrane
The standardized mean difference (SMD) is used as a summary statistic in meta-analysis when the studies all assess the same outcome, but measure it in a variety of ways... The overall intervention effect can also be difficult to interpret as it is reported in units of SD rather than in units of any of the measurement scales used in the review.