Supplements and Surveys, Correlation and Causation

I

In my attempts to find definitive evidence of supplements increasing cognition, I waded through more studies than I care to remember.  One frustration that I encountered fairly early on is that most vitamin and mineral data comes from longitudinal and cross-sectional surveys.  I can see why:  it takes a long time for some supplement levels to make a difference, and trying to run a six month to two-year double-blind supplement vs. placebo trial is no minor undertaking.  Better to use the vast sources of longitudinal data and infer from them.  More than one person I’ve talked to about those studies, however, has quoted the ever-present adage “correlation does not imply causation” criticism about them.  That got me curious as to exactly where that phrase came from, and how to apply it properly to all of these supplement long-term studies.

The phrase was coined by the very men we owe modern statistics to:  Karl Pearson and G. Udny Yule, who took the nascent science of 19^th-century correlation and made it practical in the 1900s.  He swiftly identified the problem of what he named spurious correlations, which are cases where the math makes it appear that two data sets are related but in fact it’s pure coincidence.  I got a chuckle out of this quote:

“Such high correlations as arise from common growth or decline with time, when interpreted as causal or semicausal relationships, are in our opinion perfectly idle, indeed are only too apt to be mischievous, and we shall reach nothing, or less than nothing–knighthoods–by the investigation of them [sic].” ^[1]Aldrich, John. “Correlations Genuine and Spurious in Pearson and Yule.” Statistical Science 10, no. 4 (November 1995): 364–76. https://doi.org/10.1214/ss/1177009870.

That makes me want to call all bad correlations mischievous—it has a better ring to it than spurious, right?  It’s also no surprise Pearson refused a knighthood when offered it.  More importantly, it’s also clear that, even when this branch of statistics was first forming, scientists knew full well to watch out for false correlations between them.  In other words, anyone whose first response to survey interpretations is “correlation does not imply causation” can easily be greeted with “Absence of evidence is not evidence of absence.” ^[2]“Project Cyclops: A Design Study of a System for Detecting Extraterrestrial Intelligent Life.” Accessed December 3, 2020. https://ntrs.nasa.gov/citations/19730010095.

I actually fully agree with the literal interpretation of the phrase; however, that needs to be clearly defined.  In mathematics, saying that something “implies” something else means there is likely a relationship.  What it doesn’t mean is that the relationship is proved.  In other words, “A implies B” and “Because of A, then B” are not one and the same.

All right then, enough history and background, let’s apply this to supplement surveys.  The trend I’ve seen over and over again is that they’ll have a large sample size, which is a good start, and then they’ll compare data among supplement use, demographics, and health markers.

It’s in the comparison part that the situation gets murky.  Supplement users do tend to have better health outcomes; however, they also have different demographics, and those very demographics would explain some of the health differences.  Long story short, educated high-income folk take more supplements and have better health markers . . . but they as a demographic already do so in general, for various reasons such as being able to afford better health care in the first place. (You can read a more detailed analysis of a study that shows this behavior in https://www.adaptabledeveloper.com/the-demographics-of-multi-supplement-daily-users/)