Statistics Pitfall #3: Correlation vs. Causation

There are plenty of arguments for letting data guide your decisions. “Numbers don't lie” is a phrase you hear often. At first glance, that's entirely true. Data is an unbiased reflection of reality and a good check on assumptions and gut feelings. Unfortunately, it's not always easy to interpret data correctly. It's no coincidence that universities in the Netherlands, almost without exception, fill part of their curriculum with statistics courses. Anyone who works with data runs the risk of falling into one of many pitfalls. In this series of blogs, we'll explain a number of common pitfalls and give you concrete tips to avoid them. In this third blog of the series, we discuss 'Correlation vs. Causation'.

Why finding a relationship is not the end but the beginning of research:

You know the drill: one moment a study is published claiming that a glass of red wine a day is good for your health, and the next day a study comes out contradicting it immediately. It seems to happen very often that a relationship found in food research later turns out not to be true after all. How does that happen? The answer is that many scientific researchers, too, regularly fall into one of the biggest statistical pitfalls there is: a correlation between 2 factors is not the same as a causal relationship.

What do correlation and causation actually mean?

For anyone whose head is spinning right now, don't worry, we'll explain this step by step. First, we need to look at what these two phenomena mean. Put simply, a correlation means that two phenomena we're studying can be said to often occur at the same time or to the same degree. A well-known example is the observation that when the number of ice creams sold goes up, the number of people who die from drowning also rises proportionally.

In that case, there's a positive correlation between the phenomenon “number of ice creams sold” and “number of drowning deaths.” Conversely, with a negative correlation, when one phenomenon rises, the other falls. As soon as there's a correlation between two phenomena, the next question is usually whether one phenomenon might be the cause of the other, in other words: is there a causal relationship?

Looking at our example of ice cream sales and the number of drowning deaths, it doesn't seem realistic that ice cream causes drowning. Nor does it sound logical that people buy more ice cream when they see others drowning. In this example, there's secretly a third phenomenon we haven't accounted for in our “research”: the weather. The warmer the weather, the more ice cream is sold because people are looking to cool off. Likewise, more people go swimming, and the more people swim, the greater the chance that something goes wrong.

Causal relationship

For good research, a correlation between two phenomena is therefore only a starting point. Once you see that two effects are correlated, the next step is to find out whether a causal relationship also exists. Only then do you know whether reducing phenomenon A will actually have an effect on phenomenon B. Based on our ice cream study, you could decide to ban all ice cream sales in the hope of reducing the number of drownings, but you'll probably soon see that it doesn't really help.

Unfortunately, there are many examples where correlation is confused with causation. The tricky part is that there are quite a few phenomena that can easily be correlated with each other. So easily, in fact, that websites exist where you can construct the most bizarre correlations yourself using real data, such as this site by author Tyler Vigen, who has also written an entire book full of charts like this one:

Another classic example of a correlation that occurs very often is the fact that a lot of geographic data is strongly correlated with population density. In the clip below, comedian John Oliver shows how such correlations are used as the basis for, for example, the “5G conspiracy,” which claims that the rollout of the 5G network is the cause of Covid infections. The argument there is that the 5G towers appear to be located in the same places as the areas where the virus spreads quickly. The underlying cause is that the virus spreads more easily in densely populated areas, and those just happen to also be the locations that are attractive for rolling out new infrastructure.

A correlation between two facts can therefore be the result of a causal relationship between those two variables, or of a shared third cause, but it can also happen that it's pure coincidence that two things appear to be correlated. Put simply, some things go up and some things go down, so depending on the range of your data, you can always find phenomena that seem to correlate with whatever you're studying. This is also the answer to the question we posed at the start of this blog about why there's so much contradictory food research. In such studies, the scope of the research often turns out to be too narrow. As early as 2005, researcher John P. A. Ioannidis published a study in which he found that most published research makes claims that aren't true. Fortunately, in his paper he also offers a few guidelines to help you avoid falling into the trap yourself. According to Ioannidis, reliable research has the following characteristics:

• The study has a large sample and is as broad as possible (for example, “ice cream sales” rather than “sales of rocket-shaped popsicles in Delft”).
• Changes in the causal variable have a large effect on the outcome variable.
• There's as little selection and discrimination as possible in the relationships between the two factors.
• The causal relationship holds up across different research methods.
• The research is as independent as possible (not just financially, but also, for example, in relation to popularity or how easily it can be published).

In addition to the characteristics above, we'd also like to give you a few tips to help make your own analyses more resistant to this pitfall:

• Try to come up with a logical explanation yourself for why two facts might be related. In that process, you often already have to make one or two assumptions that you can test.
• If you do think you've found a causal relationship, you can test it. Try to predict what will happen if you make a change to the causal factor, and only then run an experiment.
• Without a proven causal relationship, it's dangerous to let a correlation guide your decisions. Stay critical and avoid costly changes that have no effect.

Written by Lennaert van den Brink
Senior Consultant