Statistics pitfall #3: Correlation vs. Causation

There are many arguments for letting data guide your decisions. “Numbers don't lie” is a common refrain. At first glance, that is entirely true. Data is an impartial representation of reality and a good test for assumptions and gut feelings. Unfortunately, it is not always easy to interpret data correctly. It is no coincidence that almost every university in the Netherlands devotes part of its curriculum to statistics classes. Anyone who works with data runs the risk of falling into one of the many pitfalls. In this blog series we explain a number of common pitfalls and give you concrete tips to avoid them. In this third blog of the series we discuss ‘Correlation vs. Causation’.

Why finding a relationship is not the end but the beginning of research:

You know how it goes: one moment a study is published saying a glass of red wine a day is good for your health, and the next day another study immediately contradicts it. It seems to happen very often that a relationship found in a nutrition study later turns out not to be true after all. Why is that? The answer is that many scientific researchers also regularly fall into one of the biggest statistical pitfalls there is: a correlation between two factors is not the same as a causal relationship.

What do correlation and causation mean?

If this all sounds like double Dutch, don't worry, we will explain it step by step. First, we need to look at what these two phenomena mean. Put simply, a correlation means that two phenomena we are studying can be said to often occur at the same time or to the same degree. A well-known example is the observation that when the number of ice creams sold goes up, proportionally more people also die from drowning.

In that case there is a positive correlation between the phenomenon “number of ice creams sold” and “number of drowning deaths”. Conversely, with a negative correlation, when one phenomenon rises the other falls. As soon as there is a correlation between two phenomena, the next question is usually whether one phenomenon might be the cause of the other; in other words, whether there is a causal relationship.

Looking at our example of ice creams sold and the number of drowning deaths, it does not seem realistic that ice creams cause drownings. Nor does it sound logical that people buy more ice creams when they see others drowning. In this example there is actually a hidden third phenomenon that we did not include in our “research”: the weather. The warmer the weather, the more ice creams are sold because people want to cool down. Likewise, more people go swimming, and the more people swim, the greater the chance that something goes wrong.

Causal relationship

For sound research, a correlation between two phenomena is therefore only a starting point. If you see that two effects are correlated, the next step is to find out whether there is also a causal relationship. Only then do you know whether reducing phenomenon A will have an effect on phenomenon B. Based on our ice cream research, you could decide to ban all ice cream sales in the hope of reducing the number of drownings, but you would probably soon see that it does not really help.

Unfortunately, there are many examples where a correlation is confused with a causal relationship. The tricky thing is that quite a lot of phenomena can easily be correlated with each other. So easily, in fact, that there are websites where you can put together the most bizarre correlations yourself based on real data, such as this site by author Tyler Vigen, who has also written an entire book full of charts like this one:

Another classic example of a very common correlation is the fact that a great deal of geographic data is strongly correlated with population density. In the clip below, comedian John Oliver shows how such correlations are used as the basis for things like the “5G conspiracy”, which claims that the rollout of the 5G network is the cause of coronavirus infections. The argument is that 5G towers appear to be in the same places where the virus spreads quickly. The underlying cause is that the virus spreads more easily in densely populated areas, and those happen to be exactly the locations that are attractive for rolling out new infrastructure.

A correlation between two facts can therefore be the result of a causal relationship between those two variables or of a common third cause, but it can also be pure coincidence that two things appear to be correlated. Simply put, there are things that rise and things that fall, so depending on the range of your data you can always find phenomena that are related to whatever you are studying. This is also the answer to the question we asked at the start of this blog about why there is so much contradictory nutrition research. In such studies, the range (or “scope”) of the research often turns out to be too small. As early as 2005, researcher John P. A. Ioannidis published a study in which he concluded that most published research findings make claims that are not true. Fortunately, his paper also offers a few guidelines to prevent you from falling into the pitfall yourself. According to Ioannidis, reliable research has the following characteristics:

• The study has a large sample and is as broad as possible (for example “ice cream sales” versus “sales of rocket ice lollies in Delft”).
• Changes in the cause variable have a large impact on the effect variable.
• There has been as little selection and discrimination as possible in the relationships between the two factors.
• The causal relationship holds up under different research methods.
• The research is as independent as possible (not only financially, but also, for example, in relation to popularity or how easily it can be published).

In addition to the characteristics above, we would also like to give you a few tips to make your own analyses more resistant to this pitfall:

• Try to come up with a logical explanation yourself of why two facts might be related. In that process you often already have to make one or two assumptions that you can test.
• If you do think you have found a causal relationship, you can test it. Try to predict what will happen if you make a change to the causing factor, and only then carry out an experiment.
• Without a proven causal relationship, it is dangerous to let a correlation guide your decision. Stay critical and avoid costly changes that have no effect.

More statistics pitfalls?

Has reading this blog made you curious about the other blogs in this series? Read all the blogs at your leisure via the buttons below.

Pitfall 1 Pitfall 2 Pitfall 4