Correlation, Causation and Coincidence

Correlation, coincidence and causation: three words that are often confused.

Two things happen together – either at the same time, or sequentially.  Why?

This is a correlation, and the natural human tendency is to assume that there is a reason for the correation; that something caused the two events to occur together.  Daniel Kahneman and Amos Tversky identified this as resulting from one of the more common cognitive biases that humans are prone to: the representativeness bias.

This bias in the way we think creates patterns in our minds that link events in a way that follow established (or merely perceived) rules.  So, when two things occur one after the other, we perceive causation, because it conforms better to our perceptions of how the world works. When two things coincide, we think there must be a reason, and when we spot statistical anomalies, we suspect that something must be going on.

In the UK, the ratio of boys to girls (under 16 years of age) is approximately 51% to 49%.  I don’t think anyone really knows why the gender ratio at birth (allowing for selective terminations) is not 1:1.  The 51:49 ration does seem to be fairly universal in humans.

But what if we found a town with a ratio of 45:55, or an occupation with 59:41?  Would we think there is something in the water, or some occupational hazard affecting reproductive processes?

In fact, in the 1990s, someone measured the proportion of boys to girls among the children of Israeli fighter pilots 16:84.  Since the fighter pilots were all men a the time, what was their occupation doing to their Y chromosome?

However, any sample is likely to diverge from the average.  Take enough samples and some will diverge a lot.  Taken on its own, that sample will look so odd that something must be going on.  Taken with thousands of other samples and all you have is an outlier.  But it is the outliers that we notice.

Two things happen together: there are three possibilities

  1. Coincidence – things do happen together
  2. Direct Causation  – the two things are related, so that one triggers the other
  3. Indirect Causation – the two things are each related to a third thing, that trigger each
Advertisements

2 thoughts on “Correlation, Causation and Coincidence

  1. Glen B. Alleman

    Mike,
    Great timing, we’re in the midst of a “triage” effort for a large DoD program and the notion of correlation and causaton are confused. When asked “why did this thing happne (some process failure)?” The answer is “well that’s what happened last month when we ran to same reports.”
    Yes there is a correlation between how you run the month end report and how you upload status and how you upload all the other direct costs. But the cause is you aren’t following the Work Instructions for the Earned Value Management System month end close process.
    This is a simple minded example of the failure of our public school system in the freshman statistics class.
    The classic in the agile management world though is – “because I’ve observed this outcome and read some books about the same thing, I can assume it’ll work in your domain.” Yea, right

    Reply
  2. walthe310

    I like to collect coincidences. Most coincidences are random occurrences, but I still enjoy looking for causal connections and possible insights. The hunt adds interest to life. For example, when I worked as an aide in a hospital, I noted that most patients select a doctor whose last name is the same length, or nearly so, as their last name. Probably meaningless, but still fun to spot it. My own last name is five letters long and my current doctor’s last name is also five letters long.

    I am reading Interface by Neal Stephenson written under the pen name of Stephen Bury. Stephenson is also one of my favorite writers of science fiction and I have read all of his books. Interface takes place in the near future and is set in Illinois and Denver, just as Dan Simmons has near future books set in both places. And both authors have last names beginning with “S” and ending in “on.”

    Reply

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s