The Great Correlation vs Causation Debate: SMX East New York City 2011, Day 1!

Sep 14, 2011 | 2,013 views | by Navneet Kaushal
VN:F [1.9.20_1166]
Rating: 5.0/5 (2 votes cast)

The second session of SMX East 2011 Day 1was “The Great Correlation vs Causation Debate”. In this session, top SEO experts share their views after understanding search algorithms debate correlation vs. causation and make their cases for the types of ranking factors that do (or don’t) have significant meaning. Through this session, they focus on what changes actually cause ranking changes, and which changes merely correlate with rankings (e.g. they’re coincidental byproducts of change).


  • Danny Sullivan, Editor-in-Chief, Search Engine Land (@dannysullivan)

Q&A Moderator:

  • Max Thomas, President, Thunder SEO (@thundermax)


  • Eric Enge, President, Stone Temple Consulting (@stonetemple)
  • Micah Fisher-Kirshner, Senior SEO Manager, Become, Inc
  • Mitul Gandhi, Founder and Chief Architect, seoClarity
  • Kristine Schachinger, Owner, (@schachin)
  • Tony Wright, CEO, WrightIMC (@tonynwright)

The first speaker of the session was Kristine Schachinger, Owner, She begins the session by defining correlation and causation. She states that when you have a mutual relationship between two or more things and where one has a measurable effect on the other, it is called correlation. Further, she explains causation as “a relationship in which one action or event is the direct consequence of another”. Kristine, gives an example to explain that in every casual relationship there is a correlative one, but you will not find a casual effect in every correlative one. She says, “If I hit a ball with a tennis racket, racket hits ball, ball goes so far, the length of how far the ball goes is a causal effect. But the breakfast you had that morning might be correlated to how far you hit the ball”.

Kristine then says that we often mistake correlation for causation and in turn make assumptions based on old ideas or incorrect perception. Perception is like an optical illusion for example, site code on rankings – pre-Google's site speed announcement. And if you have a misperception, then we have no correlation or a spurious correlation. A Spurious Correlation is the perception that A effects B, but in actuality a third hidden variable does. She further, explains the delegates about spurious correlation by giving an example. She says that there is an increase in the sales of beer in NYC and we all might think that SEOs are here because of SMX and corelate it. However, we cam into this conclusion because we are not aware that Fashion Week is also going on. Thus, such variable also exists. Kristine says, “In a casual relationship, you can directly measure that A does CAUSE B. In correlative relationships, you can only measure the strength of the effect of A on B”.

Kristine talks of Confidence Level and says that you can show correlative strength in many ways and the best way to show is the confidence interval. This tells you that there is a relationship which is correlative where X is affecting Y. Because whenever you measure something you have to do testing. You have to know:

  • Your variable types – random, snowball, etc.
  • What is your sample size – is it large enough to normalize your data? Do you need it to?
  • Are you controlling for outliers?
  • Are you choosing the proper analysis method?
  • If you are not controlling these, are you able to understand the limitations of your conclusions?

Next up was Micah Fisher-Kirshner, Senior SEO Manager, Become, Inc. He address the delegates by asking a question Public Correlations: how do you tell the good vs the bad? Micah Fisher-Kirshner, further explains correlation with the following points:

  • Correlation should be shown with scatter plots, it helps show to visualize what it represents.
  • Correlation is a place to start, it is not a place to end.
  • SCheck the results with an SEO
  • Does this make sense, what if we factor for X

He further raises questions and explain it to the audience and other panel experts.

  •  How much time an effort will it take if you go down that route, and is there ROI on those efforts?
  • Is the data trustworthy?
  • Are you factoring all the relevant data that matters?

They looked at Panda and the average time on site. They saw a 42% correlation, but the scatter plot did not showed a correlation?So they looked deeper at other sites like theirs, by categories. Maybe, there was a group punishment? ie. How tos, CSEs, marketplaces, News, etc. Micah, emphasize and says that everything should be looked into detail over a period of time and then we need to look for seasonalitym sub-directories, sub-domains and domains.

The less amount of data you have, the longer you need to run the test.

  • Common Correlating Pitfalls
  • Other marketing channels
  • Extraneous online events, like other Google updates.
  • Did you break something, did you launch something new, etc.
  • Offline events, such as holidays, world or national events. Such as the impact of the DC earthquake.
  • For about two hours, people stopped shopping during the earthquake. It lowered their traffic by 2% that day.

The main purpose of stating the above mentioned points by Micah Fisher-Kirshner is to stress that you need to question everything!

Mitul Gandhi from seoClarity shares his views on it by saying that Measurement : How do you report on Correlation vs. Causation. What can you actually do with it? He then mentions the following points:

  • Avoid the pitfall of looking at all the variables in mind, like said before.
  • Look at it as not as snap shots in time.
  • It has to be reproducible.

Tony Wright, CEO, WrightIMC (@tonynwright) comes up next and begins by sharing on how to do SEO. He states the following points that should be taken into consideration for doing SEO:

  • Code: clean
  • Content: good content
  • Connections: links from other web sites
  • Conversations: social media

The penultimate speaker of the session was Eric Enge, President, Stone Temple Consulting (@stonetemple), who diagrees on what Tony said. Eric is in the opinion that you don’t have time to change one variable and wait six months before changing anything else. According to him, Correlation and Causation DON’T MATTER. You want to build great stuff that causes both of those things to happen. He further, says that you need not follow every search trend you see because the basis have not changed in the past 6-7 years. Eric gives the following examples to show that we neither have time for such things nor it is necessary:

  • Does Correlation cause Causation? Does a FB share result in links?
  • Does correlation cause traffic?
  • Aren't the signals just going to change anyway?

The session ends here with more Q & A and proved beneficial for the audience. Stay tuned for the upcoming coverage of the sessions.

The Great Correlation vs Causation Debate: SMX East New York City 2011, Day 1!, 5.0 out of 5 based on 2 ratings
4.thumbnail The Great Correlation vs Causation Debate: SMX East New York City 2011, Day 1!

Navneet Kaushal

Navneet Kaushal is the founder and CEO of PageTraffic, an SEO Agency in India with offices in Chicago, Mumbai and London. A leading search strategist, Navneet helps clients maintain an edge in search engines and the online media. Navneet's expertise has established PageTraffic as one of the most awarded and successful search marketing agencies.
4.thumbnail The Great Correlation vs Causation Debate: SMX East New York City 2011, Day 1!
4.thumbnail The Great Correlation vs Causation Debate: SMX East New York City 2011, Day 1!