Survivor bias, also known as survivorship bias, is a phenomenon that causes your marketing or market research to fail. It’s the reason why certain kinds of marketing advice never seem to work.
You must be wondering why I’m saying this, being a digital marketer, myself. While I’m definitely not saying that all marketing advice is destined to fail and that you should ignore it when it comes your way, survivor bias is something you need to be aware of in this market.
Above all, follow the advice that forces you to dig deeper and answer the tough questions – not advice that simply tells you to march in a certain direction.
What is survivor bias?
This behaviour can lead to false conclusions or overly optimistic beliefs, simply because failures are ignored. It can also lead to the false belief that our successes are the result of special tactics rather than mere coincidence or luck.
It might be difficult for you to grasp why survivor bias is bad, because why shouldn’t we focus on how success came about? The problem, here, is that survivor bias creates this false idea that these successes will work for anyone at any time, which is simply not the case.
We are taught from an early age to learn from our mistakes. Survivor bias is the exact opposite of this – it focuses on what worked over what did not. It, therefore, creates a template for others to follow and gives you the false assurance that this template is sure to work every time.
What’s scary about this is how we tend to adopt these opinions to structure businesses and make decisions, without examining all the data, and then stare in disbelief when it leads to failure. Learning how to avoid survivor bias is, therefore, a crucial step in making sure that your efforts are not wasted.
The history of survivor bias
Before I explain how survivor bias affects market research, I’m going to tell you a story. It might not seem relevant to business and marketing, but it’s a story that should be known because this is where many consider the principle of survivor bias to have emerged from.
In World War II, the Allied forces wanted to add protective armour to their war planes. Since resources were limited, experts had to decide where exactly they should include it for the planes to benefit from additional protection.
In order to do so, the experts decided to study the planes that had been shot but successfully made it back home. They found out that these planes had incurred bullets around the tail, body, and wings, except the cockpit and engine.
So, this seemingly obvious train of thought led them to place armour around the places that had incurred bullets and avoid the places that had no bullet holes. Sound logic, right?
If you really think about it, though, the planes they studied are the ones that made it back to the base, meaning that the bullets weren’t fatal to the plane – the tail, body, and wings could survive a rain of bullets.
Mathematician and statistician, Abraham Wald, was the one who pointed this out and recommended that the military attach the armour to the areas that the surviving aircrafts had no bullet holes, instead. In all probability, he saved many lives and also gave birth to the concept of survivor bias.
How does survivor bias apply to market research?
Similar to the story of war planes, we see this error of perception even in market research.
Let’s imagine, hypothetically, that I have a client who sells wooden furniture and they want to venture into producing metal furniture to increase their profit. Our team then analyses types of furniture that were profitable over the past year and find out only 10% of them are made out of metal.
Based on this market research, we will convey to the client that they should not venture into this idea and that it’s profitable to stick to wooden furniture because research shows that that’s what made the most profit.
However, what if in the unprofitable segment of furniture models, only 1% are metal furniture? What if out of the rest of the 9%, most are wooden furniture?
The lesson we need to learn from this data set is that it would make more sense for the client to expand their collection of furniture to include metal furniture if they want to increase profit. Just by looking at the success rates, we become victims of survivor bias and are prone to make the wrong decisions.
How does survivor bias apply to marketing?
In marketing, we rarely write about techniques, strategies or people that failed to succeed. We only document winners. Why? The truth is that nobody wants to position themselves as a loser and lose their competitive edge.
Just imagine, if all that’s on the internet are stories about winners – then you only read tactics as part of success stories.
Say I write about an email campaign that worked well for our company. In it, I point out how I used a blue background for the attached brochure instead of a white background. After reading this post, someone decides to use blue backgrounds in their email campaigns.
But what if 99 other people who used blue backgrounds for the emails in their email campaign failed? No one reads about these experiences! The only story out there that’s read is my success story, the survivor story. So, the audience believes that emails with blue backgrounds are the winning tactic for email campaigns.
It isn’t necessarily bad to stick to a strategy with a 1% success rate. It’s better than a 0% success rate, but only if you know what made it work in the first place.
The risk of survivor bias is that it often makes the wrong tactics look good. When readers see this on the internet, they stop looking, don’t make further inquiries, and stop digging. Since these tactics aren’t surefire ways to succeed, this leads to disappointing results more often than not.
How do you avoid survivor bias in the marketing field?
Be aware
The good news is that you have already taken the first step to resolve this issue – awareness.
You are now aware of what survivor bias is and how it can present itself in market research and marketing. Meaning, you are less likely to be fooled by it. So, even if I did put myself in a position for you to question what I have documented in our blogs, this article has helped you understand that what worked for me, may not necessarily work for you.
Recognise that what works for one person may not work for another. Understand that with one success there could be a thousand attempts that took the same path and failed and that luck and chance play a big role in success.
Don’t stop questioning
Always keep digging! Don’t stop at the success stories. Question why something worked for it to be successful. Dig deeper into what did not work and what went wrong on their path to success. Do some research to see if that tactic worked for other people/businesses too.
Be very sceptical of individual gurus, books, podcasts, and other resources that appear to give you the ‘answer’ or ‘secret formula’ to achieving wealth, success, and happiness. This all may sound hypocritical because I write a lot of tactical blog posts – in fact, I read and use this material because they’re fantastic resources.
What’s important, though, is that you ask yourself ‘why did this work?’ before you blindly use it.
Take a balanced approach
Don’t dismiss the most successful, influential and popular success stories out there. Clearly, there’s something that’s done right. All I’m saying is consider the potentially devastating downsides and consequences that lead to these successes too.
Instead of trying to mirror these wins, draw some key takeaways and come up with your own way to incorporate them into your business.
Do what you need to do but always account for survivor bias
In conclusion, don’t look for quick fixes to your problems. Try your best to understand what’s truly necessary. Do your research but always look at both sides of the story. Look for stories about marketing failures because there are some of those out there! I mean, survivor bias isn’t 100% either; it’s a bias, not an absolute.
Remember – don’t take any advice as the law. Take it as one of several considerations. Only by being aware, questioning and taking a balanced approach will you be able to find your own path to success.