For most of the marketeers, ecommerce or product people Conversion Rate (CVR) is one of the main metric to monitor. While it can be a main indicator of a webshop or an apps’ overall health, it can also be misleading as a business evolves.
Its simple formula is a number of conversions divided by total traffic (usually shown as a percentage). How it often goes with such “percentage” metrics is that they’re dependent on absolute metrics that can often be volatile in time or even meaningless without context. How can such a relation affect a digital business, and how to mitigate the effects of wrong conclusions stemming from it?
Vanity Metrics
I believe most of the readers are more than familiar with Conversion Rate, but a “vanity metric” can be a fresh term for some. In brief, it is a metric that looks good “on paper” but does not necessarily give meaningful insights on the performance. Usually, these are any “ever-growing” metrics such as the total number of users or social media followers. Norman Nielsen wrote in a blog article:
Vanity metrics appear impressive, but their fluctuations are not operational.
Nielsen Norman Group
As a remedy for steering your business on vanity metrics, using ratios instead is often advised. CVR may seem like a perfect fit but there are certain flaws to it. One lies in its formula, the other relates to a bias present in many firms.
Relativeness of CVR
The first issue of CVR is its denominator – usually a total number of users or sessions. Such metrics tend to be extremely volatile and they are often mentioned as vanity metrics themselves. This linear dependency between traffic and CVR makes the latter seemingly vain, too.
Secondly, the problem is how we measure and report on CVR. Out of pure laziness, most of the reporting and dashboards will probably focus on the site-wide CVR. This is where the vanity of CVR begins.
CVR vs Strategy
In the beginning of a digital business’ journey, having only a few performance-driven aqusition channels and a homepage, closely monitoring your CVR can be insightful (and extremely addictive). It’s a simple metric that a slight change of which can affect your bottom line. It can even be a strong factor in identyfing a product/market fit.
As a business grows, and marketing channels evolve, site-wide CVR will become a resultant of different sources of traffic landing on various pages. Branded-search or retargeting will most likely yield high CVR, mid-funnel performance media will rather be slightly less converting.
Now, let’s assume that diminishing returns hit the performance-driven campaigns and we want to invest in brand awareness (more and more brands come to this conclusion, like for example Airbnb).
At this point, the same metric that could signal a product/market fit, becames a mere weighted average of different sources’ CVR. In a scenario of growing top-funnel or mid-funnel investments, it will be in decline. If an organisation’s reporting has been built to cater to a performance-driven marketing, it will lead to some execs’ worrisome calls.
Dangers of Following a Vain CVR
Certainly, a hardcore example of taking actions on a declining CVR from a high, “vain” level is making bad strategic decisions based on it. Some of them may include even concluding that any brand investment doesn’t work (because “conversion rate is getting low”).
On another occassion, a declining site-wide CVR will push a CEO to hire a whole CRO (Conversion Rate Optimisation) team. CRO usually seems like a valid investment, although this is a countinous process and the team would probably fail to generate any short-term wins, to a dissapointment of each party involved.
Alternative Costs
I believe that most common pitfal to an organisation relying too heavily on CVR is trying to fix what’s not broken. Alerted execs will ask why CVR is so low and the poor employees will be obliged to dive into numbers to explain it to them (I’ll try to show how to do it effectively later).
I’ll present an example that happened to me recently. In a highly seasonal business, during the “peak” period CVR on one site went significantly down. It alarmed the upper management including the CEO, who mostly looked at the site-wide CVR. First conclusions were “something’s wrong with the website, we need to investigate it!”. Fortunately, before the whole team of devs were enagaged and resources burned, we had been able to show how a less “salesy” blog article started ranking 1st organically and got us 10000s of new sessions (and even despite the CVR of around 0.05% it translated to a massive ROI from the blog content).
The example above shows that it is possible to grow your business while decreasing CVR. Keeping CVR at a high level can be just vain.
In a radical scenario, if the management sticked to the strategy of focusing on a high CVR as a KPI target, they may have concluded that creating such articles is bad for the business. If you feel that could’ve happened in your company – best if you quit it.
How to make it actionable (again)
First of all, the key metric that you should always bear in mind is revenue or absolute number of conversions. If these numbers are growing despite slightly decreasing CVR, then this shouldn’t prevent you from sleeping well at night.
Secondly, always try to look at CVR within some context: e.g., per channel, device or landing page. This way you’ll minimise the “weighted average” bias of a site-wide CVR.
An another approach to the problem would be updating your dashboards and reports to show “adjusted” CVR that would for example filter out traffic from temporary top-of-funnel campaigns or any other source which CVR is not their main KPI. This would require a good discipline in tagging and understanding your traffic, thought.
Related considerations on CVR
Benchmarking
A most common question related to CVR is “What is the average CVR?”. Unless you’re a super-beginner who needs a clue what can a CVR value look like, such benchmarking means absolutely nothing. Marketing mixes and customer bases of each of the business can vary so greatly that any ballpark would not provide us with valuable information. If you should benchmark to anything, then it is your own performance, for example in a previous year or period.
Don’t Trust the Gurus
A similar, but reverse flaw of CVR is often used by so-called marketing “gurus”. They will often make their actions look surprisingly good and show inflated results by showing only partial data. A landing page may have an extremaly good CVR if it is used as a last touchpoint in a funnel or a remarketing campaign may show great CVR and ROAS but we had to invest a lot of money in all the steps that led to them. Again, it is crucial to look at the bigger picture, knowing how easy CVR can be influenced by other factors.
Troubleshooting CVR
Now, let’s assume your Conversion Rate is plummeting. What are the steps to detect any potential causes in an effective way? The points can be shuffled based on the particular case:
- Check the traffic sources. Are there any new traffic campaigns? Maybe we started to rank in Google for more general terms?
- Look at the locations of the traffic. Maybe there is bot traffic from one particular place?
- Check devices and browsers, maybe a recent update caused the app to break on one of the smartphone models.
- If you find anything in steps 1 – 3 then break it down by the others to isolate the exact cause.
- Check the site speed. Maybe the server is having some issues and users bounce.
- Look at the conversion funnel. Are there any leaking steps in it? Maybe there were some un-tested new features that harm the perfomance?
- Are there any seasonal products or services that are no longer in high demand?
- Checked for all of the above and still no clue? If the trend continues for a longer period then maybe your offering becomes less relevant to the customers? This is a good example how CVR can be an important indicator. Also, we cannot rule out external forced to be at play – has the competition increased or the market is in decline?
Wrapping up
Conversion Rate is an important metric when evaluating a performance marketing campaign or a website funnel. It meaning dillutes when looking at it site-wide and without the whole marketing mix context in mind. It is especially common for growing and evolving businesses.
It’s crucial not to focus on keeping CVR rate high at all means. You need to look at the bigger picture and try to understand what does it consist of.
Certainly, a higher Conversion Rate is something to strive for, but this should be ideally achieved with continous testing that take the flaws of CVR into account, like statistical interference.
A business should not set a higher CVR as its top-level goals. This is the simplest path to make a good metric vain.