A quick guide to customer retention

There are two main ways you can grow the customer base of a business: you can either acquire more customers by increasing your customer acquisition rate, or you can improve your customer experience and increase the number of customers the business retains.

As customers are constantly being lost due to attrition, churn, or defection, it is vital for a business to top up the losses with new customer acquisitions and try to reduce customer churn. If the customer acquisition rate exceeds the customer churn rate, the overall number of customers will grow. If the acquisition rate is insufficient to top up the customers being lost to churn, then the volume of customers will decline.

As most marketers or ecommerce data scientists are aware, it is generally much more expensive to acquire new customers than it is to retain existing ones, therefore, increasing the customer retention rate is a major objective in most businesses as it can have a significant impact on profits.

In this guide, I’ll cover several competing ways to define customer retention rate, explain the data science techniques used to measure, analyse, and optimise retention, and explain how you can build the underlying models required to measure retention rate properly.

What is customer retention rate?

Customer retention rate, like Customer Lifetime Value (CLV), is one of those unusual marketing metrics that businesses say are critical to their success but that are actually quite poorly defined and rarely measured in a consistent manner.

One very interesting study on customer retention metrics was undertaken by Aspinall et al. (2001). They surveyed UK businesses (including online retailers) to see if they considered retention a critical marketing metric and asked them if they could define it or measure it. Surprisingly, most could not.

“Many companies that claim to consider customer retention as an important business objective do not define it well or measure it. On this basis, companies’ claims to focus on customer retention need to be treated with a pinch of salt”, Aspinall et al. (2001).

When companies were asked to define customer retention, most of them gave quite vague answers, which would impact their ability to calculate it. If you can’t define something clearly, then data scientists or marketers are going to struggle to implement it mathematically to create customer retention metrics for the business to track.

There are many definitions of customer retention in the academic literature, but most of them are rather vague, and wouldn’t make it easy for a data scientist to use the definition to implement a metric a business could use. My preferred metric is the one proposed by Farris et al. (2010), who define customer retention rate as follows:

“Customer retention rate is the ratio of customers retained to the ratio of customers at risk of churning”, Farris et al. (2010)

Farris’ definition of customer retention rate is one of the few to actually suggest how it ought to be calculated. However, it can make it sound quite simple to understand and easy to generate, but the reality is rather different and many businesses may struggle to calculate retention rate correctly.

Picture by Kampus.

What makes customer retention rate hard to calculate?

If we go back to Farris’ definition of retention we’ll see that it requires two values:

1. The number of customers retained
2. The number of customers at risk of churning

These two figures are easier to obtain in contractual businesses, because you can easily see how many customers you have by counting the number of active subscriptions or contracts. Similarly, the number at risk of churning might crudely be considered as those customers whose contract expires the following month.

However, neither metric is easily available to those in non-contractual settings, such as ecommerce businesses, or in environments where the customer isn’t tracked - such as most bricks-and-mortar retail stores. Most articles you’ll read on customer retention tend to skirt around this important point.

The reality is that customer retention is actually very challenging to calculate in non-contractual settings and extremely difficult in retail, unless you have the benefit of a loyalty card to help you track customers better.

In order to calculate customer retention rates correctly, you will really need a customer churn model, and these are beyond the capabilities of most marketers. As a result, it’s very common to see customer retention rates being calculated differently (or incorrectly) in articles and even in off-the-shelf customer analytics software.

What models are needed to calculate customer retention?

Customer retention rate is most commonly applied to the measurement of retention of existing customers in contractual situations and is harder to apply to non-contractual settings, such as traditional or online retail, primarily due to the complexities surrounding its measurement outside contractual businesses.

Since customer retention rate is defined as “the ratio of customers retained to the ratio of customers at risk of churning” (Farris et al., 2010) you obviously need to be able to measure the number of retained customers and the number of customers at risk of churning to calculate it.

The only way to accurately obtain those metrics is by building a customer churn model. These aren’t trivial to create. Customer churn models are divided into two main categories: contractual churn models and non-contractual churn models.

Neither model calculates the actual retention rate, as defined by Farris, but will identify which of your customers are still customers and which are likely to churn, so you can more easily calculate the metric.

Contractual churn models

Contractual churn models are designed for businesses where customers have a contract or subscription, such as an insurance or telecommunications provider. These models are classifiers that aim to predict whether a customer will churn or will be retained based on a set of features that describe them, their usage of the service, the charges they’ve paid, and their experiences with the company and its customer service department.

As you might imagine, happy customers tend to be those who’ve used the service without issues or have had any problems resolved quickly and politely through service recovery. Disgruntled customers will be those who’ve had problems with the product or service that haven’t been adequately resolved.

A wide range of popular machine learning classification algorithms can be incorporated into contractual churn models, but the most effective are typically tree-based or gradient boosting algorithms, including decision trees, random forests, XGBoost, and Catboost, as well as ensemble models and stacking models that combine various algorithms in a single model.

Recent machine learning research covering contractual churn models has demonstrated their ability to reach very high levels of accuracy - often up to 96% - typically through the use of ensemble models.

For further information, check out my step-by-step guide to creating a contractual churn model.

Non-contractual churn models

Non-contractual churn models are used in businesses where there is no contract but where customer data is held, such as online retail, or in traditional retail when loyalty cards record individual customer usage. They’re much more complicated to understand and implement than contractual churn models, and are actually quite rarely used, especially in small to medium-sized companies.

Non-contractual churn models also require the use of complex machine learning algorithms, but unlike contractual churn models, they don’t use classification algorithms, because the time of a customer’s “death” (in Customer Lifetime Value terminology) isn’t known in non-contractual businesses.

In non-contractual settings, a company can never really be sure exactly how many of its customers are still customers, or are “alive” (in CLV terminology). This requires the use of a predictive model to estimate how many customers from the past are still customers today.

The Beta Geometric Negative Binomial Distribution or BG/NBD model is one of the most common models used for this technique. The model is quite complex and hard to create from scratch, but there is an excellent implementation for R called Buy Til You Die or BTYD, and an equivalent Python package called Lifetimes, which also implements the same BG/NBD model.

For further information, check out my step-by-step guide to creating a non-contractual churn model.

Picture by Rodnae.

What heuristics are used in place of retention?

Since many companies don’t really understand the customer retention metric, can’t measure individual customer activity accurately because customers aren’t tracked (such as most brick and mortar retail businesses), or don’t have data scientists on their teams to create the customer churn models required, they need an alternative approach.

It’s very common for companies to use heuristics aimed at providing a similar metric to retention rate that can be measured more easily. While the metrics may still often be inappropriately named “retention rate” internally, they are still often sufficient to give marketers a general steer on whether the company is increasing its actual average customer retention rate in the absence of the real value.

Common heuristics for those who can’t measure retention

Picture by Kampus.

Can cohort analysis help?

Cohort analysis can be a useful tool in helping to analyse how customer behaviour varies over time. It is particularly useful for retention studies that examine different marketing channels or marketing campaigns, since customers acquired via different channels often behave differently. However, you can apply it with many types of customer segmentation.

One interesting observation (and the topic of my Master’s degree research) is that deep discounting in particular can have a striking impact on the likelihood that customers will be retained. My research used cohort analysis to examine the behavioural differences of customers acquired via affiliate marketing websites via deep discounts.

Customers acquired using deep discounts behaved completely differently to those acquired via more conventional means. They were very expensive to acquire, didn’t come back unless they were given further deep discounts, and they spent far less than other customers, making the approach completely unprofitable.

How can you identify what drives customer churn?

In order to identify the factors influencing customer churn and could be improved to help increase retention, you first need to capture the right kind of data on your customers, typically via your CRM or customer help desk platform. You can also use Natural Language Processing (NLP) to examine customer support tickets or customer reviews in order to identify the drivers of churn using machine learning.

By identifying what influences retention rates the most, you can discover what you need to focus upon to improve your offering and improve the customer experience. You can also find out if the features of your retention strategy or loyalty programme actually work, or if you’re just rewarding customers needlessly or giving them things that don’t encourage them to stay customers.

Cox’s Proportional Hazards model is one of the best ways to analyse which factors influence customer retention and attrition the most. It can help statistically pinpoint which aspects of the customer experience have the strongest positive effects on retention
and which ones are causing most of your attrition.

Importantly, a higher retention doesn’t always mean that customers are satisfied. There are other ways to increase retention without making customers happier. According to Aspinall et al. (2001), some customers may be apathetic and not bother to switch, while others may resent the company but be experiencing vendor lock-in, which makes the task of switching to a rival much harder. Similarly, customers that don’t appear to be loyal but themselves feel brand loyalty may be persuaded to purchase if barriers are removed.

Common factors influencing customer attrition

There are literally dozens of factors that can drive customer attrition. Here are some of the most common:

Picture by Rodnae.

How can you increase your customer retention rate?

In order to improve your customer retention, you need to create a customer retention strategy. Larger companies sometimes create a dedicated customer success team that focuses specifically on improving customer retention.

A customer success team will work with data scientists to analyse the causes of churn, and will work to improve the offering to help keep customers happy and track metrics that allow them to monitor whether the techniques are working.

A step-by-step approach to improving retention

How can you identify customers about to churn?

Customer latency is perhaps the easiest metric you can use to identify which customers are most likely to churn. Customer latency (or purchase latency) measures the gap between each individual customer’s orders and is often analysed on its own as part of an order gap analysis.

The reason latency is so important is that customers typically purchase somewhere around the mean of their purchase latency. As the below graph shows, when the customer is placing orders within the mean period, their probability of being “alive” (in Customer Lifetime Value terms) remains high. However, as the customer latency gap gets bigger, they are much more likely to have lapsed as a customer, and are considered “at risk” of churning.

This time between orders is central to most Customer Lifetime Value models, such as the BG/NBD model and the Buy Til You Die implementation used by some data scientists. It’s easy to calculate purchase latency and the metric itself can be used to make simple estimates of whether a customer is due to order again soon or not. That in itself can be a useful tool for sales team in B2B ecommerce.

What is a good retention rate?

Since we know from the academic literature that most companies don’t know how to define or measure retention, and that most are likely to be using a crude heuristic, it’s difficult to state what “good” looks like, because the metrics they’re using aren’t directly comparable.

It will, of course, also vary for every sector and business, and it will be completely different in contractual settings compared to non-contractual settings, like retail.

My advice would be to define your retention metric internally, calculate and record it regularly and undertake analysis to understand what behaviours make customers leave. Keep making your products, prices, and service better so more customers stay and remain loyal.

Further reading

• Aspinall, E., Nancarrow, C. and Stone, M., 2001. The meaning and measurement of customer retention. Journal of Targeting, Measurement and Analysis for Marketing, 10(1), pp.79-87.
• Farris, P.W., Bendle, N., Pfeifer, P.E. and Reibstein, D., 2010. Marketing metrics: The definitive guide to measuring marketing performance. Pearson Education.
• Keiningham, T.L., Cooil, B., Aksoy, L., Andreassen, T.W. and Weiner, J., 2007. The value of different customer satisfaction and loyalty metrics in predicting customer retention, recommendation, and share‐of‐wallet. Managing service quality: An international Journal.
• Weinstein, A., 2002. Customer retention: A usage segmentation and customer value approach. Journal of Targeting, Measurement and analysis for Marketing, 10(3), pp.259-268.

Matt Clarke, Monday, November 01, 2021