360 Commerce, Data & Intelligence September 11, 2017
Predicting Conversion Intent in Real-Time
Most businesses allocate a significant proportion of their advertising budget to retargeting campaigns, but most of these retargeting audiences are constructed using gut-feeling. In more detail, retargeting audiences are predominantly defined by simple rules such as “all people that have visited the website” or “all visitors that have viewed a product”. This is the flaw in the current remarketing practises these days and dates back to the famous quote from John Wanamaker “Half the money I spend on advertising is wasted, the trouble is I don’t know which half”.
The flaw in current retargeting PRACTICES
From a business perspective, we do not want to retarget the website visitors that had a specific interaction with the website, but we want to retarget the visitors that are most likely to convert when retargeted and thus experience the adds as relevant and informative (see image 1). If we are able to get an understanding about the conversion intent of an individual visitor, we are in a position in which we can integrate this conversion intent into our campaigns to target only the relevant people. Additionally, it is expected that this adjustment will positively affect the conversion rate. Also, it will avoid the well known frustration of irrelevant advertisements and improve the brand experience of the targeted person because he will conceive the adds as being relevant to him. So how can we obtain the conversion intent of a website visitor instantly so we can use it for retargeting purposes?
Predictive modelling process
This question can be answered by smartly leveraging the power of raw clickstream-data and using several machine learning techniques to build a model that can predict the conversion intent of an individual website visitor in real-time. This may sound like it requires many sophisticated technologies with expensive licensing fees, but the opposite is true. Only open-source technologies are used to build and deploy this predictive model at one of our clients. The following steps briefly explain the process (see image 2):
- 1. Firstly, we collected the raw data from over 500.000 sessions (unique website visits) including all the interactions, scrolls and engagement indicators the visitors had with the website during their visit.
- 2. Secondly, we applied an algorithm suite to the data to find out which algorithm is best in recognizing the browsing patterns that are typical for buying customers. Over 30 different predictors, which are related to the current and historical session of the visitor, are used to detect these patterns. Eventually, after many testing iterations, the algorithm that most accurately predicts which visitors are going to convert is used in production.
- 4. Lastly, now that we have a predictive model that predicts the conversion intent of each individual visitor in real-time, we can push these predictions to all advertising platforms of choice to use for enhanced audience building.
Image 2: Machine Learning Process
How to use the output of the model
Now everything is in place, we can use the predicted conversion intent to enhance and improve our retargeting audiences used in campaigns. This will avoid targeting irrelevant and uninterested visitors (low conversion intent) and improve the conversion rate. The experiment we did was aimed at confirming whether the model made relevant and valuable predictions in real-time when deployed. In more detail, we determined a threshold which made the best split between low and high conversion intent of website visitors. Consequently, we set up an experiment that splits the current retargeting audience, which consists of all people that viewed a product or added a product to their cart, into two groups. The first group is the experiment group low intent and the second the experiment group high intent.
Both groups had the same bid strategy, frequency cap and were retargeted via Facebook. In short, the hypothesis for this experiment was as follows: “The conversion rate of the high conversion intent group is statistically better than that of the low conversion intent group”. We analysed the difference in performance between the two groups and the result were impressive.
It is not a surprise that the high conversion intent group is one-third of the total retargeting audiences, but it is interesting to see that the difference in conversion rate between the two audiences is significantly different. In other words, if we would retarget only the visitors classified as having a high conversion intent, we would generate an uplift in conversion of 233% compared to low conversion intent group (see image 3 below). But when the objective is to maximize revenue, we would not advise to exclude customers classified as low conversion intent simply because there are still converting customers in this group. Alternatively, we would advise to treat this group differently.
Image 3: Results first experiment
Although the model is not making perfect predictions yet, the first experiment confirms that it is capable of making solid predictions about which visitors are more likely to convert than others. Now the next question is: “How to use the predicted conversion intent in campaigns to improve conversion rate or generate more revenue?”. We are currently running experiments to find the answer to this question and are looking for pioneering tech-savvy marketers and business managers to join our quest.
This model could serve different purposes, for example, it could be used to deliver dynamic content to tailor the customer experience of individual website visitors. Additionally, the predicted conversion intent can be used for enhanced reporting. Since this metric involves not only single interactions but the actual intent of website visitors, it is a much richer metric that provides insights about the overall user intent of specific groups of visitors. Also, it would require a small adjustment to convert this model into predicting the expected revenue a visitor is likely to deliver when retargeted or what frequency cap should be used for each individual visitor for retargeting.
To conclude, by smartly leveraging the power of big data, machine learning techniques and open-source software it is possible to build a predictive model that helps online marketers to retarget more relevant people and avoid disturbing non-interested visitors. Wouldn’t we all want that? When integrating the output of such a predictive model into the current remarketing campaigns, we can generate a much better conversion rate when the objective is to spend a limited budget most efficiently. Besides that, the predictive model can serve different purposes and can be used for dynamic content, reporting, or predicting expected revenue. This creates a new dimension for tech-savvy marketers to innovate and be creative. In other words, this model conveys many different applications and therefore creates a new playing field in online marketing in which business problems can be translated into data solutions using machine learning and big data technologies.