Main results:
- Proved impact on revenue – A clear link between a product initiative and the impact on revenue that gave the organization the confidence to expose more users to the feature.
- Data Science tools for the product group
- Leveled up the analytical discussion internally
In Short
Taboola is one of the leading ad-tech platforms in the industry, serving over 360 billion content recommendations each month.
Developing dynamic recommendations for advertisers was one of the 2022 primary objectives. There were already several recommendation features in place which were exposed to a limited group of users, but Taboola had difficulty measuring their impact. A solution was needed to justify further investment in these types of features*.*
Using Loops, the product team could show that a single recommendation contributed an uplift of over 3% to the revenue of users exposed to it. That validation opened the opportunity to expand the feature release to thousands of new campaigns, which showed an uplift of over 5% in revenue.
With full integration to Loops, Taboola is now equipped with a scientific tool to measure the impact of other new or existing features, eliminating seasonality noise and neutralizing the traffic’s variance.
In Long
Taboola is the world’s largest discovery platform having a massive scale, unique content consumption data, and world-class AI technology. They help thousands of advertisers reach their audiences, increasing effectiveness across the buyer journey – From increasing awareness to driving online purchases.
One of the areas the product team was planning to improve was the Recommendation Center. Though the team believed in the content shown in it, the Recommendation Center was a secluded page in the product, which meant that only advertisers who actively navigated to this page were exposed to this valuable content, which is what the product team wanted to change.
Why Loops?
To justify this project, the team had to prove a connection between exposing an advertiser to a recommendation and business impact. Considering the many constraints that were to take into account (user tenure, type, seasonality, etc.), it was clear that a simple correlation analysis would not do the job. On the other hand – Shifting data scientists to create such a tool in-house took a lot of work, especially before any evidence of this project had a positive business impact.
Loops’ “Feature Impact” model was a perfect fit. This model uses Causal Inference methodologies and compares each user to itself in different timestamps. It also creates comparable groups by their properties, tenure, and behavior, ensuring the “treatment” event (AKA the exposure to the recommendation) is fully controlled. A zero-code integration meant that the only thing the product analytics team had to do was to map the relevant data into the system.
Loops was able to present a clear result showing that the recommendation significantly improves performance by over 3%.
“Thanks to Loops, we got the validation we needed. We could justify the expansion with high accuracy and significance, creating the “next generation” of user recommendations which would have greater exposure, and therefore, greater impact” – Amir, Product Analytics Director.
Being fully integrated with Loops, once that “next generation” infrastructure was launched, it made perfect sense to use the exact same model and recommendation to measure its current impact. The results amazed us all – not only did the exposure improve, but the change to an in-context placement showed an increased impact of a 5.7% uplift in revenue.
“Having Loops in our toolbox created a much more advanced analytical discussion in our group. It made us smarter, more creative, and ambitious, as sophistication does not equal high development effort anymore”.
