Introduction
A leading advertising platform that connects users with personalized content recommendations leverages Loops to help serve over one billion users each month. Loops plays a critical role in monitoring KPI performance across their complex operation.
A Loops customer for two years, they recently extended use of the Loops platform to their R&D team. Beyond their usual monitoring of key metrics such as revenues, page views per visit, and more, and their focus on optimizing core KPIs to enhance the user experience and maximize ROI for their publisher clients, their R&D team will now review the effect of development releases on top KPIs. Overall, their highly data-driven approach ensures relevant content is delivered to audiences when it matters most, making them an essential tool for modern digital marketing strategies.
When the Impact of R&D Development Releases on KPIs are out of the Loop
Like many companies today, the Publishers R&D team was busy making product adjustments and pushing these into production. However, monitoring the impact of every development release on the performance of publishers’ core KPIs was not prioritized, primarily because of the traffic, time, and cost that traditional methods would require.
Earlier in 2024, some teams at the company, including the data analytics and finance teams, noticed that a payments-related KPI was increasing. However, the R&D team were not directly involved in monitoring KPIs and were unaware of any problem. It wasn’t until the KPI had continuously increased for a month that they were informed of the change and began reviewing their code.
Finding the Bugs Behind KPI Spikes, the Hard Way
The team needed to determine if the increase was caused by a recent development release. Typically, identifying the impact of product changes on KPIs is a slow and manual process of examining the performance of multiple KPIs for every individual development change. This requires a massive effort and is the reason why many companies don’t actually measure the KPI impact of the majority of their development releases.
In this particular case, it took the R&D team over a month to manually identify the development release that triggered the change in the KPI. With Loops, finding the root cause of KPI trends is far more immediate and conclusive, often saving customers from significant costs and losses occurring over time.
Loops Root Cause Analysis Quickly Confirms a Development Release Error
The AI-based Loops Root Cause Analysis model helped the R&D team confirm that a bug had been introduced within one of the development releases a month earlier. As depicted in the chart below, the new code was incorrectly causing the KPI to spike upward, subsequently increasing payables.
Knowing exactly what and where to act, the bug was fixed by the following month.
Monitoring the KPI Impact of Every Development Release with the Loops Release-Impact Model
Being able to pinpoint specific code releases that have a dramatic impact on critical performance metrics with the Loops Root Cause analysis is a key feature that empowered the R&D team. They were able to easily identify a specific release and then investigate the relevant sections in the code.
The next step is to ensure that the impact of every development release on KPIs is monitored going forward. Loops has helped many companies make measuring the impact of every release a fast and easy part of their development release process.
Read more about measuring the causal effect of product launches with the Loops Release-Impact Model.
No-Code, Easy Setup - and Slack Notifications
The Loops setup for the R&D team was simple and fast. Now, they have the peace of mind that they can efficiently and quickly use Loops to find the root cause of KPI trends, anomalies, and spikes.
Closing the loop, the company decided to also take advantage of Loops’ Slack notifications to issue weekly updates on KPI changes, ensuring the R&D team and other stakeholders are kept informed, further mitigating risk and facilitating rapid response.
Conclusion - Ensuring R&D Analytics are in the Product Growth Mix
R&D analytics are sometimes overlooked. There’s a rush to launch new releases to market but measuring the impact of these launches on various metrics, from latency, performance, to core business KPIs can be easy to overlook, especially fast-moving, complex environments. Loops handles this important step efficiently, with minimal traffic and resources.
Loops has revolutionized the way this company, and many others, monitor and optimize their core KPIs, getting straight to the root cause of problems and being able to leverage detailed information on the explainers and segments behind them. Automating the impact analysis of every development release, can drive rapid identification and resolutions, ensuring optimal performance and reducing unnecessary costs.
When it comes to protecting revenue, monitoring the impact of R&D releases is imperative. Adopting the proactive, AI-enabled approach offered through the Loops platform mitigates risk and minimizes loss. Further, the operational efficiency provided also strengthens a company’s ability to serve its customers across vast user bases. The success of this case study demonstrates the significant, broad value that Loops can bring to the table. An indispensable tool for modern digital companies, Loops is not only for data and analytics, and product growth teams, it also delivers value, efficiency and confidence for those in leading R&D.
