Introduction
In the competitive world of consumer goods, maximizing the performance of your online app and the users’ experience is crucial for driving sales, customer satisfaction, and long-term loyalty. This case study explores how a global leader in the fast-moving consumer goods industry utilized Loops product analytics to identify and resolve a drop in a core Key Performance Indicator (KPI), ultimately avoiding a $5 million hit to revenue.
Existing Consumer Goods Customer
The Loops customer, a well-known global company, had been using the Loops product analytics platform for a year, relying on Loops insights into their app’s performance and user behavior. Recently, they started a phased launch new version of their online app, the primary channel for their online business division.
Performance and UX Issues Affecting Device Types
Shortly after the initial launch of the new app version, Loops surfaced a decline in a core KPI. Upon further investigation, it was discovered that the new app version had performance and UX issues that affected certain consumer device types. Fortunately, the majority of users were not on the new version when this issue was identified. The issues led to a decrease in KPIs for the affected segments, even though the overall KPI average did not show a significant decline. Typically, just using an average can mask the presence of an underlying problem, making it difficult to identify the root cause had they used traditional analytics methods. Loops alerts on these segments facilitated the early detection of an issue empowering the team to halt the deployment before major losses were incurred.
The Hidden Segments Behind KPI Drops:
Because the app version was new, it’s share of users was small. Therefore, looking at the average didn’t surface any problem. However, the analysis from the Loops platform identified a combination of attributes such as specific devices, specific apps, and specific countries that were leading the decline. It analyzed all of the data, millions of permutations, in determining the cause and effect. This level of exhaustive analysis would take take a very difficult and lengthy process to detect manually.
The graph and table below show an example where the KPI trend appears positive, yet the detailed segment analysis shows a 40.92% drop in KPI value related to users sourced from LinkedIn.
A Loops platform table showing segments and their impact on a KPI
The behavior of hidden user segments are a common reason for KPI Drops. Other reasons include seasonality, marketing activity, product performance, data pipelines, experiments, feature launches, other KPIs and external events, which are not always represented within your data. To learn more about KPI drops, read the Loops eBook: Protecting Revenue - The Ultimate Playbook for Understanding Your Product KPI Changes and Avoiding Loss.
Identification of Decline Reasons with Loops
Loops’ platform, quickly identified the reasons for the KPI decline by leveraging its advanced AI-based causal inference models. The platform’s automated segment analysis pinpointed the specific devices that were experiencing issues, which would have otherwise been challenging to detect if relying on average KPI performance alone.
Leveraging Proprietary Models for Root Cause Analysis
Loops employs proprietary models to analyze all the data from the systems connected to it and then ranks the potential reasons for the KPI decline. The platform examines various factors, including marketing campaigns, A/B tests, product bugs and changes, seasonality impacts, and even finds user segments that the customer had not predefined. By ranking these potential “explainers” based on their impact, Loops provided clear insight into the root cause of the KPI drop. For instance, had all countries underperformed, it would have been evident that the issue was not country-specific. Rather, Loops identified the specific segments affecting the KPI drop. In addition, Loops automatically generates summaries of insights and seamlessly pushes them to the customers’ Slack, MS-Teams, and email, ensuring timely and actionable information is distributed to the relevant individuals and teams.
Pausing the New App Version and Fixing the Problem
Armed with the insights discovered through the Loops platform, the customer decided to pause the new app version and address the identified issues. By focusing on the specific device types that were affected, they were able to implement targeted fixes. This proactive approach not only resolved the performance and UX issues but also prevented further revenue loss. The estimated revenue saved from this intervention amounted to approximately $5 million.
Learning: Protecting Revenue Before Losses Occur
This case study underscores the importance of protecting revenue by proactively identifying and addressing potential issues. Companies need to implement automated measures that allow them to detect and resolve problems before they lead to significant financial losses. Key points to remember are:
Setting alerts, not only on averages but across the board, is critical to early notification and prevention of potential revenue loss
- Setting alerts, not only on averages but across the board, is critical to early notification and prevention of potential revenue loss
- Root Cause Analysis must be automated to avoid the significant time many companies waste trying to understand why a KPI, or revenue, has dropped.
Proactive Measures for Revenue Protection
To safeguard their revenue and ensure continued success, the customer implemented several proactive measures:
- Weekly Business Reviews: The customer conducted weekly business reviews to identify trends, drops, and anomalies in their KPIs. This regular monitoring meeting allowed them to stay ahead of potential issues and address them promptly.
- Automated Analysis and Alerts Mechanism: Loops’ automated analysis and alerts mechanism provided real-time notifications of any significant changes in KPIs. The analysis included examining segments and dozens of other potential reasons that could influence KPI performance. The alert enabled the customer to quickly investigate and respond to potential problems. Without this level of automatic, early detection, they might otherwise have missed the issue. The KPI average had not significantly changed and would typically not have warranted deeper analysis. Loops not only provides in-platform alerts, the systems also automatically generates personalized alerts and Insights Summaries that are pushed to MS-Teams, Slack and email.
Conclusion
Companies waste too much time trying to understand why KPIs are changing instead of automating the process, which enables them to focus on driving insights. This case study highlights the importance of advanced analytics and proactive measures in monitoring the KPIs of your product and protecting the revenue you achieve. By leveraging Loops product analytics, the customer was able to quickly identify and resolve the root cause of a KPI decline, avoiding a $5 million loss of revenue in the process. It also underscores the value of using Loops’ sophisticated AI-based causal analysis tools to gain automated, valuable insights, know where to act on them, and to drive business growth.
