KPI Drop Checklist

It’s 2024 and analysts are still wasting time figuring out why their KPIs dropped.

At, we know this from our interviews with 40 growth analytics leaders. They all said the same thing - investigating why KPIs go down is a huge headache and where they spend their most time. Further, they said this happens all the time, from every week to quarterly. Growth teams get stuck for days/weeks trying to solve the mystery. Most of the time, they still don’t get the answer.Based on our research

We put together this prioritized checklist to help you understand KPI drops...

...hopefully in less time and with greater confidence:

1. Data Issues
   1. If your data quality or quantity suddenly changes, check your pipelines for errors and anomalies. Are data sources sending data, is there null/bad data or changes in data formats/schemas.
2. Seasonality
   1. Look at both the natural patterns of your business and those driven by external factors that impact user behavior, such as holidays. You’ll need to compare with previous year(s) data. Use anomaly detection models that account for seasonality to see if this explains your KPI decline.It’s important to remember that each KPI requires its own model adjustment - some KPIs are very volatile by nature, triggering “false alarms.”
3. Marketing Traffic
   1. Examine your marketing mix to identify if there are specific marketing channels that are driving the KPI change. Did the KPI drop for a specific channel, a new channel, or because the channels mix changed and you attracted more users of less quality?
4. User Segmentation
   1. KPI performance varies across different user segments. For example, with apps KPI changes very often pertain to a user device type. Also consider country, new versus existing users, vertical, product and persona, etc. It’s important to rank the segments based on their “contribution to the change”, which is a function of:> Change in the size of the segment> Change in the KPI for a segment.
5. Product Issues
   1. Your KPI drop could be due to product-related problems. Common product issues include errors, bugs, and usability, especially across devices and onboarding flows. Other issues such as latency and performance are increasingly becoming top KPI drop culprits.
6. New Features/Launches
   1. Here you need to perform a “feature or release impact analysis” to understand the effect of the launch in the following days/weekly. Analyze the user journeys, relevant funnels, adoptions across segments, the impact of this feature launch on other KPIs, etc. You’ll be surprised how one feature may affect other product areas in unexpected ways.
7. A/B tests
   1. Incorrect A/B test setup is one thing but also when a significant number of users, 50% for example, are exposed to an experiment that is underperforming, your KPIs will drop.

Want to know how Loops does this automatically, Book a Demo with me.

I would love to hear if investigating KPI drops is a problem for you.