The explosion in the use of data has upended the way product and growth professionals make critical decisions about how they build their products. Product teams do their best to leverage near-infinite amounts of data to discover insights that help them develop better products. They’re on a mission to gain clarity regarding how users interact with their products and more specifically, discover the features and flows that delight users to the point that they become ambassadors that drive viral growth.
However, as many product teams around the world have already learned, implementing this novel strategy is no walk in the park. The most valuable insights, the ones that deliver the biggest boosts to KPIs, are hidden behind mountains of data. Many product professionals spend countless hours staring at dashboards on their screens in an endless pursuit of that one golden insight that will unlock the floodgates of growth. They wade through endless dashboards, events, segments and correlations to better understand user behavior; but the unfortunate reality is that even with an army of analysts and data scientists behind them, there’s no way they can process it all and many great growth opportunities are overlooked.
All this means that product professionals are still making decisions based on a partial understanding of their data instead of real, data-driven insights on opportunities to improve their products. Existing product analytics solutions don’t do the trick; they fall woefully short when it comes to translating all those graphs and events into meaningful actions that support product KPIs. At the end of the day, they leave the product manager with piles of graphs and mountains of raw data that are difficult to interpret and nearly impossible to act upon.
Too many dashboards, not enough action
In today’s data landscape, data sources, segments and KPI tracking are scattered across dozens of different platforms and dashboards, making the discovery of even simple insights an extremely time-consuming, inefficient process.
Existing product analytics platforms do excel at visualization that supports independent analysis. However, drawing actionable conclusions takes more than just seeing your data; it requires context and a deep understanding of the story behind the graphs and raw data, which product analytics platforms do not provide. The result is a manual effort left completely up to the product professional, and product decisions that are based on only a partial understanding of the data.
No alignment with strategy
Product professionals are mission-focused, goal-oriented creatures by nature. They concentrate on KPIs such as retention, engagement and conversion as well as use cases, user personas and on improving the features that support their business strategy. They approach their products as never-ending works in progress that deliver those features, flows and processes that generate the most delightful user experience.
However, existing product analytic solutions built for product professionals weren’t necessarily built to support the way they work. These solutions leverage raw data as a function of the way the engineers defined it in the code. This results in a flood of ambiguous terms such as user properties, events, event properties and data points that no one can understand, like “Click_payment_start_2”.
But product professionals don’t think in terms of ‘events’ and ‘properties, they concentrate on the things that matter to their product-focused audience: segments, personas, features and user experience. It’s up to the product professionals to do the heavy lifting and understand how the abstract technical terms that litter their dashboards translate into insights on how they can improve their product and lift their KPIs.
Correlation leads to biased optimization actions
Different users engage with the same products in different ways; great product professionals excel at understanding exactly how each feature, user journey, user persona, etc., interacts with their product and delivers higher retention and conversion. Yet, most product analytics solutions are based on identifying how different user behaviors are correlated with user success.
The problem is that correlations are highly misleading and error-prone, causing product professionals to make biased decisions and optimize on the wrong opportunities. Correlations can be influenced by a wide range of different elements that are often challenging to identify.
By the way, Loops‘ causal inference models help data and product teams gain actionable insights that move the needle. Talk to us and we’ll show you how.
For example, we may come to the conclusion that if new users in a certain segment interact with a specific flow or feature within their first week, they are more likely to continue using the product after a month. We’d then run tests that encourage users to use that same feature/flow more frequently, and even change the onboarding process and update the tooltips to support its use. Now it’s time to wait a few weeks and verify the results, which are often disappointing as while the insight was great, chances are that we’ll see no impact on the retention rate. This is because our insight was based on a biased conclusion; apparently, there were other factors–user savviness or high intent–that we didn’t take into account and in reality, were the source of the user behavior we attempted to recreate.
You see, correlation without causation holds limited value and proving causation is a difficult process; connecting a certain effect to its true cause requires the application of complex algorithmic and data science. This nascent domain, known as Causal Inference, also demands a deep understanding of the data’s context and knowledge regarding which data should be examined when searching for these factors. That’s why most existing product analytics solutions prefer to remain confined to the realm of correlation, even though it provides less value and could negatively impact the work of their product manager users.
Working with partial data
The majority of the existing product analytics solutions were developed before the data warehouse became a staple of data infrastructure. This means that for them to operate, their users must send them huge volumes of data for analysis and processing which entails a great deal of data transfer and storage.
Since most solution providers charge according to the number of data points they receive and store, costs can quickly spiral out of control. On top of this, companies typically store their data in a variety of different sources and formats. Consequently, they often only send the top events or sampled data. This unfortunate reality limits and narrows the depth of analysis that can be conducted, resulting in subpar insights. Hidden opportunities and insights can be found where you least expect them, otherwise, they wouldn’t be considered hidden. By using only parts of their data organizations often miss out on valuable insights that may be waiting in the data left behind, not to mention the fact that the analyses they do receive are far less accurate as they’re based on partial data.
Take for example two data sets: user actions and revenue. While it’s clear that the way users behave can impact revenues, and hence business goals, the insights regarding exactly how this happens can only be unlocked by cross-referencing and analyzing both. In many cases though, companies store the revenue data on Stripe or other payment platforms – very different data sources and formats than the traditional “events” format used by most product analytics solutions. The company might choose to hold back from sending this data, and as a result overlook this critical metric and miss out on major value multipliers.
Before we go
In our big data-driven world, any product professional worth their backlog knows that they need product analytics in their arsenal. It’s the only possible way to make sense out of terabytes of data that their products generate at any given moment.
Most existing product analytics solutions were built to make data accessible as we moved from offline to online operations and began collecting data. While they can help product professionals visualize their data, these legacy platforms were built for a different world, one with much less data.
Today’s complex data landscape calls for a new, more advanced approach to product analytics, one that does away with dashboards and focuses on generating the valuable, actionable insights product professionals need to meet their goals. In other words, the future of product analytics should focus on making data–all of it–work for the product professional instead of forcing the product professional to work for the data.
