Our story with Loops kind of tells itself – we started with a free trial and we gained key insights. We took that and enhanced the product. Once that shipped, we used another Loops feature to prove the impact it would drive. I’d say we’re pretty excited about the impact on growth that Loops has helped us drive at Wahi”Simon Trudeau, Head of Analytics and Optimization, Wahi
The Old Way at Wahi
Wahi has always had a focus on product analytics, but quickly determined that their correlation-based approach of formulating hypotheses, trying to optimize campaigns and the product, and waiting weeks to months on A/B tests trying to identify causation, was not optimal.
Before using Loops, Wahi had never incorporated causal inference in a statistically rigorous way.
Key challenges of Wahi’s old approach included:
- Dashboard-centric, correlation-based analytics that did not shed light on the cause of KPI variations.
- Traditional A/B testing required tradeoffs, to either run a traditional A/B experiment, also taking a hit to user website performance; go manual, taking on the difficult task of manually trying to filter out the noise and isolate the lift of the change in a before/after analysis with a lower level of confidence; or not run any experiment – because of these tradeoffs but also because not everything was always “testable”.
The New Way, with Loops
“I heard about Loops on an analytics-related Slack community, and when I landed on the homepage the first thing I saw was causal analysis of KPI variations, which was exactly the issue I needed to address. We did a free trial for that specific use case and it was actually hugely successful. Ever since, Loops has been very valuable and foundational to our product analytics process. We use Loops for every important move we make.”
Simon Trudeau, Head of Analytics and Optimization
How Wahi Drove 50% Lift in Engagement - from the Loops Free Trial
STEP 1: Discovered a High-Conversion feature – while in a Loops Free Trial
Rather than the traditional dashboards, Wahi started out using the Loops’ Opportunities feed in a free trial.
They wanted to identify what drove the highest lift in their top KPIs. One of the first insights they gained from Loops was they discovered that a unique feature of their mobile app, which presented each user with a customized, homepage of recommended properties, had a major causal impact on converting users into leads.
STEP 2: Added the Feature for Web Users
The majority of Wahi’s users engaged via their website versus the mobile app and Wahi wanted to know if adding the mobile app feature “recommended properties page” to their website would increase conversion-to-lead rates and help drive subsequent lift in retention and engagement KPIs.
They added the capability to their website for users with an existing Wahi account, who were then automatically redirected to the new personalized page featuring their recommended properties. This meant that more users would experience this high-impact conversion rate feature.
STEP 3: Proved the Impact Using the Loops Release Impact Model
Wahi leveraged a powerful capability of the Loops platform – the Release Impact Model – to measure the impact the new feature would have on their KPI.
Typically, when a new feature or release comes out, there could be a KPI variation that follows. However, it’s not necessarily because of the feature that you’ve just shipped. In Wahi’s case, seasonality and the high velocity of marketing and product updates are major factors that can render traditional before-and-after analysis unreliable. However, with the Loops Release Impact model, all of their changes become “testable” and running the release impact analysis did not slow website performance. The model simulates A/B tests, provides accurate impact measurements, and accounts for “noise” factors with minimal traffic and no wait. – Learn more about Loops Release Impact.
STEP 4: Enjoyed a Fast 50% Lift in Engagement and 5.4% Lift in 1st Week Retention
Wahi saw a 5.4% lift in their New User 1st week Retention KPI within just two weeks of releasing the new recommended properties feature on their website. Further, users not only found relevant listings, they also saved and shared them 50% more frequently than users not exposed to the new feature. The conversion KPI lift was very positive, particularly because conversions at Wahi also help drive leads, retention, and engagement.
How Wahi Drove a 200% Increase in Marketing Conversions - with Loops "User Journey" Insight
STEP 1: Discover Two Negative-Impact User Journeys
In addition to gaining causal insight on product features, Wahi decided to leverage the Loops platform to run the same Loops models on their different marketing user-touchpoints to gain new multi-touch (user journey) impact and attribution insight. Wahi discovered that two specific combinations of channel and landing page (user journeys) had a negative impact on lead creation.
STEP 2: Understand and Apply What Worked Best
Instead of simply deprioritizing these user journeys, Wahi also used Loops to look at why other combinations of channel and landing page had better conversion rates. This helped them to improve the UX flow of the two underperforming combinations.
STEP 3: Enjoy a fast 200% Conversion-To-Next-Step Lifted
To date, the first-improved user journey has driven an incredible 2X increase in Conversion to Next Step. Changes for the 2nd user journey are set for release in late April.
Discovering the Unknown with Causal Analysis:
In just a few months, Wahi has been able to automatically find causal inference on different types of behavior, features adoption, or mix of marketing campaigns. Causal insights have provided the most value for Wahi.
With the Loops Opportunities feed and impact sizing, the Wahi team can respond very quickly, as they did with the underperforming marketing journeys and after discovering a high-impact feature of their mobile app.
Making a Profound Impact on Business Process
Today, Wahi runs most initiatives through a potential impact determination process, empowered by the Loops platform.
Loops has given the Wahi team new confidence in their decisions and has become an integral part of the quantitative analysis process, as well as the weekly, monthly and quarterly business reviews in which they prioritize their product and marketing initiatives.
