Summary
Executives thought they could drive even more gasoline sales through further discounting to their members. They needed to measure if dropping the price per gallon drove more sales volume, validating their beliefs. Instead of relying on ‘tribal’ knowledge.
Take Our Content to Go
Background
A leading operator of quick service convenience stores & gas stations across North America sought Mosaic out to provide insights around their loyalty program. The firm manages a loyalty program through a mobile app to drive brand loyalty with customers and to increase the volume of goods sold. Executives thought they could drive even more gasoline sales through further discounting to their members. They needed to measure if dropping the price per gallon drove more sales volume, validating their beliefs. Instead of relying on ‘tribal’ knowledge, the company reached out to Mosaic to take a data driven approach on loyalty program analytics.
Experiment Design & Execution
In collaboration with our customer, Mosaic helped design, execute & measure a controlled experiment at just over 200 locations in a specific region for over 6 months. The goals of this test were to measure which promotion has the highest lift, measured both in terms of frequency and sales from loyalty customers. One of the promotions offered a flat per gallon discount every day and the other offered the same per gallon discount every day plus a much larger one-time discount when a customer reached a minimum monthly purchase threshold. To access these discounts, a consumer needed to download and sign up for an app-based loyalty program, and registered app users were randomly assigned to receive one of the two promotions.
Mosaic needed to design a live test for new loyalty offers to be delivered through a retail fuel loyalty program with controls for seasonality, regional differences, and past behavior for existing customers. The key metric for comparison in the tests was determined to be the total gallons purchased per month (30 days) for the users in each treatment group. Mosaic proposed a user-partitioning experiment with a staggered rollout of the new promotions. The staggered approach provided a more reliable estimate of the effects of the two test promotions relative to no rollback promotions. Geographic segments with sufficient sales volume were selected in conjunction with the gas station operator and a promotional schedule was rolled out to segmented regions at 14 day intervals. This facilitated a comparison between markets with and without the new promotions that controls for seasonal and market level changes.
The formal test of the primary hypothesis were regression based algorithms that tested the difference in average monthly fuel purchases between the two test groups overall. In addition to testing statistical significance of the hypothesis effect, a confidence interval on the relative lift in fuel purchases was developed to provide decision makers with 95% accuracy of additional purchases made due to the promotions.
Insightful Results
The pilot program and associated marketing efforts with discounting fuel sales substantially increased the app user base and participation in fuel promotions. The rate of new user signups more than quadrupled during the pilot, utilization of the per gallon promotion far exceeded legacy promotions and 20% more users engaged with the app on a weekly basis. The one-time quantity-based discount did not show an increase in purchases but did highlight some surprising insights. Not only did users interact with the app more, there was a slight reduction in time between consecutive transactions when the users had a one-time discount available to be redeemed.
The gas station operator requested these insights be distributed efficiently & effectively to different decision makers across the business. Mosaic developed an interactive dashboard to track consumer activity throughout the test. This dashboard was effective in communicating how various user groups responded to specific loyalty benefits with recommendations for how to structure future benefits more effectively.
Conclusion
This test proved the gas station operator should continue its per gallon discount, expanding to other states, explore more targeted challenge-based promotions with an eye towards driving down time between transactions and leverage newly engaged app user base for A/B testing and personalized targeting.
Contact Mosaic if you are interested in having us build you a similar pricing experiment!
