AB Testing: Bringing down delivery times by ~50%

Challenge

My client faced inefficiencies in the delivery process due to the current order batching system. With a rider batching limit of 4 orders per batch, instances of delayed order delivery occurred, where nearby orders were held back despite available riders, simply because the current riders had not yet reached their batch limit. This led to potentially longer delivery times, even with surplus riders on hand. As a food delivery app that challenged the legacy food delivery apps on an under 10 minute delivery, it was crucial to address this.

Solution

To address this, an AB test was designed to evaluate the impact of reducing the maximum number of orders batched per rider. Two test variations were implemented across different stores:

Test A : Reduced the maximum batch size from 4 to 2
Test B : Reduced the maximum batch size from 4 to 3

The independent variable was the change in the maximum number of orders per batch (2 and 3), and the primary dependent variable (success metric) was the average delivery time per day.

Impact

A 2 order per batch could effectively bring down delivery time to under 10 minutes without resulting in a rider deficit, or otherwise impacting operations.

Established AB testing as future tool to experiment for revenue and customer satisfaction optimization

Improve customer satisfaction and potentially increase order volume due to faster service

Topics:

AB Testing, Experimentation Design, Operational Efficiency, Revenue Optimization, Customer Satisfaction

Tools:

Python, Excel, BigQuery, MySQL, Google Cloud Platform, Scipy, Numpy, Pandas

Data:

Transactional Data from Vendor and internal systems accessed via BigQuery on Google Cloud and MySQL

QuickLook Images

Page updated

Google Sites

Report abuse