Can your control share the treatment’s drivers?
A dispatch change speeds treated deliveries while both groups use one driver pool.
The question
Larkspur, a parcel courier, pilots a new parcel-dispatch rule. Customers are randomly assigned, but every eligible order competes for the same 100 available couriers. Treated orders get earlier access to nearby couriers. The team reports customer-level outcomes as if the groups did not affect one another.
| Orders | Late orders | |
|---|---|---|
| Control | 200 | 50 |
| Treatment | 200 | 30 |
Authored data. One shared courier pool serves both arms. No zone-level or time-block outcomes are supplied.
- DispatcherA courier reserved for a treated order is unavailable to control orders until that trip ends.
- Product leadCan we say the rule cuts lateness by ten points for everyone?
“Interpret the readout and design a more useful next experiment.”
Not saved by this site. Your browser may restore text when you navigate back.
A strong answer, step by step
Open each step in turn. This is one strong answer, not the only one.
Step 1: Calculate without overstating
What would you do here? Open the move you’d make.
Report the contrast with its limit
The strong move. The observed arm contrast and the all-market rollout effect are different claims.
Promise a ten-point market-wide reduction
Tempting, but it misses something. At full rollout there is no untreated group from which to take couriers.
See how a strong candidate says it
“Observed lateness is 50/200 = 25% in control and 30/200 = 15% in treatment: a −10 percentage-point contrast. That does not establish a ten-point reduction if the rule is used for everyone.”
Why it works: The observed arm contrast and the all-market rollout effect are different claims.
Step 2: Trace the interference
What would you do here? Open the move you’d make.
Map the shared-supply path
The strong move. Assignment is randomized, but outcomes can still depend on other customers’ assignments.
Add more customer rows to solve interference
Tempting, but it misses something. More observations do not remove a design mismatch.
See how a strong candidate says it
“Giving treated orders first access changes the supply available to control. Treatment can affect control outcomes through courier capacity. Customer randomization does not remove that path.”
Why it works: Assignment is randomized, but outcomes can still depend on other customers’ assignments.
Step 3: Change the assignment design
What would you do here? Open the move you’d make.
Use exactly two zones and count every order as independent
Tempting, but it misses something. Few assignment units and cross-zone travel can leave uncertainty understated and spillovers unresolved.
Test a defensible market/time design
The strong move. A cluster or switchback design has its own spillover, carryover and balance assumptions.
See how a strong candidate says it
“Consider assigning operationally separate markets or time blocks to the rule. Check courier movement across boundaries and trip carryover between blocks. Choose washout and block length from operating evidence, not a convenient clock interval.”
Why it works: A cluster or switchback design has its own spillover, carryover and balance assumptions.
Step 4: Measure the whole marketplace
What would you do here? Open the move you’d make.
Measure net outcomes and assignment-unit uncertainty
The strong move. A faster treated order may displace service elsewhere; the business needs the net effect.
Optimize treated-order speed alone
Tempting, but it misses something. That can reward taking capacity from other customers without improving the marketplace.
See how a strong candidate says it
“Pre-specify overall late deliveries, throughput and courier earnings, plus cancellations and customer guardrails. Analyze at the assignment structure with enough independent units. Ask for assignment, trip and supply logs before a power plan.”
Why it works: A faster treated order may displace service elsewhere; the business needs the net effect.
The answer, in short
- Decision
- Do not infer the full-rollout effect from this customer-level contrast.
- Because
- The arms compete for the same couriers, so the control is exposed to a treatment spillover.
- What would change it
- A design with credible separation/carryover handling and enough independent assignment units could support a rollout estimate.
Common traps
- Treating 400 orders as 400 independent assignment units in a new cluster design.
- Ignoring displaced service and courier outcomes.
Check your answer
- Did you calculate 25% and 15%?
- Did you name the shared-courier interference path?
- Did you test separation and carryover assumptions?
- Did you ask for net marketplace outcomes?
Last reviewed Nov 10, 2026