datainterview.org
Company deep dives

Booking.com interviews

Experimentation practice · quality and legitimate extremes · Mar–Jul 2026

Before the test starts

“Did the experiment include a power calculation?”

Edgar Cano, Daisy Duursma, Nils Skotara and Melanie MuellerCo-authors, Booking.com ML & DS Blog

Booking.com ML & DS Blog · 2026-03-24

Plan detectability

What evidence is needed before setting the test duration?

Example

“Did the experiment adhere to its planned runtime?”

Edgar Cano, Daisy Duursma, Nils Skotara and Melanie MuellerCo-authors, Booking.com ML & DS Blog

Booking.com ML & DS Blog · 2026-03-24

Keep the stopping rule

Can you extend the test after seeing the result?

Example

Keep the original decision

“Did the decision align with the hypothesis?”

Edgar Cano, Daisy Duursma, Nils Skotara and Melanie MuellerCo-authors, Booking.com ML & DS Blog

Booking.com ML & DS Blog · 2026-03-24

Preserve the hypothesis

Did the result answer the pre-specified question?

Example

When large users are the point

“When the intervention genuinely moves the tail”

Margarida Moreira da SilvaAuthor, Booking.com ML & DS Blog

Booking.com ML & DS Blog · 2026-07-01

Keep the intended outcome

Does a capped mean still measure the intended benefit?

Example

“winsorization will mask this real effect”

Margarida Moreira da SilvaAuthor, Booking.com ML & DS Blog

Booking.com ML & DS Blog · 2026-07-01

Report the capped estimand

Would a cleaner metric hide the agency feature’s purpose?

Example

Last reviewed