🍪 CRO Bites #14: CUPED Applied - 8 Tech Companies Sharing Insights

Hi there 👋

Most of us want to test faster, but random noise often forces us to keep experiments running longer than they should.

Enter CUPED (Controlled-experiment Using Pre-Experiment Data). Coined by Microsoft back in 2013, this powerful variance reduction technique has become an absolute game-changer for experimentation programs.

In a nutshell: CUPED uses historical data from before your experiment starts to clean up the noise in your current test. By filtering out this background noise, it dramatically increases both the speed and accuracy of your experimentation program, allowing you to detect true winning variations much faster.

What is particularly exciting is the wealth of practical knowledge that has been shared over the years. Several tech companies have published excellent papers and articles explaining how they apply CUPED to scale their testing velocity.

We curated the most valuable publications from 8 leading tech companies:

1. Microsoft (2013) | Improving the Sensitivity of Online Controlled Experiments by Utilizing Pre-Experiment Data

2. Netflix (2016) | Improving the Sensitivity of Online Controlled Experiments: Case Studies at Netflix

3. Booking.com (2018) | How Booking.com increases the power of online experiments with CUPED

4. Meta (2020) | Increasing the sensitivity of A/B tests by utilizing the variance estimates of experimental units

5. DoorDash (2020) | Improving Online Experiment Capacity by 4X with Parallelization and Increased Sensitivity

6. Airbnb (2021) | Zero to Hero: Exploiting Null Effects to Achieve Variance Reduction in Experiments with One-sided Triggering

7. Etsy

   1. (2021) | Increasing experimentation accuracy and speed by using control variates

   2. (2025) | Reducing experiment duration with predicted control variates

8. Google Cloud (2025) | Tune Out the Noise: Use CUPED to Save A/B Tests from Relaunching

Special thanks to all authors of the individual papers and blog articles for sharing your expertise!

Looking for more engineering insights? We curate these and other experimentation blogs directly at Conversion Stash.

Browse all Experimentation Blogs

Confidence Blog by Spotify

Explore expert articles on experimentation and technical insights from Spotify’s R&D team. Confidence is an experimentation platform by Spotify that helps you run experiments and make data-informed decisions throughout your entire business.

Explore Resource of the Month

Discover all CRO Resources

Join 200+ in-house leaders & innovators to explore and shape the future of how we build, ship, test, and work.

👉 Claim your FREE ticket: The organizers are looking for moderators to help out and are offering a few free tickets in return. Interested? Reach out to Ben Labay via LinkedIn DM (mention Conversion Stash) or email us directly.

Claim you Space in the Ring

• 🇳🇱 MeasureCamp Amsterdam | April 18, 2026 | Amsterdam, Netherlands

• 🇦🇹 Experimentation Unite Vienna | April 22, 2026 | Vienna, Austria

• 🇺🇸 Marketing Analytics Summit | April 27-30, 2026 | Santa Barbara, USA

• 🇩🇪 Berlin Experimentation Meetup #13 | April 28, 2026 | Berlin, Germany

• 🇬🇧 Space Academy by ExElite | AI That Works | May 6, 2026 | London, UK | 🏷️ Save 12 % with code CS12

• 🇬🇧 Space Academy by ExElite | Tomorrow’s World Today | May 7, 2026 | London, UK | 🏷️ Save 12 % with code CS12

• 🇬🇧 Circus (formerly known as EXL) | May 11-12, 2026 | London, UK

• 🇳🇱 Optimization Meetup Utrecht #2 | May 28, 2026 | Utrecht, Netherlands

• 💻 Best In Test Awards #2 | June 10, 2026 | Online

• 💻 The Women in Experimentation Summit | September 14-15, 2026 | Online

• 🇬🇧 Experimentation Elite Awards | September 24, 2026 | London, United Kingdom

• 🇳🇱 DDMA Experimentation Heroes #10 | November 5, 2026 | Amsterdam, Netherlands

• 🇳🇱 Conversion Hotel | November 20-22, 2026 | Texel, Netherlands

View Full Calendar on Luma

Compare the best experimentation tools and find the perfect fit for your organization.

Compare A/B Testing Tools

50 Experimentation Interview Questions
Hiring for CRO or preparing for your next role? This list of 50 essential questions will help assess candidates’ expertise in experimentation, analytics, and optimization strategy.

CRO Glossary Cheatsheet
Master the language of experimentation with our CRO Glossary Cheatsheet. This quick reference guide covers 100+ key terms, helping you navigate A/B testing, statistical significance, and optimization frameworks like a pro.

Discover all CRO Assets

Find 100+ experimentation related jobs on ExperimentationJobs.com. These jobs are from all over the world, on-site, fully remote or hybrid. Take a look and start pivoting your career.

This week’s featured roles:

• Medior CRO specialist at De Mandemakers Groep (Waalwijk, Netherlands)

• Data-analist (Experimentation & Digital Analytics) at Online Dialogue (Utrecht, Netherlands)

• Senior CRO Specialist at sofatutor (Germany & EMEA)

• Director, Web Experience and Optimization at Mimecast (Lexington, USA)

• Head of Digital Optimisation at Frasers Group (London, United Kingdom)

Are you hiring? Submit here

We love partnering with brands that resonate with our audience. Drop us a line at info@conversion-stash.com to explore sponsorship options!

Stay optimized,
Niklas & Nils from Conversion Stash