The Value of Verifying Your Testing Data – By Microsoft

Microsoft runs thousands of A/B test experiments per month. And, bets millions of dollars on the outcomes of these experiments.
 
Not surprisingly, the company needs to be fully confident they can trust their A/B testing results.
 
Listen to this informative 17-minute interview to learn about:
  • Microsoft’s unique approach for validating their A/B test results, and what defines a trustworthy experiment.

  • The importance of page load speed. And, how a 4 millisecond increase in page load speed can fund the salary of one Microsoft engineer for a year!

  • Several real-life Microsoft experiments that would have been wrong, had the team not caught the issues, how the problem was detected, and what was done to fix the problem.

Applying this insight, you’ll walk away from this talk a better, more informed data analyst, with a true appreciation for what comprises meaningful A/B test results.

Microsoft runs thousands of A/B test experiments per month. And, bets millions of dollars on the outcomes of these experiments.
Not surprisingly, the company needs to be fully confident they can trust their A/B testing results.
Listen to this informative 17-minute talk to learn about:
  • Microsoft’s unique approach for validating their A/B test results, and what defines a trustworthy experiment.
  • The importance of page load speed. And, how a 4 millisecond increase in page load speed can fund the salary of one Microsoft engineer for a year!
  • Several real-life Microsoft experiments that would have been wrong, had the team not caught the issues, how the problem was detected, and what was done to fix the problem.

archived_tests-paywall_image jpg
oh_snap-mobile