Also just referred to as MDE.
The larger the MDE, the smaller the sample size needed to run your experiment.
And vice versa. The smaller the MDE, the bigger the sample required for your experiment to be adequately powered.
As a very basic guideline, you should consider setting the MDE to a relative percentage uplift of up to 5%.
If the experiment isn't powered enough to detect a 5% effect, the test results can't considered trustworthy.
Note, the MDE is different from what some experimenters call the Minimum Effect of Interest (MEI) which is viewed by some an an "input parameter" where as the MDE is seen as an "output parameter".
However, typically, the terms MEI and MDE are used interchangeably, and MDE is what's displayed on most sample size calculators.
An additional note, the observed effect, or treatment effect, is altogether different. The observed/treatment effect is the conversion difference detected.
Test results are only trustworthy when the treatment effect is larger than the MDE.« Back to Glossary Index