The opposite of beta (β).
Assesses the likelihood a correct decision has been made that there is, in fact, a real effect, or conversion difference between variants.
Power is calculated as 1 - β which gives a decimal value.
A power of 0.80 is the standard.
It means 80% of the time a meaningful effect will be detected. As such, there's only a (0.20) 20% chance of missing detecting the effect and ending up with a false negative.
In testing, your aim is to ensure you have enough power to meaningfully detect a significant effect, or difference in conversion rates to reduce your chances of a type II error.
The higher the power, the more likely a real effect has been detected.
However, the probability of finding the effect, within a given degree of confidence, is impacted by sample size. The larger your sample size, the higher your power.
« Back to Glossary Index