📄 Article
Free
Intermediate
Most founders call a paid campaign a winner or loser off a handful of clicks, and this piece walks through exactly why that read is usually noise. It puts real numbers behind the question: what baseline conversion rate, what minimum detectable effect, and how many conversions you need before the result means anything. Treat it as a starting point for setting your own threshold before you launch, not after you are staring at the dashboard.
Calculating Sample Size in A/B Testing: Everything You Need to Know
From GuessTheTest by Deborah O'Malley
- Decide your required sample size and minimum detectable effect before the campaign runs, so you are not tempted to stop early on a lucky day.
- Rough reliability wants thousands of visitors and hundreds of conversions per variant, far more than the trickle most early tests collect.
- Let a test run at least two full weeks to cover normal weekday and weekend swings, and avoid peeking and calling it early.
