Article

Calculating Sample Size in A/B Testing: Everything You Need to Know

Deborah O'Malley · GuessTheTest Free Intermediate

Most founders call a paid campaign a winner or loser off a handful of clicks, and this piece walks through exactly why that read is usually noise. It puts real numbers behind the question: what baseline conversion rate, what minimum detectable effect, and how many conversions you need before the result means anything. Treat it as a starting point for setting your own threshold before you launch, not after you are staring at the dashboard.

• Decide your required sample size and minimum detectable effect before the campaign runs, so you are not tempted to stop early on a lucky day.
• Rough reliability wants thousands of visitors and hundreds of conversions per variant, far more than the trickle most early tests collect.
• Let a test run at least two full weeks to cover normal weekday and weekend swings, and avoid peeking and calling it early.