Essay

The Case Against A/B Testing at Early-Stage Startups

Andrew Cohen · Entrepreneur Free Beginner

This is the honest counter-argument most growth advice skips: when you have low traffic, formal A/B tests usually cannot reach significance for weeks or months, and the setup cost is real. Cohen, a founder himself, says name the problem plainly and lean on focus groups, hallway usability tests, and judgment until you actually have the volume. Read it as a starting point for deciding whether a test is even worth running yet, not as a verdict against ever testing.

• Young products rarely have the sample size to make an A/B result trustworthy, and running one anyway burns founder time better spent on product-market fit.
• Early traffic is often skewed (one launch, one ad blast), so even a significant-looking result may not reflect real users.
• Small qualitative checks like a quick usability session or talking to a few users can be enough signal at this stage.