Article

How to Do AI Analysis You Can Actually Trust

Caitlin Sullivan · Lenny's Newsletter Freemium Intermediate

If you are tempted to dump 20 transcripts into an AI tool and ask for the pattern, read this first. It explains why AI defaults to the loudest, most generic theme and can bury a sharper, less common signal that actually matters more. It gives a workflow for forcing an AI, or yourself, to keep exact quotes and flag contradictions rather than smoothing everything into one tidy summary. Useful if you want a second pass on your transcripts without fooling yourself.

  • AI analysis defaults to consensus themes and can miss a small but important minority signal.
  • Keep the customer's exact words in your working notes, do not let a tool paraphrase them away.
  • Explicitly look for contradictions between what people say and what they actually do.
Open on lennysnewsletter.com ↗

The question it answers

  1. I've done 20 interviews and every person described a slightly different problem. How do I know if there's a real pattern or just noise?

Read with it

  1. The single best thing ever written on customer conversations. It teaches you to ask about the customer's life and past... momtestbook.com Book · ~130 pages The Mom Test Rob Fitzpatrick
  2. A YC partner turns 'talk to users' into a repeatable 5-question interview script you can use this afternoon. It's The Mom Test... YC Startup Library Video · ~25 min How to Talk to Users Eric Migicovsky (YC / founder of Pebble)
  3. The definitive essay on where good ideas come from: notice problems you personally have, don't force it. Use it as the lens for... paulgraham.com Essay · ~20 min read How to Get Startup Ideas Paul Graham
  4. Turns the fuzzy phrase 'product-market fit' into a number you can move. Vohra's survey (the '40% would be very disappointed'... First Round Review Article · ~25 min read How Superhuman Built an Engine to Find Product/Market Fit Rahul Vohra (CEO, Superhuman)
  5. The essay that put 'product-market fit' into the startup vocabulary. Read it for the gut-level description of what PMF feels... pmarchive.com Essay · ~15 min read The Only Thing That Matters Marc Andreessen
  6. The author of Crossing the Chasm explaining beachhead selection in plain, modern language, the fastest way to absorb the... Lenny's Podcast / Lenny's Newsletter Podcast · ~75 min Geoffrey Moore on finding your beachhead, crossing the chasm, and dominating a market Lenny Rachitsky & Geoffrey Moore
eChai Partner Brands The partnership →