Article

Synthetic users vs real user research: why AI falls short

Radical Product Thinking · radicalproduct.com

The crispest articulation of the unknown-unknowns problem with AI feedback.

Open on radicalproduct.com ↗

It answers 2 questions

  1. Can synthetic users and AI personas replace talking to real customers?
  2. How do I test my AI prototype with real users, and can AI simulate user feedback?

Read with it

  1. The most rigorous framework for when AI research participants are and are not legitimate. nngroup.com Article Synthetic users: if, when, and how to use AI-generated research Nielsen Norman Group
  2. Purpose-built for this exact workflow, including prototypes from Figma Make, Lovable, Bolt, and Replit. maze.co Guide How to test and validate AI prototypes with users Maze
  3. The fastest way to get task-based usability data on a clickable prototype. maze.co Tool Maze prototype testing Maze
  4. A neutral overview of where AI genuinely speeds up usability testing. figma.com Guide AI usability testing guide: modern workflow and tools Figma
  5. Turns 'send the link around' into a structured test with real findings in about an hour. reforge.com Article How to get structured user feedback on your AI prototypes Reforge
  6. The foundational YC lesson on extracting truth from user conversations. ycombinator.com Video How to talk to users Y Combinator
eChai Partner Brands The partnership →