Article · ~30 min

Things we learned about LLMs in 2024

Simon Willison · simonwillison.net Free Intermediate

Willison is a working engineer who writes plainly about what the technology can and cannot reliably do, which is the single hardest thing for a non-technical founder to gauge. He is honest about hallucinations, confidently wrong answers, and the gap between an impressive demo and a dependable product. This gives you the vocabulary to ask a builder sharp questions about what breaks in production.

  • Capable models still produce confident wrong answers, so reliability is the real test
  • A model that dazzles in a demo can fail on messy real inputs
  • Treat these tools as fast but fallible assistants, not oracles
Open on simonwillison.net ↗

The question it answers

  1. As a non-technical founder, how do I evaluate a tech-heavy trend like AI without getting fooled by hype?

Read with it

  1. The single best thing ever written on customer conversations. It teaches you to ask about the customer's life and past... momtestbook.com Book · ~130 pages The Mom Test Rob Fitzpatrick
  2. The definitive essay on where good ideas come from: notice problems you personally have, don't force it. Use it as the lens for... paulgraham.com Essay · ~20 min read How to Get Startup Ideas Paul Graham
  3. Video · ~25 min How to Talk to Users Eric Migicovsky
  4. This is the canonical piece that put jobs to be done on the map, written by the people who coined it. It uses the famous... Harvard Business Review Article · ~20 min read Know Your Customers' Jobs to Be Done Clayton Christensen et al.
  5. Video · ~50 min How to Evaluate Startup Ideas Kevin Hale, Y Combinator
  6. Perez shows that real technological revolutions run on 50 year cycles with a predictable frenzy, a crash, and then a long... Goodreads Book · 224 pages Technological Revolutions and Financial Capital Carlota Perez
eChai Partner Brands The partnership →