Article

Things we learned about LLMs in 2024

Simon Willison · simonwillison.net

The most-cited year-in-review; explains why the leader changes constantly and why lock-in is a trap.

Open on simonwillison.net ↗

The question it answers

  1. Which AI model should I build on (GPT, Claude, Gemini, open-source), and how do I avoid locking myself in?

Read with it

  1. The independent standard for comparing intelligence, speed, and price on one chart. artificialanalysis.ai Tool LLM Leaderboard: Comparison of over 100 AI models Artificial Analysis
  2. Continuously updated rankings pulled straight from provider APIs; good second opinion. llm-stats.com Tool LLM Stats: Compare 300+ models by intelligence, speed, price LLM Stats
  3. Product-builder-oriented leaderboard that maps benchmark scores to practical use cases. vellum.ai Tool Vellum LLM Leaderboard Vellum
  4. Fastest way to sanity-check what any model actually costs before you wire it in. pricepertoken.com Tool Price Per Token leaderboards Price Per Token
  5. A structured comparison across reasoning, coding, context, and compliance rather than hype. intuitionlabs.ai Article Claude vs ChatGPT vs Copilot vs Gemini: Enterprise Guide IntuitionLabs
  6. Side-by-side output quality on real writing tasks, the kind of eval founders should copy. blog.type.ai Article Who Wrote it Better? Claude vs ChatGPT vs Gemini Type.ai
eChai Partner Brands The partnership →