Article

LLM Token Optimization: Cut Costs and Latency

Redis · redis.io

Engineering-level detail on token budgets and semantic caching from an infra company that measures it.

Open on redis.io ↗

The question it answers

  1. How much does it cost to run AI features, and how do I keep costs sane?

Read with it

  1. The definitive chart: equivalent-quality inference gets 10x cheaper per year, which changes what you can afford to build. a16z.com Article Welcome to LLMflation: LLM inference cost is going down fast Guido Appenzeller (a16z)
  2. The five-minute version of the cost-decline argument, useful for convincing a worried co-founder. x.com Thread a16z on LLMflation (thread) a16z
  3. Cached reads cost 10% of normal input price; the single highest-leverage cost lever, from the primary source. platform.claude.com Guide Prompt caching (Claude docs) Anthropic
  4. A flat 50% discount for anything that can wait up to 24 hours, like digests, enrichment, and backfills. developers.openai.com Guide Batch API (OpenAI docs) OpenAI
  5. Grounded numbers on what AI COGS actually look like across real startups, not vibes. tanayj.com Article The State of AI Gross Margins in 2025 Tanay Jaipuria (Wing VC)
  6. Fresh survey data showing margins improving from 41% to 45% and what the improvers did differently. upstartsmedia.com Article Exclusive Data: Startups Are Learning What AI Profits Look Like Upstarts Media
eChai Partner Brands The partnership →