Scott Williams with the bull case, made as a scenario: every company already running an agent that reviews each ticket, invoice or claim through a frontier model is paying seconds and cents for a call that is usually obvious, and has years of humans making the same call on record to fit a cheap decision model to.
Prediction: millionaires will be made using custom Jev style models (parallel constrained decoding) to make the agent systems companies already run more token efficient.
Let me explain with a scenario:
Imagine a company already has an agent workflow running where an llm reviews every item before it moves on: a support ticket gets triaged, an invoice gets approved or held, a claim gets flagged.
Every one of those goes through a frontier model today, a few seconds and a few cents each, on the way to a decision that in most cases is obvious. Behind that flow sits years of humans (or agents) making the exact same call, with the outcome attached.
Now imagine you first run each item through a custom PCD or similar model that costs a fraction of the llm and returns a classification of what to do at that step, with a mathematically accurate probability attached.
When it's confident, the item skips the llm entirely.
When it isn't, the llm handles it as normal.
The model has seen years of your team making this exact decision, usually a constrained set of decisions, so it should be right most of the time. Say it comes back confident on 6 out of 10 items. That's more than half your llm spend potentially gone from that step, likely with comparable accuracy.
This pre processing idea works in a bunch of other use cases too, such as:
- model/request routing: cheap model, frontier model, or a human
- picking which skill or subagent to load for a turn instead of stuffing the whole catalog into context
- reranking retrieved context so only the relevant chunks reach the window
- guardrails on every agent turn: contradictions, policy issues, prompt injection
- extracting typed fields from unstructured data emails, PDFs and transcripts before anything expensive touches them
Every one of those is a decision an llm makes today, that could potentially be done by another, cheaper model class. Very excited to see Jev/PCD-based pre processing use cases get deployed to agents at scale.
