I got access to Jev earlier today (thank you @hackgoofer). I have run ~5,000 requests so far, (which cost me around $2!), across classification, model routing, intent, steering, and many other things.
tl;dr, Jev enables a new intelligent decision-making primitive, separate from deterministic code and LLM calls. This allows a class of decision-making that was neither suited to dumb, unintelligent code, nor to slow, expensive LLMs.
It is super fast and cheap, and I think I will likely end up making a few Jev calls to every LLM call I make in my product. I think probably any company using LLM requests today can probably add a Jev call pre and/or post LLM calls to quite literally make their product much better for free, and have better tool calling behavior in many cases.
I happened to have a personal benchmark for this as I’d been working on a ton of proactivity and classification tasks. I have been using the deepseek flash and more recently gpt 5.6 luna family of models as reasonably smart classifiers with low latency. Think questions like:
- Did this conversation output contradict something they’ve mentioned before?
- Should we send a followup message to this user based on our rules?
- It’s been a few seconds of silence. Should we proactively send a message?
In the past, I’ve been forced to write a bunch of what I call decision chains, mostly because an LLM classification call is very expensive in TIME (avg 4s), and less importantly can cost quite a bit if run on every message.
Imagine a normal chat app. If you added 4s to every response to figure out if the response is good before sending it out, that ends up being pretty bad. So instead, I usually have to write some code that is a crude heuristic that runs quickly and decides whether to run the classifier. Obviously this sucks because you call the classifier many times that you don’t want to, which makes your p95 bad, and you also miss cases with the heuristic, and you also have to manage all of these weird chains.
With Jev, it’s cheap enough, and fast enough (p50 ~150ms, p95 ~350ms in my testing!) that you can easily run it every turn. Heck you can reasonably run it before generation AND post generation, for any application that isn’t realtime voice, and still feel snappy.
But this is just one use case. Think: smarter model routing, better context packing, better responses, better observability for intent/tags/safety, smarter retries and so much more. By simply thinking about the inputs and outcomes you want to enforce, you can use Jev to supercharge most model calls and reduce bad user outcomes. The more “quirks” a model has, the more valuable it ends up being.
It’s a bit weird and unintuitive using Jev. Generally, you want to decrease the # of questions you ask a classifier, or it makes more mistakes. In fact, you might want to ask your questions kind of in a compound way, because the reasoning happens in a shared scratchpad of sorts. Adding questions muddies the scratchpad and makes it take longer.
With Jev, you feel incentivized to go the other way, to formulate your query as a set of independent questions. It doesn’t feel like adding more questions decreases your performance on others.
You can go a bit deeper to improve tool calls. Many model tools are things like turning on settings, or other things. You can easily improve models that are not very good at tool calling with Jev, by simply figuring out when to run them. You can do a pre-LLM call to figure out when to unfurl different tool definitions, in order to make your main LLM run better, you could run a background task with Jev + another LLM to reduce tool and context burden on your main LLM, and free it to be responsive.
I’ve only scratched the surface of my testing but very excited!
