Dan Shipper is the CEO of Every. Mike Taylor, its head of evals, fed Jev 37 documents with 21 questions each and got 777 judgments back in 0.7 seconds for about a quarter of a cent. Every's second test is the one independent number so far: on 12 passages with planted writing defects Jev caught six of seven where Fable 5.1 caught all seven, at 0.35 seconds a passage against 8.83, which is 25 times faster rather than the 200 in the launch claim.
Dan Shipper
we almost never test new foundation models but we've been testing this for ~a week @every and it's pretty wild.
the kind of things that will be obviously indispensible in 6-12 months
it doesn't produce words as output, it produces probabilities. so it can efficiently act as a judge in cases where you'd need a Fable-level model—but in our testing was 25x faster and 600x lower priced
excellent vibe check by @hammer_mt on @every:
Andre Landgraf's showcase of the yes/no gate, several worked examples at the link.
Andre Landgraf
1h
Ok, jev is really cool. So many great use cases where a fast and accurate yes/no gate can deliver meaningful performance gains.
Played around and built some showcase examples here →
Vipul Sharma's Minecraft agents ask Jev whether to fight or run, from what the agent can see, the mob count, its weapon and its health; one broke off mid-fight as the odds turned.
Vipul Sharma
1h
I hooked up Jev by @typesafeai to a few agents on my Minecraft world. Look at how Jev decides when they should fight and when they should run away.
Jev rates the odds from what the agent can see, mob count, weapon, how hurt it is, and runs away when they are bad. One broke off mid-fight as its odds turned.
Mahmoud's simulation of fifteen autonomous drones through an asteroid field, Jev deciding for each in under 300 ms, all fifteen arriving.
Mahmoud
2h
Jev can control a swarm of 15 autonomous drones in real time!
I built a simulation to test it navigating an asteroid field:
• Decision latency: < 300ms
• Survival rate: 100% (all 15 reached the destination)
Built with @typesafeai by @CompleteSkeptic
