Harsha Gundala's answer to the two years in stealth, posted the morning after: a Qwen 2.5 1B set up to batch every key of a JSON schema in one pass and hand back probabilities over the categories, running on an M4 MacBook, on Hugging Face. His point is that any LLM can be made to do this without new training.
Harsha Gundala
They were building in stealth for 2 years, I was building in stealth for 2 hours…
Happy to open source Qwen-2.5-1B-RLCD, 5x faster on-device inference for JSON workloads that need to be type-safe.
⚡️Demo below on a M4 MacBook⚡️
every LLM has the ability to efficiently batch inference every key of a JSON at the same time and generate probabilities from a set of possible categories. No new training required, but it’s easy to optimize if you need!
On hugging face now!
Andre Landgraf's showcase of the yes/no gate, several worked examples at the link.
Andre Landgraf
1h
Ok, jev is really cool. So many great use cases where a fast and accurate yes/no gate can deliver meaningful performance gains.
Played around and built some showcase examples here →
Vipul Sharma's Minecraft agents ask Jev whether to fight or run, from what the agent can see, the mob count, its weapon and its health; one broke off mid-fight as the odds turned.
Vipul Sharma
1h
I hooked up Jev by @typesafeai to a few agents on my Minecraft world. Look at how Jev decides when they should fight and when they should run away.
Jev rates the odds from what the agent can see, mob count, weapon, how hurt it is, and runs away when they are bad. One broke off mid-fight as its odds turned.
Mahmoud's simulation of fifteen autonomous drones through an asteroid field, Jev deciding for each in under 300 ms, all fifteen arriving.
Mahmoud
2h
Jev can control a swarm of 15 autonomous drones in real time!
I built a simulation to test it navigating an asteroid field:
• Decision latency: < 300ms
• Survival rate: 100% (all 15 reached the destination)
Built with @typesafeai by @CompleteSkeptic
