-
Harsha Gundala's answer to the two years in stealth, posted the morning after: a Qwen 2.5 1B set up to batch every key of a JSON schema in one pass and hand back probabilities over the categories, running on an M4 MacBook, on Hugging Face. His point is that any LLM can be made to do this without new training.
Harsha Gundala
They were building in stealth for 2 years, I was building in stealth for 2 hours…
Happy to open source Qwen-2.5-1B-RLCD, 5x faster on-device inference for JSON workloads that need to be type-safe.
⚡️Demo below on a M4 MacBook⚡️
every LLM has the ability to efficiently batch inference every key of a JSON at the same time and generate probabilities from a set of possible categories. No new training required, but it’s easy to optimize if you need!
On hugging face now!
Harsha Gundala's post on X, curated on eChai
eChai Partner Brands
The partnership →
