-
Sean Goedecke is a staff engineer at GitHub who writes about how software is built, and this is the most careful engineer's read of the launch. He takes the interface seriously as something new and the moat less so: most of what Jev does, he argues, is available today by prefilling an ordinary model's answer and sampling one token, and he expects other labs and open-weights tinkerers to catch up. The re-creations on this page are the early evidence for that part of it.
Sean Goedecke, in seangoedecke.com
Fast structured output could be a genuinely new computational primitive for intelligence. So far we've built a lot of programs on top of autoregressive token generation, and they all look like fancy chatbots. Leaning hard into structured output might conceivably unlock a bunch of non-chatbot use cases for AI.
[...]
If you want fast, parallelized structured output against limited choices, you don't strictly need to do autoregressive generation at all. You can simply prefill the response with "choice": " and generate one token, restricted to the user-provided choices. [...]
eChai Partner Brands
The partnership →
