Deciding with Jev AI

Hassan El Mghari's post on X, curated on eChai

  • Triage and classification

    Curated from X →

    Hassan El Mghari, of Together AI, with the pattern several people arrived at this week: Jev first, and anything it is unsure about goes to a bigger model. Jev classified 100 emails in 1.42 seconds, the 31 under 95% confidence went to Kimi K3, and the pair got 96 of 100 for about seven cents, of which Jev's share was a third of a cent.

    Hassan El Mghari Hassan El Mghari

    Jev + Kimi K3 for fraud detection!

    TLDR: Jev classified 100 emails in 1.42 seconds, then I routed the uncertain cases to Kimi K3. The full pipeline got 96/100 correct for only ~$0.07.

    Video is not sped up, check out the live run!

    Here was my process:

    I gave Jev 100 emails to classify (a mix of 50 legit & 50 fraudelent emails). It classified all of them in 1.42 seconds.

    An underrated feature about Jev is it will give you the confidence score for a classification, so I routed any prediction under 95% confidence to Kimi K3 to be fully sure.

    31 emails fell below that threshold. After routing those to Kimi K3, the combined pipeline reached 96% accuracy.

    The full run took 16 seconds & ~$0.07 in inference costs:

    - $0.068 from Kimi K3 on @togethercompute
    - $0.003 (1/3 of a cent) from Jev on @typesafeai.

    I think this is a really interesting pattern: use a fast specialized model like Jev for the narrow task, then route the uncertain cases to a larger LLM.

    I feel like this kind of approach could be a game changer for use cases like fraud or anything realtime. You can use the speed & low cost of Jev while having a larger LLM as a fallback to ensure high accuracy.

eChai Partner Brands The partnership →