JEV is a transformer that produces no text. You give it unstructured input and a set of questions whose answers you have defined in advance, and it returns typed answers with a confidence score, in a single forward pass rather than token by token. Shark Labs treats it as evidence for a pattern rather than a dependency: most decisions inside a product were never essay questions, and routing them through a text generator is the most common avoidable cost in a production AI feature.
A lot of the AI conversation this month is about JEV. Much less of it explains what the thing actually is, where it would sit in a system you already run, or what it is bad at. This is the version we would give a client, in plain language, followed by the part that matters more than the model itself.
What JEV is
JEV
A transformer that generates no text at all. It takes unstructured input plus a set of questions whose possible answers you defined in advance, and returns typed answers with a probability attached to each one. It cannot write you a paragraph, and that removal is the entire design.
A message lands in your queue: "The deploy failed twice and customers are seeing 500s. Can someone look now?" You ask two questions you defined earlier, whether it is urgent and which team owns it. A few hundred milliseconds later you get back urgent: yes (0.94) and category: infra (0.81).
There are three question shapes: a choice picks one of up to 255 options, a score rates something on ordered levels, and a noul is a yes or no that returns the probability the statement is true. Questions are evaluated in parallel, so asking ten costs roughly what asking one costs.
Why it is fast, and why that is structural
This is the part worth understanding, because it is the reason the numbers are not a tuning trick.
A language model writes one token, feeds it back into itself, and writes the next. That loop is why latency grows with how much it says, and why a one-word answer wrapped in JSON still has to be spelled out character by character. You pay for the packaging.
JEV has no loop. The answer comes out of a single forward pass, all at once. There is no output to stream because there is no output in the usual sense, only a value and a number.
- 70 to 500 ms
- End-to-end latency, per the vendor
- $0.042
- Per million input tokens. Output is not metered
- 255
- Maximum options in a single choice question
TypeSafe, who built it, call this a "System One" model, borrowing Kahneman's split between fast intuition and slow deliberation. Everything shipped so far is System 2: thoughtful, verbose and expensive. Their bet is that most decisions inside real software never needed System 2, and that the cheapest way to get a yes or no has been to make an essay-writing machine produce a one-word essay.
It does not replace your LLM. It sits in front of it
This is the part the hype skips, and it is the only part that changes an architecture. Four places it fits:
- Routing. Read the incoming request, decide whether it needs the cheap model or the expensive one. Most requests do not need the expensive one, and today most of them get it anyway.
- Guardrails. Classify a tool call as risky or safe before an agent is allowed to execute it. Cheap enough to run on every call, which is the only way a guardrail is worth having.
- Volume classification. Ticket triage, email sorting, sentiment over millions of rows. The work that is too expensive for a frontier model and too fuzzy for a regex.
- Real-time loops. Browser agents, trading, games. Anywhere one second is too slow.
The mechanism that makes all four work is the confidence number. It is trained with what TypeSafe calls Reinforcement Learning for Calibrated Decisions, the goal being that a 0.7 actually means 0.7. So you set a threshold. Above it, resolve immediately. Below it, escalate to the big model. Cheap layer in front, expensive layer behind, clean handoff between them.
const verdict = await classify(ticket, {
urgent: { kind: "noul" },
category: { kind: "choice", options: TEAMS },
});
// Confident enough to act on: route it and stop. No LLM call.
if (verdict.urgent.confidence >= THRESHOLD) {
return route(ticket, verdict.category.value);
}
// Not confident: this is the minority of traffic, and the only
// traffic that pays frontier-model prices.
return escalateToModel(ticket);The arithmetic
Numbers make this concrete, so here is a worked example. The assumptions are stated so you can put your own in: two million items classified a month, around 400 tokens each, so 800 million input tokens.
| Approach | Input price | Monthly input cost |
|---|---|---|
| Frontier model, one call per item | ~$3.00 / M tokens | ~$2,400 |
| JEV, one call per item | $0.042 / M tokens | ~$34 |
| Two-tier, 85% resolved by the cheap layer | mixed | ~$394 |
The third row is the honest one, and it is the row we would actually build. It still costs ten times the second row, because the hard 15% is exactly the traffic that needs a real model. What went away is the 85% that never did.
Notice the two-tier row does not depend on JEV. It depends on having *something* cheap and calibrated in front. That is the point of this post.
What it is bad at
- It never explains itself. There is no reasoning to log, so it is the wrong tool anywhere a decision has to be auditable. Lending, hiring, moderation appeals, anything a regulator or a customer can demand a reason for.
- It will not design your schema. A badly shaped question produces a confidently useless answer, and the quality of your options list is now the quality of your feature.
- The benchmarks are the vendor's own, measured against model-derived references rather than independent ground truth. The headline 193x faster and 445x cheaper are workload-specific claims, not universal multipliers.
- It is in early access. Early access opened mid-September 2026. That is a fine thing to prototype against and a poor thing to put on the critical path of a product you have to support for three years.
There is also a fair criticism doing the rounds, that this is a zero-shot classifier with unusually good marketing, and that fast structured output is already available by prefilling an existing model. We think that is right about the architecture and wrong about the framing, which brings us to the actual recommendation.
What we would do on a client project
Not adopt JEV. Adopt the shape of it, which you can do this quarter with tools that are already boring.
- 01Find the decisions. Go through your AI feature and mark every model call whose output is one of a fixed set of values. Route, classify, flag, score, is-this-safe. In most products we audit, this is the majority of calls by volume and a minority by cost awareness, which is why nobody has noticed.
- 02Put a cheap layer in front of them. A fine-tuned small classifier, or a small model with constrained output, or JEV once it leaves early access. Which one matters far less than the fact that something is there.
- 03Make it return a confidence, and set a threshold. This is the load-bearing piece. Without calibration you have a cheap layer you cannot trust, so you end up calling the big model anyway and paying for both.
- 04Measure the escalation rate, not the accuracy. The number that tells you whether this is working is what fraction of traffic the cheap layer resolves. If it is 30%, your schema is wrong. If it is 99%, your threshold is too loose and errors are getting through silently.
- 05Keep the expensive path. The two-tier design only works because the hard cases still reach a real model. A cheap layer with nothing behind it is just a worse feature.
Done in that order, the vendor becomes swappable and the architecture is the asset. If JEV wins, you point step two at it. If it does not, you have already banked most of the saving. Five patterns for putting an LLM into software that already exists covers where this sits relative to the other shapes, and what an AI feature actually costs has the rest of the cost model.
The one-paragraph version
The last three years taught everyone to reach for a text generator whenever a decision looked hard, and to pay text-generator prices for answers worth one bit. JEV is a correction to that, and it may or may not be the model that wins. The shape it is pointing at, intelligence as a cheap typed sub-second function call with an expensive model behind it for the hard cases, is going to stay, and you do not have to wait for early access to start building that way.