Recipe: Cascade Routing — Jev Coarse, Jev Fine, LLM Last

The pattern: arrange judgment in stages, cheapest first. A coarse Jev pass screens the entire firehose; a finer Jev pass refines whatever survives; an LLM (or a human) sees only the fraction where generation or deliberation actually pays for itself. Each stage is one or two orders of magnitude more expensive than the one before it, and each stage shrinks the volume handed to the next.

all items ──▶ Jev coarse filter ──▶ Jev fine classification ──▶ LLM / human
             (Noul: keep/drop)      (Choice: route; Score: rank)   (only the few %)

#Why the cascade shape

Because the cost asymmetry is the whole point of the paradigm. At $0.042/M input tokens with free output, Jev screening is cheap enough to apply to everything — that's the Jevons-paradox bet behind the name. Frontier LLMs are two-plus orders of magnitude more expensive per item once output tokens count, so every item a Jev stage eliminates is pure savings. Early-access demos frame the end-to-end gap as hundreds of times cheaper; treat the exact multiplier skeptically, but the direction is arithmetic, not marketing.

#A three-stage example: inbound email

def triage(email: dict) -> str:
    state = {
        "from": email["from"],
        "subject": email["subject"],
        "body_excerpt": email["body"][:2000],
    }

    # Stage 1 — coarse: is this worth ANY expensive attention?
    r1 = jev(state, {
        "needs_attention": {
            "type": "noul",
            "instructions": "Answer yes if a human or a strong AI should read this email today.",
        }
    })
    if r1["needs_attention"]["probability"] < 0.30:
        return "archive"

    # Stage 2 — fine: route + prioritize the survivors
    r2 = jev(state, {
        "queue": {
            "type": "choice",
            "instructions": "Pick the queue that should own this email.",
            "options": ["support", "sales", "billing", "legal", "personal"],
        },
        "urgency": {
            "type": "score",
            "instructions": "Rate how time-sensitive this email is.",
            "scale": 5,
        },
    })
    queue = r2["queue"]

    # Stage 3 — LLM only where confidence says the cheap stages weren't sure,
    # or the queue's SLA demands a drafted reply.
    if queue["confidence"] < 0.80 or r2["urgency"]["score"] >= 4:
        return llm_draft_reply(email, queue=queue["choice"])

    return f"auto_file:{queue['choice']}"

Note stage 2 uses Speculative Fan-Out — routing and urgency in one call — and stage 3 is a Confidence Gate with an LLM as the fallback target.

#Design rules

1. Each stage needs its own calibrated threshold. Don't reuse stage-1's cutoff at stage 2; measure each stage against its own labeled sample.
2. Keep stages independent in what they read. If the fine stage needs more fields than the coarse stage, re-fetch and rebuild state — don't smuggle a giant state through stage 1 "just in case" (State Design).
3. Mind the error compounding. Two stages at 90% each are 81% end-to-end on the surviving path. Cascades multiply mistakes as happily as they divide costs; that's why each gate keeps a fallback instead of forcing a verdict.
4. Respect the aggregate rate limits. Cascades make several Jev calls per item — fan out within a stage to keep the request count per item low (limits: 1200 req/min, subject to change).

#Where to go next

• The Jev → LLM pipeline — the two-stage minimal version
• Confidence Gating — the fallback mechanics
• Models & Pricing — the cost figures behind the math

#Sources

• learnjev.com — Cost & benchmarks (community documentation).
• jevai.wiki — API reference (community documentation).
• jev101.com — 什么是 Jev（中文） (community documentation, Chinese; application patterns like email triage and RAG filtering).

Unofficial fan-made handbook. Not affiliated with TypeSafe AI or jev.com.