Recipe: Confidence Gating — Automate the Sure, Escalate the Rest

The pattern: never let a borderline Jev verdict drive an irreversible action. Define a high-confidence zone (act automatically), a low-confidence zone (fall back to a human or an LLM), and — for yes/no questions — a confident-negative zone (auto-reject). The gate is where Jev's calibration meets your risk tolerance.

#The three zones

Noul probability p:

p ≥ T_yes        → auto-YES   (approve, send, publish)
T_no < p < T_yes → ESCALATE   (human queue or LLM second opinion)
p ≤ T_no         → auto-NO    (discard, archive, skip)

For Choice and Score, use the returned confidence field (and the distribution shape) instead of a single probability — the concept is identical: confident → act, uncertain → escalate.

Concepts first: Confidence & Calibration explains what the fields mean and why you must measure thresholds on your own data rather than trusting launch numbers.

#Reference implementation

from typesafe import Client

client = Client()

T_YES = 0.95   # auto-approve above this — set from YOUR calibration run
T_NO = 0.10    # auto-reject below this

def moderate_review(review: dict) -> str:
    result = client.systemone(
        model="jev-1.13.0",
        state={
            "review_text": review["text"],
            "reviewer_account_age_days": review["account_age_days"],
            "reviewer_prior_removals": review["prior_removals"],
        },
        questions={
            "is_fake": {
                "type": "noul",
                "instructions": "Answer yes if this review is fake, paid, or bot-generated.",
                "criteria": "Genuine negative reviews with specifics are not fake.",
            }
        },
    )
    p = result["answers"]["is_fake"]["probability"]

    if p >= T_YES:
        return "auto_remove"
    if p <= T_NO:
        return "auto_publish"
    return "human_queue"      # the middle zone is a feature, not a failure

#Choosing thresholds honestly

• Pull thresholds from your calibration curve, not from vibes or launch posts. Independent re-tests have found Jev accuracy below official claims and highly task-dependent — the recipe is only as safe as your measurement. See the measurement protocol in Confidence & Calibration.
• Price the two error directions separately. Auto-removing a genuine review (false positive) and publishing a fake one (false negative) rarely cost the same. Asymmetric costs → asymmetric thresholds.
• Size the middle zone by budget. A wide middle zone means more human/LLM spend and fewer mistakes; narrow means the reverse. The gate turns "how much accuracy can we afford?" into a slider you actually control.
• Log every gated decision with its probability. That log is the calibration set for your next threshold revision — and your early-warning system when input distributions drift.

#Escalation targets: human or LLM?

• Human when the judgment needs accountability, taste, or context nobody has written down (content policy edge cases, fraud).
• LLM when the item needs reading and reasoning at length — the LLM's verdict is slower and pricier but it can explain itself, and you can capture that reasoning for the record. This is one stage of the broader Cascade Routing pattern.

#Where to go next

• Confidence & Calibration — the measurement protocol
• Cascade Routing — gating as one stage of a bigger funnel
• Speculative Fan-Out — gate on a bundle of verdicts, not one

#Sources

• learnjev.com — Cost & benchmarks (community documentation; calibration measurement and honest accuracy reporting).
• jevai.wiki — API reference (community documentation; response fields).
• jev101.com — 什么是 Jev（中文） (community documentation, Chinese).

Unofficial fan-made handbook. Not affiliated with TypeSafe AI or jev.com.