Sentiment Analysis with a Judgment-Only Model

#The scenario

You track what people say: product reviews, support tickets, social mentions, survey verbatims. Leadership wants a dashboard — positive/negative trends by product area — not ten thousand LLM-written summaries. Volume is high, the labels are known, and consistency matters more than eloquence.

#Why Jev fits

Sentiment classification is a textbook closed-label task. A judgment model gives you the same label vocabulary applied the same way to the millionth item as to the first — no drift in tone, no verbose caveats, no per-token generation cost. Because judging is cheap, you can classify everything: every review, every mention, every day, and re-run history when your label scheme evolves.

#Judgment design example

Structured input per item:

Judgments to define:

1. Choice: "Polarity?" → positive / negative / neutral / mixed.
2. Choice: "Primary aspect?" → pricing / quality / support / usability / performance / other — turns raw sentiment into actionable sentiment.
3. Score: "Intensity 0–100" → separates mild grumbling from rage-quit language; drives alert thresholds.
4. Binary: "Is this actually about us?" → filters out lookalike mentions and sarcasm-adjacent noise before it pollutes the dashboard.

#Where the LLM sits

Two places, both downstream:

• Ambiguity resolution: items where the polarity judgment is mixed or low-margin get an LLM second read with full context.
• Narrative reporting: the weekly digest humans read — "negativity around pricing up 12%, driven by annual-plan complaints" — is a generation task over aggregated verdicts, not over raw text. The LLM reads statistics and a few exemplars, not 50,000 reviews. That is the Jev → LLM pipeline economics applied to analytics.

#Watch-outs

• Sarcasm and domain slang are the classic failure modes — route low-margin cases to the LLM band rather than accepting the verdict blindly.
• Define "mixed" explicitly. Without it, mixed-sentiment items get force-fit into positive/negative and your trend lines lie.
• Re-baseline after scheme changes. Adding an aspect label makes this month's counts incomparable to last month's; re-judge history (cheap) rather than stitching datasets.

#Sources

• "Jev explained in 7min" — Caleb Writes Code — the classification-paradigm overview.
• "Jev is HERE. How to use it" — Greg Isenberg (~293K views, 1 day).
• "Jev + GPT-6 Astra = 400X Cheaper" — Jack Roberts (~21K views, 11 hours) — why judging everything and generating rarely is the cost win.

Unofficial fan-made handbook. Not affiliated with TypeSafe AI or jev.com.