Sharing a small thing I built. It turns an LLM into a constrained decision-maker instead of a text generator.
The idea: you give it a situation, a question, and your own list of 2–26 options. It returns exactly one of them in a single forward pass the choice is read from the next-token logits over the option markers (A, B, C…), not generated as text. So the output is structurally guaranteed to be one of your keys. Nothing to parse, nothing to hallucinate outside the set.
I know constraining output to a set isn't new constrained decoding, classifier heads, and JSON-schema modes all do it. The constraint is the easy part. What I actually spent the effort on: a small LoRA (rank 8, q_proj/v_proj only) trained on contrastive counterfactual pairs so it reads the state instead of latching onto marker position, then merged back into the base. Plus a fitted temperature so the confidence is calibrated and you can gate act if conf ≥ 0.9 else escalate.
How it's made: base Qwen3-4B, behaviour merged in (only 72 weight matrices differ from base, everything else bit-identical). Loads with stock transformers. Also exposes an MCP tool so agents can call it directly.
Honest eval (this exact checkpoint):
- 81.6% on 500 held-out contrastive items; ECE 0.143 (hence the temperature).
- ~85% on a hard contrastive reasoning probe (double-negation, quantifier-negation, implicature, counterfactual…).
- I just stress-tested it on toxicity + PII classification. Clear cases: 12/12, confidence ~1.0.
Hard/adversarial cases: toxicity 3/6, PII 4/6. The genuinely ambiguous ones mostly landed in the low-confidence band (0.4–0.6) where the gate sends them to review but it has confident misses too: a sarcastic insult read as "clean" at 0.98, and a venue address flagged as personal "contact" at 0.98. Sarcasm and reported/quoted speech are its blind spots and I'm not hiding that.
- Known weak spots: multi-step arithmetic and Winograd-style coreference, and errors there are often confident gate and fall back.
Use cases I've found it good for: agent control flow (tool routing, RAG-vs-answer, escalation), guardrails (it refused a phishing draft and blocked rm -rf / on prod in my tests), ticket triage, intent, sentiment any fixed-label routing, since the options are defined per request so one model covers all of them.
Live demo (Hugging Face ZeroGPU, no setup has a prev/next example cycler incl. agentic cases):
https://huggingface.co/spaces/mo19788/mini-typesafe-demo
Weights (Apache-2.0):
https://huggingface.co/mo19788/mini-typesafe-qwen3-4b
Would genuinely like feedback on the approach and the calibration and if anyone has a labeled routing/moderation set they'd point it at, I'd love to see where it breaks.