When should a local decision model replace an LLM classification call?

An agent can mix two different jobs: producing new content and choosing from options its developer already knows. I built JevStyle for the second job. Its typed questions ask for a yes/no probability, a probability distribution over choices, or a score on an ordered scale.

Start with one decision

For a support inbox, that decision might be choosing the team that should receive a ticket. Keep the route descriptions the same across examples and include a review path for ambiguous inputs. A local model can propose the destination without being allowed to issue a refund, edit account data, or execute a tool.

Here is the shape of a request with the Python package:

from jev_style import JevStyle, choice

js = JevStyle()  # default: the local 0.8B text model
result = js.decide("My card was charged twice for the same order.", {
    "team": choice("Which team should handle the main request?", {
        "billing": "Charges, payment failures, invoices, or duplicate payments.",
        "shipping": "Delivery tracking, delays, or a parcel that has not arrived.",
        "returns": "Return or exchange of an item that arrived damaged or in the wrong size.",
    }),
})
print(result["answers"]["team"]["probabilities"])

The local routing walkthrough includes installation and a review fallback. The support-ticket example shows how to keep the model's recommendation separate from the action your application takes.

Evaluate the handoff, not just the label

The important question is how much of your own workload the model can handle at an acceptable error rate. A benchmark score on another dataset does not answer that. Collect representative labeled requests, hold back a final test set, and use separate development and calibration data for the question wording and threshold.

The jev-style eval command reports accuracy, Brier score, expected calibration error, and observed coverage at several error budgets. The automation threshold applies to the top option probability, not the API's separate normalized confidence field. A threshold selected from the same sample whose error is reported is exploratory, not an independent validation result.

Start in shadow mode: keep the current workflow in control while logging the proposed route, its probability, and the eventual human label. Then examine wrong high-confidence decisions as well as the proportion that would go to review. Measure latency in your own deployment, including the parts that users actually wait for.

The evaluation walkthrough explains the metrics and the split between selecting a threshold and testing it.

Choose the model for the input

For a small text-only starting point, use Jev-Style-0.8B-Decision-v3. For images such as receipts or screenshots, use Jev-Style-2B-Decision-v4 with the PyTorch runtime. The 2B v4 MLX and GGUF files contain its language model only. All builds require the matching decision runtime rather than a generic text-generation command.

The v4 Collection groups the model, formats, and browser demo. The demo uses ZeroGPU and is subject to its usage limits. The separate JevStyle hosted API is temporarily under maintenance as of October 28, 2026; the local package and model downloads are available.

The repository also includes an adoption skill that helps identify label-only LLM calls and prepare a migration with the original model as a fallback. The goal is a measured change to one decision step: keep it only if your quality, latency, and operational requirements improve.

Code and installation: lawrence3699/jev-style.

This is an independent project, not affiliated with TypeSafe AI. Model cards document the training data, licenses, evaluation protocols, and limitations. Published self-runs are not official leaderboard entries. Concrete failure cases on ambiguous routing, uncommon categories, and mixed-language inputs are welcome.