usejul/jul-decision-minicpm5-2b-mlx-4bit

jul-decision-minicpm5-2b (MLX, 4-bit) — v2

A decision model: one text (the state) and typed questions in (Choice, Noul yes/no, Score), a probability per option out, in one forward pass. It never generates text. MiniCPM5-2B with a merged LoRA (r=16) and a pointer head, converted to MLX and quantized to 4 bits (group size 64, 1.4 GB) for Apple Silicon. The unquantized PyTorch weights are usejul/jul-decision-minicpm5-2b. Against them, on 40 held-out questions, 1 answer differs and the median probability gap is 0.002: 4 bits flip near ties.

Made for jul: local typed decisions with the interface of TypeSafe's (Jev) Python SDK. This repo was called minicpm5-2b-decision-mlx-4bit until v1.1; the old name redirects here.

What changed in v2

• Trained on human-written data. v1 was trained on Kev's decision mix (with LLM-written and templated texts); v2 on about 30,000 typed decisions whose texts were written by people, labelled by the datasets' own annotators, from commercially usable sources, English and French (30%), plus 20% generated rows (emotion traps, date comparisons, compositions) and 1,500 extra sentiment scores, decontaminated against our evaluation sets (8-gram and embedding filters).
• Score is read by the vectors. The pointer head reads Choice and Noul better than the vector reading of the same weights, and Score worse (it rarely picks an end of the scale). decision.json now says so ("routing": {"types": ["score"]}) and jul reads Score with the vector reading it fits on these weights. This needs a jul with type routing (merged in main, next release); an older jul reads Score with the pointer head.
• One epoch, temperature 1.26 fitted on held-out rows.

The previous versions stay available: revision="v1.1" and revision="v1.0".

Use it with jul

jul models add jul-decision-minicpm5-2b --repo usejul/jul-decision-minicpm5-2b-mlx-4bit
jul ask choice "Which team should handle this ticket?" -o billing -o shipping -o access \
    --state "I was charged twice for order 4411" --model jul-decision-minicpm5-2b

from jul import TypeSafeClient, Choice, Noul, Score

client = TypeSafeClient(model="jul-decision-minicpm5-2b")
answer = client.system_one(
    state={"subject": "Charged twice", "body": "I see two charges for order #4411."},
    questions={"team": Choice(instructions="Which team should handle this ticket?",
                              criteria={"billing": "Payments and refunds",
                                        "shipping": "Delivery problems",
                                        "access": "Login and account access"}),
               "angry": Noul(instructions="Is the customer angry?"),
               "priority": Score(instructions="How urgent is this ticket?",
                                 criteria=["low", "normal", "high"])})

jul models add also fits the vector reading that Score (and questions with very many options) go to. jul needs no code specific to this model: the input format, the readout, the pointer head, its temperature and the routing are in decision.json, next to the weights.

Results

jev-bench: 300 hand-written questions (5 domains: sentiment, finance, support, agent routing, moderation; 100 per type), measured with the PyTorch weights, no 8-gram shared with the training data (one six-word command, "Add milk to my shopping list.", appears in both under other questions), one question per call, PyTorch on an NVIDIA A10G.

	all	Choice	Noul	Score	p50
MiniCPM5-2B, untrained (jul vectors)	0.630	0.75	0.66	0.48	142 ms
minicpm5-2b-decision v1.1	0.637	0.82	0.66	0.43	85 ms
jul-decision-minicpm5-2b v2	0.680	0.87	0.76	0.41	85 ms

95% intervals are about ±9 points per type (n = 100).

Training

LoRA r=16 on every attention and MLP projection plus a pointer head (dim 256), on Kev's architecture and training code (Kev, Jared Palmer, Apache-2.0): learning rate 5e-5, batch 8, one epoch, options reshuffled on every pass, "none of the above" pairs. Base: openbmb/MiniCPM5-2B at 12a3808a.

License

Apache-2.0, like the base model and Kev's code.