Independent 4-way benchmark incl. this engine (751 identical states)
Benchmarked this approach as a fourth column next to Laya and Jev, same 751 states, same questions, seed 42. Note for readers: this repo ships no fine-tuned weights, so I ran stock mlx-community/Qwen2.5-1.5B-Instruct-4bit under the vendored parallel engine, local MLX on an M2. Score questions asked as enums over rubric levels.
Results: triage 0.825, guardrails 0.900, moderation 0.700, agnews 0.730, emotion 0.540, banking77 (12 intents) 0.500, mnli 0.733, sst5 0.617 (ties Jev 1.13.0), multilingual intent 0.880. Latency 700-1150 ms per call on M2.
Two findings for the author. First-token logit slicing degrades as option count grows: 12-way banking at 0.500 while 4-6 way stays competitive. Emotion is a three-model wall at 0.54-0.55 with ECE near 0.3.
Full harness, raw JSONs, all four runs: https://github.com/instax-dutta/sysone-bench