chaoliangUNSW/MacJev-0.8B-Decision-Data
Preview • Updated • 1
chaoliang yan PRO
chaoliangUNSW
AI & ML interests
None yet
Recent Activity
repliedto their post about 8 hours ago
Jev-style decisions on your laptop: Jev-Style-0.8B-Decision-v3
Give it any text and a question. It returns a calibrated probability for every option in one forward pass: pick one, yes/no, or a score. No text generation, nothing to parse.
On data it never trained on:
• Banking77, all 77 intents in one pass (400 test rows): 68.2% vs 49.2% for the best official Laya checkpoint (+19.0, paired 95% CI +14.0 to +24.0)
• tweet_topic, zero-shot: 75.5% vs 63.2% for English Laya, within 4 points of Jev (79.3%), and better calibrated than Jev (ECE 0.027 vs 0.063)
• JevBench v1.4.1, 231 public items, zero-shot: 64.1%, ahead of every Qwen3.5-0.8B-based system on the board; ahead of Laya (58.4%) too, though that lead is inside the CI
Jev is still ahead on all three. v3 is the open, local option that gets closest: 0.53 GB in 4-bit, runs on CPU.
Typed decisions (in-domain): 79.2% vs 76.6% for Laya typed, both trained on the same train split (paired CI +1.1 to +4.2). Read this as agreement with the dataset's teacher labels, not decision skill:
• teacher-noise reference: one draw from the teacher's own distribution matches the gold label 65.9% of the time (test split); the dataset card puts teacher self-agreement at 73.5%
• on the teacher's near-ties, v3 and Laya typed score the same (51.7% vs 51.7%); v3's lead comes from cases where the teacher is clear
• Jev's 72.7% there is zero-shot, so not a like-for-like comparison
Also: up to 25,600 tokens per call (98.3% on real 24K-token documents); about 44 ms per short decision on an M1 Max; one read of a document answers every question in one call; ahead of Laya multilingual in 51 of 51 languages.
Try it in your browser, no install:
https://huggingface.co/spaces/chaoliangUNSW/jev-style-v3
Model (transformers, with GGUF and MLX builds linked):
https://huggingface.co/chaoliangUNSW/Jev-Style-0.8B-Decision-v3
Apache-2.0. Not affiliated with TypeSafe, Jev or Laya. Thanks to @dipankarsarkar for pushing on the typed-decisions framing. new activity about 8 hours ago
chaoliangUNSW/Jev-Style-Qwen3.5-2B-Decision-v2-GGUF:Bad Answers with Ollama liked a model about 10 hours ago
chaoliangUNSW/Jev-Style-0.8B-Decision-v3Organizations
None yet