Scombat PRO
scombat
AI & ML interests
None yet
Recent Activity
posted an update 3 days ago
We published an open, replayable bench for prompt-complexity classifiers: three local LLM judges (Gemma 4 31B, Qwen3.8 27B, gpt-oss-20b), 360 prompts in en/fr/ru/zh, a length baseline, bootstrap intervals, a 2 σ rule before any "better than".
s1d1-router (mmBERT-base, Apache-2.0, 25–310 MB int8) agrees with the judges' median at QWK 0.807; the judges agree with each other at 0.774; prompt length gets 0.543. Caveat first: Gemma 4 labelled the training data and is one of the judges (0.785 without it).
Article: https://huggingface.co/blog/scombat/measuring-a-prompt-complexity-router-against-local · bench: https://github.com/scombat/s1d1-bench · try it in the browser: https://huggingface.co/spaces/scombat/s1d1-router-demo published an article 3 days ago
Measuring a prompt-complexity router against local LLM judges: an open, replayable bench updated a model 16 days ago
scombat/s1d1-router-6src-tiny-onnx-int8Organizations
None yet