Results on a Brazilian Portuguese typed-decision benchmark
Hi! We ran hopper-g on a PT-BR typed-decisions benchmark: 7 tasks over native Brazilian Portuguese text, plus a held-out Central Bank FAQ task. We served it with hopper-serve, unchanged except for --allow-slow-kernels: causal-conv1d did not build on our RTX 5090, so the timings do not apply, but the answers are the same. It scored 63.0% mean balanced accuracy, fourth among the 13 untuned open decision models up to 4B we tested, and 96.2% on the FAQ task. A 4B fine-tuned on Portuguese reached 72.9% on the benchmark.
Full table and per-task results: https://huggingface.co/datasets/felhen-ai/ptbr-typed-decisions-bench
Evaluator: https://github.com/felhen-ai/saracura/blob/main/research/eval_http.py
If anything looks off for your model, like settings or server flags, tell us and we will rerun it. Thanks for releasing the weights.