A PRO

Done in Run 9. Each model is now scored with and without the wrapper, and raw overrun is reported next to the trimmed score. The wrapper adds 1.8 points for SFT and 0 for GRPO, and it cuts generated tokens by about 8 times. Every results file now carries its serving settings, grouped and hashed, and a diff tool names which group changed. Runs before Run 9 lack these records, so only changes from Run 9 onward can be attributed.

Thank you for reading it this closely. That's a really useful distinction, and I agree with it.

It's the same principle as counting blocked attempts: a reported number should describe exactly what produced it. Our results describe the adapter together with its serving wrapper, not the adapter alone. The wrapper uses greedy decoding, stops on every Tiny Aya end-of-turn token and on the string "Tool result for", then keeps only the first tool call (trim_turn()). As you noted, that handling alone moved invalid calls from 16.7% to 0% with no extra training, so the 0% belongs to the pair.

I've added a Serving contract section to the README listing these settings (prompt template, decoding, token limits, stop tokens and string, trimming), with a note that anyone serving the adapter through another wrapper should apply the same settings or expect different numbers. The article now says this too.

As a next step, I'd like to record these settings in every results file and report the raw, untrimmed overrun rate next to the trimmed score, so "model alone" and "model plus wrapper" can be compared directly.

Thanks again for the thoughtful feedback.