Pennywise router (ONNX)
This repo holds the ONNX weights for Pennywise, an RL-trained decision model (Laya, ModernBERT-large) that picks the cheapest model tier that will still answer a request correctly:
- small:
nvidia/nemotron-3.5-bolt-e4b - medium:
openai/gpt-oss-20b - large:
nvidia/nemotron-3-ultra-550b-a55b
The weights are downloaded and sha256-verified automatically by the npm MCP server pennywise-mcp:
npx -y pennywise-mcp
In the Alan desktop app: go to Settings β Plugins β MCP β Add custom connector β Command.
- Windows: Command
cmd, with arguments (one per line)/c,npx,-y,pennywise-mcp. - macOS / Linux: Command
npx, with arguments-y,pennywise-mcp.
Then set the environment variable NVIDIA_API_KEY = ${NVIDIA_API_KEY} and click Enter API key. The key is only needed for the ask_routed tool.
Files
| File | Size | Role |
|---|---|---|
lower.onnx |
1.38 GB | Shared ModernBERT trunk: input_ids, attention_mask (int64 [1,L]) β h (float32 [1,L,1024]) |
upper_lam0.onnx |
0.30 GB | Top layers + decision head trained with Ξ» = 0 (quality-leaning) |
upper_lam0.5.onnx |
0.30 GB | Same, Ξ» = 0.5 (balanced; the default auto mode) |
The upper graphs take h, attention_mask, marker_pos (int64 [1,3]) and marker_mask (bool [1,3]) and output probs [1,3] over (small, medium, large). The tokenizer and the fixed routing-question prefix ship inside the npm package.
Results
On a 10-task agent coding benchmark, Pennywise finished 10/10 in 11.0 min. Alan's built-in Auto Router also finished 10/10, but took 36.9 min, so Pennywise was 3.4Γ faster at the same success rate.
More: https://getpennywise.pages.dev
Author: Saksham Sharma (BITS Pilani)