s6-nano
Passed its pre-registered gate. The gate did not measure whether it reads the questions (study of 15/11/2026, docs/s6-nano2-study.md). The gate asked for more than the majority class of
typed_decisions_bench(0.398), and a constant answer per task passes it: answering each question with its most common training answer scores 0.4815, and s6-nano scores 0.4805. On 17 of the bench's 20 tasks it gives one answer to at least 95% of the items. The bench's tasks were 0.4% of what it drew in decision training. A linear model of at most 715,136 bytes (the size of this file) reaches 0.703 on the validation split of the same 20 tasks, and 0.947 / 0.944 on MASSIVE English / French validation once trained on MASSIVE (a validation rehearsal, not a test). Its French training data are two yes/no tasks. On When2Call,tool_callis 13.9% of the training split and 35% of the test, and the model predicts it 39 times in 2,000. NOTAM has no training data. The verdict stands (it passed the gate as written); a narrower second version was studied (the study put its chance of clearing all its gates, against size-matched baselines, at about 10 to 20%), then pre-registered and trained once as s6-route-nano, a tool router: it failed its gate, 0.8395 on unseen tools against 0.8500 for BM25 and 38.9 ms per request in WASM against 2 ms (result).
One 715 KB ternary model for four games and for typed questions in free text. 1,486,225
parameters. It reads a state as text and answers typed questions (choice, score, noul) with
calibrated probabilities, one forward pass per question. It plays the four skycmd games at the
oracle's level, and of the skycmd models trained from scratch and measured on typed_decisions_bench, it is the
first above the majority class. It passed a publication gate fixed before training.
- Demo: huggingface.co/spaces/maxgfr/s6-nano (the games, and typed questions you can edit)
- Code: github.com/maxgfr/skycmd (100 % Rust)
- All results, failures included: results page
- Technical report: maxgfr.github.io/skycmd/paper
- Pre-registration: docs/s6-gen-prereg.md
- Data: trained on skycmd-decisions, skycmd-glm-labels, skycmd-games, skycmd-numeric, skycmd-text; tested on skycmd-decisions, skycmd-zero-shot
- Every demo, with its verdict: the skycmd Space
Results
Test sets, at most 2,000 questions per source, full depth:
| set | s6-nano | reference |
|---|---|---|
typed_decisions_bench |
0.480 | majority class 0.398; the most common training answer of each question 0.4815; on the same records s3-pico 0.382, s4-micro 0.352, s5-lite 0.696 |
| MASSIVE intents, English | 0.425 | s3-pico 0.187 (see the limitations) |
| MASSIVE intents, French | 0.239 | s3-pico 0.117 |
| When2Call (test split of a trained source) | 0.404 | s3-pico 0.368 |
| NOTAM subject / NOTAM multi (never trained on) | 0.055 / 0.493 | s3-pico 0.163 / 0.319 |
| ECE over all these sources | 0.0479 | the gate allows 0.08 |
Closed loop (sky-arena, 30 episodes per game, the same seeds as s3-pico, full depth):
| game | result |
|---|---|
| Flappy Drone | 200 / 200 pipes, 0 crashes |
| Rescue Run | 100 % |
| Obstacle Dodge | 87 % (= oracle), 23 reflex vetoes |
| Doom-like | 100 % of levels, kill rate 0.772 (= oracle) |
Latency: median 1,488 Β΅s per decision natively over the test sources; in WASM under Node, about 15 ms on the long JSON states of the bench and 0.44 ms per Flappy frame (two questions and the physics step). WASM and native agree to 9.5e-7 on 200 bench questions, with the same answer on all 200.
Full depth
The file answers at full depth: all 4 block passes, every time. Its three exit thresholds are set
to 1 in the GGUF (skycmd-s2.exit_threshold.{choice,score,noul}). The model has an exit after
each pass, and conformal risk control (Ξ± = 0.02) chose earlier thresholds, but at those
thresholds Rescue Run fell to 93 % and the Dodge vetoes rose from 23 to 1,016. The
pre-registered rule allows 2 %, so they were rejected. The calibration had left the thresholds at
0.5, a setting that was never evaluated, which is why they are set to 1 here. You can still pass
another threshold at run time (setExitThreshold in the browser).
Architecture
The s3 graph at language scale (skycmd-s2 GGUF tag), as in s4: question and options form a
static stream encoded once and cached; the state, one token per key value pair for numbers,
goes through 2 weight-shared blocks looped twice, with an exit after each of the 4 passes;
options never attend to each other; ternary weights trained with QAT. d 144, 2 heads,
vocabulary 2,048, windows of 512 state tokens and 160 question tokens.
The s6 code adds Parallax attention, Muon v2 and other options. The pre-registered ablation adopted none of them, so this model uses softmax attention and AdamW.
Limitations
- A modest generalist: 0.480 on the bench clears the majority class, but 17M models reach about 0.69 (s5-lite 0.696, oscar-1-17m 0.6775). The bench score is that of a constant answer per task (see the notice above).
- It cannot read NOTAMs (0.055 on the subject question) and stays at 0.404 on When2Call.
- The English MASSIVE score is not strictly zero-shot: 206 of the 2,000 test utterances also appear in the training split, 201 of them in a source derived from MASSIVE. None of the 1,000 French ones does.
- One run, one seed. The first version of its larger sibling (5.6M) reached 0.547 on the bench but failed the ECE guard (0.0821): it failed its gate, and its file is published as a result (owner's decision of 15/11/2026) as s6-micro-v1. A second version, designed after that reading and trained twice as long on decisions, passed the same gate on the same test records (0.553, ECE 0.0588; a second look): s6-micro.
- The games are civil drone scenarios and a Doom-like shooter drawn with Freedoom art; the model only advises and coded rules always keep a veto. This is a demo and a tutorial, not flight software.
Usage
Runs in the skycmd engine only (browser, Node, Python, CLI, server):
cargo install --git https://github.com/maxgfr/skycmd sky-serve
skycmd decide --model s6-nano.gguf --request req.json --pretty
Request format and bindings: install guide.
Training
From scratch in Rust (mlx-rs) on an Apple M5, in a braked mode, so the budgets are working minutes: masked-language pretraining in float (90 minutes, 53,360 steps, 0.87B tokens, 588 per parameter), then decision training with ternary QAT after a 10 % float warm-up (90 minutes, 19,893 steps) and three DAgger rounds on the games (15 minutes each). Data: the typed-decision sets of Hugging Face, classic datasets converted into typed questions, game states labelled by exact oracles and a numeric curriculum. Pretraining text: SimpleStories, TinyStories, FineWeb and the non-web parts of Dolma 3 and TxT360. Soft labels on 747,077 training questions come from GLM-5.3-flash (open weights, MIT, called through the z.ai API), kept only where they agree with the gold answer; no test question was sent. Licenses: DATA_LICENSES.md.
Files
s6-nano.gguf: ternary weights, tokenizer, temperatures, and exit thresholds set to 1 (full depth). 715,136 bytes, sha256b31514b536ae50ebd475f7bf99811148cf3b4d6f9a9b241fc588614b30a9fdf5.
- Downloads last month
- 57
We're not able to determine the quantization variants.