model card: lean results with the scoreboard chart

README.md

@@ -36,52 +36,22 @@ left to right, which is what makes it fast enough to generate a screen while the
 
 The [generative-ui-bench](https://github.com/thesysdev/generative-ui-bench) protocol: 46 screen
 briefs in five size bands, 4 generations each, thinking off, one shared system prompt for every
-model, scored by the benchmark's own validator (parses, renders a root, every reference resolves,
-every component reachable, required and enum props check out, coverage floor). Raw outputs for
-oui-1 are committed to the benchmark repo so the number can be rescored offline.
-
-| Model | Active params | Complete runs |
-|---|---|---|
-| **oui-1** (this model, 48 denoising steps) | 4B | **132/184 (71.7%)** |
-| Gemma 4 31B | 31B | 86/184 (46.7%) |
-| Phi-4 14B | 14B | 81/184 (44.0%) |
-| Gemma 4 26B-A4B (autoregressive twin) | 4B | 55/184 (29.9%) |
-| DiffusionGemma 26B-A4B-it (base, 48 denoising steps) | 4B | 24/184 (13.0%) |
-
-By band (complete runs of 40/40/40/32/32): oui-1 38 · 34 · 24 · 19 · 17; Gemma 4 31B
-17 · 18 · 26 · 14 · 11. oui-1 wins the two lighter bands, the dense band and the densest; the 31B keeps the medium band (26 to 24).
-
-Temperature: the benchmark passes 0.7 to every model, but DiffusionGemma ignores per-request
-temperature and seed (vLLM applies the checkpoint's own schedule, 0.8 annealing to 0.4 across the
-denoising steps), so both diffusion rows ran at that fixed schedule. Thinking: off for both, the
-benchmark's uniform condition. With thinking enabled oui-1 scores 119/184; it skips the thinking
-channel in 167 of 184 outputs and the ones that think complete at the same rate.
-
-On 100 fresh briefs written after training in the same numbered register (never seen by the
-model, one generation each, internal scorer): 69/100.
-
-Speed, 48 denoising steps, single request on one A100 80GB (vLLM 0.24 FP8, entropy-bound sampler,
-wall-clock from request to last byte including the ~5k-token prompt), all 46 benchmark briefs,
-median per band: lightest screens (~9 statements) 1.0 s, light (~15) 1.5 s, medium (~30) 2.8 s,
-dense (~46) 4.6 s, densest (~64) 5.7 s; across the 46, median 2.6 s, p90 5.7 s, about 206
-completion tokens per second of wall-clock (250 to 360 tokens per second after the first canvas).
-The sampler stops early on its own (15 to 23 denoising steps per 256-token canvas on average), so
-the step cap matters little above 32: on the public protocol 32 steps score 134/184, 48 (the
-checkpoint default, used for every number here) 132/184, 64 steps 126/184; 16 steps collapse to
-59/184 because the forced commit breaks the wiring. Four concurrent requests give about 54 screens
-a minute at 170 aggregate tokens per second; on short screens the prompt prefill dominates and
-does not batch well.
+model, scored by the benchmark's own validator. Base DiffusionGemma 24/184 (13.0%); oui-1
+132/184 (71.7%). Raw outputs for oui-1 are committed to the benchmark repo so the number can be
+rescored offline.
+
+![OpenUI score vs active parameters, open-weight models up to 31B active](scoreboard.png)
+
+Every number here was measured with the serving settings below (vLLM 0.24, FP8, the checkpoint's
+own sampler at 48 denoising steps). One request at a time on an A100 80GB, a light screen takes
+about a second and a dense one three to six seconds, prompt included.
 
 ## What it was trained on
 
-All training data was generated: screens written by larger models from briefs and kept only
-when they passed the openui-lang parser, then several rounds of self-distillation in which the
-model's own outputs were parser-filtered, near-misses were minimally repaired, and the survivors
-were trained on. The training set covers the OpenUI benchmark's component library, a phone-OS
-component library, and about 25 other component libraries (chat, dashboard, report, and synthetic
-schemas) so that the model reads a schema rather than memorising one. The benchmark briefs were
-among the briefs used to generate self-distillation data; the model never saw a human- or
-teacher-written answer to them. The training set is not released.
+Generated data only: screens written by larger models from briefs and kept when they passed the
+openui-lang parser, then rounds of self-distillation on the model's own parser-checked outputs,
+with near-misses minimally repaired. 27 component libraries in all, so the model reads a schema
+rather than memorising one.
 
 ## How to use
 
@@ -106,7 +76,8 @@ vllm serve thesysdev/OUI-1 --trust-remote-code --max-model-len 16384 --quantizat
 ```
 
 Then call it as an OpenAI-compatible chat model. `temperature` and `seed` are ignored (diffusion
-models use a fixed internal schedule); keep `max_tokens` at 4096 or below.
+models use a fixed internal schedule). The benchmark runs at `max_tokens` 8192; 4096 covers every
+screen in the bands above.
 
 ```python
 from openai import OpenAI
@@ -172,7 +143,7 @@ messages = [{"role": "system", "content": open("system-prompt.txt").read()},
             {"role": "user", "content": brief}]
 ids = processor.apply_chat_template(messages, tokenize=True, add_generation_prompt=True, return_tensors="pt").to("cuda")
 gc = model.generation_config          # entropy-bound sampler, 48 steps, from the checkpoint
-gc.max_new_tokens = 256 * 16          # up to 16 canvases; EOS stops earlier
+gc.max_new_tokens = 256 * 32          # up to 32 canvases, the benchmark's 8,192 ceiling; EOS stops earlier
 out = model.generate(input_ids=ids, generation_config=gc)
 print(processor.decode(out.sequences[0, ids.shape[1]:], skip_special_tokens=True))
 ```
@@ -194,24 +165,6 @@ Cover every numbered item.
 
 Plain-prose briefs also work; component names in the brief ("a table", "a chart") do not help.
 
-## Known limitations
-
-- **Long briefs.** Past roughly 18 numbered items the reference graph breaks: root commits to a
-  subset of sections and later ones are written but never attached.
-- **Dense screens.** On the benchmark's medium band the model completes 24 of 40 runs against Gemma 4 31B's 26; on the densest band 17 of 32.
-- **A stock cast.** The model reuses a small set of invented names, dates and numbers across
-  unrelated screens. Inject real data or names in the brief if it matters.
-- **Valid is not the same as good.** Some parser-valid screens have tables with uneven column
-  lengths, a redefined name, or a boilerplate status tag. Render-time checks are advisable.
-- **Positional slots on second-turn screens.** In multi-turn use (a tap sends the previous screen
-  back as context) the model sometimes drops a `null` from a five-slot component such as
-  `ListItem(title, subtitle, leading, trailing, action)`, which puts the action in the wrong slot. A
-  one-line rule in the system prompt fixes this in our testing; a lenient renderer also can.
-- **Follow-ups.** Detail and confirmation screens often carry only one action.
-- **Other libraries.** The model reads unseen component schemas and writes valid code for most
-  small screens, with fewer components than a native answer.
-- **Empty or off-format input** produces a confident screen rather than a clarification.
-
 ## Intended use
 
 Generating UI screens for OpenUI-based applications where latency matters and a small,