card: how to run (vLLM, OpenAI client, function calling, transformers)

README.md

@@ -85,33 +85,104 @@ teacher-written answer to them. The training set is not released.
 
 ## How to use
 
-Serve with vLLM 0.24 or newer. The sampler settings come from the checkpoint's own
-`generation_config.json` (entropy-bound sampler, entropy bound 0.1, 48 denoising steps, 256-token
-canvas); every number above was measured with them. To change the step cap use
-`--diffusion-config '{"canvas_length":256,"max_denoising_steps":N}'`; note that `--hf-overrides`
-keys such as `diffusion_max_denoising_steps` are not read by vLLM.
+The model reads the component library from the system prompt and answers with an openui-lang
+program. Build the system prompt for your own component library with `@openuidev/cli`
+(`npx @openuidev/cli generate <library.ts> --out system-prompt.txt`), or use the benchmark's
+reference prompt (`protocols/openui/prompt.ts` in generative-ui-bench) to try it as is. Render
+the output with `@openuidev/react-lang` (or the Vue and Svelte renderers) and validate it with
+`@openuidev/lang-core`.
+
+### vLLM (recommended)
+
+vLLM 0.24 or newer. The sampler settings come from the checkpoint's own `generation_config.json`
+(entropy-bound sampler, entropy bound 0.1, 48 denoising steps, 256-token canvas); every number
+above was measured with them. About 26 GB of VRAM at FP8.
 
 ```bash
+pip install "vllm>=0.24"
 vllm serve thesysdev/OUI-1 --trust-remote-code --max-model-len 16384 --quantization fp8 \
   --served-model-name oui-1 --max-num-seqs 4 \
   --enable-auto-tool-choice --tool-call-parser gemma4
 ```
 
-Function calling: the model keeps Gemma 4's native tool-call format, so with the `gemma4` parser
-it returns standard OpenAI `tool_calls` for tools you pass in the request, then writes the screen
-from the tool result on the next turn. It needs one instruction in the system prompt to prefer the
-tool over inventing data, e.g.: "When the user asks about the weather or a stock price you MUST
-call the matching tool first and output nothing else in that turn; write the screen from the
-returned values on the next turn." In our test (weather, stock price tools) it called the right
-tool with the right arguments on 9 of 9 asks, never called one on 3 unrelated asks, and used the
-returned numbers verbatim. `tool_choice: "required"` is not available (vLLM has no structured
-outputs for diffusion models yet).
-
-Then call it as an OpenAI-compatible chat model. The system prompt must carry the component
-library: use `@openuidev/cli`'s prompt generator for your library, or the benchmark's
-`protocols/openui/prompt.ts` for the reference library. Recommended: `max_tokens` 4096 (the
-`temperature` and `seed` fields are ignored by DiffusionGemma), briefs in the form the model was
-trained on:
+Then call it as an OpenAI-compatible chat model. `temperature` and `seed` are ignored (diffusion
+models use a fixed internal schedule); keep `max_tokens` at 4096 or below.
+
+```python
+from openai import OpenAI
+
+client = OpenAI(base_url="http://localhost:8000/v1", api_key="local")
+system = open("system-prompt.txt").read()  # your component library, from @openuidev/cli
+brief = """A fitness app's daily summary. Build a single screen for this. It must show:
+1. step count today against the daily goal
+2. calories burned
+3. a way to log a new workout
+Cover every numbered item."""
+r = client.chat.completions.create(
+    model="oui-1",
+    messages=[{"role": "system", "content": system}, {"role": "user", "content": brief}],
+    max_tokens=4096,
+    stream=False,
+)
+print(r.choices[0].message.content)  # openui-lang: one component per line, wired into root
+```
+
+Streaming works the same way (`stream=True`); tokens arrive one 256-token canvas at a time.
+
+To change the step cap use `--diffusion-config '{"canvas_length":256,"max_denoising_steps":N}'`
+(32 and 48 score the same on the benchmark, 16 breaks the reference graph). `--hf-overrides` keys
+such as `diffusion_max_denoising_steps` are not read by vLLM.
+
+### Function calling
+
+The model keeps Gemma 4's native tool-call format. With `--tool-call-parser gemma4` it returns
+standard OpenAI `tool_calls` for tools you pass in the request, then writes the screen from the
+tool result on the next turn. It needs one instruction in the system prompt to prefer the tool
+over inventing data, e.g.: "When the user asks about the weather or a stock price you MUST call the
+matching tool first and output nothing else in that turn; write the screen from the returned
+values on the next turn." In our test (weather, stock price tools) it called the right tool with
+the right arguments on 9 of 9 asks, never called one on 3 unrelated asks, and used the returned
+numbers verbatim. `tool_choice: "required"` is not available (vLLM has no structured outputs for
+diffusion models yet).
+
+```python
+tools = [{"type": "function", "function": {"name": "get_weather",
+          "description": "Current weather for a city",
+          "parameters": {"type": "object", "properties": {"city": {"type": "string"}}, "required": ["city"]}}}]
+r = client.chat.completions.create(model="oui-1", tools=tools, max_tokens=4096,
+    messages=[{"role": "system", "content": system + TOOL_RULE}, {"role": "user", "content": "weather in Goa this weekend"}])
+call = r.choices[0].message.tool_calls[0]           # get_weather({"city": "Goa"})
+# run the tool, then append the assistant message and a {"role": "tool", ...} result and call again
+```
+
+### Transformers
+
+Transformers 5.11 or newer with the DiffusionGemma model class; bf16 needs about 52 GB of GPU
+memory (one A100 80GB or H100). `generate` chains 256-token canvases internally and stops at EOS.
+
+```python
+import torch
+from transformers import AutoModelForCausalLM, AutoProcessor
+
+model_id = "thesysdev/OUI-1"
+processor = AutoProcessor.from_pretrained(model_id)
+model = AutoModelForCausalLM.from_pretrained(model_id, torch_dtype=torch.bfloat16, device_map="cuda")
+
+messages = [{"role": "system", "content": open("system-prompt.txt").read()},
+            {"role": "user", "content": brief}]
+ids = processor.apply_chat_template(messages, tokenize=True, add_generation_prompt=True, return_tensors="pt").to("cuda")
+gc = model.generation_config          # entropy-bound sampler, 48 steps, from the checkpoint
+gc.max_new_tokens = 256 * 16          # up to 16 canvases; EOS stops earlier
+out = model.generate(input_ids=ids, generation_config=gc)
+print(processor.decode(out.sequences[0, ids.shape[1]:], skip_special_tokens=True))
+```
+
+The LoRA adapter that produced these weights is in `adapter/` (PEFT format, base
+`google/diffusiongemma-26B-A4B-it`), for anyone who prefers to load base plus adapter.
+
+### Briefs
+
+Briefs in the form the model was trained on work best:
 
 ```
 A fitness app's daily summary. Build a single screen for this. It must show: