# wald-serve

The TypeSafe `POST /v1/systemone` server for Wald-4B: the option-letter readout over a vLLM OpenAI-compatible server or
llama.cpp's `llama-server`, the effort gate (none / low / medium / high / high-k), the knockout for more than 26
options, and the temperature table. It needs only public packages: `pydantic`, plus `vllm==0.30.0` or llama.cpp on the
machine that runs the weights.

```sh
pip install ".[vllm]"                              # on a Linux CUDA machine
wald-serve --model /path/to/Wald-4B --port 8000    # starts vLLM on the weights and serves /v1/systemone
```

GGUF weights (CPU, Apple Silicon or any GPU llama.cpp supports):

```sh
pip install .
wald-serve --gguf ./Wald-4B-Q8_0.gguf --port 8000                     # starts llama-server (on PATH) on the file
wald-serve --llamacpp http://127.0.0.1:8080 --temperature temperature.json \
  --prompt-format repeat_state_plain --effort none --port 8000        # or attach to a running llama-server
```

With `--gguf`, `serving.json` and `temperature.json` are read from the GGUF's folder. The llama.cpp backend sends token
ids to `/completion` and reads the letters from its top 100 next-token logprobs (`n_probs`); a letter outside them gets
the same floor as in vLLM. llama.cpp's context is `--max-model-len` × `--llama-parallel` (default 4 slots).

Tests (no model, no GPU; a fake server stands in for vLLM and llama-server): `pip install ".[test]" && cd tests && pytest -q`.
`tests/test_parity.py` compares prompts and answers byte for byte with the reference implementation the published
numbers were measured with; it is skipped unless that implementation is importable (WALD_PARITY_SRC set).

## Video input (0.2.2, `wald-serve-native-vision`)

Install with the decoder: `pip install ".[vllm,video]"` (OpenCV; vLLM already depends on it). A request may carry **one
video**, either as a top-level `videos` list (the Decision 3.0 / d3 request shape) or as an OpenAI-style message part:

```sh
curl -s localhost:8000/v1/systemone -H 'Content-Type: application/json' -d '{
  "state": "Aisle camera, last 15 seconds.",
  "videos": ["data:video/mp4;base64,AAAAIGZ0eXBpc29t..."],
  "questions": {"conceal": {"type": "noul", "instructions": "Does the shopper put an item into a bag or pocket?"}}}'

# the same video inside the state, at its position among the text parts
{"state": {"messages": [{"role": "user", "content": [
   {"type": "text", "text": "Aisle camera."},
   {"type": "video_url", "video_url": {"url": "data:video/mp4;base64,..."}}]}]},
 "video_route": "frames16", "questions": {...}}
```

- **Sources:** a `data:video/{mp4,quicktime,webm,x-matroska,x-msvideo,mpeg,x-m4v};base64,` URI, raw base64, an http(s)
  URL (`--video-urls public`, the default, fetches only from public addresses; `off` refuses URLs; `any` allows every
  host), or a local path / `file://` URL inside `--video-dir DIR` (refused without that flag).
- **`video_route`** (per request; server default `--video-route`, default `native32`):
  - `native32`: 32 frames spread evenly over the whole clip, given to vLLM as one Qwen3.5 video with its timestamps
    (`<t seconds>` per frame pair), 131,072 px per frame.
  - `frames16`: 16 frames spread evenly, each ≤ 131,072 px, sent as 16 images, each after a text label
    `Frame k/16 at t = S s:`. This is exactly the client protocol of the published VisualDecisionBench-video read.
- **Decode once:** the server decodes the video once per request and every question reuses the frames; with `native32`
  vLLM also processes the video once (later questions reference it by a content uuid). An LRU of decoded videos
  (`--video-cache-mb`, default 512, keyed by the sha256 of the bytes) serves repeat requests on the same video.
- **Images and video together:** `frames16` only. A request with images and no `video_route` uses `frames16` and says
  so (`"video_route_fallback": "images in the request"`); `"video_route": "native32"` with images is refused (400),
  because vLLM's per-request pixel budget applies to images as well.
- **Limits (400 with a reason prefix):** one video per request (`video_count`); ≤ 32 MB (`--max-video-mb`,
  `video_too_large`); ≤ 300 s (`--max-video-seconds`, `video_too_long`); ≤ 3840 × 2160 source frames (`video_pixels`);
  MP4 / MOV, WebM / MKV, AVI or MPEG containers that OpenCV's FFmpeg backend opens (`video_format`, `video_malformed`);
  request bodies ≤ 64 MB. Longer clips are thinned, not refused: 32 frames over 300 s is one frame every ~9.7 s. For
  event timing in long streams, send the window you care about.
- **Response:** `videos` (count), `video_route`, `video_frames` (frames used per video) and `images` (the caller's own
  images; frames16 frames are not counted). `usage.input_tokens` counts text tokens only, as for images.
- **vLLM:** `--model` starts vLLM with `--limit-mm-per-prompt {"image": 32, "video": 2}` (16 frames next to up to 16
  images; a repeated state holds its video twice). `--no-video` keeps serving.json's limits and refuses videos. With
  `--vllm URL`, start that vLLM with the same limit.
- **Calibration:** unchanged. Video questions use the frozen image-route reader, prompt and temperature table.
  `--vllm` mode now takes the table's hash pin from the table's own directory (sibling `serving.json`
  `temperature_sha256`, else its `SHA256SUMS` line, else the frozen v2.1 hash), so an attached front-end can serve a
  checkpoint with its own table. The hash check still runs.