server/README.md

wald-serve

The TypeSafe POST /v1/systemone server for Wald-4B: the option-letter readout over a vLLM OpenAI-compatible server or llama.cpp's llama-server, the effort gate (none / low / medium / high / high-k), the knockout for more than 26 options, and the temperature table. It needs only public packages: pydantic, plus vllm==0.30.0 or llama.cpp on the machine that runs the weights.

pip install ".[vllm]"                              # on a Linux CUDA machine
wald-serve --model /path/to/Wald-4B --port 8000    # starts vLLM on the weights and serves /v1/systemone

GGUF weights (CPU, Apple Silicon or any GPU llama.cpp supports):

pip install .
wald-serve --gguf ./Wald-4B-Q8_0.gguf --port 8000                     # starts llama-server (on PATH) on the file
wald-serve --llamacpp http://127.0.0.1:8080 --temperature temperature.json \
  --prompt-format repeat_state_plain --effort none --port 8000        # or attach to a running llama-server

With --gguf, serving.json and temperature.json are read from the GGUF's folder. The llama.cpp backend sends token ids to /completion and reads the letters from its top 100 next-token logprobs (n_probs); a letter outside them gets the same floor as in vLLM. llama.cpp's context is --max-model-len × --llama-parallel (default 4 slots).

Tests (no model, no GPU; a fake server stands in for vLLM and llama-server): pip install ".[test]" && cd tests && pytest -q. tests/test_parity.py compares prompts and answers byte for byte with the reference implementation the published numbers were measured with; it is skipped unless that implementation is importable (WALD_PARITY_SRC set).

Video input (0.2.2, wald-serve-native-vision)

Install with the decoder: pip install ".[vllm,video]" (OpenCV; vLLM already depends on it). A request may carry one video, either as a top-level videos list (the Decision 3.0 / d3 request shape) or as an OpenAI-style message part:

curl -s localhost:8000/v1/systemone -H 'Content-Type: application/json' -d '{
  "state": "Aisle camera, last 15 seconds.",
  "videos": ["data:video/mp4;base64,AAAAIGZ0eXBpc29t..."],
  "questions": {"conceal": {"type": "noul", "instructions": "Does the shopper put an item into a bag or pocket?"}}}'

# the same video inside the state, at its position among the text parts
{"state": {"messages": [{"role": "user", "content": [
   {"type": "text", "text": "Aisle camera."},
   {"type": "video_url", "video_url": {"url": "data:video/mp4;base64,..."}}]}]},
 "video_route": "frames16", "questions": {...}}

• Sources: a data:video/{mp4,quicktime,webm,x-matroska,x-msvideo,mpeg,x-m4v};base64, URI, raw base64, an http(s) URL (--video-urls public, the default, fetches only from public addresses; off refuses URLs; any allows every host), or a local path / file:// URL inside --video-dir DIR (refused without that flag).
• video_route (per request; server default --video-route, default native32):
  • native32: 32 frames spread evenly over the whole clip, given to vLLM as one Qwen3.5 video with its timestamps (<t seconds> per frame pair), 131,072 px per frame.
  • frames16: 16 frames spread evenly, each ≤ 131,072 px, sent as 16 images, each after a text label Frame k/16 at t = S s:. This is exactly the client protocol of the published VisualDecisionBench-video read.
• Decode once: the server decodes the video once per request and every question reuses the frames; with native32 vLLM also processes the video once (later questions reference it by a content uuid). An LRU of decoded videos (--video-cache-mb, default 512, keyed by the sha256 of the bytes) serves repeat requests on the same video.
• Images and video together: frames16 only. A request with images and no video_route uses frames16 and says so ("video_route_fallback": "images in the request"); "video_route": "native32" with images is refused (400), because vLLM's per-request pixel budget applies to images as well.
• Limits (400 with a reason prefix): one video per request (video_count); ≤ 32 MB (--max-video-mb, video_too_large); ≤ 300 s (--max-video-seconds, video_too_long); ≤ 3840 × 2160 source frames (video_pixels); MP4 / MOV, WebM / MKV, AVI or MPEG containers that OpenCV's FFmpeg backend opens (video_format, video_malformed); request bodies ≤ 64 MB. Longer clips are thinned, not refused: 32 frames over 300 s is one frame every ~9.7 s. For event timing in long streams, send the window you care about.
• Response: videos (count), video_route, video_frames (frames used per video) and images (the caller's own images; frames16 frames are not counted). usage.input_tokens counts text tokens only, as for images.
• vLLM: --model starts vLLM with --limit-mm-per-prompt {"image": 32, "video": 2} (16 frames next to up to 16 images; a repeated state holds its video twice). --no-video keeps serving.json's limits and refuses videos. With --vllm URL, start that vLLM with the same limit.
• Calibration: unchanged. Video questions use the frozen image-route reader, prompt and temperature table. --vllm mode now takes the table's hash pin from the table's own directory (sibling serving.json temperature_sha256, else its SHA256SUMS line, else the frozen v2.1 hash), so an attached front-end can serve a checkpoint with its own table. The hash check still runs.