docs/ds4-api-surface-matrix.md

DS4 API surface matrix

This guide describes the Rust host's HTTP behavior. Configuration and resolved plans belong to the serving contract; qualification and workload limits belong to the release ledgers. The original route oracle came from the v0.5.6/v0.6.0 API-promotion arc.

Wire contracts are shared across the supported families. Native state and serving-lane support remain family-specific. Tokenizer, prompt/tool syntax, and stop-token handling are dispatched by the loaded model family without changing the endpoint schemas.

DS4 serves four wire surfaces. "Surface" means a distinct wire contract (object types, stream events, ID scheme, finish mapping, error envelope), not an API vendor. OpenAI Chat and legacy Completions are different surfaces: they share an endpoint family but stream different objects.

Surface	Endpoint	Buffered object	Stream objects
OpenAI Chat	POST /v1/chat/completions	chat.completion	chat.completion.chunk deltas, [DONE]
OpenAI Completion	POST /v1/completions	text_completion	text_completion chunks with choices[].text, [DONE]
Anthropic Messages	POST /v1/messages	type:"message"	message_start .. content_block_* .. message_delta, message_stop
OpenAI Responses	POST /v1/responses	object:"response"	response.created .. item/delta events with sequence_number, response.completed

Compatibility boundary

The project attempts to preserve these public contracts through v0.x:

• core ds4-* CLI flags and DS4_* environment variables;
• HTTP request/response behavior and observable sampling semantics;
• KVC files, EXT_TOOL_MAP, KTM trailers, recurrent payloads, and image replay keys across C-save/Rust-load and Rust-save/C-load;
• explicit distributed byte codecs.

The following are internal and may change during v0.x: crate APIs, Rust types, file layout, scheduler internals, and the native bridge ABI. Native and Rust objects must be built from the same commit. Distributed transport has no wire version, encryption, or authentication; use matching binaries on a trusted network.

Serving lanes and current routing

Three lanes serve generation requests:

• serial (generate_job): one request at a time on the session graph. Full feature set, including live tool continuation and corrective tool recovery.
• continuous (generate_continuous_jobs): the batched engine with per-row sampling, streaming, stops, and tools. The Rust scheduler operates over the configured/native-fitted N-bank width, including width one, and refills idle rows from the live queue during an epoch.
• static (generate_batch_jobs): coalesced buffered greedy batches; the computed routing-needs mask must be zero.

The shared route_decide function selects a lane from each request's computed needs. The routing table is:

Surface	serial	continuous	static
OpenAI Chat	fallback	yes (prompt fits one bank)	buffered + greedy + non-thinking + no stops/tools
OpenAI Completion	fallback	yes (no tools/echo)	same conditions as Chat
Anthropic Messages	buffered tools, prefill-only zero, serial-owned live frontiers, fallback	streaming tools and bank-owned output-only continuations	needs-free buffered
OpenAI Responses	buffered tools, live reasoning state, serial-owned live frontiers, fallback	same as Anthropic	needs-free buffered

Buffered tool generation stays serial because its model-visible corrective retry has no row-local batched equivalent. Streaming tool turns publish bank-owned continuation records. Output-only follow-ups reclaim the bank only when its generation and frontier still match; eviction respects the record's grace and pin windows.

DS4_SERVER_CONT_ANTHROPIC and DS4_SERVER_CONT_RESPONSES control stateless continuous routing. DS4_SERVER_CONT_TOOLS_ANTHROPIC and DS4_SERVER_CONT_TOOLS_RESPONSES control tool routing, and take effect only while the corresponding stateless switch is on.

Within OpenAI, still serial: non-streaming return_token_ids chat, completion-kind requests with return_token_ids, and completion-kind requests carrying tools.

GLM 5.3 text and image requests use the serial lane. Inline PNG/JPEG inputs have message shapes in Chat Completions, Responses and Anthropic Messages; legacy Completions has no image form. GLM live evidence covers text/image Chat with the exact Q2 main GGUF and vision sidecar. Responses and Anthropic image shapes have model-free parser coverage, not equivalent GLM live gates. See the artifact and API scope.

Inkling Small uses serial CUDA serving with text output. Chat, Responses and Messages accept PNG/JPEG images; only Chat accepts 16 kHz PCM/float WAV audio. Legacy Completions accepts text only. The optional eight-layer MTP sidecar preserves the wire contracts and uses speculative decoding for unforced greedy tokens. See MQ85GB validation and limits.

Inline images

Qwen and GLM image content is accepted in the existing API-native shapes:

Chat:      {"type":"image_url","image_url":{"url":"data:image/jpeg;base64,..."}}
Responses: {"type":"input_image","image_url":"data:image/jpeg;base64,..."}
Anthropic: {"type":"image","source":{"type":"base64","media_type":"image/jpeg","data":"..."}}

Output-budget (max_tokens) semantics today

All four parsers reject negative decode budgets with an endpoint-native error naming the supplied field. Numeric values above i32::MAX are capped; fractional values are truncated, so this is not strict integer validation. The shared request_decode_budget helper distinguishes:

• omitted — the server default (--tokens, default 393216);
• explicit zero — zero requested decode tokens;
• positive — the requested budget.

The serial lane supports zero-decode prefill. Anthropic explicitly adds the prefill-only routing need for a zero budget, preserving its cache-prewarm contract (stop_reason: "max_tokens", empty content). Other surfaces can route zero-budget requests to a batched lane, whose native engine floors max_new at one to retire the admission. Those requests may decode one token. Do not rely on route-independent zero-decode behavior outside Anthropic.

The Anthropic parser requires messages but does not require max_tokens (upstream requires it); an omitted value gets the server default like every other surface.

Trust domain

The server is one trust domain. The continuation registry and exact-DSML tool memory are global, with no authentication or tenant namespace. Registry records are keyed by (protocol, call_id) for each Anthropic/Responses tool-call turn.

A tool-call ID grants access to its conversation: an output-only continuation resumes the owning serial session or batch bank. Queued continuations can evict other clients' serial work or prevent bank eviction until the bounded grace/pin deadline. IDs are unguessable but appear in responses. Serve mutually untrusted clients through an authenticating proxy or separate servers; see the compatibility boundary.

Explicitly unsupported

• Schema-constrained output (v0.6.3): response_format with type json_object/json_schema (OpenAI Chat and Completion), text.format (Responses), and output_format / output_config.format (Anthropic) are refused at parse time with HTTP 400 and a typed message naming the mode, in the endpoint's native envelope (ds4-on-spark#10). {"type":"text"}, null, omitted, and a typeless object are accepted unchanged; the string spelling of a schema mode is refused too. Decode is never schema-constrained.
• Responses durable references: non-null previous_response_id or conversation values are rejected at parse time with "not supported; replay full input instead". DS4 serves a stateless Responses subset; clients replay full history. Literal null is accepted and ignored.
• Idempotency-Key: the HTTP reader parses only Content-Length, Transfer-Encoding (chunked request bodies, v0.6.3; DS4_SERVER_CHUNKED=0 restores the Content-Length-only reader), and Accept; the header is accepted and discarded, so a retry is a new generation with new IDs.
• /v1/batch is not implemented by the Rust host (404); the retained C bulk path does not make it a production Rust endpoint.
• return_token_ids is a DS4 extension on OpenAI Chat and Completion, with the serial-routing restrictions described above; it is not a shared four-surface feature.

Historical defects and retained fixtures

1. FIXED (Inc 0b): continuous legacy-Completion streaming emitted chat deltas. cont_on_token projected every streaming row through the chat delta machine, so a text_completion client received chat.completion.chunk objects. Completion rows now stream the serial oracle's plain text_completion chunks (cont_stream_emit_plain); the Inc 0a negative fixture is inverted (test_cont_completion_stream_matches_serial_oracle) and speed-bench/completion_stream_gate.sh holds the live schema + cont-engagement line.
2. FIXED (Rust host RC.3): engine failures after SSE starts now terminate with the surface-native error event. Responses preserve their live sequence_number, Anthropic emits event: error with its error envelope, and OpenAI streams emit an error object instead of a misleading length finish.
3. FIXED (Inc 2b): error envelopes are endpoint-native. Anthropic buffered errors carry the documented {"type":"error","error":{...}} envelope with a status-mapped type (400/409 invalid_request_error, 404 not_found_error, 429 rate_limit_error, 500 api_error, 503 overloaded_error — retryable in the native SDKs); Responses stream errors are protocol data: events ({"type":"error",...}) whose spliced sequence_number continues a live machine's counter. OpenAI chat/completions and Responses buffered errors keep the OpenAI envelope — it is the native family shape there. The flip happened inside wire_http_error/wire_stream_error (Inc 1c made the surface explicit at every call site, including the two 409 continuation-state refusals). Negative decode budgets now reject at parse with the client's own field name (max_tokens / max_completion_tokens / max_output_tokens); explicit ZERO stays supported on every surface with the lane-dependent behavior described under output budgets above.
4. FIXED (Inc 0b): admission accounting dropped decode-growth commitments once a bank's prefill landed (the old outstanding charge covered pending prompt targets only). Every continuous admission now holds a lifetime credit — its full normalized target min(prompt + decode budget, seq_cap) — from install until the row ends, and both admission verdicts (comp-cache budget and live memory floor) charge the page UNION of all live credits plus the candidate. The union matters: per-layer bank strides are narrower than VMM pages are wide, so neighbor banks share edge pages and the true union of k full banks is far below k x virtual/bank — summing per-bank rounded projections would silently shrink live width. The verdict total (resident + projected credits) is timing-independent: the promise holds no matter how much of a row's growth has faulted in. Gate: speed-bench/admission_credit_gate.sh (achieved width at the default budget, pinned-budget hard-promise reject with serial fallback, credit release on row death and on mid-prefill abort).
5. FIXED (Inc 0b): the v0.5.5 budget-cut honesty fix (#13) was serial-only. On the continuous lane — where chat+tools actually routes — an unrepairable max_tokens cut inside a tool call reported finish="error" (and a repairable one still silently completes to tool_calls, matching serial's repair tier). The cont lane now mirrors serial: the finish stays the honest length, the partial call returns as assistant content, and the same "tool call cut by token budget" marker is logged — so finish_reason_gate.sh's engagement oracle finally fires on the lane it actually exercises. Found by the Inc 0a baseline battery (deterministic fail, reproduced byte-identical on the tip-parent binary — the gate had never actually proven the cont lane). The full failure-cause taxonomy (stranding, aborts) remains Inc 2 typed-outcome work.

Recorded quirks (current behavior, not upstream-shaped)

• Anthropic response IDs: the host mints a shared chatcmpl-N / cmpl-N job ID for every surface, and anthropic_final_response / the Anthropic stream use it directly — so Anthropic clients see "id":"chatcmpl-N" instead of an upstream-shaped msg_* ID. Responses is unaffected (responses_final_response and responses_stream_init mint their own resp_*/rs_*/msg_* IDs). This is the current compatibility behavior.

Route observation metrics

GET /metrics exposes ds4_route_requests_total{surface=...,lane=...} (fixed cardinality: 4 surfaces x 3 lanes, all cells always emitted); GET /v1/stats mirrors it as the routes section. One increment per job at the moment a lane takes it — a failed batched attempt that falls back to serial increments both lanes. These counters are observation only; they exist so route promotion can prove engagement (an eligible request actually moved lanes) instead of inferring it.

Retained C oracle fixtures (ds4_test --server)

These fixtures replay deterministic token/text tapes through the C projectors, validated by protocol event validators (event order, one open/close per block/item, contiguous Responses sequence_number, UTF-8 hold-back):

• test_tape_openai_chat_stream_projection (thinking + UTF-8-split tapes)
• test_tape_openai_completion_stream_projection (the oracle for defect 1)
• test_tape_anthropic_stream_projection
• test_tape_responses_stream_projection
• test_tape_buffered_final_responses (all four buffered objects + finish mapping, including Anthropic length -> max_tokens)
• test_cont_completion_stream_matches_serial_oracle (the inverted Inc 0a negative fixture for defect 1 — cont and serial now share the legacy-Completion stream shape)
• test_route_decisions_record_current_dispatch
• test_idempotency_key_header_is_ignored
• test_responses_durable_references_rejected_at_parse
• test_error_envelopes_native_shapes (defect 3, now asserting the native shapes + the negative-budget parse rejections)

Existing per-feature streaming tests (test_openai_tool_stream_*, test_anthropic_tool_stream_*, test_responses_*) cover tool-call projection per surface and remain part of the oracle. Current Rust checks include make test-server-parity and the ds4-server crate tests; see CONTRIBUTING.md.

Live-sampled output is never a byte oracle: continuous temp-0 emissions jitter run-to-run, so live end-to-end gates assert schema, event automata, route engagement, and semantic equivalence — not byte identity.