docs/VALIDATION.md

# Validation status and required gates

## Current evidence boundary

| Scope | Status in this source release |
|---|---|
| Adapter identity and immutable training provenance | Included and hash-locked |
| Static package and prohibited-artifact scan | Must pass before archive release |
| External Raven foundation identity | Locked; verified by the receiving operator |
| Adapter-specific ONNX export | Not included; generated and verified on target |
| Adapter-specific TensorRT correctness | Not included; required after target build |
| Adapter-specific TeaCache quality/latency selection | Passed on B200; threshold 0.3 selected |
| RTX 4090 physical validation | Not completed |
| RTX 4090 200 ms / 5 Hz certification | No claim |
| Full DAgger observation-to-action validation | Not completed |

The acceleration method was exercised previously on the parent adapter and
other GPU classes. Those results demonstrate implementation feasibility only.
They are not transferable correctness, quality, memory, or latency evidence
for the update-5000 adapter on an RTX 4090. This release deliberately omits
old engines, old adapter-fused ONNX, and parent-adapter headline numbers.

The completed adapter-specific B200 campaign is retained under
`validation/b200_rgb030_depth030_20260730/` and summarized in
`SUMMARY.json`. It selects threshold `0.3`; it does not certify RTX 4090
latency or memory use.

## Package-release gates

Before creating the ZIP:

1. regenerate `config/SOURCE_MANIFEST.sha256`;
2. run `python3 bin/doctor.py --stage package`;
3. run Python compilation, shell syntax, and CPU contract tests;
4. inspect the archive for `.onnx`, `.onnx.data`, `.engine`, `.plan`, timing
   caches, CUDA Graph artifacts, foundation weights, generated outputs, and
   stale step-228270 model binaries;
5. test the ZIP with `unzip -t`;
6. calculate the final ZIP SHA-256; and
7. re-extract to a clean directory and rerun package-stage doctor.

Passing these checks means the source bundle is intact and thin. It does not
certify inference.

## Target export gates

`bin/export_onnx.sh` must produce:

- an ONNX graph and external-data file;
- a PyTorch correctness capture;
- an export summary; and
- `export_manifest.json`.

The export manifest must match:

- adapter SHA-256
  `c68de259ce5ee8e4df5a45296be8328242b3ebba351990df1d86f4e3a7a2d588`;
- the foundation DiT SHA in `config/foundation_models.lock.json`;
- the packaged exporter SHA;
- the packaged source-manifest SHA;
- the exporter Python/PyTorch/ONNX/ONNXScript/SafeTensors versions; and
- opset 20, batch 1, `ext1/ext2/wrist`, `416x240`, one-frame, lag-256
  reference profile.

Reject an export derived from the parent step-228270 adapter.

## TensorRT build gates

The target build must:

1. record the exact GPU, compute capability, driver, PyTorch/CUDA, TensorRT,
   precision, workspace, adapter, and ONNX lineage;
2. produce finite output from the captured input;
3. measure relative RMS error against PyTorch at or below `0.05`;
4. write an engine SHA-256 and target-keyed build manifest;
5. promote `assets/generated/engines/current` only after passing; and
6. pass `doctor --stage runtime`.

CUDA Graph remains disabled unless a separate target-local campaign proves
correctness, clean teardown, stability, memory behavior, and a positive
end-to-end latency delta.

## Functional smoke gates

Using `assets/fixtures/lag256/request.json`:

- every condition image is RGB `416x240`;
- view order is `ext1`, `ext2`, `wrist`;
- reference and target source indices differ by exactly 256;
- the latent is finite with shape `[1, 16, 1, 90, 52]`;
- decoded RGB contains three complete views at the original per-view
  resolution;
- no condition view is silently duplicated, resized, or omitted; and
- output hashes and the runtime/engine manifests are retained.

The fixture is a regression case, not a representative quality benchmark.

## TeaCache and five-step quality campaign

Use matched prompt, conditions, seed, model, and target for:

| Variant | Purpose |
|---|---|
| 50-step PyTorch | High-step visual reference |
| 5-step PyTorch, TeaCache off | Coarse-schedule reference |
| 5-step TensorRT, TeaCache off | Isolate TensorRT numerical effect |
| 5-step TensorRT, thresholds 0.1/0.2/0.3/0.4 | Measure cache speed/quality frontier |

Report per-view and aggregate PSNR, SSIM, and a perceptual metric, but do not
select a configuration on scalar metrics alone. Review:

- fine material texture and color;
- object identity and sharpness;
- robot/object contacts;
- depth and mask adherence;
- cross-view geometry; and
- prompt/task consistency.

The configured threshold `0.3` passed this campaign on B200. It remains
provisional on each new deployment target until the matched target-local
campaign passes.
Record the actual TeaCache block calls/skips rather than assuming a threshold
always produces a fixed skip pattern.

## Temporal and closed-loop campaign

Evaluate four distinct regimes:

1. frame-0 bootstrap for targets `t=0..255`;
2. teacher-conditioned true lag-256;
3. autoregressive true lag-256 using stored model outputs; and
4. deliberately missing one RGB or depth view, matching the fine-tuning
   dropout semantics.

Do not label an all-three-RGB-none test as in-distribution: this fine-tune
drops exactly one RGB view on a triggered sample, not the complete modality.

For autoregressive validation, generate at least 600 sequential frames
(20 seconds at 30 fps). Examine offsets 256 and 512 for delayed recurrences.
Plot quality/drift versus frame index and compare early, middle, and final
segments. Include both quantitative metrics where targets exist and a
side-by-side video.

## RTX 4090 latency, memory, and endurance

On the physical final target, separate:

- process/model startup;
- prompt-cache miss;
- warm exact-prompt latent generation;
- warm exact-prompt RGB generation;
- sensor/preprocessing and input transfer;
- VAE decode;
- output handoff/serialization;
- DAgger policy computation; and
- complete observation-to-action latency.

For every reported path retain raw iterations and report mean, median, p90,
p99, minimum, maximum, failures, GPU/process memory, system RAM, clocks,
temperature, power, throttling, and display/co-resident workload.

The primary 5 Hz gate is p90 complete full-RGB application latency at or below
200 ms. A best-case or latent-only iteration cannot satisfy that gate.

Endurance must cover the longest intended demonstration and show:

- no OOM, NaN/Inf, illegal access, TensorRT error, or missed teardown;
- bounded process/device memory growth;
- stable thermals and clocks;
- no stale-input queue growth; and
- acceptable autoregressive image quality.

## Evidence record

Retain per target:

- package and foundation doctor reports;
- `export_manifest.json`;
- `engine_build_manifest.json`;
- TensorRT correctness and microbenchmark JSON;
- raw latent and RGB timing records;
- exact prompt, seed, input hashes, and output hashes;
- representative comparison grids and closed-loop video;
- `nvidia-smi -q` or equivalent power/thermal snapshot;
- container digest or bare-metal dependency freeze; and
- application-level DAgger timing and task-quality report.

Only completed evidence may change `config/deployment.json`,
`RELEASE_READY.json`, this document, or the model card from “pending” to
“validated.”