Hyde validation update — September 17, 2026

See TRAINING.md and validation/hyde-20260917/ for the newly observed ROCm tests, full head-training cycle, live Jev comparison, and optional LoRA smoke. The prior CPU development record follows unchanged.

Validation record — September 17, 2026

Follow-up revision: TypeSafe contract and Hyde runner

49 tests passed and 3 optional Fullcollar tests skipped because Fullcollar is unavailable in the current environment: 52 discovered, no failures. Transcript: validation/unit-tests-update.txt. The original 44-test transcript remains intact as earlier evidence, not a claim about this later run.
New coverage checks structured requests, array state, null Choice descriptions, the 10-level Score limit, model-name enforcement, and equal predictions under question renaming, subsets, and aliases. Unknown question content is rejected before model work. Device enumeration alone cannot satisfy the offload check.
Python compilation, CLI discovery, structured-example validation, and shell syntax checks passed. The native C++ patch and model artifact are unchanged.
deploy/run-hyde.sh now runs the native build and interface checks with retained step/server logs. Its GPU success gate requires a ROCm device and positive layer-offload log evidence for both serving modes. The runner has not been executed on Fedora/ROCm; its log parser is covered by unit tests.
SSH using the supplied address and username returned Network is unreachable before authentication. No commands ran on Hyde and no files there changed.
Real-model inference was not rerun for this revision. The CPU evidence below records the earlier source revision. New general-purpose decision weights, question-conditioned parallel inference, and Jev parity remain unimplemented; see TYPESAFE-COMPATIBILITY.md.

Original CPU validation

44 tests passed with no skipped tests, including the actual Fullcollar 0.1.0 Python contract. The Fullcollar source was read from the existing workspace; it was not modified. Transcript: validation/unit-tests.txt.
Real IBM Granite 4.1 3B Q4_K_M inference through a locally compiled native llama.cpp CPU server passed for Choice, Score, and Noul. The application returned typed distributions and correctly abstained because the baseline was uncalibrated.
Real Granite feature extraction returned four finite, L2-normalized vectors with 2,560 dimensions each. A tiny linear head was fitted to those real vectors and produced finite two-class logits. This tests the native feature and head-training interface; it is not a trained production classifier.
The complete train/calibrate/test/save/reload flow passed on clearly labeled synthetic test features, including schema/runtime mismatch rejection and weight-integrity checks.
The native C++ probability patch compiled and passed the real-model test. git diff --check and reverse-application validation passed.
Python package editable installation, module import/CLI discovery, Python compilation, and shell-script syntax checks passed.

Failure found and repaired

The initial real-model integration test failed: upstream llama.cpp returned an out-of-grammar token in its post-sampling top probabilities. At the inspected revision, grammar rejection sampling can skip masking the full distribution when the sampled token is already valid. The client correctly rejected the incomplete candidate set. The included patch applies the grammar first when post-sampling probabilities are requested. The repaired path passed on the real model.

An initial attempt to contact local test servers across separate execution contexts failed because those contexts had separate network namespaces. The reproducible smoke script launches and tests its server subprocesses in the same execution context. This was an environment issue, not a model-service failure.

Exact reference configuration

Component	Observed value
Model repository	`ibm-granite/granite-4.1-3b-GGUF`
Repository revision	`ab4701481089b58a082ef63cc1cee738887293ff`
Weight filename	`granite-4.1-3b-Q4_K_M.gguf`
Weight size	2,099,501,664 bytes
Weight SHA-256	`662b0626cd58f443baea23559b469df6576a81d349649c59413b36a9fb32eb29`
llama.cpp revision	`aa39d7a3e145a88202793a89462d65e94a5fc25f` plus included patch
Build	GNU 13.3.0, Linux x86_64, CPU
Python	3.12.14
Native threads	4 generation, 4 prompt processing
Context / microbatch	4,096 / 4,096 tokens
Application input budget	2,048 tokens; overflow rejected
Embedding pooling	Last token, L2 normalization
GPU devices	None in the reference environment

The raw successful run is in validation/native-cpu-smoke.json. Its single three-question baseline request took approximately 4.51 seconds after model load. Four feature extractions plus a tiny head fit took approximately 1.46 seconds in total. These measurements use different inputs and do not establish a speedup, throughput curve, latency guarantee, or hardware comparison. They are CPU smoke timings, not Hyde performance estimates. No model-quality percentage is claimed.

Not yet validated

Building or running on Hyde: the original hostname did not resolve; the later attempt with the supplied address returned Network is unreachable. No command ran on Hyde, and no files or services there were changed.
Fedora/ROCm installation, driver version, actual GPU target, device permissions, successful layer offload, VRAM use, and thermal/performance behavior on Hyde.
Domain quality, calibration under distribution shift, out-of-distribution detection, rare-event risk, or reliability on real Fullcollar traffic.
Backbone fine-tuning, RLCD, or a new general-purpose trained model. This release trains supervised heads while keeping Granite frozen.
Jev quality/speed/cost parity. No Jev API was called and no comparable workload benchmark was run.
Speculative decoding and GPU/backend sampling with the probability patch. The included launchers do not enable these features.
Production load, concurrency beyond the one-active-request service policy, multi-host deployment, or a public API.

The unit test transcript includes a deprecation warning from the installed Starlette version's httpx test-client adapter. All HTTP tests still passed.

Hyde acceptance procedure

bash deploy/run-hyde.sh automates steps 1–3 and retains the run evidence. The domain-training and workflow checks in steps 4–5 require labeled data.

Run deploy/hyde-native.sh inspect and retain its host/ROCm/device output.
Run deploy/hyde-native.sh build in a new workspace. Inspect llama-server --list-devices and confirm the expected ROCm device.
Run scripts/smoke_native.py with its default GPU-layer setting and retain hyde-smoke.json. The smoke test now rejects absent/zero offload evidence; retain its server logs for inspection. A CPU run must be labeled explicitly with --gpu-layers 0.
Bind the live embedding runtime, train on verified domain labels, and use independent calibration/test data. Review selective error and coverage at prespecified thresholds, including errors for consequential routes.
Run in Fullcollar shadow mode, collect independently verified outcomes, and benchmark p50/p95 latency at representative input lengths. Choose activation criteria before inspecting the final acceptance set. Retain the old artifact for rollback. Serving selects a bundle explicitly; feedback cannot activate it.