docs/VALIDATION.md

Validation evidence and interpretation

Tested hardware and software

Item	Verified value
CPU/APU	AMD Ryzen AI MAX+ 395
GPU	Radeon 8060S, gfx1151
RAM	63.65 GiB visible
OS	Windows 11 Pro, build 26200
AMD driver	32.0.31041.1004
TheRock	gfx1151-7.13.0
HIP	7.13.99004-3309c6114a
Compiler	AMD Clang 23.0.0
llama.cpp	b10830 / 465e49b + one-file patch

Tested inference configuration

Setting	Value
Target	Qwen3.8-27B UD-Q4_K_XL
Draft	Qwen3.8-27B DFlash2 Q8_0
Target GPU layers	99
Draft GPU layers	99
Context	32,768
Batch / micro-batch	2,048 / 512
KV cache	q8_0 K and V
Parallel slots	1
DFlash p_min	0.90
Requested / effective n_max	8 / 7

Model files are not redistributed here. Names document the measured configuration, not an endorsement or compatibility promise for similarly named checkpoints.

Evidence hierarchy

The investigation used progressively stronger checks:

1. PE and dependency check — distinguishes a real 86.8 MB executable from the earlier 9,728-byte stub.
2. Version and command-line check — proves which build and model paths are live.
3. Health and model endpoints — proves the server loaded, but not correctness.
4. Metrics — proves DFlash actually drafted and the target accepted draft tokens.
5. Perplexity invariant — tests whether changing only micro-batch splitting changes model probabilities.
6. Positional needles — exercises early, middle, and late retrieval through the OpenAI-compatible user surface.
7. Full longctx/deep-reasoning battery — verifies the original failing consumer workload.

Only the last three establish the correctness repair.

Published and retained evidence

• evidence/pass1-old-hip/: exact first-pass local source shims and cache-init file
• evidence/validation-summary.json: machine-readable aggregate results for the first pass, control run, and repaired build
• scripts/probe_gfx1151_long_prompt.py: public synthetic split-prefill probe

Raw private battery JSONL files were retained by the project but are not included. They contain answer text that would contaminate future evaluation if published. The aggregate scores, counts, token budgets, timings, and interpretation needed to audit this release are preserved in the sanitized summary and this documentation.

What may be concluded

• The first-pass HIP executable corrupted the tested split-prefill long prompts.
• The fixed HIP executable did not reproduce that corruption.
• The fixed executable successfully exercised DFlash2 on full GPU offload.
• The repaired configuration scored 11/11 on the retained longctx slice.

What may not be concluded

• That every gfx1151 machine, driver, or ROCm build behaves identically.
• That DFlash2 inherently improves answer quality.
• That the fixed binary is faster than every Vulkan or upstream ROCm build.
• That rung 9 is a stable mean rather than a favorable single sample.
• That 262K context or parallel production serving is validated.