Solomon BF16 CUDA parity validation — 21 September 2026
Full BF16 inference was exercised on the Apple M5 Max / 128 GB Mac. No weights or temperatures were changed. New temperature fitting is out of scope.
Confirmed results
| Check | Result |
|---|---|
| Fresh text fixtures, all five answer types | 22/22 decisions match CUDA |
| Fresh page-image fixtures | 4/4 decisions match CUDA |
| Saved text subset | 645/645 branch decisions and 327/327 whole-question decisions match CUDA |
| Prefix tokens and captured full token sequences | Exact match with CUDA |
| Replay and fresh process versus previous run | Zero logit drift on this Mac |
| Public API cache isolation, evidence and invalid-input checks | Passed |
| Existing package tests | 21 passed |
| Parity workflow and reserved-answer regression tests | 2 passed |
| Lint | Passed |
The saved subset covers 196 documents and all five tasks. It comes from previously scored calibration-fit inputs and is a diagnostic subset, not a complete held-out qualification. It was used only for comparison; nothing was fitted.
Numerical differences
With the same frozen CUDA serving temperatures applied once on both backends, maximum probability drift on the saved subset is 2.5871 percentage points; mean drift is 0.0126 points. At identical T=1, maximum drift is 4.1508 points and mean drift is 0.0078 points. All compared decisions still agree.
CUDA and MLX logits are not bitwise equal. The report preserves the largest probability differences for inspection. No claim is made that the CUDA confidence calibration has been independently validated for MLX.
The focused runs used 59.90 GB peak Metal allocation. This is not a measurement of the maximum supported context or many-page memory use.
Remaining work
The untouched certification panel is now scored directly: 1,200 documents / 21,718 branches. It skips fit/development scoring and temperature fitting. The complete agreement gate is at least 99.9% for both branches and whole questions, at T=1 and at the frozen CUDA temperatures. Probability drift and task accuracy are reported separately.
Image fixtures and API evidence checks are limited tests; broad image accuracy and CUDA evidence-selection equivalence are not established by them.
Runtime fingerprint: 33c9b63f036c5c039a2db4ba944f5b0042ae8b8e485e68c2aeddb57c407f8908.
Detailed evaluation inputs and outputs remain private; the source release includes aggregate results only.
Adapter-only packaging checks
The 21 September source update passed 24 portable package tests, including seven Hub-loader tests. The local development suite passed 30 tests. The new verifier also checked all 37 files of the existing full BF16 conversion. The built wheel contains the original converter with its exact recorded source hash and both pinned-input manifests. These checks cover packaging and integrity, not additional trained-model parity.