docs/modules/sha256.md

bankML/sha256.rs — SHA-256 for the model pin, and the FORK.json pin scanner

Summary

sha256.rs is a dependency-free SHA-256 (FIPS 180-4) and the scanner that finds a file's pinned hash in a FORK.json. A GGUF plays only if its sha256 equals the record in the PYTHAI fork's FORK.json (files[].sha256 for its path).

On x86_64 it uses the CPU's SHA extensions (SHA-NI: AMD Zen, Intel Ice Lake and later) when present and the portable rounds otherwise. Both give the same digest.

Callers: bankml::pin and bankml::verify (the second half of the gate, see bankml.md), so every bankml serve start and every model switch hashes the whole model. Also bankml sha256 FILE; the receipt's request_sha256 and response_sha256 (serve::Tally); since 0.3.8 the checksum that closes every slot file (Native::save_slot / restore_slot); and the kernel oracles, which hash dequantized tensors to compare them with ggml's.

Technical usage

pub struct Sha256 { /* private */ }          // impl Default
impl Sha256 {
    pub fn update(&mut self, data: &[u8])
    pub fn finish(self) -> [u8; 32]
}
pub fn hex(b: &[u8]) -> String
pub fn file_hex(p: &std::path::Path) -> std::io::Result<String>
pub fn pinned_sha256(fork_json: &str, name: &str) -> Option<String>

• Sha256::default() starts a hash; update streams any number of bytes; finish pads and returns the digest.
• file_hex hashes a file in 1 MiB reads and returns lowercase hex.
• pinned_sha256 returns the lowercase 64-hex "sha256" of the object in "files" whose "path" equals name. It returns None (unpinned, refuse) when there is no such object, or when its sha256 is missing or not 64 hex digits. Key order inside the object does not matter.

Environment:

variable	effect
BANKML_NO_SHANI	set: hash with the portable rounds instead of SHA-NI (read once, then cached)

bankml sha256 .models/Bonsai-8B-Q1_0.gguf          # prints "<hex>  <file>", like sha256sum
bankml pin .models/Bonsai-8B-Q1_0.gguf --fork FORK.json

let mut h = bankml::sha256::Sha256::default();
h.update(b"abc");
assert_eq!(bankml::sha256::hex(&h.finish()), "ba7816bf8f01cfea414140de5dae2223b00361a396177a9cb410ff61f20015ad");

How it is verified

• fips_vectors: the FIPS 180-4 vectors for "", "abc", the 448-bit message, and one million 'a' streamed in 1,000-byte updates across block boundaries.
• hardware_path_equals_portable_on_every_length: the dispatching path (SHA-NI when the CPU has it) against the portable rounds on every length 0–1,000, on 4,095 / 4,096 / 4,097 / 65,536 / 69,999 bytes, and on split updates. The test prints which hardware path it exercised.
• fork_json_pin: lowercase normalisation, a non-64-hex record treated as unpinned, a missing file, sha256 before path, and the compact shape fork_bonsai.py writes.
• Per oracles.md §4, a real 1.16 GB model must hash to coreutils sha256sum's value and its published pin. The Rust pin was also checked against the real PYTHAI/Bonsai-8B-gguf-fork FORK.json.
• testing/cli.rs: pin_and_verify.

Advantages and efficiency

• No crate for the trust root. The hash that decides whether a model may answer is one file in the crate, with published test vectors as its oracle.
• Hardware rounds, chosen at run time. The SHA-NI path does two rounds per sha256rnds2 and the message schedule with sha256msg1/sha256msg2, following Intel's reference layout. Detection (is_x86_feature_detected! for sha, sse4.1, ssse3) runs once and is cached in an atomic. The release build needs no target-cpu flag.
• Measured (laptop, Ryzen 3 3200U, PERFORMANCE.md, CHANGELOG 0.1.8): 0.23 s against 1.25 s portable for a 248 MB model (5.5×); 2.9 s for the 1.16 GB 8B model; restarting the 8B carrier (hash and load) went from 25 s to 7.0 s. Cold against warm hashing of the 1.7B model measured 364 against 419 MB/s; the disk keeps up with SHA-NI, so no read-ahead hint was adopted.
• Streaming, bounded memory. file_hex uses one 1 MiB buffer whatever the file size. update hands whole 64-byte blocks to the compressor in one call and keeps only a partial block between calls.
• Narrow scanner, not a JSON parser. pinned_sha256 looks for exactly one shape: anything it cannot read as a 64-hex record for that path is None, which refuses.
• Practice. unsafe is confined to the #[target_feature(enable = "sha,sse2,ssse3,sse4.1")] intrinsic function, called only after detection, with the safety condition stated at the call.

Limitations

• The scanner is deliberately narrow. It expects the path and sha256 as plain JSON strings (the comment notes no escapes are needed); it is not a general JSON parser.
• SHA-NI is used only on x86_64. Other targets run the portable rounds.
• Hashing reads the whole file, so the pin costs time proportional to model size on every verification. The guard runs first so a bad header is refused before this cost.

See also

• bankml.md (pin, verify), gguf.md (the guard that runs first)
• ../oracles.md §4, ../PERFORMANCE.md (0.1.8 – 0.2.0 table)
• ../usage.md §9 (receipts), §13 (environment)