deterministic-engine / MATRIX_FORMAT.md
mirdax25-ux's picture
Release portable deterministic core 1.1.0: frozen formats, offline recovery and regression checks
ecad52f verified
|
Raw History Blame Contribute Delete
5.57 kB

Matrix checksum format: bodul-matrix-v1

Status: frozen compatibility specification, published 2026-11-03. Reference implementation: Matrix Core 3.6.0-OMEGA, Apache-2.0. Changes to this document may clarify examples but MUST NOT silently change the v1 byte sequence.

Inputs and normalization

The payload is a non-empty sequence of Unicode scalar values. Encode with strict UTF-8, without adding a byte order mark. An existing U+FEFF in input is data. Lone UTF-16 surrogates are invalid. The reference entry points impose a limit of 1,048,576 raw UTF-8 bytes BEFORE normalization; oversized input receives no seal.

Normalization profile bodul-whitespace-v1 uses exactly these 29 code points:

0009 000A 000B 000C 000D 001C 001D 001E 001F 0020
0085 00A0 1680 2000 2001 2002 2003 2004 2005 2006
2007 2008 2009 200A 2028 2029 202F 205F 3000

Replace each maximal run with one U+0020, then remove leading and trailing U+0020. Reject an empty result. No NFC, NFKC, case folding, punctuation changes, locale rules, or future Unicode property lookup applies. U+180E, U+200B, U+2060, and U+FEFF remain data. Visually similar text may intentionally have different checksums. This table preserves the legacy Python 3.12 whitespace behavior without depending on future Unicode databases.

UTC date and exact message bytes

Convert an explicitly supplied aware datetime to UTC FIRST. Select its date in the proleptic Gregorian calendar, years 0001 through 9999 inclusive. Format the date as ten ASCII characters YYYY-MM-DD, including leading zeroes.

Concatenate exactly, without quotes, BOM, terminal newline, or NUL terminator:

BODUL_MATRIX::<normalized text>::<YYYY-MM-DD>::OMEGA_FLOW

Encode as strict UTF-8 and apply SHA-256 as specified by FIPS 180-4. Return 64 lowercase hexadecimal characters, preserving leading zeroes.

The suffix is not a parser for recovering text: verification uses the supplied payload and date. Literal :: in the payload is ordinary data. There are no secret keys. The engine version, full timestamp, and response labels are NOT part of the hash input.

Timestamp interchange grammar

The portable JSON/CLI boundary accepts only ASCII:

YYYY-MM-DDTHH:MM:SS[.ffffff](Z|+HH:MM|-HH:MM)

Fractional seconds, when present, have 1 through 6 digits. Offset hours are 00..23; offset minutes are 00..59. Calendar validity is checked. -00:00 denotes an unknown offset and is rejected. Leap seconds, offsets containing seconds, week dates, compact dates, spaces instead of T, and missing offsets are rejected. Results use a UTC +00:00 offset; nonzero microseconds use six digits. UTC conversion outside 0001..9999 is rejected. This deliberately narrow grammar prevents a future parser expansion from changing accepted input implicitly.

The Python datetime API also accepts aware datetime objects; archive replay uses the stricter text interchange grammar. With no now, the Python core captures UTC once. Offline CLI replay requires --at explicitly.

Output and historical verification

Success includes valid: true, normalized input_payload, seal_date, matrix_verification_seal, timestamp, engine_version, checksum_format, normalization_profile, and hash_algorithm. An invalid payload includes valid: false, an error code and metadata, with NO seal. Machine schema: schemas/matrix-result-v1.json; independent fixtures: test_vectors.json.

verify_result verifies the consistency of a successful exported result. It requires the exact field set, supported format/profile/hash identifiers, canonical normalized input, a timestamp matching the UTC date, and the correct seal. Original unlabeled 3.5.1-OMEGA records remain readable. Other unlabeled records and unknown future formats fail closed. Invalid-input records cannot be verified as successful checksums.

Existing valid 3.5.1 checksums remain unchanged. Version 3.6 adds explicit format metadata and controlled Unicode/size errors; clients comparing complete objects must account for the additional fields and engine version.

The full result repeats only when text, time and engine metadata also match. The seal repeats for matching normalized text and UTC date. Metadata may be edited without changing a same-day seal; verification does not authenticate that metadata, authorship, original acquisition time, or external truth.

Future migrations

Never redefine v1 or infer the format from digest length alone. If a new hash, normalization profile or message layout is needed, publish a NEW format ID, domain-separated bytes, independent vectors, reader and migration notes. Keep the old reader for historical checks. Store original source text, old result, new result, migration date and source revision together; do not overwrite the only historical record. Rehashing does not prove when the original existed.

SHA-256 security is not guaranteed for a century. Review cryptographic guidance periodically. Archive manifests include SHA-256 and SHA-512 for byte comparison; neither is a signature, and both belong to the SHA-2 family.

References

These links explain current conventions. The explicit rules and bundled fixtures above are the preservation contract if the websites disappear.