| # Lossless index — value and capacity unchanged |
|
|
| ## What would lose capacity |
|
|
| | move | effect | |
| |---|---| |
| | reduce D | C_addr and k* both move (k* ∝ D) | |
| | INT8 / PQ / shorter codes on packed HVs | Hamming distances warp → k* cliffs earlier | |
| | drop traces | value gone | |
| |
| A random HV in `{+1,-1}^D` has ~D bits of entropy. You cannot store it |
| in o(D) bits and keep the same geometry. |
| |
| ## What does not lose either |
| |
| Traces are already `a = bind(x, ctx)`, `v = token_hv`, with `x`/`ctx` |
| built from `Encoder._atom` / `_role` (sha256 → `random_hv`). |
|
|
| Store the **recipe** (atom keys + bind/bundle/sequence). Rebuild with |
| the same Encoder law. Measured here: |
|
|
| ``` |
| LOSSLESS_OK D=4096 bits_identical=3/3 |
| ``` |
|
|
| Bit-identical packed buffers ⇒ identical Hamming ⇒ identical LSH |
| buckets ⇒ **C_addr and k* unchanged**. The value HV is the same object |
| in `{+1,-1}^D`. |
| |
| Disk: integer keys instead of `2 * ceil(D/8)` bytes. D stays 20 000. |
| |
| ## Fallback |
| |
| If a buffer is not in the Encoder algebra, keep it packed. That |
| preserves value. It does not shrink that trace. |
| |
| ## Code |
| |
| `palimseste/lossless.py` — `LosslessCodec.materialize` + `assert_bit_identical`. |
| |
| The 10.4 GB `memory.bin` is still the packed ULTRA dump. New writes |
| can be recipes. Converting the old dump is a separate pass: invert |
| each packed `a` against `encode_sequence` of the training windows. |
| Until that pass, do not delete packed parts. |
| |
