Add frozen BASIS experiment inputs

.gitattributes

@@ -36,3 +36,6 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 low_rank/llama31_8b_c4only_w512_whitening_only_lowrank/r0p4/tokenizer.json filter=lfs diff=lfs merge=lfs -text
 low_rank/llama31_8b_c4only_w512_whitening_only_lowrank/r0p6/tokenizer.json filter=lfs diff=lfs merge=lfs -text
 low_rank/llama31_8b_c4only_w512_whitening_only_lowrank/r0p8/tokenizer.json filter=lfs diff=lfs merge=lfs -text
+basis/sources/llama31/asvd_r080/model/tokenizer.json filter=lfs diff=lfs merge=lfs -text
+basis/sources/llama31/modegpt_r080/model/tokenizer.json filter=lfs diff=lfs merge=lfs -text
+basis/sources/llama31/modegpt_r060/model/tokenizer.json filter=lfs diff=lfs merge=lfs -text

README.md

@@ -1,9 +1,26 @@
-# FlashSVD Example Checkpoints
+# FlashSVD checkpoints and BASIS experiment inputs
+
+## BASIS: download experiment inputs here
+
+For the [Additional Experiments](https://github.com/Zishan-Shao/Basis/tree/handoff/paper-20260915/experiments/intern), start with the [BASIS data guide](https://huggingface.co/Duke-CEI-SVD/FlashSVD/blob/refs%2Fpr%2F1/basis/README.md).
+
+| You need… | Go to… |
+|---|---|
+| **C4 training data** | [Six frozen C4 streams: two backbones × three seeds](https://huggingface.co/Duke-CEI-SVD/FlashSVD/tree/refs%2Fpr%2F1/basis/data/c4) |
+| Q64/Q128 probe and held-out tensors | [Probe inputs](https://huggingface.co/Duke-CEI-SVD/FlashSVD/tree/refs%2Fpr%2F1/basis/data/probes) |
+| Cached per-window gradients | [Q128](https://huggingface.co/Duke-CEI-SVD/FlashSVD/tree/refs%2Fpr%2F1/basis/gradients/q128) · [Original Q64](https://huggingface.co/Duke-CEI-SVD/FlashSVD/tree/refs%2Fpr%2F1/basis/gradients/q64) |
+| Exact frozen compressed Sources | [16 paper checkpoints plus four LowRankArena imports](https://huggingface.co/Duke-CEI-SVD/FlashSVD/tree/refs%2Fpr%2F1/basis/sources) |
+| Tokenizers, original paths, and checksums | [Tokenizers](https://huggingface.co/Duke-CEI-SVD/FlashSVD/tree/refs%2Fpr%2F1/basis/tokenizers) · [Artifact manifest](https://huggingface.co/Duke-CEI-SVD/FlashSVD/blob/refs%2Fpr%2F1/basis/manifest.json) |
+
+The shared data and caches total approximately **4.4 GiB**; the 16 hosted checkpoints total **178.6 GiB**. Download only the Source files needed for your task. The guide includes selective download commands, loader instructions, checkpoint coverage, and the remaining Task 05 Source requests. Tasks 01–02 already include their numerical inputs in GitHub.
+
+## Original FlashSVD example checkpoints
 
 This repository contains a small set of curated example low-rank checkpoints for FlashSVD-v1.5 benchmarking and integration tests.
 The current examples cover SVD-LLM v1, Basis Sharing, and DobiSVD variants.
 
-Properties:
+Properties of these original example checkpoints:
+
 - all uploaded checkpoints remain factorized / low-rank; they are not fused into dense tensors
 - removable RoPE runtime buffers are omitted from the SVD-LLM examples
 - each example is a self-contained HF export subfolder

basis/README.md

@@ -0,0 +1,111 @@
+# BASIS experiment inputs
+
+Shared inputs for the [BASIS Additional Experiments](https://github.com/Zishan-Shao/Basis/tree/handoff/paper-20260915/experiments/intern). Download only the folders needed for your assignment.
+
+## Where to find your inputs
+
+| Input | Folder | Size | Use |
+|---|---|---:|---|
+| **Frozen C4 recovery streams** | [`data/c4/`](https://huggingface.co/Duke-CEI-SVD/FlashSVD/tree/refs%2Fpr%2F1/basis/data/c4/) | 0.604 GiB | Task 05 training; Task 03 overlap exclusions |
+| Original Q64/V64 | [`data/probes/q64/`](https://huggingface.co/Duke-CEI-SVD/FlashSVD/tree/refs%2Fpr%2F1/basis/data/probes/q64/) | <1 MiB | Original Q0/P0 uses the first 16 Q64 windows |
+| Independent Q128/V128 | [`data/probes/q128/`](https://huggingface.co/Duke-CEI-SVD/FlashSVD/tree/refs%2Fpr%2F1/basis/data/probes/q128/) | 1.1 MiB | Tasks 03–04 |
+| C4 validation shard for extending Q128 | [`data/c4_probe_shard/`](https://huggingface.co/Duke-CEI-SVD/FlashSVD/tree/refs%2Fpr%2F1/basis/data/c4_probe_shard/) | 38.6 MiB | Task 03 candidate documents |
+| Q128 Source/dense gradient cache | [`gradients/q128/`](https://huggingface.co/Duke-CEI-SVD/FlashSVD/tree/refs%2Fpr%2F1/basis/gradients/q128/) | 1.978 GiB | Task 03: reuse the existing first 128 windows |
+| Original Q64 gradient cache | [`gradients/q64/`](https://huggingface.co/Duke-CEI-SVD/FlashSVD/tree/refs%2Fpr%2F1/basis/gradients/q64/) | 1.764 GiB | Original pressure references and dense Q0 reuse |
+| Frozen tokenizers | [`tokenizers/`](https://huggingface.co/Duke-CEI-SVD/FlashSVD/tree/refs%2Fpr%2F1/basis/tokenizers/) | 18.3 MiB | Both backbones |
+| Frozen compressed Sources | [`sources/`](https://huggingface.co/Duke-CEI-SVD/FlashSVD/tree/refs%2Fpr%2F1/basis/sources/) | 178.633 GiB hosted here | Download individual checkpoints, usually 7–14 GiB |
+| Original manifests and example C4 run specs | [`provenance/`](https://huggingface.co/Duke-CEI-SVD/FlashSVD/tree/refs%2Fpr%2F1/basis/provenance/) | <1 MiB | Input identity and protocol lookup |
+
+[manifest.json](manifest.json) records every artifact's relative path, original path, byte count, and SHA-256, together with Source identities and dense-parent revisions. The linked GitHub repository requires collaborator access.
+
+## C4: start here for Task 05
+
+Each stream is stored as:
+
+```text
+data/c4/seed_7711/llama31/
+  tokens.pt
+  manifest.json
+  manifest.sha256
+  provenance.jsonl
+```
+
+The same layout exists for both `llama31` and `qwen3`, with seeds **7711, 7712, and 7713**. Each contains the original 3,072 × 2,048-token training windows, ordering, document provenance, and overlap audit. Use the same backbone/seed stream across methods and retained ratios. These are tokenized recovery inputs; the separate `data/c4_probe_shard/` folder supplies raw C4 validation candidates for Task 03.
+
+From the BASIS checkout:
+
+```bash
+python -m pip install huggingface_hub
+python paper/download_inputs.py --group c4 --group q64 --group tokenizers
+```
+
+This downloads to `data/flashsvd/basis/`, pins the published revision, verifies SHA-256 checksums, and writes `path_map.json` for locating inputs referenced by historical manifests.
+
+## Download for each assignment
+
+**Tasks 01–02:** numerical inputs and reference figures are already in GitHub. The current assignments do not need model checkpoints or recovery adapters.
+
+**Task 03:** reuse the Q128 cache and obtain inputs for the additional 384 documents:
+
+```bash
+python paper/download_inputs.py \
+  --group q128 --group q128_gradients --group q64 --group c4 \
+  --group c4_probe_shard --group tokenizers
+python paper/download_inputs.py --source llama31/swiftsvd_r080
+```
+
+Repeat `--source` for the 12 r0.8 Sources in the assignment. Q128 contains **14 states × 8 blocks × 16 windows**: 12 Sources plus two dense parents, all at step 0. Preserve global window indices. Smaller prefixes in the new sweep come from Q128; original Q64 uses different documents.
+
+**Task 04:** download Q128, tokenizers, and the six r0.6 SVD-LLM / Swift-SVD / MoDeGPT Sources. The cached Q128 dense gradients can be reused when the measurement identities match.
+
+```bash
+python paper/download_inputs.py --group q128 --group tokenizers
+python paper/download_inputs.py --source qwen3/modegpt_r060
+```
+
+**Task 05:** use the C4 command above, obtain the original dense Q0 references from `q64_gradients`, and fill the 36-Source inventory. The new ratio panel still requires the task-specific training/evaluation workers and new recovery runs.
+
+Use `--list` to inspect groups and Source IDs, or add `--dry-run` to check download size.
+
+## Checkpoint coverage and loading
+
+The [Source index (CSV)](sources/index.csv) lists all checkpoint paths, sizes, loader kinds, and identities. This package supports **20 frozen compressed Sources**: **16 checkpoints are hosted here**, and **four reuse existing LowRankArena weights** through pinned download references. The downloader selects the correct repository automatically.
+
+- All six methods at **r0.8** for both backbones: 12 Sources for Task 03.
+- **SVD-LLM, Swift-SVD, and MoDeGPT at r0.6** for both backbones: six Sources for Task 04.
+- **GFWSVD at r0.4** for both backbones: two existing constructions for the Task 05 inventory. Their manifests specify nested leading-factor truncation of the frozen r0.8 Source, with no new calibration; retain that recipe when assessing reuse.
+
+### Reuse existing LowRankArena weights
+
+The downloader reuses **Qwen3 ASVD r0.8, MoDeGPT r0.8/r0.6, and Basis Sharing r0.8** from LowRankArena. Their weight files are not duplicated in FlashSVD. Original runtime/config metadata is supplied here where needed to preserve the paper Source.
+
+For **Qwen3 Basis Sharing**, use `kind="basis_sharing"` from `source.json`. The existing loader restores the published weights to the exact paper parameters, buffer values, and shared coordinates; the [equivalence audit](sources/qwen3/basis_sharing_r080/equivalence.json) records the check. Its original serialized `.pt` identity is retained as `paper_checkpoint_identity`, separately from the provided HF directory identity. Record both when matching the old Q128 cache.
+
+The **Llama Swift-SVD paper Sources** use C4 calibration with **512 windows, seed 20260819**. LowRankArena's published Swift-SVD variants use **256 windows, seed 42**, with C4 or WikiText-2. Use the paper Sources here when reusing the paper gradient cache.
+
+Each assembled Source folder includes `source.json`, its checkpoint, and available construction provenance. For the four imports, the downloader combines LowRankArena weights with the metadata hosted here. Single-file `model.pt` checkpoints retain their native BASIS format; ASVD/MoDeGPT directories contain their original sharded weights and runtime metadata. Load them using the BASIS checkout:
+
+```python
+from basis.compression.checkpoints import load_checkpoint
+
+model = load_checkpoint(
+    "data/flashsvd/basis/sources/llama31/swiftsvd_r080/model.pt"
+)
+```
+
+Use the corresponding `source.json` path and `kind`, and follow the assignment's dtype and tokenizer conventions.
+
+Additional compressed model candidates are available from [LowRankArena](https://huggingface.co/Duke-CEI-SVD/LowRankArena), including Llama Swift-SVD under `new_ckpt/`. Match the construction recipe and calibration as well as method and ratio. At the audited LowRankArena revision `d95d6ebddec25227ebc87d4814a60362dfda5cd8`, the Task 05 panel has nominal method/backbone/ratio coverage for 27 of 36 configurations. The two GFWSVD r0.4 files here add two candidates. **GFWSVD r0.5/r0.7 for both backbones and Qwen3 Swift-SVD r0.4/r0.5/r0.7 remain unprovided**; record these seven entries in the inventory and coordinate with the project lead for prepared Sources.
+
+Dense weights are available from their original repositories, pinned to:
+
+| Backbone | Dense parent | Revision |
+|---|---|---|
+| Llama-3.1-8B | [meta-llama/Llama-3.1-8B](https://huggingface.co/meta-llama/Llama-3.1-8B) | `d04e592bb4f6aa9cfee91e2e20afa771667e1d4b` |
+| Qwen3-8B-Base | [Qwen/Qwen3-8B-Base](https://huggingface.co/Qwen/Qwen3-8B-Base) | `49e3418fbbbca6ecbdf9608b4d22e5a407081db4` |
+
+## Preserve the frozen inputs
+
+Original manifests retain their historical paths and hashes. Use the package inventory or downloader's path map to create fresh task manifests, while retaining the originals for provenance. The path map describes the full inventory; download an input before using its mapped path. Relocation does not make old execution locks valid on a new machine.
+
+This package includes Source/dense measurements, not recovered adapters or optimizer states. Avg7 benchmark data must be prepared using the frozen task definitions linked from the assignment. C4 originates from [allenai/c4](https://huggingface.co/datasets/allenai/c4); retain the original dataset and model access conditions.

basis/data/c4/seed_7711/llama31/manifest.json

@@ -0,0 +1,102 @@
+{
+  "backbone": "llama31",
+  "corpus": "c4",
+  "dataset": "allenai/c4",
+  "dataset_recycling": false,
+  "documents_read": 76150,
+  "documents_selected": 2074,
+  "drop_last": false,
+  "effective_batch_size": 8,
+  "eos_appended": false,
+  "epoch_count": 1,
+  "final_effective_batch_sequences": 8,
+  "final_sequence_tokens": 2048,
+  "fixed_measurement_overlap": {
+    "exact_match": true,
+    "overlap_windows": 0,
+    "shingle_tokens": 32,
+    "training_windows_audited": 3072
+  },
+  "fixed_measurements": {
+    "evaluation": {
+      "artifact_sha256": "5cb74049e7a5039fd32dcbf7581807431c4fd828d6ad5c09bda4a5ae02bf7813",
+      "path": "/home/zs89/Basis/archives/20260910_recovery_protocol_correction/payload/h2_exp/artifacts/llama31/evaluation.pt",
+      "shape": [
+        64,
+        256
+      ],
+      "tensor_sha256": "fb197f8496a8abec6b950c4a81971e3579ab571075da494b43a8aa3927b96c45"
+    },
+    "probe": {
+      "artifact_sha256": "386c2f19333c195c47cea4adc2a882f47615c8d526d5f903bc8911a36a851a9b",
+      "path": "/home/zs89/Basis/archives/20260910_recovery_protocol_correction/payload/h2_exp/artifacts/llama31/probe.pt",
+      "shape": [
+        64,
+        256
+      ],
+      "tensor_sha256": "117bcb58782b37ea6878eee9de789f11f64aacc90a0f1ceb41ef2bf07ebf9015"
+    }
+  },
+  "input_positions": 6291456,
+  "measurement_overlap_audit": {
+    "evaluation_tensor_sha256": "fb197f8496a8abec6b950c4a81971e3579ab571075da494b43a8aa3927b96c45",
+    "overlap_windows": 0,
+    "passed": true,
+    "probe_tensor_sha256": "117bcb58782b37ea6878eee9de789f11f64aacc90a0f1ceb41ef2bf07ebf9015",
+    "shingle_tokens": 32
+  },
+  "nonoverlapping_source_spans": true,
+  "nonpadding_tokens": 6291456,
+  "num_sequences": 3072,
+  "objective": "causal LM on full unpadded C4 windows",
+  "optimizer_steps": 384,
+  "padded_tokens": 0,
+  "provenance_file_sha256": "12d474f0a12d5f88cd679c009ff6a583e779384ca6cb002abfeeb538dc0c9fc5",
+  "provenance_path": "/home/zs89/Basis/h1_exp/artifacts/joint_single_seed_20260910/data/c4/llama31/provenance.jsonl",
+  "revision": "1588ec454efa1a09f29cd18ddd04fe05fc8653a2",
+  "same_document_chunk_candidates_across_backbones": true,
+  "save_steps": [
+    64,
+    128,
+    192,
+    256,
+    320,
+    384
+  ],
+  "schema": "joint-recovery-data-v1",
+  "seed": 7711,
+  "selection": "first 3072 admissible paired nonoverlapping within-document chunks in pinned shard row order; one shared seed shuffle after selection",
+  "selection_exclusions": {
+    "documents_short_in_at_least_one_tokenizer": 74076
+  },
+  "sequence_length": 2048,
+  "shape": [
+    3072,
+    2048
+  ],
+  "source_shard": "/home/zs89/.cache/huggingface/hub/datasets--allenai--c4/blobs/8ef8d75b0e045dec4aa5123a671b4564466b0707086a7ed1ba8721626dfffbc9",
+  "source_shard_sha256": "8ef8d75b0e045dec4aa5123a671b4564466b0707086a7ed1ba8721626dfffbc9",
+  "split": "train",
+  "status": "frozen_and_audited",
+  "supervised_targets": 6288384,
+  "tensor_hash_schema": "sha256-contiguous-c-order-int64-little-endian-or-bool-bytes-v1",
+  "tensor_sha256": {
+    "attention_mask": "856a7d62e6bf8b1e8cd815bd086af8cd63158e81eeaacf2228e7179fe5f9dc7c",
+    "input_ids": "6ab85ae0c181172d79701a74cf783c6fde8b636e32702b78ed89d59e255b17dc",
+    "labels": "6ab85ae0c181172d79701a74cf783c6fde8b636e32702b78ed89d59e255b17dc"
+  },
+  "tokenizer": {
+    "add_special_tokens": false,
+    "eos_token_id": 128001,
+    "files_sha256": {
+      "special_tokens_map.json": "462d91939dbc37178aa5a3eae7068d1990ccc92e09f288cc71f42cdf139d69cc",
+      "tokenizer.json": "76e48799b099d43365bd24ccd8ecc5aedac831718da780552f03b0a6eb4412aa",
+      "tokenizer_config.json": "8004530facf809ac432114de2a4dcc65fcb632da5ec16d666091aeb6a2ee444a"
+    },
+    "pad_token_id": 128001,
+    "path": "/home/zs89/.cache/huggingface/hub/models--meta-llama--Llama-3.1-8B/snapshots/d04e592bb4f6aa9cfee91e2e20afa771667e1d4b"
+  },
+  "tokens_file_sha256": "7dd056c4836f5e4c38f3e9fc0fc746d987aaf3b0e2b06dca634867e45a871ee7",
+  "tokens_path": "/home/zs89/Basis/h1_exp/artifacts/joint_single_seed_20260910/data/c4/llama31/tokens.pt",
+  "unique_full_windows": 3072
+}

basis/data/c4/seed_7711/llama31/manifest.sha256

@@ -0,0 +1 @@
+b2c294bfd31ceb80cb66e2de0fba1d06f492b4597b9f2a93a379386986ae5dcc  manifest.json

basis/data/c4/seed_7711/llama31/tokens.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:7dd056c4836f5e4c38f3e9fc0fc746d987aaf3b0e2b06dca634867e45a871ee7
+size 106956915

basis/data/c4/seed_7711/qwen3/manifest.json

@@ -0,0 +1,101 @@
+{
+  "backbone": "qwen3",
+  "corpus": "c4",
+  "dataset": "allenai/c4",
+  "dataset_recycling": false,
+  "documents_read": 76150,
+  "documents_selected": 2074,
+  "drop_last": false,
+  "effective_batch_size": 8,
+  "eos_appended": false,
+  "epoch_count": 1,
+  "final_effective_batch_sequences": 8,
+  "final_sequence_tokens": 2048,
+  "fixed_measurement_overlap": {
+    "exact_match": true,
+    "overlap_windows": 0,
+    "shingle_tokens": 32,
+    "training_windows_audited": 3072
+  },
+  "fixed_measurements": {
+    "evaluation": {
+      "artifact_sha256": "6d0d654ec4775bb250d458c7c5d2e4ab6060718cd588919d69d8d243fe3f886e",
+      "path": "/home/zs89/Basis/archives/20260910_recovery_protocol_correction/payload/h2_exp/artifacts/qwen3/evaluation.pt",
+      "shape": [
+        64,
+        256
+      ],
+      "tensor_sha256": "39776fef3736ac0e761aecc088db0cf858afa82ae56e8e2593b8f724f63e6abb"
+    },
+    "probe": {
+      "artifact_sha256": "aaafc696c0f30d57c385f5833e1583e6262da3847b87904923985734a1994b42",
+      "path": "/home/zs89/Basis/archives/20260910_recovery_protocol_correction/payload/h2_exp/artifacts/qwen3/probe.pt",
+      "shape": [
+        64,
+        256
+      ],
+      "tensor_sha256": "9d8f5bf750276d5509cc21632b77d298269445420739311c379aee46955aacac"
+    }
+  },
+  "input_positions": 6291456,
+  "measurement_overlap_audit": {
+    "evaluation_tensor_sha256": "39776fef3736ac0e761aecc088db0cf858afa82ae56e8e2593b8f724f63e6abb",
+    "overlap_windows": 0,
+    "passed": true,
+    "probe_tensor_sha256": "9d8f5bf750276d5509cc21632b77d298269445420739311c379aee46955aacac",
+    "shingle_tokens": 32
+  },
+  "nonoverlapping_source_spans": true,
+  "nonpadding_tokens": 6291456,
+  "num_sequences": 3072,
+  "objective": "causal LM on full unpadded C4 windows",
+  "optimizer_steps": 384,
+  "padded_tokens": 0,
+  "provenance_file_sha256": "12d474f0a12d5f88cd679c009ff6a583e779384ca6cb002abfeeb538dc0c9fc5",
+  "provenance_path": "/home/zs89/Basis/h1_exp/artifacts/joint_single_seed_20260910/data/c4/qwen3/provenance.jsonl",
+  "revision": "1588ec454efa1a09f29cd18ddd04fe05fc8653a2",
+  "same_document_chunk_candidates_across_backbones": true,
+  "save_steps": [
+    64,
+    128,
+    192,
+    256,
+    320,
+    384
+  ],
+  "schema": "joint-recovery-data-v1",
+  "seed": 7711,
+  "selection": "first 3072 admissible paired nonoverlapping within-document chunks in pinned shard row order; one shared seed shuffle after selection",
+  "selection_exclusions": {
+    "documents_short_in_at_least_one_tokenizer": 74076
+  },
+  "sequence_length": 2048,
+  "shape": [
+    3072,
+    2048
+  ],
+  "source_shard": "/home/zs89/.cache/huggingface/hub/datasets--allenai--c4/blobs/8ef8d75b0e045dec4aa5123a671b4564466b0707086a7ed1ba8721626dfffbc9",
+  "source_shard_sha256": "8ef8d75b0e045dec4aa5123a671b4564466b0707086a7ed1ba8721626dfffbc9",
+  "split": "train",
+  "status": "frozen_and_audited",
+  "supervised_targets": 6288384,
+  "tensor_hash_schema": "sha256-contiguous-c-order-int64-little-endian-or-bool-bytes-v1",
+  "tensor_sha256": {
+    "attention_mask": "856a7d62e6bf8b1e8cd815bd086af8cd63158e81eeaacf2228e7179fe5f9dc7c",
+    "input_ids": "e9da0d1cb73b4166880be90a4c93132ee3f153efc5a7a20cbd53918ff7919066",
+    "labels": "e9da0d1cb73b4166880be90a4c93132ee3f153efc5a7a20cbd53918ff7919066"
+  },
+  "tokenizer": {
+    "add_special_tokens": false,
+    "eos_token_id": 151643,
+    "files_sha256": {
+      "tokenizer.json": "c0382117ea329cdf097041132f6d735924b697924d6f6fc3945713e96ce87539",
+      "tokenizer_config.json": "3c04ed3ca964ea2f6b2b5faf0dc4d31aec1cb1e8b4bcf63f402d295046b422b5"
+    },
+    "pad_token_id": 151643,
+    "path": "/home/zs89/.cache/huggingface/hub/models--Qwen--Qwen3-8B-Base/snapshots/49e3418fbbbca6ecbdf9608b4d22e5a407081db4"
+  },
+  "tokens_file_sha256": "52f32bfe7d634c39e18f3f0c7242def4229564b29094ac95376fe3cc0df18519",
+  "tokens_path": "/home/zs89/Basis/h1_exp/artifacts/joint_single_seed_20260910/data/c4/qwen3/tokens.pt",
+  "unique_full_windows": 3072
+}

basis/data/c4/seed_7711/qwen3/manifest.sha256

@@ -0,0 +1 @@
+23d130968fe7270152db9a4e48b5210f1874ce6263279e84b6c3b00f2c543943  manifest.json

basis/data/c4/seed_7711/qwen3/tokens.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:52f32bfe7d634c39e18f3f0c7242def4229564b29094ac95376fe3cc0df18519
+size 106956915

basis/data/c4/seed_7712/llama31/manifest.json

@@ -0,0 +1,102 @@
+{
+  "backbone": "llama31",
+  "corpus": "c4",
+  "dataset": "allenai/c4",
+  "dataset_recycling": false,
+  "documents_read": 76150,
+  "documents_selected": 2074,
+  "drop_last": false,
+  "effective_batch_size": 8,
+  "eos_appended": false,
+  "epoch_count": 1,
+  "final_effective_batch_sequences": 8,
+  "final_sequence_tokens": 2048,
+  "fixed_measurement_overlap": {
+    "exact_match": true,
+    "overlap_windows": 0,
+    "shingle_tokens": 32,
+    "training_windows_audited": 3072
+  },
+  "fixed_measurements": {
+    "evaluation": {
+      "artifact_sha256": "5cb74049e7a5039fd32dcbf7581807431c4fd828d6ad5c09bda4a5ae02bf7813",
+      "path": "/home/zs89/Basis/archives/20260910_recovery_protocol_correction/payload/h2_exp/artifacts/llama31/evaluation.pt",
+      "shape": [
+        64,
+        256
+      ],
+      "tensor_sha256": "fb197f8496a8abec6b950c4a81971e3579ab571075da494b43a8aa3927b96c45"
+    },
+    "probe": {
+      "artifact_sha256": "386c2f19333c195c47cea4adc2a882f47615c8d526d5f903bc8911a36a851a9b",
+      "path": "/home/zs89/Basis/archives/20260910_recovery_protocol_correction/payload/h2_exp/artifacts/llama31/probe.pt",
+      "shape": [
+        64,
+        256
+      ],
+      "tensor_sha256": "117bcb58782b37ea6878eee9de789f11f64aacc90a0f1ceb41ef2bf07ebf9015"
+    }
+  },
+  "input_positions": 6291456,
+  "measurement_overlap_audit": {
+    "evaluation_tensor_sha256": "fb197f8496a8abec6b950c4a81971e3579ab571075da494b43a8aa3927b96c45",
+    "overlap_windows": 0,
+    "passed": true,
+    "probe_tensor_sha256": "117bcb58782b37ea6878eee9de789f11f64aacc90a0f1ceb41ef2bf07ebf9015",
+    "shingle_tokens": 32
+  },
+  "nonoverlapping_source_spans": true,
+  "nonpadding_tokens": 6291456,
+  "num_sequences": 3072,
+  "objective": "causal LM on full unpadded C4 windows",
+  "optimizer_steps": 384,
+  "padded_tokens": 0,
+  "provenance_file_sha256": "3bbdd48d1b30e799e14267ca6a9b91f767b4d5b3f747efa5a61bf2ee9757c105",
+  "provenance_path": "/home/zs89/Basis/h1_exp/artifacts/joint_seed_robustness_20260911/data/seed_7712/c4/llama31/provenance.jsonl",
+  "revision": "1588ec454efa1a09f29cd18ddd04fe05fc8653a2",
+  "same_document_chunk_candidates_across_backbones": true,
+  "save_steps": [
+    64,
+    128,
+    192,
+    256,
+    320,
+    384
+  ],
+  "schema": "joint-recovery-data-v1",
+  "seed": 7712,
+  "selection": "first 3072 admissible paired nonoverlapping within-document chunks in pinned shard row order; one shared seed shuffle after selection",
+  "selection_exclusions": {
+    "documents_short_in_at_least_one_tokenizer": 74076
+  },
+  "sequence_length": 2048,
+  "shape": [
+    3072,
+    2048
+  ],
+  "source_shard": "/home/zs89/.cache/huggingface/hub/datasets--allenai--c4/blobs/8ef8d75b0e045dec4aa5123a671b4564466b0707086a7ed1ba8721626dfffbc9",
+  "source_shard_sha256": "8ef8d75b0e045dec4aa5123a671b4564466b0707086a7ed1ba8721626dfffbc9",
+  "split": "train",
+  "status": "frozen_and_audited",
+  "supervised_targets": 6288384,
+  "tensor_hash_schema": "sha256-contiguous-c-order-int64-little-endian-or-bool-bytes-v1",
+  "tensor_sha256": {
+    "attention_mask": "856a7d62e6bf8b1e8cd815bd086af8cd63158e81eeaacf2228e7179fe5f9dc7c",
+    "input_ids": "97111afc128374039ea9da0b8a96725bd4047831c87c5c0b14bfe4a2122ef50f",
+    "labels": "97111afc128374039ea9da0b8a96725bd4047831c87c5c0b14bfe4a2122ef50f"
+  },
+  "tokenizer": {
+    "add_special_tokens": false,
+    "eos_token_id": 128001,
+    "files_sha256": {
+      "special_tokens_map.json": "462d91939dbc37178aa5a3eae7068d1990ccc92e09f288cc71f42cdf139d69cc",
+      "tokenizer.json": "76e48799b099d43365bd24ccd8ecc5aedac831718da780552f03b0a6eb4412aa",
+      "tokenizer_config.json": "8004530facf809ac432114de2a4dcc65fcb632da5ec16d666091aeb6a2ee444a"
+    },
+    "pad_token_id": 128001,
+    "path": "/home/zs89/Basis/checkpoints/frozen_sources/llama31/dense/d04e592bb4f6aa9cfee91e2e20afa771667e1d4b"
+  },
+  "tokens_file_sha256": "98974aa8dc83d86a0fdac3b50b2cbab1b8b41420ccc072b28e35ef70693548ef",
+  "tokens_path": "/home/zs89/Basis/h1_exp/artifacts/joint_seed_robustness_20260911/data/seed_7712/c4/llama31/tokens.pt",
+  "unique_full_windows": 3072
+}

basis/data/c4/seed_7712/llama31/manifest.sha256

@@ -0,0 +1 @@
+d131bff67650471910131a97b62e8b3a8a5736051e8f447dd5b8e548883e6958  manifest.json

basis/data/c4/seed_7712/llama31/tokens.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:98974aa8dc83d86a0fdac3b50b2cbab1b8b41420ccc072b28e35ef70693548ef
+size 106956915

basis/data/c4/seed_7712/qwen3/manifest.json

@@ -0,0 +1,101 @@
+{
+  "backbone": "qwen3",
+  "corpus": "c4",
+  "dataset": "allenai/c4",
+  "dataset_recycling": false,
+  "documents_read": 76150,
+  "documents_selected": 2074,
+  "drop_last": false,
+  "effective_batch_size": 8,
+  "eos_appended": false,
+  "epoch_count": 1,
+  "final_effective_batch_sequences": 8,
+  "final_sequence_tokens": 2048,
+  "fixed_measurement_overlap": {
+    "exact_match": true,
+    "overlap_windows": 0,
+    "shingle_tokens": 32,
+    "training_windows_audited": 3072
+  },
+  "fixed_measurements": {
+    "evaluation": {
+      "artifact_sha256": "6d0d654ec4775bb250d458c7c5d2e4ab6060718cd588919d69d8d243fe3f886e",
+      "path": "/home/zs89/Basis/archives/20260910_recovery_protocol_correction/payload/h2_exp/artifacts/qwen3/evaluation.pt",
+      "shape": [
+        64,
+        256
+      ],
+      "tensor_sha256": "39776fef3736ac0e761aecc088db0cf858afa82ae56e8e2593b8f724f63e6abb"
+    },
+    "probe": {
+      "artifact_sha256": "aaafc696c0f30d57c385f5833e1583e6262da3847b87904923985734a1994b42",
+      "path": "/home/zs89/Basis/archives/20260910_recovery_protocol_correction/payload/h2_exp/artifacts/qwen3/probe.pt",
+      "shape": [
+        64,
+        256
+      ],
+      "tensor_sha256": "9d8f5bf750276d5509cc21632b77d298269445420739311c379aee46955aacac"
+    }
+  },
+  "input_positions": 6291456,
+  "measurement_overlap_audit": {
+    "evaluation_tensor_sha256": "39776fef3736ac0e761aecc088db0cf858afa82ae56e8e2593b8f724f63e6abb",
+    "overlap_windows": 0,
+    "passed": true,
+    "probe_tensor_sha256": "9d8f5bf750276d5509cc21632b77d298269445420739311c379aee46955aacac",
+    "shingle_tokens": 32
+  },
+  "nonoverlapping_source_spans": true,
+  "nonpadding_tokens": 6291456,
+  "num_sequences": 3072,
+  "objective": "causal LM on full unpadded C4 windows",
+  "optimizer_steps": 384,
+  "padded_tokens": 0,
+  "provenance_file_sha256": "3bbdd48d1b30e799e14267ca6a9b91f767b4d5b3f747efa5a61bf2ee9757c105",
+  "provenance_path": "/home/zs89/Basis/h1_exp/artifacts/joint_seed_robustness_20260911/data/seed_7712/c4/qwen3/provenance.jsonl",
+  "revision": "1588ec454efa1a09f29cd18ddd04fe05fc8653a2",
+  "same_document_chunk_candidates_across_backbones": true,
+  "save_steps": [
+    64,
+    128,
+    192,
+    256,
+    320,
+    384
+  ],
+  "schema": "joint-recovery-data-v1",
+  "seed": 7712,
+  "selection": "first 3072 admissible paired nonoverlapping within-document chunks in pinned shard row order; one shared seed shuffle after selection",
+  "selection_exclusions": {
+    "documents_short_in_at_least_one_tokenizer": 74076
+  },
+  "sequence_length": 2048,
+  "shape": [
+    3072,
+    2048
+  ],
+  "source_shard": "/home/zs89/.cache/huggingface/hub/datasets--allenai--c4/blobs/8ef8d75b0e045dec4aa5123a671b4564466b0707086a7ed1ba8721626dfffbc9",
+  "source_shard_sha256": "8ef8d75b0e045dec4aa5123a671b4564466b0707086a7ed1ba8721626dfffbc9",
+  "split": "train",
+  "status": "frozen_and_audited",
+  "supervised_targets": 6288384,
+  "tensor_hash_schema": "sha256-contiguous-c-order-int64-little-endian-or-bool-bytes-v1",
+  "tensor_sha256": {
+    "attention_mask": "856a7d62e6bf8b1e8cd815bd086af8cd63158e81eeaacf2228e7179fe5f9dc7c",
+    "input_ids": "9102ad1c76335ede73eefbefe66347b5231e6d69e0c2ad52c76e70fc46e5e378",
+    "labels": "9102ad1c76335ede73eefbefe66347b5231e6d69e0c2ad52c76e70fc46e5e378"
+  },
+  "tokenizer": {
+    "add_special_tokens": false,
+    "eos_token_id": 151643,
+    "files_sha256": {
+      "tokenizer.json": "c0382117ea329cdf097041132f6d735924b697924d6f6fc3945713e96ce87539",
+      "tokenizer_config.json": "3c04ed3ca964ea2f6b2b5faf0dc4d31aec1cb1e8b4bcf63f402d295046b422b5"
+    },
+    "pad_token_id": 151643,
+    "path": "/home/zs89/Basis/checkpoints/frozen_sources/qwen3/dense/49e3418fbbbca6ecbdf9608b4d22e5a407081db4"
+  },
+  "tokens_file_sha256": "7e5825c0eb13d29a78c524e335ed312da5e08ca12f47d48eda6f0f15bff9231f",
+  "tokens_path": "/home/zs89/Basis/h1_exp/artifacts/joint_seed_robustness_20260911/data/seed_7712/c4/qwen3/tokens.pt",
+  "unique_full_windows": 3072
+}

basis/data/c4/seed_7712/qwen3/manifest.sha256

@@ -0,0 +1 @@
+920d5a07f5d0759d8b9bab06a33ba08c11d4221b304db59cd0e6779ee4a5192d  manifest.json

basis/data/c4/seed_7712/qwen3/tokens.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:7e5825c0eb13d29a78c524e335ed312da5e08ca12f47d48eda6f0f15bff9231f
+size 106956915

basis/data/c4/seed_7713/llama31/manifest.json

@@ -0,0 +1,102 @@
+{
+  "backbone": "llama31",
+  "corpus": "c4",
+  "dataset": "allenai/c4",
+  "dataset_recycling": false,
+  "documents_read": 76150,
+  "documents_selected": 2074,
+  "drop_last": false,
+  "effective_batch_size": 8,
+  "eos_appended": false,
+  "epoch_count": 1,
+  "final_effective_batch_sequences": 8,
+  "final_sequence_tokens": 2048,
+  "fixed_measurement_overlap": {
+    "exact_match": true,
+    "overlap_windows": 0,
+    "shingle_tokens": 32,
+    "training_windows_audited": 3072
+  },
+  "fixed_measurements": {
+    "evaluation": {
+      "artifact_sha256": "5cb74049e7a5039fd32dcbf7581807431c4fd828d6ad5c09bda4a5ae02bf7813",
+      "path": "/home/zs89/Basis/archives/20260910_recovery_protocol_correction/payload/h2_exp/artifacts/llama31/evaluation.pt",
+      "shape": [
+        64,
+        256
+      ],
+      "tensor_sha256": "fb197f8496a8abec6b950c4a81971e3579ab571075da494b43a8aa3927b96c45"
+    },
+    "probe": {
+      "artifact_sha256": "386c2f19333c195c47cea4adc2a882f47615c8d526d5f903bc8911a36a851a9b",
+      "path": "/home/zs89/Basis/archives/20260910_recovery_protocol_correction/payload/h2_exp/artifacts/llama31/probe.pt",
+      "shape": [
+        64,
+        256
+      ],
+      "tensor_sha256": "117bcb58782b37ea6878eee9de789f11f64aacc90a0f1ceb41ef2bf07ebf9015"
+    }
+  },
+  "input_positions": 6291456,
+  "measurement_overlap_audit": {
+    "evaluation_tensor_sha256": "fb197f8496a8abec6b950c4a81971e3579ab571075da494b43a8aa3927b96c45",
+    "overlap_windows": 0,
+    "passed": true,
+    "probe_tensor_sha256": "117bcb58782b37ea6878eee9de789f11f64aacc90a0f1ceb41ef2bf07ebf9015",
+    "shingle_tokens": 32
+  },
+  "nonoverlapping_source_spans": true,
+  "nonpadding_tokens": 6291456,
+  "num_sequences": 3072,
+  "objective": "causal LM on full unpadded C4 windows",
+  "optimizer_steps": 384,
+  "padded_tokens": 0,
+  "provenance_file_sha256": "ac7fc3225306abf7672472f490646295a0270d71705344f23e24f9d6e492a302",
+  "provenance_path": "/home/zs89/Basis/h1_exp/artifacts/joint_seed_robustness_20260911/data/seed_7713/c4/llama31/provenance.jsonl",
+  "revision": "1588ec454efa1a09f29cd18ddd04fe05fc8653a2",
+  "same_document_chunk_candidates_across_backbones": true,
+  "save_steps": [
+    64,
+    128,
+    192,
+    256,
+    320,
+    384
+  ],
+  "schema": "joint-recovery-data-v1",
+  "seed": 7713,
+  "selection": "first 3072 admissible paired nonoverlapping within-document chunks in pinned shard row order; one shared seed shuffle after selection",
+  "selection_exclusions": {
+    "documents_short_in_at_least_one_tokenizer": 74076
+  },
+  "sequence_length": 2048,
+  "shape": [
+    3072,
+    2048
+  ],
+  "source_shard": "/home/zs89/.cache/huggingface/hub/datasets--allenai--c4/blobs/8ef8d75b0e045dec4aa5123a671b4564466b0707086a7ed1ba8721626dfffbc9",
+  "source_shard_sha256": "8ef8d75b0e045dec4aa5123a671b4564466b0707086a7ed1ba8721626dfffbc9",
+  "split": "train",
+  "status": "frozen_and_audited",
+  "supervised_targets": 6288384,
+  "tensor_hash_schema": "sha256-contiguous-c-order-int64-little-endian-or-bool-bytes-v1",
+  "tensor_sha256": {
+    "attention_mask": "856a7d62e6bf8b1e8cd815bd086af8cd63158e81eeaacf2228e7179fe5f9dc7c",
+    "input_ids": "6880e58e01bd1adf55df977396e2b06ca556dfd840c4e16da879ff94e37157ef",
+    "labels": "6880e58e01bd1adf55df977396e2b06ca556dfd840c4e16da879ff94e37157ef"
+  },
+  "tokenizer": {
+    "add_special_tokens": false,
+    "eos_token_id": 128001,
+    "files_sha256": {
+      "special_tokens_map.json": "462d91939dbc37178aa5a3eae7068d1990ccc92e09f288cc71f42cdf139d69cc",
+      "tokenizer.json": "76e48799b099d43365bd24ccd8ecc5aedac831718da780552f03b0a6eb4412aa",
+      "tokenizer_config.json": "8004530facf809ac432114de2a4dcc65fcb632da5ec16d666091aeb6a2ee444a"
+    },
+    "pad_token_id": 128001,
+    "path": "/home/zs89/Basis/checkpoints/frozen_sources/llama31/dense/d04e592bb4f6aa9cfee91e2e20afa771667e1d4b"
+  },
+  "tokens_file_sha256": "5a578c5aa7e674221b0d5e7a7e7789cc3e67e8ce3117760e0bbd4e5ef1354bf7",
+  "tokens_path": "/home/zs89/Basis/h1_exp/artifacts/joint_seed_robustness_20260911/data/seed_7713/c4/llama31/tokens.pt",
+  "unique_full_windows": 3072
+}

basis/data/c4/seed_7713/llama31/manifest.sha256

@@ -0,0 +1 @@
+da16188c35280956802c422a382be1620fa5d9afed1cce8a1ab1fc33d22c40ef  manifest.json

basis/data/c4/seed_7713/llama31/tokens.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5a578c5aa7e674221b0d5e7a7e7789cc3e67e8ce3117760e0bbd4e5ef1354bf7
+size 106956915

basis/data/c4/seed_7713/qwen3/manifest.json

@@ -0,0 +1,101 @@
+{
+  "backbone": "qwen3",
+  "corpus": "c4",
+  "dataset": "allenai/c4",
+  "dataset_recycling": false,
+  "documents_read": 76150,
+  "documents_selected": 2074,
+  "drop_last": false,
+  "effective_batch_size": 8,
+  "eos_appended": false,
+  "epoch_count": 1,
+  "final_effective_batch_sequences": 8,
+  "final_sequence_tokens": 2048,
+  "fixed_measurement_overlap": {
+    "exact_match": true,
+    "overlap_windows": 0,
+    "shingle_tokens": 32,
+    "training_windows_audited": 3072
+  },
+  "fixed_measurements": {
+    "evaluation": {
+      "artifact_sha256": "6d0d654ec4775bb250d458c7c5d2e4ab6060718cd588919d69d8d243fe3f886e",
+      "path": "/home/zs89/Basis/archives/20260910_recovery_protocol_correction/payload/h2_exp/artifacts/qwen3/evaluation.pt",
+      "shape": [
+        64,
+        256
+      ],
+      "tensor_sha256": "39776fef3736ac0e761aecc088db0cf858afa82ae56e8e2593b8f724f63e6abb"
+    },
+    "probe": {
+      "artifact_sha256": "aaafc696c0f30d57c385f5833e1583e6262da3847b87904923985734a1994b42",
+      "path": "/home/zs89/Basis/archives/20260910_recovery_protocol_correction/payload/h2_exp/artifacts/qwen3/probe.pt",
+      "shape": [
+        64,
+        256
+      ],
+      "tensor_sha256": "9d8f5bf750276d5509cc21632b77d298269445420739311c379aee46955aacac"
+    }
+  },
+  "input_positions": 6291456,
+  "measurement_overlap_audit": {
+    "evaluation_tensor_sha256": "39776fef3736ac0e761aecc088db0cf858afa82ae56e8e2593b8f724f63e6abb",
+    "overlap_windows": 0,
+    "passed": true,
+    "probe_tensor_sha256": "9d8f5bf750276d5509cc21632b77d298269445420739311c379aee46955aacac",
+    "shingle_tokens": 32
+  },
+  "nonoverlapping_source_spans": true,
+  "nonpadding_tokens": 6291456,
+  "num_sequences": 3072,
+  "objective": "causal LM on full unpadded C4 windows",
+  "optimizer_steps": 384,
+  "padded_tokens": 0,
+  "provenance_file_sha256": "ac7fc3225306abf7672472f490646295a0270d71705344f23e24f9d6e492a302",
+  "provenance_path": "/home/zs89/Basis/h1_exp/artifacts/joint_seed_robustness_20260911/data/seed_7713/c4/qwen3/provenance.jsonl",
+  "revision": "1588ec454efa1a09f29cd18ddd04fe05fc8653a2",
+  "same_document_chunk_candidates_across_backbones": true,
+  "save_steps": [
+    64,
+    128,
+    192,
+    256,
+    320,
+    384
+  ],
+  "schema": "joint-recovery-data-v1",
+  "seed": 7713,
+  "selection": "first 3072 admissible paired nonoverlapping within-document chunks in pinned shard row order; one shared seed shuffle after selection",
+  "selection_exclusions": {
+    "documents_short_in_at_least_one_tokenizer": 74076
+  },
+  "sequence_length": 2048,
+  "shape": [
+    3072,
+    2048
+  ],
+  "source_shard": "/home/zs89/.cache/huggingface/hub/datasets--allenai--c4/blobs/8ef8d75b0e045dec4aa5123a671b4564466b0707086a7ed1ba8721626dfffbc9",
+  "source_shard_sha256": "8ef8d75b0e045dec4aa5123a671b4564466b0707086a7ed1ba8721626dfffbc9",
+  "split": "train",
+  "status": "frozen_and_audited",
+  "supervised_targets": 6288384,
+  "tensor_hash_schema": "sha256-contiguous-c-order-int64-little-endian-or-bool-bytes-v1",
+  "tensor_sha256": {
+    "attention_mask": "856a7d62e6bf8b1e8cd815bd086af8cd63158e81eeaacf2228e7179fe5f9dc7c",
+    "input_ids": "b2975ce04e31d38a1ca8e975f5568e8b5ec2b4373b6d4456b02c9c8c64a7ffce",
+    "labels": "b2975ce04e31d38a1ca8e975f5568e8b5ec2b4373b6d4456b02c9c8c64a7ffce"
+  },
+  "tokenizer": {
+    "add_special_tokens": false,
+    "eos_token_id": 151643,
+    "files_sha256": {
+      "tokenizer.json": "c0382117ea329cdf097041132f6d735924b697924d6f6fc3945713e96ce87539",
+      "tokenizer_config.json": "3c04ed3ca964ea2f6b2b5faf0dc4d31aec1cb1e8b4bcf63f402d295046b422b5"
+    },
+    "pad_token_id": 151643,
+    "path": "/home/zs89/Basis/checkpoints/frozen_sources/qwen3/dense/49e3418fbbbca6ecbdf9608b4d22e5a407081db4"
+  },
+  "tokens_file_sha256": "98e7d7de6e39b82fc86ee931c76bb498c1435d1a023ea070e26d1174efa69ba7",
+  "tokens_path": "/home/zs89/Basis/h1_exp/artifacts/joint_seed_robustness_20260911/data/seed_7713/c4/qwen3/tokens.pt",
+  "unique_full_windows": 3072
+}

basis/data/c4/seed_7713/qwen3/manifest.sha256

@@ -0,0 +1 @@
+bafbc8fdb939103966ec20bd5a5e9dc9e7a47e548554d95b70098204d0ac87ac  manifest.json

basis/data/c4/seed_7713/qwen3/tokens.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:98e7d7de6e39b82fc86ee931c76bb498c1435d1a023ea070e26d1174efa69ba7
+size 106956915

basis/data/c4_probe_shard/c4-validation.00007-of-00008.json.gz

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:0bf87534c7050091c60466df50f174c3147d052ec7dc645da9574c902445beaf
+size 40446172

basis/data/probes/q128/llama31_evaluation.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:7801da597e5f2005d4e042126d2e04d3cc75b08d7919b699c67690877c1676d6
+size 263862

basis/data/probes/q128/llama31_probe.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b676f3fd028b2527df5582b9183f9317e9144db7b3d72a8e7e0ae0f1bc66281a
+size 263763

basis/data/probes/q128/manifest.json

@@ -0,0 +1,1976 @@
+{
+  "status": "frozen",
+  "protocol": {
+    "study_id": "hx_independent_confirmation_20260912",
+    "created_utc": "2026-09-12T16:45:08.227175+00:00",
+    "backbones": [
+      "llama31",
+      "qwen3"
+    ],
+    "methods": [
+      "asvd",
+      "svdllm",
+      "basis_sharing",
+      "modegpt",
+      "gfwsvd",
+      "swiftsvd"
+    ],
+    "retention": 0.8,
+    "recovery_seed": 7711,
+    "steps": [
+      0,
+      64,
+      384
+    ],
+    "probe_windows": 128,
+    "evaluation_windows": 128,
+    "window_length": 256,
+    "selection_seed": 2026091201,
+    "candidate_pool_size": 1024,
+    "minimum_source_row": 10000,
+    "shard": "/home/zs89/.cache/huggingface/hub/datasets--allenai--c4/snapshots/1588ec454efa1a09f29cd18ddd04fe05fc8653a2/en/c4-validation.00007-of-00008.json.gz",
+    "no_new_recovery_training": true,
+    "analyses": [
+      "Source magnitude/profile across new Q blocks",
+      "uniform energy scaling and paired bootstrap uncertainty",
+      "held-out H on new V",
+      "new-P Source-to-H associations",
+      "native BASIS implementation checks"
+    ],
+    "no_outcome_based_stopping": true,
+    "primary_new_readout": "all128; all eight contiguous16 blocks also reported",
+    "bootstrap_seed": 2026091202,
+    "bootstrap_replicates": 999
+  },
+  "data": {
+    "llama31": {
+      "probe": {
+        "path": "/home/zs89/Basis/hx_exp/artifacts/confirmation_data/llama31_probe.pt",
+        "shape": [
+          128,
+          256
+        ],
+        "tensor_sha256": "f19c31b43250bb8f5904e38874b21aca10fff6f736a76ada81e86121553f7e1b",
+        "artifact_sha256": "b676f3fd028b2527df5582b9183f9317e9144db7b3d72a8e7e0ae0f1bc66281a"
+      },
+      "evaluation": {
+        "path": "/home/zs89/Basis/hx_exp/artifacts/confirmation_data/llama31_evaluation.pt",
+        "shape": [
+          128,
+          256
+        ],
+        "tensor_sha256": "736e3a4086111f686021cff0b3beaf1cd1ae4feae52bfd6719cdb3680a9e400d",
+        "artifact_sha256": "7801da597e5f2005d4e042126d2e04d3cc75b08d7919b699c67690877c1676d6"
+      }
+    },
+    "qwen3": {
+      "probe": {
+        "path": "/home/zs89/Basis/hx_exp/artifacts/confirmation_data/qwen3_probe.pt",
+        "shape": [
+          128,
+          256
+        ],
+        "tensor_sha256": "0cfd945084a5b6adb291f206a37394a316b1300e49860297c5b30a6527a5f38e",
+        "artifact_sha256": "240a585a6764a10b59da60304e7ba88697d3335469f88054324909c216a06f14"
+      },
+      "evaluation": {
+        "path": "/home/zs89/Basis/hx_exp/artifacts/confirmation_data/qwen3_evaluation.pt",
+        "shape": [
+          128,
+          256
+        ],
+        "tensor_sha256": "5011899796a87c32c242e373011eea486b68fdd46d12ecc05966342bcb5b4037",
+        "artifact_sha256": "1bc7629428ba8896ee0710f562dbaca59df13181ed00f3e5549948dd5da47c48"
+      }
+    }
+  },
+  "tokenizers": {
+    "llama31": {
+      "path": "/home/zs89/.cache/huggingface/hub/models--meta-llama--Llama-3.1-8B/snapshots/d04e592bb4f6aa9cfee91e2e20afa771667e1d4b",
+      "files_sha256": {
+        "tokenizer.json": "76e48799b099d43365bd24ccd8ecc5aedac831718da780552f03b0a6eb4412aa",
+        "tokenizer_config.json": "8004530facf809ac432114de2a4dcc65fcb632da5ec16d666091aeb6a2ee444a",
+        "special_tokens_map.json": "462d91939dbc37178aa5a3eae7068d1990ccc92e09f288cc71f42cdf139d69cc"
+      },
+      "add_special_tokens": false,
+      "eos_token_id": 128001,
+      "pad_token_id": 128001
+    },
+    "qwen3": {
+      "path": "/home/zs89/.cache/huggingface/hub/models--Qwen--Qwen3-8B-Base/snapshots/49e3418fbbbca6ecbdf9608b4d22e5a407081db4",
+      "files_sha256": {
+        "tokenizer.json": "c0382117ea329cdf097041132f6d735924b697924d6f6fc3945713e96ce87539",
+        "tokenizer_config.json": "3c04ed3ca964ea2f6b2b5faf0dc4d31aec1cb1e8b4bcf63f402d295046b422b5"
+      },
+      "add_special_tokens": false,
+      "eos_token_id": 151643,
+      "pad_token_id": 151643
+    }
+  },
+  "selected_documents": [
+    {
+      "window": 0,
+      "partition": "probe",
+      "candidate_index": 571,
+      "source_row": 11099,
+      "text_sha256": "567e54e72a40945a406d33e2350c5f4032d5f16e5b7a2b77c4ce0d1a2fa9f70e"
+    },
+    {
+      "window": 1,
+      "partition": "probe",
+      "candidate_index": 704,
+      "source_row": 11367,
+      "text_sha256": "d5fa1985e18369f2d2798aeac394fdf52e6bbbe4ea17d592e22737d12e06042f"
+    },
+    {
+      "window": 2,
+      "partition": "probe",
+      "candidate_index": 626,
+      "source_row": 11220,
+      "text_sha256": "d440a7a7435769822b0a391efd8b8a185f61330aea2772465eda8e0ddf12fca6"
+    },
+    {
+      "window": 3,
+      "partition": "probe",
+      "candidate_index": 862,
+      "source_row": 11704,
+      "text_sha256": "2bc391f9a6d54c7e27224c84d882d20530061201263dd4776347ac61cba99673"
+    },
+    {
+      "window": 4,
+      "partition": "probe",
+      "candidate_index": 589,
+      "source_row": 11150,
+      "text_sha256": "7a3a8ddc176f84cbc145503e1927f8bedac6cabc4d7d61882f30333de08c4f41"
+    },
+    {
+      "window": 5,
+      "partition": "probe",
+      "candidate_index": 485,
+      "source_row": 10938,
+      "text_sha256": "9554ba1d75d146db4e497419a60fd086af562bbf69b3be3ed8d98f73e342fcbe"
+    },
+    {
+      "window": 6,
+      "partition": "probe",
+      "candidate_index": 556,
+      "source_row": 11065,
+      "text_sha256": "3e3f650c57e805790926390e625cd7e1cff13ba26661090fec87d426b15fcbb7"
+    },
+    {
+      "window": 7,
+      "partition": "probe",
+      "candidate_index": 114,
+      "source_row": 10210,
+      "text_sha256": "5b7f9eb15d805f5b1662341e813019acfecf554ca75013fa507896818217d9bc"
+    },
+    {
+      "window": 8,
+      "partition": "probe",
+      "candidate_index": 995,
+      "source_row": 11978,
+      "text_sha256": "2e6d848561fb4fdbf928260896a35ea97d594262e1219ccb7d3dfffc198f72c7"
+    },
+    {
+      "window": 9,
+      "partition": "probe",
+      "candidate_index": 413,
+      "source_row": 10816,
+      "text_sha256": "a582719f663169064fe3e32a4299e2eb9ed927c8fd574e48438d91a10a774c6e"
+    },
+    {
+      "window": 10,
+      "partition": "probe",
+      "candidate_index": 544,
+      "source_row": 11046,
+      "text_sha256": "c9a9e29b414843a00690412e7ee956dc8f795426dddff5eae70a58458260d3b9"
+    },
+    {
+      "window": 11,
+      "partition": "probe",
+      "candidate_index": 767,
+      "source_row": 11495,
+      "text_sha256": "751a12fa2273f1a0824fa331a7fb0ede1a8ecf773df7fbfb17e2eef080ed6f1e"
+    },
+    {
+      "window": 12,
+      "partition": "probe",
+      "candidate_index": 619,
+      "source_row": 11209,
+      "text_sha256": "39f78acface5d7709617f35d8404e3322228d3a13aad08a7bd8e63102b35e592"
+    },
+    {
+      "window": 13,
+      "partition": "probe",
+      "candidate_index": 982,
+      "source_row": 11956,
+      "text_sha256": "84ff2bbc16d8e4a95b358d9320975fef47432841ea027baa4490a35b554f9329"
+    },
+    {
+      "window": 14,
+      "partition": "probe",
+      "candidate_index": 760,
+      "source_row": 11486,
+      "text_sha256": "31e0e6a87ecdfc04b9be8a5f71cbaedaf24d62841bba020b7c544f20ee5d59e2"
+    },
+    {
+      "window": 15,
+      "partition": "probe",
+      "candidate_index": 32,
+      "source_row": 10057,
+      "text_sha256": "ffeb479e1d33728792764d4aa9092fe1a5fd4db73ed3ebb8f80d0fe51b446305"
+    },
+    {
+      "window": 16,
+      "partition": "probe",
+      "candidate_index": 762,
+      "source_row": 11488,
+      "text_sha256": "fe0393bdf37ca1491ae80925da0993fb4522543b22af77e152989c70675c1ac3"
+    },
+    {
+      "window": 17,
+      "partition": "probe",
+      "candidate_index": 416,
+      "source_row": 10825,
+      "text_sha256": "ad426f54ded5a9132371f6d50fa8a3d1063cfaae70c092b3162fb796a3ebdfec"
+    },
+    {
+      "window": 18,
+      "partition": "probe",
+      "candidate_index": 345,
+      "source_row": 10683,
+      "text_sha256": "b1afa35b09d3b312250eab1c749a352d1bb56e58af9a9c0f642e962565f71bfe"
+    },
+    {
+      "window": 19,
+      "partition": "probe",
+      "candidate_index": 38,
+      "source_row": 10071,
+      "text_sha256": "7403a2c76a4b357df8e720df43707bb59d7d84067755b3004b63b9aa61f45164"
+    },
+    {
+      "window": 20,
+      "partition": "probe",
+      "candidate_index": 427,
+      "source_row": 10841,
+      "text_sha256": "61cf8634f2744b040bb04c42c11af2a1e73ac6d711c1766c7a3bc1aea9667198"
+    },
+    {
+      "window": 21,
+      "partition": "probe",
+      "candidate_index": 246,
+      "source_row": 10465,
+      "text_sha256": "ed66d9d434be7f4753888d56ed73bfa7920b23dcca78c7703794214385255ef5"
+    },
+    {
+      "window": 22,
+      "partition": "probe",
+      "candidate_index": 457,
+      "source_row": 10888,
+      "text_sha256": "1bdb102c5ca9ea3ca85eed55a9ba01629d2bb6a1d2546ed0463511bc45bf9ffb"
+    },
+    {
+      "window": 23,
+      "partition": "probe",
+      "candidate_index": 939,
+      "source_row": 11853,
+      "text_sha256": "ba583d6eb33484045fb4f88f90a5fab15ece7bac5f88749470a8a9d6683733e3"
+    },
+    {
+      "window": 24,
+      "partition": "probe",
+      "candidate_index": 377,
+      "source_row": 10746,
+      "text_sha256": "46f956c75604fc38df8ddc1dae7da3241d1bb4bba92b9407c472d0de6da0cca8"
+    },
+    {
+      "window": 25,
+      "partition": "probe",
+      "candidate_index": 610,
+      "source_row": 11190,
+      "text_sha256": "cb194966e872736de4347ba4b40e475b32a6742b4266dd9d8597a1cd94169248"
+    },
+    {
+      "window": 26,
+      "partition": "probe",
+      "candidate_index": 793,
+      "source_row": 11554,
+      "text_sha256": "dbf6c0f338141c22372c77fe82f9cbdd3cde770bcab909d1b3cccac3e5356aa8"
+    },
+    {
+      "window": 27,
+      "partition": "probe",
+      "candidate_index": 821,
+      "source_row": 11616,
+      "text_sha256": "9848411098d1ef573fdd53f559f69a432a6794cc47b2bc96bc31e90ba862dcc3"
+    },
+    {
+      "window": 28,
+      "partition": "probe",
+      "candidate_index": 21,
+      "source_row": 10033,
+      "text_sha256": "edb1b77797f4d75f0f83532cc1dc9c7f5adf7459d635e5f365192510515709c7"
+    },
+    {
+      "window": 29,
+      "partition": "probe",
+      "candidate_index": 119,
+      "source_row": 10217,
+      "text_sha256": "daddec4af40958a2d8b9f29d7a68283aef48604cbd097b3506ce130001bba6d3"
+    },
+    {
+      "window": 30,
+      "partition": "probe",
+      "candidate_index": 107,
+      "source_row": 10202,
+      "text_sha256": "251a461afd4a9aef8c5077db92bfcec073639e674ddd50fce930c99e99c99cd4"
+    },
+    {
+      "window": 31,
+      "partition": "probe",
+      "candidate_index": 92,
+      "source_row": 10179,
+      "text_sha256": "0c6984b28d1fdb774dc5d1c8f7f786292e94f12804191b76dae6ed3bfb6d947b"
+    },
+    {
+      "window": 32,
+      "partition": "probe",
+      "candidate_index": 605,
+      "source_row": 11178,
+      "text_sha256": "9e7ebd62e6bdf561fdcb2c8a10e3b5600a66f27709e641c43f4994cf5afc6edb"
+    },
+    {
+      "window": 33,
+      "partition": "probe",
+      "candidate_index": 237,
+      "source_row": 10447,
+      "text_sha256": "4e5ff6579793fef1b12423641af2e9c3066ad7a8f20922853be7da43be3fee78"
+    },
+    {
+      "window": 34,
+      "partition": "probe",
+      "candidate_index": 819,
+      "source_row": 11614,
+      "text_sha256": "9567d72faf2aa042f4a744263240f36f0b631935510c6cacda3036ab935810ec"
+    },
+    {
+      "window": 35,
+      "partition": "probe",
+      "candidate_index": 1009,
+      "source_row": 12010,
+      "text_sha256": "d89886bb8916987dbe9b21b2c7e6378679d6b9fef1fa0a37f24469bce5437d97"
+    },
+    {
+      "window": 36,
+      "partition": "probe",
+      "candidate_index": 497,
+      "source_row": 10965,
+      "text_sha256": "2f84222d045f68283811132d72d91574bd92546354732fdd6159e5cf4a727596"
+    },
+    {
+      "window": 37,
+      "partition": "probe",
+      "candidate_index": 642,
+      "source_row": 11251,
+      "text_sha256": "ad21784528ba64356427af9c0e812ed03b7e4c6555dba0065eac87cb4113206a"
+    },
+    {
+      "window": 38,
+      "partition": "probe",
+      "candidate_index": 411,
+      "source_row": 10813,
+      "text_sha256": "0544bfc04db5b61aa5fba43f6a4ab2cb5feeeb781acd7f3ec1b33c854308015f"
+    },
+    {
+      "window": 39,
+      "partition": "probe",
+      "candidate_index": 981,
+      "source_row": 11955,
+      "text_sha256": "933103ef68bdecbced758cf520c53c5014d4e689c8578343fe62c96233f2bc71"
+    },
+    {
+      "window": 40,
+      "partition": "probe",
+      "candidate_index": 964,
+      "source_row": 11917,
+      "text_sha256": "998af0e2bf4cee978963c2e844c3abdec29fac01198bfa48bf9ea58ddaef1434"
+    },
+    {
+      "window": 41,
+      "partition": "probe",
+      "candidate_index": 900,
+      "source_row": 11788,
+      "text_sha256": "1f9bd1ab3f79b8b2a39d55ed99cd1d008c3b96dd31fb08a213c5f49b2fef735e"
+    },
+    {
+      "window": 42,
+      "partition": "probe",
+      "candidate_index": 11,
+      "source_row": 10015,
+      "text_sha256": "6088abaeed0bdd18d4aaea31ccccfd1dd8fd291532ace8a5d6c9a40e4b88b250"
+    },
+    {
+      "window": 43,
+      "partition": "probe",
+      "candidate_index": 546,
+      "source_row": 11048,
+      "text_sha256": "73fe6b53fd6a26f2a40d83b5296435a4c9e4496f30e8ffc2accf73c1eedf866a"
+    },
+    {
+      "window": 44,
+      "partition": "probe",
+      "candidate_index": 113,
+      "source_row": 10209,
+      "text_sha256": "859428ccf869af298ade1286c7d21cfa594fec06f68026cf9ccc200c9a35d8da"
+    },
+    {
+      "window": 45,
+      "partition": "probe",
+      "candidate_index": 790,
+      "source_row": 11542,
+      "text_sha256": "fa6061e3903cd901878e48ad9f99ff0c4766392d483e3f9c34e17615e252b833"
+    },
+    {
+      "window": 46,
+      "partition": "probe",
+      "candidate_index": 558,
+      "source_row": 11071,
+      "text_sha256": "d3fc95ab5ecc2e3466cff7b911ae0ace20ac3c7a625c5eb77548e1fe7040fa66"
+    },
+    {
+      "window": 47,
+      "partition": "probe",
+      "candidate_index": 499,
+      "source_row": 10970,
+      "text_sha256": "6c04d7939d50eb57d3dedb71d0eb7cd880e7dfdf7ffb13f1d33369c9b65507b9"
+    },
+    {
+      "window": 48,
+      "partition": "probe",
+      "candidate_index": 787,
+      "source_row": 11537,
+      "text_sha256": "9cce869f914ae16ccf184ac6dfb4c85f0b47d6d4519b1401002b91e608936b6b"
+    },
+    {
+      "window": 49,
+      "partition": "probe",
+      "candidate_index": 536,
+      "source_row": 11029,
+      "text_sha256": "a3f641fc34d0ef0b6d0110cca9e2bdc2eb3c19461eb4a6ae45f5370950da4f7c"
+    },
+    {
+      "window": 50,
+      "partition": "probe",
+      "candidate_index": 745,
+      "source_row": 11455,
+      "text_sha256": "98249f8c02b62f754c8e0796c6062ffd41d14f30f07b74230ceae323e2fc6a35"
+    },
+    {
+      "window": 51,
+      "partition": "probe",
+      "candidate_index": 328,
+      "source_row": 10645,
+      "text_sha256": "f3c853741d90f9942609ec5e3427ce477cd3a139eeae0764c28e564e57b22d52"
+    },
+    {
+      "window": 52,
+      "partition": "probe",
+      "candidate_index": 473,
+      "source_row": 10920,
+      "text_sha256": "6a598ee346a88aff72c8499849c47c3995f84d2648b641db24468e2cd8de8a45"
+    },
+    {
+      "window": 53,
+      "partition": "probe",
+      "candidate_index": 668,
+      "source_row": 11300,
+      "text_sha256": "089a04d390eb65a856133b148b7d9f051a97ad68722d3de146abc4a1c176dfcd"
+    },
+    {
+      "window": 54,
+      "partition": "probe",
+      "candidate_index": 740,
+      "source_row": 11445,
+      "text_sha256": "a1c747d3c6cee4731398ec0aa3494296ad59ad2041450530522449330f3d5b1f"
+    },
+    {
+      "window": 55,
+      "partition": "probe",
+      "candidate_index": 302,
+      "source_row": 10585,
+      "text_sha256": "b5561ed0dc880dbce836b0fab3795dcccd54e4bb332f699d1daad11724c8e694"
+    },
+    {
+      "window": 56,
+      "partition": "probe",
+      "candidate_index": 859,
+      "source_row": 11699,
+      "text_sha256": "bb2b8d8ef0399ee101a47ba5c74f0d59430cc658afd43f4004b2b4ff14f97066"
+    },
+    {
+      "window": 57,
+      "partition": "probe",
+      "candidate_index": 380,
+      "source_row": 10750,
+      "text_sha256": "11e416b61cece5d25387f653f523d360ee2497617cc24ec7cac31d64f8def52d"
+    },
+    {
+      "window": 58,
+      "partition": "probe",
+      "candidate_index": 483,
+      "source_row": 10936,
+      "text_sha256": "804e07e5e885164dc4dbad3209304eef81afbf85e6d77f30c6310835b16c0864"
+    },
+    {
+      "window": 59,
+      "partition": "probe",
+      "candidate_index": 428,
+      "source_row": 10842,
+      "text_sha256": "452caca72efc366446e3a2a5a3ee382548fdedf18b7db36cf42af35e9089ecf5"
+    },
+    {
+      "window": 60,
+      "partition": "probe",
+      "candidate_index": 207,
+      "source_row": 10394,
+      "text_sha256": "1df4bc8d7e2ff31cfd1c130fd9b461ac10c1f3f0524320f18cf17b821e8ae4b9"
+    },
+    {
+      "window": 61,
+      "partition": "probe",
+      "candidate_index": 607,
+      "source_row": 11181,
+      "text_sha256": "f50be97c4df31128c71c7d15ba57891c31ea18c35bc30179b91c16db4088442e"
+    },
+    {
+      "window": 62,
+      "partition": "probe",
+      "candidate_index": 727,
+      "source_row": 11409,
+      "text_sha256": "48e657c0d4085513293a9c7daa73e3cbd17cf8d64cff670ff460045903ba6874"
+    },
+    {
+      "window": 63,
+      "partition": "probe",
+      "candidate_index": 344,
+      "source_row": 10682,
+      "text_sha256": "bbcc359566c7083636794a9fbdfe05a9e6e672d312ca0d16338d3bf5dec0acd0"
+    },
+    {
+      "window": 64,
+      "partition": "probe",
+      "candidate_index": 636,
+      "source_row": 11239,
+      "text_sha256": "c997e4fec3305316f29dfd85104e2d7a3ae73eaa9efde7b4f967768a29ce46d1"
+    },
+    {
+      "window": 65,
+      "partition": "probe",
+      "candidate_index": 336,
+      "source_row": 10665,
+      "text_sha256": "7a3af45f3a26c05da97bd566e77ba69d297674e5950d6b8b417a63db90189981"
+    },
+    {
+      "window": 66,
+      "partition": "probe",
+      "candidate_index": 834,
+      "source_row": 11648,
+      "text_sha256": "c1292486d512ca318feb9bc78d2c2eb78e90e54325e58f015758e492f3582c66"
+    },
+    {
+      "window": 67,
+      "partition": "probe",
+      "candidate_index": 285,
+      "source_row": 10544,
+      "text_sha256": "4d1a610df17ad9ef638bb9f553f809afc6a8339edc19859682c60e0c4e62074a"
+    },
+    {
+      "window": 68,
+      "partition": "probe",
+      "candidate_index": 722,
+      "source_row": 11403,
+      "text_sha256": "000288937f36b961d450a1e75622b2779bb8cc99d7645547dbed541196d45298"
+    },
+    {
+      "window": 69,
+      "partition": "probe",
+      "candidate_index": 218,
+      "source_row": 10415,
+      "text_sha256": "df6fbb0c07ce1955332dbcd4578714d5951e92fb5ca9942e7d4256bfaa38298f"
+    },
+    {
+      "window": 70,
+      "partition": "probe",
+      "candidate_index": 679,
+      "source_row": 11318,
+      "text_sha256": "695ce980ae460bb8e886c5bea5f5e3860530ff7dbbced21ba653bf05fad87abb"
+    },
+    {
+      "window": 71,
+      "partition": "probe",
+      "candidate_index": 488,
+      "source_row": 10945,
+      "text_sha256": "f35d966d4f15cd1aeb812bc5c783669a9013f251a31c3dfe2f3706a2bc22b960"
+    },
+    {
+      "window": 72,
+      "partition": "probe",
+      "candidate_index": 988,
+      "source_row": 11963,
+      "text_sha256": "23a0e9bc6d9b6256fe8e14f5863444980ace297e1125bda50a50c0fa65e2d1df"
+    },
+    {
+      "window": 73,
+      "partition": "probe",
+      "candidate_index": 126,
+      "source_row": 10231,
+      "text_sha256": "68d3ed5c111e2d99db3b4a9359eb83ea7e205305db7e9ea8286a5d009642b6a8"
+    },
+    {
+      "window": 74,
+      "partition": "probe",
+      "candidate_index": 526,
+      "source_row": 11014,
+      "text_sha256": "54029cb2ad3c1047b9cea25f62f51e2c532f65d97bdc2c56c816eea685f85233"
+    },
+    {
+      "window": 75,
+      "partition": "probe",
+      "candidate_index": 889,
+      "source_row": 11761,
+      "text_sha256": "d6d8b84ef4cd7b48407d14f89211b473f2a2b847a3b649d3bde6d44cd0c20433"
+    },
+    {
+      "window": 76,
+      "partition": "probe",
+      "candidate_index": 258,
+      "source_row": 10499,
+      "text_sha256": "3387c8f43f926b4f5a7189a584c0c8e09139ca0390a4e8ca9a0da4327004c5cd"
+    },
+    {
+      "window": 77,
+      "partition": "probe",
+      "candidate_index": 801,
+      "source_row": 11569,
+      "text_sha256": "dc7173938b477ae4d0e6ed09b77d9d185bb53275014363d689455062a1877a4d"
+    },
+    {
+      "window": 78,
+      "partition": "probe",
+      "candidate_index": 83,
+      "source_row": 10168,
+      "text_sha256": "986c91b70ccfd7d48dc9a56a4d4e3c051927faa806a54f9cb7ce82226af126ee"
+    },
+    {
+      "window": 79,
+      "partition": "probe",
+      "candidate_index": 361,
+      "source_row": 10718,
+      "text_sha256": "5b54d8333e7ec66fe3bae5137ee22fc592aa4bad9bab3b5e97e6b458df99a874"
+    },
+    {
+      "window": 80,
+      "partition": "probe",
+      "candidate_index": 346,
+      "source_row": 10687,
+      "text_sha256": "192ebb4a4f08151485b722d7af06cc8770a158aebfca858a9cd5a9ab2328ccd4"
+    },
+    {
+      "window": 81,
+      "partition": "probe",
+      "candidate_index": 252,
+      "source_row": 10476,
+      "text_sha256": "62bc4aed9452e985e4deb247bc035902d99b56b7315a60f9d0822329c6304f42"
+    },
+    {
+      "window": 82,
+      "partition": "probe",
+      "candidate_index": 699,
+      "source_row": 11353,
+      "text_sha256": "6cad5c225aacc557fe9b0b3d6b7db903469a8e40c92707e0a3c9b108ca8486a7"
+    },
+    {
+      "window": 83,
+      "partition": "probe",
+      "candidate_index": 868,
+      "source_row": 11714,
+      "text_sha256": "6e5ed9f76a561d1f218e3bd89aac3147688608f0ec2da9d104761c57bbbd551e"
+    },
+    {
+      "window": 84,
+      "partition": "probe",
+      "candidate_index": 25,
+      "source_row": 10041,
+      "text_sha256": "1969ed2f999794d9a003bbc68d7f5a8405afa99a42c9daa51e862fcc43d8ebd2"
+    },
+    {
+      "window": 85,
+      "partition": "probe",
+      "candidate_index": 84,
+      "source_row": 10169,
+      "text_sha256": "9e9f1a321539e8889653e725dfb3b0ccb0b1f5335d769cd8371e18c2b6e5ef12"
+    },
+    {
+      "window": 86,
+      "partition": "probe",
+      "candidate_index": 330,
+      "source_row": 10647,
+      "text_sha256": "087ef9758757cba2a94c181d622a75a80d29f8488fa66faf4a17699c3a0b2e9d"
+    },
+    {
+      "window": 87,
+      "partition": "probe",
+      "candidate_index": 549,
+      "source_row": 11054,
+      "text_sha256": "12c3aa817890ec7a0ddf2953c244eb182aafa4ca2bf1aca208c5b51295c4ee4d"
+    },
+    {
+      "window": 88,
+      "partition": "probe",
+      "candidate_index": 17,
+      "source_row": 10026,
+      "text_sha256": "0f99a3883fbcd47aee5289a8f97561e4b98d51d72e9862e36001ed587c8584d4"
+    },
+    {
+      "window": 89,
+      "partition": "probe",
+      "candidate_index": 291,
+      "source_row": 10559,
+      "text_sha256": "e52c8e1e4b049822f5538f0b42a4dda0790989805dbe33f9d1b652e846f43d34"
+    },
+    {
+      "window": 90,
+      "partition": "probe",
+      "candidate_index": 460,
+      "source_row": 10895,
+      "text_sha256": "9149f51e799c9d2ee80d4991e3742ef0687a2eddbae374beff5792c57ca6b0c3"
+    },
+    {
+      "window": 91,
+      "partition": "probe",
+      "candidate_index": 383,
+      "source_row": 10759,
+      "text_sha256": "18c6a380f22a3cdaf0041d7f6616335f3a2d020ec6a8d23175750168e597f49d"
+    },
+    {
+      "window": 92,
+      "partition": "probe",
+      "candidate_index": 612,
+      "source_row": 11193,
+      "text_sha256": "cc234e73d909245e2c70e8fad59a2cbbacf8e62789f8fff86ba03148e6337006"
+    },
+    {
+      "window": 93,
+      "partition": "probe",
+      "candidate_index": 942,
+      "source_row": 11868,
+      "text_sha256": "6f2dd6e72ab5c15a621f85a5239916cc37aa7219f017f75bb00fab16afa9bfeb"
+    },
+    {
+      "window": 94,
+      "partition": "probe",
+      "candidate_index": 205,
+      "source_row": 10383,
+      "text_sha256": "659f56c55eb09213a5d88ba7070c424ea8bc64907d60f1fb9a8ad5f43b47f9ba"
+    },
+    {
+      "window": 95,
+      "partition": "probe",
+      "candidate_index": 951,
+      "source_row": 11885,
+      "text_sha256": "9b70f7ef43827ed26adcab21c71572d581ebc57cf28b2b4b72ed57471d1bdb52"
+    },
+    {
+      "window": 96,
+      "partition": "probe",
+      "candidate_index": 323,
+      "source_row": 10630,
+      "text_sha256": "0d3e92d345b656906b3229085e83879b8749441f2fbe485c4b6569fe3dae5182"
+    },
+    {
+      "window": 97,
+      "partition": "probe",
+      "candidate_index": 493,
+      "source_row": 10954,
+      "text_sha256": "a769abb2d410a13d644ed02070835ea0f15137cd59315fe0c84319f268c49706"
+    },
+    {
+      "window": 98,
+      "partition": "probe",
+      "candidate_index": 509,
+      "source_row": 10984,
+      "text_sha256": "d33a631bcb0fb4a5d270a522c8e3af9dc8412524cadc8f466040faea467057d0"
+    },
+    {
+      "window": 99,
+      "partition": "probe",
+      "candidate_index": 104,
+      "source_row": 10197,
+      "text_sha256": "1f1472bb875c09db04790bbecbd84baa65b930b758e3bff2c302c86f89645c38"
+    },
+    {
+      "window": 100,
+      "partition": "probe",
+      "candidate_index": 743,
+      "source_row": 11453,
+      "text_sha256": "3a68c972725ce52b952c2624f2383b01a105c074178f1e358887f7875bb905a9"
+    },
+    {
+      "window": 101,
+      "partition": "probe",
+      "candidate_index": 406,
+      "source_row": 10804,
+      "text_sha256": "5b08e98424d75eceb794233b5b60e27ab756f8d34b2404b3d59a75e8a9a83982"
+    },
+    {
+      "window": 102,
+      "partition": "probe",
+      "candidate_index": 375,
+      "source_row": 10744,
+      "text_sha256": "37497193d54ccdfc569f8cc5e1b05942b660d139d90207a4c3f4d6c7c0f9dcea"
+    },
+    {
+      "window": 103,
+      "partition": "probe",
+      "candidate_index": 907,
+      "source_row": 11797,
+      "text_sha256": "f423ef77a55b860d104b40abb0a6250a1445e364859c15f44ab2aabb6b1aa90a"
+    },
+    {
+      "window": 104,
+      "partition": "probe",
+      "candidate_index": 753,
+      "source_row": 11473,
+      "text_sha256": "ad3ca77cf7462de785c6b3483c061cfe1a49c6bb506bd8383d8ab46532614c7e"
+    },
+    {
+      "window": 105,
+      "partition": "probe",
+      "candidate_index": 98,
+      "source_row": 10188,
+      "text_sha256": "e108f245e0a52a5ef65c0c418c467ea068a4f72bdb463e57cbb4a8c0054ad8c2"
+    },
+    {
+      "window": 106,
+      "partition": "probe",
+      "candidate_index": 358,
+      "source_row": 10712,
+      "text_sha256": "3e062746a3de850e194bc640d54958f60a322bfdef7b7ef3aa9de35dcfb1522b"
+    },
+    {
+      "window": 107,
+      "partition": "probe",
+      "candidate_index": 193,
+      "source_row": 10363,
+      "text_sha256": "3e13689da7ad48975d80041880104cc070d35d3514ea083c21db856f0f487a57"
+    },
+    {
+      "window": 108,
+      "partition": "probe",
+      "candidate_index": 266,
+      "source_row": 10510,
+      "text_sha256": "f8d5b9a8c7503e00db0c109703bcc8585c5b3482306af92de8912dad8a7c68c2"
+    },
+    {
+      "window": 109,
+      "partition": "probe",
+      "candidate_index": 649,
+      "source_row": 11262,
+      "text_sha256": "34c671ef9eb03c0589026fb491e62517f2ca2f4f410cc74386d3d14a05faad36"
+    },
+    {
+      "window": 110,
+      "partition": "probe",
+      "candidate_index": 89,
+      "source_row": 10175,
+      "text_sha256": "357ebf1af84b7496f34bd877eae20db3be43d05471ac204ff307bad3746ee7da"
+    },
+    {
+      "window": 111,
+      "partition": "probe",
+      "candidate_index": 439,
+      "source_row": 10856,
+      "text_sha256": "b4bafb930329f179ee40bcfc686dde3ea39660615040959b8733a434a4e7c22e"
+    },
+    {
+      "window": 112,
+      "partition": "probe",
+      "candidate_index": 35,
+      "source_row": 10061,
+      "text_sha256": "def8cc2385507839c29d739d14ff305332f4d2557bc2fa3158bf5a24312aeebd"
+    },
+    {
+      "window": 113,
+      "partition": "probe",
+      "candidate_index": 43,
+      "source_row": 10083,
+      "text_sha256": "0f0d92fcfa3fc14f4f39fd35f4b61424c1478364edfde695ef476a8d1cc63638"
+    },
+    {
+      "window": 114,
+      "partition": "probe",
+      "candidate_index": 354,
+      "source_row": 10703,
+      "text_sha256": "c630be8b77081575c30eaab9a21d3e0b3bccc07f838dcf6c5675426802857098"
+    },
+    {
+      "window": 115,
+      "partition": "probe",
+      "candidate_index": 891,
+      "source_row": 11765,
+      "text_sha256": "bc64b886f84f2dff083dd7c3e0ba1eeb68b2caac9268837805780741884e862e"
+    },
+    {
+      "window": 116,
+      "partition": "probe",
+      "candidate_index": 553,
+      "source_row": 11059,
+      "text_sha256": "94d6df5a7a19df682c23bd566de736dffb82cb27dd6867b11eaafb8580ab7b8e"
+    },
+    {
+      "window": 117,
+      "partition": "probe",
+      "candidate_index": 415,
+      "source_row": 10823,
+      "text_sha256": "d152156760f9cd25aed6b6951b134d2344ace2a33f5ab71fa6edb06cb26ef194"
+    },
+    {
+      "window": 118,
+      "partition": "probe",
+      "candidate_index": 912,
+      "source_row": 11810,
+      "text_sha256": "b16b7327342fe3b7477525ed9ef35aa1ddd8fe73497ed1d353033dfaf60674b2"
+    },
+    {
+      "window": 119,
+      "partition": "probe",
+      "candidate_index": 140,
+      "source_row": 10259,
+      "text_sha256": "a145669acc17d6663294fb2562526fd2de9e9ab24fce21bd929910220432cf76"
+    },
+    {
+      "window": 120,
+      "partition": "probe",
+      "candidate_index": 48,
+      "source_row": 10090,
+      "text_sha256": "71d435277f462935a8239b5c54bc9cef7e90b53fe2e1867647c9d95db3a38319"
+    },
+    {
+      "window": 121,
+      "partition": "probe",
+      "candidate_index": 34,
+      "source_row": 10059,
+      "text_sha256": "78e52c6ac6a4d7b7b1fbd51c64a9f5c16ef87975eba4d3133fe7c3d9d8321419"
+    },
+    {
+      "window": 122,
+      "partition": "probe",
+      "candidate_index": 272,
+      "source_row": 10520,
+      "text_sha256": "afcde444671cde4f80c6a5b1deda0d5c1deba3d6c272a6f13da661a68b728b5e"
+    },
+    {
+      "window": 123,
+      "partition": "probe",
+      "candidate_index": 728,
+      "source_row": 11411,
+      "text_sha256": "ff669f1fb147e7c9eb38efb31a539af740597d85d1659286eb040773ec76bc6f"
+    },
+    {
+      "window": 124,
+      "partition": "probe",
+      "candidate_index": 418,
+      "source_row": 10827,
+      "text_sha256": "937c618999d065d0f20ce3a48882e2d28a7bc74b73fade5ad502eba1b4044c61"
+    },
+    {
+      "window": 125,
+      "partition": "probe",
+      "candidate_index": 85,
+      "source_row": 10170,
+      "text_sha256": "eea6a485f45e6e8ec92b031dbcd8bfaec6fca83a1a6c8c2dc8df274b7129c94d"
+    },
+    {
+      "window": 126,
+      "partition": "probe",
+      "candidate_index": 318,
+      "source_row": 10619,
+      "text_sha256": "beda0cf4ac83d2e969ce3cc87fbb5490340bc9ba3898b9d6fc12fa112d3b6365"
+    },
+    {
+      "window": 127,
+      "partition": "probe",
+      "candidate_index": 294,
+      "source_row": 10563,
+      "text_sha256": "5b3012d0465709dd85773028793cf4fc0143bb7b0ec75a8cec5e11012972050a"
+    },
+    {
+      "window": 128,
+      "partition": "evaluation",
+      "candidate_index": 666,
+      "source_row": 11298,
+      "text_sha256": "34dbac0f8b95d24356ade006dad8ade20ba49dd4a50be0fc2760f1150027e4fd"
+    },
+    {
+      "window": 129,
+      "partition": "evaluation",
+      "candidate_index": 1020,
+      "source_row": 12047,
+      "text_sha256": "b44be99f4103a53c1f69b45dc9fd749fa399b9e33518079d6058158d819fe83b"
+    },
+    {
+      "window": 130,
+      "partition": "evaluation",
+      "candidate_index": 72,
+      "source_row": 10148,
+      "text_sha256": "5b452d047f278871b1629f5bea3b2889827d4a4d656872cf08fdc094b760c393"
+    },
+    {
+      "window": 131,
+      "partition": "evaluation",
+      "candidate_index": 22,
+      "source_row": 10036,
+      "text_sha256": "4cbcd1dea09ec039a1f915fa15089f5592c817fc5cfac071a507e86aa2668a87"
+    },
+    {
+      "window": 132,
+      "partition": "evaluation",
+      "candidate_index": 716,
+      "source_row": 11389,
+      "text_sha256": "e06ce7ff5a9afb837e21bb08967c84318dc5724981dd2c3828b0788a9235feb1"
+    },
+    {
+      "window": 133,
+      "partition": "evaluation",
+      "candidate_index": 435,
+      "source_row": 10850,
+      "text_sha256": "6c053247bf8e33c81475bc4f35d588b8f67552fe913dfee9e64a1b0ca974292c"
+    },
+    {
+      "window": 134,
+      "partition": "evaluation",
+      "candidate_index": 138,
+      "source_row": 10252,
+      "text_sha256": "86f3cfed6a4bc7b4e5e4b8dfe2e8fd9b2af1871149f972097c5980736d367b7b"
+    },
+    {
+      "window": 135,
+      "partition": "evaluation",
+      "candidate_index": 910,
+      "source_row": 11806,
+      "text_sha256": "78a9b92bbb1684d804f487a2a7f0a6d2131b46d9ab9f8acd6cb2848da384aa5f"
+    },
+    {
+      "window": 136,
+      "partition": "evaluation",
+      "candidate_index": 698,
+      "source_row": 11352,
+      "text_sha256": "2f79d8103e5618384a5db391342258ad062e026c6cb6b97bdd71b148c3a00f9e"
+    },
+    {
+      "window": 137,
+      "partition": "evaluation",
+      "candidate_index": 749,
+      "source_row": 11461,
+      "text_sha256": "2d7239662ec75b139369309bc016b5ee5a0f143621bc9d6f6cd66636526bb6b0"
+    },
+    {
+      "window": 138,
+      "partition": "evaluation",
+      "candidate_index": 796,
+      "source_row": 11558,
+      "text_sha256": "793da0657458ebe25dda4d18bb6fc4e4a817cc5975fe05ebd81db518bc8be800"
+    },
+    {
+      "window": 139,
+      "partition": "evaluation",
+      "candidate_index": 866,
+      "source_row": 11712,
+      "text_sha256": "94b96d8e3d80e8e5b48b76e1dc8cc9b34811cd21a61d3679d42a4735af748bea"
+    },
+    {
+      "window": 140,
+      "partition": "evaluation",
+      "candidate_index": 894,
+      "source_row": 11775,
+      "text_sha256": "ab7e29a43c4419dc79758462353890f79fe165fe131cca774b11bfca2c9c68b8"
+    },
+    {
+      "window": 141,
+      "partition": "evaluation",
+      "candidate_index": 102,
+      "source_row": 10195,
+      "text_sha256": "7c0b9e166fbede84e5c773a6bcbe1f1bca44fdc03ea5ec864580408d0045bea6"
+    },
+    {
+      "window": 142,
+      "partition": "evaluation",
+      "candidate_index": 436,
+      "source_row": 10851,
+      "text_sha256": "80cbf501091ab59b9c8c5772b7d0e09241c51c476836b249a7a29eba7c1533bb"
+    },
+    {
+      "window": 143,
+      "partition": "evaluation",
+      "candidate_index": 937,
+      "source_row": 11851,
+      "text_sha256": "51aed63f54974de47373c544eccdb3d888c2fdd5377ecd445538b09a6aa372f0"
+    },
+    {
+      "window": 144,
+      "partition": "evaluation",
+      "candidate_index": 802,
+      "source_row": 11574,
+      "text_sha256": "32654fe05ba788c3e01f050794ce99bb72cbd30ac7bbb01516d578a8de61ecbd"
+    },
+    {
+      "window": 145,
+      "partition": "evaluation",
+      "candidate_index": 381,
+      "source_row": 10751,
+      "text_sha256": "5ce681f3ea61acd284cad5ed026e5024c1e81bfacc4d942a3f5aed80b79602cc"
+    },
+    {
+      "window": 146,
+      "partition": "evaluation",
+      "candidate_index": 655,
+      "source_row": 11276,
+      "text_sha256": "d8a39d4053841b841ec9fd629ebe02754d788ada7aeec452fef3d62db7d309a6"
+    },
+    {
+      "window": 147,
+      "partition": "evaluation",
+      "candidate_index": 877,
+      "source_row": 11732,
+      "text_sha256": "ab7e9312b72b4a806298321159af4a927bd961b07c851902e9e939cb18e0963d"
+    },
+    {
+      "window": 148,
+      "partition": "evaluation",
+      "candidate_index": 989,
+      "source_row": 11966,
+      "text_sha256": "1f3347f2d296d90400ec327e4e9cd6bdb0e6ca2e605b7a2172fe4e50e52fbadd"
+    },
+    {
+      "window": 149,
+      "partition": "evaluation",
+      "candidate_index": 788,
+      "source_row": 11539,
+      "text_sha256": "c143746dd12042c682898615b35d07a927d93d220340ccab80280b8f00ad6f1e"
+    },
+    {
+      "window": 150,
+      "partition": "evaluation",
+      "candidate_index": 644,
+      "source_row": 11254,
+      "text_sha256": "39f7bc1be4ae69e2a9753af6b52a9acce86f18ab2c28bf469bf12b6940f86599"
+    },
+    {
+      "window": 151,
+      "partition": "evaluation",
+      "candidate_index": 1019,
+      "source_row": 12043,
+      "text_sha256": "b2933a81630cd91690547da25934fb7ac55e28afd345521674c80041a0e80c53"
+    },
+    {
+      "window": 152,
+      "partition": "evaluation",
+      "candidate_index": 1007,
+      "source_row": 12007,
+      "text_sha256": "9e7b771d4b342fd58d44eccfb2dfd181df3894d91ecc5c4f03a9b72e7ae9e8b3"
+    },
+    {
+      "window": 153,
+      "partition": "evaluation",
+      "candidate_index": 160,
+      "source_row": 10297,
+      "text_sha256": "4559b151a2673bb8f50502dfa42de6452c8ea8de7e04924b19626ea58dc10206"
+    },
+    {
+      "window": 154,
+      "partition": "evaluation",
+      "candidate_index": 66,
+      "source_row": 10133,
+      "text_sha256": "13f773d8dbebc600d8850c848eff9b4ff93b11200f9f796d847ca94ca028f288"
+    },
+    {
+      "window": 155,
+      "partition": "evaluation",
+      "candidate_index": 391,
+      "source_row": 10775,
+      "text_sha256": "413b5fee52c561c91fdfb78fcd278ac9b21bd81728ed2631b66d01ecfb03d894"
+    },
+    {
+      "window": 156,
+      "partition": "evaluation",
+      "candidate_index": 407,
+      "source_row": 10806,
+      "text_sha256": "5c65a4341d8769704768e0c528871c0207c5de85ad53e28dfa0017049e72776e"
+    },
+    {
+      "window": 157,
+      "partition": "evaluation",
+      "candidate_index": 658,
+      "source_row": 11283,
+      "text_sha256": "f9ebdcadb8b40e7f3eb30dff7d6eff52d2fe286492dd22c3c7482f42a1c7ebae"
+    },
+    {
+      "window": 158,
+      "partition": "evaluation",
+      "candidate_index": 973,
+      "source_row": 11938,
+      "text_sha256": "f94992207daf79583edc83fc4c7d90da99e26f96f52d86b24fed851d5fee0619"
+    },
+    {
+      "window": 159,
+      "partition": "evaluation",
+      "candidate_index": 899,
+      "source_row": 11787,
+      "text_sha256": "5f816c9399ecdb9d23273ba9b421e19dc8401c5e5dac852dbe1ad384de1c2fc8"
+    },
+    {
+      "window": 160,
+      "partition": "evaluation",
+      "candidate_index": 103,
+      "source_row": 10196,
+      "text_sha256": "063182d602b99a828a8e8125e172a5bb7625be0ecfae22aec110732e6b5bb71d"
+    },
+    {
+      "window": 161,
+      "partition": "evaluation",
+      "candidate_index": 448,
+      "source_row": 10873,
+      "text_sha256": "0183ab4c821341e83ed9a22bbd3122dbb0c7908739d4d95b0d39459b1c942cc8"
+    },
+    {
+      "window": 162,
+      "partition": "evaluation",
+      "candidate_index": 919,
+      "source_row": 11823,
+      "text_sha256": "154720a19a666a7728f97cdbd9cbf049e8f773d4d7a7f5ebb1330ec184618473"
+    },
+    {
+      "window": 163,
+      "partition": "evaluation",
+      "candidate_index": 901,
+      "source_row": 11789,
+      "text_sha256": "84362b8623175dbfd6603f9e7ad202bfd334c053f7dc3b5b6ff792ab51012ac6"
+    },
+    {
+      "window": 164,
+      "partition": "evaluation",
+      "candidate_index": 306,
+      "source_row": 10599,
+      "text_sha256": "93e71f5689a40094065ee8874b6144d33c8549805239e0caf050cc59a7e68f7d"
+    },
+    {
+      "window": 165,
+      "partition": "evaluation",
+      "candidate_index": 197,
+      "source_row": 10371,
+      "text_sha256": "b6fb782b402133ecd7bedc09228399d4f7fef368e036513b1aaec2bdb0ac3b63"
+    },
+    {
+      "window": 166,
+      "partition": "evaluation",
+      "candidate_index": 446,
+      "source_row": 10867,
+      "text_sha256": "9f1a0bfef6a9c5f13c7e80751f3354d01f50723ffd2da3656b2a5d30ac48f749"
+    },
+    {
+      "window": 167,
+      "partition": "evaluation",
+      "candidate_index": 686,
+      "source_row": 11329,
+      "text_sha256": "efd4083a01e21e68760021352ffaf57c4a0e27bf8c44ead0caaea6e8d6e7303d"
+    },
+    {
+      "window": 168,
+      "partition": "evaluation",
+      "candidate_index": 587,
+      "source_row": 11148,
+      "text_sha256": "9e85ac6af0c0515f7ab4257c438ecc8aeba9de3c4ed3d6f41fad5cc65a14b0dd"
+    },
+    {
+      "window": 169,
+      "partition": "evaluation",
+      "candidate_index": 872,
+      "source_row": 11721,
+      "text_sha256": "af408960cfc685309cce2fd89d13da8a446fd6480bddeebedcefb2c0a12c00ea"
+    },
+    {
+      "window": 170,
+      "partition": "evaluation",
+      "candidate_index": 927,
+      "source_row": 11838,
+      "text_sha256": "da49df36fbc672887281ce7c26ad753fe63f0df1529528e963906a4ca9a90b9b"
+    },
+    {
+      "window": 171,
+      "partition": "evaluation",
+      "candidate_index": 840,
+      "source_row": 11660,
+      "text_sha256": "e4925f294fb51b6ba88080fde67c52d7a697423e238a58b403be30bf898c81de"
+    },
+    {
+      "window": 172,
+      "partition": "evaluation",
+      "candidate_index": 471,
+      "source_row": 10917,
+      "text_sha256": "352e4b8140875a34540baf92ec82e59644b6171663622bf93dd8e2f4b23c4826"
+    },
+    {
+      "window": 173,
+      "partition": "evaluation",
+      "candidate_index": 339,
+      "source_row": 10670,
+      "text_sha256": "7dbc60b91e916b5fcbcca2ac4b48d7d5d665b1bf9baa5de4610055565b985534"
+    },
+    {
+      "window": 174,
+      "partition": "evaluation",
+      "candidate_index": 423,
+      "source_row": 10834,
+      "text_sha256": "6e737b59aad5755fb4ea9b1fe95f3966b3b13238acf94ac68ed6f6d4d4f82a1b"
+    },
+    {
+      "window": 175,
+      "partition": "evaluation",
+      "candidate_index": 807,
+      "source_row": 11588,
+      "text_sha256": "ce99c8f9c72edc1a912605f310c0f74ff650af7ae66e8edcee8d4b4abd724850"
+    },
+    {
+      "window": 176,
+      "partition": "evaluation",
+      "candidate_index": 487,
+      "source_row": 10944,
+      "text_sha256": "21ce899e4eec8b5af2f58099ebbc3abd4e7ded6278fe633b2cb73f2616ba9371"
+    },
+    {
+      "window": 177,
+      "partition": "evaluation",
+      "candidate_index": 268,
+      "source_row": 10512,
+      "text_sha256": "c6564486b6212b3848a191c509f2219bc4f0c9b198bb660e8d8a862eba406e2d"
+    },
+    {
+      "window": 178,
+      "partition": "evaluation",
+      "candidate_index": 674,
+      "source_row": 11309,
+      "text_sha256": "98efe134a8a0f8c36c50b49fc164ada4e3da6ac5a08e212cbff3e111065bf22f"
+    },
+    {
+      "window": 179,
+      "partition": "evaluation",
+      "candidate_index": 627,
+      "source_row": 11224,
+      "text_sha256": "25b122d0032ca7798ef738d8ae013605cce78f7063ed812495e3a55509156a60"
+    },
+    {
+      "window": 180,
+      "partition": "evaluation",
+      "candidate_index": 581,
+      "source_row": 11128,
+      "text_sha256": "bf77acdc53c5ff57d2a2cb65e917ddd98f175ca7ff28327506d26a4c8dc530e8"
+    },
+    {
+      "window": 181,
+      "partition": "evaluation",
+      "candidate_index": 878,
+      "source_row": 11733,
+      "text_sha256": "44e41210cfeaa68466885ec745afb03abd3f8583f12a6fb1160eaef07eacf706"
+    },
+    {
+      "window": 182,
+      "partition": "evaluation",
+      "candidate_index": 432,
+      "source_row": 10846,
+      "text_sha256": "a1d8719f05de1f9bc850092c84b0a05a70c0a39820a7a023ecddca490106c0e7"
+    },
+    {
+      "window": 183,
+      "partition": "evaluation",
+      "candidate_index": 1004,
+      "source_row": 11995,
+      "text_sha256": "896b43a6a1824bf3cb7bf45206191ef10827d588303c1c3879c6608b3274edc0"
+    },
+    {
+      "window": 184,
+      "partition": "evaluation",
+      "candidate_index": 597,
+      "source_row": 11165,
+      "text_sha256": "c0b9e59bf272b13b007f8d42831c33e723865bb1e250e71a1044cb2ab335c3da"
+    },
+    {
+      "window": 185,
+      "partition": "evaluation",
+      "candidate_index": 804,
+      "source_row": 11579,
+      "text_sha256": "307b9c5f5b741fd02fafbf694291c6fc24b97ba6e9415b4d92ebb2b176bfdcd6"
+    },
+    {
+      "window": 186,
+      "partition": "evaluation",
+      "candidate_index": 127,
+      "source_row": 10233,
+      "text_sha256": "4858803959c1c0732f5a256814f6c210408f68b63b80b27611a37f9a285865f6"
+    },
+    {
+      "window": 187,
+      "partition": "evaluation",
+      "candidate_index": 565,
+      "source_row": 11083,
+      "text_sha256": "1f984996aa976c1022543c1c3634c5de9b62f31f0ef9170b41c8c34383db7b14"
+    },
+    {
+      "window": 188,
+      "partition": "evaluation",
+      "candidate_index": 535,
+      "source_row": 11028,
+      "text_sha256": "e38058c1aaeeec6d58999c522eb9534b5d4eb982d284c4f4622072a17d51be4a"
+    },
+    {
+      "window": 189,
+      "partition": "evaluation",
+      "candidate_index": 280,
+      "source_row": 10533,
+      "text_sha256": "f2dbeb6c1b374b590c63df50d72723a34290e4130c4b8bbb4c3b7c7a5a780c68"
+    },
+    {
+      "window": 190,
+      "partition": "evaluation",
+      "candidate_index": 135,
+      "source_row": 10248,
+      "text_sha256": "2b8c155a84eab632fc34146914c975a77cadcda9a0abec83de79419265bdd902"
+    },
+    {
+      "window": 191,
+      "partition": "evaluation",
+      "candidate_index": 469,
+      "source_row": 10914,
+      "text_sha256": "8759bffc480c20c5a9675d86b33546c2d376462b35a16c7fb5749104a7385d22"
+    },
+    {
+      "window": 192,
+      "partition": "evaluation",
+      "candidate_index": 566,
+      "source_row": 11088,
+      "text_sha256": "3690e3444ecc8391c3f2aac882873ce2e5e636e72fff555ccb1cba2ebec5bf04"
+    },
+    {
+      "window": 193,
+      "partition": "evaluation",
+      "candidate_index": 695,
+      "source_row": 11346,
+      "text_sha256": "495b8217015297cb282fc6e9005e359b11a6a6e6f42993854c07798112954a06"
+    },
+    {
+      "window": 194,
+      "partition": "evaluation",
+      "candidate_index": 747,
+      "source_row": 11458,
+      "text_sha256": "4e023bd24366cc68d21f841c8f9b1d31838cb0c2e2371dadb59410906e8b955e"
+    },
+    {
+      "window": 195,
+      "partition": "evaluation",
+      "candidate_index": 598,
+      "source_row": 11167,
+      "text_sha256": "a80fb36e40bca99eb031b13b7edeba01cee4d1582b516e47b205a94422518ea3"
+    },
+    {
+      "window": 196,
+      "partition": "evaluation",
+      "candidate_index": 321,
+      "source_row": 10628,
+      "text_sha256": "609474602bb3a783d3c722809520580358c115930d48ab9656f3a9a894728c2a"
+    },
+    {
+      "window": 197,
+      "partition": "evaluation",
+      "candidate_index": 671,
+      "source_row": 11306,
+      "text_sha256": "55a29e81a7526f470e906e0e22f341c1a1baac804fd6a95dff2203abdfc53a83"
+    },
+    {
+      "window": 198,
+      "partition": "evaluation",
+      "candidate_index": 86,
+      "source_row": 10171,
+      "text_sha256": "16d780d3887de55f645bb4554e50b792c7451b40d87514ba7e7162d233861311"
+    },
+    {
+      "window": 199,
+      "partition": "evaluation",
+      "candidate_index": 993,
+      "source_row": 11973,
+      "text_sha256": "96288901ef173ec300499408c26a25f6ce7d68b346018f9d00150e86e4067a0c"
+    },
+    {
+      "window": 200,
+      "partition": "evaluation",
+      "candidate_index": 822,
+      "source_row": 11622,
+      "text_sha256": "199389c44f51f9f118d19b3685b738436278defa81b8ca809e6fc8507a7effc0"
+    },
+    {
+      "window": 201,
+      "partition": "evaluation",
+      "candidate_index": 734,
+      "source_row": 11424,
+      "text_sha256": "dca785d18f1064669d30b5d20c5d1619b4e3551633727cee2cf00835232f7820"
+    },
+    {
+      "window": 202,
+      "partition": "evaluation",
+      "candidate_index": 506,
+      "source_row": 10981,
+      "text_sha256": "b92aa94c55eec19b1838886f60e35fbe38683b774f6a26b1dcdd856b40f338b9"
+    },
+    {
+      "window": 203,
+      "partition": "evaluation",
+      "candidate_index": 170,
+      "source_row": 10318,
+      "text_sha256": "7491799cc2631e44c386548f3c35f756efa26b89f51dd40b6bce52ccb52a3e4d"
+    },
+    {
+      "window": 204,
+      "partition": "evaluation",
+      "candidate_index": 784,
+      "source_row": 11533,
+      "text_sha256": "fc09aebd5d653af875223e9470aa1b85b15f0b69e5cb21f99998d6c11bce155d"
+    },
+    {
+      "window": 205,
+      "partition": "evaluation",
+      "candidate_index": 831,
+      "source_row": 11641,
+      "text_sha256": "72bdde219c2dda3ff15f36ef7ae924236e7881163e00f33caa134bafd73a55ea"
+    },
+    {
+      "window": 206,
+      "partition": "evaluation",
+      "candidate_index": 836,
+      "source_row": 11651,
+      "text_sha256": "b4fd997a286b9bf06d040c840ce3f8eada206e5cc04fb05e302fdc2a4e4e3150"
+    },
+    {
+      "window": 207,
+      "partition": "evaluation",
+      "candidate_index": 366,
+      "source_row": 10727,
+      "text_sha256": "9e33b80e2db8d1210a08401c51d5a72075665f056db11f00615d102649198707"
+    },
+    {
+      "window": 208,
+      "partition": "evaluation",
+      "candidate_index": 295,
+      "source_row": 10565,
+      "text_sha256": "b6a5d36fdcb6e757650dfc6260a403b225c85534bee6df4d09190cd72260bdb2"
+    },
+    {
+      "window": 209,
+      "partition": "evaluation",
+      "candidate_index": 188,
+      "source_row": 10351,
+      "text_sha256": "7a50502f9ebed9ed250443c7c4129d9f18623b4987cad0df8fd8f6ee8907b1ca"
+    },
+    {
+      "window": 210,
+      "partition": "evaluation",
+      "candidate_index": 244,
+      "source_row": 10461,
+      "text_sha256": "5584d70c5237ca4c3d4cd4b4829f53e7f776643d83dcffa049fca05196544b7d"
+    },
+    {
+      "window": 211,
+      "partition": "evaluation",
+      "candidate_index": 552,
+      "source_row": 11058,
+      "text_sha256": "e23c921aebab824266478b9e8d933a32b32f08ab0a3baad70ad8eab184202e20"
+    },
+    {
+      "window": 212,
+      "partition": "evaluation",
+      "candidate_index": 736,
+      "source_row": 11433,
+      "text_sha256": "a435d6ff32c6ae13c9a63d95b03121786e52feb22a37b2982dbf674926db8814"
+    },
+    {
+      "window": 213,
+      "partition": "evaluation",
+      "candidate_index": 884,
+      "source_row": 11748,
+      "text_sha256": "2f1bc84d599f42142870d64d40f91e0fa516eb691258ce472e6429081b66f835"
+    },
+    {
+      "window": 214,
+      "partition": "evaluation",
+      "candidate_index": 482,
+      "source_row": 10933,
+      "text_sha256": "e45b616dc30ae2a50ff74214778dcadc02b833317ca0c6552baa37fc2b0833cb"
+    },
+    {
+      "window": 215,
+      "partition": "evaluation",
+      "candidate_index": 265,
+      "source_row": 10508,
+      "text_sha256": "021d03470d6ba599361d0b396afdd57a2aea3e73a23ae6a81d11635cc86e9f15"
+    },
+    {
+      "window": 216,
+      "partition": "evaluation",
+      "candidate_index": 440,
+      "source_row": 10859,
+      "text_sha256": "f396af611113df98715185f8688de6f41f40dc4a51df28a1e578e4c9c258c1a2"
+    },
+    {
+      "window": 217,
+      "partition": "evaluation",
+      "candidate_index": 925,
+      "source_row": 11834,
+      "text_sha256": "e5680c0598660396e5c570453071e83ec70fd2f824a3e7921ccbce5b575b8acb"
+    },
+    {
+      "window": 218,
+      "partition": "evaluation",
+      "candidate_index": 953,
+      "source_row": 11889,
+      "text_sha256": "2a09f90879c7065b6566dd1f0f9774c7756bbaf4b71267bdf406ee4d4c7ca733"
+    },
+    {
+      "window": 219,
+      "partition": "evaluation",
+      "candidate_index": 676,
+      "source_row": 11312,
+      "text_sha256": "159364896867d83e22f213990c78c0392b9dbf7ae2e52e639df012e7defb7e55"
+    },
+    {
+      "window": 220,
+      "partition": "evaluation",
+      "candidate_index": 688,
+      "source_row": 11333,
+      "text_sha256": "ac8124f1891b9e0d1604754ba354c47492ce2a9bae302d0e28982d75ca1c626b"
+    },
+    {
+      "window": 221,
+      "partition": "evaluation",
+      "candidate_index": 388,
+      "source_row": 10772,
+      "text_sha256": "40ba65aa4aef08d99bb4e74bc414cd32e73c6d3cfc881e8e87e827abd51117e9"
+    },
+    {
+      "window": 222,
+      "partition": "evaluation",
+      "candidate_index": 930,
+      "source_row": 11841,
+      "text_sha256": "62c995b5001d903ac94f21942efedb63c68754106826eaa493e8cae429bb0c27"
+    },
+    {
+      "window": 223,
+      "partition": "evaluation",
+      "candidate_index": 152,
+      "source_row": 10284,
+      "text_sha256": "90c6d43ce25d379a3855e51f81e471469a21498ccd4403528a82b59d510b00cb"
+    },
+    {
+      "window": 224,
+      "partition": "evaluation",
+      "candidate_index": 729,
+      "source_row": 11415,
+      "text_sha256": "934e0c1aa0f0afa019edb0b9e0f84f854be6414f1fff259d0247a7cddd49d3eb"
+    },
+    {
+      "window": 225,
+      "partition": "evaluation",
+      "candidate_index": 350,
+      "source_row": 10694,
+      "text_sha256": "e079b4533b6a7ec9072ba639891e190e2c33bfd5f8c2bbd62e2e9c07c180f843"
+    },
+    {
+      "window": 226,
+      "partition": "evaluation",
+      "candidate_index": 706,
+      "source_row": 11369,
+      "text_sha256": "147cd9c6bd1d0cbdfa9eea27dafa843f3d25cb59ec16ff23eab530f64cadcfa4"
+    },
+    {
+      "window": 227,
+      "partition": "evaluation",
+      "candidate_index": 758,
+      "source_row": 11478,
+      "text_sha256": "0071ca64aa78237e34d3ff6f28ffc474235bf2b3f2c339f68e15dc832bf5be7c"
+    },
+    {
+      "window": 228,
+      "partition": "evaluation",
+      "candidate_index": 297,
+      "source_row": 10573,
+      "text_sha256": "245afe82fb9fca241c621d30ee82a2a4cd17d7c24b6125dce423b721f7d31a88"
+    },
+    {
+      "window": 229,
+      "partition": "evaluation",
+      "candidate_index": 545,
+      "source_row": 11047,
+      "text_sha256": "6cd0aa164619315b67d461455a8f7e0af0e20eac5d7abae1b61eef5eb45803ab"
+    },
+    {
+      "window": 230,
+      "partition": "evaluation",
+      "candidate_index": 235,
+      "source_row": 10443,
+      "text_sha256": "3cda1e7a6ae1b04d7f40613a723c42e86054a622014f1737d9e7a812800d0392"
+    },
+    {
+      "window": 231,
+      "partition": "evaluation",
+      "candidate_index": 789,
+      "source_row": 11540,
+      "text_sha256": "05f7d607581fd8d83395071e4e1d0f905e336568233ed014606b43e33471635c"
+    },
+    {
+      "window": 232,
+      "partition": "evaluation",
+      "candidate_index": 165,
+      "source_row": 10308,
+      "text_sha256": "cec11a35e2aa0146868e72bc2ef76c4c43018235cc2c8563fc4973afe26bf7f2"
+    },
+    {
+      "window": 233,
+      "partition": "evaluation",
+      "candidate_index": 858,
+      "source_row": 11697,
+      "text_sha256": "7cc077685056335d6a34ec6a857294597e16be82b2fddfd0fb52f46e38aa7d02"
+    },
+    {
+      "window": 234,
+      "partition": "evaluation",
+      "candidate_index": 79,
+      "source_row": 10160,
+      "text_sha256": "26b42b8cadf4474d03de63fbc2972c378ccf4cb67a6baa408b5ade1bdf57f3ad"
+    },
+    {
+      "window": 235,
+      "partition": "evaluation",
+      "candidate_index": 465,
+      "source_row": 10906,
+      "text_sha256": "3442a6b2b76f8e35677efda0c3f7f5d0edcbbe868db3091d86ffc0b1b90ddab3"
+    },
+    {
+      "window": 236,
+      "partition": "evaluation",
+      "candidate_index": 783,
+      "source_row": 11532,
+      "text_sha256": "4d2be24847012079516c253c175494948e269cc0deb37e3567247e5ec356f6a4"
+    },
+    {
+      "window": 237,
+      "partition": "evaluation",
+      "candidate_index": 240,
+      "source_row": 10451,
+      "text_sha256": "63238b8a23421ab2909360c12d4a93ef840dc24ce1c31aed3d475f635f536aed"
+    },
+    {
+      "window": 238,
+      "partition": "evaluation",
+      "candidate_index": 665,
+      "source_row": 11294,
+      "text_sha256": "a3e3a742eb9b3566cff43712984299196b66f34980892372d3c27e1063d47711"
+    },
+    {
+      "window": 239,
+      "partition": "evaluation",
+      "candidate_index": 799,
+      "source_row": 11562,
+      "text_sha256": "d6ee457a24c75286f9fa2f9f9fa8fee4e265cb1507ac2278b646af805c42e3c2"
+    },
+    {
+      "window": 240,
+      "partition": "evaluation",
+      "candidate_index": 892,
+      "source_row": 11772,
+      "text_sha256": "655b21dca769950b250adc3d47764583543dc02cf5a70b975b680da2be30b1ad"
+    },
+    {
+      "window": 241,
+      "partition": "evaluation",
+      "candidate_index": 936,
+      "source_row": 11850,
+      "text_sha256": "a93fac3a24b452b5ebb5ec3adef4c86688e560a90d182beba0af034007c9ddf8"
+    },
+    {
+      "window": 242,
+      "partition": "evaluation",
+      "candidate_index": 37,
+      "source_row": 10070,
+      "text_sha256": "17538ed83793b278dc81903235e2a8ebb22efaab3ee97fbdb3c904564644df07"
+    },
+    {
+      "window": 243,
+      "partition": "evaluation",
+      "candidate_index": 592,
+      "source_row": 11154,
+      "text_sha256": "d7461a280775ad41a43a0358f7ac24f43794cceeece4ff85c0d4aeff86b18424"
+    },
+    {
+      "window": 244,
+      "partition": "evaluation",
+      "candidate_index": 362,
+      "source_row": 10719,
+      "text_sha256": "7b75f437040e9f152e069b24b16b228c7a9960d2b6ee740649761c8fb7140125"
+    },
+    {
+      "window": 245,
+      "partition": "evaluation",
+      "candidate_index": 780,
+      "source_row": 11519,
+      "text_sha256": "510c842892e0a46f40c347f1b2344f6b2dcfe2ee9b28a3d1ea91fb9b281e7cb6"
+    },
+    {
+      "window": 246,
+      "partition": "evaluation",
+      "candidate_index": 455,
+      "source_row": 10884,
+      "text_sha256": "b6c57a261463b5677f5e3b91695a9e539ae9d52cdd879db53276826a44970ea6"
+    },
+    {
+      "window": 247,
+      "partition": "evaluation",
+      "candidate_index": 131,
+      "source_row": 10243,
+      "text_sha256": "cb987531c978f277bebf7b9fb80a265283097fcf65ae2259d232678db9d29c91"
+    },
+    {
+      "window": 248,
+      "partition": "evaluation",
+      "candidate_index": 584,
+      "source_row": 11141,
+      "text_sha256": "309307b5acdc6c61e13a8568b608ac42b1fc6b0cd207e4d80c6c687faf502494"
+    },
+    {
+      "window": 249,
+      "partition": "evaluation",
+      "candidate_index": 228,
+      "source_row": 10433,
+      "text_sha256": "29e29068a20f9c6a10323abf9699ddad2c165c5dd87572ba8dfff7aef230ea01"
+    },
+    {
+      "window": 250,
+      "partition": "evaluation",
+      "candidate_index": 87,
+      "source_row": 10172,
+      "text_sha256": "ea208539f639e4ba2e1f2da46fa1cc0eec38ac54e82cc94eb6ed01d29dfcde70"
+    },
+    {
+      "window": 251,
+      "partition": "evaluation",
+      "candidate_index": 167,
+      "source_row": 10311,
+      "text_sha256": "694b8b061d57ab9126ef6ed59d733c66a9e440725c9db21f93e1f0887c9dccb9"
+    },
+    {
+      "window": 252,
+      "partition": "evaluation",
+      "candidate_index": 748,
+      "source_row": 11460,
+      "text_sha256": "a426082cefa4188e7bd2da177c4afbb59fc281219aa371d6d5621ed81b305706"
+    },
+    {
+      "window": 253,
+      "partition": "evaluation",
+      "candidate_index": 735,
+      "source_row": 11430,
+      "text_sha256": "c9511366ab28339868079694d5b8e313f33d1907cc5830d2dcf289582e1be2a2"
+    },
+    {
+      "window": 254,
+      "partition": "evaluation",
+      "candidate_index": 29,
+      "source_row": 10050,
+      "text_sha256": "98b4c275372fb302d14195858e664d3e7c501f489bc75a4cea407369da1168aa"
+    },
+    {
+      "window": 255,
+      "partition": "evaluation",
+      "candidate_index": 590,
+      "source_row": 11151,
+      "text_sha256": "98829f5c625564da1cc8987ce06be932bab9c169507622ed50550abf5006b6db"
+    }
+  ],
+  "excluded_old_document_count": 2202,
+  "training_old_probe_overlap_rejected": [
+    125,
+    782
+  ],
+  "within_new_data_overlap_rejected": [],
+  "audits": {
+    "llama31": {
+      "forbidden_artifacts": [
+        {
+          "path": "/home/zs89/Basis/h1_exp/artifacts/joint_single_seed_20260910/data/c4/llama31/tokens.pt",
+          "artifact_sha256": "7dd056c4836f5e4c38f3e9fc0fc746d987aaf3b0e2b06dca634867e45a871ee7",
+          "role": "recovery_train"
+        },
+        {
+          "path": "/home/zs89/Basis/archives/20260910_recovery_protocol_correction/payload/h2_exp/artifacts/llama31/probe.pt",
+          "shape": [
+            64,
+            256
+          ],
+          "tensor_sha256": "117bcb58782b37ea6878eee9de789f11f64aacc90a0f1ceb41ef2bf07ebf9015",
+          "artifact_sha256": "386c2f19333c195c47cea4adc2a882f47615c8d526d5f903bc8911a36a851a9b",
+          "role": "old_probe"
+        },
+        {
+          "path": "/home/zs89/Basis/archives/20260910_recovery_protocol_correction/payload/h2_exp/artifacts/llama31/evaluation.pt",
+          "shape": [
+            64,
+            256
+          ],
+          "tensor_sha256": "fb197f8496a8abec6b950c4a81971e3579ab571075da494b43a8aa3927b96c45",
+          "artifact_sha256": "5cb74049e7a5039fd32dcbf7581807431c4fd828d6ad5c09bda4a5ae02bf7813",
+          "role": "old_evaluation"
+        }
+      ],
+      "candidate_overlap_count": 2,
+      "exact_shingle_tokens": 32
+    },
+    "qwen3": {
+      "forbidden_artifacts": [
+        {
+          "path": "/home/zs89/Basis/h1_exp/artifacts/joint_single_seed_20260910/data/c4/qwen3/tokens.pt",
+          "artifact_sha256": "52f32bfe7d634c39e18f3f0c7242def4229564b29094ac95376fe3cc0df18519",
+          "role": "recovery_train"
+        },
+        {
+          "path": "/home/zs89/Basis/archives/20260910_recovery_protocol_correction/payload/h2_exp/artifacts/qwen3/probe.pt",
+          "shape": [
+            64,
+            256
+          ],
+          "tensor_sha256": "9d8f5bf750276d5509cc21632b77d298269445420739311c379aee46955aacac",
+          "artifact_sha256": "aaafc696c0f30d57c385f5833e1583e6262da3847b87904923985734a1994b42",
+          "role": "old_probe"
+        },
+        {
+          "path": "/home/zs89/Basis/archives/20260910_recovery_protocol_correction/payload/h2_exp/artifacts/qwen3/evaluation.pt",
+          "shape": [
+            64,
+            256
+          ],
+          "tensor_sha256": "39776fef3736ac0e761aecc088db0cf858afa82ae56e8e2593b8f724f63e6abb",
+          "artifact_sha256": "6d0d654ec4775bb250d458c7c5d2e4ab6060718cd588919d69d8d243fe3f886e",
+          "role": "old_evaluation"
+        }
+      ],
+      "candidate_overlap_count": 2,
+      "exact_shingle_tokens": 32
+    }
+  },
+  "probe_evaluation_document_overlap": 0,
+  "limitations": "Shared held-out shard, new documents/windows. Exact 32-token separation from current C4 training and old Q/V verified. No claim of universal pretraining-data or all historical compression-calibration separation."
+}

basis/data/probes/q128/qwen3_evaluation.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:1bc7629428ba8896ee0710f562dbaca59df13181ed00f3e5549948dd5da47c48
+size 263848

basis/data/probes/q128/qwen3_probe.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:240a585a6764a10b59da60304e7ba88697d3335469f88054324909c216a06f14
+size 263749

basis/data/probes/q64/llama31/evaluation.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5cb74049e7a5039fd32dcbf7581807431c4fd828d6ad5c09bda4a5ae02bf7813
+size 263834

basis/data/probes/q64/llama31/probe.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:386c2f19333c195c47cea4adc2a882f47615c8d526d5f903bc8911a36a851a9b
+size 263735

basis/data/probes/q64/manifest.json

@@ -0,0 +1,875 @@
+{
+  "status": "frozen",
+  "protocol_sha256": "71e34d0f1573aaa84cd889e983fd388c5e818915fe3da6ee9fab21e4d9decf09",
+  "code_commit": "e48cae0ea8e8226390d829ff1387eaabcf4279eb",
+  "dataset": "allenai/c4",
+  "dataset_revision": "1588ec454efa1a09f29cd18ddd04fe05fc8653a2",
+  "dataset_file": "en/c4-validation.00007-of-00008.json.gz",
+  "dataset_file_sha256": "0bf87534c7050091c60466df50f174c3147d052ec7dc645da9574c902445beaf",
+  "pool_size": 1024,
+  "selection_seed": 20260910,
+  "selected_documents": [
+    {
+      "window": 0,
+      "partition": "probe0",
+      "source_row": 73,
+      "text_sha256": "fa58e4569506be5a48409cc161a82075000615e914c3032297e0dd1799205cd7"
+    },
+    {
+      "window": 1,
+      "partition": "probe0",
+      "source_row": 216,
+      "text_sha256": "43343785332214f39d49018869ed878cfc4afa331f649f60221a0d3c5be90721"
+    },
+    {
+      "window": 2,
+      "partition": "probe0",
+      "source_row": 862,
+      "text_sha256": "0a302e5a5ef6d3a5d39a259ab030450e61bb1d77eabda290e22f2688e4e87c97"
+    },
+    {
+      "window": 3,
+      "partition": "probe0",
+      "source_row": 1653,
+      "text_sha256": "d8a9a4feebf046e4c4071a05af770b2a17daaba84b9a0335ba7a3a6793ca29ac"
+    },
+    {
+      "window": 4,
+      "partition": "probe0",
+      "source_row": 1539,
+      "text_sha256": "8a5c0fd9a1f8600ef2c0c6bc12722745ad6c16a6aea69448c9672daeca0c1285"
+    },
+    {
+      "window": 5,
+      "partition": "probe0",
+      "source_row": 1561,
+      "text_sha256": "8d6282472b061194d28e30fef7c5c7b318019fc11ce74faa6988a4e90884eba7"
+    },
+    {
+      "window": 6,
+      "partition": "probe0",
+      "source_row": 1731,
+      "text_sha256": "ea374e536f317b3718b2a8a4f1a4264202563d049760bdbe1f4b8146dfd0cd65"
+    },
+    {
+      "window": 7,
+      "partition": "probe0",
+      "source_row": 556,
+      "text_sha256": "8023a52529f741db4f6eb1e7d64e1c8c65050bedb22e9446d43a26c0c3cfa017"
+    },
+    {
+      "window": 8,
+      "partition": "probe0",
+      "source_row": 1366,
+      "text_sha256": "83447e7d9e5714533caef2d50d500e44b400a5dae44d4922a9fb077e5a30696f"
+    },
+    {
+      "window": 9,
+      "partition": "probe0",
+      "source_row": 881,
+      "text_sha256": "f9c09f897476374e7fe1ebd1552146c815e6633236e00da7701584a47eedc75f"
+    },
+    {
+      "window": 10,
+      "partition": "probe0",
+      "source_row": 918,
+      "text_sha256": "c831b4293f0a37a29ce1c394d687df5ac976b6f22b2eeccf5656aac03a03ec62"
+    },
+    {
+      "window": 11,
+      "partition": "probe0",
+      "source_row": 745,
+      "text_sha256": "d08b198c127f7b96d202eb695a5038c55a306126dfb196a81edbc32a922d6fe8"
+    },
+    {
+      "window": 12,
+      "partition": "probe0",
+      "source_row": 641,
+      "text_sha256": "5d89196eb70c56eb92b8e84aab7e9f3efd81a4a89d9c1ea42cdcaa67887165b4"
+    },
+    {
+      "window": 13,
+      "partition": "probe0",
+      "source_row": 601,
+      "text_sha256": "3beec659048724f4b42a12179de23449ffdfd006e669f17182e145656e477c82"
+    },
+    {
+      "window": 14,
+      "partition": "probe0",
+      "source_row": 822,
+      "text_sha256": "271ee03e6e9ebd79c4e85f17a30569345eff7001ad847569d7cac7b66753ed3d"
+    },
+    {
+      "window": 15,
+      "partition": "probe0",
+      "source_row": 170,
+      "text_sha256": "895029a702d220de14ad07ffe6ef46b9e5451c8811d5a200e38e832523770921"
+    },
+    {
+      "window": 16,
+      "partition": "probe1",
+      "source_row": 2144,
+      "text_sha256": "05b5c5f12880cf73a434627105e0458e8108c0a57f87804fb684b0cbdeed9f34"
+    },
+    {
+      "window": 17,
+      "partition": "probe1",
+      "source_row": 1903,
+      "text_sha256": "68dd2eb8e9062a16d582fbf75182189c25ce6f8cdd504aed955b195bf35fa885"
+    },
+    {
+      "window": 18,
+      "partition": "probe1",
+      "source_row": 933,
+      "text_sha256": "a0b9088cef0f4c53daaec2f4b825bf6da2a8606433edae4eaa98c3f304b4b18e"
+    },
+    {
+      "window": 19,
+      "partition": "probe1",
+      "source_row": 2010,
+      "text_sha256": "7ca6c128d228ecc3368a0db3618ec9d4aef4870be81eb83bc038da20325a2d0d"
+    },
+    {
+      "window": 20,
+      "partition": "probe1",
+      "source_row": 78,
+      "text_sha256": "59cf1b53fbe6d903b8c8ee9ecd408aca6c596b99fe03e951e16c923ff0941d2b"
+    },
+    {
+      "window": 21,
+      "partition": "probe1",
+      "source_row": 986,
+      "text_sha256": "c89f5f980f0691cf4bfb95e888d6245bed090899390cb1aff63f573e318ac579"
+    },
+    {
+      "window": 22,
+      "partition": "probe1",
+      "source_row": 2009,
+      "text_sha256": "bfdf9545ff9861d83c52a2245d8259def9e2fa4df7ded44464a595275c9b1eff"
+    },
+    {
+      "window": 23,
+      "partition": "probe1",
+      "source_row": 674,
+      "text_sha256": "c6e7bd319cbd7d506ea774d2b45e7a6ffd564a6aabdc606fbbcba284c601ea10"
+    },
+    {
+      "window": 24,
+      "partition": "probe1",
+      "source_row": 554,
+      "text_sha256": "4b0561012ef0d5f74573b5db0cbff5a730599c0c9afab2a8e5c2a5eea6626146"
+    },
+    {
+      "window": 25,
+      "partition": "probe1",
+      "source_row": 70,
+      "text_sha256": "cf8ac3086cc322da021f905537d7c989d905ba49cea348b8fd2a70475a3438a6"
+    },
+    {
+      "window": 26,
+      "partition": "probe1",
+      "source_row": 1614,
+      "text_sha256": "a0b3befa0a8046e68d64127e7464f92ddc1cd6aa81c911568eec3c62ba059d76"
+    },
+    {
+      "window": 27,
+      "partition": "probe1",
+      "source_row": 2102,
+      "text_sha256": "8e9ac8a237de2f0fd81892c236e2d36eeafea8a714b9b0de2d96920beb97e53a"
+    },
+    {
+      "window": 28,
+      "partition": "probe1",
+      "source_row": 2066,
+      "text_sha256": "b5efb5617272be29b84acb8c53afbdbc4bfe9623c42f3f925a69fd6d712665be"
+    },
+    {
+      "window": 29,
+      "partition": "probe1",
+      "source_row": 542,
+      "text_sha256": "9688247c072837eda08b96ae3c9350c75c6e18267c67b906846042a10deb96bb"
+    },
+    {
+      "window": 30,
+      "partition": "probe1",
+      "source_row": 1666,
+      "text_sha256": "80c27555a095ee4e1d15963bfae5420a89a25307f2fe9d832eb0edd7ecbfa96b"
+    },
+    {
+      "window": 31,
+      "partition": "probe1",
+      "source_row": 292,
+      "text_sha256": "d1dab59f39e86db5b41c8dd1ab142c2e4f77abca3c328532764cae0b44bceaec"
+    },
+    {
+      "window": 32,
+      "partition": "probe2",
+      "source_row": 927,
+      "text_sha256": "78c7390397dda9b94dac31ef640ddc2c543ea13470e642e264f418f89b1a7b2c"
+    },
+    {
+      "window": 33,
+      "partition": "probe2",
+      "source_row": 706,
+      "text_sha256": "465358271c87c7ae177d782516d7c88f36c71555853a013cd425a760404b298c"
+    },
+    {
+      "window": 34,
+      "partition": "probe2",
+      "source_row": 814,
+      "text_sha256": "c5a784358e39b829c6b28e1c4d796db6ffad09de5f9792af50442c4183c26a48"
+    },
+    {
+      "window": 35,
+      "partition": "probe2",
+      "source_row": 2071,
+      "text_sha256": "795b6cb4f9cb7e2351c905fec991a5d9da8b3b05d9f6bd09c4d9f9827a6262cd"
+    },
+    {
+      "window": 36,
+      "partition": "probe2",
+      "source_row": 614,
+      "text_sha256": "ba9983265097a47af1c2dbf22cceb244c7fd0721928ecbedd632449a9ad69f42"
+    },
+    {
+      "window": 37,
+      "partition": "probe2",
+      "source_row": 1442,
+      "text_sha256": "315c8408640aac9309585c178469195861f8b034f4524b98e98fe2e39a2db8fd"
+    },
+    {
+      "window": 38,
+      "partition": "probe2",
+      "source_row": 1273,
+      "text_sha256": "4af0c2cbc9eb4fc229c01c74fff1d0aca76051d3f0407163b92cfdc54cb72dd5"
+    },
+    {
+      "window": 39,
+      "partition": "probe2",
+      "source_row": 322,
+      "text_sha256": "0fd83cb3edf06bba9b0c4be5babf3728b554fdfdbd9951938a05d8f1fbc834ea"
+    },
+    {
+      "window": 40,
+      "partition": "probe2",
+      "source_row": 184,
+      "text_sha256": "ea8d0bc374546d37782fd7e62023c3d477379ec1cb8ba4c9879c93cc266903c7"
+    },
+    {
+      "window": 41,
+      "partition": "probe2",
+      "source_row": 178,
+      "text_sha256": "22345c36f9c635605e65bcd0959dd9c4f3833ec5f0df45e084d2e091aa1bae3b"
+    },
+    {
+      "window": 42,
+      "partition": "probe2",
+      "source_row": 1316,
+      "text_sha256": "4d0593e4b571b1c07bf8d6ba3f7e378f734bf1243139a64a90b547e5366ae65d"
+    },
+    {
+      "window": 43,
+      "partition": "probe2",
+      "source_row": 704,
+      "text_sha256": "777a62b75dc479fb6e75ce9b2860f32eea23b547c965cb0382c66e517357cbfe"
+    },
+    {
+      "window": 44,
+      "partition": "probe2",
+      "source_row": 917,
+      "text_sha256": "abea2244f0a07bab6de1ce866bcb52b62ba958ebd5888bc069b7edeefede2d53"
+    },
+    {
+      "window": 45,
+      "partition": "probe2",
+      "source_row": 223,
+      "text_sha256": "ab9828bfd4f8edf1694c179ad3bba9112f68d5ef19b0fb17943bbcfabce8d2fa"
+    },
+    {
+      "window": 46,
+      "partition": "probe2",
+      "source_row": 422,
+      "text_sha256": "faa650ac83270636b3bbd96e120490627b33a62756fa14887792909843c8a866"
+    },
+    {
+      "window": 47,
+      "partition": "probe2",
+      "source_row": 1512,
+      "text_sha256": "b02da7702a73f0c6049331b088d4827ab3cfdb003cf832503bb777235926c318"
+    },
+    {
+      "window": 48,
+      "partition": "probe3",
+      "source_row": 119,
+      "text_sha256": "ac06b8936829ddcc01c508aacb24a1e114f0dc4f7d59c1e00759b71d568626e1"
+    },
+    {
+      "window": 49,
+      "partition": "probe3",
+      "source_row": 2036,
+      "text_sha256": "00048ad05b1da3e41eba59d05c5a2457361146dc90842fbbd62e7bba514aa355"
+    },
+    {
+      "window": 50,
+      "partition": "probe3",
+      "source_row": 717,
+      "text_sha256": "8d395f4076c5898cb04c748e76c37156acf7bc99965dbd35acd181470c45c350"
+    },
+    {
+      "window": 51,
+      "partition": "probe3",
+      "source_row": 2048,
+      "text_sha256": "424d908158f9639dac712da545f2976798e57aea6868fa37932590ca59242a65"
+    },
+    {
+      "window": 52,
+      "partition": "probe3",
+      "source_row": 1857,
+      "text_sha256": "752c0dcdf557ff6f36cdc15a66f7997338ecaa4103f2ad91ad9203c1e8e8c1c5"
+    },
+    {
+      "window": 53,
+      "partition": "probe3",
+      "source_row": 648,
+      "text_sha256": "7b74cbeaa87703949d6b82298b8d9205f9a2e237ad7600e5b93a1fa8becbe9c3"
+    },
+    {
+      "window": 54,
+      "partition": "probe3",
+      "source_row": 1743,
+      "text_sha256": "d6c0b76a11eba050beb588b69b2cbff39db6447ca9f8cd56e8a5dcdd644be758"
+    },
+    {
+      "window": 55,
+      "partition": "probe3",
+      "source_row": 1594,
+      "text_sha256": "85a323dc4b998db74e57f2fdd7713623513f208f6893f019d416562048593328"
+    },
+    {
+      "window": 56,
+      "partition": "probe3",
+      "source_row": 2028,
+      "text_sha256": "6c6878ed7f9ce68d87b3f512d96088fb375731f0f9c792b85234572a3f4a05d5"
+    },
+    {
+      "window": 57,
+      "partition": "probe3",
+      "source_row": 1773,
+      "text_sha256": "32e6165c460d090976a51a7ae788f6b6ade7fa672a51ed078c0ab34bf5a79030"
+    },
+    {
+      "window": 58,
+      "partition": "probe3",
+      "source_row": 66,
+      "text_sha256": "6659e3fc7f4119f3421ce62ed3219f4387d7411969e71b92c8bf4bac60f5a6ab"
+    },
+    {
+      "window": 59,
+      "partition": "probe3",
+      "source_row": 1300,
+      "text_sha256": "1dc2d9a60f5cb36c034b14f8369b5909543a21842e7d84d304f9f60d40f58390"
+    },
+    {
+      "window": 60,
+      "partition": "probe3",
+      "source_row": 1872,
+      "text_sha256": "c13969a5e66e1b0ae3d220c31d01317dca123eed1b93f11c7f3e876775427032"
+    },
+    {
+      "window": 61,
+      "partition": "probe3",
+      "source_row": 1485,
+      "text_sha256": "e9aada23f955bdb5168cb5b5e44861fa292d75999578e1684a84061473a6ab06"
+    },
+    {
+      "window": 62,
+      "partition": "probe3",
+      "source_row": 440,
+      "text_sha256": "6010c9d689cda8ef5134d55883ab9c4ab9a542b13ed68e93d42a11851afa8499"
+    },
+    {
+      "window": 63,
+      "partition": "probe3",
+      "source_row": 1002,
+      "text_sha256": "815f6c7b5a9b7977373b3bdf31fb703323b950b1e55969105c322a18551fa7df"
+    },
+    {
+      "window": 64,
+      "partition": "evaluation",
+      "source_row": 144,
+      "text_sha256": "911c431f2abd182ea0a68861cf0c75462234b4d7df3f465091f0b0976eaf3f6c"
+    },
+    {
+      "window": 65,
+      "partition": "evaluation",
+      "source_row": 1118,
+      "text_sha256": "bec5d5fcf9fbfc8b2409b70c3758b407b089b28d66f8e5c5ac7ad09c45bbe180"
+    },
+    {
+      "window": 66,
+      "partition": "evaluation",
+      "source_row": 2001,
+      "text_sha256": "c63081a2bf4b73f4cfa4ded5242c45f9d2bae008ee7d1a43d3fc92d28d5012da"
+    },
+    {
+      "window": 67,
+      "partition": "evaluation",
+      "source_row": 1887,
+      "text_sha256": "cc7b9c3def893be82cf5ed652ece52373d967ae93c0d48e93b79e3a715cf5a6c"
+    },
+    {
+      "window": 68,
+      "partition": "evaluation",
+      "source_row": 1076,
+      "text_sha256": "cfd4694dc95e2f26caf13e267457d7ffe57aab7ef1603e42879c94d7ad6f7799"
+    },
+    {
+      "window": 69,
+      "partition": "evaluation",
+      "source_row": 460,
+      "text_sha256": "c405da5eed4391592b86ed97d596c5d72e61adb2db39187314a147eef2ee7cd0"
+    },
+    {
+      "window": 70,
+      "partition": "evaluation",
+      "source_row": 699,
+      "text_sha256": "114124ad68b52ab41c6b0f6c0d20f021d3b8af45951a59c19010c5e90f5a82ca"
+    },
+    {
+      "window": 71,
+      "partition": "evaluation",
+      "source_row": 2081,
+      "text_sha256": "062edc3e388c3c30120a1b58448a53a5360194d44e55f563da2a34d5ef687f75"
+    },
+    {
+      "window": 72,
+      "partition": "evaluation",
+      "source_row": 656,
+      "text_sha256": "ca2306c285a9fa2e52dd2657f63d2ec4c1436e7b6d275accae68b35286f37e3e"
+    },
+    {
+      "window": 73,
+      "partition": "evaluation",
+      "source_row": 894,
+      "text_sha256": "fc4d8a2130cc4ae16de1d72acbaa320ed504c3d47fe338272c1e662ffc20e55b"
+    },
+    {
+      "window": 74,
+      "partition": "evaluation",
+      "source_row": 1014,
+      "text_sha256": "e29d7b53675bcf94079332826c2444825a82d34c4ef93858232368df6c889a03"
+    },
+    {
+      "window": 75,
+      "partition": "evaluation",
+      "source_row": 1740,
+      "text_sha256": "beff97d947eda7f594777a9ba740c2fe16be22ae5d8144f3c1de43e203c04be2"
+    },
+    {
+      "window": 76,
+      "partition": "evaluation",
+      "source_row": 1724,
+      "text_sha256": "97a997822100dadf4f23788d74da3b6794f474b3ae677b64fb2786b5bd5474d4"
+    },
+    {
+      "window": 77,
+      "partition": "evaluation",
+      "source_row": 453,
+      "text_sha256": "c1b63570df553bff11da5480cbf2d17c5d94d6212fca140a4170994db9b9787a"
+    },
+    {
+      "window": 78,
+      "partition": "evaluation",
+      "source_row": 492,
+      "text_sha256": "c2cd75d4d1a1d50cbb620517ce36ae7fbce05ef9b752c53a88407175810dc885"
+    },
+    {
+      "window": 79,
+      "partition": "evaluation",
+      "source_row": 28,
+      "text_sha256": "ab6f397025317fc09740fc038384538ada26add6bf5f76412ca4b2770d072c32"
+    },
+    {
+      "window": 80,
+      "partition": "evaluation",
+      "source_row": 795,
+      "text_sha256": "da7f679af071160e300074add2fbe4eafcd4aed186d04ef2e7177decb2e0785c"
+    },
+    {
+      "window": 81,
+      "partition": "evaluation",
+      "source_row": 1541,
+      "text_sha256": "20d0c6cb9f4dcf26a2d3f3f77571c524028791dd813cc9dc0b9ec1d29f0ea170"
+    },
+    {
+      "window": 82,
+      "partition": "evaluation",
+      "source_row": 1012,
+      "text_sha256": "4c1e3bacb35cf4a00dfd4d823129e28df8b880ced4109d1bb713c4bb4df41591"
+    },
+    {
+      "window": 83,
+      "partition": "evaluation",
+      "source_row": 2049,
+      "text_sha256": "fcab03a45f28403db692e4323ea332e2654489fb7ca823cdaeee180a6267b27f"
+    },
+    {
+      "window": 84,
+      "partition": "evaluation",
+      "source_row": 2073,
+      "text_sha256": "8aff2e9b513baff268e235c4ed7f64ba414c2832d48879e605e7bc10a8284b4c"
+    },
+    {
+      "window": 85,
+      "partition": "evaluation",
+      "source_row": 1921,
+      "text_sha256": "f34c09e1f0f252dd1e09009dcb1a067b07681282b33aec0bcb842e3494bcf88f"
+    },
+    {
+      "window": 86,
+      "partition": "evaluation",
+      "source_row": 757,
+      "text_sha256": "a3919ed4c5319aeb11b11ae3fa509cff2d863a6753bc93223aa6341f911c3f51"
+    },
+    {
+      "window": 87,
+      "partition": "evaluation",
+      "source_row": 869,
+      "text_sha256": "6c478b8dd1e0aca6b31f7a699be7bb859e3a374e0393fcd3f23212561d6e5f0e"
+    },
+    {
+      "window": 88,
+      "partition": "evaluation",
+      "source_row": 635,
+      "text_sha256": "8c1b60b5892c5dd49551375d85400998057898a78140f71fd1665c873f6dd06e"
+    },
+    {
+      "window": 89,
+      "partition": "evaluation",
+      "source_row": 1215,
+      "text_sha256": "8eecb9d179feec3e48664c2e5cd32ef4019c03c6c5f7e3a378fcb7e673c207d4"
+    },
+    {
+      "window": 90,
+      "partition": "evaluation",
+      "source_row": 1759,
+      "text_sha256": "678be1937dc9bfa60feb75509b6df18fb2307ad2d3aa80abd1515d5f32b6b8ed"
+    },
+    {
+      "window": 91,
+      "partition": "evaluation",
+      "source_row": 466,
+      "text_sha256": "9b9f623736bf62e6c0a9567da27496d44bd37cce5d1561234534d8ee9f8b2719"
+    },
+    {
+      "window": 92,
+      "partition": "evaluation",
+      "source_row": 819,
+      "text_sha256": "e155822a68cd5479547fbec8cade53b9ad0ee99cf1fc049c1cfa1b615bb774b1"
+    },
+    {
+      "window": 93,
+      "partition": "evaluation",
+      "source_row": 1640,
+      "text_sha256": "a45b41d5241a027750271d797d80c63a28a13c42c83f49aaedd0b7a4b6f7f4fe"
+    },
+    {
+      "window": 94,
+      "partition": "evaluation",
+      "source_row": 1091,
+      "text_sha256": "19d238175a738101a176b94efc47e8fc07ce8e24bb270306c33aadd8758543ec"
+    },
+    {
+      "window": 95,
+      "partition": "evaluation",
+      "source_row": 552,
+      "text_sha256": "5c75b0c948da9f0ba3174ef60b1ff1dca9b1ef09a67adc540950173f596cbf79"
+    },
+    {
+      "window": 96,
+      "partition": "evaluation",
+      "source_row": 853,
+      "text_sha256": "c3117ea2fa3e8c20380715ab834a6e32c88ca6608a625384d025d746310c92b2"
+    },
+    {
+      "window": 97,
+      "partition": "evaluation",
+      "source_row": 1863,
+      "text_sha256": "df53de3b3263fc01064ca82b60e2c2d851edb4fdb344ef47d00d6a906883adaf"
+    },
+    {
+      "window": 98,
+      "partition": "evaluation",
+      "source_row": 212,
+      "text_sha256": "cd48cc8fe42240219a55997c612f0f9f0d736920db59b61d7a2f465dc89ae076"
+    },
+    {
+      "window": 99,
+      "partition": "evaluation",
+      "source_row": 1201,
+      "text_sha256": "d3d36c9f5e1a07d19d92cbcd5370bf331c532c8055f85ee7c94d88fe4f398ac2"
+    },
+    {
+      "window": 100,
+      "partition": "evaluation",
+      "source_row": 1741,
+      "text_sha256": "4178aa353e76235f3ef4ddca74b85b21d598375041a4e6586fcc2e60927e6325"
+    },
+    {
+      "window": 101,
+      "partition": "evaluation",
+      "source_row": 628,
+      "text_sha256": "662d9ac3712b288846bdec84846efd83c05bb9b6e332d0c4387591f03499aec7"
+    },
+    {
+      "window": 102,
+      "partition": "evaluation",
+      "source_row": 1055,
+      "text_sha256": "d6e5ca1e46229411dbfc4da28dfd4807a89a0be0df3970ddf0c0c2bb5cdb3fc7"
+    },
+    {
+      "window": 103,
+      "partition": "evaluation",
+      "source_row": 1493,
+      "text_sha256": "d821de2d2cf30b906e7250b45a0e367424ed658d5eea9fc4fcbc870e5edcb139"
+    },
+    {
+      "window": 104,
+      "partition": "evaluation",
+      "source_row": 1718,
+      "text_sha256": "5095f28931627adbf5d2c0ad92399c1b6f628b2ab8c1a46c301a46f9afc59cc4"
+    },
+    {
+      "window": 105,
+      "partition": "evaluation",
+      "source_row": 274,
+      "text_sha256": "cd646837b9ae670f0d0910f9e9f2fcd6ea4b4d81d096d79380862ea94ba291bc"
+    },
+    {
+      "window": 106,
+      "partition": "evaluation",
+      "source_row": 356,
+      "text_sha256": "bb7e08ba07a358fdeac3978c26928c09cc152474f65861cb28040178557899fb"
+    },
+    {
+      "window": 107,
+      "partition": "evaluation",
+      "source_row": 829,
+      "text_sha256": "7415772f240948e566b380da414c29de1efca50af571a47a573c0f07793e44e2"
+    },
+    {
+      "window": 108,
+      "partition": "evaluation",
+      "source_row": 1236,
+      "text_sha256": "28985fa5a620e9cc405d6542b84c7cb3119806a4c90bfe19945adb7915288168"
+    },
+    {
+      "window": 109,
+      "partition": "evaluation",
+      "source_row": 773,
+      "text_sha256": "cc25948225615213da35ea980c721b4bd035c8dae588c8ea7f88eb1ff4208d95"
+    },
+    {
+      "window": 110,
+      "partition": "evaluation",
+      "source_row": 934,
+      "text_sha256": "282311e90b243d0251ac74234cd5ea911483963a8e97ba443b28f9b8ff0b7bdd"
+    },
+    {
+      "window": 111,
+      "partition": "evaluation",
+      "source_row": 265,
+      "text_sha256": "b65c9299d8f7bf6be9db82beec3a8372ec59b2005b1eb8256ef0c110493e2b9a"
+    },
+    {
+      "window": 112,
+      "partition": "evaluation",
+      "source_row": 802,
+      "text_sha256": "5de652fe7ab2bcc90011a412371a4b3d2eed07e9907c09759e6858f7ca90e3fa"
+    },
+    {
+      "window": 113,
+      "partition": "evaluation",
+      "source_row": 1946,
+      "text_sha256": "e20263641c5257a5a5d6245a45da1d287d4352590ae7f1615b83a380199281d0"
+    },
+    {
+      "window": 114,
+      "partition": "evaluation",
+      "source_row": 1074,
+      "text_sha256": "63ca2b4cee65240dfd101c68cc5d75f0f0d466a5ca2cf607566d6e1dbc49e159"
+    },
+    {
+      "window": 115,
+      "partition": "evaluation",
+      "source_row": 888,
+      "text_sha256": "95aeff310def651e5e653c67eb6d289e6b5388ef1488c5575734aa7c285d9578"
+    },
+    {
+      "window": 116,
+      "partition": "evaluation",
+      "source_row": 852,
+      "text_sha256": "f987887dc60516d98a869716fc06ca337464860e5b953f08cb0131137229bcae"
+    },
+    {
+      "window": 117,
+      "partition": "evaluation",
+      "source_row": 1482,
+      "text_sha256": "40bab508377d7befd6cf149e41f37faacef9e047570cb96c73d31e46a6b4062b"
+    },
+    {
+      "window": 118,
+      "partition": "evaluation",
+      "source_row": 692,
+      "text_sha256": "28676c0dc9a23e95ccf7ae45a3c5aa39a6df0735ce02de6c5550c03b65a3b8f5"
+    },
+    {
+      "window": 119,
+      "partition": "evaluation",
+      "source_row": 747,
+      "text_sha256": "a0cd86b59f890b9f05118c6b798753916047670b0a077d1610da8d2d212fb204"
+    },
+    {
+      "window": 120,
+      "partition": "evaluation",
+      "source_row": 1582,
+      "text_sha256": "11f073ef6c3e38b448db331fc57214b1a2de7de9d82bdd622484aebed2abd1a9"
+    },
+    {
+      "window": 121,
+      "partition": "evaluation",
+      "source_row": 2016,
+      "text_sha256": "eb9ed5bcac43fd896c36ffb23aa400afcda6c1b24bee468d7731860e72d2b6bb"
+    },
+    {
+      "window": 122,
+      "partition": "evaluation",
+      "source_row": 644,
+      "text_sha256": "e933f8aa6434edb6a220b2f8751440134218da7ab721ff7d512fc9c9422252c9"
+    },
+    {
+      "window": 123,
+      "partition": "evaluation",
+      "source_row": 1039,
+      "text_sha256": "2962ea5c560250a86b51d5a2c322e797491613369ff28b8095e7697abfd52aac"
+    },
+    {
+      "window": 124,
+      "partition": "evaluation",
+      "source_row": 1257,
+      "text_sha256": "891b49605714f1236ff2990d12338f096b31db370a6124f3afa0159f312e07a0"
+    },
+    {
+      "window": 125,
+      "partition": "evaluation",
+      "source_row": 720,
+      "text_sha256": "59a1d27a172b1369fb8143edb0b3c488dd5fa9e3e707ec4b1761ced886542a2b"
+    },
+    {
+      "window": 126,
+      "partition": "evaluation",
+      "source_row": 480,
+      "text_sha256": "4a45b6338f5466b2f001d06b46888aaf056b2a20fd6278daeb469316a92f994f"
+    },
+    {
+      "window": 127,
+      "partition": "evaluation",
+      "source_row": 224,
+      "text_sha256": "35eab0413e99deb5b05406aa0a86c87ba3c00d17de528dec1c8ab27ad1fc8b26"
+    }
+  ],
+  "probe_evaluation_document_overlap": 0,
+  "data": {
+    "llama31": {
+      "probe": {
+        "path": "/home/zs89/Basis/h2_exp/artifacts/llama31/probe.pt",
+        "shape": [
+          64,
+          256
+        ],
+        "tensor_sha256": "117bcb58782b37ea6878eee9de789f11f64aacc90a0f1ceb41ef2bf07ebf9015",
+        "artifact_sha256": "386c2f19333c195c47cea4adc2a882f47615c8d526d5f903bc8911a36a851a9b"
+      },
+      "evaluation": {
+        "path": "/home/zs89/Basis/h2_exp/artifacts/llama31/evaluation.pt",
+        "shape": [
+          64,
+          256
+        ],
+        "tensor_sha256": "fb197f8496a8abec6b950c4a81971e3579ab571075da494b43a8aa3927b96c45",
+        "artifact_sha256": "5cb74049e7a5039fd32dcbf7581807431c4fd828d6ad5c09bda4a5ae02bf7813"
+      }
+    },
+    "qwen3": {
+      "probe": {
+        "path": "/home/zs89/Basis/h2_exp/artifacts/qwen3/probe.pt",
+        "shape": [
+          64,
+          256
+        ],
+        "tensor_sha256": "9d8f5bf750276d5509cc21632b77d298269445420739311c379aee46955aacac",
+        "artifact_sha256": "aaafc696c0f30d57c385f5833e1583e6262da3847b87904923985734a1994b42"
+      },
+      "evaluation": {
+        "path": "/home/zs89/Basis/h2_exp/artifacts/qwen3/evaluation.pt",
+        "shape": [
+          64,
+          256
+        ],
+        "tensor_sha256": "39776fef3736ac0e761aecc088db0cf858afa82ae56e8e2593b8f724f63e6abb",
+        "artifact_sha256": "6d0d654ec4775bb250d458c7c5d2e4ab6060718cd588919d69d8d243fe3f886e"
+      }
+    }
+  },
+  "audits": {
+    "llama31": {
+      "train_artifact": {
+        "backbone": "llama31",
+        "corpus": "c4",
+        "model": "meta-llama/Llama-3.1-8B",
+        "manifest_path": "/home/zs89/FlashSVDTrain/diagnosis/main/01_motivation/experiments/05_llama_modegpt_dual_corpus_contrast/data/c4/manifest.json",
+        "manifest_sha256": "2d7454aa956400d36529a41ec0314f4f94c59f4e33e5d922e594b2619c0cab7d",
+        "tensor_path": "/home/zs89/FlashSVDTrain/diagnosis/main/01_motivation/experiments/05_llama_modegpt_dual_corpus_contrast/data/c4/c4_full_train_384x256.pt",
+        "tensor_sha256": "c8a4e1c6527f6ce6f901766aa60ea5ba3f06e4caa0ad31c92646c67bd4754fc5",
+        "artifact_sha256": "f77158ede5e2aaa138dd899e10bc256fd537caf86f597b0e308ddada8f031493",
+        "shape": [
+          384,
+          256
+        ]
+      },
+      "training_overlap_windows": 0,
+      "unique_windows": 128,
+      "tokenizer_path": "/home/zs89/.cache/huggingface/hub/models--meta-llama--Llama-3.1-8B/snapshots/d04e592bb4f6aa9cfee91e2e20afa771667e1d4b",
+      "tokenizer_files": {
+        "special_tokens_map.json": "462d91939dbc37178aa5a3eae7068d1990ccc92e09f288cc71f42cdf139d69cc",
+        "tokenizer.json": "76e48799b099d43365bd24ccd8ecc5aedac831718da780552f03b0a6eb4412aa",
+        "tokenizer_config.json": "8004530facf809ac432114de2a4dcc65fcb632da5ec16d666091aeb6a2ee444a"
+      }
+    },
+    "qwen3": {
+      "train_artifact": {
+        "backbone": "qwen3",
+        "corpus": "c4",
+        "model": "Qwen/Qwen3-8B-Base",
+        "manifest_path": "/home/zs89/FlashSVDTrain/diagnosis/main/01_motivation/experiments/04_qwen_matched_source_generality/data/manifest.json",
+        "manifest_sha256": "8454cc2590b785a92a9e0dd911fac7859d3dcf90b6d5ff3c1727023987e2e9eb",
+        "tensor_path": "/home/zs89/FlashSVDTrain/diagnosis/main/01_motivation/experiments/04_qwen_matched_source_generality/data/c4_full_train_384x256_repeat4.pt",
+        "tensor_sha256": "c40533d07ab5c2c9c281872dfc38aeb5b60e4084a84ef8d47e745d52debce505",
+        "artifact_sha256": "ae869fc7d350a0be962f883ed5ce89d8fd3546c0b0e4a98ae80d5a1becee2e05",
+        "shape": [
+          384,
+          256
+        ]
+      },
+      "training_overlap_windows": 0,
+      "unique_windows": 128,
+      "tokenizer_path": "/home/zs89/.cache/huggingface/hub/models--Qwen--Qwen3-8B-Base/snapshots/49e3418fbbbca6ecbdf9608b4d22e5a407081db4",
+      "tokenizer_files": {
+        "merges.txt": "8831e4f1a044471340f7c0a83d7bd71306a5b867e95fd870f74d0c5308a904d5",
+        "tokenizer.json": "c0382117ea329cdf097041132f6d735924b697924d6f6fc3945713e96ce87539",
+        "tokenizer_config.json": "3c04ed3ca964ea2f6b2b5faf0dc4d31aec1cb1e8b4bcf63f402d295046b422b5",
+        "vocab.json": "ca10d7e9fb3ed18575dd1e277a2579c16d108e32f27439684afa0e10b1440910"
+      }
+    }
+  }
+}

basis/data/probes/q64/qwen3/evaluation.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6d0d654ec4775bb250d458c7c5d2e4ab6060718cd588919d69d8d243fe3f886e
+size 263834

basis/data/probes/q64/qwen3/probe.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:aaafc696c0f30d57c385f5833e1583e6262da3847b87904923985734a1994b42
+size 263735

basis/gradients/q128/llama31/asvd_r080/step_0000/block_0/summary.json

@@ -0,0 +1,995 @@
+{
+  "status": "complete",
+  "job_id": "llama31/asvd_r080",
+  "backbone": "llama31",
+  "step": 0,
+  "state_id": "6f1b8bc457a7ab3a928da64325d85a47efcb711c337504eb5b08a0597793b221",
+  "manifest_sha256": "d77eb1a1b9fd2801372ec781f1052b3ca276dffccfe20b34e3c7c44dcfed52a2",
+  "probe_tensor_sha256": "f19c31b43250bb8f5904e38874b21aca10fff6f736a76ada81e86121553f7e1b",
+  "evaluation_tensor_sha256": "736e3a4086111f686021cff0b3beaf1cd1ae4feae52bfd6719cdb3680a9e400d",
+  "block": 0,
+  "global_window_indices": [
+    0,
+    1,
+    2,
+    3,
+    4,
+    5,
+    6,
+    7,
+    8,
+    9,
+    10,
+    11,
+    12,
+    13,
+    14,
+    15
+  ],
+  "vectors": {
+    "path": "/home/zs89/Basis/hx_exp/outputs/confirmation/llama31/asvd_r080/step_0000/block_0/vectors.pt",
+    "sha256": "30b407fb074cbb73d8aab3186f0f4e6b953d109f24a1d4204e955ca8a8bf99ed"
+  },
+  "audit": {
+    "window_nll": [
+      8.116129875183105,
+      8.684808731079102,
+      7.616055965423584,
+      8.126176834106445,
+      8.447510719299316,
+      8.912871360778809,
+      8.447684288024902,
+      9.058097839355469,
+      9.518945693969727,
+      8.312564849853516,
+      7.976768970489502,
+      7.2854766845703125,
+      7.603358268737793,
+      7.888175010681152,
+      7.658417701721191,
+      7.621391296386719
+    ],
+    "coordinate_count": 262144,
+    "module_records": [
+      {
+        "name": "layer00.attn",
+        "layer": 0,
+        "module_type": "attention",
+        "coordinate_dim": 4096,
+        "start": 0,
+        "stop": 4096
+      },
+      {
+        "name": "layer00.mlp",
+        "layer": 0,
+        "module_type": "MLP",
+        "coordinate_dim": 4096,
+        "start": 4096,
+        "stop": 8192
+      },
+      {
+        "name": "layer01.attn",
+        "layer": 1,
+        "module_type": "attention",
+        "coordinate_dim": 4096,
+        "start": 8192,
+        "stop": 12288
+      },
+      {
+        "name": "layer01.mlp",
+        "layer": 1,
+        "module_type": "MLP",
+        "coordinate_dim": 4096,
+        "start": 12288,
+        "stop": 16384
+      },
+      {
+        "name": "layer02.attn",
+        "layer": 2,
+        "module_type": "attention",
+        "coordinate_dim": 4096,
+        "start": 16384,
+        "stop": 20480
+      },
+      {
+        "name": "layer02.mlp",
+        "layer": 2,
+        "module_type": "MLP",
+        "coordinate_dim": 4096,
+        "start": 20480,
+        "stop": 24576
+      },
+      {
+        "name": "layer03.attn",
+        "layer": 3,
+        "module_type": "attention",
+        "coordinate_dim": 4096,
+        "start": 24576,
+        "stop": 28672
+      },
+      {
+        "name": "layer03.mlp",
+        "layer": 3,
+        "module_type": "MLP",
+        "coordinate_dim": 4096,
+        "start": 28672,
+        "stop": 32768
+      },
+      {
+        "name": "layer04.attn",
+        "layer": 4,
+        "module_type": "attention",
+        "coordinate_dim": 4096,
+        "start": 32768,
+        "stop": 36864
+      },
+      {
+        "name": "layer04.mlp",
+        "layer": 4,
+        "module_type": "MLP",
+        "coordinate_dim": 4096,
+        "start": 36864,
+        "stop": 40960
+      },
+      {
+        "name": "layer05.attn",
+        "layer": 5,
+        "module_type": "attention",
+        "coordinate_dim": 4096,
+        "start": 40960,
+        "stop": 45056
+      },
+      {
+        "name": "layer05.mlp",
+        "layer": 5,
+        "module_type": "MLP",
+        "coordinate_dim": 4096,
+        "start": 45056,
+        "stop": 49152
+      },
+      {
+        "name": "layer06.attn",
+        "layer": 6,
+        "module_type": "attention",
+        "coordinate_dim": 4096,
+        "start": 49152,
+        "stop": 53248
+      },
+      {
+        "name": "layer06.mlp",
+        "layer": 6,
+        "module_type": "MLP",
+        "coordinate_dim": 4096,
+        "start": 53248,
+        "stop": 57344
+      },
+      {
+        "name": "layer07.attn",
+        "layer": 7,
+        "module_type": "attention",
+        "coordinate_dim": 4096,
+        "start": 57344,
+        "stop": 61440
+      },
+      {
+        "name": "layer07.mlp",
+        "layer": 7,
+        "module_type": "MLP",
+        "coordinate_dim": 4096,
+        "start": 61440,
+        "stop": 65536
+      },
+      {
+        "name": "layer08.attn",
+        "layer": 8,
+        "module_type": "attention",
+        "coordinate_dim": 4096,
+        "start": 65536,
+        "stop": 69632
+      },
+      {
+        "name": "layer08.mlp",
+        "layer": 8,
+        "module_type": "MLP",
+        "coordinate_dim": 4096,
+        "start": 69632,
+        "stop": 73728
+      },
+      {
+        "name": "layer09.attn",
+        "layer": 9,
+        "module_type": "attention",
+        "coordinate_dim": 4096,
+        "start": 73728,
+        "stop": 77824
+      },
+      {
+        "name": "layer09.mlp",
+        "layer": 9,
+        "module_type": "MLP",
+        "coordinate_dim": 4096,
+        "start": 77824,
+        "stop": 81920
+      },
+      {
+        "name": "layer10.attn",
+        "layer": 10,
+        "module_type": "attention",
+        "coordinate_dim": 4096,
+        "start": 81920,
+        "stop": 86016
+      },
+      {
+        "name": "layer10.mlp",
+        "layer": 10,
+        "module_type": "MLP",
+        "coordinate_dim": 4096,
+        "start": 86016,
+        "stop": 90112
+      },
+      {
+        "name": "layer11.attn",
+        "layer": 11,
+        "module_type": "attention",
+        "coordinate_dim": 4096,
+        "start": 90112,
+        "stop": 94208
+      },
+      {
+        "name": "layer11.mlp",
+        "layer": 11,
+        "module_type": "MLP",
+        "coordinate_dim": 4096,
+        "start": 94208,
+        "stop": 98304
+      },
+      {
+        "name": "layer12.attn",
+        "layer": 12,
+        "module_type": "attention",
+        "coordinate_dim": 4096,
+        "start": 98304,
+        "stop": 102400
+      },
+      {
+        "name": "layer12.mlp",
+        "layer": 12,
+        "module_type": "MLP",
+        "coordinate_dim": 4096,
+        "start": 102400,
+        "stop": 106496
+      },
+      {
+        "name": "layer13.attn",
+        "layer": 13,
+        "module_type": "attention",
+        "coordinate_dim": 4096,
+        "start": 106496,
+        "stop": 110592
+      },
+      {
+        "name": "layer13.mlp",
+        "layer": 13,
+        "module_type": "MLP",
+        "coordinate_dim": 4096,
+        "start": 110592,
+        "stop": 114688
+      },
+      {
+        "name": "layer14.attn",
+        "layer": 14,
+        "module_type": "attention",
+        "coordinate_dim": 4096,
+        "start": 114688,
+        "stop": 118784
+      },
+      {
+        "name": "layer14.mlp",
+        "layer": 14,
+        "module_type": "MLP",
+        "coordinate_dim": 4096,
+        "start": 118784,
+        "stop": 122880
+      },
+      {
+        "name": "layer15.attn",
+        "layer": 15,
+        "module_type": "attention",
+        "coordinate_dim": 4096,
+        "start": 122880,
+        "stop": 126976
+      },
+      {
+        "name": "layer15.mlp",
+        "layer": 15,
+        "module_type": "MLP",
+        "coordinate_dim": 4096,
+        "start": 126976,
+        "stop": 131072
+      },
+      {
+        "name": "layer16.attn",
+        "layer": 16,
+        "module_type": "attention",
+        "coordinate_dim": 4096,
+        "start": 131072,
+        "stop": 135168
+      },
+      {
+        "name": "layer16.mlp",
+        "layer": 16,
+        "module_type": "MLP",
+        "coordinate_dim": 4096,
+        "start": 135168,
+        "stop": 139264
+      },
+      {
+        "name": "layer17.attn",
+        "layer": 17,
+        "module_type": "attention",
+        "coordinate_dim": 4096,
+        "start": 139264,
+        "stop": 143360
+      },
+      {
+        "name": "layer17.mlp",
+        "layer": 17,
+        "module_type": "MLP",
+        "coordinate_dim": 4096,
+        "start": 143360,
+        "stop": 147456
+      },
+      {
+        "name": "layer18.attn",
+        "layer": 18,
+        "module_type": "attention",
+        "coordinate_dim": 4096,
+        "start": 147456,
+        "stop": 151552
+      },
+      {
+        "name": "layer18.mlp",
+        "layer": 18,
+        "module_type": "MLP",
+        "coordinate_dim": 4096,
+        "start": 151552,
+        "stop": 155648
+      },
+      {
+        "name": "layer19.attn",
+        "layer": 19,
+        "module_type": "attention",
+        "coordinate_dim": 4096,
+        "start": 155648,
+        "stop": 159744
+      },
+      {
+        "name": "layer19.mlp",
+        "layer": 19,
+        "module_type": "MLP",
+        "coordinate_dim": 4096,
+        "start": 159744,
+        "stop": 163840
+      },
+      {
+        "name": "layer20.attn",
+        "layer": 20,
+        "module_type": "attention",
+        "coordinate_dim": 4096,
+        "start": 163840,
+        "stop": 167936
+      },
+      {
+        "name": "layer20.mlp",
+        "layer": 20,
+        "module_type": "MLP",
+        "coordinate_dim": 4096,
+        "start": 167936,
+        "stop": 172032
+      },
+      {
+        "name": "layer21.attn",
+        "layer": 21,
+        "module_type": "attention",
+        "coordinate_dim": 4096,
+        "start": 172032,
+        "stop": 176128
+      },
+      {
+        "name": "layer21.mlp",
+        "layer": 21,
+        "module_type": "MLP",
+        "coordinate_dim": 4096,
+        "start": 176128,
+        "stop": 180224
+      },
+      {
+        "name": "layer22.attn",
+        "layer": 22,
+        "module_type": "attention",
+        "coordinate_dim": 4096,
+        "start": 180224,
+        "stop": 184320
+      },
+      {
+        "name": "layer22.mlp",
+        "layer": 22,
+        "module_type": "MLP",
+        "coordinate_dim": 4096,
+        "start": 184320,
+        "stop": 188416
+      },
+      {
+        "name": "layer23.attn",
+        "layer": 23,
+        "module_type": "attention",
+        "coordinate_dim": 4096,
+        "start": 188416,
+        "stop": 192512
+      },
+      {
+        "name": "layer23.mlp",
+        "layer": 23,
+        "module_type": "MLP",
+        "coordinate_dim": 4096,
+        "start": 192512,
+        "stop": 196608
+      },
+      {
+        "name": "layer24.attn",
+        "layer": 24,
+        "module_type": "attention",
+        "coordinate_dim": 4096,
+        "start": 196608,
+        "stop": 200704
+      },
+      {
+        "name": "layer24.mlp",
+        "layer": 24,
+        "module_type": "MLP",
+        "coordinate_dim": 4096,
+        "start": 200704,
+        "stop": 204800
+      },
+      {
+        "name": "layer25.attn",
+        "layer": 25,
+        "module_type": "attention",
+        "coordinate_dim": 4096,
+        "start": 204800,
+        "stop": 208896
+      },
+      {
+        "name": "layer25.mlp",
+        "layer": 25,
+        "module_type": "MLP",
+        "coordinate_dim": 4096,
+        "start": 208896,
+        "stop": 212992
+      },
+      {
+        "name": "layer26.attn",
+        "layer": 26,
+        "module_type": "attention",
+        "coordinate_dim": 4096,
+        "start": 212992,
+        "stop": 217088
+      },
+      {
+        "name": "layer26.mlp",
+        "layer": 26,
+        "module_type": "MLP",
+        "coordinate_dim": 4096,
+        "start": 217088,
+        "stop": 221184
+      },
+      {
+        "name": "layer27.attn",
+        "layer": 27,
+        "module_type": "attention",
+        "coordinate_dim": 4096,
+        "start": 221184,
+        "stop": 225280
+      },
+      {
+        "name": "layer27.mlp",
+        "layer": 27,
+        "module_type": "MLP",
+        "coordinate_dim": 4096,
+        "start": 225280,
+        "stop": 229376
+      },
+      {
+        "name": "layer28.attn",
+        "layer": 28,
+        "module_type": "attention",
+        "coordinate_dim": 4096,
+        "start": 229376,
+        "stop": 233472
+      },
+      {
+        "name": "layer28.mlp",
+        "layer": 28,
+        "module_type": "MLP",
+        "coordinate_dim": 4096,
+        "start": 233472,
+        "stop": 237568
+      },
+      {
+        "name": "layer29.attn",
+        "layer": 29,
+        "module_type": "attention",
+        "coordinate_dim": 4096,
+        "start": 237568,
+        "stop": 241664
+      },
+      {
+        "name": "layer29.mlp",
+        "layer": 29,
+        "module_type": "MLP",
+        "coordinate_dim": 4096,
+        "start": 241664,
+        "stop": 245760
+      },
+      {
+        "name": "layer30.attn",
+        "layer": 30,
+        "module_type": "attention",
+        "coordinate_dim": 4096,
+        "start": 245760,
+        "stop": 249856
+      },
+      {
+        "name": "layer30.mlp",
+        "layer": 30,
+        "module_type": "MLP",
+        "coordinate_dim": 4096,
+        "start": 249856,
+        "stop": 253952
+      },
+      {
+        "name": "layer31.attn",
+        "layer": 31,
+        "module_type": "attention",
+        "coordinate_dim": 4096,
+        "start": 253952,
+        "stop": 258048
+      },
+      {
+        "name": "layer31.mlp",
+        "layer": 31,
+        "module_type": "MLP",
+        "coordinate_dim": 4096,
+        "start": 258048,
+        "stop": 262144
+      }
+    ],
+    "module_names": [
+      "layer00.attn",
+      "layer00.mlp",
+      "layer01.attn",
+      "layer01.mlp",
+      "layer02.attn",
+      "layer02.mlp",
+      "layer03.attn",
+      "layer03.mlp",
+      "layer04.attn",
+      "layer04.mlp",
+      "layer05.attn",
+      "layer05.mlp",
+      "layer06.attn",
+      "layer06.mlp",
+      "layer07.attn",
+      "layer07.mlp",
+      "layer08.attn",
+      "layer08.mlp",
+      "layer09.attn",
+      "layer09.mlp",
+      "layer10.attn",
+      "layer10.mlp",
+      "layer11.attn",
+      "layer11.mlp",
+      "layer12.attn",
+      "layer12.mlp",
+      "layer13.attn",
+      "layer13.mlp",
+      "layer14.attn",
+      "layer14.mlp",
+      "layer15.attn",
+      "layer15.mlp",
+      "layer16.attn",
+      "layer16.mlp",
+      "layer17.attn",
+      "layer17.mlp",
+      "layer18.attn",
+      "layer18.mlp",
+      "layer19.attn",
+      "layer19.mlp",
+      "layer20.attn",
+      "layer20.mlp",
+      "layer21.attn",
+      "layer21.mlp",
+      "layer22.attn",
+      "layer22.mlp",
+      "layer23.attn",
+      "layer23.mlp",
+      "layer24.attn",
+      "layer24.mlp",
+      "layer25.attn",
+      "layer25.mlp",
+      "layer26.attn",
+      "layer26.mlp",
+      "layer27.attn",
+      "layer27.mlp",
+      "layer28.attn",
+      "layer28.mlp",
+      "layer29.attn",
+      "layer29.mlp",
+      "layer30.attn",
+      "layer30.mlp",
+      "layer31.attn",
+      "layer31.mlp"
+    ],
+    "module_dimensions": [
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096
+    ],
+    "module_ranges": {
+      "layer00.attn": [
+        0,
+        4096
+      ],
+      "layer00.mlp": [
+        4096,
+        8192
+      ],
+      "layer01.attn": [
+        8192,
+        12288
+      ],
+      "layer01.mlp": [
+        12288,
+        16384
+      ],
+      "layer02.attn": [
+        16384,
+        20480
+      ],
+      "layer02.mlp": [
+        20480,
+        24576
+      ],
+      "layer03.attn": [
+        24576,
+        28672
+      ],
+      "layer03.mlp": [
+        28672,
+        32768
+      ],
+      "layer04.attn": [
+        32768,
+        36864
+      ],
+      "layer04.mlp": [
+        36864,
+        40960
+      ],
+      "layer05.attn": [
+        40960,
+        45056
+      ],
+      "layer05.mlp": [
+        45056,
+        49152
+      ],
+      "layer06.attn": [
+        49152,
+        53248
+      ],
+      "layer06.mlp": [
+        53248,
+        57344
+      ],
+      "layer07.attn": [
+        57344,
+        61440
+      ],
+      "layer07.mlp": [
+        61440,
+        65536
+      ],
+      "layer08.attn": [
+        65536,
+        69632
+      ],
+      "layer08.mlp": [
+        69632,
+        73728
+      ],
+      "layer09.attn": [
+        73728,
+        77824
+      ],
+      "layer09.mlp": [
+        77824,
+        81920
+      ],
+      "layer10.attn": [
+        81920,
+        86016
+      ],
+      "layer10.mlp": [
+        86016,
+        90112
+      ],
+      "layer11.attn": [
+        90112,
+        94208
+      ],
+      "layer11.mlp": [
+        94208,
+        98304
+      ],
+      "layer12.attn": [
+        98304,
+        102400
+      ],
+      "layer12.mlp": [
+        102400,
+        106496
+      ],
+      "layer13.attn": [
+        106496,
+        110592
+      ],
+      "layer13.mlp": [
+        110592,
+        114688
+      ],
+      "layer14.attn": [
+        114688,
+        118784
+      ],
+      "layer14.mlp": [
+        118784,
+        122880
+      ],
+      "layer15.attn": [
+        122880,
+        126976
+      ],
+      "layer15.mlp": [
+        126976,
+        131072
+      ],
+      "layer16.attn": [
+        131072,
+        135168
+      ],
+      "layer16.mlp": [
+        135168,
+        139264
+      ],
+      "layer17.attn": [
+        139264,
+        143360
+      ],
+      "layer17.mlp": [
+        143360,
+        147456
+      ],
+      "layer18.attn": [
+        147456,
+        151552
+      ],
+      "layer18.mlp": [
+        151552,
+        155648
+      ],
+      "layer19.attn": [
+        155648,
+        159744
+      ],
+      "layer19.mlp": [
+        159744,
+        163840
+      ],
+      "layer20.attn": [
+        163840,
+        167936
+      ],
+      "layer20.mlp": [
+        167936,
+        172032
+      ],
+      "layer21.attn": [
+        172032,
+        176128
+      ],
+      "layer21.mlp": [
+        176128,
+        180224
+      ],
+      "layer22.attn": [
+        180224,
+        184320
+      ],
+      "layer22.mlp": [
+        184320,
+        188416
+      ],
+      "layer23.attn": [
+        188416,
+        192512
+      ],
+      "layer23.mlp": [
+        192512,
+        196608
+      ],
+      "layer24.attn": [
+        196608,
+        200704
+      ],
+      "layer24.mlp": [
+        200704,
+        204800
+      ],
+      "layer25.attn": [
+        204800,
+        208896
+      ],
+      "layer25.mlp": [
+        208896,
+        212992
+      ],
+      "layer26.attn": [
+        212992,
+        217088
+      ],
+      "layer26.mlp": [
+        217088,
+        221184
+      ],
+      "layer27.attn": [
+        221184,
+        225280
+      ],
+      "layer27.mlp": [
+        225280,
+        229376
+      ],
+      "layer28.attn": [
+        229376,
+        233472
+      ],
+      "layer28.mlp": [
+        233472,
+        237568
+      ],
+      "layer29.attn": [
+        237568,
+        241664
+      ],
+      "layer29.mlp": [
+        241664,
+        245760
+      ],
+      "layer30.attn": [
+        245760,
+        249856
+      ],
+      "layer30.mlp": [
+        249856,
+        253952
+      ],
+      "layer31.attn": [
+        253952,
+        258048
+      ],
+      "layer31.mlp": [
+        258048,
+        262144
+      ]
+    },
+    "layer_count": 32,
+    "hidden_size": 4096,
+    "uniform_module_width": true,
+    "probe_blocks": [
+      [
+        0,
+        1,
+        2,
+        3,
+        4,
+        5,
+        6,
+        7,
+        8,
+        9,
+        10,
+        11,
+        12,
+        13,
+        14,
+        15
+      ]
+    ],
+    "gate_dtype": "float32",
+    "ungated_first_window_nll": 8.116129875183105,
+    "unit_gate_nll_error": 0.0,
+    "repeat_window_index": 0,
+    "repeat_vector_exact": true,
+    "repeat_vector_l2_error": 0.0,
+    "repeat_window_nll": 8.116129875183105,
+    "source_parameter_versions_unchanged": true,
+    "non_gate_trainable_parameters": 0,
+    "gates_removed_before_native_restore": true,
+    "windows": 16,
+    "module_graph_restored": true,
+    "raw_mean_norm": 1.3242629036422449
+  },
+  "optimization_steps": 0
+}

basis/gradients/q128/llama31/asvd_r080/step_0000/block_0/vectors.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:30b407fb074cbb73d8aab3186f0f4e6b953d109f24a1d4204e955ca8a8bf99ed
+size 17831065

basis/gradients/q128/llama31/asvd_r080/step_0000/block_1/summary.json

@@ -0,0 +1,995 @@
+{
+  "status": "complete",
+  "job_id": "llama31/asvd_r080",
+  "backbone": "llama31",
+  "step": 0,
+  "state_id": "6f1b8bc457a7ab3a928da64325d85a47efcb711c337504eb5b08a0597793b221",
+  "manifest_sha256": "d77eb1a1b9fd2801372ec781f1052b3ca276dffccfe20b34e3c7c44dcfed52a2",
+  "probe_tensor_sha256": "f19c31b43250bb8f5904e38874b21aca10fff6f736a76ada81e86121553f7e1b",
+  "evaluation_tensor_sha256": "736e3a4086111f686021cff0b3beaf1cd1ae4feae52bfd6719cdb3680a9e400d",
+  "block": 1,
+  "global_window_indices": [
+    16,
+    17,
+    18,
+    19,
+    20,
+    21,
+    22,
+    23,
+    24,
+    25,
+    26,
+    27,
+    28,
+    29,
+    30,
+    31
+  ],
+  "vectors": {
+    "path": "/home/zs89/Basis/hx_exp/outputs/confirmation/llama31/asvd_r080/step_0000/block_1/vectors.pt",
+    "sha256": "4c586fefd4492ec4ef20cc3767d7699d63634f7004629b71b03903707edab8f7"
+  },
+  "audit": {
+    "window_nll": [
+      8.030828475952148,
+      9.496520042419434,
+      8.715717315673828,
+      8.263007164001465,
+      6.266981601715088,
+      8.636651992797852,
+      7.717964172363281,
+      9.228141784667969,
+      8.48049259185791,
+      8.259269714355469,
+      8.278332710266113,
+      6.939949989318848,
+      7.611452102661133,
+      8.084861755371094,
+      8.405790328979492,
+      8.357336044311523
+    ],
+    "coordinate_count": 262144,
+    "module_records": [
+      {
+        "name": "layer00.attn",
+        "layer": 0,
+        "module_type": "attention",
+        "coordinate_dim": 4096,
+        "start": 0,
+        "stop": 4096
+      },
+      {
+        "name": "layer00.mlp",
+        "layer": 0,
+        "module_type": "MLP",
+        "coordinate_dim": 4096,
+        "start": 4096,
+        "stop": 8192
+      },
+      {
+        "name": "layer01.attn",
+        "layer": 1,
+        "module_type": "attention",
+        "coordinate_dim": 4096,
+        "start": 8192,
+        "stop": 12288
+      },
+      {
+        "name": "layer01.mlp",
+        "layer": 1,
+        "module_type": "MLP",
+        "coordinate_dim": 4096,
+        "start": 12288,
+        "stop": 16384
+      },
+      {
+        "name": "layer02.attn",
+        "layer": 2,
+        "module_type": "attention",
+        "coordinate_dim": 4096,
+        "start": 16384,
+        "stop": 20480
+      },
+      {
+        "name": "layer02.mlp",
+        "layer": 2,
+        "module_type": "MLP",
+        "coordinate_dim": 4096,
+        "start": 20480,
+        "stop": 24576
+      },
+      {
+        "name": "layer03.attn",
+        "layer": 3,
+        "module_type": "attention",
+        "coordinate_dim": 4096,
+        "start": 24576,
+        "stop": 28672
+      },
+      {
+        "name": "layer03.mlp",
+        "layer": 3,
+        "module_type": "MLP",
+        "coordinate_dim": 4096,
+        "start": 28672,
+        "stop": 32768
+      },
+      {
+        "name": "layer04.attn",
+        "layer": 4,
+        "module_type": "attention",
+        "coordinate_dim": 4096,
+        "start": 32768,
+        "stop": 36864
+      },
+      {
+        "name": "layer04.mlp",
+        "layer": 4,
+        "module_type": "MLP",
+        "coordinate_dim": 4096,
+        "start": 36864,
+        "stop": 40960
+      },
+      {
+        "name": "layer05.attn",
+        "layer": 5,
+        "module_type": "attention",
+        "coordinate_dim": 4096,
+        "start": 40960,
+        "stop": 45056
+      },
+      {
+        "name": "layer05.mlp",
+        "layer": 5,
+        "module_type": "MLP",
+        "coordinate_dim": 4096,
+        "start": 45056,
+        "stop": 49152
+      },
+      {
+        "name": "layer06.attn",
+        "layer": 6,
+        "module_type": "attention",
+        "coordinate_dim": 4096,
+        "start": 49152,
+        "stop": 53248
+      },
+      {
+        "name": "layer06.mlp",
+        "layer": 6,
+        "module_type": "MLP",
+        "coordinate_dim": 4096,
+        "start": 53248,
+        "stop": 57344
+      },
+      {
+        "name": "layer07.attn",
+        "layer": 7,
+        "module_type": "attention",
+        "coordinate_dim": 4096,
+        "start": 57344,
+        "stop": 61440
+      },
+      {
+        "name": "layer07.mlp",
+        "layer": 7,
+        "module_type": "MLP",
+        "coordinate_dim": 4096,
+        "start": 61440,
+        "stop": 65536
+      },
+      {
+        "name": "layer08.attn",
+        "layer": 8,
+        "module_type": "attention",
+        "coordinate_dim": 4096,
+        "start": 65536,
+        "stop": 69632
+      },
+      {
+        "name": "layer08.mlp",
+        "layer": 8,
+        "module_type": "MLP",
+        "coordinate_dim": 4096,
+        "start": 69632,
+        "stop": 73728
+      },
+      {
+        "name": "layer09.attn",
+        "layer": 9,
+        "module_type": "attention",
+        "coordinate_dim": 4096,
+        "start": 73728,
+        "stop": 77824
+      },
+      {
+        "name": "layer09.mlp",
+        "layer": 9,
+        "module_type": "MLP",
+        "coordinate_dim": 4096,
+        "start": 77824,
+        "stop": 81920
+      },
+      {
+        "name": "layer10.attn",
+        "layer": 10,
+        "module_type": "attention",
+        "coordinate_dim": 4096,
+        "start": 81920,
+        "stop": 86016
+      },
+      {
+        "name": "layer10.mlp",
+        "layer": 10,
+        "module_type": "MLP",
+        "coordinate_dim": 4096,
+        "start": 86016,
+        "stop": 90112
+      },
+      {
+        "name": "layer11.attn",
+        "layer": 11,
+        "module_type": "attention",
+        "coordinate_dim": 4096,
+        "start": 90112,
+        "stop": 94208
+      },
+      {
+        "name": "layer11.mlp",
+        "layer": 11,
+        "module_type": "MLP",
+        "coordinate_dim": 4096,
+        "start": 94208,
+        "stop": 98304
+      },
+      {
+        "name": "layer12.attn",
+        "layer": 12,
+        "module_type": "attention",
+        "coordinate_dim": 4096,
+        "start": 98304,
+        "stop": 102400
+      },
+      {
+        "name": "layer12.mlp",
+        "layer": 12,
+        "module_type": "MLP",
+        "coordinate_dim": 4096,
+        "start": 102400,
+        "stop": 106496
+      },
+      {
+        "name": "layer13.attn",
+        "layer": 13,
+        "module_type": "attention",
+        "coordinate_dim": 4096,
+        "start": 106496,
+        "stop": 110592
+      },
+      {
+        "name": "layer13.mlp",
+        "layer": 13,
+        "module_type": "MLP",
+        "coordinate_dim": 4096,
+        "start": 110592,
+        "stop": 114688
+      },
+      {
+        "name": "layer14.attn",
+        "layer": 14,
+        "module_type": "attention",
+        "coordinate_dim": 4096,
+        "start": 114688,
+        "stop": 118784
+      },
+      {
+        "name": "layer14.mlp",
+        "layer": 14,
+        "module_type": "MLP",
+        "coordinate_dim": 4096,
+        "start": 118784,
+        "stop": 122880
+      },
+      {
+        "name": "layer15.attn",
+        "layer": 15,
+        "module_type": "attention",
+        "coordinate_dim": 4096,
+        "start": 122880,
+        "stop": 126976
+      },
+      {
+        "name": "layer15.mlp",
+        "layer": 15,
+        "module_type": "MLP",
+        "coordinate_dim": 4096,
+        "start": 126976,
+        "stop": 131072
+      },
+      {
+        "name": "layer16.attn",
+        "layer": 16,
+        "module_type": "attention",
+        "coordinate_dim": 4096,
+        "start": 131072,
+        "stop": 135168
+      },
+      {
+        "name": "layer16.mlp",
+        "layer": 16,
+        "module_type": "MLP",
+        "coordinate_dim": 4096,
+        "start": 135168,
+        "stop": 139264
+      },
+      {
+        "name": "layer17.attn",
+        "layer": 17,
+        "module_type": "attention",
+        "coordinate_dim": 4096,
+        "start": 139264,
+        "stop": 143360
+      },
+      {
+        "name": "layer17.mlp",
+        "layer": 17,
+        "module_type": "MLP",
+        "coordinate_dim": 4096,
+        "start": 143360,
+        "stop": 147456
+      },
+      {
+        "name": "layer18.attn",
+        "layer": 18,
+        "module_type": "attention",
+        "coordinate_dim": 4096,
+        "start": 147456,
+        "stop": 151552
+      },
+      {
+        "name": "layer18.mlp",
+        "layer": 18,
+        "module_type": "MLP",
+        "coordinate_dim": 4096,
+        "start": 151552,
+        "stop": 155648
+      },
+      {
+        "name": "layer19.attn",
+        "layer": 19,
+        "module_type": "attention",
+        "coordinate_dim": 4096,
+        "start": 155648,
+        "stop": 159744
+      },
+      {
+        "name": "layer19.mlp",
+        "layer": 19,
+        "module_type": "MLP",
+        "coordinate_dim": 4096,
+        "start": 159744,
+        "stop": 163840
+      },
+      {
+        "name": "layer20.attn",
+        "layer": 20,
+        "module_type": "attention",
+        "coordinate_dim": 4096,
+        "start": 163840,
+        "stop": 167936
+      },
+      {
+        "name": "layer20.mlp",
+        "layer": 20,
+        "module_type": "MLP",
+        "coordinate_dim": 4096,
+        "start": 167936,
+        "stop": 172032
+      },
+      {
+        "name": "layer21.attn",
+        "layer": 21,
+        "module_type": "attention",
+        "coordinate_dim": 4096,
+        "start": 172032,
+        "stop": 176128
+      },
+      {
+        "name": "layer21.mlp",
+        "layer": 21,
+        "module_type": "MLP",
+        "coordinate_dim": 4096,
+        "start": 176128,
+        "stop": 180224
+      },
+      {
+        "name": "layer22.attn",
+        "layer": 22,
+        "module_type": "attention",
+        "coordinate_dim": 4096,
+        "start": 180224,
+        "stop": 184320
+      },
+      {
+        "name": "layer22.mlp",
+        "layer": 22,
+        "module_type": "MLP",
+        "coordinate_dim": 4096,
+        "start": 184320,
+        "stop": 188416
+      },
+      {
+        "name": "layer23.attn",
+        "layer": 23,
+        "module_type": "attention",
+        "coordinate_dim": 4096,
+        "start": 188416,
+        "stop": 192512
+      },
+      {
+        "name": "layer23.mlp",
+        "layer": 23,
+        "module_type": "MLP",
+        "coordinate_dim": 4096,
+        "start": 192512,
+        "stop": 196608
+      },
+      {
+        "name": "layer24.attn",
+        "layer": 24,
+        "module_type": "attention",
+        "coordinate_dim": 4096,
+        "start": 196608,
+        "stop": 200704
+      },
+      {
+        "name": "layer24.mlp",
+        "layer": 24,
+        "module_type": "MLP",
+        "coordinate_dim": 4096,
+        "start": 200704,
+        "stop": 204800
+      },
+      {
+        "name": "layer25.attn",
+        "layer": 25,
+        "module_type": "attention",
+        "coordinate_dim": 4096,
+        "start": 204800,
+        "stop": 208896
+      },
+      {
+        "name": "layer25.mlp",
+        "layer": 25,
+        "module_type": "MLP",
+        "coordinate_dim": 4096,
+        "start": 208896,
+        "stop": 212992
+      },
+      {
+        "name": "layer26.attn",
+        "layer": 26,
+        "module_type": "attention",
+        "coordinate_dim": 4096,
+        "start": 212992,
+        "stop": 217088
+      },
+      {
+        "name": "layer26.mlp",
+        "layer": 26,
+        "module_type": "MLP",
+        "coordinate_dim": 4096,
+        "start": 217088,
+        "stop": 221184
+      },
+      {
+        "name": "layer27.attn",
+        "layer": 27,
+        "module_type": "attention",
+        "coordinate_dim": 4096,
+        "start": 221184,
+        "stop": 225280
+      },
+      {
+        "name": "layer27.mlp",
+        "layer": 27,
+        "module_type": "MLP",
+        "coordinate_dim": 4096,
+        "start": 225280,
+        "stop": 229376
+      },
+      {
+        "name": "layer28.attn",
+        "layer": 28,
+        "module_type": "attention",
+        "coordinate_dim": 4096,
+        "start": 229376,
+        "stop": 233472
+      },
+      {
+        "name": "layer28.mlp",
+        "layer": 28,
+        "module_type": "MLP",
+        "coordinate_dim": 4096,
+        "start": 233472,
+        "stop": 237568
+      },
+      {
+        "name": "layer29.attn",
+        "layer": 29,
+        "module_type": "attention",
+        "coordinate_dim": 4096,
+        "start": 237568,
+        "stop": 241664
+      },
+      {
+        "name": "layer29.mlp",
+        "layer": 29,
+        "module_type": "MLP",
+        "coordinate_dim": 4096,
+        "start": 241664,
+        "stop": 245760
+      },
+      {
+        "name": "layer30.attn",
+        "layer": 30,
+        "module_type": "attention",
+        "coordinate_dim": 4096,
+        "start": 245760,
+        "stop": 249856
+      },
+      {
+        "name": "layer30.mlp",
+        "layer": 30,
+        "module_type": "MLP",
+        "coordinate_dim": 4096,
+        "start": 249856,
+        "stop": 253952
+      },
+      {
+        "name": "layer31.attn",
+        "layer": 31,
+        "module_type": "attention",
+        "coordinate_dim": 4096,
+        "start": 253952,
+        "stop": 258048
+      },
+      {
+        "name": "layer31.mlp",
+        "layer": 31,
+        "module_type": "MLP",
+        "coordinate_dim": 4096,
+        "start": 258048,
+        "stop": 262144
+      }
+    ],
+    "module_names": [
+      "layer00.attn",
+      "layer00.mlp",
+      "layer01.attn",
+      "layer01.mlp",
+      "layer02.attn",
+      "layer02.mlp",
+      "layer03.attn",
+      "layer03.mlp",
+      "layer04.attn",
+      "layer04.mlp",
+      "layer05.attn",
+      "layer05.mlp",
+      "layer06.attn",
+      "layer06.mlp",
+      "layer07.attn",
+      "layer07.mlp",
+      "layer08.attn",
+      "layer08.mlp",
+      "layer09.attn",
+      "layer09.mlp",
+      "layer10.attn",
+      "layer10.mlp",
+      "layer11.attn",
+      "layer11.mlp",
+      "layer12.attn",
+      "layer12.mlp",
+      "layer13.attn",
+      "layer13.mlp",
+      "layer14.attn",
+      "layer14.mlp",
+      "layer15.attn",
+      "layer15.mlp",
+      "layer16.attn",
+      "layer16.mlp",
+      "layer17.attn",
+      "layer17.mlp",
+      "layer18.attn",
+      "layer18.mlp",
+      "layer19.attn",
+      "layer19.mlp",
+      "layer20.attn",
+      "layer20.mlp",
+      "layer21.attn",
+      "layer21.mlp",
+      "layer22.attn",
+      "layer22.mlp",
+      "layer23.attn",
+      "layer23.mlp",
+      "layer24.attn",
+      "layer24.mlp",
+      "layer25.attn",
+      "layer25.mlp",
+      "layer26.attn",
+      "layer26.mlp",
+      "layer27.attn",
+      "layer27.mlp",
+      "layer28.attn",
+      "layer28.mlp",
+      "layer29.attn",
+      "layer29.mlp",
+      "layer30.attn",
+      "layer30.mlp",
+      "layer31.attn",
+      "layer31.mlp"
+    ],
+    "module_dimensions": [
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096
+    ],
+    "module_ranges": {
+      "layer00.attn": [
+        0,
+        4096
+      ],
+      "layer00.mlp": [
+        4096,
+        8192
+      ],
+      "layer01.attn": [
+        8192,
+        12288
+      ],
+      "layer01.mlp": [
+        12288,
+        16384
+      ],
+      "layer02.attn": [
+        16384,
+        20480
+      ],
+      "layer02.mlp": [
+        20480,
+        24576
+      ],
+      "layer03.attn": [
+        24576,
+        28672
+      ],
+      "layer03.mlp": [
+        28672,
+        32768
+      ],
+      "layer04.attn": [
+        32768,
+        36864
+      ],
+      "layer04.mlp": [
+        36864,
+        40960
+      ],
+      "layer05.attn": [
+        40960,
+        45056
+      ],
+      "layer05.mlp": [
+        45056,
+        49152
+      ],
+      "layer06.attn": [
+        49152,
+        53248
+      ],
+      "layer06.mlp": [
+        53248,
+        57344
+      ],
+      "layer07.attn": [
+        57344,
+        61440
+      ],
+      "layer07.mlp": [
+        61440,
+        65536
+      ],
+      "layer08.attn": [
+        65536,
+        69632
+      ],
+      "layer08.mlp": [
+        69632,
+        73728
+      ],
+      "layer09.attn": [
+        73728,
+        77824
+      ],
+      "layer09.mlp": [
+        77824,
+        81920
+      ],
+      "layer10.attn": [
+        81920,
+        86016
+      ],
+      "layer10.mlp": [
+        86016,
+        90112
+      ],
+      "layer11.attn": [
+        90112,
+        94208
+      ],
+      "layer11.mlp": [
+        94208,
+        98304
+      ],
+      "layer12.attn": [
+        98304,
+        102400
+      ],
+      "layer12.mlp": [
+        102400,
+        106496
+      ],
+      "layer13.attn": [
+        106496,
+        110592
+      ],
+      "layer13.mlp": [
+        110592,
+        114688
+      ],
+      "layer14.attn": [
+        114688,
+        118784
+      ],
+      "layer14.mlp": [
+        118784,
+        122880
+      ],
+      "layer15.attn": [
+        122880,
+        126976
+      ],
+      "layer15.mlp": [
+        126976,
+        131072
+      ],
+      "layer16.attn": [
+        131072,
+        135168
+      ],
+      "layer16.mlp": [
+        135168,
+        139264
+      ],
+      "layer17.attn": [
+        139264,
+        143360
+      ],
+      "layer17.mlp": [
+        143360,
+        147456
+      ],
+      "layer18.attn": [
+        147456,
+        151552
+      ],
+      "layer18.mlp": [
+        151552,
+        155648
+      ],
+      "layer19.attn": [
+        155648,
+        159744
+      ],
+      "layer19.mlp": [
+        159744,
+        163840
+      ],
+      "layer20.attn": [
+        163840,
+        167936
+      ],
+      "layer20.mlp": [
+        167936,
+        172032
+      ],
+      "layer21.attn": [
+        172032,
+        176128
+      ],
+      "layer21.mlp": [
+        176128,
+        180224
+      ],
+      "layer22.attn": [
+        180224,
+        184320
+      ],
+      "layer22.mlp": [
+        184320,
+        188416
+      ],
+      "layer23.attn": [
+        188416,
+        192512
+      ],
+      "layer23.mlp": [
+        192512,
+        196608
+      ],
+      "layer24.attn": [
+        196608,
+        200704
+      ],
+      "layer24.mlp": [
+        200704,
+        204800
+      ],
+      "layer25.attn": [
+        204800,
+        208896
+      ],
+      "layer25.mlp": [
+        208896,
+        212992
+      ],
+      "layer26.attn": [
+        212992,
+        217088
+      ],
+      "layer26.mlp": [
+        217088,
+        221184
+      ],
+      "layer27.attn": [
+        221184,
+        225280
+      ],
+      "layer27.mlp": [
+        225280,
+        229376
+      ],
+      "layer28.attn": [
+        229376,
+        233472
+      ],
+      "layer28.mlp": [
+        233472,
+        237568
+      ],
+      "layer29.attn": [
+        237568,
+        241664
+      ],
+      "layer29.mlp": [
+        241664,
+        245760
+      ],
+      "layer30.attn": [
+        245760,
+        249856
+      ],
+      "layer30.mlp": [
+        249856,
+        253952
+      ],
+      "layer31.attn": [
+        253952,
+        258048
+      ],
+      "layer31.mlp": [
+        258048,
+        262144
+      ]
+    },
+    "layer_count": 32,
+    "hidden_size": 4096,
+    "uniform_module_width": true,
+    "probe_blocks": [
+      [
+        0,
+        1,
+        2,
+        3,
+        4,
+        5,
+        6,
+        7,
+        8,
+        9,
+        10,
+        11,
+        12,
+        13,
+        14,
+        15
+      ]
+    ],
+    "gate_dtype": "float32",
+    "ungated_first_window_nll": 8.030828475952148,
+    "unit_gate_nll_error": 0.0,
+    "repeat_window_index": 0,
+    "repeat_vector_exact": true,
+    "repeat_vector_l2_error": 0.0,
+    "repeat_window_nll": 8.030828475952148,
+    "source_parameter_versions_unchanged": true,
+    "non_gate_trainable_parameters": 0,
+    "gates_removed_before_native_restore": true,
+    "windows": 16,
+    "module_graph_restored": true,
+    "raw_mean_norm": 1.5504645437273508
+  },
+  "optimization_steps": 0
+}

basis/gradients/q128/llama31/asvd_r080/step_0000/block_1/vectors.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4c586fefd4492ec4ef20cc3767d7699d63634f7004629b71b03903707edab8f7
+size 17831065

basis/gradients/q128/llama31/asvd_r080/step_0000/block_2/summary.json

@@ -0,0 +1,995 @@
+{
+  "status": "complete",
+  "job_id": "llama31/asvd_r080",
+  "backbone": "llama31",
+  "step": 0,
+  "state_id": "6f1b8bc457a7ab3a928da64325d85a47efcb711c337504eb5b08a0597793b221",
+  "manifest_sha256": "d77eb1a1b9fd2801372ec781f1052b3ca276dffccfe20b34e3c7c44dcfed52a2",
+  "probe_tensor_sha256": "f19c31b43250bb8f5904e38874b21aca10fff6f736a76ada81e86121553f7e1b",
+  "evaluation_tensor_sha256": "736e3a4086111f686021cff0b3beaf1cd1ae4feae52bfd6719cdb3680a9e400d",
+  "block": 2,
+  "global_window_indices": [
+    32,
+    33,
+    34,
+    35,
+    36,
+    37,
+    38,
+    39,
+    40,
+    41,
+    42,
+    43,
+    44,
+    45,
+    46,
+    47
+  ],
+  "vectors": {
+    "path": "/home/zs89/Basis/hx_exp/outputs/confirmation/llama31/asvd_r080/step_0000/block_2/vectors.pt",
+    "sha256": "0c3775aef0f0aea760ac1202527504fb710a5aa9512ca5fcb1e7f94b0832e42f"
+  },
+  "audit": {
+    "window_nll": [
+      9.359926223754883,
+      8.008647918701172,
+      9.712199211120605,
+      10.204842567443848,
+      6.498856067657471,
+      8.299304008483887,
+      7.603578090667725,
+      8.236316680908203,
+      8.487428665161133,
+      7.834820747375488,
+      8.098539352416992,
+      8.343332290649414,
+      8.432135581970215,
+      7.436379909515381,
+      9.0822172164917,
+      8.146020889282227
+    ],
+    "coordinate_count": 262144,
+    "module_records": [
+      {
+        "name": "layer00.attn",
+        "layer": 0,
+        "module_type": "attention",
+        "coordinate_dim": 4096,
+        "start": 0,
+        "stop": 4096
+      },
+      {
+        "name": "layer00.mlp",
+        "layer": 0,
+        "module_type": "MLP",
+        "coordinate_dim": 4096,
+        "start": 4096,
+        "stop": 8192
+      },
+      {
+        "name": "layer01.attn",
+        "layer": 1,
+        "module_type": "attention",
+        "coordinate_dim": 4096,
+        "start": 8192,
+        "stop": 12288
+      },
+      {
+        "name": "layer01.mlp",
+        "layer": 1,
+        "module_type": "MLP",
+        "coordinate_dim": 4096,
+        "start": 12288,
+        "stop": 16384
+      },
+      {
+        "name": "layer02.attn",
+        "layer": 2,
+        "module_type": "attention",
+        "coordinate_dim": 4096,
+        "start": 16384,
+        "stop": 20480
+      },
+      {
+        "name": "layer02.mlp",
+        "layer": 2,
+        "module_type": "MLP",
+        "coordinate_dim": 4096,
+        "start": 20480,
+        "stop": 24576
+      },
+      {
+        "name": "layer03.attn",
+        "layer": 3,
+        "module_type": "attention",
+        "coordinate_dim": 4096,
+        "start": 24576,
+        "stop": 28672
+      },
+      {
+        "name": "layer03.mlp",
+        "layer": 3,
+        "module_type": "MLP",
+        "coordinate_dim": 4096,
+        "start": 28672,
+        "stop": 32768
+      },
+      {
+        "name": "layer04.attn",
+        "layer": 4,
+        "module_type": "attention",
+        "coordinate_dim": 4096,
+        "start": 32768,
+        "stop": 36864
+      },
+      {
+        "name": "layer04.mlp",
+        "layer": 4,
+        "module_type": "MLP",
+        "coordinate_dim": 4096,
+        "start": 36864,
+        "stop": 40960
+      },
+      {
+        "name": "layer05.attn",
+        "layer": 5,
+        "module_type": "attention",
+        "coordinate_dim": 4096,
+        "start": 40960,
+        "stop": 45056
+      },
+      {
+        "name": "layer05.mlp",
+        "layer": 5,
+        "module_type": "MLP",
+        "coordinate_dim": 4096,
+        "start": 45056,
+        "stop": 49152
+      },
+      {
+        "name": "layer06.attn",
+        "layer": 6,
+        "module_type": "attention",
+        "coordinate_dim": 4096,
+        "start": 49152,
+        "stop": 53248
+      },
+      {
+        "name": "layer06.mlp",
+        "layer": 6,
+        "module_type": "MLP",
+        "coordinate_dim": 4096,
+        "start": 53248,
+        "stop": 57344
+      },
+      {
+        "name": "layer07.attn",
+        "layer": 7,
+        "module_type": "attention",
+        "coordinate_dim": 4096,
+        "start": 57344,
+        "stop": 61440
+      },
+      {
+        "name": "layer07.mlp",
+        "layer": 7,
+        "module_type": "MLP",
+        "coordinate_dim": 4096,
+        "start": 61440,
+        "stop": 65536
+      },
+      {
+        "name": "layer08.attn",
+        "layer": 8,
+        "module_type": "attention",
+        "coordinate_dim": 4096,
+        "start": 65536,
+        "stop": 69632
+      },
+      {
+        "name": "layer08.mlp",
+        "layer": 8,
+        "module_type": "MLP",
+        "coordinate_dim": 4096,
+        "start": 69632,
+        "stop": 73728
+      },
+      {
+        "name": "layer09.attn",
+        "layer": 9,
+        "module_type": "attention",
+        "coordinate_dim": 4096,
+        "start": 73728,
+        "stop": 77824
+      },
+      {
+        "name": "layer09.mlp",
+        "layer": 9,
+        "module_type": "MLP",
+        "coordinate_dim": 4096,
+        "start": 77824,
+        "stop": 81920
+      },
+      {
+        "name": "layer10.attn",
+        "layer": 10,
+        "module_type": "attention",
+        "coordinate_dim": 4096,
+        "start": 81920,
+        "stop": 86016
+      },
+      {
+        "name": "layer10.mlp",
+        "layer": 10,
+        "module_type": "MLP",
+        "coordinate_dim": 4096,
+        "start": 86016,
+        "stop": 90112
+      },
+      {
+        "name": "layer11.attn",
+        "layer": 11,
+        "module_type": "attention",
+        "coordinate_dim": 4096,
+        "start": 90112,
+        "stop": 94208
+      },
+      {
+        "name": "layer11.mlp",
+        "layer": 11,
+        "module_type": "MLP",
+        "coordinate_dim": 4096,
+        "start": 94208,
+        "stop": 98304
+      },
+      {
+        "name": "layer12.attn",
+        "layer": 12,
+        "module_type": "attention",
+        "coordinate_dim": 4096,
+        "start": 98304,
+        "stop": 102400
+      },
+      {
+        "name": "layer12.mlp",
+        "layer": 12,
+        "module_type": "MLP",
+        "coordinate_dim": 4096,
+        "start": 102400,
+        "stop": 106496
+      },
+      {
+        "name": "layer13.attn",
+        "layer": 13,
+        "module_type": "attention",
+        "coordinate_dim": 4096,
+        "start": 106496,
+        "stop": 110592
+      },
+      {
+        "name": "layer13.mlp",
+        "layer": 13,
+        "module_type": "MLP",
+        "coordinate_dim": 4096,
+        "start": 110592,
+        "stop": 114688
+      },
+      {
+        "name": "layer14.attn",
+        "layer": 14,
+        "module_type": "attention",
+        "coordinate_dim": 4096,
+        "start": 114688,
+        "stop": 118784
+      },
+      {
+        "name": "layer14.mlp",
+        "layer": 14,
+        "module_type": "MLP",
+        "coordinate_dim": 4096,
+        "start": 118784,
+        "stop": 122880
+      },
+      {
+        "name": "layer15.attn",
+        "layer": 15,
+        "module_type": "attention",
+        "coordinate_dim": 4096,
+        "start": 122880,
+        "stop": 126976
+      },
+      {
+        "name": "layer15.mlp",
+        "layer": 15,
+        "module_type": "MLP",
+        "coordinate_dim": 4096,
+        "start": 126976,
+        "stop": 131072
+      },
+      {
+        "name": "layer16.attn",
+        "layer": 16,
+        "module_type": "attention",
+        "coordinate_dim": 4096,
+        "start": 131072,
+        "stop": 135168
+      },
+      {
+        "name": "layer16.mlp",
+        "layer": 16,
+        "module_type": "MLP",
+        "coordinate_dim": 4096,
+        "start": 135168,
+        "stop": 139264
+      },
+      {
+        "name": "layer17.attn",
+        "layer": 17,
+        "module_type": "attention",
+        "coordinate_dim": 4096,
+        "start": 139264,
+        "stop": 143360
+      },
+      {
+        "name": "layer17.mlp",
+        "layer": 17,
+        "module_type": "MLP",
+        "coordinate_dim": 4096,
+        "start": 143360,
+        "stop": 147456
+      },
+      {
+        "name": "layer18.attn",
+        "layer": 18,
+        "module_type": "attention",
+        "coordinate_dim": 4096,
+        "start": 147456,
+        "stop": 151552
+      },
+      {
+        "name": "layer18.mlp",
+        "layer": 18,
+        "module_type": "MLP",
+        "coordinate_dim": 4096,
+        "start": 151552,
+        "stop": 155648
+      },
+      {
+        "name": "layer19.attn",
+        "layer": 19,
+        "module_type": "attention",
+        "coordinate_dim": 4096,
+        "start": 155648,
+        "stop": 159744
+      },
+      {
+        "name": "layer19.mlp",
+        "layer": 19,
+        "module_type": "MLP",
+        "coordinate_dim": 4096,
+        "start": 159744,
+        "stop": 163840
+      },
+      {
+        "name": "layer20.attn",
+        "layer": 20,
+        "module_type": "attention",
+        "coordinate_dim": 4096,
+        "start": 163840,
+        "stop": 167936
+      },
+      {
+        "name": "layer20.mlp",
+        "layer": 20,
+        "module_type": "MLP",
+        "coordinate_dim": 4096,
+        "start": 167936,
+        "stop": 172032
+      },
+      {
+        "name": "layer21.attn",
+        "layer": 21,
+        "module_type": "attention",
+        "coordinate_dim": 4096,
+        "start": 172032,
+        "stop": 176128
+      },
+      {
+        "name": "layer21.mlp",
+        "layer": 21,
+        "module_type": "MLP",
+        "coordinate_dim": 4096,
+        "start": 176128,
+        "stop": 180224
+      },
+      {
+        "name": "layer22.attn",
+        "layer": 22,
+        "module_type": "attention",
+        "coordinate_dim": 4096,
+        "start": 180224,
+        "stop": 184320
+      },
+      {
+        "name": "layer22.mlp",
+        "layer": 22,
+        "module_type": "MLP",
+        "coordinate_dim": 4096,
+        "start": 184320,
+        "stop": 188416
+      },
+      {
+        "name": "layer23.attn",
+        "layer": 23,
+        "module_type": "attention",
+        "coordinate_dim": 4096,
+        "start": 188416,
+        "stop": 192512
+      },
+      {
+        "name": "layer23.mlp",
+        "layer": 23,
+        "module_type": "MLP",
+        "coordinate_dim": 4096,
+        "start": 192512,
+        "stop": 196608
+      },
+      {
+        "name": "layer24.attn",
+        "layer": 24,
+        "module_type": "attention",
+        "coordinate_dim": 4096,
+        "start": 196608,
+        "stop": 200704
+      },
+      {
+        "name": "layer24.mlp",
+        "layer": 24,
+        "module_type": "MLP",
+        "coordinate_dim": 4096,
+        "start": 200704,
+        "stop": 204800
+      },
+      {
+        "name": "layer25.attn",
+        "layer": 25,
+        "module_type": "attention",
+        "coordinate_dim": 4096,
+        "start": 204800,
+        "stop": 208896
+      },
+      {
+        "name": "layer25.mlp",
+        "layer": 25,
+        "module_type": "MLP",
+        "coordinate_dim": 4096,
+        "start": 208896,
+        "stop": 212992
+      },
+      {
+        "name": "layer26.attn",
+        "layer": 26,
+        "module_type": "attention",
+        "coordinate_dim": 4096,
+        "start": 212992,
+        "stop": 217088
+      },
+      {
+        "name": "layer26.mlp",
+        "layer": 26,
+        "module_type": "MLP",
+        "coordinate_dim": 4096,
+        "start": 217088,
+        "stop": 221184
+      },
+      {
+        "name": "layer27.attn",
+        "layer": 27,
+        "module_type": "attention",
+        "coordinate_dim": 4096,
+        "start": 221184,
+        "stop": 225280
+      },
+      {
+        "name": "layer27.mlp",
+        "layer": 27,
+        "module_type": "MLP",
+        "coordinate_dim": 4096,
+        "start": 225280,
+        "stop": 229376
+      },
+      {
+        "name": "layer28.attn",
+        "layer": 28,
+        "module_type": "attention",
+        "coordinate_dim": 4096,
+        "start": 229376,
+        "stop": 233472
+      },
+      {
+        "name": "layer28.mlp",
+        "layer": 28,
+        "module_type": "MLP",
+        "coordinate_dim": 4096,
+        "start": 233472,
+        "stop": 237568
+      },
+      {
+        "name": "layer29.attn",
+        "layer": 29,
+        "module_type": "attention",
+        "coordinate_dim": 4096,
+        "start": 237568,
+        "stop": 241664
+      },
+      {
+        "name": "layer29.mlp",
+        "layer": 29,
+        "module_type": "MLP",
+        "coordinate_dim": 4096,
+        "start": 241664,
+        "stop": 245760
+      },
+      {
+        "name": "layer30.attn",
+        "layer": 30,
+        "module_type": "attention",
+        "coordinate_dim": 4096,
+        "start": 245760,
+        "stop": 249856
+      },
+      {
+        "name": "layer30.mlp",
+        "layer": 30,
+        "module_type": "MLP",
+        "coordinate_dim": 4096,
+        "start": 249856,
+        "stop": 253952
+      },
+      {
+        "name": "layer31.attn",
+        "layer": 31,
+        "module_type": "attention",
+        "coordinate_dim": 4096,
+        "start": 253952,
+        "stop": 258048
+      },
+      {
+        "name": "layer31.mlp",
+        "layer": 31,
+        "module_type": "MLP",
+        "coordinate_dim": 4096,
+        "start": 258048,
+        "stop": 262144
+      }
+    ],
+    "module_names": [
+      "layer00.attn",
+      "layer00.mlp",
+      "layer01.attn",
+      "layer01.mlp",
+      "layer02.attn",
+      "layer02.mlp",
+      "layer03.attn",
+      "layer03.mlp",
+      "layer04.attn",
+      "layer04.mlp",
+      "layer05.attn",
+      "layer05.mlp",
+      "layer06.attn",
+      "layer06.mlp",
+      "layer07.attn",
+      "layer07.mlp",
+      "layer08.attn",
+      "layer08.mlp",
+      "layer09.attn",
+      "layer09.mlp",
+      "layer10.attn",
+      "layer10.mlp",
+      "layer11.attn",
+      "layer11.mlp",
+      "layer12.attn",
+      "layer12.mlp",
+      "layer13.attn",
+      "layer13.mlp",
+      "layer14.attn",
+      "layer14.mlp",
+      "layer15.attn",
+      "layer15.mlp",
+      "layer16.attn",
+      "layer16.mlp",
+      "layer17.attn",
+      "layer17.mlp",
+      "layer18.attn",
+      "layer18.mlp",
+      "layer19.attn",
+      "layer19.mlp",
+      "layer20.attn",
+      "layer20.mlp",
+      "layer21.attn",
+      "layer21.mlp",
+      "layer22.attn",
+      "layer22.mlp",
+      "layer23.attn",
+      "layer23.mlp",
+      "layer24.attn",
+      "layer24.mlp",
+      "layer25.attn",
+      "layer25.mlp",
+      "layer26.attn",
+      "layer26.mlp",
+      "layer27.attn",
+      "layer27.mlp",
+      "layer28.attn",
+      "layer28.mlp",
+      "layer29.attn",
+      "layer29.mlp",
+      "layer30.attn",
+      "layer30.mlp",
+      "layer31.attn",
+      "layer31.mlp"
+    ],
+    "module_dimensions": [
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096
+    ],
+    "module_ranges": {
+      "layer00.attn": [
+        0,
+        4096
+      ],
+      "layer00.mlp": [
+        4096,
+        8192
+      ],
+      "layer01.attn": [
+        8192,
+        12288
+      ],
+      "layer01.mlp": [
+        12288,
+        16384
+      ],
+      "layer02.attn": [
+        16384,
+        20480
+      ],
+      "layer02.mlp": [
+        20480,
+        24576
+      ],
+      "layer03.attn": [
+        24576,
+        28672
+      ],
+      "layer03.mlp": [
+        28672,
+        32768
+      ],
+      "layer04.attn": [
+        32768,
+        36864
+      ],
+      "layer04.mlp": [
+        36864,
+        40960
+      ],
+      "layer05.attn": [
+        40960,
+        45056
+      ],
+      "layer05.mlp": [
+        45056,
+        49152
+      ],
+      "layer06.attn": [
+        49152,
+        53248
+      ],
+      "layer06.mlp": [
+        53248,
+        57344
+      ],
+      "layer07.attn": [
+        57344,
+        61440
+      ],
+      "layer07.mlp": [
+        61440,
+        65536
+      ],
+      "layer08.attn": [
+        65536,
+        69632
+      ],
+      "layer08.mlp": [
+        69632,
+        73728
+      ],
+      "layer09.attn": [
+        73728,
+        77824
+      ],
+      "layer09.mlp": [
+        77824,
+        81920
+      ],
+      "layer10.attn": [
+        81920,
+        86016
+      ],
+      "layer10.mlp": [
+        86016,
+        90112
+      ],
+      "layer11.attn": [
+        90112,
+        94208
+      ],
+      "layer11.mlp": [
+        94208,
+        98304
+      ],
+      "layer12.attn": [
+        98304,
+        102400
+      ],
+      "layer12.mlp": [
+        102400,
+        106496
+      ],
+      "layer13.attn": [
+        106496,
+        110592
+      ],
+      "layer13.mlp": [
+        110592,
+        114688
+      ],
+      "layer14.attn": [
+        114688,
+        118784
+      ],
+      "layer14.mlp": [
+        118784,
+        122880
+      ],
+      "layer15.attn": [
+        122880,
+        126976
+      ],
+      "layer15.mlp": [
+        126976,
+        131072
+      ],
+      "layer16.attn": [
+        131072,
+        135168
+      ],
+      "layer16.mlp": [
+        135168,
+        139264
+      ],
+      "layer17.attn": [
+        139264,
+        143360
+      ],
+      "layer17.mlp": [
+        143360,
+        147456
+      ],
+      "layer18.attn": [
+        147456,
+        151552
+      ],
+      "layer18.mlp": [
+        151552,
+        155648
+      ],
+      "layer19.attn": [
+        155648,
+        159744
+      ],
+      "layer19.mlp": [
+        159744,
+        163840
+      ],
+      "layer20.attn": [
+        163840,
+        167936
+      ],
+      "layer20.mlp": [
+        167936,
+        172032
+      ],
+      "layer21.attn": [
+        172032,
+        176128
+      ],
+      "layer21.mlp": [
+        176128,
+        180224
+      ],
+      "layer22.attn": [
+        180224,
+        184320
+      ],
+      "layer22.mlp": [
+        184320,
+        188416
+      ],
+      "layer23.attn": [
+        188416,
+        192512
+      ],
+      "layer23.mlp": [
+        192512,
+        196608
+      ],
+      "layer24.attn": [
+        196608,
+        200704
+      ],
+      "layer24.mlp": [
+        200704,
+        204800
+      ],
+      "layer25.attn": [
+        204800,
+        208896
+      ],
+      "layer25.mlp": [
+        208896,
+        212992
+      ],
+      "layer26.attn": [
+        212992,
+        217088
+      ],
+      "layer26.mlp": [
+        217088,
+        221184
+      ],
+      "layer27.attn": [
+        221184,
+        225280
+      ],
+      "layer27.mlp": [
+        225280,
+        229376
+      ],
+      "layer28.attn": [
+        229376,
+        233472
+      ],
+      "layer28.mlp": [
+        233472,
+        237568
+      ],
+      "layer29.attn": [
+        237568,
+        241664
+      ],
+      "layer29.mlp": [
+        241664,
+        245760
+      ],
+      "layer30.attn": [
+        245760,
+        249856
+      ],
+      "layer30.mlp": [
+        249856,
+        253952
+      ],
+      "layer31.attn": [
+        253952,
+        258048
+      ],
+      "layer31.mlp": [
+        258048,
+        262144
+      ]
+    },
+    "layer_count": 32,
+    "hidden_size": 4096,
+    "uniform_module_width": true,
+    "probe_blocks": [
+      [
+        0,
+        1,
+        2,
+        3,
+        4,
+        5,
+        6,
+        7,
+        8,
+        9,
+        10,
+        11,
+        12,
+        13,
+        14,
+        15
+      ]
+    ],
+    "gate_dtype": "float32",
+    "ungated_first_window_nll": 9.359926223754883,
+    "unit_gate_nll_error": 0.0,
+    "repeat_window_index": 0,
+    "repeat_vector_exact": true,
+    "repeat_vector_l2_error": 0.0,
+    "repeat_window_nll": 9.359926223754883,
+    "source_parameter_versions_unchanged": true,
+    "non_gate_trainable_parameters": 0,
+    "gates_removed_before_native_restore": true,
+    "windows": 16,
+    "module_graph_restored": true,
+    "raw_mean_norm": 1.6006406472862853
+  },
+  "optimization_steps": 0
+}

basis/gradients/q128/llama31/asvd_r080/step_0000/block_2/vectors.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:0c3775aef0f0aea760ac1202527504fb710a5aa9512ca5fcb1e7f94b0832e42f
+size 17831065

basis/gradients/q128/llama31/asvd_r080/step_0000/block_3/summary.json

@@ -0,0 +1,995 @@
+{
+  "status": "complete",
+  "job_id": "llama31/asvd_r080",
+  "backbone": "llama31",
+  "step": 0,
+  "state_id": "6f1b8bc457a7ab3a928da64325d85a47efcb711c337504eb5b08a0597793b221",
+  "manifest_sha256": "d77eb1a1b9fd2801372ec781f1052b3ca276dffccfe20b34e3c7c44dcfed52a2",
+  "probe_tensor_sha256": "f19c31b43250bb8f5904e38874b21aca10fff6f736a76ada81e86121553f7e1b",
+  "evaluation_tensor_sha256": "736e3a4086111f686021cff0b3beaf1cd1ae4feae52bfd6719cdb3680a9e400d",
+  "block": 3,
+  "global_window_indices": [
+    48,
+    49,
+    50,
+    51,
+    52,
+    53,
+    54,
+    55,
+    56,
+    57,
+    58,
+    59,
+    60,
+    61,
+    62,
+    63
+  ],
+  "vectors": {
+    "path": "/home/zs89/Basis/hx_exp/outputs/confirmation/llama31/asvd_r080/step_0000/block_3/vectors.pt",
+    "sha256": "feb7f9c27889d5e2f27773d0bb048f55f6c7fa64be75f6a16dacc0c392194305"
+  },
+  "audit": {
+    "window_nll": [
+      7.977346897125244,
+      9.111451148986816,
+      8.200664520263672,
+      6.93148946762085,
+      7.273468494415283,
+      8.202545166015625,
+      8.73422908782959,
+      9.010284423828125,
+      8.469447135925293,
+      7.767333984375,
+      7.7148213386535645,
+      8.083393096923828,
+      6.961669445037842,
+      8.176446914672852,
+      7.891605377197266,
+      8.451583862304688
+    ],
+    "coordinate_count": 262144,
+    "module_records": [
+      {
+        "name": "layer00.attn",
+        "layer": 0,
+        "module_type": "attention",
+        "coordinate_dim": 4096,
+        "start": 0,
+        "stop": 4096
+      },
+      {
+        "name": "layer00.mlp",
+        "layer": 0,
+        "module_type": "MLP",
+        "coordinate_dim": 4096,
+        "start": 4096,
+        "stop": 8192
+      },
+      {
+        "name": "layer01.attn",
+        "layer": 1,
+        "module_type": "attention",
+        "coordinate_dim": 4096,
+        "start": 8192,
+        "stop": 12288
+      },
+      {
+        "name": "layer01.mlp",
+        "layer": 1,
+        "module_type": "MLP",
+        "coordinate_dim": 4096,
+        "start": 12288,
+        "stop": 16384
+      },
+      {
+        "name": "layer02.attn",
+        "layer": 2,
+        "module_type": "attention",
+        "coordinate_dim": 4096,
+        "start": 16384,
+        "stop": 20480
+      },
+      {
+        "name": "layer02.mlp",
+        "layer": 2,
+        "module_type": "MLP",
+        "coordinate_dim": 4096,
+        "start": 20480,
+        "stop": 24576
+      },
+      {
+        "name": "layer03.attn",
+        "layer": 3,
+        "module_type": "attention",
+        "coordinate_dim": 4096,
+        "start": 24576,
+        "stop": 28672
+      },
+      {
+        "name": "layer03.mlp",
+        "layer": 3,
+        "module_type": "MLP",
+        "coordinate_dim": 4096,
+        "start": 28672,
+        "stop": 32768
+      },
+      {
+        "name": "layer04.attn",
+        "layer": 4,
+        "module_type": "attention",
+        "coordinate_dim": 4096,
+        "start": 32768,
+        "stop": 36864
+      },
+      {
+        "name": "layer04.mlp",
+        "layer": 4,
+        "module_type": "MLP",
+        "coordinate_dim": 4096,
+        "start": 36864,
+        "stop": 40960
+      },
+      {
+        "name": "layer05.attn",
+        "layer": 5,
+        "module_type": "attention",
+        "coordinate_dim": 4096,
+        "start": 40960,
+        "stop": 45056
+      },
+      {
+        "name": "layer05.mlp",
+        "layer": 5,
+        "module_type": "MLP",
+        "coordinate_dim": 4096,
+        "start": 45056,
+        "stop": 49152
+      },
+      {
+        "name": "layer06.attn",
+        "layer": 6,
+        "module_type": "attention",
+        "coordinate_dim": 4096,
+        "start": 49152,
+        "stop": 53248
+      },
+      {
+        "name": "layer06.mlp",
+        "layer": 6,
+        "module_type": "MLP",
+        "coordinate_dim": 4096,
+        "start": 53248,
+        "stop": 57344
+      },
+      {
+        "name": "layer07.attn",
+        "layer": 7,
+        "module_type": "attention",
+        "coordinate_dim": 4096,
+        "start": 57344,
+        "stop": 61440
+      },
+      {
+        "name": "layer07.mlp",
+        "layer": 7,
+        "module_type": "MLP",
+        "coordinate_dim": 4096,
+        "start": 61440,
+        "stop": 65536
+      },
+      {
+        "name": "layer08.attn",
+        "layer": 8,
+        "module_type": "attention",
+        "coordinate_dim": 4096,
+        "start": 65536,
+        "stop": 69632
+      },
+      {
+        "name": "layer08.mlp",
+        "layer": 8,
+        "module_type": "MLP",
+        "coordinate_dim": 4096,
+        "start": 69632,
+        "stop": 73728
+      },
+      {
+        "name": "layer09.attn",
+        "layer": 9,
+        "module_type": "attention",
+        "coordinate_dim": 4096,
+        "start": 73728,
+        "stop": 77824
+      },
+      {
+        "name": "layer09.mlp",
+        "layer": 9,
+        "module_type": "MLP",
+        "coordinate_dim": 4096,
+        "start": 77824,
+        "stop": 81920
+      },
+      {
+        "name": "layer10.attn",
+        "layer": 10,
+        "module_type": "attention",
+        "coordinate_dim": 4096,
+        "start": 81920,
+        "stop": 86016
+      },
+      {
+        "name": "layer10.mlp",
+        "layer": 10,
+        "module_type": "MLP",
+        "coordinate_dim": 4096,
+        "start": 86016,
+        "stop": 90112
+      },
+      {
+        "name": "layer11.attn",
+        "layer": 11,
+        "module_type": "attention",
+        "coordinate_dim": 4096,
+        "start": 90112,
+        "stop": 94208
+      },
+      {
+        "name": "layer11.mlp",
+        "layer": 11,
+        "module_type": "MLP",
+        "coordinate_dim": 4096,
+        "start": 94208,
+        "stop": 98304
+      },
+      {
+        "name": "layer12.attn",
+        "layer": 12,
+        "module_type": "attention",
+        "coordinate_dim": 4096,
+        "start": 98304,
+        "stop": 102400
+      },
+      {
+        "name": "layer12.mlp",
+        "layer": 12,
+        "module_type": "MLP",
+        "coordinate_dim": 4096,
+        "start": 102400,
+        "stop": 106496
+      },
+      {
+        "name": "layer13.attn",
+        "layer": 13,
+        "module_type": "attention",
+        "coordinate_dim": 4096,
+        "start": 106496,
+        "stop": 110592
+      },
+      {
+        "name": "layer13.mlp",
+        "layer": 13,
+        "module_type": "MLP",
+        "coordinate_dim": 4096,
+        "start": 110592,
+        "stop": 114688
+      },
+      {
+        "name": "layer14.attn",
+        "layer": 14,
+        "module_type": "attention",
+        "coordinate_dim": 4096,
+        "start": 114688,
+        "stop": 118784
+      },
+      {
+        "name": "layer14.mlp",
+        "layer": 14,
+        "module_type": "MLP",
+        "coordinate_dim": 4096,
+        "start": 118784,
+        "stop": 122880
+      },
+      {
+        "name": "layer15.attn",
+        "layer": 15,
+        "module_type": "attention",
+        "coordinate_dim": 4096,
+        "start": 122880,
+        "stop": 126976
+      },
+      {
+        "name": "layer15.mlp",
+        "layer": 15,
+        "module_type": "MLP",
+        "coordinate_dim": 4096,
+        "start": 126976,
+        "stop": 131072
+      },
+      {
+        "name": "layer16.attn",
+        "layer": 16,
+        "module_type": "attention",
+        "coordinate_dim": 4096,
+        "start": 131072,
+        "stop": 135168
+      },
+      {
+        "name": "layer16.mlp",
+        "layer": 16,
+        "module_type": "MLP",
+        "coordinate_dim": 4096,
+        "start": 135168,
+        "stop": 139264
+      },
+      {
+        "name": "layer17.attn",
+        "layer": 17,
+        "module_type": "attention",
+        "coordinate_dim": 4096,
+        "start": 139264,
+        "stop": 143360
+      },
+      {
+        "name": "layer17.mlp",
+        "layer": 17,
+        "module_type": "MLP",
+        "coordinate_dim": 4096,
+        "start": 143360,
+        "stop": 147456
+      },
+      {
+        "name": "layer18.attn",
+        "layer": 18,
+        "module_type": "attention",
+        "coordinate_dim": 4096,
+        "start": 147456,
+        "stop": 151552
+      },
+      {
+        "name": "layer18.mlp",
+        "layer": 18,
+        "module_type": "MLP",
+        "coordinate_dim": 4096,
+        "start": 151552,
+        "stop": 155648
+      },
+      {
+        "name": "layer19.attn",
+        "layer": 19,
+        "module_type": "attention",
+        "coordinate_dim": 4096,
+        "start": 155648,
+        "stop": 159744
+      },
+      {
+        "name": "layer19.mlp",
+        "layer": 19,
+        "module_type": "MLP",
+        "coordinate_dim": 4096,
+        "start": 159744,
+        "stop": 163840
+      },
+      {
+        "name": "layer20.attn",
+        "layer": 20,
+        "module_type": "attention",
+        "coordinate_dim": 4096,
+        "start": 163840,
+        "stop": 167936
+      },
+      {
+        "name": "layer20.mlp",
+        "layer": 20,
+        "module_type": "MLP",
+        "coordinate_dim": 4096,
+        "start": 167936,
+        "stop": 172032
+      },
+      {
+        "name": "layer21.attn",
+        "layer": 21,
+        "module_type": "attention",
+        "coordinate_dim": 4096,
+        "start": 172032,
+        "stop": 176128
+      },
+      {
+        "name": "layer21.mlp",
+        "layer": 21,
+        "module_type": "MLP",
+        "coordinate_dim": 4096,
+        "start": 176128,
+        "stop": 180224
+      },
+      {
+        "name": "layer22.attn",
+        "layer": 22,
+        "module_type": "attention",
+        "coordinate_dim": 4096,
+        "start": 180224,
+        "stop": 184320
+      },
+      {
+        "name": "layer22.mlp",
+        "layer": 22,
+        "module_type": "MLP",
+        "coordinate_dim": 4096,
+        "start": 184320,
+        "stop": 188416
+      },
+      {
+        "name": "layer23.attn",
+        "layer": 23,
+        "module_type": "attention",
+        "coordinate_dim": 4096,
+        "start": 188416,
+        "stop": 192512
+      },
+      {
+        "name": "layer23.mlp",
+        "layer": 23,
+        "module_type": "MLP",
+        "coordinate_dim": 4096,
+        "start": 192512,
+        "stop": 196608
+      },
+      {
+        "name": "layer24.attn",
+        "layer": 24,
+        "module_type": "attention",
+        "coordinate_dim": 4096,
+        "start": 196608,
+        "stop": 200704
+      },
+      {
+        "name": "layer24.mlp",
+        "layer": 24,
+        "module_type": "MLP",
+        "coordinate_dim": 4096,
+        "start": 200704,
+        "stop": 204800
+      },
+      {
+        "name": "layer25.attn",
+        "layer": 25,
+        "module_type": "attention",
+        "coordinate_dim": 4096,
+        "start": 204800,
+        "stop": 208896
+      },
+      {
+        "name": "layer25.mlp",
+        "layer": 25,
+        "module_type": "MLP",
+        "coordinate_dim": 4096,
+        "start": 208896,
+        "stop": 212992
+      },
+      {
+        "name": "layer26.attn",
+        "layer": 26,
+        "module_type": "attention",
+        "coordinate_dim": 4096,
+        "start": 212992,
+        "stop": 217088
+      },
+      {
+        "name": "layer26.mlp",
+        "layer": 26,
+        "module_type": "MLP",
+        "coordinate_dim": 4096,
+        "start": 217088,
+        "stop": 221184
+      },
+      {
+        "name": "layer27.attn",
+        "layer": 27,
+        "module_type": "attention",
+        "coordinate_dim": 4096,
+        "start": 221184,
+        "stop": 225280
+      },
+      {
+        "name": "layer27.mlp",
+        "layer": 27,
+        "module_type": "MLP",
+        "coordinate_dim": 4096,
+        "start": 225280,
+        "stop": 229376
+      },
+      {
+        "name": "layer28.attn",
+        "layer": 28,
+        "module_type": "attention",
+        "coordinate_dim": 4096,
+        "start": 229376,
+        "stop": 233472
+      },
+      {
+        "name": "layer28.mlp",
+        "layer": 28,
+        "module_type": "MLP",
+        "coordinate_dim": 4096,
+        "start": 233472,
+        "stop": 237568
+      },
+      {
+        "name": "layer29.attn",
+        "layer": 29,
+        "module_type": "attention",
+        "coordinate_dim": 4096,
+        "start": 237568,
+        "stop": 241664
+      },
+      {
+        "name": "layer29.mlp",
+        "layer": 29,
+        "module_type": "MLP",
+        "coordinate_dim": 4096,
+        "start": 241664,
+        "stop": 245760
+      },
+      {
+        "name": "layer30.attn",
+        "layer": 30,
+        "module_type": "attention",
+        "coordinate_dim": 4096,
+        "start": 245760,
+        "stop": 249856
+      },
+      {
+        "name": "layer30.mlp",
+        "layer": 30,
+        "module_type": "MLP",
+        "coordinate_dim": 4096,
+        "start": 249856,
+        "stop": 253952
+      },
+      {
+        "name": "layer31.attn",
+        "layer": 31,
+        "module_type": "attention",
+        "coordinate_dim": 4096,
+        "start": 253952,
+        "stop": 258048
+      },
+      {
+        "name": "layer31.mlp",
+        "layer": 31,
+        "module_type": "MLP",
+        "coordinate_dim": 4096,
+        "start": 258048,
+        "stop": 262144
+      }
+    ],
+    "module_names": [
+      "layer00.attn",
+      "layer00.mlp",
+      "layer01.attn",
+      "layer01.mlp",
+      "layer02.attn",
+      "layer02.mlp",
+      "layer03.attn",
+      "layer03.mlp",
+      "layer04.attn",
+      "layer04.mlp",
+      "layer05.attn",
+      "layer05.mlp",
+      "layer06.attn",
+      "layer06.mlp",
+      "layer07.attn",
+      "layer07.mlp",
+      "layer08.attn",
+      "layer08.mlp",
+      "layer09.attn",
+      "layer09.mlp",
+      "layer10.attn",
+      "layer10.mlp",
+      "layer11.attn",
+      "layer11.mlp",
+      "layer12.attn",
+      "layer12.mlp",
+      "layer13.attn",
+      "layer13.mlp",
+      "layer14.attn",
+      "layer14.mlp",
+      "layer15.attn",
+      "layer15.mlp",
+      "layer16.attn",
+      "layer16.mlp",
+      "layer17.attn",
+      "layer17.mlp",
+      "layer18.attn",
+      "layer18.mlp",
+      "layer19.attn",
+      "layer19.mlp",
+      "layer20.attn",
+      "layer20.mlp",
+      "layer21.attn",
+      "layer21.mlp",
+      "layer22.attn",
+      "layer22.mlp",
+      "layer23.attn",
+      "layer23.mlp",
+      "layer24.attn",
+      "layer24.mlp",
+      "layer25.attn",
+      "layer25.mlp",
+      "layer26.attn",
+      "layer26.mlp",
+      "layer27.attn",
+      "layer27.mlp",
+      "layer28.attn",
+      "layer28.mlp",
+      "layer29.attn",
+      "layer29.mlp",
+      "layer30.attn",
+      "layer30.mlp",
+      "layer31.attn",
+      "layer31.mlp"
+    ],
+    "module_dimensions": [
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096
+    ],
+    "module_ranges": {
+      "layer00.attn": [
+        0,
+        4096
+      ],
+      "layer00.mlp": [
+        4096,
+        8192
+      ],
+      "layer01.attn": [
+        8192,
+        12288
+      ],
+      "layer01.mlp": [
+        12288,
+        16384
+      ],
+      "layer02.attn": [
+        16384,
+        20480
+      ],
+      "layer02.mlp": [
+        20480,
+        24576
+      ],
+      "layer03.attn": [
+        24576,
+        28672
+      ],
+      "layer03.mlp": [
+        28672,
+        32768
+      ],
+      "layer04.attn": [
+        32768,
+        36864
+      ],
+      "layer04.mlp": [
+        36864,
+        40960
+      ],
+      "layer05.attn": [
+        40960,
+        45056
+      ],
+      "layer05.mlp": [
+        45056,
+        49152
+      ],
+      "layer06.attn": [
+        49152,
+        53248
+      ],
+      "layer06.mlp": [
+        53248,
+        57344
+      ],
+      "layer07.attn": [
+        57344,
+        61440
+      ],
+      "layer07.mlp": [
+        61440,
+        65536
+      ],
+      "layer08.attn": [
+        65536,
+        69632
+      ],
+      "layer08.mlp": [
+        69632,
+        73728
+      ],
+      "layer09.attn": [
+        73728,
+        77824
+      ],
+      "layer09.mlp": [
+        77824,
+        81920
+      ],
+      "layer10.attn": [
+        81920,
+        86016
+      ],
+      "layer10.mlp": [
+        86016,
+        90112
+      ],
+      "layer11.attn": [
+        90112,
+        94208
+      ],
+      "layer11.mlp": [
+        94208,
+        98304
+      ],
+      "layer12.attn": [
+        98304,
+        102400
+      ],
+      "layer12.mlp": [
+        102400,
+        106496
+      ],
+      "layer13.attn": [
+        106496,
+        110592
+      ],
+      "layer13.mlp": [
+        110592,
+        114688
+      ],
+      "layer14.attn": [
+        114688,
+        118784
+      ],
+      "layer14.mlp": [
+        118784,
+        122880
+      ],
+      "layer15.attn": [
+        122880,
+        126976
+      ],
+      "layer15.mlp": [
+        126976,
+        131072
+      ],
+      "layer16.attn": [
+        131072,
+        135168
+      ],
+      "layer16.mlp": [
+        135168,
+        139264
+      ],
+      "layer17.attn": [
+        139264,
+        143360
+      ],
+      "layer17.mlp": [
+        143360,
+        147456
+      ],
+      "layer18.attn": [
+        147456,
+        151552
+      ],
+      "layer18.mlp": [
+        151552,
+        155648
+      ],
+      "layer19.attn": [
+        155648,
+        159744
+      ],
+      "layer19.mlp": [
+        159744,
+        163840
+      ],
+      "layer20.attn": [
+        163840,
+        167936
+      ],
+      "layer20.mlp": [
+        167936,
+        172032
+      ],
+      "layer21.attn": [
+        172032,
+        176128
+      ],
+      "layer21.mlp": [
+        176128,
+        180224
+      ],
+      "layer22.attn": [
+        180224,
+        184320
+      ],
+      "layer22.mlp": [
+        184320,
+        188416
+      ],
+      "layer23.attn": [
+        188416,
+        192512
+      ],
+      "layer23.mlp": [
+        192512,
+        196608
+      ],
+      "layer24.attn": [
+        196608,
+        200704
+      ],
+      "layer24.mlp": [
+        200704,
+        204800
+      ],
+      "layer25.attn": [
+        204800,
+        208896
+      ],
+      "layer25.mlp": [
+        208896,
+        212992
+      ],
+      "layer26.attn": [
+        212992,
+        217088
+      ],
+      "layer26.mlp": [
+        217088,
+        221184
+      ],
+      "layer27.attn": [
+        221184,
+        225280
+      ],
+      "layer27.mlp": [
+        225280,
+        229376
+      ],
+      "layer28.attn": [
+        229376,
+        233472
+      ],
+      "layer28.mlp": [
+        233472,
+        237568
+      ],
+      "layer29.attn": [
+        237568,
+        241664
+      ],
+      "layer29.mlp": [
+        241664,
+        245760
+      ],
+      "layer30.attn": [
+        245760,
+        249856
+      ],
+      "layer30.mlp": [
+        249856,
+        253952
+      ],
+      "layer31.attn": [
+        253952,
+        258048
+      ],
+      "layer31.mlp": [
+        258048,
+        262144
+      ]
+    },
+    "layer_count": 32,
+    "hidden_size": 4096,
+    "uniform_module_width": true,
+    "probe_blocks": [
+      [
+        0,
+        1,
+        2,
+        3,
+        4,
+        5,
+        6,
+        7,
+        8,
+        9,
+        10,
+        11,
+        12,
+        13,
+        14,
+        15
+      ]
+    ],
+    "gate_dtype": "float32",
+    "ungated_first_window_nll": 7.977346897125244,
+    "unit_gate_nll_error": 0.0,
+    "repeat_window_index": 0,
+    "repeat_vector_exact": true,
+    "repeat_vector_l2_error": 0.0,
+    "repeat_window_nll": 7.977346897125244,
+    "source_parameter_versions_unchanged": true,
+    "non_gate_trainable_parameters": 0,
+    "gates_removed_before_native_restore": true,
+    "windows": 16,
+    "module_graph_restored": true,
+    "raw_mean_norm": 1.4127550554234807
+  },
+  "optimization_steps": 0
+}

basis/gradients/q128/llama31/asvd_r080/step_0000/block_3/vectors.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:feb7f9c27889d5e2f27773d0bb048f55f6c7fa64be75f6a16dacc0c392194305
+size 17831065

basis/gradients/q128/llama31/asvd_r080/step_0000/block_4/summary.json

@@ -0,0 +1,995 @@
+{
+  "status": "complete",
+  "job_id": "llama31/asvd_r080",
+  "backbone": "llama31",
+  "step": 0,
+  "state_id": "6f1b8bc457a7ab3a928da64325d85a47efcb711c337504eb5b08a0597793b221",
+  "manifest_sha256": "d77eb1a1b9fd2801372ec781f1052b3ca276dffccfe20b34e3c7c44dcfed52a2",
+  "probe_tensor_sha256": "f19c31b43250bb8f5904e38874b21aca10fff6f736a76ada81e86121553f7e1b",
+  "evaluation_tensor_sha256": "736e3a4086111f686021cff0b3beaf1cd1ae4feae52bfd6719cdb3680a9e400d",
+  "block": 4,
+  "global_window_indices": [
+    64,
+    65,
+    66,
+    67,
+    68,
+    69,
+    70,
+    71,
+    72,
+    73,
+    74,
+    75,
+    76,
+    77,
+    78,
+    79
+  ],
+  "vectors": {
+    "path": "/home/zs89/Basis/hx_exp/outputs/confirmation/llama31/asvd_r080/step_0000/block_4/vectors.pt",
+    "sha256": "04849bd0a47a2b5a4701a1b4ee6b669562ed64b80eeb289bb353e694284fc559"
+  },
+  "audit": {
+    "window_nll": [
+      7.647074222564697,
+      7.756343841552734,
+      8.068008422851562,
+      8.404354095458984,
+      7.586843967437744,
+      9.268815040588379,
+      10.216001510620117,
+      9.152825355529785,
+      9.130943298339844,
+      8.416171073913574,
+      8.557043075561523,
+      8.05533218383789,
+      7.994378089904785,
+      8.100312232971191,
+      8.155213356018066,
+      8.680834770202637
+    ],
+    "coordinate_count": 262144,
+    "module_records": [
+      {
+        "name": "layer00.attn",
+        "layer": 0,
+        "module_type": "attention",
+        "coordinate_dim": 4096,
+        "start": 0,
+        "stop": 4096
+      },
+      {
+        "name": "layer00.mlp",
+        "layer": 0,
+        "module_type": "MLP",
+        "coordinate_dim": 4096,
+        "start": 4096,
+        "stop": 8192
+      },
+      {
+        "name": "layer01.attn",
+        "layer": 1,
+        "module_type": "attention",
+        "coordinate_dim": 4096,
+        "start": 8192,
+        "stop": 12288
+      },
+      {
+        "name": "layer01.mlp",
+        "layer": 1,
+        "module_type": "MLP",
+        "coordinate_dim": 4096,
+        "start": 12288,
+        "stop": 16384
+      },
+      {
+        "name": "layer02.attn",
+        "layer": 2,
+        "module_type": "attention",
+        "coordinate_dim": 4096,
+        "start": 16384,
+        "stop": 20480
+      },
+      {
+        "name": "layer02.mlp",
+        "layer": 2,
+        "module_type": "MLP",
+        "coordinate_dim": 4096,
+        "start": 20480,
+        "stop": 24576
+      },
+      {
+        "name": "layer03.attn",
+        "layer": 3,
+        "module_type": "attention",
+        "coordinate_dim": 4096,
+        "start": 24576,
+        "stop": 28672
+      },
+      {
+        "name": "layer03.mlp",
+        "layer": 3,
+        "module_type": "MLP",
+        "coordinate_dim": 4096,
+        "start": 28672,
+        "stop": 32768
+      },
+      {
+        "name": "layer04.attn",
+        "layer": 4,
+        "module_type": "attention",
+        "coordinate_dim": 4096,
+        "start": 32768,
+        "stop": 36864
+      },
+      {
+        "name": "layer04.mlp",
+        "layer": 4,
+        "module_type": "MLP",
+        "coordinate_dim": 4096,
+        "start": 36864,
+        "stop": 40960
+      },
+      {
+        "name": "layer05.attn",
+        "layer": 5,
+        "module_type": "attention",
+        "coordinate_dim": 4096,
+        "start": 40960,
+        "stop": 45056
+      },
+      {
+        "name": "layer05.mlp",
+        "layer": 5,
+        "module_type": "MLP",
+        "coordinate_dim": 4096,
+        "start": 45056,
+        "stop": 49152
+      },
+      {
+        "name": "layer06.attn",
+        "layer": 6,
+        "module_type": "attention",
+        "coordinate_dim": 4096,
+        "start": 49152,
+        "stop": 53248
+      },
+      {
+        "name": "layer06.mlp",
+        "layer": 6,
+        "module_type": "MLP",
+        "coordinate_dim": 4096,
+        "start": 53248,
+        "stop": 57344
+      },
+      {
+        "name": "layer07.attn",
+        "layer": 7,
+        "module_type": "attention",
+        "coordinate_dim": 4096,
+        "start": 57344,
+        "stop": 61440
+      },
+      {
+        "name": "layer07.mlp",
+        "layer": 7,
+        "module_type": "MLP",
+        "coordinate_dim": 4096,
+        "start": 61440,
+        "stop": 65536
+      },
+      {
+        "name": "layer08.attn",
+        "layer": 8,
+        "module_type": "attention",
+        "coordinate_dim": 4096,
+        "start": 65536,
+        "stop": 69632
+      },
+      {
+        "name": "layer08.mlp",
+        "layer": 8,
+        "module_type": "MLP",
+        "coordinate_dim": 4096,
+        "start": 69632,
+        "stop": 73728
+      },
+      {
+        "name": "layer09.attn",
+        "layer": 9,
+        "module_type": "attention",
+        "coordinate_dim": 4096,
+        "start": 73728,
+        "stop": 77824
+      },
+      {
+        "name": "layer09.mlp",
+        "layer": 9,
+        "module_type": "MLP",
+        "coordinate_dim": 4096,
+        "start": 77824,
+        "stop": 81920
+      },
+      {
+        "name": "layer10.attn",
+        "layer": 10,
+        "module_type": "attention",
+        "coordinate_dim": 4096,
+        "start": 81920,
+        "stop": 86016
+      },
+      {
+        "name": "layer10.mlp",
+        "layer": 10,
+        "module_type": "MLP",
+        "coordinate_dim": 4096,
+        "start": 86016,
+        "stop": 90112
+      },
+      {
+        "name": "layer11.attn",
+        "layer": 11,
+        "module_type": "attention",
+        "coordinate_dim": 4096,
+        "start": 90112,
+        "stop": 94208
+      },
+      {
+        "name": "layer11.mlp",
+        "layer": 11,
+        "module_type": "MLP",
+        "coordinate_dim": 4096,
+        "start": 94208,
+        "stop": 98304
+      },
+      {
+        "name": "layer12.attn",
+        "layer": 12,
+        "module_type": "attention",
+        "coordinate_dim": 4096,
+        "start": 98304,
+        "stop": 102400
+      },
+      {
+        "name": "layer12.mlp",
+        "layer": 12,
+        "module_type": "MLP",
+        "coordinate_dim": 4096,
+        "start": 102400,
+        "stop": 106496
+      },
+      {
+        "name": "layer13.attn",
+        "layer": 13,
+        "module_type": "attention",
+        "coordinate_dim": 4096,
+        "start": 106496,
+        "stop": 110592
+      },
+      {
+        "name": "layer13.mlp",
+        "layer": 13,
+        "module_type": "MLP",
+        "coordinate_dim": 4096,
+        "start": 110592,
+        "stop": 114688
+      },
+      {
+        "name": "layer14.attn",
+        "layer": 14,
+        "module_type": "attention",
+        "coordinate_dim": 4096,
+        "start": 114688,
+        "stop": 118784
+      },
+      {
+        "name": "layer14.mlp",
+        "layer": 14,
+        "module_type": "MLP",
+        "coordinate_dim": 4096,
+        "start": 118784,
+        "stop": 122880
+      },
+      {
+        "name": "layer15.attn",
+        "layer": 15,
+        "module_type": "attention",
+        "coordinate_dim": 4096,
+        "start": 122880,
+        "stop": 126976
+      },
+      {
+        "name": "layer15.mlp",
+        "layer": 15,
+        "module_type": "MLP",
+        "coordinate_dim": 4096,
+        "start": 126976,
+        "stop": 131072
+      },
+      {
+        "name": "layer16.attn",
+        "layer": 16,
+        "module_type": "attention",
+        "coordinate_dim": 4096,
+        "start": 131072,
+        "stop": 135168
+      },
+      {
+        "name": "layer16.mlp",
+        "layer": 16,
+        "module_type": "MLP",
+        "coordinate_dim": 4096,
+        "start": 135168,
+        "stop": 139264
+      },
+      {
+        "name": "layer17.attn",
+        "layer": 17,
+        "module_type": "attention",
+        "coordinate_dim": 4096,
+        "start": 139264,
+        "stop": 143360
+      },
+      {
+        "name": "layer17.mlp",
+        "layer": 17,
+        "module_type": "MLP",
+        "coordinate_dim": 4096,
+        "start": 143360,
+        "stop": 147456
+      },
+      {
+        "name": "layer18.attn",
+        "layer": 18,
+        "module_type": "attention",
+        "coordinate_dim": 4096,
+        "start": 147456,
+        "stop": 151552
+      },
+      {
+        "name": "layer18.mlp",
+        "layer": 18,
+        "module_type": "MLP",
+        "coordinate_dim": 4096,
+        "start": 151552,
+        "stop": 155648
+      },
+      {
+        "name": "layer19.attn",
+        "layer": 19,
+        "module_type": "attention",
+        "coordinate_dim": 4096,
+        "start": 155648,
+        "stop": 159744
+      },
+      {
+        "name": "layer19.mlp",
+        "layer": 19,
+        "module_type": "MLP",
+        "coordinate_dim": 4096,
+        "start": 159744,
+        "stop": 163840
+      },
+      {
+        "name": "layer20.attn",
+        "layer": 20,
+        "module_type": "attention",
+        "coordinate_dim": 4096,
+        "start": 163840,
+        "stop": 167936
+      },
+      {
+        "name": "layer20.mlp",
+        "layer": 20,
+        "module_type": "MLP",
+        "coordinate_dim": 4096,
+        "start": 167936,
+        "stop": 172032
+      },
+      {
+        "name": "layer21.attn",
+        "layer": 21,
+        "module_type": "attention",
+        "coordinate_dim": 4096,
+        "start": 172032,
+        "stop": 176128
+      },
+      {
+        "name": "layer21.mlp",
+        "layer": 21,
+        "module_type": "MLP",
+        "coordinate_dim": 4096,
+        "start": 176128,
+        "stop": 180224
+      },
+      {
+        "name": "layer22.attn",
+        "layer": 22,
+        "module_type": "attention",
+        "coordinate_dim": 4096,
+        "start": 180224,
+        "stop": 184320
+      },
+      {
+        "name": "layer22.mlp",
+        "layer": 22,
+        "module_type": "MLP",
+        "coordinate_dim": 4096,
+        "start": 184320,
+        "stop": 188416
+      },
+      {
+        "name": "layer23.attn",
+        "layer": 23,
+        "module_type": "attention",
+        "coordinate_dim": 4096,
+        "start": 188416,
+        "stop": 192512
+      },
+      {
+        "name": "layer23.mlp",
+        "layer": 23,
+        "module_type": "MLP",
+        "coordinate_dim": 4096,
+        "start": 192512,
+        "stop": 196608
+      },
+      {
+        "name": "layer24.attn",
+        "layer": 24,
+        "module_type": "attention",
+        "coordinate_dim": 4096,
+        "start": 196608,
+        "stop": 200704
+      },
+      {
+        "name": "layer24.mlp",
+        "layer": 24,
+        "module_type": "MLP",
+        "coordinate_dim": 4096,
+        "start": 200704,
+        "stop": 204800
+      },
+      {
+        "name": "layer25.attn",
+        "layer": 25,
+        "module_type": "attention",
+        "coordinate_dim": 4096,
+        "start": 204800,
+        "stop": 208896
+      },
+      {
+        "name": "layer25.mlp",
+        "layer": 25,
+        "module_type": "MLP",
+        "coordinate_dim": 4096,
+        "start": 208896,
+        "stop": 212992
+      },
+      {
+        "name": "layer26.attn",
+        "layer": 26,
+        "module_type": "attention",
+        "coordinate_dim": 4096,
+        "start": 212992,
+        "stop": 217088
+      },
+      {
+        "name": "layer26.mlp",
+        "layer": 26,
+        "module_type": "MLP",
+        "coordinate_dim": 4096,
+        "start": 217088,
+        "stop": 221184
+      },
+      {
+        "name": "layer27.attn",
+        "layer": 27,
+        "module_type": "attention",
+        "coordinate_dim": 4096,
+        "start": 221184,
+        "stop": 225280
+      },
+      {
+        "name": "layer27.mlp",
+        "layer": 27,
+        "module_type": "MLP",
+        "coordinate_dim": 4096,
+        "start": 225280,
+        "stop": 229376
+      },
+      {
+        "name": "layer28.attn",
+        "layer": 28,
+        "module_type": "attention",
+        "coordinate_dim": 4096,
+        "start": 229376,
+        "stop": 233472
+      },
+      {
+        "name": "layer28.mlp",
+        "layer": 28,
+        "module_type": "MLP",
+        "coordinate_dim": 4096,
+        "start": 233472,
+        "stop": 237568
+      },
+      {
+        "name": "layer29.attn",
+        "layer": 29,
+        "module_type": "attention",
+        "coordinate_dim": 4096,
+        "start": 237568,
+        "stop": 241664
+      },
+      {
+        "name": "layer29.mlp",
+        "layer": 29,
+        "module_type": "MLP",
+        "coordinate_dim": 4096,
+        "start": 241664,
+        "stop": 245760
+      },
+      {
+        "name": "layer30.attn",
+        "layer": 30,
+        "module_type": "attention",
+        "coordinate_dim": 4096,
+        "start": 245760,
+        "stop": 249856
+      },
+      {
+        "name": "layer30.mlp",
+        "layer": 30,
+        "module_type": "MLP",
+        "coordinate_dim": 4096,
+        "start": 249856,
+        "stop": 253952
+      },
+      {
+        "name": "layer31.attn",
+        "layer": 31,
+        "module_type": "attention",
+        "coordinate_dim": 4096,
+        "start": 253952,
+        "stop": 258048
+      },
+      {
+        "name": "layer31.mlp",
+        "layer": 31,
+        "module_type": "MLP",
+        "coordinate_dim": 4096,
+        "start": 258048,
+        "stop": 262144
+      }
+    ],
+    "module_names": [
+      "layer00.attn",
+      "layer00.mlp",
+      "layer01.attn",
+      "layer01.mlp",
+      "layer02.attn",
+      "layer02.mlp",
+      "layer03.attn",
+      "layer03.mlp",
+      "layer04.attn",
+      "layer04.mlp",
+      "layer05.attn",
+      "layer05.mlp",
+      "layer06.attn",
+      "layer06.mlp",
+      "layer07.attn",
+      "layer07.mlp",
+      "layer08.attn",
+      "layer08.mlp",
+      "layer09.attn",
+      "layer09.mlp",
+      "layer10.attn",
+      "layer10.mlp",
+      "layer11.attn",
+      "layer11.mlp",
+      "layer12.attn",
+      "layer12.mlp",
+      "layer13.attn",
+      "layer13.mlp",
+      "layer14.attn",
+      "layer14.mlp",
+      "layer15.attn",
+      "layer15.mlp",
+      "layer16.attn",
+      "layer16.mlp",
+      "layer17.attn",
+      "layer17.mlp",
+      "layer18.attn",
+      "layer18.mlp",
+      "layer19.attn",
+      "layer19.mlp",
+      "layer20.attn",
+      "layer20.mlp",
+      "layer21.attn",
+      "layer21.mlp",
+      "layer22.attn",
+      "layer22.mlp",
+      "layer23.attn",
+      "layer23.mlp",
+      "layer24.attn",
+      "layer24.mlp",
+      "layer25.attn",
+      "layer25.mlp",
+      "layer26.attn",
+      "layer26.mlp",
+      "layer27.attn",
+      "layer27.mlp",
+      "layer28.attn",
+      "layer28.mlp",
+      "layer29.attn",
+      "layer29.mlp",
+      "layer30.attn",
+      "layer30.mlp",
+      "layer31.attn",
+      "layer31.mlp"
+    ],
+    "module_dimensions": [
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096
+    ],
+    "module_ranges": {
+      "layer00.attn": [
+        0,
+        4096
+      ],
+      "layer00.mlp": [
+        4096,
+        8192
+      ],
+      "layer01.attn": [
+        8192,
+        12288
+      ],
+      "layer01.mlp": [
+        12288,
+        16384
+      ],
+      "layer02.attn": [
+        16384,
+        20480
+      ],
+      "layer02.mlp": [
+        20480,
+        24576
+      ],
+      "layer03.attn": [
+        24576,
+        28672
+      ],
+      "layer03.mlp": [
+        28672,
+        32768
+      ],
+      "layer04.attn": [
+        32768,
+        36864
+      ],
+      "layer04.mlp": [
+        36864,
+        40960
+      ],
+      "layer05.attn": [
+        40960,
+        45056
+      ],
+      "layer05.mlp": [
+        45056,
+        49152
+      ],
+      "layer06.attn": [
+        49152,
+        53248
+      ],
+      "layer06.mlp": [
+        53248,
+        57344
+      ],
+      "layer07.attn": [
+        57344,
+        61440
+      ],
+      "layer07.mlp": [
+        61440,
+        65536
+      ],
+      "layer08.attn": [
+        65536,
+        69632
+      ],
+      "layer08.mlp": [
+        69632,
+        73728
+      ],
+      "layer09.attn": [
+        73728,
+        77824
+      ],
+      "layer09.mlp": [
+        77824,
+        81920
+      ],
+      "layer10.attn": [
+        81920,
+        86016
+      ],
+      "layer10.mlp": [
+        86016,
+        90112
+      ],
+      "layer11.attn": [
+        90112,
+        94208
+      ],
+      "layer11.mlp": [
+        94208,
+        98304
+      ],
+      "layer12.attn": [
+        98304,
+        102400
+      ],
+      "layer12.mlp": [
+        102400,
+        106496
+      ],
+      "layer13.attn": [
+        106496,
+        110592
+      ],
+      "layer13.mlp": [
+        110592,
+        114688
+      ],
+      "layer14.attn": [
+        114688,
+        118784
+      ],
+      "layer14.mlp": [
+        118784,
+        122880
+      ],
+      "layer15.attn": [
+        122880,
+        126976
+      ],
+      "layer15.mlp": [
+        126976,
+        131072
+      ],
+      "layer16.attn": [
+        131072,
+        135168
+      ],
+      "layer16.mlp": [
+        135168,
+        139264
+      ],
+      "layer17.attn": [
+        139264,
+        143360
+      ],
+      "layer17.mlp": [
+        143360,
+        147456
+      ],
+      "layer18.attn": [
+        147456,
+        151552
+      ],
+      "layer18.mlp": [
+        151552,
+        155648
+      ],
+      "layer19.attn": [
+        155648,
+        159744
+      ],
+      "layer19.mlp": [
+        159744,
+        163840
+      ],
+      "layer20.attn": [
+        163840,
+        167936
+      ],
+      "layer20.mlp": [
+        167936,
+        172032
+      ],
+      "layer21.attn": [
+        172032,
+        176128
+      ],
+      "layer21.mlp": [
+        176128,
+        180224
+      ],
+      "layer22.attn": [
+        180224,
+        184320
+      ],
+      "layer22.mlp": [
+        184320,
+        188416
+      ],
+      "layer23.attn": [
+        188416,
+        192512
+      ],
+      "layer23.mlp": [
+        192512,
+        196608
+      ],
+      "layer24.attn": [
+        196608,
+        200704
+      ],
+      "layer24.mlp": [
+        200704,
+        204800
+      ],
+      "layer25.attn": [
+        204800,
+        208896
+      ],
+      "layer25.mlp": [
+        208896,
+        212992
+      ],
+      "layer26.attn": [
+        212992,
+        217088
+      ],
+      "layer26.mlp": [
+        217088,
+        221184
+      ],
+      "layer27.attn": [
+        221184,
+        225280
+      ],
+      "layer27.mlp": [
+        225280,
+        229376
+      ],
+      "layer28.attn": [
+        229376,
+        233472
+      ],
+      "layer28.mlp": [
+        233472,
+        237568
+      ],
+      "layer29.attn": [
+        237568,
+        241664
+      ],
+      "layer29.mlp": [
+        241664,
+        245760
+      ],
+      "layer30.attn": [
+        245760,
+        249856
+      ],
+      "layer30.mlp": [
+        249856,
+        253952
+      ],
+      "layer31.attn": [
+        253952,
+        258048
+      ],
+      "layer31.mlp": [
+        258048,
+        262144
+      ]
+    },
+    "layer_count": 32,
+    "hidden_size": 4096,
+    "uniform_module_width": true,
+    "probe_blocks": [
+      [
+        0,
+        1,
+        2,
+        3,
+        4,
+        5,
+        6,
+        7,
+        8,
+        9,
+        10,
+        11,
+        12,
+        13,
+        14,
+        15
+      ]
+    ],
+    "gate_dtype": "float32",
+    "ungated_first_window_nll": 7.647074222564697,
+    "unit_gate_nll_error": 0.0,
+    "repeat_window_index": 0,
+    "repeat_vector_exact": true,
+    "repeat_vector_l2_error": 0.0,
+    "repeat_window_nll": 7.647074222564697,
+    "source_parameter_versions_unchanged": true,
+    "non_gate_trainable_parameters": 0,
+    "gates_removed_before_native_restore": true,
+    "windows": 16,
+    "module_graph_restored": true,
+    "raw_mean_norm": 1.4278121317586596
+  },
+  "optimization_steps": 0
+}

basis/gradients/q128/llama31/asvd_r080/step_0000/block_4/vectors.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:04849bd0a47a2b5a4701a1b4ee6b669562ed64b80eeb289bb353e694284fc559
+size 17831065

basis/gradients/q128/llama31/asvd_r080/step_0000/block_5/summary.json

@@ -0,0 +1,995 @@
+{
+  "status": "complete",
+  "job_id": "llama31/asvd_r080",
+  "backbone": "llama31",
+  "step": 0,
+  "state_id": "6f1b8bc457a7ab3a928da64325d85a47efcb711c337504eb5b08a0597793b221",
+  "manifest_sha256": "d77eb1a1b9fd2801372ec781f1052b3ca276dffccfe20b34e3c7c44dcfed52a2",
+  "probe_tensor_sha256": "f19c31b43250bb8f5904e38874b21aca10fff6f736a76ada81e86121553f7e1b",
+  "evaluation_tensor_sha256": "736e3a4086111f686021cff0b3beaf1cd1ae4feae52bfd6719cdb3680a9e400d",
+  "block": 5,
+  "global_window_indices": [
+    80,
+    81,
+    82,
+    83,
+    84,
+    85,
+    86,
+    87,
+    88,
+    89,
+    90,
+    91,
+    92,
+    93,
+    94,
+    95
+  ],
+  "vectors": {
+    "path": "/home/zs89/Basis/hx_exp/outputs/confirmation/llama31/asvd_r080/step_0000/block_5/vectors.pt",
+    "sha256": "972c13792b4e4cbcb5dfe215953b95d6ac38e20f155a40f639ed726f6576e9f0"
+  },
+  "audit": {
+    "window_nll": [
+      7.346001625061035,
+      8.51166820526123,
+      8.103764533996582,
+      8.430168151855469,
+      8.659966468811035,
+      8.30048942565918,
+      7.958211898803711,
+      7.64803409576416,
+      8.626933097839355,
+      7.8045430183410645,
+      7.895693778991699,
+      7.832529067993164,
+      10.556968688964844,
+      8.87231731414795,
+      9.21268367767334,
+      7.377403259277344
+    ],
+    "coordinate_count": 262144,
+    "module_records": [
+      {
+        "name": "layer00.attn",
+        "layer": 0,
+        "module_type": "attention",
+        "coordinate_dim": 4096,
+        "start": 0,
+        "stop": 4096
+      },
+      {
+        "name": "layer00.mlp",
+        "layer": 0,
+        "module_type": "MLP",
+        "coordinate_dim": 4096,
+        "start": 4096,
+        "stop": 8192
+      },
+      {
+        "name": "layer01.attn",
+        "layer": 1,
+        "module_type": "attention",
+        "coordinate_dim": 4096,
+        "start": 8192,
+        "stop": 12288
+      },
+      {
+        "name": "layer01.mlp",
+        "layer": 1,
+        "module_type": "MLP",
+        "coordinate_dim": 4096,
+        "start": 12288,
+        "stop": 16384
+      },
+      {
+        "name": "layer02.attn",
+        "layer": 2,
+        "module_type": "attention",
+        "coordinate_dim": 4096,
+        "start": 16384,
+        "stop": 20480
+      },
+      {
+        "name": "layer02.mlp",
+        "layer": 2,
+        "module_type": "MLP",
+        "coordinate_dim": 4096,
+        "start": 20480,
+        "stop": 24576
+      },
+      {
+        "name": "layer03.attn",
+        "layer": 3,
+        "module_type": "attention",
+        "coordinate_dim": 4096,
+        "start": 24576,
+        "stop": 28672
+      },
+      {
+        "name": "layer03.mlp",
+        "layer": 3,
+        "module_type": "MLP",
+        "coordinate_dim": 4096,
+        "start": 28672,
+        "stop": 32768
+      },
+      {
+        "name": "layer04.attn",
+        "layer": 4,
+        "module_type": "attention",
+        "coordinate_dim": 4096,
+        "start": 32768,
+        "stop": 36864
+      },
+      {
+        "name": "layer04.mlp",
+        "layer": 4,
+        "module_type": "MLP",
+        "coordinate_dim": 4096,
+        "start": 36864,
+        "stop": 40960
+      },
+      {
+        "name": "layer05.attn",
+        "layer": 5,
+        "module_type": "attention",
+        "coordinate_dim": 4096,
+        "start": 40960,
+        "stop": 45056
+      },
+      {
+        "name": "layer05.mlp",
+        "layer": 5,
+        "module_type": "MLP",
+        "coordinate_dim": 4096,
+        "start": 45056,
+        "stop": 49152
+      },
+      {
+        "name": "layer06.attn",
+        "layer": 6,
+        "module_type": "attention",
+        "coordinate_dim": 4096,
+        "start": 49152,
+        "stop": 53248
+      },
+      {
+        "name": "layer06.mlp",
+        "layer": 6,
+        "module_type": "MLP",
+        "coordinate_dim": 4096,
+        "start": 53248,
+        "stop": 57344
+      },
+      {
+        "name": "layer07.attn",
+        "layer": 7,
+        "module_type": "attention",
+        "coordinate_dim": 4096,
+        "start": 57344,
+        "stop": 61440
+      },
+      {
+        "name": "layer07.mlp",
+        "layer": 7,
+        "module_type": "MLP",
+        "coordinate_dim": 4096,
+        "start": 61440,
+        "stop": 65536
+      },
+      {
+        "name": "layer08.attn",
+        "layer": 8,
+        "module_type": "attention",
+        "coordinate_dim": 4096,
+        "start": 65536,
+        "stop": 69632
+      },
+      {
+        "name": "layer08.mlp",
+        "layer": 8,
+        "module_type": "MLP",
+        "coordinate_dim": 4096,
+        "start": 69632,
+        "stop": 73728
+      },
+      {
+        "name": "layer09.attn",
+        "layer": 9,
+        "module_type": "attention",
+        "coordinate_dim": 4096,
+        "start": 73728,
+        "stop": 77824
+      },
+      {
+        "name": "layer09.mlp",
+        "layer": 9,
+        "module_type": "MLP",
+        "coordinate_dim": 4096,
+        "start": 77824,
+        "stop": 81920
+      },
+      {
+        "name": "layer10.attn",
+        "layer": 10,
+        "module_type": "attention",
+        "coordinate_dim": 4096,
+        "start": 81920,
+        "stop": 86016
+      },
+      {
+        "name": "layer10.mlp",
+        "layer": 10,
+        "module_type": "MLP",
+        "coordinate_dim": 4096,
+        "start": 86016,
+        "stop": 90112
+      },
+      {
+        "name": "layer11.attn",
+        "layer": 11,
+        "module_type": "attention",
+        "coordinate_dim": 4096,
+        "start": 90112,
+        "stop": 94208
+      },
+      {
+        "name": "layer11.mlp",
+        "layer": 11,
+        "module_type": "MLP",
+        "coordinate_dim": 4096,
+        "start": 94208,
+        "stop": 98304
+      },
+      {
+        "name": "layer12.attn",
+        "layer": 12,
+        "module_type": "attention",
+        "coordinate_dim": 4096,
+        "start": 98304,
+        "stop": 102400
+      },
+      {
+        "name": "layer12.mlp",
+        "layer": 12,
+        "module_type": "MLP",
+        "coordinate_dim": 4096,
+        "start": 102400,
+        "stop": 106496
+      },
+      {
+        "name": "layer13.attn",
+        "layer": 13,
+        "module_type": "attention",
+        "coordinate_dim": 4096,
+        "start": 106496,
+        "stop": 110592
+      },
+      {
+        "name": "layer13.mlp",
+        "layer": 13,
+        "module_type": "MLP",
+        "coordinate_dim": 4096,
+        "start": 110592,
+        "stop": 114688
+      },
+      {
+        "name": "layer14.attn",
+        "layer": 14,
+        "module_type": "attention",
+        "coordinate_dim": 4096,
+        "start": 114688,
+        "stop": 118784
+      },
+      {
+        "name": "layer14.mlp",
+        "layer": 14,
+        "module_type": "MLP",
+        "coordinate_dim": 4096,
+        "start": 118784,
+        "stop": 122880
+      },
+      {
+        "name": "layer15.attn",
+        "layer": 15,
+        "module_type": "attention",
+        "coordinate_dim": 4096,
+        "start": 122880,
+        "stop": 126976
+      },
+      {
+        "name": "layer15.mlp",
+        "layer": 15,
+        "module_type": "MLP",
+        "coordinate_dim": 4096,
+        "start": 126976,
+        "stop": 131072
+      },
+      {
+        "name": "layer16.attn",
+        "layer": 16,
+        "module_type": "attention",
+        "coordinate_dim": 4096,
+        "start": 131072,
+        "stop": 135168
+      },
+      {
+        "name": "layer16.mlp",
+        "layer": 16,
+        "module_type": "MLP",
+        "coordinate_dim": 4096,
+        "start": 135168,
+        "stop": 139264
+      },
+      {
+        "name": "layer17.attn",
+        "layer": 17,
+        "module_type": "attention",
+        "coordinate_dim": 4096,
+        "start": 139264,
+        "stop": 143360
+      },
+      {
+        "name": "layer17.mlp",
+        "layer": 17,
+        "module_type": "MLP",
+        "coordinate_dim": 4096,
+        "start": 143360,
+        "stop": 147456
+      },
+      {
+        "name": "layer18.attn",
+        "layer": 18,
+        "module_type": "attention",
+        "coordinate_dim": 4096,
+        "start": 147456,
+        "stop": 151552
+      },
+      {
+        "name": "layer18.mlp",
+        "layer": 18,
+        "module_type": "MLP",
+        "coordinate_dim": 4096,
+        "start": 151552,
+        "stop": 155648
+      },
+      {
+        "name": "layer19.attn",
+        "layer": 19,
+        "module_type": "attention",
+        "coordinate_dim": 4096,
+        "start": 155648,
+        "stop": 159744
+      },
+      {
+        "name": "layer19.mlp",
+        "layer": 19,
+        "module_type": "MLP",
+        "coordinate_dim": 4096,
+        "start": 159744,
+        "stop": 163840
+      },
+      {
+        "name": "layer20.attn",
+        "layer": 20,
+        "module_type": "attention",
+        "coordinate_dim": 4096,
+        "start": 163840,
+        "stop": 167936
+      },
+      {
+        "name": "layer20.mlp",
+        "layer": 20,
+        "module_type": "MLP",
+        "coordinate_dim": 4096,
+        "start": 167936,
+        "stop": 172032
+      },
+      {
+        "name": "layer21.attn",
+        "layer": 21,
+        "module_type": "attention",
+        "coordinate_dim": 4096,
+        "start": 172032,
+        "stop": 176128
+      },
+      {
+        "name": "layer21.mlp",
+        "layer": 21,
+        "module_type": "MLP",
+        "coordinate_dim": 4096,
+        "start": 176128,
+        "stop": 180224
+      },
+      {
+        "name": "layer22.attn",
+        "layer": 22,
+        "module_type": "attention",
+        "coordinate_dim": 4096,
+        "start": 180224,
+        "stop": 184320
+      },
+      {
+        "name": "layer22.mlp",
+        "layer": 22,
+        "module_type": "MLP",
+        "coordinate_dim": 4096,
+        "start": 184320,
+        "stop": 188416
+      },
+      {
+        "name": "layer23.attn",
+        "layer": 23,
+        "module_type": "attention",
+        "coordinate_dim": 4096,
+        "start": 188416,
+        "stop": 192512
+      },
+      {
+        "name": "layer23.mlp",
+        "layer": 23,
+        "module_type": "MLP",
+        "coordinate_dim": 4096,
+        "start": 192512,
+        "stop": 196608
+      },
+      {
+        "name": "layer24.attn",
+        "layer": 24,
+        "module_type": "attention",
+        "coordinate_dim": 4096,
+        "start": 196608,
+        "stop": 200704
+      },
+      {
+        "name": "layer24.mlp",
+        "layer": 24,
+        "module_type": "MLP",
+        "coordinate_dim": 4096,
+        "start": 200704,
+        "stop": 204800
+      },
+      {
+        "name": "layer25.attn",
+        "layer": 25,
+        "module_type": "attention",
+        "coordinate_dim": 4096,
+        "start": 204800,
+        "stop": 208896
+      },
+      {
+        "name": "layer25.mlp",
+        "layer": 25,
+        "module_type": "MLP",
+        "coordinate_dim": 4096,
+        "start": 208896,
+        "stop": 212992
+      },
+      {
+        "name": "layer26.attn",
+        "layer": 26,
+        "module_type": "attention",
+        "coordinate_dim": 4096,
+        "start": 212992,
+        "stop": 217088
+      },
+      {
+        "name": "layer26.mlp",
+        "layer": 26,
+        "module_type": "MLP",
+        "coordinate_dim": 4096,
+        "start": 217088,
+        "stop": 221184
+      },
+      {
+        "name": "layer27.attn",
+        "layer": 27,
+        "module_type": "attention",
+        "coordinate_dim": 4096,
+        "start": 221184,
+        "stop": 225280
+      },
+      {
+        "name": "layer27.mlp",
+        "layer": 27,
+        "module_type": "MLP",
+        "coordinate_dim": 4096,
+        "start": 225280,
+        "stop": 229376
+      },
+      {
+        "name": "layer28.attn",
+        "layer": 28,
+        "module_type": "attention",
+        "coordinate_dim": 4096,
+        "start": 229376,
+        "stop": 233472
+      },
+      {
+        "name": "layer28.mlp",
+        "layer": 28,
+        "module_type": "MLP",
+        "coordinate_dim": 4096,
+        "start": 233472,
+        "stop": 237568
+      },
+      {
+        "name": "layer29.attn",
+        "layer": 29,
+        "module_type": "attention",
+        "coordinate_dim": 4096,
+        "start": 237568,
+        "stop": 241664
+      },
+      {
+        "name": "layer29.mlp",
+        "layer": 29,
+        "module_type": "MLP",
+        "coordinate_dim": 4096,
+        "start": 241664,
+        "stop": 245760
+      },
+      {
+        "name": "layer30.attn",
+        "layer": 30,
+        "module_type": "attention",
+        "coordinate_dim": 4096,
+        "start": 245760,
+        "stop": 249856
+      },
+      {
+        "name": "layer30.mlp",
+        "layer": 30,
+        "module_type": "MLP",
+        "coordinate_dim": 4096,
+        "start": 249856,
+        "stop": 253952
+      },
+      {
+        "name": "layer31.attn",
+        "layer": 31,
+        "module_type": "attention",
+        "coordinate_dim": 4096,
+        "start": 253952,
+        "stop": 258048
+      },
+      {
+        "name": "layer31.mlp",
+        "layer": 31,
+        "module_type": "MLP",
+        "coordinate_dim": 4096,
+        "start": 258048,
+        "stop": 262144
+      }
+    ],
+    "module_names": [
+      "layer00.attn",
+      "layer00.mlp",
+      "layer01.attn",
+      "layer01.mlp",
+      "layer02.attn",
+      "layer02.mlp",
+      "layer03.attn",
+      "layer03.mlp",
+      "layer04.attn",
+      "layer04.mlp",
+      "layer05.attn",
+      "layer05.mlp",
+      "layer06.attn",
+      "layer06.mlp",
+      "layer07.attn",
+      "layer07.mlp",
+      "layer08.attn",
+      "layer08.mlp",
+      "layer09.attn",
+      "layer09.mlp",
+      "layer10.attn",
+      "layer10.mlp",
+      "layer11.attn",
+      "layer11.mlp",
+      "layer12.attn",
+      "layer12.mlp",
+      "layer13.attn",
+      "layer13.mlp",
+      "layer14.attn",
+      "layer14.mlp",
+      "layer15.attn",
+      "layer15.mlp",
+      "layer16.attn",
+      "layer16.mlp",
+      "layer17.attn",
+      "layer17.mlp",
+      "layer18.attn",
+      "layer18.mlp",
+      "layer19.attn",
+      "layer19.mlp",
+      "layer20.attn",
+      "layer20.mlp",
+      "layer21.attn",
+      "layer21.mlp",
+      "layer22.attn",
+      "layer22.mlp",
+      "layer23.attn",
+      "layer23.mlp",
+      "layer24.attn",
+      "layer24.mlp",
+      "layer25.attn",
+      "layer25.mlp",
+      "layer26.attn",
+      "layer26.mlp",
+      "layer27.attn",
+      "layer27.mlp",
+      "layer28.attn",
+      "layer28.mlp",
+      "layer29.attn",
+      "layer29.mlp",
+      "layer30.attn",
+      "layer30.mlp",
+      "layer31.attn",
+      "layer31.mlp"
+    ],
+    "module_dimensions": [
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096,
+      4096
+    ],
+    "module_ranges": {
+      "layer00.attn": [
+        0,
+        4096
+      ],
+      "layer00.mlp": [
+        4096,
+        8192
+      ],
+      "layer01.attn": [
+        8192,
+        12288
+      ],
+      "layer01.mlp": [
+        12288,
+        16384
+      ],
+      "layer02.attn": [
+        16384,
+        20480
+      ],
+      "layer02.mlp": [
+        20480,
+        24576
+      ],
+      "layer03.attn": [
+        24576,
+        28672
+      ],
+      "layer03.mlp": [
+        28672,
+        32768
+      ],
+      "layer04.attn": [
+        32768,
+        36864
+      ],
+      "layer04.mlp": [
+        36864,
+        40960
+      ],
+      "layer05.attn": [
+        40960,
+        45056
+      ],
+      "layer05.mlp": [
+        45056,
+        49152
+      ],
+      "layer06.attn": [
+        49152,
+        53248
+      ],
+      "layer06.mlp": [
+        53248,
+        57344
+      ],
+      "layer07.attn": [
+        57344,
+        61440
+      ],
+      "layer07.mlp": [
+        61440,
+        65536
+      ],
+      "layer08.attn": [
+        65536,
+        69632
+      ],
+      "layer08.mlp": [
+        69632,
+        73728
+      ],
+      "layer09.attn": [
+        73728,
+        77824
+      ],
+      "layer09.mlp": [
+        77824,
+        81920
+      ],
+      "layer10.attn": [
+        81920,
+        86016
+      ],
+      "layer10.mlp": [
+        86016,
+        90112
+      ],
+      "layer11.attn": [
+        90112,
+        94208
+      ],
+      "layer11.mlp": [
+        94208,
+        98304
+      ],
+      "layer12.attn": [
+        98304,
+        102400
+      ],
+      "layer12.mlp": [
+        102400,
+        106496
+      ],
+      "layer13.attn": [
+        106496,
+        110592
+      ],
+      "layer13.mlp": [
+        110592,
+        114688
+      ],
+      "layer14.attn": [
+        114688,
+        118784
+      ],
+      "layer14.mlp": [
+        118784,
+        122880
+      ],
+      "layer15.attn": [
+        122880,
+        126976
+      ],
+      "layer15.mlp": [
+        126976,
+        131072
+      ],
+      "layer16.attn": [
+        131072,
+        135168
+      ],
+      "layer16.mlp": [
+        135168,
+        139264
+      ],
+      "layer17.attn": [
+        139264,
+        143360
+      ],
+      "layer17.mlp": [
+        143360,
+        147456
+      ],
+      "layer18.attn": [
+        147456,
+        151552
+      ],
+      "layer18.mlp": [
+        151552,
+        155648
+      ],
+      "layer19.attn": [
+        155648,
+        159744
+      ],
+      "layer19.mlp": [
+        159744,
+        163840
+      ],
+      "layer20.attn": [
+        163840,
+        167936
+      ],
+      "layer20.mlp": [
+        167936,
+        172032
+      ],
+      "layer21.attn": [
+        172032,
+        176128
+      ],
+      "layer21.mlp": [
+        176128,
+        180224
+      ],
+      "layer22.attn": [
+        180224,
+        184320
+      ],
+      "layer22.mlp": [
+        184320,
+        188416
+      ],
+      "layer23.attn": [
+        188416,
+        192512
+      ],
+      "layer23.mlp": [
+        192512,
+        196608
+      ],
+      "layer24.attn": [
+        196608,
+        200704
+      ],
+      "layer24.mlp": [
+        200704,
+        204800
+      ],
+      "layer25.attn": [
+        204800,
+        208896
+      ],
+      "layer25.mlp": [
+        208896,
+        212992
+      ],
+      "layer26.attn": [
+        212992,
+        217088
+      ],
+      "layer26.mlp": [
+        217088,
+        221184
+      ],
+      "layer27.attn": [
+        221184,
+        225280
+      ],
+      "layer27.mlp": [
+        225280,
+        229376
+      ],
+      "layer28.attn": [
+        229376,
+        233472
+      ],
+      "layer28.mlp": [
+        233472,
+        237568
+      ],
+      "layer29.attn": [
+        237568,
+        241664
+      ],
+      "layer29.mlp": [
+        241664,
+        245760
+      ],
+      "layer30.attn": [
+        245760,
+        249856
+      ],
+      "layer30.mlp": [
+        249856,
+        253952
+      ],
+      "layer31.attn": [
+        253952,
+        258048
+      ],
+      "layer31.mlp": [
+        258048,
+        262144
+      ]
+    },
+    "layer_count": 32,
+    "hidden_size": 4096,
+    "uniform_module_width": true,
+    "probe_blocks": [
+      [
+        0,
+        1,
+        2,
+        3,
+        4,
+        5,
+        6,
+        7,
+        8,
+        9,
+        10,
+        11,
+        12,
+        13,
+        14,
+        15
+      ]
+    ],
+    "gate_dtype": "float32",
+    "ungated_first_window_nll": 7.346001625061035,
+    "unit_gate_nll_error": 0.0,
+    "repeat_window_index": 0,
+    "repeat_vector_exact": true,
+    "repeat_vector_l2_error": 0.0,
+    "repeat_window_nll": 7.346001625061035,
+    "source_parameter_versions_unchanged": true,
+    "non_gate_trainable_parameters": 0,
+    "gates_removed_before_native_restore": true,
+    "windows": 16,
+    "module_graph_restored": true,
+    "raw_mean_norm": 1.5685211541040713
+  },
+  "optimization_steps": 0
+}

basis/gradients/q128/llama31/asvd_r080/step_0000/block_5/vectors.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:972c13792b4e4cbcb5dfe215953b95d6ac38e20f155a40f639ed726f6576e9f0
+size 17831065