Add seven calibrated GFWSVD and Qwen Swift-SVD Source checkpoints

README.md

@@ -2,17 +2,19 @@
 
 ## BASIS: download experiment inputs here
 
-For the [Additional Experiments](https://github.com/Zishan-Shao/Basis/tree/handoff/paper-20260915/experiments/intern), start with the [BASIS data guide](https://huggingface.co/Duke-CEI-SVD/FlashSVD/blob/refs%2Fpr%2F1/basis/README.md).
+For the [Additional Experiments](https://github.com/Zishan-Shao/Basis/tree/handoff/paper-20260915/experiments/intern), start with the [BASIS data guide](https://huggingface.co/Duke-CEI-SVD/FlashSVD/blob/main/basis/README.md).
 
 | You need… | Go to… |
 |---|---|
-| **C4 training data** | [Six frozen C4 streams: two backbones × three seeds](https://huggingface.co/Duke-CEI-SVD/FlashSVD/tree/refs%2Fpr%2F1/basis/data/c4) |
-| Q64/Q128 probe and held-out tensors | [Probe inputs](https://huggingface.co/Duke-CEI-SVD/FlashSVD/tree/refs%2Fpr%2F1/basis/data/probes) |
-| Cached per-window gradients | [Q128](https://huggingface.co/Duke-CEI-SVD/FlashSVD/tree/refs%2Fpr%2F1/basis/gradients/q128) · [Original Q64](https://huggingface.co/Duke-CEI-SVD/FlashSVD/tree/refs%2Fpr%2F1/basis/gradients/q64) |
-| Exact frozen compressed Sources | [16 paper checkpoints plus four LowRankArena imports](https://huggingface.co/Duke-CEI-SVD/FlashSVD/tree/refs%2Fpr%2F1/basis/sources) |
-| Tokenizers, original paths, and checksums | [Tokenizers](https://huggingface.co/Duke-CEI-SVD/FlashSVD/tree/refs%2Fpr%2F1/basis/tokenizers) · [Artifact manifest](https://huggingface.co/Duke-CEI-SVD/FlashSVD/blob/refs%2Fpr%2F1/basis/manifest.json) |
+| **C4 training data** | [Six frozen C4 streams: two backbones × three seeds](https://huggingface.co/Duke-CEI-SVD/FlashSVD/tree/main/basis/data/c4) |
+| Q64/Q128 probe and held-out tensors | [Probe inputs](https://huggingface.co/Duke-CEI-SVD/FlashSVD/tree/main/basis/data/probes) |
+| Cached per-window gradients | [Q128](https://huggingface.co/Duke-CEI-SVD/FlashSVD/tree/main/basis/gradients/q128) · [Original Q64](https://huggingface.co/Duke-CEI-SVD/FlashSVD/tree/main/basis/gradients/q64) |
+| Exact frozen compressed Sources | [23 hosted checkpoints plus four LowRankArena imports](https://huggingface.co/Duke-CEI-SVD/FlashSVD/tree/main/basis/sources) |
+| Tokenizers, original paths, and checksums | [Tokenizers](https://huggingface.co/Duke-CEI-SVD/FlashSVD/tree/main/basis/tokenizers) · [Artifact manifest](https://huggingface.co/Duke-CEI-SVD/FlashSVD/blob/main/basis/manifest.json) |
 
-The shared data and caches total approximately **4.4 GiB**; the 16 hosted checkpoints total **178.6 GiB**. Download only the Source files needed for your task. The guide includes selective download commands, loader instructions, checkpoint coverage, and the remaining Task 05 Source requests. Tasks 01–02 already include their numerical inputs in GitHub.
+The shared data and caches total approximately **4.4 GiB**; the 23 hosted checkpoints total **246.0 GiB**. Download only the Source files needed for your task. The guide includes selective download commands, loader instructions, checkpoint coverage, and the seven newly supplied Task 05 GFWSVD/Swift-SVD ratio Sources. Tasks 01–02 already include their numerical inputs in GitHub.
+
+**Added 2026-09-17:** [GFWSVD r0.5/r0.7 (Llama/Qwen) and Swift-SVD r0.4/r0.5/r0.7 (Qwen), with download instructions and checksums](basis/releases/source_ratios_20260917/README.md).
 
 ## Original FlashSVD example checkpoints
 
@@ -35,3 +37,7 @@ source = "Duke-CEI-SVD/FlashSVD/llama_7b/svdllm_v1/whitening_only_0.5_hf"
 model, tok = get_model_from_source(source)
 model = model.eval().to("cuda", dtype=torch.bfloat16)
 ```
+
+## BASIS intern code
+
+Use the [intern GitHub branch](https://github.com/Zishan-Shao/Basis/tree/handoff/paper-20260915) for installation, Source loading/checking, compression, and the five assignments. [Tested code revision](https://github.com/Zishan-Shao/Basis/tree/57218ca8c09e12464adaddddc4924394eac668f5): `57218ca`. The GitHub downloader pins the verified FlashSVD checkpoint revision; download weights/data here and run the commands from that branch.

basis/README.md

@@ -6,18 +6,20 @@ Shared inputs for the [BASIS Additional Experiments](https://github.com/Zishan-S
 
 | Input | Folder | Size | Use |
 |---|---|---:|---|
-| **Frozen C4 recovery streams** | [`data/c4/`](https://huggingface.co/Duke-CEI-SVD/FlashSVD/tree/refs%2Fpr%2F1/basis/data/c4/) | 0.604 GiB | Task 05 training; Task 03 overlap exclusions |
-| Original Q64/V64 | [`data/probes/q64/`](https://huggingface.co/Duke-CEI-SVD/FlashSVD/tree/refs%2Fpr%2F1/basis/data/probes/q64/) | <1 MiB | Original Q0/P0 uses the first 16 Q64 windows |
-| Independent Q128/V128 | [`data/probes/q128/`](https://huggingface.co/Duke-CEI-SVD/FlashSVD/tree/refs%2Fpr%2F1/basis/data/probes/q128/) | 1.1 MiB | Tasks 03–04 |
-| C4 validation shard for extending Q128 | [`data/c4_probe_shard/`](https://huggingface.co/Duke-CEI-SVD/FlashSVD/tree/refs%2Fpr%2F1/basis/data/c4_probe_shard/) | 38.6 MiB | Task 03 candidate documents |
-| Q128 Source/dense gradient cache | [`gradients/q128/`](https://huggingface.co/Duke-CEI-SVD/FlashSVD/tree/refs%2Fpr%2F1/basis/gradients/q128/) | 1.978 GiB | Task 03: reuse the existing first 128 windows |
-| Original Q64 gradient cache | [`gradients/q64/`](https://huggingface.co/Duke-CEI-SVD/FlashSVD/tree/refs%2Fpr%2F1/basis/gradients/q64/) | 1.764 GiB | Original pressure references and dense Q0 reuse |
-| Frozen tokenizers | [`tokenizers/`](https://huggingface.co/Duke-CEI-SVD/FlashSVD/tree/refs%2Fpr%2F1/basis/tokenizers/) | 18.3 MiB | Both backbones |
-| Frozen compressed Sources | [`sources/`](https://huggingface.co/Duke-CEI-SVD/FlashSVD/tree/refs%2Fpr%2F1/basis/sources/) | 178.633 GiB hosted here | Download individual checkpoints, usually 7–14 GiB |
-| Original manifests and example C4 run specs | [`provenance/`](https://huggingface.co/Duke-CEI-SVD/FlashSVD/tree/refs%2Fpr%2F1/basis/provenance/) | <1 MiB | Input identity and protocol lookup |
+| **Frozen C4 recovery streams** | [`data/c4/`](https://huggingface.co/Duke-CEI-SVD/FlashSVD/tree/main/basis/data/c4/) | 0.604 GiB | Task 05 training; Task 03 overlap exclusions |
+| Original Q64/V64 | [`data/probes/q64/`](https://huggingface.co/Duke-CEI-SVD/FlashSVD/tree/main/basis/data/probes/q64/) | <1 MiB | Original Q0/P0 uses the first 16 Q64 windows |
+| Independent Q128/V128 | [`data/probes/q128/`](https://huggingface.co/Duke-CEI-SVD/FlashSVD/tree/main/basis/data/probes/q128/) | 1.1 MiB | Tasks 03–04 |
+| C4 validation shard for extending Q128 | [`data/c4_probe_shard/`](https://huggingface.co/Duke-CEI-SVD/FlashSVD/tree/main/basis/data/c4_probe_shard/) | 38.6 MiB | Task 03 candidate documents |
+| Q128 Source/dense gradient cache | [`gradients/q128/`](https://huggingface.co/Duke-CEI-SVD/FlashSVD/tree/main/basis/gradients/q128/) | 1.978 GiB | Task 03: reuse the existing first 128 windows |
+| Original Q64 gradient cache | [`gradients/q64/`](https://huggingface.co/Duke-CEI-SVD/FlashSVD/tree/main/basis/gradients/q64/) | 1.764 GiB | Original pressure references and dense Q0 reuse |
+| Frozen tokenizers | [`tokenizers/`](https://huggingface.co/Duke-CEI-SVD/FlashSVD/tree/main/basis/tokenizers/) | 18.3 MiB | Both backbones |
+| Frozen compressed Sources | [`sources/`](https://huggingface.co/Duke-CEI-SVD/FlashSVD/tree/main/basis/sources/) | 245.958 GiB hosted here | Download individual checkpoints, usually 7–14 GiB |
+| Original manifests and example C4 run specs | [`provenance/`](https://huggingface.co/Duke-CEI-SVD/FlashSVD/tree/main/basis/provenance/) | <1 MiB | Input identity and protocol lookup |
 
 [manifest.json](manifest.json) records every artifact's relative path, original path, byte count, and SHA-256, together with Source identities and dense-parent revisions. The linked GitHub repository requires collaborator access.
 
+> **New ratios (2026-09-17):** [seven newly published GFWSVD/Swift-SVD Sources](releases/source_ratios_20260917/README.md). Old BASIS download scripts pinned to `3462c6bc...` will not list them; use the direct-download example or update the bundle pin.
+
 ## C4: start here for Task 05
 
 Each stream is stored as:
@@ -69,12 +71,14 @@ Use `--list` to inspect groups and Source IDs, or add `--dry-run` to check downl
 
 ## Checkpoint coverage and loading
 
-The [Source index (CSV)](sources/index.csv) lists all checkpoint paths, sizes, loader kinds, and identities. This package supports **20 frozen compressed Sources**: **16 checkpoints are hosted here**, and **four reuse existing LowRankArena weights** through pinned download references. The downloader selects the correct repository automatically.
+The [Source index (CSV)](sources/index.csv) lists all checkpoint paths, sizes, loader kinds, and identities. This package supports **27 frozen compressed Sources**: **23 checkpoints are hosted here**, and **four reuse existing LowRankArena weights** through pinned download references. The downloader selects the correct repository automatically.
 
 - All six methods at **r0.8** for both backbones: 12 Sources for Task 03.
 - **SVD-LLM, Swift-SVD, and MoDeGPT at r0.6** for both backbones: six Sources for Task 04.
 - **GFWSVD at r0.4** for both backbones: two existing constructions for the Task 05 inventory. Their manifests specify nested leading-factor truncation of the frozen r0.8 Source, with no new calibration; retain that recipe when assessing reuse.
 
+- **Seven new Task 05 Sources:** GFWSVD r0.5/r0.7 for Llama/Qwen and Swift-SVD r0.4/r0.5/r0.7 for Qwen. Each has a construction manifest and validation record. [Download and load them](releases/source_ratios_20260917/README.md).
+
 ### Reuse existing LowRankArena weights
 
 The downloader reuses **Qwen3 ASVD r0.8, MoDeGPT r0.8/r0.6, and Basis Sharing r0.8** from LowRankArena. Their weight files are not duplicated in FlashSVD. Original runtime/config metadata is supplied here where needed to preserve the paper Source.
@@ -95,7 +99,7 @@ model = load_checkpoint(
 
 Use the corresponding `source.json` path and `kind`, and follow the assignment's dtype and tokenizer conventions.
 
-Additional compressed model candidates are available from [LowRankArena](https://huggingface.co/Duke-CEI-SVD/LowRankArena), including Llama Swift-SVD under `new_ckpt/`. Match the construction recipe and calibration as well as method and ratio. At the audited LowRankArena revision `d95d6ebddec25227ebc87d4814a60362dfda5cd8`, the Task 05 panel has nominal method/backbone/ratio coverage for 27 of 36 configurations. The two GFWSVD r0.4 files here add two candidates. **GFWSVD r0.5/r0.7 for both backbones and Qwen3 Swift-SVD r0.4/r0.5/r0.7 remain unprovided**; record these seven entries in the inventory and coordinate with the project lead for prepared Sources.
+Additional compressed model candidates are available from [LowRankArena](https://huggingface.co/Duke-CEI-SVD/LowRankArena), including Llama Swift-SVD under `new_ckpt/`. Match the construction recipe and calibration as well as method and ratio. At the audited LowRankArena revision `d95d6ebddec25227ebc87d4814a60362dfda5cd8`, the Task 05 panel has nominal method/backbone/ratio coverage for 27 of 36 configurations. The two GFWSVD r0.4 files here add two candidates. **The seven remaining Sources are now supplied here:** GFWSVD r0.5/r0.7 for both backbones and Qwen3 Swift-SVD r0.4/r0.5/r0.7. See the [new ratio Sources and direct-download instructions](releases/source_ratios_20260917/README.md). Together with the audited LowRankArena catalog this supplies nominal method/backbone/ratio candidates for all 36 Task 05 entries. Recipe/calibration matching, Source evaluations and recovery runs still require verification; nominal coverage is not a completed experiment.
 
 Dense weights are available from their original repositories, pinned to:
 
@@ -109,3 +113,7 @@ Dense weights are available from their original repositories, pinned to:
 Original manifests retain their historical paths and hashes. Use the package inventory or downloader's path map to create fresh task manifests, while retaining the originals for provenance. The path map describes the full inventory; download an input before using its mapped path. Relocation does not make old execution locks valid on a new machine.
 
 This package includes Source/dense measurements, not recovered adapters or optimizer states. Avg7 benchmark data must be prepared using the frozen task definitions linked from the assignment. C4 originates from [allenai/c4](https://huggingface.co/datasets/allenai/c4); retain the original dataset and model access conditions.
+
+## BASIS intern code
+
+Use the [intern GitHub branch](https://github.com/Zishan-Shao/Basis/tree/handoff/paper-20260915) for installation, Source loading/checking, compression, and the five assignments. [Tested code revision](https://github.com/Zishan-Shao/Basis/tree/57218ca8c09e12464adaddddc4924394eac668f5): `57218ca`. The GitHub downloader pins the verified FlashSVD checkpoint revision; download weights/data here and run the commands from that branch.

basis/manifest.json

@@ -4163,6 +4163,230 @@
       "group": "sources",
       "hash_basis": "sha256_verified",
       "generated_metadata": true
+    },
+    {
+      "path": "basis/sources/llama31/gfwsvd_r050/source.json",
+      "original_path": "/home/zs89/Basis/outputs/source_delivery_20260917/publication/stage/basis/sources/llama31/gfwsvd_r050/source.json",
+      "bytes": 800,
+      "sha256": "e9d85e7435cb11db56f2e516dd41e6c4f525d73ab19a72e4581e04ddbba8f9f3",
+      "group": "sources",
+      "hash_basis": "sha256_verified"
+    },
+    {
+      "path": "basis/sources/llama31/gfwsvd_r050/provenance/manifest.json",
+      "original_path": "/home/zs89/Basis/outputs/source_delivery_20260917/publication/stage/basis/sources/llama31/gfwsvd_r050/provenance/manifest.json",
+      "bytes": 89994,
+      "sha256": "5bf7ec28f4543876afed572238b4a3cfc3ac68c74a11d98d9db1427e358eb96c",
+      "group": "sources",
+      "hash_basis": "sha256_verified"
+    },
+    {
+      "path": "basis/sources/llama31/gfwsvd_r050/validation.json",
+      "original_path": "/home/zs89/Basis/outputs/source_delivery_20260917/publication/stage/basis/sources/llama31/gfwsvd_r050/validation.json",
+      "bytes": 670,
+      "sha256": "77b47ee966740c81f5b4d0fe4d3b8a1d78f4e6d5de9c095b93b1851678638c44",
+      "group": "sources",
+      "hash_basis": "sha256_verified"
+    },
+    {
+      "path": "basis/sources/llama31/gfwsvd_r050/model.pt",
+      "original_path": "/home/zs89/Basis/outputs/source_delivery_20260917/sources/llama31/gfwsvd_r050/model.pt",
+      "bytes": 9078011199,
+      "sha256": "1c1f65a0d57817c76d64b00466be8aa533d22673d8aaf567d7960d138499d266",
+      "group": "sources",
+      "hash_basis": "sha256_verified"
+    },
+    {
+      "path": "basis/sources/llama31/gfwsvd_r070/source.json",
+      "original_path": "/home/zs89/Basis/outputs/source_delivery_20260917/publication/stage/basis/sources/llama31/gfwsvd_r070/source.json",
+      "bytes": 800,
+      "sha256": "9c12788b813ffef5eea93c86849d6186b7b860bd31031a08372938260688d6bd",
+      "group": "sources",
+      "hash_basis": "sha256_verified"
+    },
+    {
+      "path": "basis/sources/llama31/gfwsvd_r070/provenance/manifest.json",
+      "original_path": "/home/zs89/Basis/outputs/source_delivery_20260917/publication/stage/basis/sources/llama31/gfwsvd_r070/provenance/manifest.json",
+      "bytes": 89993,
+      "sha256": "2477352f1ed98e53b170abacc8ab619969de142aa87b4834301aa2092bd2525a",
+      "group": "sources",
+      "hash_basis": "sha256_verified"
+    },
+    {
+      "path": "basis/sources/llama31/gfwsvd_r070/validation.json",
+      "original_path": "/home/zs89/Basis/outputs/source_delivery_20260917/publication/stage/basis/sources/llama31/gfwsvd_r070/validation.json",
+      "bytes": 669,
+      "sha256": "d91cf92c8f1119496b7e79e07287e2046ecc50809efa00d0a60e03d4c21a46b3",
+      "group": "sources",
+      "hash_basis": "sha256_verified"
+    },
+    {
+      "path": "basis/sources/llama31/gfwsvd_r070/model.pt",
+      "original_path": "/home/zs89/Basis/outputs/source_delivery_20260917/sources/llama31/gfwsvd_r070/model.pt",
+      "bytes": 11872204875,
+      "sha256": "b43045553b40a499d73f7460fc395622c0b2c737cc81555f8febe995a115f913",
+      "group": "sources",
+      "hash_basis": "sha256_verified"
+    },
+    {
+      "path": "basis/sources/qwen3/gfwsvd_r050/source.json",
+      "original_path": "/home/zs89/Basis/outputs/source_delivery_20260917/publication/stage/basis/sources/qwen3/gfwsvd_r050/source.json",
+      "bytes": 788,
+      "sha256": "54ac8a893d32152c916bae70b2725da394928a89261864414f9e829bdf66630d",
+      "group": "sources",
+      "hash_basis": "sha256_verified"
+    },
+    {
+      "path": "basis/sources/qwen3/gfwsvd_r050/provenance/manifest.json",
+      "original_path": "/home/zs89/Basis/outputs/source_delivery_20260917/publication/stage/basis/sources/qwen3/gfwsvd_r050/provenance/manifest.json",
+      "bytes": 109960,
+      "sha256": "b25a983025b02d0cdf7fe7dde12917a4eaed6b6102388abf2b12fc260f378267",
+      "group": "sources",
+      "hash_basis": "sha256_verified"
+    },
+    {
+      "path": "basis/sources/qwen3/gfwsvd_r050/validation.json",
+      "original_path": "/home/zs89/Basis/outputs/source_delivery_20260917/publication/stage/basis/sources/qwen3/gfwsvd_r050/validation.json",
+      "bytes": 665,
+      "sha256": "b40feb692c673a322323e0851e59664815ab3fade69b7ac6fa6e1fd01b107de6",
+      "group": "sources",
+      "hash_basis": "sha256_verified"
+    },
+    {
+      "path": "basis/sources/qwen3/gfwsvd_r050/model.pt",
+      "original_path": "/home/zs89/Basis/outputs/source_delivery_20260917/sources/qwen3/gfwsvd_r050/model.pt",
+      "bytes": 9435690527,
+      "sha256": "9c4537ab505b781236533d549c829e5150ac4108116646320a1522b2893fd226",
+      "group": "sources",
+      "hash_basis": "sha256_verified"
+    },
+    {
+      "path": "basis/sources/qwen3/gfwsvd_r070/source.json",
+      "original_path": "/home/zs89/Basis/outputs/source_delivery_20260917/publication/stage/basis/sources/qwen3/gfwsvd_r070/source.json",
+      "bytes": 788,
+      "sha256": "039e729a574ffbf728090bc5e86e275f4cf52360ed60dd446e9f12b15013b71f",
+      "group": "sources",
+      "hash_basis": "sha256_verified"
+    },
+    {
+      "path": "basis/sources/qwen3/gfwsvd_r070/provenance/manifest.json",
+      "original_path": "/home/zs89/Basis/outputs/source_delivery_20260917/publication/stage/basis/sources/qwen3/gfwsvd_r070/provenance/manifest.json",
+      "bytes": 109959,
+      "sha256": "2d50dd15762262d31f8943585f48ff7cfc554d178b7f2a27070e2a11d96eeff6",
+      "group": "sources",
+      "hash_basis": "sha256_verified"
+    },
+    {
+      "path": "basis/sources/qwen3/gfwsvd_r070/validation.json",
+      "original_path": "/home/zs89/Basis/outputs/source_delivery_20260917/publication/stage/basis/sources/qwen3/gfwsvd_r070/validation.json",
+      "bytes": 665,
+      "sha256": "bf07f5171834d4a1281271f8a91b6aa7b6f8233cf3487539199f529632f3a8bb",
+      "group": "sources",
+      "hash_basis": "sha256_verified"
+    },
+    {
+      "path": "basis/sources/qwen3/gfwsvd_r070/model.pt",
+      "original_path": "/home/zs89/Basis/outputs/source_delivery_20260917/sources/qwen3/gfwsvd_r070/model.pt",
+      "bytes": 12211992995,
+      "sha256": "4aed75e0ba422ec94593d437883917ba8eefb542bfe899b9dc6e4184cd9f9281",
+      "group": "sources",
+      "hash_basis": "sha256_verified"
+    },
+    {
+      "path": "basis/sources/qwen3/swiftsvd_r040/source.json",
+      "original_path": "/home/zs89/Basis/outputs/source_delivery_20260917/publication/stage/basis/sources/qwen3/swiftsvd_r040/source.json",
+      "bytes": 800,
+      "sha256": "94814f2c74d750743197437e96bbf9902c6ed1a77ef4ee041c26def6deebf551",
+      "group": "sources",
+      "hash_basis": "sha256_verified"
+    },
+    {
+      "path": "basis/sources/qwen3/swiftsvd_r040/provenance/manifest.json",
+      "original_path": "/home/zs89/Basis/outputs/source_delivery_20260917/publication/stage/basis/sources/qwen3/swiftsvd_r040/provenance/manifest.json",
+      "bytes": 152209,
+      "sha256": "577ce96ae7e12172c5f8760ebfb1872e3a654f8730ad84432fe5e2b128b49a73",
+      "group": "sources",
+      "hash_basis": "sha256_verified"
+    },
+    {
+      "path": "basis/sources/qwen3/swiftsvd_r040/validation.json",
+      "original_path": "/home/zs89/Basis/outputs/source_delivery_20260917/publication/stage/basis/sources/qwen3/swiftsvd_r040/validation.json",
+      "bytes": 737,
+      "sha256": "7a26f1d3e5bf8814bdbf4cf832144acb82082f85f1686ca6620916107aa59087",
+      "group": "sources",
+      "hash_basis": "sha256_verified"
+    },
+    {
+      "path": "basis/sources/qwen3/swiftsvd_r040/model.pt",
+      "original_path": "/home/zs89/Basis/outputs/source_delivery_20260917/sources/qwen3/swiftsvd_r040/model.pt",
+      "bytes": 8043409011,
+      "sha256": "1e8b5a3db8feb63dd574f42b78913e6d5d55b0bcc8f4ef630c78db3f12305a6b",
+      "group": "sources",
+      "hash_basis": "sha256_verified"
+    },
+    {
+      "path": "basis/sources/qwen3/swiftsvd_r050/source.json",
+      "original_path": "/home/zs89/Basis/outputs/source_delivery_20260917/publication/stage/basis/sources/qwen3/swiftsvd_r050/source.json",
+      "bytes": 800,
+      "sha256": "4a51bf0e7f49732b26adfa64c2785a5d95451d0875a5ecc9b08e1ce01c68327d",
+      "group": "sources",
+      "hash_basis": "sha256_verified"
+    },
+    {
+      "path": "basis/sources/qwen3/swiftsvd_r050/provenance/manifest.json",
+      "original_path": "/home/zs89/Basis/outputs/source_delivery_20260917/publication/stage/basis/sources/qwen3/swiftsvd_r050/provenance/manifest.json",
+      "bytes": 152281,
+      "sha256": "ee289fac111ccc271db732ea2f5fbd6bc72bed45102c1062fd9b1bffb99ace14",
+      "group": "sources",
+      "hash_basis": "sha256_verified"
+    },
+    {
+      "path": "basis/sources/qwen3/swiftsvd_r050/validation.json",
+      "original_path": "/home/zs89/Basis/outputs/source_delivery_20260917/publication/stage/basis/sources/qwen3/swiftsvd_r050/validation.json",
+      "bytes": 670,
+      "sha256": "b2c6966e131f3ed0c6844cd028e08b9293e8fea4e76a3a8b5c7c6e0b51868310",
+      "group": "sources",
+      "hash_basis": "sha256_verified"
+    },
+    {
+      "path": "basis/sources/qwen3/swiftsvd_r050/model.pt",
+      "original_path": "/home/zs89/Basis/outputs/source_delivery_20260917/sources/qwen3/swiftsvd_r050/model.pt",
+      "bytes": 9435689427,
+      "sha256": "2a5281ae0ab3a32efe8bf1c0d8906461b0bfae774fba4c38da38996b4250a60f",
+      "group": "sources",
+      "hash_basis": "sha256_verified"
+    },
+    {
+      "path": "basis/sources/qwen3/swiftsvd_r070/source.json",
+      "original_path": "/home/zs89/Basis/outputs/source_delivery_20260917/publication/stage/basis/sources/qwen3/swiftsvd_r070/source.json",
+      "bytes": 800,
+      "sha256": "9581afbff019c1fbe6311923dc252af45b2e8a4f92969365f7c5cd51b8c700f8",
+      "group": "sources",
+      "hash_basis": "sha256_verified"
+    },
+    {
+      "path": "basis/sources/qwen3/swiftsvd_r070/provenance/manifest.json",
+      "original_path": "/home/zs89/Basis/outputs/source_delivery_20260917/publication/stage/basis/sources/qwen3/swiftsvd_r070/provenance/manifest.json",
+      "bytes": 152280,
+      "sha256": "dc81dec65cb434607ad45dd4d7c69d0e089404d1a68c3c5108e565257577ad4a",
+      "group": "sources",
+      "hash_basis": "sha256_verified"
+    },
+    {
+      "path": "basis/sources/qwen3/swiftsvd_r070/validation.json",
+      "original_path": "/home/zs89/Basis/outputs/source_delivery_20260917/publication/stage/basis/sources/qwen3/swiftsvd_r070/validation.json",
+      "bytes": 669,
+      "sha256": "0c7900220b0129555097642dc3f6f6b2815e4f0065be12731079e9e4e7287e78",
+      "group": "sources",
+      "hash_basis": "sha256_verified"
+    },
+    {
+      "path": "basis/sources/qwen3/swiftsvd_r070/model.pt",
+      "original_path": "/home/zs89/Basis/outputs/source_delivery_20260917/sources/qwen3/swiftsvd_r070/model.pt",
+      "bytes": 12211991895,
+      "sha256": "4069d003526013ba3ecdd1155a86cfffa66a29610132eb822ede22cb776d036a",
+      "group": "sources",
+      "hash_basis": "sha256_verified"
     }
   ],
   "sources": [
@@ -4671,6 +4895,153 @@
       ],
       "note": "Existing nested Source construction; verify the recorded recipe and actual retention in the Task 05 inventory.",
       "weight_repository": "Duke-CEI-SVD/FlashSVD"
+    },
+    {
+      "backbone": "llama31",
+      "checkpoint_identity": {
+        "checkpoint_sha256": "1c1f65a0d57817c76d64b00466be8aa533d22673d8aaf567d7960d138499d266"
+      },
+      "id": "llama31/gfwsvd_r050",
+      "kind": null,
+      "method": "gfwsvd",
+      "note": "New Task 05 Source from ordered frozen r=0.8 paper factors; same calibration, new checkpoint identity. Evaluate new pressure/recovery results.",
+      "path": "basis/sources/llama31/gfwsvd_r050/model.pt",
+      "provenance_files": [
+        "basis/sources/llama31/gfwsvd_r050/provenance/manifest.json"
+      ],
+      "retention": 0.5,
+      "tasks": [
+        "05"
+      ],
+      "original_path": "/home/zs89/Basis/outputs/source_delivery_20260917/sources/llama31/gfwsvd_r050/model.pt",
+      "validation": "basis/sources/llama31/gfwsvd_r050/validation.json",
+      "weight_repository": "Duke-CEI-SVD/FlashSVD"
+    },
+    {
+      "backbone": "llama31",
+      "checkpoint_identity": {
+        "checkpoint_sha256": "b43045553b40a499d73f7460fc395622c0b2c737cc81555f8febe995a115f913"
+      },
+      "id": "llama31/gfwsvd_r070",
+      "kind": null,
+      "method": "gfwsvd",
+      "note": "New Task 05 Source from ordered frozen r=0.8 paper factors; same calibration, new checkpoint identity. Evaluate new pressure/recovery results.",
+      "path": "basis/sources/llama31/gfwsvd_r070/model.pt",
+      "provenance_files": [
+        "basis/sources/llama31/gfwsvd_r070/provenance/manifest.json"
+      ],
+      "retention": 0.7,
+      "tasks": [
+        "05"
+      ],
+      "original_path": "/home/zs89/Basis/outputs/source_delivery_20260917/sources/llama31/gfwsvd_r070/model.pt",
+      "validation": "basis/sources/llama31/gfwsvd_r070/validation.json",
+      "weight_repository": "Duke-CEI-SVD/FlashSVD"
+    },
+    {
+      "backbone": "qwen3",
+      "checkpoint_identity": {
+        "checkpoint_sha256": "9c4537ab505b781236533d549c829e5150ac4108116646320a1522b2893fd226"
+      },
+      "id": "qwen3/gfwsvd_r050",
+      "kind": null,
+      "method": "gfwsvd",
+      "note": "New Task 05 Source from ordered frozen r=0.8 paper factors; same calibration, new checkpoint identity. Evaluate new pressure/recovery results.",
+      "path": "basis/sources/qwen3/gfwsvd_r050/model.pt",
+      "provenance_files": [
+        "basis/sources/qwen3/gfwsvd_r050/provenance/manifest.json"
+      ],
+      "retention": 0.5,
+      "tasks": [
+        "05"
+      ],
+      "original_path": "/home/zs89/Basis/outputs/source_delivery_20260917/sources/qwen3/gfwsvd_r050/model.pt",
+      "validation": "basis/sources/qwen3/gfwsvd_r050/validation.json",
+      "weight_repository": "Duke-CEI-SVD/FlashSVD"
+    },
+    {
+      "backbone": "qwen3",
+      "checkpoint_identity": {
+        "checkpoint_sha256": "4aed75e0ba422ec94593d437883917ba8eefb542bfe899b9dc6e4184cd9f9281"
+      },
+      "id": "qwen3/gfwsvd_r070",
+      "kind": null,
+      "method": "gfwsvd",
+      "note": "New Task 05 Source from ordered frozen r=0.8 paper factors; same calibration, new checkpoint identity. Evaluate new pressure/recovery results.",
+      "path": "basis/sources/qwen3/gfwsvd_r070/model.pt",
+      "provenance_files": [
+        "basis/sources/qwen3/gfwsvd_r070/provenance/manifest.json"
+      ],
+      "retention": 0.7,
+      "tasks": [
+        "05"
+      ],
+      "original_path": "/home/zs89/Basis/outputs/source_delivery_20260917/sources/qwen3/gfwsvd_r070/model.pt",
+      "validation": "basis/sources/qwen3/gfwsvd_r070/validation.json",
+      "weight_repository": "Duke-CEI-SVD/FlashSVD"
+    },
+    {
+      "backbone": "qwen3",
+      "checkpoint_identity": {
+        "checkpoint_sha256": "1e8b5a3db8feb63dd574f42b78913e6d5d55b0bcc8f4ef630c78db3f12305a6b"
+      },
+      "id": "qwen3/swiftsvd_r040",
+      "kind": null,
+      "method": "swiftsvd",
+      "note": "New Task 05 Source from ordered frozen r=0.8 paper factors; same calibration, new checkpoint identity. Evaluate new pressure/recovery results.",
+      "path": "basis/sources/qwen3/swiftsvd_r040/model.pt",
+      "provenance_files": [
+        "basis/sources/qwen3/swiftsvd_r040/provenance/manifest.json"
+      ],
+      "retention": 0.4,
+      "tasks": [
+        "05"
+      ],
+      "original_path": "/home/zs89/Basis/outputs/source_delivery_20260917/sources/qwen3/swiftsvd_r040/model.pt",
+      "validation": "basis/sources/qwen3/swiftsvd_r040/validation.json",
+      "weight_repository": "Duke-CEI-SVD/FlashSVD"
+    },
+    {
+      "backbone": "qwen3",
+      "checkpoint_identity": {
+        "checkpoint_sha256": "2a5281ae0ab3a32efe8bf1c0d8906461b0bfae774fba4c38da38996b4250a60f"
+      },
+      "id": "qwen3/swiftsvd_r050",
+      "kind": null,
+      "method": "swiftsvd",
+      "note": "New Task 05 Source from ordered frozen r=0.8 paper factors; same calibration, new checkpoint identity. Evaluate new pressure/recovery results.",
+      "path": "basis/sources/qwen3/swiftsvd_r050/model.pt",
+      "provenance_files": [
+        "basis/sources/qwen3/swiftsvd_r050/provenance/manifest.json"
+      ],
+      "retention": 0.5,
+      "tasks": [
+        "05"
+      ],
+      "original_path": "/home/zs89/Basis/outputs/source_delivery_20260917/sources/qwen3/swiftsvd_r050/model.pt",
+      "validation": "basis/sources/qwen3/swiftsvd_r050/validation.json",
+      "weight_repository": "Duke-CEI-SVD/FlashSVD"
+    },
+    {
+      "backbone": "qwen3",
+      "checkpoint_identity": {
+        "checkpoint_sha256": "4069d003526013ba3ecdd1155a86cfffa66a29610132eb822ede22cb776d036a"
+      },
+      "id": "qwen3/swiftsvd_r070",
+      "kind": null,
+      "method": "swiftsvd",
+      "note": "New Task 05 Source from ordered frozen r=0.8 paper factors; same calibration, new checkpoint identity. Evaluate new pressure/recovery results.",
+      "path": "basis/sources/qwen3/swiftsvd_r070/model.pt",
+      "provenance_files": [
+        "basis/sources/qwen3/swiftsvd_r070/provenance/manifest.json"
+      ],
+      "retention": 0.7,
+      "tasks": [
+        "05"
+      ],
+      "original_path": "/home/zs89/Basis/outputs/source_delivery_20260917/sources/qwen3/swiftsvd_r070/model.pt",
+      "validation": "basis/sources/qwen3/swiftsvd_r070/validation.json",
+      "weight_repository": "Duke-CEI-SVD/FlashSVD"
     }
   ],
   "dense_parents": [
@@ -4726,7 +5097,8 @@
     "Original manifests retain historical paths and hashes. Map paths through this inventory when preparing new task manifests.",
     "Q0 is the first 16 windows of original Q64. Q128 is an independent panel.",
     "Only Source/dense gradients are included; no recovered adapters or optimizer states.",
-    "Four Sources reuse LowRankArena weight files through pinned per-file download references. Qwen Basis Sharing uses the existing exact identity-restoring loader; its original serialized paper identity is recorded separately."
+    "Four Sources reuse LowRankArena weight files through pinned per-file download references. Qwen Basis Sharing uses the existing exact identity-restoring loader; its original serialized paper identity is recorded separately.",
+    "2026-09-17: seven Task 05 GFWSVD/independent Swift-SVD Sources added using frozen r=0.8 factor prefixes. Their original calibration is inherited; new checkpoints do not inherit paper gradient cache identities. See basis/releases/source_ratios_20260917/README.md."
   ],
   "external_reuse": {
     "sources": [

basis/releases/source_ratios_20260917/README.md

@@ -0,0 +1,63 @@
+# Seven new Task 05 Source checkpoints
+
+Added 2026-09-17. These files fill the seven missing method/backbone/ratio candidates in the previous BASIS + LowRankArena inventory. All are compressed, pre-recovery Sources, with new serialization identities.
+
+| Source | Checkpoint | Actual projection retention | SHA256 |
+|---|---|---:|---|
+| llama31/gfwsvd_r050 | [model.pt](../../sources/llama31/gfwsvd_r050/model.pt) | 0.49974647 | `1c1f65a0d57817c76d64b00466be8aa533d22673d8aaf567d7960d138499d266` |
+| llama31/gfwsvd_r070 | [model.pt](../../sources/llama31/gfwsvd_r070/model.pt) | 0.69992300 | `b43045553b40a499d73f7460fc395622c0b2c737cc81555f8febe995a115f913` |
+| qwen3/gfwsvd_r050 | [model.pt](../../sources/qwen3/gfwsvd_r050/model.pt) | 0.49996816 | `9c4537ab505b781236533d549c829e5150ac4108116646320a1522b2893fd226` |
+| qwen3/gfwsvd_r070 | [model.pt](../../sources/qwen3/gfwsvd_r070/model.pt) | 0.69982379 | `4aed75e0ba422ec94593d437883917ba8eefb542bfe899b9dc6e4184cd9f9281` |
+| qwen3/swiftsvd_r040 | [model.pt](../../sources/qwen3/swiftsvd_r040/model.pt) | 0.39974312 | `1e8b5a3db8feb63dd574f42b78913e6d5d55b0bcc8f4ef630c78db3f12305a6b` |
+| qwen3/swiftsvd_r050 | [model.pt](../../sources/qwen3/swiftsvd_r050/model.pt) | 0.49996816 | `2a5281ae0ab3a32efe8bf1c0d8906461b0bfae774fba4c38da38996b4250a60f` |
+| qwen3/swiftsvd_r070 | [model.pt](../../sources/qwen3/swiftsvd_r070/model.pt) | 0.69982379 | `4069d003526013ba3ecdd1155a86cfffa66a29610132eb822ede22cb776d036a` |
+
+## Download and load
+
+Existing BASIS checkouts may pin an older manifest. This standalone command pins the verified release commit and downloads one complete Source; change `source_id` for another row above. Install `huggingface_hub` and the BASIS package in the usual environment.
+
+```python
+import json
+from pathlib import Path
+from huggingface_hub import snapshot_download
+from basis.compression.checkpoints import load_checkpoint
+from basis.utils.io import sha256_file
+
+repo = "Duke-CEI-SVD/FlashSVD"
+source_id = "qwen3/swiftsvd_r050"
+revision = "08d28e1337c4fec6a106c959c4ca8a6e2a0d6acb"  # verified release commit
+root = Path("data/flashsvd")
+snapshot_download(repo, revision=revision, local_dir=root,
+                  allow_patterns=[f"basis/sources/{source_id}/*"])
+source = json.loads((root / "basis/sources" / source_id / "source.json").read_text())
+checkpoint = root / source["path"]
+assert sha256_file(checkpoint) == source["checkpoint_identity"]["checkpoint_sha256"]
+model = load_checkpoint(checkpoint, kind=source["kind"]).eval()
+model = model.to("cuda:0")  # Move device only; preserve stored buffer dtypes.
+print("Pinned revision:", revision)
+```
+
+All seven use `kind=null` in the Source record and a native BASIS `model.pt`; the existing BASIS loader supports their packaged classes. Use the original Llama/Qwen tokenizer and dense parent declared in the package manifest. Downloading does not require a `kind` option. Each source.json path is relative to the download root (`data/flashsvd` above).
+
+## Construction and verification
+
+- GFWSVD: ordered leading-factor truncation of the frozen r=0.8 diagonal-KFAC Source, consistent with the already published r=0.4 recipe. No new rank-specific calibration or optimization is claimed.
+- Swift-SVD: ordered factor prefixes from the independently calibrated dense output-PCA r=0.8 Source. Calibration is the original 512 × 2048 C4 tensor (seed 20260819), not the 256-window LowRankArena variant or a historical SVD-LLM gauge control.
+- Every checkpoint was reloaded; all factor/untouched-parameter/buffer hashes and a finite CPU forward pass passed. Qwen Swift r=0.4 also matches the previous FP32-PCA-cache export exactly (651 parameter tensors and one buffer).
+- Suggested BASIS capacity is recorded and lies within 11M ±5%. Full recovery preflight, Avg7, pressure, and three-seed recovery remain subsequent experiments. New Sources must not silently reuse old gradient caches.
+- The seven checkpoints total 67.324 GiB. Per-Source `provenance/manifest.json` retains construction/calibration lineage; `validation.json` records the local checks. [Aggregate audit](validation.json).
+- All prior Source entries and identities are preserved. The package now has 27 Sources: 23 hosted checkpoints and four pinned LowRankArena imports. Prior frozen revisions remain available.
+
+## Updated selective downloader
+
+Download [download_inputs.py](download_inputs.py) and [data_bundle.json](data_bundle.json) into the same folder, then run:
+
+```bash
+python download_inputs.py --source qwen3/swiftsvd_r050 --output data/flashsvd
+```
+
+This pinned registry includes all 27 Sources. [Remote publication checks](publication_validation.json) verify the new weights and unchanged prior files.
+
+## BASIS intern code
+
+Use the [intern GitHub branch](https://github.com/Zishan-Shao/Basis/tree/handoff/paper-20260915) for installation, Source loading/checking, compression, and the five assignments. [Tested code revision](https://github.com/Zishan-Shao/Basis/tree/57218ca8c09e12464adaddddc4924394eac668f5): `57218ca`. The GitHub downloader pins the verified FlashSVD checkpoint revision; download weights/data here and run the commands from that branch.

basis/releases/source_ratios_20260917/data_bundle.json

@@ -0,0 +1,15 @@
+{
+  "status": "published",
+  "repo_id": "Duke-CEI-SVD/FlashSVD",
+  "revision": "08d28e1337c4fec6a106c959c4ca8a6e2a0d6acb",
+  "manifest_path": "basis/manifest.json",
+  "manifest_sha256": "81b74f1fab6c7aaf040e174a1a8f5046f0f8dafee035ec2cf01b76502e4fe0c7",
+  "publication": "pull_request",
+  "pr_url": "https://huggingface.co/Duke-CEI-SVD/FlashSVD/discussions/2",
+  "data_bytes": 4728774592,
+  "source_bytes": 264095245023,
+  "external_weight_bytes": 53054325445,
+  "source_count": 27,
+  "full_checkpoints_in_flashsvd": 23,
+  "sources_reusing_lowrankarena": 4
+}

basis/releases/source_ratios_20260917/download_inputs.py

@@ -0,0 +1,131 @@
+#!/usr/bin/env python3
+"""Download selected frozen BASIS inputs and verify their SHA-256 checksums."""
+from __future__ import annotations
+
+import argparse
+import hashlib
+import json
+import os
+import shutil
+from concurrent.futures import ThreadPoolExecutor
+from pathlib import Path
+
+BUNDLE = Path(__file__).with_name('data_bundle.json')
+
+
+def sha256(path):
+    digest = hashlib.sha256()
+    with Path(path).open('rb') as stream:
+        for block in iter(lambda: stream.read(8 * 1024 * 1024), b''):
+            digest.update(block)
+    return digest.hexdigest()
+
+
+def select_files(manifest, groups, source_ids):
+    available_groups = {r['group'] for r in manifest['files']} - {'sources'}
+    unknown = set(groups) - available_groups
+    sources = {s['id']: s for s in manifest['sources']}
+    if unknown or set(source_ids) - sources.keys():
+        raise ValueError(f'Unknown groups or Sources: {sorted(unknown | (set(source_ids) - sources.keys()))}')
+    selected = []
+    prefixes = tuple(f'basis/sources/{s}/' for s in source_ids)
+    for record in manifest['files']:
+        path = Path(record['path'])
+        if path.is_absolute() or '..' in path.parts or path.parts[0] != 'basis':
+            raise ValueError(f'Invalid package path: {path}')
+        if record['group'] in set(groups) | {'provenance'} or (prefixes and record['path'].startswith(prefixes)):
+            selected.append(record)
+    return selected
+
+
+def verify_files(records, root):
+    for record in records:
+        path = root / record['path']
+        if path.stat().st_size != record['bytes'] or sha256(path) != record['sha256']:
+            raise ValueError(f'Checksum mismatch: {path}')
+
+
+def download_external(records, root):
+    """Reuse pinned LowRankArena files and assemble the declared local layout."""
+    from huggingface_hub import hf_hub_download
+
+    def fetch(record):
+        remote = record['download']
+        target = root / record['path']
+        if target.is_file() and target.stat().st_size == record['bytes'] and sha256(target) == record['sha256']:
+            return
+        cached = Path(hf_hub_download(remote['repo_id'], remote['path'],
+                                     revision=remote['revision'], cache_dir=root / '.cache' / 'external')).resolve()
+        if sha256(cached) != record['sha256']:
+            raise ValueError(f'External checksum mismatch: {remote["path"]}')
+        target.parent.mkdir(parents=True, exist_ok=True)
+        if target.exists() or target.is_symlink():
+            target.unlink()
+        try:
+            os.link(cached, target)
+        except OSError:
+            shutil.copyfile(cached, target)
+
+    with ThreadPoolExecutor(max_workers=4) as pool:
+        list(pool.map(fetch, records))
+
+
+def write_path_map(manifest, root, revision):
+    # This maps the full inventory; only requested groups have been downloaded.
+    paths = {r['original_path']: str(root / r['path']) for r in manifest['files']}
+    paths.update({s['original_path']: str(root / s['path']) for s in manifest['sources']})
+    target = root / 'basis' / 'path_map.json'
+    target.write_text(json.dumps({
+        'repo_id': manifest['repo_id'], 'revision': revision,
+        'note': 'Full inventory mapping; check that an input was downloaded before use.',
+        'paths': paths, 'dense_parents': manifest['dense_parents'],
+    }, indent=2) + '\n')
+    return target
+
+
+def main():
+    parser = argparse.ArgumentParser(description=__doc__)
+    parser.add_argument('--group', action='append', default=[], help='Repeat for c4, c4_probe_shard, q64, q128, q64_gradients, q128_gradients, tokenizers, or provenance')
+    parser.add_argument('--source', action='append', default=[], help='Exact Source ID, e.g. llama31/swiftsvd_r080; repeat as needed')
+    parser.add_argument('--output', type=Path, default=Path('data/flashsvd'))
+    parser.add_argument('--list', action='store_true', help='List available input groups and Sources')
+    parser.add_argument('--dry-run', action='store_true', help='Show selected sizes; download only the small manifest')
+    args = parser.parse_args()
+    bundle = json.loads(BUNDLE.read_text())
+    if bundle.get('status') != 'published' or not bundle.get('revision'):
+        parser.error('The package is prepared but has not yet been published. See paper/DOWNLOADS.md.')
+    from huggingface_hub import hf_hub_download, snapshot_download
+    root = args.output.expanduser().resolve()
+    manifest_path = hf_hub_download(bundle['repo_id'], bundle['manifest_path'],
+                                    revision=bundle['revision'], local_dir=root)
+    if sha256(manifest_path) != bundle['manifest_sha256']:
+        raise ValueError('Package manifest checksum mismatch')
+    manifest = json.loads(Path(manifest_path).read_text())
+    if args.list:
+        for group in sorted({r['group'] for r in manifest['files']} - {'sources'}):
+            records = [r for r in manifest['files'] if r['group'] == group]
+            print(f'{group}: {len(records)} files, {sum(r["bytes"] for r in records) / 2**30:.3f} GiB')
+        for source in manifest['sources']:
+            print(f'Source {source["id"]}: Tasks {", ".join(source["tasks"])}')
+        return
+    if not args.group and not args.source:
+        parser.error('Select --group / --source, or use --list.')
+    selected = select_files(manifest, args.group, args.source)
+    print(f'Selected {len(selected)} files, {sum(r["bytes"] for r in selected) / 2**30:.3f} GiB')
+    if args.dry_run:
+        return
+    local = [r for r in selected if 'download' not in r]
+    external = [r for r in selected if 'download' in r]
+    if local:
+        snapshot_download(bundle['repo_id'], revision=bundle['revision'], local_dir=root,
+                          allow_patterns=[r['path'] for r in local], max_workers=4)
+    if external:
+        print(f'Reusing {len(external)} weight files from LowRankArena.')
+        download_external(external, root)
+    verify_files(selected, root)
+    print('Verified every selected file. Path map:', write_path_map(manifest, root, bundle['revision']))
+    print('Create fresh task manifests using these paths; preserve the original frozen manifests.')
+
+
+if __name__ == '__main__':
+    main()

basis/releases/source_ratios_20260917/publication_validation.json

@@ -0,0 +1,14 @@
+{
+  "status": "passed",
+  "verified_revision": "08d28e1337c4fec6a106c959c4ca8a6e2a0d6acb",
+  "uploaded_objects_verified": 35,
+  "new_weight_files_verified": 7,
+  "new_source_count": 7,
+  "source_count": 27,
+  "weight_verification": "All seven remote LFS byte counts and SHA256 identities match freshly hashed local artifacts",
+  "metadata_verification": "Git blob IDs or LFS identities match for every uploaded metadata object",
+  "original_files_preserved": 586,
+  "original_source_records_preserved": 20,
+  "anonymous_manifest_and_seven_source_records_downloaded": true,
+  "manifest_sha256": "81b74f1fab6c7aaf040e174a1a8f5046f0f8dafee035ec2cf01b76502e4fe0c7"
+}

basis/releases/source_ratios_20260917/upload_status.json

@@ -0,0 +1,7 @@
+{
+  "status": "uploaded",
+  "new_sources": 7,
+  "source_count": 27,
+  "base_revision": "3462c6bc5405247da1a618105ca50774e75c4489",
+  "note": "Checkpoint objects and complete source registry uploaded together; remote verification is recorded by the publisher."
+}

basis/releases/source_ratios_20260917/validation.json

@@ -0,0 +1,134 @@
+{
+  "status": "local_checks_passed",
+  "base_revision": "3462c6bc5405247da1a618105ca50774e75c4489",
+  "sources": [
+    {
+      "actual_projection_retention": 0.49974646935096156,
+      "buffers_verified": 1,
+      "calibration_tensor_sha256": "95ca6fc95ac62db49af1d7c4e5b4d7d5e100c29e82376929268bfb14987ad82b",
+      "checkpoint_identity": {
+        "checkpoint_sha256": "1c1f65a0d57817c76d64b00466be8aa533d22673d8aaf567d7960d138499d266"
+      },
+      "elapsed_seconds": 92.98302385210991,
+      "factor_pairs": 224,
+      "finite_forward": true,
+      "parameters_verified": 515,
+      "predicted_adapter_parameters": 10759936,
+      "source_id": "llama31/gfwsvd_r050",
+      "source_json": "/home/zs89/Basis/outputs/source_delivery_20260917/sources/llama31/gfwsvd_r050/source.json",
+      "status": "passed",
+      "suggested_adapter_rank": 22
+    },
+    {
+      "actual_projection_retention": 0.6999230018028846,
+      "buffers_verified": 1,
+      "calibration_tensor_sha256": "95ca6fc95ac62db49af1d7c4e5b4d7d5e100c29e82376929268bfb14987ad82b",
+      "checkpoint_identity": {
+        "checkpoint_sha256": "b43045553b40a499d73f7460fc395622c0b2c737cc81555f8febe995a115f913"
+      },
+      "elapsed_seconds": 87.71806664485484,
+      "factor_pairs": 224,
+      "finite_forward": true,
+      "parameters_verified": 515,
+      "predicted_adapter_parameters": 10958848,
+      "source_id": "llama31/gfwsvd_r070",
+      "source_json": "/home/zs89/Basis/outputs/source_delivery_20260917/sources/llama31/gfwsvd_r070/source.json",
+      "status": "passed",
+      "suggested_adapter_rank": 16
+    },
+    {
+      "actual_projection_retention": 0.49996815557065216,
+      "buffers_verified": 1,
+      "calibration_tensor_sha256": "138935390d601b875fd3624b405634f177e285b0f0dc55985dabaf02de0c7d44",
+      "checkpoint_identity": {
+        "checkpoint_sha256": "9c4537ab505b781236533d549c829e5150ac4108116646320a1522b2893fd226"
+      },
+      "elapsed_seconds": 81.4608334293589,
+      "factor_pairs": 252,
+      "finite_forward": true,
+      "parameters_verified": 651,
+      "predicted_adapter_parameters": 10762560,
+      "source_id": "qwen3/gfwsvd_r050",
+      "source_json": "/home/zs89/Basis/outputs/source_delivery_20260917/sources/qwen3/gfwsvd_r050/source.json",
+      "status": "passed",
+      "suggested_adapter_rank": 20
+    },
+    {
+      "actual_projection_retention": 0.6998237941576086,
+      "buffers_verified": 1,
+      "calibration_tensor_sha256": "138935390d601b875fd3624b405634f177e285b0f0dc55985dabaf02de0c7d44",
+      "checkpoint_identity": {
+        "checkpoint_sha256": "4aed75e0ba422ec94593d437883917ba8eefb542bfe899b9dc6e4184cd9f9281"
+      },
+      "elapsed_seconds": 86.43985069356859,
+      "factor_pairs": 252,
+      "finite_forward": true,
+      "parameters_verified": 651,
+      "predicted_adapter_parameters": 11298960,
+      "source_id": "qwen3/gfwsvd_r070",
+      "source_json": "/home/zs89/Basis/outputs/source_delivery_20260917/sources/qwen3/gfwsvd_r070/source.json",
+      "status": "passed",
+      "suggested_adapter_rank": 15
+    },
+    {
+      "actual_projection_retention": 0.39974312160326086,
+      "buffers_verified": 1,
+      "calibration_tensor_sha256": "138935390d601b875fd3624b405634f177e285b0f0dc55985dabaf02de0c7d44",
+      "checkpoint_identity": {
+        "checkpoint_sha256": "1e8b5a3db8feb63dd574f42b78913e6d5d55b0bcc8f4ef630c78db3f12305a6b"
+      },
+      "elapsed_seconds": 28.72612918820232,
+      "existing_fp32_cache_export_parameters_and_buffers_exact": true,
+      "factor_pairs": 252,
+      "finite_forward": true,
+      "parameters_verified": 651,
+      "predicted_adapter_parameters": 11187072,
+      "source_id": "qwen3/swiftsvd_r040",
+      "source_json": "/home/zs89/Basis/outputs/source_delivery_20260917/sources/qwen3/swiftsvd_r040/source.json",
+      "status": "passed",
+      "suggested_adapter_rank": 26
+    },
+    {
+      "actual_projection_retention": 0.49996815557065216,
+      "buffers_verified": 1,
+      "calibration_tensor_sha256": "138935390d601b875fd3624b405634f177e285b0f0dc55985dabaf02de0c7d44",
+      "checkpoint_identity": {
+        "checkpoint_sha256": "2a5281ae0ab3a32efe8bf1c0d8906461b0bfae774fba4c38da38996b4250a60f"
+      },
+      "elapsed_seconds": 69.60182892624289,
+      "factor_pairs": 252,
+      "finite_forward": true,
+      "parameters_verified": 651,
+      "predicted_adapter_parameters": 10762560,
+      "source_id": "qwen3/swiftsvd_r050",
+      "source_json": "/home/zs89/Basis/outputs/source_delivery_20260917/sources/qwen3/swiftsvd_r050/source.json",
+      "status": "passed",
+      "suggested_adapter_rank": 20
+    },
+    {
+      "actual_projection_retention": 0.6998237941576086,
+      "buffers_verified": 1,
+      "calibration_tensor_sha256": "138935390d601b875fd3624b405634f177e285b0f0dc55985dabaf02de0c7d44",
+      "checkpoint_identity": {
+        "checkpoint_sha256": "4069d003526013ba3ecdd1155a86cfffa66a29610132eb822ede22cb776d036a"
+      },
+      "elapsed_seconds": 95.89392327796668,
+      "factor_pairs": 252,
+      "finite_forward": true,
+      "parameters_verified": 651,
+      "predicted_adapter_parameters": 11298960,
+      "source_id": "qwen3/swiftsvd_r070",
+      "source_json": "/home/zs89/Basis/outputs/source_delivery_20260917/sources/qwen3/swiftsvd_r070/source.json",
+      "status": "passed",
+      "suggested_adapter_rank": 15
+    }
+  ],
+  "checks": [
+    "checkpoint_hash",
+    "factor_tensor_hashes",
+    "untouched_parameter_hashes",
+    "buffer_hashes",
+    "finite_forward"
+  ],
+  "publication_check": "Validate uploaded LFS size/SHA256 and metadata blob hashes against this package."
+}

basis/sources/index.csv

@@ -19,3 +19,10 @@ qwen3/modegpt_r060,04,0.6,10.061,Duke-CEI-SVD/LowRankArena,basis/sources/qwen3/m
 qwen3/swiftsvd_r060,04,0.6,10.086,Duke-CEI-SVD/FlashSVD,basis/sources/qwen3/swiftsvd_r060/model.pt,,dae155bdd1eff75f597945991f8c6b502a5b2b391e2793c33a0378d73e743e72,dae155bdd1eff75f597945991f8c6b502a5b2b391e2793c33a0378d73e743e72
 llama31/gfwsvd_r040,05,0.4,7.165,Duke-CEI-SVD/FlashSVD,basis/sources/llama31/gfwsvd_r040/model.pt,,d218e26a43f40ae3009f88329730e3702c8992d681f0c217a52ca3ba2763771d,d218e26a43f40ae3009f88329730e3702c8992d681f0c217a52ca3ba2763771d
 qwen3/gfwsvd_r040,05,0.4,7.498,Duke-CEI-SVD/FlashSVD,basis/sources/qwen3/gfwsvd_r040/model.pt,,fa53afb025fe782e525ef1a6b23e97f8cb7af5d739ba2ff892a91578594e5bcd,fa53afb025fe782e525ef1a6b23e97f8cb7af5d739ba2ff892a91578594e5bcd
+llama31/gfwsvd_r050,05,0.5,8.455,Duke-CEI-SVD/FlashSVD,basis/sources/llama31/gfwsvd_r050/model.pt,,1c1f65a0d57817c76d64b00466be8aa533d22673d8aaf567d7960d138499d266,
+llama31/gfwsvd_r070,05,0.7,11.057,Duke-CEI-SVD/FlashSVD,basis/sources/llama31/gfwsvd_r070/model.pt,,b43045553b40a499d73f7460fc395622c0b2c737cc81555f8febe995a115f913,
+qwen3/gfwsvd_r050,05,0.5,8.788,Duke-CEI-SVD/FlashSVD,basis/sources/qwen3/gfwsvd_r050/model.pt,,9c4537ab505b781236533d549c829e5150ac4108116646320a1522b2893fd226,
+qwen3/gfwsvd_r070,05,0.7,11.373,Duke-CEI-SVD/FlashSVD,basis/sources/qwen3/gfwsvd_r070/model.pt,,4aed75e0ba422ec94593d437883917ba8eefb542bfe899b9dc6e4184cd9f9281,
+qwen3/swiftsvd_r040,05,0.4,7.491,Duke-CEI-SVD/FlashSVD,basis/sources/qwen3/swiftsvd_r040/model.pt,,1e8b5a3db8feb63dd574f42b78913e6d5d55b0bcc8f4ef630c78db3f12305a6b,
+qwen3/swiftsvd_r050,05,0.5,8.788,Duke-CEI-SVD/FlashSVD,basis/sources/qwen3/swiftsvd_r050/model.pt,,2a5281ae0ab3a32efe8bf1c0d8906461b0bfae774fba4c38da38996b4250a60f,
+qwen3/swiftsvd_r070,05,0.7,11.373,Duke-CEI-SVD/FlashSVD,basis/sources/qwen3/swiftsvd_r070/model.pt,,4069d003526013ba3ecdd1155a86cfffa66a29610132eb822ede22cb776d036a,

basis/sources/llama31/gfwsvd_r050/model.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:1c1f65a0d57817c76d64b00466be8aa533d22673d8aaf567d7960d138499d266
+size 9078011199

basis/sources/llama31/gfwsvd_r050/provenance/manifest.json

@@ -0,0 +1,2149 @@
+{
+  "actual_projection_retention": 0.49974646935096156,
+  "buffer_hashes": {
+    "model.rotary_emb.inv_freq": "54e45984c6ed0110ac11befb802715adece9ed7cd0e01b44520367e80c2b1e6d"
+  },
+  "build_protocol": "basis_nested_paper_factors_v1",
+  "calibration_provenance": "Inherited exactly; no new calibration or recovery",
+  "calibration_shape": [
+    512,
+    2048
+  ],
+  "calibration_tensor_sha256": "95ca6fc95ac62db49af1d7c4e5b4d7d5e100c29e82376929268bfb14987ad82b",
+  "constructor": "ordered leading-factor truncation of the frozen r=0.8 paper Source",
+  "dense_projection_parameters": 6979321856,
+  "exporter_sha256": "095e17a71ddbdfe32c4976e562041b61197064891a40cdf1da830638b824dfdd",
+  "factor_parameters": 3487891456,
+  "keep_ratio": 0.5,
+  "method": "gfwsvd",
+  "output_sha256": "1c1f65a0d57817c76d64b00466be8aa533d22673d8aaf567d7960d138499d266",
+  "parent": {
+    "checkpoint_identity": {
+      "checkpoint_sha256": "eac75ad9381d7742a96a1d088f607c14a44d0fb8973cf5281c86514f7828718d"
+    },
+    "kind": null,
+    "path": "/tmp/basis-input-audit/source_upload/basis/sources/llama31/gfwsvd_r080/model.pt",
+    "source_id": "llama31/gfwsvd_r080",
+    "source_json_sha256": "f8374c009efde1189b82245a9245699222dbaab147a7240001f2d9dc728d99e4"
+  },
+  "parent_manifest": {
+    "base_model": "meta-llama/Llama-3.1-8B",
+    "calibration_provenance": "exact_locked_tensor_verified",
+    "calibration_shape": [
+      512,
+      2048
+    ],
+    "calibration_tensor_sha256": "95ca6fc95ac62db49af1d7c4e5b4d7d5e100c29e82376929268bfb14987ad82b",
+    "constructor": "two-sided diagonal-KFAC weighted SVD",
+    "factor_parameters": 5582028800,
+    "keep_ratio": 0.8,
+    "method": "GFWSVD",
+    "metric_scope": "diagonal A and B Fisher/KFAC approximation",
+    "module_count": 224,
+    "output": "/home/zs89/FlashSVDTrain/diagnosis/spectral_fingerprint/confirmatory/artifacts/sources/gfwsvd_shared512/model.pt",
+    "output_bytes": 13275277923,
+    "output_sha256": "eac75ad9381d7742a96a1d088f607c14a44d0fb8973cf5281c86514f7828718d",
+    "rank_profile": {
+      "down_proj": 2548,
+      "gate_proj": 2548,
+      "k_proj": 655,
+      "o_proj": 1638,
+      "q_proj": 1638,
+      "up_proj": 2548,
+      "v_proj": 655
+    },
+    "rank_rule": "floor(keep_ratio * out_dim * in_dim / (out_dim + in_dim))",
+    "status": "complete"
+  },
+  "parent_manifest_sha256": "6558e358f1a8000d5a2c0f59ab8e030d91c0fcdd26a5b6ae7c069201d15c083b",
+  "rank_records": [
+    {
+      "coordinate_dim": 1024,
+      "in_features": 4096,
+      "input_sha256": "c52e4a4e99656127782a1e39e769db0ce808d3726fc093b66f2f1dc0454a28de",
+      "module": "model.layers.0.self_attn.q_proj",
+      "out_features": 4096,
+      "output_sha256": "eaa7f0da8c85805eebf44ab0bc68b0c1aa8b7fdf393443fa1a97eb20c507230d",
+      "parent_rank": 1638
+    },
+    {
+      "coordinate_dim": 409,
+      "in_features": 4096,
+      "input_sha256": "678acc6c749cf6b9fd65630ad681c389997cf5a9315e7662fd87c7637a722533",
+      "module": "model.layers.0.self_attn.k_proj",
+      "out_features": 1024,
+      "output_sha256": "f31ba540746655ed5a6334c0f0e85ca51a1c05433329933926d05bf98a13abb2",
+      "parent_rank": 655
+    },
+    {
+      "coordinate_dim": 409,
+      "in_features": 4096,
+      "input_sha256": "dacb5fe1800d3edf3dc03394a6e85deb77f7c73aab95f38d34af757f500646c7",
+      "module": "model.layers.0.self_attn.v_proj",
+      "out_features": 1024,
+      "output_sha256": "de2959b424a93110b0e914770ae538b9db9a8032bd5b0e0bf691bda4c9944298",
+      "parent_rank": 655
+    },
+    {
+      "coordinate_dim": 1024,
+      "in_features": 4096,
+      "input_sha256": "9292b792b37c914585eb3901cd8edf17dc00bc13e20d0f12c76cc87cb5642ebb",
+      "module": "model.layers.0.self_attn.o_proj",
+      "out_features": 4096,
+      "output_sha256": "59a7cf65574c540d713221fdff9eda6e9160f1bbed0da02ba7c36335a7722ac4",
+      "parent_rank": 1638
+    },
+    {
+      "coordinate_dim": 1592,
+      "in_features": 4096,
+      "input_sha256": "dad4241c2139caa0b4d852ff5f4fab8eaa96ee3aace0e18e0151baeeccbde85a",
+      "module": "model.layers.0.mlp.gate_proj",
+      "out_features": 14336,
+      "output_sha256": "e5c1373df6e144bb99e5e4b98a3e062e98deac71a7bdaff7cf35eafbd4a1166c",
+      "parent_rank": 2548
+    },
+    {
+      "coordinate_dim": 1592,
+      "in_features": 14336,
+      "input_sha256": "da33fabaed39de32ef4ba4e2ad5a94a8253ea77c22ae2eea3bf6b65cdae098eb",
+      "module": "model.layers.0.mlp.down_proj",
+      "out_features": 4096,
+      "output_sha256": "0c90e2cd4500e2eb6b943fcc3420da89d740205bf0788137e53ae979c61a1243",
+      "parent_rank": 2548
+    },
+    {
+      "coordinate_dim": 1592,
+      "in_features": 4096,
+      "input_sha256": "326852659fec4d607fa41051a2f3793c7310a9b861b14eadb2418064ef64cedc",
+      "module": "model.layers.0.mlp.up_proj",
+      "out_features": 14336,
+      "output_sha256": "0efa47f0c1cba382cf99c01e160b8841e20796b6b9dd74febde3e1669ebb622f",
+      "parent_rank": 2548
+    },
+    {
+      "coordinate_dim": 1024,
+      "in_features": 4096,
+      "input_sha256": "2e09eb8aa28aa39fabd273bf804c4c713620189acd3f0949e128a11e883f2cfb",
+      "module": "model.layers.1.self_attn.q_proj",
+      "out_features": 4096,
+      "output_sha256": "f986f61bbdcbd2c88a0fb8c64c56033d967e5123f2433576d82dcf327574f51d",
+      "parent_rank": 1638
+    },
+    {
+      "coordinate_dim": 409,
+      "in_features": 4096,
+      "input_sha256": "a7e7f125a73de2c1da6feb7f2e2810e680bca227ef5fb4b5704d4996b31e2d06",
+      "module": "model.layers.1.self_attn.k_proj",
+      "out_features": 1024,
+      "output_sha256": "1e4272f69a0e19ddde057eea441fbfe1b75d414d292bee95d6be05777772a87f",
+      "parent_rank": 655
+    },
+    {
+      "coordinate_dim": 409,
+      "in_features": 4096,
+      "input_sha256": "cf011e1710bc964197d4e7fd12ed5fa61c6c7de8d886aef7fc5fa4d676a3cd58",
+      "module": "model.layers.1.self_attn.v_proj",
+      "out_features": 1024,
+      "output_sha256": "3c4b86f6453fff5c82518d64b2425f521816b5bb6c4948222111e216e465a3be",
+      "parent_rank": 655
+    },
+    {
+      "coordinate_dim": 1024,
+      "in_features": 4096,
+      "input_sha256": "4391332e2bee25e1df792aff1b3d7925e325178811e1bb44c190d50e9a5474ac",
+      "module": "model.layers.1.self_attn.o_proj",
+      "out_features": 4096,
+      "output_sha256": "5643489866d730c81d95cbe4de24c852c20f9368326608408267c1617422b717",
+      "parent_rank": 1638
+    },
+    {
+      "coordinate_dim": 1592,
+      "in_features": 4096,
+      "input_sha256": "cb1ee4318d49e0050679ce436de05776ff0afdf85d0557e8a8ff6a313eef348d",
+      "module": "model.layers.1.mlp.gate_proj",
+      "out_features": 14336,
+      "output_sha256": "4cc9065657f2480041543f3f6d97dab28e696e324aad6279fa6016de3ab6d1a4",
+      "parent_rank": 2548
+    },
+    {
+      "coordinate_dim": 1592,
+      "in_features": 14336,
+      "input_sha256": "ed726e7c2855ada6c5b4952e6642749392b04db5bc19ef6b620937020ab87d07",
+      "module": "model.layers.1.mlp.down_proj",
+      "out_features": 4096,
+      "output_sha256": "37e26656dfb48a9b909352e3242451cefb9112a67f9315cb140ee1b4802f8ecf",
+      "parent_rank": 2548
+    },
+    {
+      "coordinate_dim": 1592,
+      "in_features": 4096,
+      "input_sha256": "ef2fbfb6fd250c694d9675c8db090f9d697930a98441d2f2688e43c4b3fb95c8",
+      "module": "model.layers.1.mlp.up_proj",
+      "out_features": 14336,
+      "output_sha256": "a68cef4d952541b5fc32f2e996e50a0ec15b96ad16b957f9f3e3bf7180264536",
+      "parent_rank": 2548
+    },
+    {
+      "coordinate_dim": 1024,
+      "in_features": 4096,
+      "input_sha256": "0ee435af397b711e6a091aad450b709e2ae4f2e9f59b4d4e531949a2907569eb",
+      "module": "model.layers.2.self_attn.q_proj",
+      "out_features": 4096,
+      "output_sha256": "7d991ea8826dbbe29bc57afa1c76475e0db74e21da69de2baf3fa1985ab0dddf",
+      "parent_rank": 1638
+    },
+    {
+      "coordinate_dim": 409,
+      "in_features": 4096,
+      "input_sha256": "2c35daf39846fff9af410bf423c01821b1e60c45163afdb354a80d8b215152e2",
+      "module": "model.layers.2.self_attn.k_proj",
+      "out_features": 1024,
+      "output_sha256": "48955cf7c78b16543993354215e812ee80fd9e5d2aaa5226bb6ef367e708693d",
+      "parent_rank": 655
+    },
+    {
+      "coordinate_dim": 409,
+      "in_features": 4096,
+      "input_sha256": "4e55fc99b783210721d6abbd10da8250b7d0b0b5a7d569cd7830ef5e01d7b96d",
+      "module": "model.layers.2.self_attn.v_proj",
+      "out_features": 1024,
+      "output_sha256": "7f4ec4b44cb01bff24233e5e99058eb873dced628c56941956516f2dbb6a4350",
+      "parent_rank": 655
+    },
+    {
+      "coordinate_dim": 1024,
+      "in_features": 4096,
+      "input_sha256": "eb58ee3cbf35b8ab7df18c3f6246d5684c04b216400f1f435117db271feac7fd",
+      "module": "model.layers.2.self_attn.o_proj",
+      "out_features": 4096,
+      "output_sha256": "728d1e5987a50a311ccb0dca448f73b9ea2fcf7efb1fbb7952a3d69183d2c811",
+      "parent_rank": 1638
+    },
+    {
+      "coordinate_dim": 1592,
+      "in_features": 4096,
+      "input_sha256": "ee7ee8946e46c11b904116f6698339d6ee15e09a51d48cacc73f0fa5eee9751c",
+      "module": "model.layers.2.mlp.gate_proj",
+      "out_features": 14336,
+      "output_sha256": "cbf6bec6808103b6509213ed5011444efaeddebc0e2b498c26f695948e7f6f88",
+      "parent_rank": 2548
+    },
+    {
+      "coordinate_dim": 1592,
+      "in_features": 14336,
+      "input_sha256": "35c098df92477b5e64db543a590d9baab14e09fc1ab90fdaa041cabdaaf2bd06",
+      "module": "model.layers.2.mlp.down_proj",
+      "out_features": 4096,
+      "output_sha256": "c0bccc44aa3a65d295f671ab5723099f00e1b5aa831ed300dbd3ac8a961de728",
+      "parent_rank": 2548
+    },
+    {
+      "coordinate_dim": 1592,
+      "in_features": 4096,
+      "input_sha256": "1805a631333374ed0715989e0d4181405c008c6567b133ca2b88f96e46b5b020",
+      "module": "model.layers.2.mlp.up_proj",
+      "out_features": 14336,
+      "output_sha256": "c36ce2fbc444aa0e813f2b81fec6856c8fa2b2e46f5ea76037a89a39ff8be329",
+      "parent_rank": 2548
+    },
+    {
+      "coordinate_dim": 1024,
+      "in_features": 4096,
+      "input_sha256": "4d7fbfe08c049eb3e00026388f4e23b8d53fac0725cfb6b62093862f68f5fc3b",
+      "module": "model.layers.3.self_attn.q_proj",
+      "out_features": 4096,
+      "output_sha256": "70e6d8dc2b0a5e5c760eccbbbff5ea8a66f084805e9f3fbf616420d2854ab2c0",
+      "parent_rank": 1638
+    },
+    {
+      "coordinate_dim": 409,
+      "in_features": 4096,
+      "input_sha256": "4c945d3a2d02ee1635ca056ad2d22fa6513ff1b3c1ff9d7d865bec72db173a1e",
+      "module": "model.layers.3.self_attn.k_proj",
+      "out_features": 1024,
+      "output_sha256": "90598b72008600d78e3460bec6716401295e700e4d0943527896d8de58ae5a29",
+      "parent_rank": 655
+    },
+    {
+      "coordinate_dim": 409,
+      "in_features": 4096,
+      "input_sha256": "5579840e504fe4f3b543fc7e1ae9f9779bf09d17504bb08239b902b323b154a5",
+      "module": "model.layers.3.self_attn.v_proj",
+      "out_features": 1024,
+      "output_sha256": "a72d3b7d90852cac870802715b7ad18b22c52867bae16c27a07ad0af6e3f4fbe",
+      "parent_rank": 655
+    },
+    {
+      "coordinate_dim": 1024,
+      "in_features": 4096,
+      "input_sha256": "b76fb4cc4ecb35ee7239ac22ec713dc5ee65abdb2171978422303ad22cca5ab7",
+      "module": "model.layers.3.self_attn.o_proj",
+      "out_features": 4096,
+      "output_sha256": "2b0c856667cce233b1250cbf0f765f17156b2ecdc460042aa1d437b62b1e981d",
+      "parent_rank": 1638
+    },
+    {
+      "coordinate_dim": 1592,
+      "in_features": 4096,
+      "input_sha256": "c76174195067b4cbf320e516dd87addfa47f1487e7a70278d0f0bbe10b27b690",
+      "module": "model.layers.3.mlp.gate_proj",
+      "out_features": 14336,
+      "output_sha256": "5b99b22343270d2ed6db2a5fe3a12809552be7aa8f47579fc24498ec390e3ba0",
+      "parent_rank": 2548
+    },
+    {
+      "coordinate_dim": 1592,
+      "in_features": 14336,
+      "input_sha256": "469279f4044e35d7ba202e464a5dafdde67417d77071dcd0d79157db115816d4",
+      "module": "model.layers.3.mlp.down_proj",
+      "out_features": 4096,
+      "output_sha256": "8bf2148adcb68809228428c16542088b76089d9464dc63bec6fc6f2982411571",
+      "parent_rank": 2548
+    },
+    {
+      "coordinate_dim": 1592,
+      "in_features": 4096,
+      "input_sha256": "0a528fb715481cd1a29eb5d3ebdb3e5276ce97763452ecf8773d77f9f98baa80",
+      "module": "model.layers.3.mlp.up_proj",
+      "out_features": 14336,
+      "output_sha256": "aab1b988a246f53784f11b74cb8a9b03636b6f900ec95f9756be9d4d26f6c37a",
+      "parent_rank": 2548
+    },
+    {
+      "coordinate_dim": 1024,
+      "in_features": 4096,
+      "input_sha256": "0f8913bcfc9576733e49879e8b527aa155f3275b67ab89e0adefb23f56b807e6",
+      "module": "model.layers.4.self_attn.q_proj",
+      "out_features": 4096,
+      "output_sha256": "f7045ef17a1473bbd737256ae60b3c863dc897a479d4ca6e2649dc0db368867b",
+      "parent_rank": 1638
+    },
+    {
+      "coordinate_dim": 409,
+      "in_features": 4096,
+      "input_sha256": "9ff92cd09719aa71a6e38d8160c499f7d58dea8188d4b01525200cc1f0c41129",
+      "module": "model.layers.4.self_attn.k_proj",
+      "out_features": 1024,
+      "output_sha256": "07c9ec128db36d0e5a97eca40544350c2ad167fed345ae3a4a0e37f9122a9195",
+      "parent_rank": 655
+    },
+    {
+      "coordinate_dim": 409,
+      "in_features": 4096,
+      "input_sha256": "db78925aadd7e745aa39d449c29211589a4dd22ee878e739a75376bea2d88abb",
+      "module": "model.layers.4.self_attn.v_proj",
+      "out_features": 1024,
+      "output_sha256": "3411062538fa19b6124dbe4313253e7b6c70ee1d2ad2b2bf51c267e5f0323561",
+      "parent_rank": 655
+    },
+    {
+      "coordinate_dim": 1024,
+      "in_features": 4096,
+      "input_sha256": "de539de21d4f0957b626451a632ecf8c9a9063bbbc3b599cf8bde8c491cefcc7",
+      "module": "model.layers.4.self_attn.o_proj",
+      "out_features": 4096,
+      "output_sha256": "403d7aa8272a79df43d7e1b3d121274aecbcd22b0be149f671de9e8db2033423",
+      "parent_rank": 1638
+    },
+    {
+      "coordinate_dim": 1592,
+      "in_features": 4096,
+      "input_sha256": "eb488ecb917dccc1f4dd0135d4cfd850d3f3bfec8d835038933a82d6e5108634",
+      "module": "model.layers.4.mlp.gate_proj",
+      "out_features": 14336,
+      "output_sha256": "71691103b24d6031742881b0b582da8927f22876fbb33de6341e621fc4ac117c",
+      "parent_rank": 2548
+    },
+    {
+      "coordinate_dim": 1592,
+      "in_features": 14336,
+      "input_sha256": "f48b4d0a2001d6a75ce4a310e1c2a2fc4bc68dcc3676e503fbac935f6e51e68d",
+      "module": "model.layers.4.mlp.down_proj",
+      "out_features": 4096,
+      "output_sha256": "c07b057155dfe0bddfbc3d0efbc363d33f7ba5b0ec955690531c83bca9a5b974",
+      "parent_rank": 2548
+    },
+    {
+      "coordinate_dim": 1592,
+      "in_features": 4096,
+      "input_sha256": "91aac4ea610373feafa3f75da695874cf2f3083ed50a3ea335d1a2a964751156",
+      "module": "model.layers.4.mlp.up_proj",
+      "out_features": 14336,
+      "output_sha256": "da526d80764fb0d4c7c0d915d93d8cac9c3a6859f69626ab9b6a8a3971eed4e0",
+      "parent_rank": 2548
+    },
+    {
+      "coordinate_dim": 1024,
+      "in_features": 4096,
+      "input_sha256": "98fcf8dd8193819f77be57d6d97c5f0f5726a8ac2bc5886ba41d5517db476db5",
+      "module": "model.layers.5.self_attn.q_proj",
+      "out_features": 4096,
+      "output_sha256": "3b9c1432dc9cbddd2d07eb9cda4120870482a2c1cdd420aae1bf65cda07602fc",
+      "parent_rank": 1638
+    },
+    {
+      "coordinate_dim": 409,
+      "in_features": 4096,
+      "input_sha256": "6ec533443326a4178bcc258164cbf64930d6b3f37fa106a6b9bf5b06c8bc0c11",
+      "module": "model.layers.5.self_attn.k_proj",
+      "out_features": 1024,
+      "output_sha256": "cbf2b3fb15a2be673d447158eb8594b1f089147a7a692fabd72d61e529ca398a",
+      "parent_rank": 655
+    },
+    {
+      "coordinate_dim": 409,
+      "in_features": 4096,
+      "input_sha256": "3ce580f9cc9fb24b4c8483a356df4d8405dcaa3b8642def2085e1d68af3024e4",
+      "module": "model.layers.5.self_attn.v_proj",
+      "out_features": 1024,
+      "output_sha256": "a10a083c5cbf0aa9ec1a2c399762335a750cbe00a36abde0f3c6625f4b6cfa0d",
+      "parent_rank": 655
+    },
+    {
+      "coordinate_dim": 1024,
+      "in_features": 4096,
+      "input_sha256": "b694674039b8259b70d58ce95db79fa3f6dc1091f6366a592d8a09e39c0371e3",
+      "module": "model.layers.5.self_attn.o_proj",
+      "out_features": 4096,
+      "output_sha256": "f61f830eb4597e8e4bf1d68eed2d3f46b4a71e8e9116c849425d61dcb1f8aeae",
+      "parent_rank": 1638
+    },
+    {
+      "coordinate_dim": 1592,
+      "in_features": 4096,
+      "input_sha256": "a78fa9cd9276aaa8e6b6b1191978b0d0fa7b204a093ae72363add62f5e42aa95",
+      "module": "model.layers.5.mlp.gate_proj",
+      "out_features": 14336,
+      "output_sha256": "9ba9a1b79bf2a43aaf22a371be1e9402f7d54ea9648a0b094c9de12490ee81f2",
+      "parent_rank": 2548
+    },
+    {
+      "coordinate_dim": 1592,
+      "in_features": 14336,
+      "input_sha256": "53592cc6a391e95fe532bb693b462559ba286c140d92e67055350e48c873454c",
+      "module": "model.layers.5.mlp.down_proj",
+      "out_features": 4096,
+      "output_sha256": "5be903cc35692716a3066eb0dfc783a0aea873725d1b1f73a3cfe201432b943d",
+      "parent_rank": 2548
+    },
+    {
+      "coordinate_dim": 1592,
+      "in_features": 4096,
+      "input_sha256": "42f6ef7cb21b32840dab682721f6aef028b8d01b19d465f216886d87e92de6ba",
+      "module": "model.layers.5.mlp.up_proj",
+      "out_features": 14336,
+      "output_sha256": "609eb4870e873bd92052788ae208866c622ae87af7937a9d7d424490977abcba",
+      "parent_rank": 2548
+    },
+    {
+      "coordinate_dim": 1024,
+      "in_features": 4096,
+      "input_sha256": "d84e781af70c2781dab780fb9a30ac5b621b9c858f5f42c4d2a4099aacd8a8ec",
+      "module": "model.layers.6.self_attn.q_proj",
+      "out_features": 4096,
+      "output_sha256": "3fbfbafdf6ae229583107fc34d412943199e668d2aa611988570717d0b0f7daa",
+      "parent_rank": 1638
+    },
+    {
+      "coordinate_dim": 409,
+      "in_features": 4096,
+      "input_sha256": "2dcb4028b726431fc4715529bd3c33cff1c7d09a2b7d2f3e1cfe60b8f382a3b2",
+      "module": "model.layers.6.self_attn.k_proj",
+      "out_features": 1024,
+      "output_sha256": "a3fe28dfb0f4a4dd49979a5a9d6bde896aea13e6c77c7be4ea680e558cbb483a",
+      "parent_rank": 655
+    },
+    {
+      "coordinate_dim": 409,
+      "in_features": 4096,
+      "input_sha256": "332e4476d9702b0a49abd79bce06b01747b642f32f6c15f3f7760736c91701ea",
+      "module": "model.layers.6.self_attn.v_proj",
+      "out_features": 1024,
+      "output_sha256": "2884c6a7cafb0634ecb204abc1f84b18ac61d496e4fed43356c457497f61fa3b",
+      "parent_rank": 655
+    },
+    {
+      "coordinate_dim": 1024,
+      "in_features": 4096,
+      "input_sha256": "95036f1a37188ed3978af40bc69ed86a883a755a0768dcea4043d002dac3b228",
+      "module": "model.layers.6.self_attn.o_proj",
+      "out_features": 4096,
+      "output_sha256": "f3fd692f356084997d06bd0e131488f701dd3d5df675e16ee358ff4c4fb1af0d",
+      "parent_rank": 1638
+    },
+    {
+      "coordinate_dim": 1592,
+      "in_features": 4096,
+      "input_sha256": "fcb9b597a0eb1df7035444f37cbe253fddca90e4a3969a0a3861caa41448ec19",
+      "module": "model.layers.6.mlp.gate_proj",
+      "out_features": 14336,
+      "output_sha256": "17a940bb88076a1d41d5e6418b13e73970c304e4acdc053312f9b2b13cd69220",
+      "parent_rank": 2548
+    },
+    {
+      "coordinate_dim": 1592,
+      "in_features": 14336,
+      "input_sha256": "8414ca1e06a95759c93707c8e1318ee58d9a395e4323157ecf90d1e280e7ae9a",
+      "module": "model.layers.6.mlp.down_proj",
+      "out_features": 4096,
+      "output_sha256": "1a72403264cbb492c9905e72f699c51c986a0435d7dfb7152d9b84366e8f43d0",
+      "parent_rank": 2548
+    },
+    {
+      "coordinate_dim": 1592,
+      "in_features": 4096,
+      "input_sha256": "51921ed765cf6ec0cc81c7e5afbd59e175327ad0153ba06f3e070298efb05551",
+      "module": "model.layers.6.mlp.up_proj",
+      "out_features": 14336,
+      "output_sha256": "1264a3bf6cff7ae63921c4d6fb2825c53cea324abfc5de65d3fd69e60b85424d",
+      "parent_rank": 2548
+    },
+    {
+      "coordinate_dim": 1024,
+      "in_features": 4096,
+      "input_sha256": "41384288eedd000a27fc8f77319eeca7a7e6cfada748045bc2c8b50e4124ad44",
+      "module": "model.layers.7.self_attn.q_proj",
+      "out_features": 4096,
+      "output_sha256": "017f45712fa6d6060bd60b652b07436625aa80594358d5eb9e8e8ab130033f31",
+      "parent_rank": 1638
+    },
+    {
+      "coordinate_dim": 409,
+      "in_features": 4096,
+      "input_sha256": "024550285ddffda17ca78d281bfc0a1dd9f66df6ecd37038daaf28e2813016de",
+      "module": "model.layers.7.self_attn.k_proj",
+      "out_features": 1024,
+      "output_sha256": "d8ac0f96a155d784273ae622d86d33ea04cc7c3e4096c73641d4ad1f9b455b6f",
+      "parent_rank": 655
+    },
+    {
+      "coordinate_dim": 409,
+      "in_features": 4096,
+      "input_sha256": "64385aad081179ac7906d0e61e9f831e0891553c1ee36c9046d119bf97237527",
+      "module": "model.layers.7.self_attn.v_proj",
+      "out_features": 1024,
+      "output_sha256": "90b203a310318267c11d2149fbc0d036823a6aa9f771c75052fcf086cb552221",
+      "parent_rank": 655
+    },
+    {
+      "coordinate_dim": 1024,
+      "in_features": 4096,
+      "input_sha256": "c85f2dae45cbcba67ee21c96f27a64dab269192b2c6e262710f22a570fc7b81a",
+      "module": "model.layers.7.self_attn.o_proj",
+      "out_features": 4096,
+      "output_sha256": "46480e007537dfdc3796bdb56b80c9fe4aba720d2085f61c1b536c209a429fdf",
+      "parent_rank": 1638
+    },
+    {
+      "coordinate_dim": 1592,
+      "in_features": 4096,
+      "input_sha256": "d67d8b2e9a100704cbddf467a38792785ad00638dfcb0c00b5165e37e33cd9a0",
+      "module": "model.layers.7.mlp.gate_proj",
+      "out_features": 14336,
+      "output_sha256": "8f792e77b11f0a7dd83977fbf189f0456ce02564506f15671d5a37bd2c6a6de5",
+      "parent_rank": 2548
+    },
+    {
+      "coordinate_dim": 1592,
+      "in_features": 14336,
+      "input_sha256": "d1bfc70bfbd3d03c4b2c46407d48315d6babad75bbcde21a88099c6ee9f2c8fa",
+      "module": "model.layers.7.mlp.down_proj",
+      "out_features": 4096,
+      "output_sha256": "0256becb0f5be017ee5ad91813e3da55bd19a8991d15b21a1feb3279d5dc0be3",
+      "parent_rank": 2548
+    },
+    {
+      "coordinate_dim": 1592,
+      "in_features": 4096,
+      "input_sha256": "d3e21ef4f76c3f85b404544692a42631448b8fafcbd06686cf1ebd5e3a94f67b",
+      "module": "model.layers.7.mlp.up_proj",
+      "out_features": 14336,
+      "output_sha256": "a63ea6e653e114e29ab30f19061350e3fc0fe6caa2d093697b3f364cf0e9d350",
+      "parent_rank": 2548
+    },
+    {
+      "coordinate_dim": 1024,
+      "in_features": 4096,
+      "input_sha256": "0d82c5a3cffb19601464f01c1a6fdaea8d82c2265f2faa84454d1436cc9ee394",
+      "module": "model.layers.8.self_attn.q_proj",
+      "out_features": 4096,
+      "output_sha256": "b2375f837a3cea84143940e3bffd7d13bb7ef6fcfc16a4fa64345688264a2ef9",
+      "parent_rank": 1638
+    },
+    {
+      "coordinate_dim": 409,
+      "in_features": 4096,
+      "input_sha256": "1cfa4688048ee5881fb56ab0b1f034ffa76ff8b9f076388ee3cd8f91364107fc",
+      "module": "model.layers.8.self_attn.k_proj",
+      "out_features": 1024,
+      "output_sha256": "a8ee924c79d45218d67890f19844618ff36d5a83a86e7bae327a4e2e7ed1b6b6",
+      "parent_rank": 655
+    },
+    {
+      "coordinate_dim": 409,
+      "in_features": 4096,
+      "input_sha256": "da5dc164e6a2a20271581673d20122dc904172af4a55020592d815bbea03f238",
+      "module": "model.layers.8.self_attn.v_proj",
+      "out_features": 1024,
+      "output_sha256": "139f56eb96912e2012c4282c1473515ff619bf1ea5b31c218b41cb8beffd97e1",
+      "parent_rank": 655
+    },
+    {
+      "coordinate_dim": 1024,
+      "in_features": 4096,
+      "input_sha256": "31ba9c52253b85c2218327bf458618f556c51e59037a47fbe776dac724174f5f",
+      "module": "model.layers.8.self_attn.o_proj",
+      "out_features": 4096,
+      "output_sha256": "84a6ea61932bad758d9e03b0b2aa9ca9daf566469c784802068875c1e76c63c8",
+      "parent_rank": 1638
+    },
+    {
+      "coordinate_dim": 1592,
+      "in_features": 4096,
+      "input_sha256": "89bf4d2fb73cbadbbf9efab2423f809372be0ea54b6bf2ef6b52a9f35bcb37c4",
+      "module": "model.layers.8.mlp.gate_proj",
+      "out_features": 14336,
+      "output_sha256": "c8fcb1631499c044264c1690885e3007f031c6bd55b7823bbde592c3c44fe6c7",
+      "parent_rank": 2548
+    },
+    {
+      "coordinate_dim": 1592,
+      "in_features": 14336,
+      "input_sha256": "04212adc688884a741a1300979d32f7bd9683e0fd66b501eb5b779963debb1de",
+      "module": "model.layers.8.mlp.down_proj",
+      "out_features": 4096,
+      "output_sha256": "ceb6f18329afd6c3741b3081539b49dd2236eb8697ef308b058e34ce722a6acc",
+      "parent_rank": 2548
+    },
+    {
+      "coordinate_dim": 1592,
+      "in_features": 4096,
+      "input_sha256": "6fa727306fae72e58c0bdb878436fdead90cbfd2097b09d19053e16ebf8dcb42",
+      "module": "model.layers.8.mlp.up_proj",
+      "out_features": 14336,
+      "output_sha256": "c4116899607974fc55f9ed215253208dc617bda3f723f577fe2d59c000d8332e",
+      "parent_rank": 2548
+    },
+    {
+      "coordinate_dim": 1024,
+      "in_features": 4096,
+      "input_sha256": "7e3746cc02686cc65b3b15a2dfad8fd87b8d816d96d1ad90f73aa65c9a683f05",
+      "module": "model.layers.9.self_attn.q_proj",
+      "out_features": 4096,
+      "output_sha256": "4116e9f3dc5dc803f282c167712e4ab9ad0e0b31500db96f452a8599f2b8588a",
+      "parent_rank": 1638
+    },
+    {
+      "coordinate_dim": 409,
+      "in_features": 4096,
+      "input_sha256": "90f58bfe1cd8548f8196f4b4cb183542c2694c12d5e74fbb7fe2a0851bdc5f30",
+      "module": "model.layers.9.self_attn.k_proj",
+      "out_features": 1024,
+      "output_sha256": "23aa97462896f48fcf60ec8ebd67a80a0f53e9e15f7de33909f0f8249c02ec80",
+      "parent_rank": 655
+    },
+    {
+      "coordinate_dim": 409,
+      "in_features": 4096,
+      "input_sha256": "4ce7ae25da0fc51499ebe8e9e2ff842fdf30078bd9ceef8d9319c62b2332ad48",
+      "module": "model.layers.9.self_attn.v_proj",
+      "out_features": 1024,
+      "output_sha256": "dcc1f0506d6718fec96fb82cd1e71e269ff1ee967c5a7d47e7dbc2a0e97e4b8c",
+      "parent_rank": 655
+    },
+    {
+      "coordinate_dim": 1024,
+      "in_features": 4096,
+      "input_sha256": "7a2da101c0eaeaa0f32fee6306c28d8f3d7c4b0817b69c468332efc1b278af49",
+      "module": "model.layers.9.self_attn.o_proj",
+      "out_features": 4096,
+      "output_sha256": "32242df60253e4f606e92833a6720107516c8897ac20154944e44a18ae2dea4f",
+      "parent_rank": 1638
+    },
+    {
+      "coordinate_dim": 1592,
+      "in_features": 4096,
+      "input_sha256": "26e8c31106962d9d6c7c633d898489bd84fd48861304fcc4d4d08edaebe77ba1",
+      "module": "model.layers.9.mlp.gate_proj",
+      "out_features": 14336,
+      "output_sha256": "05f96f7667dcfc3bb8e9d0803e25b165779679c29feea7a95bce75a3e5f8fb7f",
+      "parent_rank": 2548
+    },
+    {
+      "coordinate_dim": 1592,
+      "in_features": 14336,
+      "input_sha256": "60541dc7853acf9ba367c31587f14c4bd783bb21ad3d0873702fa9eace534f13",
+      "module": "model.layers.9.mlp.down_proj",
+      "out_features": 4096,
+      "output_sha256": "319b5a1b7811fa1b1ebf4b1aedb9a9a70b0788ffa4ff0d23c93533e7575c4407",
+      "parent_rank": 2548
+    },
+    {
+      "coordinate_dim": 1592,
+      "in_features": 4096,
+      "input_sha256": "e1801977acc941a3073ae1807a1da53d96685c71f0a769dd525bca0e0d4e767b",
+      "module": "model.layers.9.mlp.up_proj",
+      "out_features": 14336,
+      "output_sha256": "072a665b6a90de09396926b17d2b71fff5b2882228a4ccf57405b07b76f6cc78",
+      "parent_rank": 2548
+    },
+    {
+      "coordinate_dim": 1024,
+      "in_features": 4096,
+      "input_sha256": "09a22a34754222f17297a9846a03302c60b012116bd999e2d79689d892db03a8",
+      "module": "model.layers.10.self_attn.q_proj",
+      "out_features": 4096,
+      "output_sha256": "5491a9b0a93d2c5ee0d94e4c6ea6f68d25a660152d050bcb66cfba49725e6106",
+      "parent_rank": 1638
+    },
+    {
+      "coordinate_dim": 409,
+      "in_features": 4096,
+      "input_sha256": "106381848bc520bcad388469277ab9f2e57ba89b0e4203ecc061b239d4327182",
+      "module": "model.layers.10.self_attn.k_proj",
+      "out_features": 1024,
+      "output_sha256": "68bcf28afa27d4e97d13f991a46b9f7c258eb5c39e6cc348a2d7c25b59e6b021",
+      "parent_rank": 655
+    },
+    {
+      "coordinate_dim": 409,
+      "in_features": 4096,
+      "input_sha256": "aef43c6a824b4de9c8bd3dd5eadd049841d736c4826bbec63b1d7df671f94a4a",
+      "module": "model.layers.10.self_attn.v_proj",
+      "out_features": 1024,
+      "output_sha256": "1836e613372c8a7e90ba5ab6230cbb6a572dfb228cdfc033fe4e99da197eeada",
+      "parent_rank": 655
+    },
+    {
+      "coordinate_dim": 1024,
+      "in_features": 4096,
+      "input_sha256": "5c09b109b58a83235eb0f447454cde2e44cfd373c3b0f052f230584334a5d915",
+      "module": "model.layers.10.self_attn.o_proj",
+      "out_features": 4096,
+      "output_sha256": "2978e1f979c346d1caff9bf2344870eff99619ec5f8eec04658ac1e28c29d44b",
+      "parent_rank": 1638
+    },
+    {
+      "coordinate_dim": 1592,
+      "in_features": 4096,
+      "input_sha256": "5e50e98ac686289834eed9a44f9ed1402ef733d10f84e5d2c17e1b59bedd56ed",
+      "module": "model.layers.10.mlp.gate_proj",
+      "out_features": 14336,
+      "output_sha256": "efa7278f6b8ea6d83ca3dc17e70456290b2f08a217a672912e4081d10d1a365b",
+      "parent_rank": 2548
+    },
+    {
+      "coordinate_dim": 1592,
+      "in_features": 14336,
+      "input_sha256": "a98390cee572f85ce28a5d180cd6b5c2f86fe4d451d03663e8de06ed878ec917",
+      "module": "model.layers.10.mlp.down_proj",
+      "out_features": 4096,
+      "output_sha256": "ecc4114d00531e116374d9da26b58e5725a2cb56cd9e5a1e1eb842e3f14fcd19",
+      "parent_rank": 2548
+    },
+    {
+      "coordinate_dim": 1592,
+      "in_features": 4096,
+      "input_sha256": "a557b046c528bb6891fa635294d68d90f51bd907225f9f9ee6bc261305ba61b9",
+      "module": "model.layers.10.mlp.up_proj",
+      "out_features": 14336,
+      "output_sha256": "c7f57bcea71da9c0f1f27db59db51d424b643d068fb324cbdb98743b2d1c55d6",
+      "parent_rank": 2548
+    },
+    {
+      "coordinate_dim": 1024,
+      "in_features": 4096,
+      "input_sha256": "2d75074573cc852e8f6caf9b9e765ac60dd09e4e0bb3edee11215b8eca820212",
+      "module": "model.layers.11.self_attn.q_proj",
+      "out_features": 4096,
+      "output_sha256": "74a459f8348aa644faa9b175c71b7776a9b29f947b68392bf048aa4278ed4b1d",
+      "parent_rank": 1638
+    },
+    {
+      "coordinate_dim": 409,
+      "in_features": 4096,
+      "input_sha256": "7346561736429a677134620181621280c50bb32947b70a9844a8f7afcc4ca8bb",
+      "module": "model.layers.11.self_attn.k_proj",
+      "out_features": 1024,
+      "output_sha256": "b59d074f948a92f3c6fc1a152982948f1c3af1b16844807369e1ffeefc63775e",
+      "parent_rank": 655
+    },
+    {
+      "coordinate_dim": 409,
+      "in_features": 4096,
+      "input_sha256": "60662c568e9a2d6bc4e2b1cd5bdf40e659af1011a9993bfcdef52b13903f7b06",
+      "module": "model.layers.11.self_attn.v_proj",
+      "out_features": 1024,
+      "output_sha256": "e3180981f37972ba29b24d42af3ad72668d23b756a432af9e6159b9d1d023e6b",
+      "parent_rank": 655
+    },
+    {
+      "coordinate_dim": 1024,
+      "in_features": 4096,
+      "input_sha256": "9bb40ce29029f42e1d58c0248498477626a778e10be839047f9eac0430f8da25",
+      "module": "model.layers.11.self_attn.o_proj",
+      "out_features": 4096,
+      "output_sha256": "535dd16b4da2cf290d7713e92e736625ed61b3428cbdb734d36380fb16495c32",
+      "parent_rank": 1638
+    },
+    {
+      "coordinate_dim": 1592,
+      "in_features": 4096,
+      "input_sha256": "fed6e0ed3e7715213e5292626ce15aced1066094887402248f6678e84ddd93e4",
+      "module": "model.layers.11.mlp.gate_proj",
+      "out_features": 14336,
+      "output_sha256": "6d28ad8916c6cca206b236e4108889afcb7bf37387d03d5108f4e456e101df3b",
+      "parent_rank": 2548
+    },
+    {
+      "coordinate_dim": 1592,
+      "in_features": 14336,
+      "input_sha256": "b73a29c284402b1d602ffd2675c1c98917b6867ab1615880d4d8c45fe67b793d",
+      "module": "model.layers.11.mlp.down_proj",
+      "out_features": 4096,
+      "output_sha256": "3e29a3ba65e8397a48be879a4a9ca100b33078fc04f9541bdab389b908bb5ffa",
+      "parent_rank": 2548
+    },
+    {
+      "coordinate_dim": 1592,
+      "in_features": 4096,
+      "input_sha256": "da09243d07b9991b4ffd2c4137793185ccc3ee372136607e10bc99ba86c1f8f9",
+      "module": "model.layers.11.mlp.up_proj",
+      "out_features": 14336,
+      "output_sha256": "0503ccdffb1480e937db648fbadb31997aa74d0e9b038ef42d1cb1aaaa9c10ca",
+      "parent_rank": 2548
+    },
+    {
+      "coordinate_dim": 1024,
+      "in_features": 4096,
+      "input_sha256": "fb9bf89e1431180782d3253300ec3f38ed87d88653f4f0ea3a8d96ab5b07bec3",
+      "module": "model.layers.12.self_attn.q_proj",
+      "out_features": 4096,
+      "output_sha256": "af0a28419dd7208fe5d595cad82a0d9105786b7976d2bb48255b86a09b0c9b4c",
+      "parent_rank": 1638
+    },
+    {
+      "coordinate_dim": 409,
+      "in_features": 4096,
+      "input_sha256": "58c6dd68bca02643e4a0c449644e61188614e0d64202f55858c054436eb1f0be",
+      "module": "model.layers.12.self_attn.k_proj",
+      "out_features": 1024,
+      "output_sha256": "c1ab90c39045424d20783de66f91690b0f958c3cb09b0648653e595409cac6f3",
+      "parent_rank": 655
+    },
+    {
+      "coordinate_dim": 409,
+      "in_features": 4096,
+      "input_sha256": "1c468340605cfa1c964cb3a361dd0033bd3f370db7365132953ded6b1d2aec8b",
+      "module": "model.layers.12.self_attn.v_proj",
+      "out_features": 1024,
+      "output_sha256": "964584a100fb9153772ae2257a8bf9b60c2fcbed0606a40cecdbec44a6eacf56",
+      "parent_rank": 655
+    },
+    {
+      "coordinate_dim": 1024,
+      "in_features": 4096,
+      "input_sha256": "e301b7139dcfc7b5e2ffb3eaf870bbeec7841a4c7f3e8a753ba7bed921b925e7",
+      "module": "model.layers.12.self_attn.o_proj",
+      "out_features": 4096,
+      "output_sha256": "e195399617ba9e02dffc28c7102bac40f884fad47972660e77c68b9d2f051dfe",
+      "parent_rank": 1638
+    },
+    {
+      "coordinate_dim": 1592,
+      "in_features": 4096,
+      "input_sha256": "d6f40f0728a4f62d4a447b7eaa47c864c4e8c7348218540287515ab73167d028",
+      "module": "model.layers.12.mlp.gate_proj",
+      "out_features": 14336,
+      "output_sha256": "3268ee9575e126e5123c7cbfbb1970e2f2f8a62f0615fc2f0480119704abc114",
+      "parent_rank": 2548
+    },
+    {
+      "coordinate_dim": 1592,
+      "in_features": 14336,
+      "input_sha256": "95e14ac7e9e53f1efbd594e46ce3368f3f5d7d703fb8928c8a4f94a3d26ea094",
+      "module": "model.layers.12.mlp.down_proj",
+      "out_features": 4096,
+      "output_sha256": "9c8cdb8aba0d663346f75d99c88502bc2b6fab5d4dee254b6abc75e3d686ac7b",
+      "parent_rank": 2548
+    },
+    {
+      "coordinate_dim": 1592,
+      "in_features": 4096,
+      "input_sha256": "e3e7c5d45e471baa3894933ea478d3d8f56e728dab7a99279efa03f3808e6138",
+      "module": "model.layers.12.mlp.up_proj",
+      "out_features": 14336,
+      "output_sha256": "c6c1c97858af338585b3c9c703a93ecfedfa59b068087c9e5c831824573f58fd",
+      "parent_rank": 2548
+    },
+    {
+      "coordinate_dim": 1024,
+      "in_features": 4096,
+      "input_sha256": "bcf1af937802a4d8653bd290a61b5b02e06207e0a1b90543157274cc95253540",
+      "module": "model.layers.13.self_attn.q_proj",
+      "out_features": 4096,
+      "output_sha256": "a3ec474af5ad3ecedc8cdb0f368bac0da12bf27dc93ecff7377cbc86185bbf2a",
+      "parent_rank": 1638
+    },
+    {
+      "coordinate_dim": 409,
+      "in_features": 4096,
+      "input_sha256": "9e59eb2b212a3230a7f0c120d93990490e881cb6060ae55767a1417b3842ac02",
+      "module": "model.layers.13.self_attn.k_proj",
+      "out_features": 1024,
+      "output_sha256": "74b6b356cb5b8ae5baba292e77b3656b9bd548ec78b12d89231b090c2e1f39e1",
+      "parent_rank": 655
+    },
+    {
+      "coordinate_dim": 409,
+      "in_features": 4096,
+      "input_sha256": "c77f282904286fd1b41cf7e2c1c23769e38d1a795ac4edd1463d1b8cf7cb46ce",
+      "module": "model.layers.13.self_attn.v_proj",
+      "out_features": 1024,
+      "output_sha256": "40c0813a5a8ffe8e10897fb53be05122a46ccb8103c4054cdfb7bab05bf38744",
+      "parent_rank": 655
+    },
+    {
+      "coordinate_dim": 1024,
+      "in_features": 4096,
+      "input_sha256": "df72937ad62e61d017a18ff4d571f20ceea97698472da929d951706172601a94",
+      "module": "model.layers.13.self_attn.o_proj",
+      "out_features": 4096,
+      "output_sha256": "8a6c6ed73a777faeaeae0f775eb024cea24fc6265faae2fa240b444f96259122",
+      "parent_rank": 1638
+    },
+    {
+      "coordinate_dim": 1592,
+      "in_features": 4096,
+      "input_sha256": "46922580fb6d1aca6fd43a24f9a64f423117987bd2b19d1369853508665eb9aa",
+      "module": "model.layers.13.mlp.gate_proj",
+      "out_features": 14336,
+      "output_sha256": "a37bb72bddb76e9a49a95ec23bf58b478acf9184c642d719579b65e964101aea",
+      "parent_rank": 2548
+    },
+    {
+      "coordinate_dim": 1592,
+      "in_features": 14336,
+      "input_sha256": "365b9042b31c40c416cc7ba7b6c217b178619cda9efc196afd9ffa46cc1fab14",
+      "module": "model.layers.13.mlp.down_proj",
+      "out_features": 4096,
+      "output_sha256": "73ff958dad46bf1233031412b406b9928c31e00e5ab472e5676212fac43488b6",
+      "parent_rank": 2548
+    },
+    {
+      "coordinate_dim": 1592,
+      "in_features": 4096,
+      "input_sha256": "f5eff86c0dbf4dcf5d745ee645f624d89397fe9cb282a0eb180698adfabf06a1",
+      "module": "model.layers.13.mlp.up_proj",
+      "out_features": 14336,
+      "output_sha256": "cd7d73f481aea9544b74d1a9c64452c24c71dbc280740cd561442642bc44eb6e",
+      "parent_rank": 2548
+    },
+    {
+      "coordinate_dim": 1024,
+      "in_features": 4096,
+      "input_sha256": "e283513731ab7dc151477776a5f5377726db0a99019fc45f27254c9d9deef9ea",
+      "module": "model.layers.14.self_attn.q_proj",
+      "out_features": 4096,
+      "output_sha256": "9dc37a6be1f91f6075cefa86645182d5e15719dad8e88a9e6db2f20f47e3897b",
+      "parent_rank": 1638
+    },
+    {
+      "coordinate_dim": 409,
+      "in_features": 4096,
+      "input_sha256": "c2785715a1c0ee0a3d174d398f8b42564bf98a8451e300986a14181a9b89c11c",
+      "module": "model.layers.14.self_attn.k_proj",
+      "out_features": 1024,
+      "output_sha256": "2dcb36ae4268ae95fae6334d56c40febdbde55b02e2260fe921f475c4c9c33a8",
+      "parent_rank": 655
+    },
+    {
+      "coordinate_dim": 409,
+      "in_features": 4096,
+      "input_sha256": "4953df7e3ff174a0a54837f9f68009344a8c4cbd82f320e650236775716c4934",
+      "module": "model.layers.14.self_attn.v_proj",
+      "out_features": 1024,
+      "output_sha256": "38af8ba31ff8c7ea5e8d67b97ea3e08f5876c035af3fabf56a4253fdb11eb8d1",
+      "parent_rank": 655
+    },
+    {
+      "coordinate_dim": 1024,
+      "in_features": 4096,
+      "input_sha256": "ee66a43487712f6f0405881cf0685e33814ce9bf88878556ea5714fa50e0e7c1",
+      "module": "model.layers.14.self_attn.o_proj",
+      "out_features": 4096,
+      "output_sha256": "ea9d3854d759f3450a86db92e1b3c02aaadf68a88e1378747fec90a57274598b",
+      "parent_rank": 1638
+    },
+    {
+      "coordinate_dim": 1592,
+      "in_features": 4096,
+      "input_sha256": "e11f1270ea1fb5e6b84c42c75f5636470eadd83f1312865dfff9de1bce350f4b",
+      "module": "model.layers.14.mlp.gate_proj",
+      "out_features": 14336,
+      "output_sha256": "5afb8f6f0a1701e4f96e55fd15fd44cc2aa7759fd5aabe60d7b49379b7d33011",
+      "parent_rank": 2548
+    },
+    {
+      "coordinate_dim": 1592,
+      "in_features": 14336,
+      "input_sha256": "e629cef5fd1272567805ed290389c6fcdcd15222a50635dd1651199c7b5961f5",
+      "module": "model.layers.14.mlp.down_proj",
+      "out_features": 4096,
+      "output_sha256": "80fcdacb448d63afb27dbf0ae46e4d52e9ead6c484a9d126c7da410a187fd1e8",
+      "parent_rank": 2548
+    },
+    {
+      "coordinate_dim": 1592,
+      "in_features": 4096,
+      "input_sha256": "a072e87f7b4d8338000fe25716cb739d6bd4f848e85a0edfebf87f5423c3aa35",
+      "module": "model.layers.14.mlp.up_proj",
+      "out_features": 14336,
+      "output_sha256": "4ba0084bdbf2dccadad19e9a7a099ebeb8ecd6f8305292191fa7f0eadad1ac30",
+      "parent_rank": 2548
+    },
+    {
+      "coordinate_dim": 1024,
+      "in_features": 4096,
+      "input_sha256": "17b0cb82e49b13b544f517a62cb653a4d6ea3a9d24f18e58714807bb508d670b",
+      "module": "model.layers.15.self_attn.q_proj",
+      "out_features": 4096,
+      "output_sha256": "dc367742e886dd7034f0725fc5863e7ef232cbfd6bc3840a014d958b62b13383",
+      "parent_rank": 1638
+    },
+    {
+      "coordinate_dim": 409,
+      "in_features": 4096,
+      "input_sha256": "8dbeffb08ac0d632c7a75a70988f50ba82e5624db528e98e98ed74cd25bd1d06",
+      "module": "model.layers.15.self_attn.k_proj",
+      "out_features": 1024,
+      "output_sha256": "b7bca364a5a0968926adae9ce7ebd1a8d48032a77f2666aefe360c13549e425a",
+      "parent_rank": 655
+    },
+    {
+      "coordinate_dim": 409,
+      "in_features": 4096,
+      "input_sha256": "a8bfeaa374b7376f04074b2c53362591543b1367cb2323ddc75c27ff1f30f7e6",
+      "module": "model.layers.15.self_attn.v_proj",
+      "out_features": 1024,
+      "output_sha256": "e797f50944d1952db1429f71d8e20aaafb8fd18ac9f632594714aff7d2d8f5b0",
+      "parent_rank": 655
+    },
+    {
+      "coordinate_dim": 1024,
+      "in_features": 4096,
+      "input_sha256": "0c5b4dd96e1ad058e890e8ef797b1464aa3abdc6acaa5d341cfda156f69e2c67",
+      "module": "model.layers.15.self_attn.o_proj",
+      "out_features": 4096,
+      "output_sha256": "e78ff1290b475cd9d0afcf6cbe4d8e969f2b9fafdbfde59f96d3edd4c77d7f74",
+      "parent_rank": 1638
+    },
+    {
+      "coordinate_dim": 1592,
+      "in_features": 4096,
+      "input_sha256": "57d72adba7cfbb18b0e44ec08d4f3bac20be117979acd612e9e62d63e2fa5196",
+      "module": "model.layers.15.mlp.gate_proj",
+      "out_features": 14336,
+      "output_sha256": "68060616bee6c05871ec6966fc0eb3a502bec0454a84e15bea12aab146d04766",
+      "parent_rank": 2548
+    },
+    {
+      "coordinate_dim": 1592,
+      "in_features": 14336,
+      "input_sha256": "12339957a652637dcdcf324d758ff8dcc54786d80d633cda7fc898b1e4c58236",
+      "module": "model.layers.15.mlp.down_proj",
+      "out_features": 4096,
+      "output_sha256": "0210808843b4c955ad81db9fd3988eed28956874eafbd8fa49214ccba5dcb47b",
+      "parent_rank": 2548
+    },
+    {
+      "coordinate_dim": 1592,
+      "in_features": 4096,
+      "input_sha256": "4cde28ad8cd12cadf8f03542e2ea0b22432c4e205344050433a266f494adea7d",
+      "module": "model.layers.15.mlp.up_proj",
+      "out_features": 14336,
+      "output_sha256": "e1e88a5f082a8ec7fade459e4085d95dd105ebcbc9f1ab46108e8f6d708b1d02",
+      "parent_rank": 2548
+    },
+    {
+      "coordinate_dim": 1024,
+      "in_features": 4096,
+      "input_sha256": "38a39af09789abcf63c9a20dcaace1120b4bd84b74e35c049fb5704f60686c66",
+      "module": "model.layers.16.self_attn.q_proj",
+      "out_features": 4096,
+      "output_sha256": "3f88e9a3b134d055c5a5235c089787e6e46725331b81cc9e394dcd88917373b5",
+      "parent_rank": 1638
+    },
+    {
+      "coordinate_dim": 409,
+      "in_features": 4096,
+      "input_sha256": "8c294c6cec6fee186933076fee17b0ddcf80b9fba6fdf20b83aecc9c7579d2b4",
+      "module": "model.layers.16.self_attn.k_proj",
+      "out_features": 1024,
+      "output_sha256": "3ec5587fcc9b09c32e1e05aac89de49b3ac90ecaeb0dff91d5901d380bc6db3b",
+      "parent_rank": 655
+    },
+    {
+      "coordinate_dim": 409,
+      "in_features": 4096,
+      "input_sha256": "14ed7aca6d3fec3864522ecf2b2cd18136f5bb24f3db947c0d0ae3e881dee4fd",
+      "module": "model.layers.16.self_attn.v_proj",
+      "out_features": 1024,
+      "output_sha256": "fac57cb7be160d08db295caf9c1531bef04df861b7390c62017727c0fcdb5068",
+      "parent_rank": 655
+    },
+    {
+      "coordinate_dim": 1024,
+      "in_features": 4096,
+      "input_sha256": "7eefe30b7bde8899941bd5fbdf5227aa2c38e607a5a120d3fb804f8cd0a78681",
+      "module": "model.layers.16.self_attn.o_proj",
+      "out_features": 4096,
+      "output_sha256": "ae18943b01b4c10b6bc5de22b467ac38117584b94b2b86d6ceb1e0c7a4e7e7b8",
+      "parent_rank": 1638
+    },
+    {
+      "coordinate_dim": 1592,
+      "in_features": 4096,
+      "input_sha256": "cd59aa2ac0f6558704c96fd9353c4a89a1afdccbdd20e57676a4483ef3080803",
+      "module": "model.layers.16.mlp.gate_proj",
+      "out_features": 14336,
+      "output_sha256": "e3de65c482bb617cf052ffecb1d3178a1919912c2b71c508cf73405b30d2c9a4",
+      "parent_rank": 2548
+    },
+    {
+      "coordinate_dim": 1592,
+      "in_features": 14336,
+      "input_sha256": "f6db010dd27317867efac3daaff23d2e3590a48f0a28b5337af7168dd31037ab",
+      "module": "model.layers.16.mlp.down_proj",
+      "out_features": 4096,
+      "output_sha256": "a10307dc888a29c89df673ab96195d5f20767a2e3b2e5f4aba503175fa74bf70",
+      "parent_rank": 2548
+    },
+    {
+      "coordinate_dim": 1592,
+      "in_features": 4096,
+      "input_sha256": "20091baf015962b0d4c1102165ebfd73553f68da08a5a615f5942f2704d19b3d",
+      "module": "model.layers.16.mlp.up_proj",
+      "out_features": 14336,
+      "output_sha256": "6a2852d80cbf0b9b81450244004312006c84bc058b03a739af6447986975f21f",
+      "parent_rank": 2548
+    },
+    {
+      "coordinate_dim": 1024,
+      "in_features": 4096,
+      "input_sha256": "480ae8b48244fef94c90333a5236edb505fab035d281b86f80edce9dcca0a719",
+      "module": "model.layers.17.self_attn.q_proj",
+      "out_features": 4096,
+      "output_sha256": "7618c0f3a800997ebe32b7cb367caebcccc0dd7fc923a22bd3b18f194dac8323",
+      "parent_rank": 1638
+    },
+    {
+      "coordinate_dim": 409,
+      "in_features": 4096,
+      "input_sha256": "c16ba5d30c110a199c11b5d8faee039df975a60f38d04c7d4361ff564a941143",
+      "module": "model.layers.17.self_attn.k_proj",
+      "out_features": 1024,
+      "output_sha256": "089a8b7129e2db9651c4d1ab74ea21494439ea44cb19b33e2da59a246799d5bd",
+      "parent_rank": 655
+    },
+    {
+      "coordinate_dim": 409,
+      "in_features": 4096,
+      "input_sha256": "f36f3fd8d95c8318d2147c09e66af6eec040b21b35fd1212e0f843dc1773c9b6",
+      "module": "model.layers.17.self_attn.v_proj",
+      "out_features": 1024,
+      "output_sha256": "a2eb6cf93fa1e0634de0bc8c40d103be64c9a630d9fb3a277c5e4e26ba516393",
+      "parent_rank": 655
+    },
+    {
+      "coordinate_dim": 1024,
+      "in_features": 4096,
+      "input_sha256": "95ff777adc442a31c32ae53adb9cd37d6959c881466b2c14219046be0580d0a0",
+      "module": "model.layers.17.self_attn.o_proj",
+      "out_features": 4096,
+      "output_sha256": "6419a28104b317d6cbcec93de34364bd0db067ece9909194c3b871679ef80b0c",
+      "parent_rank": 1638
+    },
+    {
+      "coordinate_dim": 1592,
+      "in_features": 4096,
+      "input_sha256": "ea21444e3922cabe590cc66d6ff112f86c4181912a5f50b0b1ea4b528cac22ff",
+      "module": "model.layers.17.mlp.gate_proj",
+      "out_features": 14336,
+      "output_sha256": "c05fa8c44b01cee1bcdd9924717785b0ee74dfd93342fee9a058ccbeee44f5cc",
+      "parent_rank": 2548
+    },
+    {
+      "coordinate_dim": 1592,
+      "in_features": 14336,
+      "input_sha256": "064effcaf81d65e1ef30d83620bddcc447e12415c72fd5c5607bee1344443abe",
+      "module": "model.layers.17.mlp.down_proj",
+      "out_features": 4096,
+      "output_sha256": "fb69b3a515aa7640d30454b6db7acd0b4714adfffcc66cae66221a8a438d6217",
+      "parent_rank": 2548
+    },
+    {
+      "coordinate_dim": 1592,
+      "in_features": 4096,
+      "input_sha256": "f4eeb26cfc191f9212c3338397d682c90257c02c88c6fc12d059e247094269c2",
+      "module": "model.layers.17.mlp.up_proj",
+      "out_features": 14336,
+      "output_sha256": "cc19bf6c200c84706ec25eee2f398cfa9e8558e900006c9f3839484de5bb411c",
+      "parent_rank": 2548
+    },
+    {
+      "coordinate_dim": 1024,
+      "in_features": 4096,
+      "input_sha256": "b26f6e587f29700c7dbc0efb05d472ff761e2c9ec38683112952baf185d21b67",
+      "module": "model.layers.18.self_attn.q_proj",
+      "out_features": 4096,
+      "output_sha256": "07dcb80b85194cac61b54b3158cf08f71081358c0be5fadb53c560decda74931",
+      "parent_rank": 1638
+    },
+    {
+      "coordinate_dim": 409,
+      "in_features": 4096,
+      "input_sha256": "1bbba2830de11eac3a3141241fe8ba6693b61cca1b48a6cdaefc4b137336e780",
+      "module": "model.layers.18.self_attn.k_proj",
+      "out_features": 1024,
+      "output_sha256": "58c925c5b1f14db26e05d9a29831aac0110262801e429db25c6728cadfc5e96f",
+      "parent_rank": 655
+    },
+    {
+      "coordinate_dim": 409,
+      "in_features": 4096,
+      "input_sha256": "366143ad5639baf593edbc4f83ed9cef8c3018919ae159cc7f9176ed906b1555",
+      "module": "model.layers.18.self_attn.v_proj",
+      "out_features": 1024,
+      "output_sha256": "5b226dd0180b7063ea17308b52b7c8e2f7d94bde1d2b4d68cac27d07eb81ddaa",
+      "parent_rank": 655
+    },
+    {
+      "coordinate_dim": 1024,
+      "in_features": 4096,
+      "input_sha256": "697ed47bf921fffcaaf1b9c4cd697c122a6e9a09604c239eb3a9c6a92f206037",
+      "module": "model.layers.18.self_attn.o_proj",
+      "out_features": 4096,
+      "output_sha256": "6771db9e9e2e908c8e6151f4a6dd2e309e6e06795d2c023ec74e57f6f10f4771",
+      "parent_rank": 1638
+    },
+    {
+      "coordinate_dim": 1592,
+      "in_features": 4096,
+      "input_sha256": "093126069ef00be3dbf2cf65ecc2f7f423d42aebd59b0cb19c7973a8d27a00aa",
+      "module": "model.layers.18.mlp.gate_proj",
+      "out_features": 14336,
+      "output_sha256": "6ed113db63078c02dab6905eb80d21ef9657574523f49b0ae868f26a5b0b61b5",
+      "parent_rank": 2548
+    },
+    {
+      "coordinate_dim": 1592,
+      "in_features": 14336,
+      "input_sha256": "de0de094f0a8390e538ac6c368d95484b3750f7b3c6e5b259b91559a50c72ec1",
+      "module": "model.layers.18.mlp.down_proj",
+      "out_features": 4096,
+      "output_sha256": "8a048bfa26f40e99fdea22d4b3e76f7e81d7f7c737e931e36b6486f7be46349e",
+      "parent_rank": 2548
+    },
+    {
+      "coordinate_dim": 1592,
+      "in_features": 4096,
+      "input_sha256": "c9cad723bc442132b845d8cdafeb461917fe2b0d1d8ffb67908c426e4e1d915c",
+      "module": "model.layers.18.mlp.up_proj",
+      "out_features": 14336,
+      "output_sha256": "e3a8b4e92e8a4d2b529be135390157563e89926e909e9fd4ef3dd571a41c70a7",
+      "parent_rank": 2548
+    },
+    {
+      "coordinate_dim": 1024,
+      "in_features": 4096,
+      "input_sha256": "1aa1fb9c0c998a60214724244fff9e8a14068e1b2d7f888b5c9bc3c10ef0bfc1",
+      "module": "model.layers.19.self_attn.q_proj",
+      "out_features": 4096,
+      "output_sha256": "0e6326803397274492f3f7e465f8de3617667998ba4d49ededf12ae26310a8f1",
+      "parent_rank": 1638
+    },
+    {
+      "coordinate_dim": 409,
+      "in_features": 4096,
+      "input_sha256": "456d0aa5edbfc987247eed555515123c85005b4c96f04144b44f99fa732dcc84",
+      "module": "model.layers.19.self_attn.k_proj",
+      "out_features": 1024,
+      "output_sha256": "572aac553f077d236617c8301f32771b9bd3c4dac7ee536e215712e7828aea88",
+      "parent_rank": 655
+    },
+    {
+      "coordinate_dim": 409,
+      "in_features": 4096,
+      "input_sha256": "cbb53efbc8f6edc458324c6d973038a42bf953ea970a25b1673ddece5765a45c",
+      "module": "model.layers.19.self_attn.v_proj",
+      "out_features": 1024,
+      "output_sha256": "8b774e1a2109362190c6c595d486b94c2d2beca2e193a2ad9c2befaaafdd63b8",
+      "parent_rank": 655
+    },
+    {
+      "coordinate_dim": 1024,
+      "in_features": 4096,
+      "input_sha256": "5f7f30f3fcce1cf953cde8f066b861ab173087d96a634d3d89a1ba994aa37f9d",
+      "module": "model.layers.19.self_attn.o_proj",
+      "out_features": 4096,
+      "output_sha256": "f8ed8a5627d4a5b71e9e40a7a2950aa7c7c37e5cc6e3692d82b768c0b2a4369d",
+      "parent_rank": 1638
+    },
+    {
+      "coordinate_dim": 1592,
+      "in_features": 4096,
+      "input_sha256": "33eddae6e5f5a6daaaa6cfa51cc526b8febf505c9f0c5518a383fe86a62fe2ab",
+      "module": "model.layers.19.mlp.gate_proj",
+      "out_features": 14336,
+      "output_sha256": "1ece7603d00e6f7733d15067bf0b32e942daf936488610b28f4601089be18d58",
+      "parent_rank": 2548
+    },
+    {
+      "coordinate_dim": 1592,
+      "in_features": 14336,
+      "input_sha256": "54e6c84339d4d172452ec4b807a7a9d300712cb3bc333b4333c0e2dd2f5fe4d9",
+      "module": "model.layers.19.mlp.down_proj",
+      "out_features": 4096,
+      "output_sha256": "401b10dad7a6f53ac17708a2306d494b592f434abc3a8dea39fb73ccbc0f5265",
+      "parent_rank": 2548
+    },
+    {
+      "coordinate_dim": 1592,
+      "in_features": 4096,
+      "input_sha256": "a7c3ef468651a65069d0dcb2a80b6adeb41001900dbae5562cd24ba454b233c0",
+      "module": "model.layers.19.mlp.up_proj",
+      "out_features": 14336,
+      "output_sha256": "1e1cc731148ac93926af43766420146994ddb9c9a66e6f2abdd67083b7247ab7",
+      "parent_rank": 2548
+    },
+    {
+      "coordinate_dim": 1024,
+      "in_features": 4096,
+      "input_sha256": "9b1f8ae84eff4e3ea0aff870b4e2fd87332ebd170d2e74c4ed30099f67dc24a7",
+      "module": "model.layers.20.self_attn.q_proj",
+      "out_features": 4096,
+      "output_sha256": "bc5c904e7bdc4bbd0aab594ca9b3c94c88910a967b6777c3e21c569130c238df",
+      "parent_rank": 1638
+    },
+    {
+      "coordinate_dim": 409,
+      "in_features": 4096,
+      "input_sha256": "6b78e0cb74473a3e74cb5d7811148972c84a454e1ae637c19b3809c945ff2bc6",
+      "module": "model.layers.20.self_attn.k_proj",
+      "out_features": 1024,
+      "output_sha256": "078b5536a518f77cb27e88d21e463b3c2f678ca4f414ea982ea2e17c92fb3265",
+      "parent_rank": 655
+    },
+    {
+      "coordinate_dim": 409,
+      "in_features": 4096,
+      "input_sha256": "890f15e53c60483a0c915dd955bcc0510935dacba713c0743083c8949ae59214",
+      "module": "model.layers.20.self_attn.v_proj",
+      "out_features": 1024,
+      "output_sha256": "3322944dedbef56816ac4ee458b4126fee4fc5a12e0eb7271b0048e253108ce1",
+      "parent_rank": 655
+    },
+    {
+      "coordinate_dim": 1024,
+      "in_features": 4096,
+      "input_sha256": "cd2b8c80d029a464c282cfda9ede4b9a4ddf4c283492cedc5f0da1dc792244b2",
+      "module": "model.layers.20.self_attn.o_proj",
+      "out_features": 4096,
+      "output_sha256": "9c7cfe9e7f36a77f3593cae081af00d9f3dc3cd03657f9e7e9646a4190948cb0",
+      "parent_rank": 1638
+    },
+    {
+      "coordinate_dim": 1592,
+      "in_features": 4096,
+      "input_sha256": "427d20c1c0ff8b2137eb9300d4bcec0a7a4993ae66c636cc7aedd34be99a6323",
+      "module": "model.layers.20.mlp.gate_proj",
+      "out_features": 14336,
+      "output_sha256": "47f97b57034d775187b3824fffe0f66f88713fbc1ee5f01c6a6aa9e949a152c8",
+      "parent_rank": 2548
+    },
+    {
+      "coordinate_dim": 1592,
+      "in_features": 14336,
+      "input_sha256": "6ca87037ae7fad7b0161f0a843d0620fd8a57a9dc1f4deb390e8b8bbac5866c7",
+      "module": "model.layers.20.mlp.down_proj",
+      "out_features": 4096,
+      "output_sha256": "be3bc86d5272009f3980cfa775de4e167c245b1cded5b4f2a5ec024e96f984c3",
+      "parent_rank": 2548
+    },
+    {
+      "coordinate_dim": 1592,
+      "in_features": 4096,
+      "input_sha256": "54c341aa2d26effdff156540c4c58cd177638266d120c286683141157220e8f0",
+      "module": "model.layers.20.mlp.up_proj",
+      "out_features": 14336,
+      "output_sha256": "909b5bf8a2ca3cec0b49838d7397e1c8e700fb4a1d7b7d0acc3645a22ff99718",
+      "parent_rank": 2548
+    },
+    {
+      "coordinate_dim": 1024,
+      "in_features": 4096,
+      "input_sha256": "b1bfa8e5e5eb78dc4c8fe666be25bb87c2ce95a6c8ea0a1879dec69f6cbc94b3",
+      "module": "model.layers.21.self_attn.q_proj",
+      "out_features": 4096,
+      "output_sha256": "7366aca48c108b65d585e30d28f1471406fc24e3a7ee4621f8fdce6e0cf2d63f",
+      "parent_rank": 1638
+    },
+    {
+      "coordinate_dim": 409,
+      "in_features": 4096,
+      "input_sha256": "a3acc2952d400115b60d8ec827ff439cf891bbef8ce473aed056369ac3e383e7",
+      "module": "model.layers.21.self_attn.k_proj",
+      "out_features": 1024,
+      "output_sha256": "6a870e4f590b6860dece0c6aa3fcfbc6f063e8d1dd809387f9fde183613b973a",
+      "parent_rank": 655
+    },
+    {
+      "coordinate_dim": 409,
+      "in_features": 4096,
+      "input_sha256": "c998d11a7fe79ce3b6e451cf817f292eecc0774e1d17d2592b729ce4cbdc684f",
+      "module": "model.layers.21.self_attn.v_proj",
+      "out_features": 1024,
+      "output_sha256": "7771bcab899ce3b7edaf1690b8a35acceb2177a667404e9500bc9802ed952876",
+      "parent_rank": 655
+    },
+    {
+      "coordinate_dim": 1024,
+      "in_features": 4096,
+      "input_sha256": "466bcc43c51fdf3b8f1bf86c1b935a3fd325b94a60aca11f42f5b29fa3c869a0",
+      "module": "model.layers.21.self_attn.o_proj",
+      "out_features": 4096,
+      "output_sha256": "4bae2f816eab221b1b2babfdad86c2e91cc67a41ed547c7a074278b0601696cb",
+      "parent_rank": 1638
+    },
+    {
+      "coordinate_dim": 1592,
+      "in_features": 4096,
+      "input_sha256": "60df51821e10355b075043ebea8cd1712fd861961f652f4c9a598b5b54e256f6",
+      "module": "model.layers.21.mlp.gate_proj",
+      "out_features": 14336,
+      "output_sha256": "74e77c6eb8f07bedfb5bfb96c9ee1eeb8058b80e1906f051679e71e9fca0e58f",
+      "parent_rank": 2548
+    },
+    {
+      "coordinate_dim": 1592,
+      "in_features": 14336,
+      "input_sha256": "f557fa6716f8ab1afb18ee9860f942401cb920685c42716c1c83e57b2e46b526",
+      "module": "model.layers.21.mlp.down_proj",
+      "out_features": 4096,
+      "output_sha256": "37f4147dc0b06f8d51d1d2af4edfa533da30321ea98b9f0983a8201eadf7ce03",
+      "parent_rank": 2548
+    },
+    {
+      "coordinate_dim": 1592,
+      "in_features": 4096,
+      "input_sha256": "52d3903b6a20d72668543c8a4164d11d5a38cde60fdd87b0076113c19fb6d9df",
+      "module": "model.layers.21.mlp.up_proj",
+      "out_features": 14336,
+      "output_sha256": "f446909b05e06f49e4965b754ed3ae13da8f115512a356d2fa95983419241aed",
+      "parent_rank": 2548
+    },
+    {
+      "coordinate_dim": 1024,
+      "in_features": 4096,
+      "input_sha256": "3d0d3694544a4014fe8ec366951ebf4f1a10babc9539f4db6268f4fda3d20dc1",
+      "module": "model.layers.22.self_attn.q_proj",
+      "out_features": 4096,
+      "output_sha256": "208c52587aacabcacef582440d2e0f24fafd25244856a6e88d8b47d47c083101",
+      "parent_rank": 1638
+    },
+    {
+      "coordinate_dim": 409,
+      "in_features": 4096,
+      "input_sha256": "eb224b4b310e39e71514cce69043f432de998f3ea2ccf9b513eb8392ebc3fc66",
+      "module": "model.layers.22.self_attn.k_proj",
+      "out_features": 1024,
+      "output_sha256": "cdad871421077caf8f574b191ec57edb022144b27e889c9d9befa48442db2a2e",
+      "parent_rank": 655
+    },
+    {
+      "coordinate_dim": 409,
+      "in_features": 4096,
+      "input_sha256": "74a4e1a3430205e0d7e5e1b32352d9f5ae2b9a088484033c87ea5991dc217f52",
+      "module": "model.layers.22.self_attn.v_proj",
+      "out_features": 1024,
+      "output_sha256": "62179fd82cefd1435cb88f53accea39aed82c472fb06980975e96845dd1d3c49",
+      "parent_rank": 655
+    },
+    {
+      "coordinate_dim": 1024,
+      "in_features": 4096,
+      "input_sha256": "4613d5c68bc4cd3bdd93e107a4f1acf293bd65b0132adc9beaef74ae89138d08",
+      "module": "model.layers.22.self_attn.o_proj",
+      "out_features": 4096,
+      "output_sha256": "ae4b26f1cc73db6283ff0e4200b8e521fa37383c830fd1b1b8a59079976c1781",
+      "parent_rank": 1638
+    },
+    {
+      "coordinate_dim": 1592,
+      "in_features": 4096,
+      "input_sha256": "82c8fc66aa6a289b0bc2d9d65500c50fd414b31411d871df945950d18dbc4c02",
+      "module": "model.layers.22.mlp.gate_proj",
+      "out_features": 14336,
+      "output_sha256": "7cf6771e883efd80b1e13871bf6d82905484ec4c89d425b1b26dfee7b27186e5",
+      "parent_rank": 2548
+    },
+    {
+      "coordinate_dim": 1592,
+      "in_features": 14336,
+      "input_sha256": "c6ab15c6d73976b8003ef3a940b5cf503d96d1faf4e06ebec3c4ea4d455470d8",
+      "module": "model.layers.22.mlp.down_proj",
+      "out_features": 4096,
+      "output_sha256": "c42315577ecbcd4bded1c0400b4ad9e1326f31a2a2e5aaadcb6d9b73f7cbff21",
+      "parent_rank": 2548
+    },
+    {
+      "coordinate_dim": 1592,
+      "in_features": 4096,
+      "input_sha256": "25ad78d6791f125d9ebf784ebaccd442a52d61a5028e426517b3300dd55746d2",
+      "module": "model.layers.22.mlp.up_proj",
+      "out_features": 14336,
+      "output_sha256": "acaaf03dab1d04478cd906833204577c039930bcd2929ef1cb2d3ec00eca3ccf",
+      "parent_rank": 2548
+    },
+    {
+      "coordinate_dim": 1024,
+      "in_features": 4096,
+      "input_sha256": "1d6fc32d66917c015710cbc1ac1105d6140992cbeef50e8465356e3b21faec9a",
+      "module": "model.layers.23.self_attn.q_proj",
+      "out_features": 4096,
+      "output_sha256": "dd4d54d1be037c84bd715a688081c0154d2da66e36b6cf289f8c88a214f9c31c",
+      "parent_rank": 1638
+    },
+    {
+      "coordinate_dim": 409,
+      "in_features": 4096,
+      "input_sha256": "c732ceb4e9571d07b3c266fcf269a2821f4ba63a595a77d49ed7439606751ed7",
+      "module": "model.layers.23.self_attn.k_proj",
+      "out_features": 1024,
+      "output_sha256": "6c7ae9e3e619be5fda511881c2187c1a86204b797c34423d9ee4be5164a7d84e",
+      "parent_rank": 655
+    },
+    {
+      "coordinate_dim": 409,
+      "in_features": 4096,
+      "input_sha256": "d55d2405213aa66bfcd1b9fbccc269635ddb2dc45fcd42658fb916797854b3fa",
+      "module": "model.layers.23.self_attn.v_proj",
+      "out_features": 1024,
+      "output_sha256": "8ffcfc18864e162cac65c3b6f613b89e3aaa3c51f1907acb8f2a8e8ab003ec48",
+      "parent_rank": 655
+    },
+    {
+      "coordinate_dim": 1024,
+      "in_features": 4096,
+      "input_sha256": "02c098dd6a41e9f7938f0f2ad1eb8d64464b3e28d0fb33a9b2e4aa86c89847cc",
+      "module": "model.layers.23.self_attn.o_proj",
+      "out_features": 4096,
+      "output_sha256": "a83aca7e7de1fada33bfcf640fac83b29771739f8ebd90812c9cef2d1ccbc8ee",
+      "parent_rank": 1638
+    },
+    {
+      "coordinate_dim": 1592,
+      "in_features": 4096,
+      "input_sha256": "a1af7040fcb006295af8fbadce5a6c80989d59be0771a28a9d8f0169248087d3",
+      "module": "model.layers.23.mlp.gate_proj",
+      "out_features": 14336,
+      "output_sha256": "a16f60a1c18645bb33f2d01d07b7624fb0f73f6c291a2605f481e5422892ed49",
+      "parent_rank": 2548
+    },
+    {
+      "coordinate_dim": 1592,
+      "in_features": 14336,
+      "input_sha256": "3851abc6dd15665c79048bb602210a0083e2068056bb965e15a779dce09f5437",
+      "module": "model.layers.23.mlp.down_proj",
+      "out_features": 4096,
+      "output_sha256": "aaba6ed34f607c5916777d435cf951659c3b27caa04859120f7806af929b27ea",
+      "parent_rank": 2548
+    },
+    {
+      "coordinate_dim": 1592,
+      "in_features": 4096,
+      "input_sha256": "c8299c5eb64cd459cc56461757aeef04b640ca3296e574ebe4b68cb5b7d62ca7",
+      "module": "model.layers.23.mlp.up_proj",
+      "out_features": 14336,
+      "output_sha256": "6c8f05d4d3cda6f0c54df166a8ac3c48ec008cc9945fc8246a50e01cf1d566ec",
+      "parent_rank": 2548
+    },
+    {
+      "coordinate_dim": 1024,
+      "in_features": 4096,
+      "input_sha256": "327e4080f4b4d60aafed7909280fbf7763abfa26e8770ecd7f4d673e6fb66d1c",
+      "module": "model.layers.24.self_attn.q_proj",
+      "out_features": 4096,
+      "output_sha256": "e3a3af7ee54cdb584b32a6e1bfd7222a7c0c7c45b912d9023e2eaf43c6beafe7",
+      "parent_rank": 1638
+    },
+    {
+      "coordinate_dim": 409,
+      "in_features": 4096,
+      "input_sha256": "56c99362747b4b0663d77faade4d09ee62e08c9b995ca2725da9dcdad107378f",
+      "module": "model.layers.24.self_attn.k_proj",
+      "out_features": 1024,
+      "output_sha256": "27d04e26325301d1f00f34eb88892adf389bb4b4462683b1dd6df750c4740332",
+      "parent_rank": 655
+    },
+    {
+      "coordinate_dim": 409,
+      "in_features": 4096,
+      "input_sha256": "8a282be8d82ee862b6426a99948d5dd00e862a078e6740cce06ec7c8ff4f2c5a",
+      "module": "model.layers.24.self_attn.v_proj",
+      "out_features": 1024,
+      "output_sha256": "08145c0520998415705996ea22557d0a66d6dd082cab8e466f6fbb3194fbc89d",
+      "parent_rank": 655
+    },
+    {
+      "coordinate_dim": 1024,
+      "in_features": 4096,
+      "input_sha256": "8412729cb76652dc922416ad7221f9796737b59dcfabd71fe56b6ebcd8357f2b",
+      "module": "model.layers.24.self_attn.o_proj",
+      "out_features": 4096,
+      "output_sha256": "440854624ae5619d61f812fbfe5a737a1278f385117cc9f201cf3c194ae7a83c",
+      "parent_rank": 1638
+    },
+    {
+      "coordinate_dim": 1592,
+      "in_features": 4096,
+      "input_sha256": "61200a3157d37387f0aafc59fb5ada583ee1296bc282a1c099dc5f5e53a7a537",
+      "module": "model.layers.24.mlp.gate_proj",
+      "out_features": 14336,
+      "output_sha256": "c84318ac9e115e607e4ffac5797f6d5b3671cb2652ffdaabd090652d32d13ab1",
+      "parent_rank": 2548
+    },
+    {
+      "coordinate_dim": 1592,
+      "in_features": 14336,
+      "input_sha256": "81807f5ce9a42f3a163d41e9d3740f7e1b765b206c199712a78c7c7c03c8a2e8",
+      "module": "model.layers.24.mlp.down_proj",
+      "out_features": 4096,
+      "output_sha256": "87726b235795ebbd8b3724b41a865d643cbcd33a61c9e78e20216f0def8ba9d9",
+      "parent_rank": 2548
+    },
+    {
+      "coordinate_dim": 1592,
+      "in_features": 4096,
+      "input_sha256": "046e14ce2591232e4bd715ad730697ec5189cd365f5ccfdd38faf0736d931578",
+      "module": "model.layers.24.mlp.up_proj",
+      "out_features": 14336,
+      "output_sha256": "8f6cd76183c80f2df98a8fbbf98cb3eeb84c2bdd42ff5672a6759fb6e89d7231",
+      "parent_rank": 2548
+    },
+    {
+      "coordinate_dim": 1024,
+      "in_features": 4096,
+      "input_sha256": "dc3e0911cc72a28b166e4253be7e1c56f8916021fb6ca659e39055f53dbf840c",
+      "module": "model.layers.25.self_attn.q_proj",
+      "out_features": 4096,
+      "output_sha256": "d9c9690100f8231f50f68449c4ca4ee615bfb8ef567e28bbac2ca930910b37bb",
+      "parent_rank": 1638
+    },
+    {
+      "coordinate_dim": 409,
+      "in_features": 4096,
+      "input_sha256": "740966b9a1c0a4db36200f61c5c583f09675a9c4ee50c9996f1544d09777ac00",
+      "module": "model.layers.25.self_attn.k_proj",
+      "out_features": 1024,
+      "output_sha256": "ba7c8cada91fefc0629c8a9c20e7741672f38212b19dcc5be8886af261ea7de1",
+      "parent_rank": 655
+    },
+    {
+      "coordinate_dim": 409,
+      "in_features": 4096,
+      "input_sha256": "bac8d8f97a72e4298a20a2f60fb54cf1218d677bc6564161ac52246b52ab1531",
+      "module": "model.layers.25.self_attn.v_proj",
+      "out_features": 1024,
+      "output_sha256": "5a0614e697e040a3803c90ec7f5fcb32e6c208db78afe0d9b305ce8c32eac276",
+      "parent_rank": 655
+    },
+    {
+      "coordinate_dim": 1024,
+      "in_features": 4096,
+      "input_sha256": "6c12474797cb2cdd6fb3da3a598f9ed53c2b93cc945f012b9c19a46c80937f40",
+      "module": "model.layers.25.self_attn.o_proj",
+      "out_features": 4096,
+      "output_sha256": "4042bbd9d2e33a3398114fdafcde1a0fd0e94f82e6a34a8aede8f22e0ad67204",
+      "parent_rank": 1638
+    },
+    {
+      "coordinate_dim": 1592,
+      "in_features": 4096,
+      "input_sha256": "b721f791ede125451254067f6d9bbefaf755acd5c282800915d0ad522c9bcf1c",
+      "module": "model.layers.25.mlp.gate_proj",
+      "out_features": 14336,
+      "output_sha256": "b5f0476b58dfcbaa8e1528e74d75e58866ad6900a9cf3be299ed36decbf171bd",
+      "parent_rank": 2548
+    },
+    {
+      "coordinate_dim": 1592,
+      "in_features": 14336,
+      "input_sha256": "f4ab3e62ed7ec29e52e22612ff9ac926ff89421251ec3abe87522289a0e60092",
+      "module": "model.layers.25.mlp.down_proj",
+      "out_features": 4096,
+      "output_sha256": "a8512de8e271439496fba3252cf1f659454fd39fe1fd5aae6331682cce38886e",
+      "parent_rank": 2548
+    },
+    {
+      "coordinate_dim": 1592,
+      "in_features": 4096,
+      "input_sha256": "739a2e7261828d48cad67b01db6e31450c43bdcf135e770ffeefd04e0a768533",
+      "module": "model.layers.25.mlp.up_proj",
+      "out_features": 14336,
+      "output_sha256": "74a0e5ee0c0a13ec61a96d87c91f7aab124689a96527fcc052df7fd1f48bdedc",
+      "parent_rank": 2548
+    },
+    {
+      "coordinate_dim": 1024,
+      "in_features": 4096,
+      "input_sha256": "659d313b5fbf86a53e36f3c6f4a62360dcd2c8ce19f3662ef4da5136e3228c3e",
+      "module": "model.layers.26.self_attn.q_proj",
+      "out_features": 4096,
+      "output_sha256": "7423baac5884e2d0d949d746f2d2d4aa7c41f30083a7493939a068324ddd35b3",
+      "parent_rank": 1638
+    },
+    {
+      "coordinate_dim": 409,
+      "in_features": 4096,
+      "input_sha256": "d01520f55f2d9ad7d305073cfd4af80a564377cb0ae3ac6930164889481ac0a8",
+      "module": "model.layers.26.self_attn.k_proj",
+      "out_features": 1024,
+      "output_sha256": "e735e23caaa93e15cd4100d6c68914c3f3a3a03ba7743ee5e55cc24e2b0332a5",
+      "parent_rank": 655
+    },
+    {
+      "coordinate_dim": 409,
+      "in_features": 4096,
+      "input_sha256": "5c9f29f022fcede3092476e439d8537ea3a7c361a9d24558f0532ec575a83a51",
+      "module": "model.layers.26.self_attn.v_proj",
+      "out_features": 1024,
+      "output_sha256": "dd923ba844eeb5e5092a7bd066730e07f9cae843b9759664e8360c11abcca4c0",
+      "parent_rank": 655
+    },
+    {
+      "coordinate_dim": 1024,
+      "in_features": 4096,
+      "input_sha256": "342de862c637b57662a790d517c468b2c9c662e44704bdbaf9e5234e3855bcd9",
+      "module": "model.layers.26.self_attn.o_proj",
+      "out_features": 4096,
+      "output_sha256": "fcd1d1ec964488ebd1d66c7408bef65820eaf6422ba4df175866672ef1c8773d",
+      "parent_rank": 1638
+    },
+    {
+      "coordinate_dim": 1592,
+      "in_features": 4096,
+      "input_sha256": "123783d20eb86e890585aab76b2588c5fa04bfa8697841532b6ee004bc48f5ae",
+      "module": "model.layers.26.mlp.gate_proj",
+      "out_features": 14336,
+      "output_sha256": "af9a522de9ef4a5df2f41ad5058d034dc7dcf7ab57f8b6dfc99e13655f9dedb5",
+      "parent_rank": 2548
+    },
+    {
+      "coordinate_dim": 1592,
+      "in_features": 14336,
+      "input_sha256": "2a0c7980c6231ff9dd4e631085f4d1d1c470732a3a8364931ce27445b7062652",
+      "module": "model.layers.26.mlp.down_proj",
+      "out_features": 4096,
+      "output_sha256": "9b56abb176f93600984ddaadd10bda77869c1f7eebf33dd2319a420e53a4e003",
+      "parent_rank": 2548
+    },
+    {
+      "coordinate_dim": 1592,
+      "in_features": 4096,
+      "input_sha256": "809bf6f90e5a36f93a8035afe2061ea71338229a7d5240bdb96b6243b0344597",
+      "module": "model.layers.26.mlp.up_proj",
+      "out_features": 14336,
+      "output_sha256": "902606cdef14ae601cefd8606349268b7159e596206cf8600784ad4922475a2b",
+      "parent_rank": 2548
+    },
+    {
+      "coordinate_dim": 1024,
+      "in_features": 4096,
+      "input_sha256": "133aa7e2a1cbb3584fbe6caca05e7473a9df2c4771c8d077f33b3fbee31424e3",
+      "module": "model.layers.27.self_attn.q_proj",
+      "out_features": 4096,
+      "output_sha256": "ad7e7c80a81a71d92df45e319b53dd794cfae0420f9d47248bd15229a14c5638",
+      "parent_rank": 1638
+    },
+    {
+      "coordinate_dim": 409,
+      "in_features": 4096,
+      "input_sha256": "6a16c435d1ceda7c828bddad766b13d4bbee8edf43611b6c6b6ea9fc7084e146",
+      "module": "model.layers.27.self_attn.k_proj",
+      "out_features": 1024,
+      "output_sha256": "586d04a5d4feb61a77ffb69a79bc50ad90cf46ef855cd62b067c4e14b95bf691",
+      "parent_rank": 655
+    },
+    {
+      "coordinate_dim": 409,
+      "in_features": 4096,
+      "input_sha256": "55ec159eb1877d1749f97baa5e6a8153f65d72ef6ae58bdac3de127054d37706",
+      "module": "model.layers.27.self_attn.v_proj",
+      "out_features": 1024,
+      "output_sha256": "cefbfa12d70a3f0b96ffb464005f8c15e9d8c54cfee1480b435878456fb0b48a",
+      "parent_rank": 655
+    },
+    {
+      "coordinate_dim": 1024,
+      "in_features": 4096,
+      "input_sha256": "a219f60859682dec9cadb94a1a7048db0fd150aea80c78a66c861596c49a3011",
+      "module": "model.layers.27.self_attn.o_proj",
+      "out_features": 4096,
+      "output_sha256": "eb8d3c6e28f7213e10bc9778aa39c03b38a5163f3f78ffd2fffd8ff1048bd68b",
+      "parent_rank": 1638
+    },
+    {
+      "coordinate_dim": 1592,
+      "in_features": 4096,
+      "input_sha256": "8836c928f6348aa9a38f24d5fc398738265a2e4f10f803b37db7dd6e12121134",
+      "module": "model.layers.27.mlp.gate_proj",
+      "out_features": 14336,
+      "output_sha256": "deb61eeb650bd83c9bebe3e25683aa22b9da76b8c5d1e065bd7dff61e5625fcb",
+      "parent_rank": 2548
+    },
+    {
+      "coordinate_dim": 1592,
+      "in_features": 14336,
+      "input_sha256": "e6f7a890e00247d403cba406a8c2394b509ddbdde6a747c391340ccbcbc1c296",
+      "module": "model.layers.27.mlp.down_proj",
+      "out_features": 4096,
+      "output_sha256": "d16d0f1c35bcc2886fedad9009ff06cfe05755ca6a61f07726c85b79012a6d06",
+      "parent_rank": 2548
+    },
+    {
+      "coordinate_dim": 1592,
+      "in_features": 4096,
+      "input_sha256": "0e23a1bf9846e4dbd4fd559ef2d9478f0e18430ae93e47bbab14752955d14b08",
+      "module": "model.layers.27.mlp.up_proj",
+      "out_features": 14336,
+      "output_sha256": "a06fc344eaa67bfe5408071aef677cb22f1e46385050742d4acb27b18b9a5386",
+      "parent_rank": 2548
+    },
+    {
+      "coordinate_dim": 1024,
+      "in_features": 4096,
+      "input_sha256": "4b227c8d0cccd31fcdc6dbaef7f8cf7ca429addad9ecf69fc0fe1a79c1a48353",
+      "module": "model.layers.28.self_attn.q_proj",
+      "out_features": 4096,
+      "output_sha256": "1731cf15b82b3555977e407670d40eda6c78bfb2b132a23bbbbab73ec7de200d",
+      "parent_rank": 1638
+    },
+    {
+      "coordinate_dim": 409,
+      "in_features": 4096,
+      "input_sha256": "eb840656be85eb572eace3afc7cd13c6d0498529f734403fef40aff74a3c3493",
+      "module": "model.layers.28.self_attn.k_proj",
+      "out_features": 1024,
+      "output_sha256": "66934dc4cc80e3544d102ca1c73d9270efadf9f088b2df3365b3347ebed84400",
+      "parent_rank": 655
+    },
+    {
+      "coordinate_dim": 409,
+      "in_features": 4096,
+      "input_sha256": "4bb41d670dc567c748553f5760792fdbee1c96ad16c9c7b0bf7bef9ece71d0c5",
+      "module": "model.layers.28.self_attn.v_proj",
+      "out_features": 1024,
+      "output_sha256": "bca1c88b9751041745bd4acf62f62b30e904afd01f99663beec5928c28551677",
+      "parent_rank": 655
+    },
+    {
+      "coordinate_dim": 1024,
+      "in_features": 4096,
+      "input_sha256": "bb1c1888fdd0387f98e26dfaaf0d8e803827c789aca0b37e3209e4406c519cc4",
+      "module": "model.layers.28.self_attn.o_proj",
+      "out_features": 4096,
+      "output_sha256": "820534dcb202c6c048d9dc286cc290e91123837f295e941da04e009b249cee3e",
+      "parent_rank": 1638
+    },
+    {
+      "coordinate_dim": 1592,
+      "in_features": 4096,
+      "input_sha256": "f6714d3fbceefa4a156aceb522fa07508f725ee43e835c962c6505862d0be774",
+      "module": "model.layers.28.mlp.gate_proj",
+      "out_features": 14336,
+      "output_sha256": "b8346a33d282c5a854482a1bad4f025a74bc2eae9cc3a19f873437da897880f6",
+      "parent_rank": 2548
+    },
+    {
+      "coordinate_dim": 1592,
+      "in_features": 14336,
+      "input_sha256": "5bd7cdb6d57d5475c0d931817ed0d1dc3e1aac8189c92e1dc8b73920d7bd06e9",
+      "module": "model.layers.28.mlp.down_proj",
+      "out_features": 4096,
+      "output_sha256": "525336ca94b74e6aedb309f3d26b70f68c55e56f415062236532183ba7cc9aee",
+      "parent_rank": 2548
+    },
+    {
+      "coordinate_dim": 1592,
+      "in_features": 4096,
+      "input_sha256": "dcd26bad9ea250d69a8606b9e763e13328807e52aac3afa4945e49a9b9ab8acd",
+      "module": "model.layers.28.mlp.up_proj",
+      "out_features": 14336,
+      "output_sha256": "d707fdab52df3a7f7d8d7b5b8d776e291fad7afd89fca3950ff20fe531271b18",
+      "parent_rank": 2548
+    },
+    {
+      "coordinate_dim": 1024,
+      "in_features": 4096,
+      "input_sha256": "84a7700ebbdca39d76a4661be3e7e3553f99aec1d5624c8df29dd2cb71a2d2f5",
+      "module": "model.layers.29.self_attn.q_proj",
+      "out_features": 4096,
+      "output_sha256": "2d96564a61351a258c578f5c43cc2a38579b595315f0356a918c314708564101",
+      "parent_rank": 1638
+    },
+    {
+      "coordinate_dim": 409,
+      "in_features": 4096,
+      "input_sha256": "960aa1b91dcc467d652b8a3247a28c8af18d5e6a274ef759e216db78843fb0d7",
+      "module": "model.layers.29.self_attn.k_proj",
+      "out_features": 1024,
+      "output_sha256": "b3796e6b5b6186695d367fd64e57449086c81aee2c6e972e05608ab26a08a77a",
+      "parent_rank": 655
+    },
+    {
+      "coordinate_dim": 409,
+      "in_features": 4096,
+      "input_sha256": "74639fe83849f73fd4aefa1bef11300455f62508e962d2222a9051806bb3a66a",
+      "module": "model.layers.29.self_attn.v_proj",
+      "out_features": 1024,
+      "output_sha256": "0b6789baacb798b0fb9a7031cbac8917cba18e1077e7f513683b37b52b15a49e",
+      "parent_rank": 655
+    },
+    {
+      "coordinate_dim": 1024,
+      "in_features": 4096,
+      "input_sha256": "b873f88d65ddc0a0e3164f93ad9e09adde038052788a4a72dbf462336b10af75",
+      "module": "model.layers.29.self_attn.o_proj",
+      "out_features": 4096,
+      "output_sha256": "d670116faf6a308395ca664f9b89f487276fcbda61365c1f74af6d8c7fa5dfb0",
+      "parent_rank": 1638
+    },
+    {
+      "coordinate_dim": 1592,
+      "in_features": 4096,
+      "input_sha256": "1063efdf767c696f954500e6115e6b24fc4767b699d34f65f3d9adb0d383f5b5",
+      "module": "model.layers.29.mlp.gate_proj",
+      "out_features": 14336,
+      "output_sha256": "0c29798c2d7efe50c1e07481eb8c0050a5cf95e150fa2337976497382397ec3f",
+      "parent_rank": 2548
+    },
+    {
+      "coordinate_dim": 1592,
+      "in_features": 14336,
+      "input_sha256": "b3624d0ed0492404e3866bedae05208ccb92db76c025435559bb6d22a8ce9b13",
+      "module": "model.layers.29.mlp.down_proj",
+      "out_features": 4096,
+      "output_sha256": "0263c49911175948bd50218dc25756d7572ec2612f963a25ded4d059a731a78d",
+      "parent_rank": 2548
+    },
+    {
+      "coordinate_dim": 1592,
+      "in_features": 4096,
+      "input_sha256": "45f6e2799f35d8ee2c99a6fcec9bec0b6c0243d110b7439a16e50cfec07b7aff",
+      "module": "model.layers.29.mlp.up_proj",
+      "out_features": 14336,
+      "output_sha256": "813aac0a03dc38cc0b9ba1eaa17199698e086b8cfd242d37cc20cbd602390eb9",
+      "parent_rank": 2548
+    },
+    {
+      "coordinate_dim": 1024,
+      "in_features": 4096,
+      "input_sha256": "8a434a7a2c99380e93dfe50b7f9cf1a86741ad6cd9f36aa3558f5500724c8688",
+      "module": "model.layers.30.self_attn.q_proj",
+      "out_features": 4096,
+      "output_sha256": "01c27ddad2ee40947c32cbc25f3694a123ad944e53ca422bdbc812943aa9b78d",
+      "parent_rank": 1638
+    },
+    {
+      "coordinate_dim": 409,
+      "in_features": 4096,
+      "input_sha256": "dcec96e62a55ddeeea61f5519d6b2c805e711a881c87b8b2aca25609f2cb7ccf",
+      "module": "model.layers.30.self_attn.k_proj",
+      "out_features": 1024,
+      "output_sha256": "2f20af86d69bd359d141a862ede4893090f3d9e3cb687a89c551cc9c5ef70e1b",
+      "parent_rank": 655
+    },
+    {
+      "coordinate_dim": 409,
+      "in_features": 4096,
+      "input_sha256": "b8cbfbd47d4198607935a6c9cbe8d35d4cb805220e326cf83e8bc27e05a4b665",
+      "module": "model.layers.30.self_attn.v_proj",
+      "out_features": 1024,
+      "output_sha256": "ba5ff5cbe0502153697f5295862b4e062df2962b7af5e720cbc3de916aeee317",
+      "parent_rank": 655
+    },
+    {
+      "coordinate_dim": 1024,
+      "in_features": 4096,
+      "input_sha256": "279dbfa2bbd5bd610d431d3b60d5c69341910f6d2f567c9aded97ed59d246c47",
+      "module": "model.layers.30.self_attn.o_proj",
+      "out_features": 4096,
+      "output_sha256": "0c7bac80bbffac08be275835a0c1bcc689db1a104205e0ff9630370ba09e0b0f",
+      "parent_rank": 1638
+    },
+    {
+      "coordinate_dim": 1592,
+      "in_features": 4096,
+      "input_sha256": "8403339e2778313b4466d7f034334fff0e9f05fe3aba19d8c61203fbe0022e29",
+      "module": "model.layers.30.mlp.gate_proj",
+      "out_features": 14336,
+      "output_sha256": "a1875886117636ba486c19caff73cabf534c2344c7e43842c588fcaea780e6e8",
+      "parent_rank": 2548
+    },
+    {
+      "coordinate_dim": 1592,
+      "in_features": 14336,
+      "input_sha256": "fb5c3fe7ef811d337a7d9bdd306d990ea14d3874339d8d90f947dcd474c2bfaf",
+      "module": "model.layers.30.mlp.down_proj",
+      "out_features": 4096,
+      "output_sha256": "db12bcf4d91ad7972aef0fa9b823e5b8fa6846914de16cc97b443bfcc60590f9",
+      "parent_rank": 2548
+    },
+    {
+      "coordinate_dim": 1592,
+      "in_features": 4096,
+      "input_sha256": "f67c63ff2eb7670518891a4bdd1fa6d5da5be1d2c4c7f2ca0f04bb5ce6dd2635",
+      "module": "model.layers.30.mlp.up_proj",
+      "out_features": 14336,
+      "output_sha256": "c3b32a32e20c3897ea6dc7819981f86833b35e362d395814917a514cc56c112c",
+      "parent_rank": 2548
+    },
+    {
+      "coordinate_dim": 1024,
+      "in_features": 4096,
+      "input_sha256": "d91da6ed2f47d876d0f6ef6b7c901b052b2745d836058bf0f01ecc34f5686c80",
+      "module": "model.layers.31.self_attn.q_proj",
+      "out_features": 4096,
+      "output_sha256": "9400eda9a67f1b59806cc3844ed8ff586ef08624b7af3b50904d28e4aa0ddb1d",
+      "parent_rank": 1638
+    },
+    {
+      "coordinate_dim": 409,
+      "in_features": 4096,
+      "input_sha256": "07d02032069289b09406bcaa3731a9bf450d814c89dc7b05e30faefa9ef08e22",
+      "module": "model.layers.31.self_attn.k_proj",
+      "out_features": 1024,
+      "output_sha256": "8950edb02bdee1d51dca43a32b2380b419ee4ec13ba662942ff35c1f74df11d9",
+      "parent_rank": 655
+    },
+    {
+      "coordinate_dim": 409,
+      "in_features": 4096,
+      "input_sha256": "419fa44c1faea2d5b5f530da5eb2358846799d91069c56cd18e1b48d8cc53c1e",
+      "module": "model.layers.31.self_attn.v_proj",
+      "out_features": 1024,
+      "output_sha256": "a8a45745d2201bb39e536665c4f5023c899b47f3cfaa563d47370eb9d495320f",
+      "parent_rank": 655
+    },
+    {
+      "coordinate_dim": 1024,
+      "in_features": 4096,
+      "input_sha256": "2e0eb08a04bdd476f8244386e22d3eb9d3a041e72d4decd0d62e539d985ea229",
+      "module": "model.layers.31.self_attn.o_proj",
+      "out_features": 4096,
+      "output_sha256": "9c70f411a9954e03821d7bbf219435b75579f036622982d022e84009bbe6dbec",
+      "parent_rank": 1638
+    },
+    {
+      "coordinate_dim": 1592,
+      "in_features": 4096,
+      "input_sha256": "674efc264d679a42b18a0eb86514c4c4617e1269652a28c1d50a6a4f87f04144",
+      "module": "model.layers.31.mlp.gate_proj",
+      "out_features": 14336,
+      "output_sha256": "d980c8766562338f1b39ee4fc31e8b8db6ba6b83262719b69a2b0c92fc89bbb4",
+      "parent_rank": 2548
+    },
+    {
+      "coordinate_dim": 1592,
+      "in_features": 14336,
+      "input_sha256": "b2221194b0bbc182b02b41709bd117581282acbf45fbb03eb30bd766927f3988",
+      "module": "model.layers.31.mlp.down_proj",
+      "out_features": 4096,
+      "output_sha256": "e1bc014f5a2a714ff50c155e795f1958999ec63279bff8d7889b8cc46b8935b2",
+      "parent_rank": 2548
+    },
+    {
+      "coordinate_dim": 1592,
+      "in_features": 4096,
+      "input_sha256": "2763f4bfbbc651ffcc34b9b51f7c70f5fda443c63aae8d1adcde5907aceb5879",
+      "module": "model.layers.31.mlp.up_proj",
+      "out_features": 14336,
+      "output_sha256": "654c08deb0efe559870d41a4c6bc5b1a52d9445043e2c47ace0ec97b4cd19274",
+      "parent_rank": 2548
+    }
+  ],
+  "rank_rule": "floor(retention * out_dim * in_dim / (out_dim + in_dim)), clamped to [1,min(out,in)]",
+  "status": "complete",
+  "total_parameters": 4538830848,
+  "untouched_parameter_hashes": {
+    "lm_head.weight": "090f81d81da902a6c63671ee46b041d86144564ab564a921a4bd1c5a7a724715",
+    "model.embed_tokens.weight": "90c94d0d053e7e1bd5d93241a814dc124dff8fc9b173e9dd156d3616715d9a7f",
+    "model.layers.0.input_layernorm.weight": "c05f2ee40bc0a8f3e8e945f6ccd5c3269613a1d40b949769a83caa9053752401",
+    "model.layers.0.post_attention_layernorm.weight": "b1f4680ce6c11f8b918f2d7c3647d67b785ee4f638cd9f26f8d36c3a1cfc36f7",
+    "model.layers.1.input_layernorm.weight": "f80ce5698eee462a1813c01111d9ae2b2860db6f73a5a3207c823ba19e434b92",
+    "model.layers.1.post_attention_layernorm.weight": "3438ea45be01c075e312ac685a7ccb8bb3c4da2533df355ffb4066a062e82d3c",
+    "model.layers.10.input_layernorm.weight": "9ed9d4e290f104df080b4ea94c1ccf68b4a16d385c1834f7efba7175283bfe49",
+    "model.layers.10.post_attention_layernorm.weight": "857a68c64ce7162abb1662185e87e68bd18926568a91eba873fc38a2a3cf0a2b",
+    "model.layers.11.input_layernorm.weight": "b907f2e38259526600874e50ed79ef4002c9d38fd82ed5f81ca6bf0f68cbd7ab",
+    "model.layers.11.post_attention_layernorm.weight": "c9a3f16d69c43365ed03317f0756b633bd915b0c7ff357b23919efe4645b7541",
+    "model.layers.12.input_layernorm.weight": "7c2d68b953362134776634d0aa2a5784dfc7a566b2d62a77e0e244c54e241387",
+    "model.layers.12.post_attention_layernorm.weight": "8527af879cced9eac501eb119821bc641bb85973041feaf1db372a44f8f98591",
+    "model.layers.13.input_layernorm.weight": "247e6e47068b55cba967ef13237e39dd20546166ebd2c4c269eeb9ac2934ce0d",
+    "model.layers.13.post_attention_layernorm.weight": "e2086ff72bb3165f3b58e1ec0d439507729cd5adb57c30566b0612f797082497",
+    "model.layers.14.input_layernorm.weight": "9055f44ac5b8b7d2b836f3646769af0a4a3ee50c37ad9293e2a38bc9a391eebe",
+    "model.layers.14.post_attention_layernorm.weight": "e5ae6d0df5d1875de1ace359dae75dbb91c0e8af740c63f5bac0d9f26ff14849",
+    "model.layers.15.input_layernorm.weight": "2c73b0cadae00127b43e266760b2d96a68add6624e688b9fed4c4bf838f438d8",
+    "model.layers.15.post_attention_layernorm.weight": "4cd167dd855091ce9e68ed182f54a605ce5e571b49b44dc2a4d2fc12f039c596",
+    "model.layers.16.input_layernorm.weight": "cbf8c798b1e12f194fa5da53d9388bb3dd614f8d19c6e5e8201bdd3a1da4edc8",
+    "model.layers.16.post_attention_layernorm.weight": "e0aad8492cce8490b7d6510eea059629c70d5cf71cad137e4dc7bf143ee28e83",
+    "model.layers.17.input_layernorm.weight": "3f98949bf36485ad022d55d8ca4f7bc744b003ec256aac845abc7a1b2382975b",
+    "model.layers.17.post_attention_layernorm.weight": "686ea0e8928dc23cd54b997aeeb261353ba1c69a036c1e38f7c53f8287963702",
+    "model.layers.18.input_layernorm.weight": "46732e00ab7f456a8db4437e95852d7905461fb63ac3516a3718fc2a685baa69",
+    "model.layers.18.post_attention_layernorm.weight": "11652fd11eec6c6b499be49b96dde4a0f0dfe2fd065edb375a44485ba03755d5",
+    "model.layers.19.input_layernorm.weight": "18a8aa238934a281f6b611f2f616e1e4b7bbb08af5cd3e95021896ff9d9f33e7",
+    "model.layers.19.post_attention_layernorm.weight": "a7b3fc24b4b0fc5a76e437c21532c77abc0bcf90543ede02d0bd36b1ea5c86c8",
+    "model.layers.2.input_layernorm.weight": "0bcc1aacf6ad43d3247e3e4da4c9f5d952b469d97fd380751b17bd5ca54a0d0c",
+    "model.layers.2.post_attention_layernorm.weight": "25a12dd2949f30109a2d0cfecedf77876d13156a0476075be13e1485abc79080",
+    "model.layers.20.input_layernorm.weight": "3b9835db6383dc1927bdea2bd37336a7314f4aae576563321e7e89133d4d5d95",
+    "model.layers.20.post_attention_layernorm.weight": "94848c536b89391dfea39e2936d2e09110aa7f02e1446c6dbd109d1f4bd655ba",
+    "model.layers.21.input_layernorm.weight": "49fb5588e289e22b47f8ef79470b78fe56ae4f399931e8a3b052d9c4b92b5609",
+    "model.layers.21.post_attention_layernorm.weight": "51e9bfa27b2f3d01c88b259665c3b54f63f2a988dbc04bc69a876305a80d5275",
+    "model.layers.22.input_layernorm.weight": "98581d697aa91ba4db68f6b441f05115da659b29fd5abf4778287d756a8e547c",
+    "model.layers.22.post_attention_layernorm.weight": "6c54b14b3b9ae7c256286d798d32bfe5192c2cf66cd5ddf7c4d9196769b75767",
+    "model.layers.23.input_layernorm.weight": "39baf05eda82d44023dda7b292b96ce0d267c2942384a35da810d0b45e3f8ece",
+    "model.layers.23.post_attention_layernorm.weight": "80a5b6a5c4e10b7b60694767267e46d05423d29de3aec81f65cff5d33c4150b0",
+    "model.layers.24.input_layernorm.weight": "1730239df84c3eb90163456559e62fd27a17c7197d4bb1bdb8b9792b8ef9da01",
+    "model.layers.24.post_attention_layernorm.weight": "97b30e15c23e1bfbbe1113c990cd357c4f3cd1af2facdee2fa8dd502461fa39c",
+    "model.layers.25.input_layernorm.weight": "85b8b0d79d5d0ae06b14700462bedfd91d85b47197734f65280c7d89d60e7411",
+    "model.layers.25.post_attention_layernorm.weight": "8094d53e9bf313bc9af96948c0c3d4983a8e376d70ea046c49eb8debf5819f13",
+    "model.layers.26.input_layernorm.weight": "ec9a244f57549da5e7c9c20830c9ff34275366063982688b6ff9e0499f40bc97",
+    "model.layers.26.post_attention_layernorm.weight": "78fead5be855e383d0d328e1402c08edc29c690f28d493d5ace0c088e185bbf0",
+    "model.layers.27.input_layernorm.weight": "35947ece34b898e5f7aef31b9f3ca0bae8ebef97ef4854ea6f61eeb8d76ca169",
+    "model.layers.27.post_attention_layernorm.weight": "cdf70e270060f657262f6ee56802ff3b5ac80df65d0b5013c1bba8bd7d224d7a",
+    "model.layers.28.input_layernorm.weight": "a7aca29f063ae1b8399bec997849fe8b56a49b7093abe7ed0664223a084eb8a3",
+    "model.layers.28.post_attention_layernorm.weight": "885f5227b6c477d5a855cc367fff3f35a374875e9e862ecf5debf5e408f233f6",
+    "model.layers.29.input_layernorm.weight": "cf6c12e4dd5d7e62187f616aab3a791aa05ed11887a491549ca4301932e5686e",
+    "model.layers.29.post_attention_layernorm.weight": "1471c1852fddd1b47022edf069267729215f68a1ffdba5407ef53c698b060479",
+    "model.layers.3.input_layernorm.weight": "014a01ef97e08598bc4d418b6d91142fa147ba364a7c34097768a59612ddf9f3",
+    "model.layers.3.post_attention_layernorm.weight": "b0bbb863584b84f839ce363eabc0788f44537766c49f9e657607d9be73665860",
+    "model.layers.30.input_layernorm.weight": "4b43c7cedefba4f34128aa04c973faeb4d6a9c7dc8459d3c56d473655fbf85f3",
+    "model.layers.30.post_attention_layernorm.weight": "b5d9a45ace2c39baa3bfadf5aea432cc0b7cbaf20c9cd574aa2c963cbd7643dc",
+    "model.layers.31.input_layernorm.weight": "5fdca8727b1e9f8145b6f700b391597ae73bb1bc1597c0fface5967b088fa1a3",
+    "model.layers.31.post_attention_layernorm.weight": "bfe99942b17344a53feb4595377e51d65708d6770bc0032549975dbbaa065537",
+    "model.layers.4.input_layernorm.weight": "471f4ded27364a38d17ab475f6df918c65752cb9d84c281b132f327c205d602b",
+    "model.layers.4.post_attention_layernorm.weight": "a0083238ae889685e7c46fda17b3859d9227b024bd3de1e0bf78e7bc96908085",
+    "model.layers.5.input_layernorm.weight": "7337480de35aadbed131d8cab06fd4e110a5dc1c5d3179ed31183a392f95aa6f",
+    "model.layers.5.post_attention_layernorm.weight": "e783b5f1b016bec0cca6160de2cef167a582484ab1066297b51343bbe9fa8748",
+    "model.layers.6.input_layernorm.weight": "dcb860363b79ce1d42367bfb99e440bfd6e09d012766db6d9786d1735a7dfdef",
+    "model.layers.6.post_attention_layernorm.weight": "a6b4b4a23071b41c4d09f29d1bf4ab20ea66c5ef84c89ca42f1f8d7d28514a34",
+    "model.layers.7.input_layernorm.weight": "58d356455c98390f802ea1c778ee07567addb75fdcbf6c4f870c2d31f258a3a5",
+    "model.layers.7.post_attention_layernorm.weight": "b28e9a2e802e83c72c8150954d272c6e8a252c811ea09bf08edceaf70e5b61f0",
+    "model.layers.8.input_layernorm.weight": "0fe756da619a3db75adbcc244c77794c26e3ec0053735399adb1bd6797608258",
+    "model.layers.8.post_attention_layernorm.weight": "9f142790d0dce0b4d860a7428558eab7281bd930b32d5bca9216812aec0c0520",
+    "model.layers.9.input_layernorm.weight": "2785c6f885cfc3b3fc7507421a4b96a3b0e4319079a2ef3be536bc774b87f49a",
+    "model.layers.9.post_attention_layernorm.weight": "4c5a6b91069f27e066212b15ae40d1e5ba146160eb8cf702f8db42b4dfc0113a",
+    "model.norm.weight": "49d4c792beddcb32ea782e0633b0bc1be6bd2979410c903fdf5300ddbc4bf55e"
+  }
+}

basis/sources/llama31/gfwsvd_r050/source.json

@@ -0,0 +1,21 @@
+{
+  "backbone": "llama31",
+  "checkpoint_identity": {
+    "checkpoint_sha256": "1c1f65a0d57817c76d64b00466be8aa533d22673d8aaf567d7960d138499d266"
+  },
+  "id": "llama31/gfwsvd_r050",
+  "kind": null,
+  "method": "gfwsvd",
+  "note": "New Task 05 Source from ordered frozen r=0.8 paper factors; same calibration, new checkpoint identity. Evaluate new pressure/recovery results.",
+  "path": "basis/sources/llama31/gfwsvd_r050/model.pt",
+  "provenance_files": [
+    "basis/sources/llama31/gfwsvd_r050/provenance/manifest.json"
+  ],
+  "retention": 0.5,
+  "tasks": [
+    "05"
+  ],
+  "original_path": "/home/zs89/Basis/outputs/source_delivery_20260917/sources/llama31/gfwsvd_r050/model.pt",
+  "validation": "basis/sources/llama31/gfwsvd_r050/validation.json",
+  "weight_repository": "Duke-CEI-SVD/FlashSVD"
+}

basis/sources/llama31/gfwsvd_r050/validation.json

@@ -0,0 +1,17 @@
+{
+  "actual_projection_retention": 0.49974646935096156,
+  "buffers_verified": 1,
+  "calibration_tensor_sha256": "95ca6fc95ac62db49af1d7c4e5b4d7d5e100c29e82376929268bfb14987ad82b",
+  "checkpoint_identity": {
+    "checkpoint_sha256": "1c1f65a0d57817c76d64b00466be8aa533d22673d8aaf567d7960d138499d266"
+  },
+  "elapsed_seconds": 92.98302385210991,
+  "factor_pairs": 224,
+  "finite_forward": true,
+  "parameters_verified": 515,
+  "predicted_adapter_parameters": 10759936,
+  "source_id": "llama31/gfwsvd_r050",
+  "source_json": "/home/zs89/Basis/outputs/source_delivery_20260917/sources/llama31/gfwsvd_r050/source.json",
+  "status": "passed",
+  "suggested_adapter_rank": 22
+}

basis/sources/llama31/gfwsvd_r070/model.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b43045553b40a499d73f7460fc395622c0b2c737cc81555f8febe995a115f913
+size 11872204875

basis/sources/llama31/gfwsvd_r070/provenance/manifest.json

@@ -0,0 +1,2149 @@
+{
+  "actual_projection_retention": 0.6999230018028846,
+  "buffer_hashes": {
+    "model.rotary_emb.inv_freq": "54e45984c6ed0110ac11befb802715adece9ed7cd0e01b44520367e80c2b1e6d"
+  },
+  "build_protocol": "basis_nested_paper_factors_v1",
+  "calibration_provenance": "Inherited exactly; no new calibration or recovery",
+  "calibration_shape": [
+    512,
+    2048
+  ],
+  "calibration_tensor_sha256": "95ca6fc95ac62db49af1d7c4e5b4d7d5e100c29e82376929268bfb14987ad82b",
+  "constructor": "ordered leading-factor truncation of the frozen r=0.8 paper Source",
+  "dense_projection_parameters": 6979321856,
+  "exporter_sha256": "095e17a71ddbdfe32c4976e562041b61197064891a40cdf1da830638b824dfdd",
+  "factor_parameters": 4884987904,
+  "keep_ratio": 0.7,
+  "method": "gfwsvd",
+  "output_sha256": "b43045553b40a499d73f7460fc395622c0b2c737cc81555f8febe995a115f913",
+  "parent": {
+    "checkpoint_identity": {
+      "checkpoint_sha256": "eac75ad9381d7742a96a1d088f607c14a44d0fb8973cf5281c86514f7828718d"
+    },
+    "kind": null,
+    "path": "/tmp/basis-input-audit/source_upload/basis/sources/llama31/gfwsvd_r080/model.pt",
+    "source_id": "llama31/gfwsvd_r080",
+    "source_json_sha256": "f8374c009efde1189b82245a9245699222dbaab147a7240001f2d9dc728d99e4"
+  },
+  "parent_manifest": {
+    "base_model": "meta-llama/Llama-3.1-8B",
+    "calibration_provenance": "exact_locked_tensor_verified",
+    "calibration_shape": [
+      512,
+      2048
+    ],
+    "calibration_tensor_sha256": "95ca6fc95ac62db49af1d7c4e5b4d7d5e100c29e82376929268bfb14987ad82b",
+    "constructor": "two-sided diagonal-KFAC weighted SVD",
+    "factor_parameters": 5582028800,
+    "keep_ratio": 0.8,
+    "method": "GFWSVD",
+    "metric_scope": "diagonal A and B Fisher/KFAC approximation",
+    "module_count": 224,
+    "output": "/home/zs89/FlashSVDTrain/diagnosis/spectral_fingerprint/confirmatory/artifacts/sources/gfwsvd_shared512/model.pt",
+    "output_bytes": 13275277923,
+    "output_sha256": "eac75ad9381d7742a96a1d088f607c14a44d0fb8973cf5281c86514f7828718d",
+    "rank_profile": {
+      "down_proj": 2548,
+      "gate_proj": 2548,
+      "k_proj": 655,
+      "o_proj": 1638,
+      "q_proj": 1638,
+      "up_proj": 2548,
+      "v_proj": 655
+    },
+    "rank_rule": "floor(keep_ratio * out_dim * in_dim / (out_dim + in_dim))",
+    "status": "complete"
+  },
+  "parent_manifest_sha256": "6558e358f1a8000d5a2c0f59ab8e030d91c0fcdd26a5b6ae7c069201d15c083b",
+  "rank_records": [
+    {
+      "coordinate_dim": 1433,
+      "in_features": 4096,
+      "input_sha256": "4daab431ff2d35d64566b0c5b51f8937983761845dbed0737f83b970830ac991",
+      "module": "model.layers.0.self_attn.q_proj",
+      "out_features": 4096,
+      "output_sha256": "03586f7d8f47156e6320f205cacedee66c03bbb0325e25046470aceb07041d78",
+      "parent_rank": 1638
+    },
+    {
+      "coordinate_dim": 573,
+      "in_features": 4096,
+      "input_sha256": "584066b18b1c9fde66ef6dacfc04611eae9759208ce241aaf9c283fb89f30bb8",
+      "module": "model.layers.0.self_attn.k_proj",
+      "out_features": 1024,
+      "output_sha256": "ef425e75b2513607d378d0e2cfa9ab751bc7de639cad73a700a8d08fab8d8ffb",
+      "parent_rank": 655
+    },
+    {
+      "coordinate_dim": 573,
+      "in_features": 4096,
+      "input_sha256": "34bec74989a3ffecb3846c501ef46eb209dfc6dcc1ee703355c45b2c5e961f36",
+      "module": "model.layers.0.self_attn.v_proj",
+      "out_features": 1024,
+      "output_sha256": "09ba39bdb952dcdd851f5a1ea201e8ebdb1550ed09baeeccf0da28db11b8ecea",
+      "parent_rank": 655
+    },
+    {
+      "coordinate_dim": 1433,
+      "in_features": 4096,
+      "input_sha256": "285d86a1b28053dd6ba08cdb5fe3633a7a8fb0ddc56bf07c86f0e7b0d709917d",
+      "module": "model.layers.0.self_attn.o_proj",
+      "out_features": 4096,
+      "output_sha256": "afc85a93c39f5206ab01782cf1b7f33816a2c887d31b8445e9414a2678ed58a8",
+      "parent_rank": 1638
+    },
+    {
+      "coordinate_dim": 2230,
+      "in_features": 4096,
+      "input_sha256": "5ab487c3ba28cf6d7ec5ff799bf9c2e7773f485a60632c951e6f8fe963366313",
+      "module": "model.layers.0.mlp.gate_proj",
+      "out_features": 14336,
+      "output_sha256": "173ddd4360a3f647aa7ba6cd5ed5ab48d34db5c0a7dc584bb223ddc5369c5f68",
+      "parent_rank": 2548
+    },
+    {
+      "coordinate_dim": 2230,
+      "in_features": 14336,
+      "input_sha256": "fbce11d1f6e5c5355190df797fd5020dab76e840ec2bfc160f9c65712f3b1cb0",
+      "module": "model.layers.0.mlp.down_proj",
+      "out_features": 4096,
+      "output_sha256": "7946614c904363b875e73e792aa96c8ef72c41c754228a8c5e9c502110850bb1",
+      "parent_rank": 2548
+    },
+    {
+      "coordinate_dim": 2230,
+      "in_features": 4096,
+      "input_sha256": "d297e81e74206b02888d9cdd26954ecf697384015d6e5a0eb0c1594bf8485afd",
+      "module": "model.layers.0.mlp.up_proj",
+      "out_features": 14336,
+      "output_sha256": "862276105b5dd5b0f35538b32e2345ab83cb7873cf23cdcda1906922aea2beb7",
+      "parent_rank": 2548
+    },
+    {
+      "coordinate_dim": 1433,
+      "in_features": 4096,
+      "input_sha256": "efe4c7fbcc72fcbb1c54d4144864cd21b0ac48cbceddcd4e471788a32bcbcb7b",
+      "module": "model.layers.1.self_attn.q_proj",
+      "out_features": 4096,
+      "output_sha256": "d08f541b555acf2c55d47e65ad89c51d0f867c7f0e47e94f159a18fea7febb2f",
+      "parent_rank": 1638
+    },
+    {
+      "coordinate_dim": 573,
+      "in_features": 4096,
+      "input_sha256": "ffa355783631fd3d39820e7af63296eb704ea59a5b1f3d0a6596aa64f983ce98",
+      "module": "model.layers.1.self_attn.k_proj",
+      "out_features": 1024,
+      "output_sha256": "8fe3982731757e4d583954b3945bdc6f59c3155f332f7fdcb21607cda450b7ae",
+      "parent_rank": 655
+    },
+    {
+      "coordinate_dim": 573,
+      "in_features": 4096,
+      "input_sha256": "c845b56d65d99ef23d19a5ea10c9183a61910987f2c38e2c9cb7d1dd96312f6b",
+      "module": "model.layers.1.self_attn.v_proj",
+      "out_features": 1024,
+      "output_sha256": "45abebcfa3e2dd00c683e0690fda2be4ceb60581ff444a336c0accaddf9b3719",
+      "parent_rank": 655
+    },
+    {
+      "coordinate_dim": 1433,
+      "in_features": 4096,
+      "input_sha256": "b673681b57f5373c38c854427ab5b3111a0ff50e7400a4fb0abe403071ea533f",
+      "module": "model.layers.1.self_attn.o_proj",
+      "out_features": 4096,
+      "output_sha256": "c408290755d1eaae36d9f21c7c0ecaf3410b115d7d53e3d29e623e0c1ee053e7",
+      "parent_rank": 1638
+    },
+    {
+      "coordinate_dim": 2230,
+      "in_features": 4096,
+      "input_sha256": "40ae353494fc059f1da91c4f795b3e750ff8ae192c834c2ebd3a36cf4d38b541",
+      "module": "model.layers.1.mlp.gate_proj",
+      "out_features": 14336,
+      "output_sha256": "22caa80667c8db0742c388ea5f5ede9041e657a30317205fb25a8fbede21ebdc",
+      "parent_rank": 2548
+    },
+    {
+      "coordinate_dim": 2230,
+      "in_features": 14336,
+      "input_sha256": "a1fdc3ee194f2c2a77b43a55e68a4a43534fd671b520ac11c51135904bd0168c",
+      "module": "model.layers.1.mlp.down_proj",
+      "out_features": 4096,
+      "output_sha256": "c1981dffa3a8cde7a0e6540dd48179fabe5b651e085d670c8d606bee422dbaea",
+      "parent_rank": 2548
+    },
+    {
+      "coordinate_dim": 2230,
+      "in_features": 4096,
+      "input_sha256": "116eb4c22592027fb186fe2781abef8f8a90d06a2e8f358e8751790439b03cb5",
+      "module": "model.layers.1.mlp.up_proj",
+      "out_features": 14336,
+      "output_sha256": "5135d961ec43de1570acd0e57973d5ebf7d379724d9b6bbc3b8df439a20f715e",
+      "parent_rank": 2548
+    },
+    {
+      "coordinate_dim": 1433,
+      "in_features": 4096,
+      "input_sha256": "12dbb059c0098c1b83a81d23032f1a5dc61a69ed6662f07353652da494212e90",
+      "module": "model.layers.2.self_attn.q_proj",
+      "out_features": 4096,
+      "output_sha256": "bec1de41e45dbbce266bca59e937037a59b245809d99e38195a28cacab0ac556",
+      "parent_rank": 1638
+    },
+    {
+      "coordinate_dim": 573,
+      "in_features": 4096,
+      "input_sha256": "d20d344cf4f2bb70577f10d2a730a8fe2cf6b4c3a831859bd40c1a7ef161a9be",
+      "module": "model.layers.2.self_attn.k_proj",
+      "out_features": 1024,
+      "output_sha256": "24ec0b50e06fecfe02a444c782c899baa998403fb903c729b3cb1b94aacf15dd",
+      "parent_rank": 655
+    },
+    {
+      "coordinate_dim": 573,
+      "in_features": 4096,
+      "input_sha256": "e76a209a02d037fde652c2d3025847936a2bd4d77ed812bf69a456d018cd4cf3",
+      "module": "model.layers.2.self_attn.v_proj",
+      "out_features": 1024,
+      "output_sha256": "09f204ce53375a1ec0c60a90cadf148f87ab5b9038812b41263c5c4ad32877ad",
+      "parent_rank": 655
+    },
+    {
+      "coordinate_dim": 1433,
+      "in_features": 4096,
+      "input_sha256": "0202ed298d047bea0f518bba0282b5c0bbf3679a7ad42dce6f5bb73c1536c2fa",
+      "module": "model.layers.2.self_attn.o_proj",
+      "out_features": 4096,
+      "output_sha256": "6e23783f7d360201f07aecd8b3a8b9b2ce91f82e363f57bd21b7ebf028be1769",
+      "parent_rank": 1638
+    },
+    {
+      "coordinate_dim": 2230,
+      "in_features": 4096,
+      "input_sha256": "12d585e9c507c4705aeb2b6628c6c19a5ac24148afcf3473b9b4cb7eda569410",
+      "module": "model.layers.2.mlp.gate_proj",
+      "out_features": 14336,
+      "output_sha256": "4d6f1a2d7afe31cc8605fbbea47a364bff3379108537a7bd4fa08ddf72c6360e",
+      "parent_rank": 2548
+    },
+    {
+      "coordinate_dim": 2230,
+      "in_features": 14336,
+      "input_sha256": "d333fb87f55a9954a94371c6c363edea7635d1267b6c88e12495aba8a71513b9",
+      "module": "model.layers.2.mlp.down_proj",
+      "out_features": 4096,
+      "output_sha256": "3c132fa56b7ca8d0078e0f1547c295e5e681fd3782f18eeb0a3cf753a722b828",
+      "parent_rank": 2548
+    },
+    {
+      "coordinate_dim": 2230,
+      "in_features": 4096,
+      "input_sha256": "9effde835d4dfa037da28d0254a3e9da7b72268f5e82b8533bd19960c369cf2d",
+      "module": "model.layers.2.mlp.up_proj",
+      "out_features": 14336,
+      "output_sha256": "eba750f1ab30f8b9af37f1afc5dafaec2d71ba64c94a6b62fd3d7692769c2f6c",
+      "parent_rank": 2548
+    },
+    {
+      "coordinate_dim": 1433,
+      "in_features": 4096,
+      "input_sha256": "a5fc961c373463159519cd5e036cffe3c1c6b0e559f0d0eb3c338df7a7c5c517",
+      "module": "model.layers.3.self_attn.q_proj",
+      "out_features": 4096,
+      "output_sha256": "8646322ecc4dca43d8bf63127ee7c25797af13b8a1980a5c8a32ab16ef6806da",
+      "parent_rank": 1638
+    },
+    {
+      "coordinate_dim": 573,
+      "in_features": 4096,
+      "input_sha256": "8ead2e328b10331775fd2dab9d2f393a39a8f945db80eb30ded528f8d91c98af",
+      "module": "model.layers.3.self_attn.k_proj",
+      "out_features": 1024,
+      "output_sha256": "8939c0962c221eed61e55f5729a1460c044a348a995f01d1a6a67ee6247992ea",
+      "parent_rank": 655
+    },
+    {
+      "coordinate_dim": 573,
+      "in_features": 4096,
+      "input_sha256": "bf5915c43529e095d47adcb787dc9281283b381b77d1754cacf0f09047d531db",
+      "module": "model.layers.3.self_attn.v_proj",
+      "out_features": 1024,
+      "output_sha256": "561167c5cac8a5acbf1b09b6114ec0a287dfc90a55b77a2f1020398f1e1a2851",
+      "parent_rank": 655
+    },
+    {
+      "coordinate_dim": 1433,
+      "in_features": 4096,
+      "input_sha256": "cbbb6d74bfbadca9156452032cec9add028d51da83394e65be126647e2d45be2",
+      "module": "model.layers.3.self_attn.o_proj",
+      "out_features": 4096,
+      "output_sha256": "9b606aec781e693e8a1bb898d74905df99a9694b2b7a44ae5964854cee35d555",
+      "parent_rank": 1638
+    },
+    {
+      "coordinate_dim": 2230,
+      "in_features": 4096,
+      "input_sha256": "3fcd0621b8f6af552b8be2c6439165fc5c17a73dc57a8bcc114940a35673a395",
+      "module": "model.layers.3.mlp.gate_proj",
+      "out_features": 14336,
+      "output_sha256": "2165a9dc178b25afa29c26eed301c55a5e0f551c29dab4a143a76a6888b05c47",
+      "parent_rank": 2548
+    },
+    {
+      "coordinate_dim": 2230,
+      "in_features": 14336,
+      "input_sha256": "03ac30edead241fe4db1768e8d9ecbdb42b73c52fadc69f3fc139a7a54e7e678",
+      "module": "model.layers.3.mlp.down_proj",
+      "out_features": 4096,
+      "output_sha256": "005adca85d25827be95f1cbea1fab23f97f089666748c860b49e010c5a332338",
+      "parent_rank": 2548
+    },
+    {
+      "coordinate_dim": 2230,
+      "in_features": 4096,
+      "input_sha256": "524c556cd1a6c35e99c4f6c8ed4d072b61fd2536e0b2c18a020c760f8a4b247a",
+      "module": "model.layers.3.mlp.up_proj",
+      "out_features": 14336,
+      "output_sha256": "08b67116ab270f933c1bddbeba7e81e15a3ea68181d58d6b16a0dbbf0dc535b5",
+      "parent_rank": 2548
+    },
+    {
+      "coordinate_dim": 1433,
+      "in_features": 4096,
+      "input_sha256": "9c669c496f960bcb73a4334ea30cea379b0f5d7e10fc73e1867482eb655eafa5",
+      "module": "model.layers.4.self_attn.q_proj",
+      "out_features": 4096,
+      "output_sha256": "bfc0ddaaa918a008a00e50fe95c3d7a18a430ab6114ca186cceb941666d12a5d",
+      "parent_rank": 1638
+    },
+    {
+      "coordinate_dim": 573,
+      "in_features": 4096,
+      "input_sha256": "48be9b5faf5c24728868d6a9081169263c02ff901e5590fe8734f7ea995c05f2",
+      "module": "model.layers.4.self_attn.k_proj",
+      "out_features": 1024,
+      "output_sha256": "01eb24085d6a27a5722b5b8f0257769e2d77c9f7c170720c7b56c0b07caa4f48",
+      "parent_rank": 655
+    },
+    {
+      "coordinate_dim": 573,
+      "in_features": 4096,
+      "input_sha256": "3ff39bead826c48691bdb753ac57aef8833f3be3e0379bbae94b0b29a6a7ff40",
+      "module": "model.layers.4.self_attn.v_proj",
+      "out_features": 1024,
+      "output_sha256": "5328d56617e17439abfe467e9698f9c032a3d662be0540a7f2a0bbd29226c77c",
+      "parent_rank": 655
+    },
+    {
+      "coordinate_dim": 1433,
+      "in_features": 4096,
+      "input_sha256": "90444a88e0587d8c589bb94ef131a01d4491e4e09fb6424d2983828430acfdba",
+      "module": "model.layers.4.self_attn.o_proj",
+      "out_features": 4096,
+      "output_sha256": "9b85fa6e86949bee59a29bb186e5c17f41050493e71c994301a1ccc598f84529",
+      "parent_rank": 1638
+    },
+    {
+      "coordinate_dim": 2230,
+      "in_features": 4096,
+      "input_sha256": "d5a5d195fcebb41aaba8b8b0a50fe1e9eaa812de2c8a677c38bcd531ae281d75",
+      "module": "model.layers.4.mlp.gate_proj",
+      "out_features": 14336,
+      "output_sha256": "6a29b5fdba0cb9d25dc924eaada5daa1dce1bdeebe50351b087b05499ec80cfd",
+      "parent_rank": 2548
+    },
+    {
+      "coordinate_dim": 2230,
+      "in_features": 14336,
+      "input_sha256": "f23ad0e2c81cd2d1cded4e07dfa998128ae87cd00f68575fbf8852840d484832",
+      "module": "model.layers.4.mlp.down_proj",
+      "out_features": 4096,
+      "output_sha256": "0cec292952087884ed7166c14039daff48b2e797855f11c711abf17e40be39f7",
+      "parent_rank": 2548
+    },
+    {
+      "coordinate_dim": 2230,
+      "in_features": 4096,
+      "input_sha256": "1435707297ce22d9759c4c02ddba94186c37bfc0cd46d16abad8dc2947a704de",
+      "module": "model.layers.4.mlp.up_proj",
+      "out_features": 14336,
+      "output_sha256": "47a993a070c80e3648844082211aba3e512a06ac48765dead201b61b0ffa2612",
+      "parent_rank": 2548
+    },
+    {
+      "coordinate_dim": 1433,
+      "in_features": 4096,
+      "input_sha256": "18d997e5f7e09a048a98f7f8b2a8d5504b11201d1e3b540139d73e0a82d96bb9",
+      "module": "model.layers.5.self_attn.q_proj",
+      "out_features": 4096,
+      "output_sha256": "acaccb24c61db8e73eb24a27e33acd3c48a58e84fd80b39cad5360a5745f50fc",
+      "parent_rank": 1638
+    },
+    {
+      "coordinate_dim": 573,
+      "in_features": 4096,
+      "input_sha256": "c003f361607a05d4b0cb27fde05aceb6ff6b50a83063c866f7679ede2b3280be",
+      "module": "model.layers.5.self_attn.k_proj",
+      "out_features": 1024,
+      "output_sha256": "39d6f32eee31b8c425c15019b752008194761a56f7438ac924d25809d0b3324d",
+      "parent_rank": 655
+    },
+    {
+      "coordinate_dim": 573,
+      "in_features": 4096,
+      "input_sha256": "dbbc026ba7910af97a9a2cfc4068a2c25c04dccca8e9e30b1bde130083dbae35",
+      "module": "model.layers.5.self_attn.v_proj",
+      "out_features": 1024,
+      "output_sha256": "f36d3a192d9e2114bf324c9295e9bcba71cf6e460338d8372bbe5c058ce6681f",
+      "parent_rank": 655
+    },
+    {
+      "coordinate_dim": 1433,
+      "in_features": 4096,
+      "input_sha256": "78970293dc2e5baaf89528de972d65f7189dba3d9f5d622750216e41c3c0aa39",
+      "module": "model.layers.5.self_attn.o_proj",
+      "out_features": 4096,
+      "output_sha256": "5a40e0c4f9ff5f9c2167013544ea473d16ed78a82d2dd91dc99809d4f0c68e60",
+      "parent_rank": 1638
+    },
+    {
+      "coordinate_dim": 2230,
+      "in_features": 4096,
+      "input_sha256": "810f41e12e60ba167826e16162a56c5b0849ae50ca766041ae9998983d723bde",
+      "module": "model.layers.5.mlp.gate_proj",
+      "out_features": 14336,
+      "output_sha256": "a725e8bdd0a8725675458ba5fbcffd0a47bef29fea3ac2875c0b6ccaad5d86f4",
+      "parent_rank": 2548
+    },
+    {
+      "coordinate_dim": 2230,
+      "in_features": 14336,
+      "input_sha256": "2f9800142114f8d5e8167401f7da78f842718c0ee3942d4b15210f1fa80ab3f2",
+      "module": "model.layers.5.mlp.down_proj",
+      "out_features": 4096,
+      "output_sha256": "2e85bd50bf4c3fd540c1031bd83d63a1a7a1aced3565b07e8859985a53aa9315",
+      "parent_rank": 2548
+    },
+    {
+      "coordinate_dim": 2230,
+      "in_features": 4096,
+      "input_sha256": "bcc0ca4be97f3895836e84425b7cfe4c0a8d963376d0be3c8094fa94346ca1ef",
+      "module": "model.layers.5.mlp.up_proj",
+      "out_features": 14336,
+      "output_sha256": "040acb7fb6ee43f0c2d5e9b3cbbd262d5cad9c204bc00e9762bc68bc012b133e",
+      "parent_rank": 2548
+    },
+    {
+      "coordinate_dim": 1433,
+      "in_features": 4096,
+      "input_sha256": "39e770d6ad49feddb5966a0fff32753d17ad949f95147d197e21248e96022019",
+      "module": "model.layers.6.self_attn.q_proj",
+      "out_features": 4096,
+      "output_sha256": "8abdb25e33036305c80cd648c240b037dace92529d02562d49bb65e40cd84d9b",
+      "parent_rank": 1638
+    },
+    {
+      "coordinate_dim": 573,
+      "in_features": 4096,
+      "input_sha256": "7e2a9d8d311cb18ca180a2c8a104ebb2b15067676aa2927d8f1a79a7e03a601e",
+      "module": "model.layers.6.self_attn.k_proj",
+      "out_features": 1024,
+      "output_sha256": "2643d9e6b2399708822f69fd21e8552581835b19cc73339ff586b2a6d060d6b2",
+      "parent_rank": 655
+    },
+    {
+      "coordinate_dim": 573,
+      "in_features": 4096,
+      "input_sha256": "1df4737a522e55ea346f8052e48caa5dc0fe4fc43d55a663b357c23015bcebd1",
+      "module": "model.layers.6.self_attn.v_proj",
+      "out_features": 1024,
+      "output_sha256": "b44266214897543128961a97a5408538c193afd50161f2ac184eb963539cbb90",
+      "parent_rank": 655
+    },
+    {
+      "coordinate_dim": 1433,
+      "in_features": 4096,
+      "input_sha256": "b25ac1e924a96ce34c0f9a858f772b0c7d083927b98956be9ad9058bd720c7ae",
+      "module": "model.layers.6.self_attn.o_proj",
+      "out_features": 4096,
+      "output_sha256": "62a856d2510d45e5751979e0adbb1956d7c5ecc4bf8b85423ae43e022ead58bc",
+      "parent_rank": 1638
+    },
+    {
+      "coordinate_dim": 2230,
+      "in_features": 4096,
+      "input_sha256": "8f3683a096fcd4adbe87a5bd08d2be809be36ffb392426f83d0f654864539091",
+      "module": "model.layers.6.mlp.gate_proj",
+      "out_features": 14336,
+      "output_sha256": "dcff05e2b123f07fde7317aed21676d0c7d579c1e29efb395b163f1aaa77e03c",
+      "parent_rank": 2548
+    },
+    {
+      "coordinate_dim": 2230,
+      "in_features": 14336,
+      "input_sha256": "65c50089a8529baaf99d090cfec2a5086935bf4bdb2cbdd365d5b708709f922a",
+      "module": "model.layers.6.mlp.down_proj",
+      "out_features": 4096,
+      "output_sha256": "24af644df3cc54a8c5d8fc5d1e4b3d1cfe9f3f51f54e60eb167989da675e4261",
+      "parent_rank": 2548
+    },
+    {
+      "coordinate_dim": 2230,
+      "in_features": 4096,
+      "input_sha256": "72ad9928af6fe7e874d523fc0b13bb7c1d6f91ce7e51c8a2ad267db6f42c2f4c",
+      "module": "model.layers.6.mlp.up_proj",
+      "out_features": 14336,
+      "output_sha256": "3d3be56bbacdb6443f22749559bfc11e6954ae2e590e57095787ebb784423618",
+      "parent_rank": 2548
+    },
+    {
+      "coordinate_dim": 1433,
+      "in_features": 4096,
+      "input_sha256": "3415eb61289afd659a70a5ad551e42c6e16e929203b9b44bc33aa73ea3a4a733",
+      "module": "model.layers.7.self_attn.q_proj",
+      "out_features": 4096,
+      "output_sha256": "8618427177f1bf02c4022fa284a2cc420669453f2bd24e76229cd8c5e8dd743e",
+      "parent_rank": 1638
+    },
+    {
+      "coordinate_dim": 573,
+      "in_features": 4096,
+      "input_sha256": "bccbc4ae3353caef49506d259c00b487eb5cf9407327c685b7934851e434775d",
+      "module": "model.layers.7.self_attn.k_proj",
+      "out_features": 1024,
+      "output_sha256": "e37b63c1c0935de3f07d12bbd7ab5f2b2fa2ef2da418b47dd6b5dfa0d2d612f0",
+      "parent_rank": 655
+    },
+    {
+      "coordinate_dim": 573,
+      "in_features": 4096,
+      "input_sha256": "fb2d5b19bd4e677303eda3b2fb6b59aac60bcd704f9ba2e2c7b2887625a8b062",
+      "module": "model.layers.7.self_attn.v_proj",
+      "out_features": 1024,
+      "output_sha256": "9d86fca16937518113a4788d948f3637a119e7acd35e4118fcd05b9f3282e747",
+      "parent_rank": 655
+    },
+    {
+      "coordinate_dim": 1433,
+      "in_features": 4096,
+      "input_sha256": "29c4703d4355add0af7aa62cff1022dd557dbb276d8ba9b3164214c34af72755",
+      "module": "model.layers.7.self_attn.o_proj",
+      "out_features": 4096,
+      "output_sha256": "c8811f0751ccc61d374dba6450addf657a1d1dec9ab1ba0694b134d40dc9dd4c",
+      "parent_rank": 1638
+    },
+    {
+      "coordinate_dim": 2230,
+      "in_features": 4096,
+      "input_sha256": "b90ae24716ef9b2f950c92e7531e6d97d50dab8f8eb3758f7cb43f1f5f77af25",
+      "module": "model.layers.7.mlp.gate_proj",
+      "out_features": 14336,
+      "output_sha256": "5c09519970a40123a3ec369eaf87663a5a671a199c1147366fbe8c2d02c5f5be",
+      "parent_rank": 2548
+    },
+    {
+      "coordinate_dim": 2230,
+      "in_features": 14336,
+      "input_sha256": "fa9ead09308a8c2fdb228137bc3535c309f31482eb739c631bbef45e3475d053",
+      "module": "model.layers.7.mlp.down_proj",
+      "out_features": 4096,
+      "output_sha256": "8dd87f08a6b813fcf6e8f4646dcfb80b38bf6c12af0a0035d67788b4933551d9",
+      "parent_rank": 2548
+    },
+    {
+      "coordinate_dim": 2230,
+      "in_features": 4096,
+      "input_sha256": "83ff177c2704d6dcd80ff4aee3b688300221ce754ae20f4ca15874951233e1f9",
+      "module": "model.layers.7.mlp.up_proj",
+      "out_features": 14336,
+      "output_sha256": "bf55ef98c2dc3049d01587c003e0172977b9125c119749a9f56e2f2720ab8691",
+      "parent_rank": 2548
+    },
+    {
+      "coordinate_dim": 1433,
+      "in_features": 4096,
+      "input_sha256": "40da0a5569d3c96eaac8b44eed98fbb1faadd7b874d27e46d142dd48fa3876a2",
+      "module": "model.layers.8.self_attn.q_proj",
+      "out_features": 4096,
+      "output_sha256": "003a64dc876acbda650f8f8b1bac28394c6f7dfad165f4937cdf3cc683db8ba8",
+      "parent_rank": 1638
+    },
+    {
+      "coordinate_dim": 573,
+      "in_features": 4096,
+      "input_sha256": "2a0a10d90a4c178417ec90e4d7164c424489c608edbd22cc4cf142a62ffe4f4d",
+      "module": "model.layers.8.self_attn.k_proj",
+      "out_features": 1024,
+      "output_sha256": "1a653e70f5cac0ca3b0bad4807046622aea11e711ae37d852e59dd1c55dcd069",
+      "parent_rank": 655
+    },
+    {
+      "coordinate_dim": 573,
+      "in_features": 4096,
+      "input_sha256": "342d123c51db556a65ce814a2c5e95983a931c5492e60751c6d05c9d69c33f7a",
+      "module": "model.layers.8.self_attn.v_proj",
+      "out_features": 1024,
+      "output_sha256": "ae4700f30c526265258237ee974bb394edb8821d0f76761f513dd2ca3ec408a8",
+      "parent_rank": 655
+    },
+    {
+      "coordinate_dim": 1433,
+      "in_features": 4096,
+      "input_sha256": "1c96ba94e51e60a9b9b3d6b658d68cb34af22ec9a040266dae1d138cf3c76b4b",
+      "module": "model.layers.8.self_attn.o_proj",
+      "out_features": 4096,
+      "output_sha256": "44a61199294af15ba75358ab8fded582ad7eff2cea7d6bc2c31a5ccfffb6729f",
+      "parent_rank": 1638
+    },
+    {
+      "coordinate_dim": 2230,
+      "in_features": 4096,
+      "input_sha256": "b0354145908f82a5e83595159f047d4c580f173c07251570debcc78b211124e0",
+      "module": "model.layers.8.mlp.gate_proj",
+      "out_features": 14336,
+      "output_sha256": "e4d8f8e215d496abf400e2959763e49904150ac114665656a584732d7a7b5b4c",
+      "parent_rank": 2548
+    },
+    {
+      "coordinate_dim": 2230,
+      "in_features": 14336,
+      "input_sha256": "4fbe343a664378954c434167f251ec174fe2fc4f077f65ab7d608ae8f7e12544",
+      "module": "model.layers.8.mlp.down_proj",
+      "out_features": 4096,
+      "output_sha256": "31a299ad46766eb57d6a550542106fb6f86cc0872957fb42916c10c0ed5f572c",
+      "parent_rank": 2548
+    },
+    {
+      "coordinate_dim": 2230,
+      "in_features": 4096,
+      "input_sha256": "8feade5a9d532b5fb636c5ce00d4bc864231a6b9e1db8f479f2585fd1c9c9f18",
+      "module": "model.layers.8.mlp.up_proj",
+      "out_features": 14336,
+      "output_sha256": "41df80ecb84db710497b751f775c9e18d48c0b461946b2fd1e9dd74a55f6abdd",
+      "parent_rank": 2548
+    },
+    {
+      "coordinate_dim": 1433,
+      "in_features": 4096,
+      "input_sha256": "be5193269c6ea98e16ba419f4ae6270809640a93b613c9a0f11f83f26d29c0ff",
+      "module": "model.layers.9.self_attn.q_proj",
+      "out_features": 4096,
+      "output_sha256": "59b0003623dbbc50bb65f712ac4561a027171cb5e61971ea961952444a730192",
+      "parent_rank": 1638
+    },
+    {
+      "coordinate_dim": 573,
+      "in_features": 4096,
+      "input_sha256": "6b347a010a8b4cad0f51077b5fa225d8c9c09fbdceaa6a0ef100c8f61f649093",
+      "module": "model.layers.9.self_attn.k_proj",
+      "out_features": 1024,
+      "output_sha256": "2c7e8020015f8eeacb0155fb6fc1600bde61160027b435e263dd0cd471505ab5",
+      "parent_rank": 655
+    },
+    {
+      "coordinate_dim": 573,
+      "in_features": 4096,
+      "input_sha256": "154ff95cf18f2543ec28eccef09584d2f36a1fef1e78e41b17690fbb63954c61",
+      "module": "model.layers.9.self_attn.v_proj",
+      "out_features": 1024,
+      "output_sha256": "1a93e98c39a2d8f956adab016b67228f5555dee430ded4e60d8bc1fe722b7ee6",
+      "parent_rank": 655
+    },
+    {
+      "coordinate_dim": 1433,
+      "in_features": 4096,
+      "input_sha256": "4731b0a6d142f495fef4b4f8f5e56177344343bfe1ed3dd1b4e91a9cc7b8b3f5",
+      "module": "model.layers.9.self_attn.o_proj",
+      "out_features": 4096,
+      "output_sha256": "8b81ebd97c70b93135805c57069d7fb25ef93e84e003f1da9c7df3cfec74cac3",
+      "parent_rank": 1638
+    },
+    {
+      "coordinate_dim": 2230,
+      "in_features": 4096,
+      "input_sha256": "d571133799c69705ac5fbb42002bf94b97cff478ca7f38a61fc22b23483cb320",
+      "module": "model.layers.9.mlp.gate_proj",
+      "out_features": 14336,
+      "output_sha256": "32d1a54355213e2b43f9012b8da428be0e542040f03f8ebdf7741aa775a1ddd5",
+      "parent_rank": 2548
+    },
+    {
+      "coordinate_dim": 2230,
+      "in_features": 14336,
+      "input_sha256": "5b7d4d5118898bddf065f79b7b8fbaabe9b02dd29e404c7b3761fcbc68dca3cd",
+      "module": "model.layers.9.mlp.down_proj",
+      "out_features": 4096,
+      "output_sha256": "383258ad60bc839d1ae4be9eb83684d944ff730ab0e85e3473e151cb388770cb",
+      "parent_rank": 2548
+    },
+    {
+      "coordinate_dim": 2230,
+      "in_features": 4096,
+      "input_sha256": "8c67d13e89ac400e97be0c955e2fa7c2e7ddc79062a53af7db206e437d862e87",
+      "module": "model.layers.9.mlp.up_proj",
+      "out_features": 14336,
+      "output_sha256": "2aaea97e2b3f7dfae0d4f41445bfb8f2c50625971ec33cc9e9c70b159e27e0e8",
+      "parent_rank": 2548
+    },
+    {
+      "coordinate_dim": 1433,
+      "in_features": 4096,
+      "input_sha256": "a55c4a5d31b414e820321e88023a43607004ce8113de193ae42d2bdee31758d7",
+      "module": "model.layers.10.self_attn.q_proj",
+      "out_features": 4096,
+      "output_sha256": "6ff1eab69aed5890775a069d4483c3534bf848b636e1e8c677d6a998200c2441",
+      "parent_rank": 1638
+    },
+    {
+      "coordinate_dim": 573,
+      "in_features": 4096,
+      "input_sha256": "7c9d2451622e5941662574641b66d4e74ca137acfe5d74783e9d636bcf6c71cc",
+      "module": "model.layers.10.self_attn.k_proj",
+      "out_features": 1024,
+      "output_sha256": "c6f9a9a1291892fe4aa895862f62b69efa18c132f256a161176da7dbe2ffe4eb",
+      "parent_rank": 655
+    },
+    {
+      "coordinate_dim": 573,
+      "in_features": 4096,
+      "input_sha256": "4b48b7384a1e6ac72333ec79c43e106b622a066de851ac4836088424856b835a",
+      "module": "model.layers.10.self_attn.v_proj",
+      "out_features": 1024,
+      "output_sha256": "d0a4cd08fb300ffef3248fb40e42659d7f0722779f5d162063625bfa0d7f681c",
+      "parent_rank": 655
+    },
+    {
+      "coordinate_dim": 1433,
+      "in_features": 4096,
+      "input_sha256": "d5683b076df344373b98aa7f3a512e7b2a1f94224488bff6e2a137d87c0c9517",
+      "module": "model.layers.10.self_attn.o_proj",
+      "out_features": 4096,
+      "output_sha256": "7b535d9b581a701eba708a06d4662dd6bf2d3379f112c402f74e6d2bdb73d62e",
+      "parent_rank": 1638
+    },
+    {
+      "coordinate_dim": 2230,
+      "in_features": 4096,
+      "input_sha256": "3b877455d431ff0a9a2fb14dc157bdb3e5ef66d6f6554779d23286ff9c64d0a8",
+      "module": "model.layers.10.mlp.gate_proj",
+      "out_features": 14336,
+      "output_sha256": "bda40368a6d5e653a4d546139972b516096a1f9c941a912fa44c16283b315db5",
+      "parent_rank": 2548
+    },
+    {
+      "coordinate_dim": 2230,
+      "in_features": 14336,
+      "input_sha256": "80b8168aa4893615236933cb41bef25c74d73a027909bf5bb906344280d51ae7",
+      "module": "model.layers.10.mlp.down_proj",
+      "out_features": 4096,
+      "output_sha256": "d1b4b12ea649f6cab6d5cbcc670ae8b454f3206b6fce965443f7c5d800378d4c",
+      "parent_rank": 2548
+    },
+    {
+      "coordinate_dim": 2230,
+      "in_features": 4096,
+      "input_sha256": "e0e65bea2ff156911ea637d02762056bb8a30b93b6488c35e09205f83e23d363",
+      "module": "model.layers.10.mlp.up_proj",
+      "out_features": 14336,
+      "output_sha256": "8aa4657a67625699430266037b80ce52b6727e26e1013c784ad5703b1cd49a37",
+      "parent_rank": 2548
+    },
+    {
+      "coordinate_dim": 1433,
+      "in_features": 4096,
+      "input_sha256": "8ce2c158d6fb3915d510837c105a782055a687d529a4e813031a4f5a3d419abf",
+      "module": "model.layers.11.self_attn.q_proj",
+      "out_features": 4096,
+      "output_sha256": "2d0100912ab11683cd1a7547ac8a4676e6a737be416109c6bd84b954746c2ba4",
+      "parent_rank": 1638
+    },
+    {
+      "coordinate_dim": 573,
+      "in_features": 4096,
+      "input_sha256": "d7a1c8e38b37188fcf0d4a92fed45a5866b14a308ba2e7cb21ee4e74b3877cd8",
+      "module": "model.layers.11.self_attn.k_proj",
+      "out_features": 1024,
+      "output_sha256": "3ffaaee6e7710095c8dbd95d997e870fc640437494b7947ce8da7aae91ef60dd",
+      "parent_rank": 655
+    },
+    {
+      "coordinate_dim": 573,
+      "in_features": 4096,
+      "input_sha256": "52aa8487eb2971df91a8e10945503afc891b9433a7a4f57d53c9a343efeff1ae",
+      "module": "model.layers.11.self_attn.v_proj",
+      "out_features": 1024,
+      "output_sha256": "1eaa5527a14183245773af1fe963bff957c621e28033ede2f063c28208eb61a0",
+      "parent_rank": 655
+    },
+    {
+      "coordinate_dim": 1433,
+      "in_features": 4096,
+      "input_sha256": "7330ce2377901d06306fe38e30f6186986f82cda433fa8468761c574e1e480cd",
+      "module": "model.layers.11.self_attn.o_proj",
+      "out_features": 4096,
+      "output_sha256": "1009211fe1d910969c0fc42d5b4966a4ddbec7b3fa443e32ee71eb39334950ba",
+      "parent_rank": 1638
+    },
+    {
+      "coordinate_dim": 2230,
+      "in_features": 4096,
+      "input_sha256": "98fac1b422d229cd66fe52c27959d85ab6b155092f3394f556e91ee80ea083b5",
+      "module": "model.layers.11.mlp.gate_proj",
+      "out_features": 14336,
+      "output_sha256": "79c8cfe63a18262ad81acf97ae5a76c459321686240a72d8facbf4b404dd72be",
+      "parent_rank": 2548
+    },
+    {
+      "coordinate_dim": 2230,
+      "in_features": 14336,
+      "input_sha256": "466826de6404f035deca1fc6b6bcf8deb81f0b0ddb6e50681dddfdf13aa9067b",
+      "module": "model.layers.11.mlp.down_proj",
+      "out_features": 4096,
+      "output_sha256": "b46627d6330ad71eae88fd8bfda43e8bb4b117099fcd0287292097e453f23474",
+      "parent_rank": 2548
+    },
+    {
+      "coordinate_dim": 2230,
+      "in_features": 4096,
+      "input_sha256": "b427f9a2f7e6092e60506a77b5b459dafff5104d380adb9201f7e90f1a5ce083",
+      "module": "model.layers.11.mlp.up_proj",
+      "out_features": 14336,
+      "output_sha256": "235c1835d249f446f09c277d0e4a456237381b6320e38d63764e956017f015bc",
+      "parent_rank": 2548
+    },
+    {
+      "coordinate_dim": 1433,
+      "in_features": 4096,
+      "input_sha256": "b380736a65e1e62738c0628cc14b9aed399ed4e2dd83d39b602e5fcb518024c3",
+      "module": "model.layers.12.self_attn.q_proj",
+      "out_features": 4096,
+      "output_sha256": "210671aa96d5573c9cfd92cd0117284992a9bcd57ffa6d217e35f535e076d5a5",
+      "parent_rank": 1638
+    },
+    {
+      "coordinate_dim": 573,
+      "in_features": 4096,
+      "input_sha256": "ce16de498f5d01d4d5d882f7684a13a327179298836f9027d44e73971b0dd7f2",
+      "module": "model.layers.12.self_attn.k_proj",
+      "out_features": 1024,
+      "output_sha256": "ec00419c2cbe505047e0e5aa8433a7de26de86dadf891b5bef0dde68f37effa2",
+      "parent_rank": 655
+    },
+    {
+      "coordinate_dim": 573,
+      "in_features": 4096,
+      "input_sha256": "f6a184fa8abd492486746ae3b379cddf874964df1581a84420e95266b8c85bc1",
+      "module": "model.layers.12.self_attn.v_proj",
+      "out_features": 1024,
+      "output_sha256": "fefeadfe55eb455cf16868cbf429318760b5b40127cd1d31a1ca8a497463ade2",
+      "parent_rank": 655
+    },
+    {
+      "coordinate_dim": 1433,
+      "in_features": 4096,
+      "input_sha256": "feac4139bcdc1288b8cc2b186172ff67c2b754ec38fe14abe40575974c597175",
+      "module": "model.layers.12.self_attn.o_proj",
+      "out_features": 4096,
+      "output_sha256": "ec578ffcc7ab5040ab42a7d1b6403ff2e860f113ecc8a56e2417acebd70b7be4",
+      "parent_rank": 1638
+    },
+    {
+      "coordinate_dim": 2230,
+      "in_features": 4096,
+      "input_sha256": "24d208c2f9ec282ef768d86698c65efbc7d92a372affb04adb8a01f8080e9f41",
+      "module": "model.layers.12.mlp.gate_proj",
+      "out_features": 14336,
+      "output_sha256": "9cbd084f222037eb398a77f27244838273f69544cd96dcd5176b0aeb6e6f7944",
+      "parent_rank": 2548
+    },
+    {
+      "coordinate_dim": 2230,
+      "in_features": 14336,
+      "input_sha256": "bbd86715077988ae6884a3b034636e2aa31d099c048f31fbcb976e4b51ad4ec3",
+      "module": "model.layers.12.mlp.down_proj",
+      "out_features": 4096,
+      "output_sha256": "7b243ae1af675fd36d99e935e2a029ecdad12ffd87a69c59bba879b12ce2776c",
+      "parent_rank": 2548
+    },
+    {
+      "coordinate_dim": 2230,
+      "in_features": 4096,
+      "input_sha256": "87e55b6bcbae8df39543de8b90e7f05d8d153bc0afcb974b873d73b6f9cefdcc",
+      "module": "model.layers.12.mlp.up_proj",
+      "out_features": 14336,
+      "output_sha256": "bcd1a8f1f3a898c59fef6fffd51ed13153e6dd71d39f6509cc8ab5cda29dd9b5",
+      "parent_rank": 2548
+    },
+    {
+      "coordinate_dim": 1433,
+      "in_features": 4096,
+      "input_sha256": "8301951972f5df8c08226c40c94d7fd2378dd91dedb20157f09769c09edb3d68",
+      "module": "model.layers.13.self_attn.q_proj",
+      "out_features": 4096,
+      "output_sha256": "d8df3c27c24ead0ca496d3115ac0a10f6108999a30763e306afe7f8116473a30",
+      "parent_rank": 1638
+    },
+    {
+      "coordinate_dim": 573,
+      "in_features": 4096,
+      "input_sha256": "16bd0a12f547d7ae39651b23a9e353b826e02ec9540698fbcafec00c9bcf891d",
+      "module": "model.layers.13.self_attn.k_proj",
+      "out_features": 1024,
+      "output_sha256": "5a77a19c0fa07bc4ca65cbf39f44c95e4484dd42c81859c7bca6a6f441511a0b",
+      "parent_rank": 655
+    },
+    {
+      "coordinate_dim": 573,
+      "in_features": 4096,
+      "input_sha256": "348f73775b490557a0dcee0ef90f0c7b7e68dca86923fcc08f5d1f6a6c0b44ed",
+      "module": "model.layers.13.self_attn.v_proj",
+      "out_features": 1024,
+      "output_sha256": "8b5965395002cccf196c1e98ea3644d16186c4d8dab32a196d117735465ab46c",
+      "parent_rank": 655
+    },
+    {
+      "coordinate_dim": 1433,
+      "in_features": 4096,
+      "input_sha256": "743f099cca74be0732d9de825c624190fc1054a7a056cd51ce9bddae87dc8c61",
+      "module": "model.layers.13.self_attn.o_proj",
+      "out_features": 4096,
+      "output_sha256": "54e4e1c09753f55dac5b4628933d4e459e3b4097075fae09fa19598a38e6009a",
+      "parent_rank": 1638
+    },
+    {
+      "coordinate_dim": 2230,
+      "in_features": 4096,
+      "input_sha256": "ee85f56687ac74451d7efe6cbf49e054971575609299aeaf28657dd390efa1f0",
+      "module": "model.layers.13.mlp.gate_proj",
+      "out_features": 14336,
+      "output_sha256": "cc4f1ad4829f20ac7f1a6ad7cb071f7938a6c6de4881f85db64dd508be341251",
+      "parent_rank": 2548
+    },
+    {
+      "coordinate_dim": 2230,
+      "in_features": 14336,
+      "input_sha256": "9bb9765a5685aca15d5b2817aa1b41ac93009b7e57a39d67345a66510409147c",
+      "module": "model.layers.13.mlp.down_proj",
+      "out_features": 4096,
+      "output_sha256": "e4403efb6d124e9a64c16b2af413351a4e65004bbe9b798edc996f92378caa8d",
+      "parent_rank": 2548
+    },
+    {
+      "coordinate_dim": 2230,
+      "in_features": 4096,
+      "input_sha256": "946dc2346a7e988c6f07b4a415a0ee90d0216be7eb0d9bb50c8a2dbe262937d1",
+      "module": "model.layers.13.mlp.up_proj",
+      "out_features": 14336,
+      "output_sha256": "5888da5f5f5acce1ea7fa54c9708d362bbc083dc00b473b1dfc94be34ad118e0",
+      "parent_rank": 2548
+    },
+    {
+      "coordinate_dim": 1433,
+      "in_features": 4096,
+      "input_sha256": "f889397c2053a60d5a99aa2664dd97510940ee648d874a042f90b49e42d00dd0",
+      "module": "model.layers.14.self_attn.q_proj",
+      "out_features": 4096,
+      "output_sha256": "cc8d46a2645bf5545aa90e3a40bf291f2c3002a834fe2ab24f7e152091641731",
+      "parent_rank": 1638
+    },
+    {
+      "coordinate_dim": 573,
+      "in_features": 4096,
+      "input_sha256": "99393d5ac423f52b431846ddaf8c59ad64d6b69e72a6008daeed870688f83e28",
+      "module": "model.layers.14.self_attn.k_proj",
+      "out_features": 1024,
+      "output_sha256": "8bad5e9fcd6a8564c6128070c3d582fab5e300c987813626d8e0a72daa2d2827",
+      "parent_rank": 655
+    },
+    {
+      "coordinate_dim": 573,
+      "in_features": 4096,
+      "input_sha256": "2a9457c349e4ababc9b979c386ff6b9751248b59214f1f058469cf219775040a",
+      "module": "model.layers.14.self_attn.v_proj",
+      "out_features": 1024,
+      "output_sha256": "7e55de6c3386fa7d048cb850608fbe310dfdf982542e0d2e0f2cacd0f9728548",
+      "parent_rank": 655
+    },
+    {
+      "coordinate_dim": 1433,
+      "in_features": 4096,
+      "input_sha256": "a0c8c62da9fc56f218fdfcb033b0f26cb464dc6e32f031ab494ecf9355183d3c",
+      "module": "model.layers.14.self_attn.o_proj",
+      "out_features": 4096,
+      "output_sha256": "190a1fdd93852232362ff7a044ac6689edb5759b5a93f3cd62709d4eca51566c",
+      "parent_rank": 1638
+    },
+    {
+      "coordinate_dim": 2230,
+      "in_features": 4096,
+      "input_sha256": "205e6840bcd7605334cce2ba73c490e6a696bbccdaf73173a26058b4df225f43",
+      "module": "model.layers.14.mlp.gate_proj",
+      "out_features": 14336,
+      "output_sha256": "c707ed1ffd043630da9af65ce1e4c9499e5d9e8f576544667f2bbfcd977f3db7",
+      "parent_rank": 2548
+    },
+    {
+      "coordinate_dim": 2230,
+      "in_features": 14336,
+      "input_sha256": "a24e04b8dcf0852f9418df96c99f85dc168de11e282619ad8dca4a173fb9da76",
+      "module": "model.layers.14.mlp.down_proj",
+      "out_features": 4096,
+      "output_sha256": "afc9765f7f93e4c3378918f53526e939f8915439d5f108175876b3d7f5ffc588",
+      "parent_rank": 2548
+    },
+    {
+      "coordinate_dim": 2230,
+      "in_features": 4096,
+      "input_sha256": "f5581059a90870eff18718cc1b5323bafc07508f4baf223f9484326e0eb6f580",
+      "module": "model.layers.14.mlp.up_proj",
+      "out_features": 14336,
+      "output_sha256": "64960e3c13e73674ea8d7cef97c332a03b3dc7e5522c55ca02d5ec439b5c2d14",
+      "parent_rank": 2548
+    },
+    {
+      "coordinate_dim": 1433,
+      "in_features": 4096,
+      "input_sha256": "78b52b3fe1b39ea459aa61e3c0ce9bd070766216e50ab786bc7ea720eb2eab3e",
+      "module": "model.layers.15.self_attn.q_proj",
+      "out_features": 4096,
+      "output_sha256": "02d119af96fc4a6c2d279f715cbc58c5ea0dc4936985f926ffbe06196c154743",
+      "parent_rank": 1638
+    },
+    {
+      "coordinate_dim": 573,
+      "in_features": 4096,
+      "input_sha256": "bf1284377305b71eb68275c6633c4964cf30e56c78f50d4fc1338b620d362334",
+      "module": "model.layers.15.self_attn.k_proj",
+      "out_features": 1024,
+      "output_sha256": "1cde0e8a6ed1bf66e18b1798b9c764811b97cba2d8454c60f9e4165757647f3a",
+      "parent_rank": 655
+    },
+    {
+      "coordinate_dim": 573,
+      "in_features": 4096,
+      "input_sha256": "14300ee3f989973da538bf4a532df08935c04064cfe947e8663f6598644c1988",
+      "module": "model.layers.15.self_attn.v_proj",
+      "out_features": 1024,
+      "output_sha256": "638b11a13edf46a441ea425b2e5eae9439e68eac86af7a6ba9e0828a8a370e4c",
+      "parent_rank": 655
+    },
+    {
+      "coordinate_dim": 1433,
+      "in_features": 4096,
+      "input_sha256": "67848d3eb189bdf72ae574e92c43a2443fece2ac149f283d1f2dedd717edc215",
+      "module": "model.layers.15.self_attn.o_proj",
+      "out_features": 4096,
+      "output_sha256": "2fe851294b6b5939be9be5277d32a1c97912485e66fbf67f85d4eef223c8399e",
+      "parent_rank": 1638
+    },
+    {
+      "coordinate_dim": 2230,
+      "in_features": 4096,
+      "input_sha256": "e335b5357113ca5126c5e5a43d72587634103af8052b5cb39422e5134b67bad2",
+      "module": "model.layers.15.mlp.gate_proj",
+      "out_features": 14336,
+      "output_sha256": "f552d09da52b274f4cf6f7257469618dc13790df7912aab37b3e19a15c2ae471",
+      "parent_rank": 2548
+    },
+    {
+      "coordinate_dim": 2230,
+      "in_features": 14336,
+      "input_sha256": "99f2361c89d555ff0bb293a2e1f80c5000275bca09c3a8b19582f98176b055dc",
+      "module": "model.layers.15.mlp.down_proj",
+      "out_features": 4096,
+      "output_sha256": "76a2a54546b5a99327da806e82475c42a0dc662a36787a2d01ddf2d6be84b2bf",
+      "parent_rank": 2548
+    },
+    {
+      "coordinate_dim": 2230,
+      "in_features": 4096,
+      "input_sha256": "6c276ab0964384e8f15e674c52a23175bc0c19b66402ee6b1e15111c2403d9c1",
+      "module": "model.layers.15.mlp.up_proj",
+      "out_features": 14336,
+      "output_sha256": "acaefd2b68764b2674566d03eea0fd439943aa6775afa3f797c879fd6532d292",
+      "parent_rank": 2548
+    },
+    {
+      "coordinate_dim": 1433,
+      "in_features": 4096,
+      "input_sha256": "d0ac7681b3eeefc2c3d78a65d6a6117b9acb412eeb180a9abe399c9b5ed7c22a",
+      "module": "model.layers.16.self_attn.q_proj",
+      "out_features": 4096,
+      "output_sha256": "10c32b0362b0335305c2cd8202363308cd2e1a67aad591125738bfd6f696bb14",
+      "parent_rank": 1638
+    },
+    {
+      "coordinate_dim": 573,
+      "in_features": 4096,
+      "input_sha256": "cf792101fba92e93c3770a48efbaccb66de7c80e68d3d8765efe588a06ccc985",
+      "module": "model.layers.16.self_attn.k_proj",
+      "out_features": 1024,
+      "output_sha256": "8b8abefb465b06fec0c4a0ebc8e212f71b76cff0e9eac1cbdfdc68058ec3dfd5",
+      "parent_rank": 655
+    },
+    {
+      "coordinate_dim": 573,
+      "in_features": 4096,
+      "input_sha256": "7a821d6b200cac98dc3456abb910a7e5369eda5a5d5f7983c05c68ce10becae5",
+      "module": "model.layers.16.self_attn.v_proj",
+      "out_features": 1024,
+      "output_sha256": "04ca5e0c9d6e910f5be65425162d459ae794f66bcede5022b7e8a3e54476806c",
+      "parent_rank": 655
+    },
+    {
+      "coordinate_dim": 1433,
+      "in_features": 4096,
+      "input_sha256": "a9331ee84584db09edc2fd42cfa7c20ca576597134e0453f73354bc613fd68fb",
+      "module": "model.layers.16.self_attn.o_proj",
+      "out_features": 4096,
+      "output_sha256": "9ef6ebcf78b79220684f9b09a80a13779d7daf5cc627a316731957eba0345db2",
+      "parent_rank": 1638
+    },
+    {
+      "coordinate_dim": 2230,
+      "in_features": 4096,
+      "input_sha256": "6d5ef8a26a841cd6ea2e62d7628396d5cadd894d9dd24597b5b0acba8e652352",
+      "module": "model.layers.16.mlp.gate_proj",
+      "out_features": 14336,
+      "output_sha256": "1401738ffebe20c48d87a092c753c469a7b90c42d1826c50fb1d81370e51aeb0",
+      "parent_rank": 2548
+    },
+    {
+      "coordinate_dim": 2230,
+      "in_features": 14336,
+      "input_sha256": "14bd4a30c9b2e4f08e5235240a3584b7dab32ad49dfed5315b6aa1b8002956e2",
+      "module": "model.layers.16.mlp.down_proj",
+      "out_features": 4096,
+      "output_sha256": "ec68a390e9e0006174ef7a17d9f556e0d5d4fa78000be7bbb609419070fb7503",
+      "parent_rank": 2548
+    },
+    {
+      "coordinate_dim": 2230,
+      "in_features": 4096,
+      "input_sha256": "961e8792e65a4fe6bdc87c9112510868b3754b616a3f0033484e5d5d811e4bed",
+      "module": "model.layers.16.mlp.up_proj",
+      "out_features": 14336,
+      "output_sha256": "76ff1f3875e02152a2bd62a15a5e030e5d895ae0633c13de194287c249b9fd77",
+      "parent_rank": 2548
+    },
+    {
+      "coordinate_dim": 1433,
+      "in_features": 4096,
+      "input_sha256": "531b4e280d2fd0b930707352d441f205983c8d1cf34ffb8a4d5f0a3448474d99",
+      "module": "model.layers.17.self_attn.q_proj",
+      "out_features": 4096,
+      "output_sha256": "6495e3780550eada0716a0b11afe332c506289df853e4db3e2730110d1b6e326",
+      "parent_rank": 1638
+    },
+    {
+      "coordinate_dim": 573,
+      "in_features": 4096,
+      "input_sha256": "51e9520b88f758451d667f9723526da12b9e7c51e77a574f90d7eb9616834e10",
+      "module": "model.layers.17.self_attn.k_proj",
+      "out_features": 1024,
+      "output_sha256": "06b83570859136f75001aa9d7600dfa8513086469aadf8ce52bfdd9c3c4a0b88",
+      "parent_rank": 655
+    },
+    {
+      "coordinate_dim": 573,
+      "in_features": 4096,
+      "input_sha256": "690049e19d3a6cba84b73ae7f5a3f10893662426e01ae58bc85cceae33a68e1e",
+      "module": "model.layers.17.self_attn.v_proj",
+      "out_features": 1024,
+      "output_sha256": "a4ad802a1a865427af73eba08c7e7b5e8134768f34252893c56a9ade14ccad64",
+      "parent_rank": 655
+    },
+    {
+      "coordinate_dim": 1433,
+      "in_features": 4096,
+      "input_sha256": "686d4d6b7356dddf4b7867b866ee74d1621785c00826cc28994800758ceb413a",
+      "module": "model.layers.17.self_attn.o_proj",
+      "out_features": 4096,
+      "output_sha256": "48de39661ae3baefa53f5fd1b6bc4ee7cae4724c55aa12dd949f425beef7321c",
+      "parent_rank": 1638
+    },
+    {
+      "coordinate_dim": 2230,
+      "in_features": 4096,
+      "input_sha256": "0102c16cfaa517aaecbb0d825e3ccd4cd99eb62c47d47d13f8229eb91d00e633",
+      "module": "model.layers.17.mlp.gate_proj",
+      "out_features": 14336,
+      "output_sha256": "bc983e6f12c04c84823a97d4788330eaecb711a42e05c7bf49642da9c780e9ed",
+      "parent_rank": 2548
+    },
+    {
+      "coordinate_dim": 2230,
+      "in_features": 14336,
+      "input_sha256": "1a1ebde62ed277f9844c8683ff4458b9854154b91d6cedb6e2e9c193f0d65b34",
+      "module": "model.layers.17.mlp.down_proj",
+      "out_features": 4096,
+      "output_sha256": "d5f5c85b00759e223b0ad0dc905377854b362a6a0a5208663b4d8029c248cacf",
+      "parent_rank": 2548
+    },
+    {
+      "coordinate_dim": 2230,
+      "in_features": 4096,
+      "input_sha256": "3c728b58b639aedb9f51252103b20c7be795a53a7fedd326b43fc58f574bdc08",
+      "module": "model.layers.17.mlp.up_proj",
+      "out_features": 14336,
+      "output_sha256": "bc88450383cde382e984378e1c0726b410e270a23287cf24d3a5aee33ebb9816",
+      "parent_rank": 2548
+    },
+    {
+      "coordinate_dim": 1433,
+      "in_features": 4096,
+      "input_sha256": "420a4364c01046e05125b1565271eb1e95f031fc65a96afe8a6e3b589e9059d5",
+      "module": "model.layers.18.self_attn.q_proj",
+      "out_features": 4096,
+      "output_sha256": "47a1ef0887b47fe6b5f49f76b5da6d09fdc9f866707364683916013358340f27",
+      "parent_rank": 1638
+    },
+    {
+      "coordinate_dim": 573,
+      "in_features": 4096,
+      "input_sha256": "229d0eebc1e3c59c0625458208cf022481087516ab53866f8f373fc95772c349",
+      "module": "model.layers.18.self_attn.k_proj",
+      "out_features": 1024,
+      "output_sha256": "85bc4e2cc8e346afcfb79e148243e6a83f31dc41a34e885d61c8141d511a3d33",
+      "parent_rank": 655
+    },
+    {
+      "coordinate_dim": 573,
+      "in_features": 4096,
+      "input_sha256": "1d277b6bed7bec3258dd12fd9ae3d2fa29ee8460458134e2a48df971aedd70ba",
+      "module": "model.layers.18.self_attn.v_proj",
+      "out_features": 1024,
+      "output_sha256": "2e6a4d22dd949c9a03cf9223cf097bad428132336b73f037bad18dcb03955d02",
+      "parent_rank": 655
+    },
+    {
+      "coordinate_dim": 1433,
+      "in_features": 4096,
+      "input_sha256": "e1728ca4181e3c1b9550cbf1c52c8f9f2a82fa252767b01f28efa8d809a16c83",
+      "module": "model.layers.18.self_attn.o_proj",
+      "out_features": 4096,
+      "output_sha256": "18e6938aa9c502733d00914ae5a8787353d44cef32b15c83a1cb1fc5db723543",
+      "parent_rank": 1638
+    },
+    {
+      "coordinate_dim": 2230,
+      "in_features": 4096,
+      "input_sha256": "abccd4aa4eb8253a1fa1d0913e3f99d072a7b1a7215e0e024ab45e187b9e3afb",
+      "module": "model.layers.18.mlp.gate_proj",
+      "out_features": 14336,
+      "output_sha256": "1673c0e751d406ce92893c46f1be63c880c5e4cc39a2db2c82af024e2991ded1",
+      "parent_rank": 2548
+    },
+    {
+      "coordinate_dim": 2230,
+      "in_features": 14336,
+      "input_sha256": "13938089791bdb9b78a424fee98ccff393ff7a70d55562564e9dbbf98d988deb",
+      "module": "model.layers.18.mlp.down_proj",
+      "out_features": 4096,
+      "output_sha256": "89573ab8ebf770919e163683903bf7e4b150f49e08cc09b37e9b122d27468ef6",
+      "parent_rank": 2548
+    },
+    {
+      "coordinate_dim": 2230,
+      "in_features": 4096,
+      "input_sha256": "89cfa0898d704c4a1b977c93a185eea5c6a36a233ebb3a31f843daa747944dd2",
+      "module": "model.layers.18.mlp.up_proj",
+      "out_features": 14336,
+      "output_sha256": "3a96fb56cafa75191840d49f40db74fed3e4c439f3bff8aec4212c76b97436d8",
+      "parent_rank": 2548
+    },
+    {
+      "coordinate_dim": 1433,
+      "in_features": 4096,
+      "input_sha256": "e00cf71ce35beb07647ce66629b2daaf1a7c1545b234ec27e2219e06383dcaa0",
+      "module": "model.layers.19.self_attn.q_proj",
+      "out_features": 4096,
+      "output_sha256": "b587207be935ca5d7862ef683a1b8535dc7ccdc14d6c5d5ef65ca687f464a8d1",
+      "parent_rank": 1638
+    },
+    {
+      "coordinate_dim": 573,
+      "in_features": 4096,
+      "input_sha256": "4543e334db84e9cae467941bf98641312e926b603204071e093204462ec4be07",
+      "module": "model.layers.19.self_attn.k_proj",
+      "out_features": 1024,
+      "output_sha256": "93d1c8914cb1499754ce2c333ee30d665b0c29e32882ba45274b28f6ffaf218f",
+      "parent_rank": 655
+    },
+    {
+      "coordinate_dim": 573,
+      "in_features": 4096,
+      "input_sha256": "f0be666b6fc5967f099960455d19036fc184718a7e4bfe38d74323270009b86e",
+      "module": "model.layers.19.self_attn.v_proj",
+      "out_features": 1024,
+      "output_sha256": "1957a27e94cca91b9bbdfbdc66d37197e4ca8c346124c904553042cbee761c8f",
+      "parent_rank": 655
+    },
+    {
+      "coordinate_dim": 1433,
+      "in_features": 4096,
+      "input_sha256": "ba510fbf6ae736232fbe662d938bad1f3671bcf3535b399bb48c5fc0ac366821",
+      "module": "model.layers.19.self_attn.o_proj",
+      "out_features": 4096,
+      "output_sha256": "a3017c311d1e3db79c2830b6ec95b0ec6cb45f923f6bdfd0f36d8e0902ffecea",
+      "parent_rank": 1638
+    },
+    {
+      "coordinate_dim": 2230,
+      "in_features": 4096,
+      "input_sha256": "0f244bc331da522b82b736359a88068eabeb3aa3ce15b94560805919ff7bfef8",
+      "module": "model.layers.19.mlp.gate_proj",
+      "out_features": 14336,
+      "output_sha256": "b6deb78c802d7a464147110cdde07845d1e41d9f3bf2ad90c16dc32adb842a22",
+      "parent_rank": 2548
+    },
+    {
+      "coordinate_dim": 2230,
+      "in_features": 14336,
+      "input_sha256": "bedf5b3fb65ed0c0c8d4f35e339ed5e26c0867b80fc0518eacbf715f53bab1f1",
+      "module": "model.layers.19.mlp.down_proj",
+      "out_features": 4096,
+      "output_sha256": "09e1708823b7e1a2a0299ac87d31dbde585837fce96779513b71da950e31e1be",
+      "parent_rank": 2548
+    },
+    {
+      "coordinate_dim": 2230,
+      "in_features": 4096,
+      "input_sha256": "0d355ebd0e8fd9252c62deb8c4f4403621520293344f182a5df39ca66f2ba0cc",
+      "module": "model.layers.19.mlp.up_proj",
+      "out_features": 14336,
+      "output_sha256": "0a3b94fc726d1925751178423df26c9bbe20b380779c7638e0ea5a764fd25c5a",
+      "parent_rank": 2548
+    },
+    {
+      "coordinate_dim": 1433,
+      "in_features": 4096,
+      "input_sha256": "cba24fd9fbb4c6aa7a6d828bcc634f26a420f6e50dcaef89595ec08b5c4de436",
+      "module": "model.layers.20.self_attn.q_proj",
+      "out_features": 4096,
+      "output_sha256": "e059353943564bfe2257eea0825fe076695860d2baaa7334ec7c322ccda8415b",
+      "parent_rank": 1638
+    },
+    {
+      "coordinate_dim": 573,
+      "in_features": 4096,
+      "input_sha256": "47c5719f9f69ec8f3f9f00cb6907527e841b8e986df7b2e147f3e24dbeb728f3",
+      "module": "model.layers.20.self_attn.k_proj",
+      "out_features": 1024,
+      "output_sha256": "c7186bcb684337e21105d76e092af6679858d5100167d40c5225a87127a503d6",
+      "parent_rank": 655
+    },
+    {
+      "coordinate_dim": 573,
+      "in_features": 4096,
+      "input_sha256": "a2a47c39ac89a379411506debe5edba141c0ee8299b4472d3fc2d1e14aebba46",
+      "module": "model.layers.20.self_attn.v_proj",
+      "out_features": 1024,
+      "output_sha256": "53fcd1f5037007d42aafbc86fe8ac4a4ad3302dfc7bf5719e4ff49bebd3f11f2",
+      "parent_rank": 655
+    },
+    {
+      "coordinate_dim": 1433,
+      "in_features": 4096,
+      "input_sha256": "0f7451647d66a2b584959255a1c7c0ad9093af8c2431806f249cd68f1e5122b4",
+      "module": "model.layers.20.self_attn.o_proj",
+      "out_features": 4096,
+      "output_sha256": "9fac0d041dae0a6fea2185f55fcbe2010a86ecceaedbc53df66eff00619cdf63",
+      "parent_rank": 1638
+    },
+    {
+      "coordinate_dim": 2230,
+      "in_features": 4096,
+      "input_sha256": "b85a372e31eab50974b934cb843f03a9bbc5fc36170905edfa53b6f390607f41",
+      "module": "model.layers.20.mlp.gate_proj",
+      "out_features": 14336,
+      "output_sha256": "013a66aa2b9e6a959871b3d55286a4a9d397c644c5b263a5e4a0d0307bf5eb32",
+      "parent_rank": 2548
+    },
+    {
+      "coordinate_dim": 2230,
+      "in_features": 14336,
+      "input_sha256": "dc745e1d2c7a17228077f0546067fb4ac2fbb400d83925bc8b5163e7c274358b",
+      "module": "model.layers.20.mlp.down_proj",
+      "out_features": 4096,
+      "output_sha256": "07bf26bb1c4bc78dd268f27cbcb373925108a15743049ce98c5183386cee5f28",
+      "parent_rank": 2548
+    },
+    {
+      "coordinate_dim": 2230,
+      "in_features": 4096,
+      "input_sha256": "14ea56bd44bbaeaba2a4fdcd07ed730609b1313c2dc3b661e2d57f7698c801e9",
+      "module": "model.layers.20.mlp.up_proj",
+      "out_features": 14336,
+      "output_sha256": "97a19135abdc3242227f43f9ff963674aeff890950f2d15b4bff9674f398f765",
+      "parent_rank": 2548
+    },
+    {
+      "coordinate_dim": 1433,
+      "in_features": 4096,
+      "input_sha256": "3eb64975350ae8d99a806829084086459c24e45c4cae9787883d99f001a2e56f",
+      "module": "model.layers.21.self_attn.q_proj",
+      "out_features": 4096,
+      "output_sha256": "75bc0b6a8a37d207b12615dacca08743cc1caea1e17f3793b46e5edfb155bf15",
+      "parent_rank": 1638
+    },
+    {
+      "coordinate_dim": 573,
+      "in_features": 4096,
+      "input_sha256": "90bcab13c206553ef0ece10b840b5aeb0d6f012f2f84583e83afdc20f80603a5",
+      "module": "model.layers.21.self_attn.k_proj",
+      "out_features": 1024,
+      "output_sha256": "5629f36761abd1816afa6c93a78204a74617456fa01a20ffe0d5065b6b8c4e5c",
+      "parent_rank": 655
+    },
+    {
+      "coordinate_dim": 573,
+      "in_features": 4096,
+      "input_sha256": "1b3b35867a9cb4e55c4f94fbef84dd64ff5c8fc8969ae2b76724c9a983978534",
+      "module": "model.layers.21.self_attn.v_proj",
+      "out_features": 1024,
+      "output_sha256": "a3701a3287c2528cf1550b962e98de5ebb3f5073743fb0a773fd6c7027a448ae",
+      "parent_rank": 655
+    },
+    {
+      "coordinate_dim": 1433,
+      "in_features": 4096,
+      "input_sha256": "10e02e0d33399a6d370115f8608093a96e51063d9c4d451e7061ad6bf47a961b",
+      "module": "model.layers.21.self_attn.o_proj",
+      "out_features": 4096,
+      "output_sha256": "ad855991ac2775dc94c0e21d5528d88c2ca77d56f51b366416a4452617fce0c8",
+      "parent_rank": 1638
+    },
+    {
+      "coordinate_dim": 2230,
+      "in_features": 4096,
+      "input_sha256": "cfb94d1de3bd4a6a94d86a14c644780116ec7302aa3ee666edc2633e63b2005d",
+      "module": "model.layers.21.mlp.gate_proj",
+      "out_features": 14336,
+      "output_sha256": "269e10ded835e6c1382d48f053bf3d72d385b0ef1f97bb058f857fb5ec6cdcf3",
+      "parent_rank": 2548
+    },
+    {
+      "coordinate_dim": 2230,
+      "in_features": 14336,
+      "input_sha256": "d6337fd058a36237bdc5b6d815d98d2d81b0af0cd7b7792d810562891163a32b",
+      "module": "model.layers.21.mlp.down_proj",
+      "out_features": 4096,
+      "output_sha256": "0d7c777e11d20742bd40732d3396d5ac993618fee3283647cc56bdb15b62ecac",
+      "parent_rank": 2548
+    },
+    {
+      "coordinate_dim": 2230,
+      "in_features": 4096,
+      "input_sha256": "95ff998d4b3fa37d46188e6c0ec3e97c57a63d708cb2c894a62627e024d537d9",
+      "module": "model.layers.21.mlp.up_proj",
+      "out_features": 14336,
+      "output_sha256": "fe3b508293d352975e1ff2eedaca7d382ea5b489b19f10d7bab75c2dbd32e7b6",
+      "parent_rank": 2548
+    },
+    {
+      "coordinate_dim": 1433,
+      "in_features": 4096,
+      "input_sha256": "917d0d93d2ad3f1c252dc6894dc54358357f327e1d48e6a730c448e6eac3f422",
+      "module": "model.layers.22.self_attn.q_proj",
+      "out_features": 4096,
+      "output_sha256": "2e9f0f983b5ac13b8b69e30e6c143b10e2f08b42b1db69c49dc3c77653c5d1f1",
+      "parent_rank": 1638
+    },
+    {
+      "coordinate_dim": 573,
+      "in_features": 4096,
+      "input_sha256": "d3d0b4ca308ec46df0d477b537d1a3f6d3d8750011f70e6cb48fc8953be1cbbd",
+      "module": "model.layers.22.self_attn.k_proj",
+      "out_features": 1024,
+      "output_sha256": "8a756aeab599a7ec4e6e728ea4620b215fa74ae340d2513be1a0027585487fef",
+      "parent_rank": 655
+    },
+    {
+      "coordinate_dim": 573,
+      "in_features": 4096,
+      "input_sha256": "28ae42b3b122c426ea33a46752d632fec2c3257c2f0c6839256c97f8541a7f4e",
+      "module": "model.layers.22.self_attn.v_proj",
+      "out_features": 1024,
+      "output_sha256": "9dbbaddd6df9827bf7afdb9cdbea2b73bb5056a2a56d72b5176c747db978b438",
+      "parent_rank": 655
+    },
+    {
+      "coordinate_dim": 1433,
+      "in_features": 4096,
+      "input_sha256": "59eaef8c0af60b47d7de97f116d1883dfcbb579311593f326d640fd0f1290906",
+      "module": "model.layers.22.self_attn.o_proj",
+      "out_features": 4096,
+      "output_sha256": "b718c5b8ce4e755b629f8a09db6c5bebc3dce088cc93908b6e308a77a1363b1f",
+      "parent_rank": 1638
+    },
+    {
+      "coordinate_dim": 2230,
+      "in_features": 4096,
+      "input_sha256": "e5cf0036d3475e42ee4898321009c8de75b0127517742ad68f4ea727338efaa0",
+      "module": "model.layers.22.mlp.gate_proj",
+      "out_features": 14336,
+      "output_sha256": "999942f6c2563708fb782f02e3fe2c9c2518c5cf6f3bff1bcd2fcfdb93f91e56",
+      "parent_rank": 2548
+    },
+    {
+      "coordinate_dim": 2230,
+      "in_features": 14336,
+      "input_sha256": "8bf7dad50d5f1a130ad9909a270473e8a149b0733757637582d521f031a1fed6",
+      "module": "model.layers.22.mlp.down_proj",
+      "out_features": 4096,
+      "output_sha256": "8e39a2ac1d79c5e82b4e69fb2b712350182507e25faa6862e44dd34c5561d7f3",
+      "parent_rank": 2548
+    },
+    {
+      "coordinate_dim": 2230,
+      "in_features": 4096,
+      "input_sha256": "23228bf55e3853da69d35f46eda3914a7ee8838e7df0e1360508c51f4c0ce01b",
+      "module": "model.layers.22.mlp.up_proj",
+      "out_features": 14336,
+      "output_sha256": "dc500fe21ca146bb4c272bde0b0f7dd0a2fb9082386086913a40911ae0433f4f",
+      "parent_rank": 2548
+    },
+    {
+      "coordinate_dim": 1433,
+      "in_features": 4096,
+      "input_sha256": "8aec10b5d82645dd56ba2b41fe2653d1172d814d69216f458ea6a431cb451b0d",
+      "module": "model.layers.23.self_attn.q_proj",
+      "out_features": 4096,
+      "output_sha256": "f23d0abeccc84b5cc41f22e6de30b905e30650e338ec4501051a5e90812de069",
+      "parent_rank": 1638
+    },
+    {
+      "coordinate_dim": 573,
+      "in_features": 4096,
+      "input_sha256": "3ccc2d16bc299ecd142b5b49e03efdfae19d0cceb444d1777ffa934deb796833",
+      "module": "model.layers.23.self_attn.k_proj",
+      "out_features": 1024,
+      "output_sha256": "f5088d42b42a6a12d9f64e345a4b0b05d683399edc4b06b757f7423d583c6d35",
+      "parent_rank": 655
+    },
+    {
+      "coordinate_dim": 573,
+      "in_features": 4096,
+      "input_sha256": "58eb9f806f7a0d3a8e3206aa792492eee2488e7e910f8620d4f547b01b6ad214",
+      "module": "model.layers.23.self_attn.v_proj",
+      "out_features": 1024,
+      "output_sha256": "0ecee6b182c03c0bad1c5fb904bb5072eaf1b904ef4fa6375c94734e412318ce",
+      "parent_rank": 655
+    },
+    {
+      "coordinate_dim": 1433,
+      "in_features": 4096,
+      "input_sha256": "b3262ac60c96c750798541cb21cbf9f0b47178f3004d95316bde68dcf4a18a9e",
+      "module": "model.layers.23.self_attn.o_proj",
+      "out_features": 4096,
+      "output_sha256": "8a383e4066ca9f25b65bdd9475893b63192cf55505de50134d6ec28c355b8b0f",
+      "parent_rank": 1638
+    },
+    {
+      "coordinate_dim": 2230,
+      "in_features": 4096,
+      "input_sha256": "e96d5e620ddf6a0874378a43a842ea2893b0c7832d9da5e8c793d8349cee73b2",
+      "module": "model.layers.23.mlp.gate_proj",
+      "out_features": 14336,
+      "output_sha256": "1484508a69fc6dd8028eb21c8374bc5006265aa471b5671e67337b194552d16d",
+      "parent_rank": 2548
+    },
+    {
+      "coordinate_dim": 2230,
+      "in_features": 14336,
+      "input_sha256": "1a8181c7811d53903ba0ab8fe2d6c015e7ce752c65bab72e254039df864e9e55",
+      "module": "model.layers.23.mlp.down_proj",
+      "out_features": 4096,
+      "output_sha256": "f4d81d603c27748c50878d9aef20000a396a65be42f400497b2c714a86ae28d8",
+      "parent_rank": 2548
+    },
+    {
+      "coordinate_dim": 2230,
+      "in_features": 4096,
+      "input_sha256": "8026af76250f3e7c6c4b4ee963167173f512b68cc4af2fe2c64d6571364cb944",
+      "module": "model.layers.23.mlp.up_proj",
+      "out_features": 14336,
+      "output_sha256": "943f16065ff11b8273e6ccad0d30129f6e9d101a00f468c2b0fa2fbe21d66809",
+      "parent_rank": 2548
+    },
+    {
+      "coordinate_dim": 1433,
+      "in_features": 4096,
+      "input_sha256": "504cedfc392390c002d6e710d698a5385b059de9dcdb5034e525f0153d52a188",
+      "module": "model.layers.24.self_attn.q_proj",
+      "out_features": 4096,
+      "output_sha256": "9ebeb4c8baa5d7c960a726db1d20dbb16aceb57f9ca875402fa8c8b451129ff3",
+      "parent_rank": 1638
+    },
+    {
+      "coordinate_dim": 573,
+      "in_features": 4096,
+      "input_sha256": "420defd8fd09c89652ad79f5a0cb87111fd9848c4a980126cf9e0e4771117c36",
+      "module": "model.layers.24.self_attn.k_proj",
+      "out_features": 1024,
+      "output_sha256": "119373a8ec386ed5364d2913c9a66e311747775acf33520ade9e00910bc710a6",
+      "parent_rank": 655
+    },
+    {
+      "coordinate_dim": 573,
+      "in_features": 4096,
+      "input_sha256": "7f91934a4f06e495c81fe1351c716a46722b635118a59988c94ef41ff5af81e0",
+      "module": "model.layers.24.self_attn.v_proj",
+      "out_features": 1024,
+      "output_sha256": "278aab5a889b804be7994f4df7bac767076e9cbbb46f8510c0f0057f96323279",
+      "parent_rank": 655
+    },
+    {
+      "coordinate_dim": 1433,
+      "in_features": 4096,
+      "input_sha256": "c859ccae7ad8cec84ea40d3f912afc5f48b0271548e735528b34d34e259047b9",
+      "module": "model.layers.24.self_attn.o_proj",
+      "out_features": 4096,
+      "output_sha256": "ff1f0853808512bbc920f5e4c23515bf0131f743b33f4f262011d37c9bb493ef",
+      "parent_rank": 1638
+    },
+    {
+      "coordinate_dim": 2230,
+      "in_features": 4096,
+      "input_sha256": "9ea6f8bbb859fe50d936f554de490986c91393f6ce3924f40c11543a6a18a2ac",
+      "module": "model.layers.24.mlp.gate_proj",
+      "out_features": 14336,
+      "output_sha256": "f8fbff13d8572c5042b480d420eec6f8aaa73eae0fefa75e20d94498bbd02578",
+      "parent_rank": 2548
+    },
+    {
+      "coordinate_dim": 2230,
+      "in_features": 14336,
+      "input_sha256": "74ff794c3529a0010777ee20a56cd7457a45a7421a040faae294dd75e49c6490",
+      "module": "model.layers.24.mlp.down_proj",
+      "out_features": 4096,
+      "output_sha256": "0166573f0e31f381e1f55283f87bbb7ccb220a28997a9a3d62fd8fe0df3b60d7",
+      "parent_rank": 2548
+    },
+    {
+      "coordinate_dim": 2230,
+      "in_features": 4096,
+      "input_sha256": "09c3350aaa68c1e865d534cbd681edf9c60ad090376a63bea2a6f5cf6ae37df1",
+      "module": "model.layers.24.mlp.up_proj",
+      "out_features": 14336,
+      "output_sha256": "53d36faf8b267a752b93d35fd511eacfcf062c8103f6fe30e9dc6c285fa2eecc",
+      "parent_rank": 2548
+    },
+    {
+      "coordinate_dim": 1433,
+      "in_features": 4096,
+      "input_sha256": "fedc04797af4630830021292c16d782122de5f6d3d94a3f27a7983a401f8a80d",
+      "module": "model.layers.25.self_attn.q_proj",
+      "out_features": 4096,
+      "output_sha256": "f1f83a850f422130ab52e71740be13448595072d09c580e2484240458aa81bd3",
+      "parent_rank": 1638
+    },
+    {
+      "coordinate_dim": 573,
+      "in_features": 4096,
+      "input_sha256": "2d2e472a40f42b8ddabe9411f8f5ec003a6f572210da9417f2efac6c28fcffea",
+      "module": "model.layers.25.self_attn.k_proj",
+      "out_features": 1024,
+      "output_sha256": "95cf4705b757c44e6bbd7bdbc5d2b9aff74c771eae616abe0add009056a24ab5",
+      "parent_rank": 655
+    },
+    {
+      "coordinate_dim": 573,
+      "in_features": 4096,
+      "input_sha256": "838d92bbd28f61518b73d1feeaf809bbbc129aaf612c2a63f32599ebce006dab",
+      "module": "model.layers.25.self_attn.v_proj",
+      "out_features": 1024,
+      "output_sha256": "9e630912751dce04d97252d47a0bd1dda48d187f3d9c42338ffa387fc6eae6df",
+      "parent_rank": 655
+    },
+    {
+      "coordinate_dim": 1433,
+      "in_features": 4096,
+      "input_sha256": "513396c32fc1e1a20c2f5a8a5f70f1574068866d72bd14409570a9505feda7bd",
+      "module": "model.layers.25.self_attn.o_proj",
+      "out_features": 4096,
+      "output_sha256": "688db2c98efa5ff7d2644fb3c235492398936d80834134afcf9411873ef0c38a",
+      "parent_rank": 1638
+    },
+    {
+      "coordinate_dim": 2230,
+      "in_features": 4096,
+      "input_sha256": "2fd2f3ca688362520f54c1dce8b1249fcd90e9a785fddc87e64857462e80d821",
+      "module": "model.layers.25.mlp.gate_proj",
+      "out_features": 14336,
+      "output_sha256": "3f9594b979bd49f0efc604c1a70b5d59883de7d8f1191dbc6feea1ecfec6ba50",
+      "parent_rank": 2548
+    },
+    {
+      "coordinate_dim": 2230,
+      "in_features": 14336,
+      "input_sha256": "474ee01d34de84a4c7a832a12306d2a170f6b17d8d7bab8f74726e33a22ebeb4",
+      "module": "model.layers.25.mlp.down_proj",
+      "out_features": 4096,
+      "output_sha256": "4c1e471ed5fa109a1d9946a1882647dc0e2728077561f5203060e723f2332c0f",
+      "parent_rank": 2548
+    },
+    {
+      "coordinate_dim": 2230,
+      "in_features": 4096,
+      "input_sha256": "c1ea158eea48faa2ed2b4ae5d85976a696e8878e29f1a25180a8fd50f5c71b63",
+      "module": "model.layers.25.mlp.up_proj",
+      "out_features": 14336,
+      "output_sha256": "ce5a7f5d57caa91be3bbed3df5bc7d9b5d2814561f23b3c9f716c43c5aa821cf",
+      "parent_rank": 2548
+    },
+    {
+      "coordinate_dim": 1433,
+      "in_features": 4096,
+      "input_sha256": "47917f9aff52ff381c23d69084653ad0ec595f7e002d3aa94aee090858c424d5",
+      "module": "model.layers.26.self_attn.q_proj",
+      "out_features": 4096,
+      "output_sha256": "9f5204bfafc28ae3aa63f15b268f35090cd3de9024d3b74754d72fb4e304fe55",
+      "parent_rank": 1638
+    },
+    {
+      "coordinate_dim": 573,
+      "in_features": 4096,
+      "input_sha256": "4fc06b796034772b042e7e66a416f5777350d093df1003f9cdea3085dae83b4f",
+      "module": "model.layers.26.self_attn.k_proj",
+      "out_features": 1024,
+      "output_sha256": "82ed0fa7fa824175cba1f5e4db1e169488fb8af302511055661ee399b9afa8a6",
+      "parent_rank": 655
+    },
+    {
+      "coordinate_dim": 573,
+      "in_features": 4096,
+      "input_sha256": "385bcf5c94b596843123e31887c8f7a86c5c16ae7aa676a3b6c3d487f1a2cbff",
+      "module": "model.layers.26.self_attn.v_proj",
+      "out_features": 1024,
+      "output_sha256": "441d69f921e55ba0c7d0df66835e44717f4fb4079b41dac85d816037478b4f3e",
+      "parent_rank": 655
+    },
+    {
+      "coordinate_dim": 1433,
+      "in_features": 4096,
+      "input_sha256": "afff7e2ff640fd65240f171ebc999ad390f182da94edcc6f7f443b52d72a8044",
+      "module": "model.layers.26.self_attn.o_proj",
+      "out_features": 4096,
+      "output_sha256": "e54de1f32080c2ba99c4c9c206a1b84d5b36f21d4c68b63832eb45a87195a377",
+      "parent_rank": 1638
+    },
+    {
+      "coordinate_dim": 2230,
+      "in_features": 4096,
+      "input_sha256": "90e2a0da0c4e9ed04c6f87e84d9e8c3c07a13e9dc4e8a29e060103f1d56e0f6c",
+      "module": "model.layers.26.mlp.gate_proj",
+      "out_features": 14336,
+      "output_sha256": "81ce4080e2b0b2919b4f41ce434c522ea7d53c4c511528807b9a50d332cfa4de",
+      "parent_rank": 2548
+    },
+    {
+      "coordinate_dim": 2230,
+      "in_features": 14336,
+      "input_sha256": "82c7abf12dd50d15275e5d69a70852b9f6152fd110c45abdb913807d084999fd",
+      "module": "model.layers.26.mlp.down_proj",
+      "out_features": 4096,
+      "output_sha256": "360090e96bf447d25235a722dac919c265fd90c7ae07c13fad263b4e5cb80a38",
+      "parent_rank": 2548
+    },
+    {
+      "coordinate_dim": 2230,
+      "in_features": 4096,
+      "input_sha256": "0822cdf95c75ae03abc94d8e05ff79fe396a1086b0f7be1b5bbd42f563ad0d7d",
+      "module": "model.layers.26.mlp.up_proj",
+      "out_features": 14336,
+      "output_sha256": "dc8a5e8bd140d9d7f5d7f3efc37a029d49542a5d6f50602a18dca5cfa9494e9e",
+      "parent_rank": 2548
+    },
+    {
+      "coordinate_dim": 1433,
+      "in_features": 4096,
+      "input_sha256": "272395d9c8f4c305abdeea44cb668d10272c09ff22131c0048ff08a0e58a8117",
+      "module": "model.layers.27.self_attn.q_proj",
+      "out_features": 4096,
+      "output_sha256": "06e57326951ebc8a89625adb849fd57aef8c5ee9c8a1e7d314febe2c207df903",
+      "parent_rank": 1638
+    },
+    {
+      "coordinate_dim": 573,
+      "in_features": 4096,
+      "input_sha256": "7f670e7aaca5d518bf353af055ed28fcce1f4400940f83db4aa96ff38397812e",
+      "module": "model.layers.27.self_attn.k_proj",
+      "out_features": 1024,
+      "output_sha256": "a0f6d5f64734ed34b27d670a99c220a4243968cbc402d5dc2570a96ee005ca2b",
+      "parent_rank": 655
+    },
+    {
+      "coordinate_dim": 573,
+      "in_features": 4096,
+      "input_sha256": "f50333d7ee03bbf4a1f507aa1814e62a118647e87be70bf5e4d34851946f45d0",
+      "module": "model.layers.27.self_attn.v_proj",
+      "out_features": 1024,
+      "output_sha256": "15b79431f3b910faab79cdd94e0842af6d6a3bd67531f3fda6c9b9924715112b",
+      "parent_rank": 655
+    },
+    {
+      "coordinate_dim": 1433,
+      "in_features": 4096,
+      "input_sha256": "03e9fbf5d9ba0b23500a4189bfd398b96419055d2af7f5f7f8d45a8b072b9dc4",
+      "module": "model.layers.27.self_attn.o_proj",
+      "out_features": 4096,
+      "output_sha256": "f478bc23f5ae9f630d434a7a2ca0db5c71052eaa47caf0f514fdde43d34eeced",
+      "parent_rank": 1638
+    },
+    {
+      "coordinate_dim": 2230,
+      "in_features": 4096,
+      "input_sha256": "7ec2124999187ab7c09962c2aaefec0deb34f75972d82bd453f6295a5ab76794",
+      "module": "model.layers.27.mlp.gate_proj",
+      "out_features": 14336,
+      "output_sha256": "be5cb4e5f8b6197d7d2ccac2944d0e4be36be4e475385b1703acbebc9684ffee",
+      "parent_rank": 2548
+    },
+    {
+      "coordinate_dim": 2230,
+      "in_features": 14336,
+      "input_sha256": "8ff4403de28b22e8bcde9eed6a3c2ac1d0f7d4b5dc8ec51642df5939c4035d9e",
+      "module": "model.layers.27.mlp.down_proj",
+      "out_features": 4096,
+      "output_sha256": "4dc205f583126fc0b8d095e824524f98d32a7b932c2082887a6957edefd9cb30",
+      "parent_rank": 2548
+    },
+    {
+      "coordinate_dim": 2230,
+      "in_features": 4096,
+      "input_sha256": "bf77ca6ad011f3385a3c4c44819ee6814133fd933e5d20c2cb1d920951fbd128",
+      "module": "model.layers.27.mlp.up_proj",
+      "out_features": 14336,
+      "output_sha256": "c6f19149380aba4eb0f48532edf47544c40cb1d713923164c90bec618e5583e3",
+      "parent_rank": 2548
+    },
+    {
+      "coordinate_dim": 1433,
+      "in_features": 4096,
+      "input_sha256": "650f8ebf9b921dfcf499cdebc1580610bc241487597316b026875aaf40696b9c",
+      "module": "model.layers.28.self_attn.q_proj",
+      "out_features": 4096,
+      "output_sha256": "900f06a5120092009b5593d6b9564d13d8a4bee0e37370a787a6fe4598c6d8e8",
+      "parent_rank": 1638
+    },
+    {
+      "coordinate_dim": 573,
+      "in_features": 4096,
+      "input_sha256": "8fcf94d2e2b610f0bdda3bc2762413917ba2fdf060db28ae0d77f5afed299f67",
+      "module": "model.layers.28.self_attn.k_proj",
+      "out_features": 1024,
+      "output_sha256": "6cd3bc3485dac5123df006aaa16054bafcda4966be52d3c6ab1e4f0a6a6dae6d",
+      "parent_rank": 655
+    },
+    {
+      "coordinate_dim": 573,
+      "in_features": 4096,
+      "input_sha256": "dd2ef35541db89ae949a1239d82a312a9d8b1244cf1a059f9a6c387e7021d351",
+      "module": "model.layers.28.self_attn.v_proj",
+      "out_features": 1024,
+      "output_sha256": "24bc269bf40857f45f0ca744dffab8924e63cf36f299cf7f1a9601639d0944eb",
+      "parent_rank": 655
+    },
+    {
+      "coordinate_dim": 1433,
+      "in_features": 4096,
+      "input_sha256": "501f5527ca8aeddc0232cdf654e37e20de55a7e37915d4ad38ede3320d57b86e",
+      "module": "model.layers.28.self_attn.o_proj",
+      "out_features": 4096,
+      "output_sha256": "e7f13797db77dead60e7b18f7ae04f8e8c7d85af66d0357d5cff7d967e10e72a",
+      "parent_rank": 1638
+    },
+    {
+      "coordinate_dim": 2230,
+      "in_features": 4096,
+      "input_sha256": "b5fa515491dd0587db42ebe59595fa0db85f3fb0c1db3d173598f309897dd009",
+      "module": "model.layers.28.mlp.gate_proj",
+      "out_features": 14336,
+      "output_sha256": "89210dad75feed26bc8decc9c2398c97193562b21b36dfcb351979cb4cb1ac58",
+      "parent_rank": 2548
+    },
+    {
+      "coordinate_dim": 2230,
+      "in_features": 14336,
+      "input_sha256": "bde748fc8001a2fb20671b0a627217bf1a5528b763680706e989b450094802d9",
+      "module": "model.layers.28.mlp.down_proj",
+      "out_features": 4096,
+      "output_sha256": "e5a48ef2127de4d096f5b7cbd7d27159e8142899db287f3ec5510de7d0dd2052",
+      "parent_rank": 2548
+    },
+    {
+      "coordinate_dim": 2230,
+      "in_features": 4096,
+      "input_sha256": "9528cd19ff829afb9df60234acaf1351df755d745817ce8f67307cf2515c12a5",
+      "module": "model.layers.28.mlp.up_proj",
+      "out_features": 14336,
+      "output_sha256": "d8bea459067317d41e4dbb394d6b7ee8a54ac9629baea8e2d7a24a569bee51df",
+      "parent_rank": 2548
+    },
+    {
+      "coordinate_dim": 1433,
+      "in_features": 4096,
+      "input_sha256": "a838aa3b91360dbe1fb604b977a24aeb4b2e842d28caf729b423df1b227f0ebb",
+      "module": "model.layers.29.self_attn.q_proj",
+      "out_features": 4096,
+      "output_sha256": "265326cfdef90d30b5c8cec358fbbed974bf21f2e3402208e565ddd9fbe36d59",
+      "parent_rank": 1638
+    },
+    {
+      "coordinate_dim": 573,
+      "in_features": 4096,
+      "input_sha256": "606bd15a5569637a6a230352f3ef01066c9721c64648fdddbde57953f75af280",
+      "module": "model.layers.29.self_attn.k_proj",
+      "out_features": 1024,
+      "output_sha256": "89f65f2ee69ae41af9e3724ce7b877aacd1452b106b0e70ab2761be65eff3c77",
+      "parent_rank": 655
+    },
+    {
+      "coordinate_dim": 573,
+      "in_features": 4096,
+      "input_sha256": "c1455adda264a5e10c1ab6b6c19612258c4d7e1b3d7476bd2d545f00a35f83a8",
+      "module": "model.layers.29.self_attn.v_proj",
+      "out_features": 1024,
+      "output_sha256": "09bbf2a27667d57e98a4264c6df1dbb55bda0f11258bf012e8f02ca66e087b1c",
+      "parent_rank": 655
+    },
+    {
+      "coordinate_dim": 1433,
+      "in_features": 4096,
+      "input_sha256": "171eed5bc361742f816a1b54c72a6fd2b3e21945e56a2f6acd14e2d7296e948a",
+      "module": "model.layers.29.self_attn.o_proj",
+      "out_features": 4096,
+      "output_sha256": "5dded4af4ed8301af48f003cf89ec6911cafab2bd50a95caffdde885da2a1aaa",
+      "parent_rank": 1638
+    },
+    {
+      "coordinate_dim": 2230,
+      "in_features": 4096,
+      "input_sha256": "ecd025d6c7dd62e5eeb49d4169e209329b9ff36de4041b67bd1f2f5d36681514",
+      "module": "model.layers.29.mlp.gate_proj",
+      "out_features": 14336,
+      "output_sha256": "da0646200182caf7dce4e8be2f8b0b91baa1973893f4eaf6db2e3ee2fb09d595",
+      "parent_rank": 2548
+    },
+    {
+      "coordinate_dim": 2230,
+      "in_features": 14336,
+      "input_sha256": "ab8fb33ab1b85de19311cc98ccde828619c7ff59205bc2f1a4e5ac901fd47047",
+      "module": "model.layers.29.mlp.down_proj",
+      "out_features": 4096,
+      "output_sha256": "c940f3ad550f7769b3249f92170dea8a206ffdc79f31bf10de0339263d0b526d",
+      "parent_rank": 2548
+    },
+    {
+      "coordinate_dim": 2230,
+      "in_features": 4096,
+      "input_sha256": "14b93249fb4b9d9b440b092550f5aeabe5d7ab61549359ed7b3fd8fce0913dc2",
+      "module": "model.layers.29.mlp.up_proj",
+      "out_features": 14336,
+      "output_sha256": "a2717cf958d72fcffd092f9a898e2cc59b20929cba0141cf34148e803fe7cae8",
+      "parent_rank": 2548
+    },
+    {
+      "coordinate_dim": 1433,
+      "in_features": 4096,
+      "input_sha256": "b84c4504fdd0f3b3f4eed38044262ad12e2b7f22b664f5f129ccfd6ed7d29e81",
+      "module": "model.layers.30.self_attn.q_proj",
+      "out_features": 4096,
+      "output_sha256": "191599057fce0a35defb6a1fe3e225342ea5b17695a0ab02a41face9d1ff80a3",
+      "parent_rank": 1638
+    },
+    {
+      "coordinate_dim": 573,
+      "in_features": 4096,
+      "input_sha256": "0e4b60895b421d4de54b7786d69b584553f4764b69011d1c014ee6260554ea5e",
+      "module": "model.layers.30.self_attn.k_proj",
+      "out_features": 1024,
+      "output_sha256": "2f1644bfca75aecabdf5c67e6a3dbdb16adabbf73038132847eb7a195eaae1fa",
+      "parent_rank": 655
+    },
+    {
+      "coordinate_dim": 573,
+      "in_features": 4096,
+      "input_sha256": "925c3ba6d401453553650cb03715fcf98188f184d98fe35574b683cfb28bd4cf",
+      "module": "model.layers.30.self_attn.v_proj",
+      "out_features": 1024,
+      "output_sha256": "65875312b9dcbad791997c043fcc7b17c9eea250c58b035a9702786a4aa3b239",
+      "parent_rank": 655
+    },
+    {
+      "coordinate_dim": 1433,
+      "in_features": 4096,
+      "input_sha256": "0978c7c84fbd635df20f6f73765901f56c68531fbcf685bea69eeeedce86a5f9",
+      "module": "model.layers.30.self_attn.o_proj",
+      "out_features": 4096,
+      "output_sha256": "95166912c3a8b985eb76d077db62d901b1c8a8a8f2ed71eb8221d4510acce846",
+      "parent_rank": 1638
+    },
+    {
+      "coordinate_dim": 2230,
+      "in_features": 4096,
+      "input_sha256": "cc3d164caed841d33fa370f52b19aa888258f843f060b374d5f2c448fd899e5c",
+      "module": "model.layers.30.mlp.gate_proj",
+      "out_features": 14336,
+      "output_sha256": "6297b6f18764c6eccff7184f495056ae7752971d3e482f7c241921e0141b022b",
+      "parent_rank": 2548
+    },
+    {
+      "coordinate_dim": 2230,
+      "in_features": 14336,
+      "input_sha256": "9d01419ae974f4e3b11771a9b960d0bdc940946719405e7211307664e4a3fd77",
+      "module": "model.layers.30.mlp.down_proj",
+      "out_features": 4096,
+      "output_sha256": "ffdb70d206fc62b819db0bd1170f9b1c3f9724f63935adec224e9b78c27aabb9",
+      "parent_rank": 2548
+    },
+    {
+      "coordinate_dim": 2230,
+      "in_features": 4096,
+      "input_sha256": "852e9ab7d1a8ea6180726f37674c13bb5e82ca9565c6c24a9c2b136b8ee2135f",
+      "module": "model.layers.30.mlp.up_proj",
+      "out_features": 14336,
+      "output_sha256": "aaf36d527372569f8e418efe76d0ab7a2164702a1c5178d265fe898e56249782",
+      "parent_rank": 2548
+    },
+    {
+      "coordinate_dim": 1433,
+      "in_features": 4096,
+      "input_sha256": "ddbf749c9cf79bcfb48a99c4d23b200cd565564c49c413122363f2e1d668ddaf",
+      "module": "model.layers.31.self_attn.q_proj",
+      "out_features": 4096,
+      "output_sha256": "a1068399691b8b96dad2603d7464b5e3d91e6f076bff96b2b30a60d6100a7d45",
+      "parent_rank": 1638
+    },
+    {
+      "coordinate_dim": 573,
+      "in_features": 4096,
+      "input_sha256": "a209355d8c0a89aff68af653df1463b4780a75d5af61f707116c1bffdc19bc74",
+      "module": "model.layers.31.self_attn.k_proj",
+      "out_features": 1024,
+      "output_sha256": "d0cc94bfa8f2b28e4181b9c87b54879e0541c9974ffa9d30d072c26154bc3242",
+      "parent_rank": 655
+    },
+    {
+      "coordinate_dim": 573,
+      "in_features": 4096,
+      "input_sha256": "fb02c8eea11c360057ccf0e1b6e1f0e62a688dbd736ce611d04368653fae8af2",
+      "module": "model.layers.31.self_attn.v_proj",
+      "out_features": 1024,
+      "output_sha256": "0dfc6297737f26d0365975ab5a802078325fc4296dcbec205759443a1355dd8f",
+      "parent_rank": 655
+    },
+    {
+      "coordinate_dim": 1433,
+      "in_features": 4096,
+      "input_sha256": "f0e39588bd9129a7c91426b757a54d74b5f0a461a4c41508d24ca830d9d9e187",
+      "module": "model.layers.31.self_attn.o_proj",
+      "out_features": 4096,
+      "output_sha256": "78ff7e139cba18bb8d5ea0397f3108ce0a267d7ba01d18aee0efe8c3f49958bc",
+      "parent_rank": 1638
+    },
+    {
+      "coordinate_dim": 2230,
+      "in_features": 4096,
+      "input_sha256": "7335728c0dc6cf602670645a51bd2551d4ed6e3580d13a72b51536b00f2cd758",
+      "module": "model.layers.31.mlp.gate_proj",
+      "out_features": 14336,
+      "output_sha256": "a992116b9e9748a48fe58b8d2d266e34d93224295e2c7c3566c51fffbd119723",
+      "parent_rank": 2548
+    },
+    {
+      "coordinate_dim": 2230,
+      "in_features": 14336,
+      "input_sha256": "ea7b34685b5a65aa5effdbb43c65296852108afa6d3c2c88cf9dc361822bb34c",
+      "module": "model.layers.31.mlp.down_proj",
+      "out_features": 4096,
+      "output_sha256": "d872868be152a91c972d765db95f3447044b2470e70f5adae07c2294d173970f",
+      "parent_rank": 2548
+    },
+    {
+      "coordinate_dim": 2230,
+      "in_features": 4096,
+      "input_sha256": "aa187d34c6f63c8d0cd89f4b0165e34ecbd933aaba5d211ed91f00620ae5a273",
+      "module": "model.layers.31.mlp.up_proj",
+      "out_features": 14336,
+      "output_sha256": "c3bdb17eef6d6a249bb6ae450fb6d3f38ffced8cfbfe3d22a770579b7704e98b",
+      "parent_rank": 2548
+    }
+  ],
+  "rank_rule": "floor(retention * out_dim * in_dim / (out_dim + in_dim)), clamped to [1,min(out,in)]",
+  "status": "complete",
+  "total_parameters": 5935927296,
+  "untouched_parameter_hashes": {
+    "lm_head.weight": "090f81d81da902a6c63671ee46b041d86144564ab564a921a4bd1c5a7a724715",
+    "model.embed_tokens.weight": "90c94d0d053e7e1bd5d93241a814dc124dff8fc9b173e9dd156d3616715d9a7f",
+    "model.layers.0.input_layernorm.weight": "c05f2ee40bc0a8f3e8e945f6ccd5c3269613a1d40b949769a83caa9053752401",
+    "model.layers.0.post_attention_layernorm.weight": "b1f4680ce6c11f8b918f2d7c3647d67b785ee4f638cd9f26f8d36c3a1cfc36f7",
+    "model.layers.1.input_layernorm.weight": "f80ce5698eee462a1813c01111d9ae2b2860db6f73a5a3207c823ba19e434b92",
+    "model.layers.1.post_attention_layernorm.weight": "3438ea45be01c075e312ac685a7ccb8bb3c4da2533df355ffb4066a062e82d3c",
+    "model.layers.10.input_layernorm.weight": "9ed9d4e290f104df080b4ea94c1ccf68b4a16d385c1834f7efba7175283bfe49",
+    "model.layers.10.post_attention_layernorm.weight": "857a68c64ce7162abb1662185e87e68bd18926568a91eba873fc38a2a3cf0a2b",
+    "model.layers.11.input_layernorm.weight": "b907f2e38259526600874e50ed79ef4002c9d38fd82ed5f81ca6bf0f68cbd7ab",
+    "model.layers.11.post_attention_layernorm.weight": "c9a3f16d69c43365ed03317f0756b633bd915b0c7ff357b23919efe4645b7541",
+    "model.layers.12.input_layernorm.weight": "7c2d68b953362134776634d0aa2a5784dfc7a566b2d62a77e0e244c54e241387",
+    "model.layers.12.post_attention_layernorm.weight": "8527af879cced9eac501eb119821bc641bb85973041feaf1db372a44f8f98591",
+    "model.layers.13.input_layernorm.weight": "247e6e47068b55cba967ef13237e39dd20546166ebd2c4c269eeb9ac2934ce0d",
+    "model.layers.13.post_attention_layernorm.weight": "e2086ff72bb3165f3b58e1ec0d439507729cd5adb57c30566b0612f797082497",
+    "model.layers.14.input_layernorm.weight": "9055f44ac5b8b7d2b836f3646769af0a4a3ee50c37ad9293e2a38bc9a391eebe",
+    "model.layers.14.post_attention_layernorm.weight": "e5ae6d0df5d1875de1ace359dae75dbb91c0e8af740c63f5bac0d9f26ff14849",
+    "model.layers.15.input_layernorm.weight": "2c73b0cadae00127b43e266760b2d96a68add6624e688b9fed4c4bf838f438d8",
+    "model.layers.15.post_attention_layernorm.weight": "4cd167dd855091ce9e68ed182f54a605ce5e571b49b44dc2a4d2fc12f039c596",
+    "model.layers.16.input_layernorm.weight": "cbf8c798b1e12f194fa5da53d9388bb3dd614f8d19c6e5e8201bdd3a1da4edc8",
+    "model.layers.16.post_attention_layernorm.weight": "e0aad8492cce8490b7d6510eea059629c70d5cf71cad137e4dc7bf143ee28e83",
+    "model.layers.17.input_layernorm.weight": "3f98949bf36485ad022d55d8ca4f7bc744b003ec256aac845abc7a1b2382975b",
+    "model.layers.17.post_attention_layernorm.weight": "686ea0e8928dc23cd54b997aeeb261353ba1c69a036c1e38f7c53f8287963702",
+    "model.layers.18.input_layernorm.weight": "46732e00ab7f456a8db4437e95852d7905461fb63ac3516a3718fc2a685baa69",
+    "model.layers.18.post_attention_layernorm.weight": "11652fd11eec6c6b499be49b96dde4a0f0dfe2fd065edb375a44485ba03755d5",
+    "model.layers.19.input_layernorm.weight": "18a8aa238934a281f6b611f2f616e1e4b7bbb08af5cd3e95021896ff9d9f33e7",
+    "model.layers.19.post_attention_layernorm.weight": "a7b3fc24b4b0fc5a76e437c21532c77abc0bcf90543ede02d0bd36b1ea5c86c8",
+    "model.layers.2.input_layernorm.weight": "0bcc1aacf6ad43d3247e3e4da4c9f5d952b469d97fd380751b17bd5ca54a0d0c",
+    "model.layers.2.post_attention_layernorm.weight": "25a12dd2949f30109a2d0cfecedf77876d13156a0476075be13e1485abc79080",
+    "model.layers.20.input_layernorm.weight": "3b9835db6383dc1927bdea2bd37336a7314f4aae576563321e7e89133d4d5d95",
+    "model.layers.20.post_attention_layernorm.weight": "94848c536b89391dfea39e2936d2e09110aa7f02e1446c6dbd109d1f4bd655ba",
+    "model.layers.21.input_layernorm.weight": "49fb5588e289e22b47f8ef79470b78fe56ae4f399931e8a3b052d9c4b92b5609",
+    "model.layers.21.post_attention_layernorm.weight": "51e9bfa27b2f3d01c88b259665c3b54f63f2a988dbc04bc69a876305a80d5275",
+    "model.layers.22.input_layernorm.weight": "98581d697aa91ba4db68f6b441f05115da659b29fd5abf4778287d756a8e547c",
+    "model.layers.22.post_attention_layernorm.weight": "6c54b14b3b9ae7c256286d798d32bfe5192c2cf66cd5ddf7c4d9196769b75767",
+    "model.layers.23.input_layernorm.weight": "39baf05eda82d44023dda7b292b96ce0d267c2942384a35da810d0b45e3f8ece",
+    "model.layers.23.post_attention_layernorm.weight": "80a5b6a5c4e10b7b60694767267e46d05423d29de3aec81f65cff5d33c4150b0",
+    "model.layers.24.input_layernorm.weight": "1730239df84c3eb90163456559e62fd27a17c7197d4bb1bdb8b9792b8ef9da01",
+    "model.layers.24.post_attention_layernorm.weight": "97b30e15c23e1bfbbe1113c990cd357c4f3cd1af2facdee2fa8dd502461fa39c",
+    "model.layers.25.input_layernorm.weight": "85b8b0d79d5d0ae06b14700462bedfd91d85b47197734f65280c7d89d60e7411",
+    "model.layers.25.post_attention_layernorm.weight": "8094d53e9bf313bc9af96948c0c3d4983a8e376d70ea046c49eb8debf5819f13",
+    "model.layers.26.input_layernorm.weight": "ec9a244f57549da5e7c9c20830c9ff34275366063982688b6ff9e0499f40bc97",
+    "model.layers.26.post_attention_layernorm.weight": "78fead5be855e383d0d328e1402c08edc29c690f28d493d5ace0c088e185bbf0",
+    "model.layers.27.input_layernorm.weight": "35947ece34b898e5f7aef31b9f3ca0bae8ebef97ef4854ea6f61eeb8d76ca169",
+    "model.layers.27.post_attention_layernorm.weight": "cdf70e270060f657262f6ee56802ff3b5ac80df65d0b5013c1bba8bd7d224d7a",
+    "model.layers.28.input_layernorm.weight": "a7aca29f063ae1b8399bec997849fe8b56a49b7093abe7ed0664223a084eb8a3",
+    "model.layers.28.post_attention_layernorm.weight": "885f5227b6c477d5a855cc367fff3f35a374875e9e862ecf5debf5e408f233f6",
+    "model.layers.29.input_layernorm.weight": "cf6c12e4dd5d7e62187f616aab3a791aa05ed11887a491549ca4301932e5686e",
+    "model.layers.29.post_attention_layernorm.weight": "1471c1852fddd1b47022edf069267729215f68a1ffdba5407ef53c698b060479",
+    "model.layers.3.input_layernorm.weight": "014a01ef97e08598bc4d418b6d91142fa147ba364a7c34097768a59612ddf9f3",
+    "model.layers.3.post_attention_layernorm.weight": "b0bbb863584b84f839ce363eabc0788f44537766c49f9e657607d9be73665860",
+    "model.layers.30.input_layernorm.weight": "4b43c7cedefba4f34128aa04c973faeb4d6a9c7dc8459d3c56d473655fbf85f3",
+    "model.layers.30.post_attention_layernorm.weight": "b5d9a45ace2c39baa3bfadf5aea432cc0b7cbaf20c9cd574aa2c963cbd7643dc",
+    "model.layers.31.input_layernorm.weight": "5fdca8727b1e9f8145b6f700b391597ae73bb1bc1597c0fface5967b088fa1a3",
+    "model.layers.31.post_attention_layernorm.weight": "bfe99942b17344a53feb4595377e51d65708d6770bc0032549975dbbaa065537",
+    "model.layers.4.input_layernorm.weight": "471f4ded27364a38d17ab475f6df918c65752cb9d84c281b132f327c205d602b",
+    "model.layers.4.post_attention_layernorm.weight": "a0083238ae889685e7c46fda17b3859d9227b024bd3de1e0bf78e7bc96908085",
+    "model.layers.5.input_layernorm.weight": "7337480de35aadbed131d8cab06fd4e110a5dc1c5d3179ed31183a392f95aa6f",
+    "model.layers.5.post_attention_layernorm.weight": "e783b5f1b016bec0cca6160de2cef167a582484ab1066297b51343bbe9fa8748",
+    "model.layers.6.input_layernorm.weight": "dcb860363b79ce1d42367bfb99e440bfd6e09d012766db6d9786d1735a7dfdef",
+    "model.layers.6.post_attention_layernorm.weight": "a6b4b4a23071b41c4d09f29d1bf4ab20ea66c5ef84c89ca42f1f8d7d28514a34",
+    "model.layers.7.input_layernorm.weight": "58d356455c98390f802ea1c778ee07567addb75fdcbf6c4f870c2d31f258a3a5",
+    "model.layers.7.post_attention_layernorm.weight": "b28e9a2e802e83c72c8150954d272c6e8a252c811ea09bf08edceaf70e5b61f0",
+    "model.layers.8.input_layernorm.weight": "0fe756da619a3db75adbcc244c77794c26e3ec0053735399adb1bd6797608258",
+    "model.layers.8.post_attention_layernorm.weight": "9f142790d0dce0b4d860a7428558eab7281bd930b32d5bca9216812aec0c0520",
+    "model.layers.9.input_layernorm.weight": "2785c6f885cfc3b3fc7507421a4b96a3b0e4319079a2ef3be536bc774b87f49a",
+    "model.layers.9.post_attention_layernorm.weight": "4c5a6b91069f27e066212b15ae40d1e5ba146160eb8cf702f8db42b4dfc0113a",
+    "model.norm.weight": "49d4c792beddcb32ea782e0633b0bc1be6bd2979410c903fdf5300ddbc4bf55e"
+  }
+}

basis/sources/llama31/gfwsvd_r070/source.json

@@ -0,0 +1,21 @@
+{
+  "backbone": "llama31",
+  "checkpoint_identity": {
+    "checkpoint_sha256": "b43045553b40a499d73f7460fc395622c0b2c737cc81555f8febe995a115f913"
+  },
+  "id": "llama31/gfwsvd_r070",
+  "kind": null,
+  "method": "gfwsvd",
+  "note": "New Task 05 Source from ordered frozen r=0.8 paper factors; same calibration, new checkpoint identity. Evaluate new pressure/recovery results.",
+  "path": "basis/sources/llama31/gfwsvd_r070/model.pt",
+  "provenance_files": [
+    "basis/sources/llama31/gfwsvd_r070/provenance/manifest.json"
+  ],
+  "retention": 0.7,
+  "tasks": [
+    "05"
+  ],
+  "original_path": "/home/zs89/Basis/outputs/source_delivery_20260917/sources/llama31/gfwsvd_r070/model.pt",
+  "validation": "basis/sources/llama31/gfwsvd_r070/validation.json",
+  "weight_repository": "Duke-CEI-SVD/FlashSVD"
+}

basis/sources/llama31/gfwsvd_r070/validation.json

@@ -0,0 +1,17 @@
+{
+  "actual_projection_retention": 0.6999230018028846,
+  "buffers_verified": 1,
+  "calibration_tensor_sha256": "95ca6fc95ac62db49af1d7c4e5b4d7d5e100c29e82376929268bfb14987ad82b",
+  "checkpoint_identity": {
+    "checkpoint_sha256": "b43045553b40a499d73f7460fc395622c0b2c737cc81555f8febe995a115f913"
+  },
+  "elapsed_seconds": 87.71806664485484,
+  "factor_pairs": 224,
+  "finite_forward": true,
+  "parameters_verified": 515,
+  "predicted_adapter_parameters": 10958848,
+  "source_id": "llama31/gfwsvd_r070",
+  "source_json": "/home/zs89/Basis/outputs/source_delivery_20260917/sources/llama31/gfwsvd_r070/source.json",
+  "status": "passed",
+  "suggested_adapter_rank": 16
+}

basis/sources/qwen3/gfwsvd_r050/model.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9c4537ab505b781236533d549c829e5150ac4108116646320a1522b2893fd226
+size 9435690527

basis/sources/qwen3/gfwsvd_r050/source.json

@@ -0,0 +1,21 @@
+{
+  "backbone": "qwen3",
+  "checkpoint_identity": {
+    "checkpoint_sha256": "9c4537ab505b781236533d549c829e5150ac4108116646320a1522b2893fd226"
+  },
+  "id": "qwen3/gfwsvd_r050",
+  "kind": null,
+  "method": "gfwsvd",
+  "note": "New Task 05 Source from ordered frozen r=0.8 paper factors; same calibration, new checkpoint identity. Evaluate new pressure/recovery results.",
+  "path": "basis/sources/qwen3/gfwsvd_r050/model.pt",
+  "provenance_files": [
+    "basis/sources/qwen3/gfwsvd_r050/provenance/manifest.json"
+  ],
+  "retention": 0.5,
+  "tasks": [
+    "05"
+  ],
+  "original_path": "/home/zs89/Basis/outputs/source_delivery_20260917/sources/qwen3/gfwsvd_r050/model.pt",
+  "validation": "basis/sources/qwen3/gfwsvd_r050/validation.json",
+  "weight_repository": "Duke-CEI-SVD/FlashSVD"
+}

basis/sources/qwen3/gfwsvd_r050/validation.json

@@ -0,0 +1,17 @@
+{
+  "actual_projection_retention": 0.49996815557065216,
+  "buffers_verified": 1,
+  "calibration_tensor_sha256": "138935390d601b875fd3624b405634f177e285b0f0dc55985dabaf02de0c7d44",
+  "checkpoint_identity": {
+    "checkpoint_sha256": "9c4537ab505b781236533d549c829e5150ac4108116646320a1522b2893fd226"
+  },
+  "elapsed_seconds": 81.4608334293589,
+  "factor_pairs": 252,
+  "finite_forward": true,
+  "parameters_verified": 651,
+  "predicted_adapter_parameters": 10762560,
+  "source_id": "qwen3/gfwsvd_r050",
+  "source_json": "/home/zs89/Basis/outputs/source_delivery_20260917/sources/qwen3/gfwsvd_r050/source.json",
+  "status": "passed",
+  "suggested_adapter_rank": 20
+}

basis/sources/qwen3/gfwsvd_r070/model.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4aed75e0ba422ec94593d437883917ba8eefb542bfe899b9dc6e4184cd9f9281
+size 12211992995

basis/sources/qwen3/gfwsvd_r070/source.json

@@ -0,0 +1,21 @@
+{
+  "backbone": "qwen3",
+  "checkpoint_identity": {
+    "checkpoint_sha256": "4aed75e0ba422ec94593d437883917ba8eefb542bfe899b9dc6e4184cd9f9281"
+  },
+  "id": "qwen3/gfwsvd_r070",
+  "kind": null,
+  "method": "gfwsvd",
+  "note": "New Task 05 Source from ordered frozen r=0.8 paper factors; same calibration, new checkpoint identity. Evaluate new pressure/recovery results.",
+  "path": "basis/sources/qwen3/gfwsvd_r070/model.pt",
+  "provenance_files": [
+    "basis/sources/qwen3/gfwsvd_r070/provenance/manifest.json"
+  ],
+  "retention": 0.7,
+  "tasks": [
+    "05"
+  ],
+  "original_path": "/home/zs89/Basis/outputs/source_delivery_20260917/sources/qwen3/gfwsvd_r070/model.pt",
+  "validation": "basis/sources/qwen3/gfwsvd_r070/validation.json",
+  "weight_repository": "Duke-CEI-SVD/FlashSVD"
+}

basis/sources/qwen3/gfwsvd_r070/validation.json

@@ -0,0 +1,17 @@
+{
+  "actual_projection_retention": 0.6998237941576086,
+  "buffers_verified": 1,
+  "calibration_tensor_sha256": "138935390d601b875fd3624b405634f177e285b0f0dc55985dabaf02de0c7d44",
+  "checkpoint_identity": {
+    "checkpoint_sha256": "4aed75e0ba422ec94593d437883917ba8eefb542bfe899b9dc6e4184cd9f9281"
+  },
+  "elapsed_seconds": 86.43985069356859,
+  "factor_pairs": 252,
+  "finite_forward": true,
+  "parameters_verified": 651,
+  "predicted_adapter_parameters": 11298960,
+  "source_id": "qwen3/gfwsvd_r070",
+  "source_json": "/home/zs89/Basis/outputs/source_delivery_20260917/sources/qwen3/gfwsvd_r070/source.json",
+  "status": "passed",
+  "suggested_adapter_rank": 15
+}

basis/sources/qwen3/swiftsvd_r040/model.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:1e8b5a3db8feb63dd574f42b78913e6d5d55b0bcc8f4ef630c78db3f12305a6b
+size 8043409011

basis/sources/qwen3/swiftsvd_r040/source.json

@@ -0,0 +1,21 @@
+{
+  "backbone": "qwen3",
+  "checkpoint_identity": {
+    "checkpoint_sha256": "1e8b5a3db8feb63dd574f42b78913e6d5d55b0bcc8f4ef630c78db3f12305a6b"
+  },
+  "id": "qwen3/swiftsvd_r040",
+  "kind": null,
+  "method": "swiftsvd",
+  "note": "New Task 05 Source from ordered frozen r=0.8 paper factors; same calibration, new checkpoint identity. Evaluate new pressure/recovery results.",
+  "path": "basis/sources/qwen3/swiftsvd_r040/model.pt",
+  "provenance_files": [
+    "basis/sources/qwen3/swiftsvd_r040/provenance/manifest.json"
+  ],
+  "retention": 0.4,
+  "tasks": [
+    "05"
+  ],
+  "original_path": "/home/zs89/Basis/outputs/source_delivery_20260917/sources/qwen3/swiftsvd_r040/model.pt",
+  "validation": "basis/sources/qwen3/swiftsvd_r040/validation.json",
+  "weight_repository": "Duke-CEI-SVD/FlashSVD"
+}

basis/sources/qwen3/swiftsvd_r040/validation.json

@@ -0,0 +1,18 @@
+{
+  "actual_projection_retention": 0.39974312160326086,
+  "buffers_verified": 1,
+  "calibration_tensor_sha256": "138935390d601b875fd3624b405634f177e285b0f0dc55985dabaf02de0c7d44",
+  "checkpoint_identity": {
+    "checkpoint_sha256": "1e8b5a3db8feb63dd574f42b78913e6d5d55b0bcc8f4ef630c78db3f12305a6b"
+  },
+  "elapsed_seconds": 28.72612918820232,
+  "existing_fp32_cache_export_parameters_and_buffers_exact": true,
+  "factor_pairs": 252,
+  "finite_forward": true,
+  "parameters_verified": 651,
+  "predicted_adapter_parameters": 11187072,
+  "source_id": "qwen3/swiftsvd_r040",
+  "source_json": "/home/zs89/Basis/outputs/source_delivery_20260917/sources/qwen3/swiftsvd_r040/source.json",
+  "status": "passed",
+  "suggested_adapter_rank": 26
+}

basis/sources/qwen3/swiftsvd_r050/model.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:2a5281ae0ab3a32efe8bf1c0d8906461b0bfae774fba4c38da38996b4250a60f
+size 9435689427

basis/sources/qwen3/swiftsvd_r050/source.json

@@ -0,0 +1,21 @@
+{
+  "backbone": "qwen3",
+  "checkpoint_identity": {
+    "checkpoint_sha256": "2a5281ae0ab3a32efe8bf1c0d8906461b0bfae774fba4c38da38996b4250a60f"
+  },
+  "id": "qwen3/swiftsvd_r050",
+  "kind": null,
+  "method": "swiftsvd",
+  "note": "New Task 05 Source from ordered frozen r=0.8 paper factors; same calibration, new checkpoint identity. Evaluate new pressure/recovery results.",
+  "path": "basis/sources/qwen3/swiftsvd_r050/model.pt",
+  "provenance_files": [
+    "basis/sources/qwen3/swiftsvd_r050/provenance/manifest.json"
+  ],
+  "retention": 0.5,
+  "tasks": [
+    "05"
+  ],
+  "original_path": "/home/zs89/Basis/outputs/source_delivery_20260917/sources/qwen3/swiftsvd_r050/model.pt",
+  "validation": "basis/sources/qwen3/swiftsvd_r050/validation.json",
+  "weight_repository": "Duke-CEI-SVD/FlashSVD"
+}

basis/sources/qwen3/swiftsvd_r050/validation.json

@@ -0,0 +1,17 @@
+{
+  "actual_projection_retention": 0.49996815557065216,
+  "buffers_verified": 1,
+  "calibration_tensor_sha256": "138935390d601b875fd3624b405634f177e285b0f0dc55985dabaf02de0c7d44",
+  "checkpoint_identity": {
+    "checkpoint_sha256": "2a5281ae0ab3a32efe8bf1c0d8906461b0bfae774fba4c38da38996b4250a60f"
+  },
+  "elapsed_seconds": 69.60182892624289,
+  "factor_pairs": 252,
+  "finite_forward": true,
+  "parameters_verified": 651,
+  "predicted_adapter_parameters": 10762560,
+  "source_id": "qwen3/swiftsvd_r050",
+  "source_json": "/home/zs89/Basis/outputs/source_delivery_20260917/sources/qwen3/swiftsvd_r050/source.json",
+  "status": "passed",
+  "suggested_adapter_rank": 20
+}

basis/sources/qwen3/swiftsvd_r070/model.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4069d003526013ba3ecdd1155a86cfffa66a29610132eb822ede22cb776d036a
+size 12211991895

basis/sources/qwen3/swiftsvd_r070/source.json

@@ -0,0 +1,21 @@
+{
+  "backbone": "qwen3",
+  "checkpoint_identity": {
+    "checkpoint_sha256": "4069d003526013ba3ecdd1155a86cfffa66a29610132eb822ede22cb776d036a"
+  },
+  "id": "qwen3/swiftsvd_r070",
+  "kind": null,
+  "method": "swiftsvd",
+  "note": "New Task 05 Source from ordered frozen r=0.8 paper factors; same calibration, new checkpoint identity. Evaluate new pressure/recovery results.",
+  "path": "basis/sources/qwen3/swiftsvd_r070/model.pt",
+  "provenance_files": [
+    "basis/sources/qwen3/swiftsvd_r070/provenance/manifest.json"
+  ],
+  "retention": 0.7,
+  "tasks": [
+    "05"
+  ],
+  "original_path": "/home/zs89/Basis/outputs/source_delivery_20260917/sources/qwen3/swiftsvd_r070/model.pt",
+  "validation": "basis/sources/qwen3/swiftsvd_r070/validation.json",
+  "weight_repository": "Duke-CEI-SVD/FlashSVD"
+}

basis/sources/qwen3/swiftsvd_r070/validation.json

@@ -0,0 +1,17 @@
+{
+  "actual_projection_retention": 0.6998237941576086,
+  "buffers_verified": 1,
+  "calibration_tensor_sha256": "138935390d601b875fd3624b405634f177e285b0f0dc55985dabaf02de0c7d44",
+  "checkpoint_identity": {
+    "checkpoint_sha256": "4069d003526013ba3ecdd1155a86cfffa66a29610132eb822ede22cb776d036a"
+  },
+  "elapsed_seconds": 95.89392327796668,
+  "factor_pairs": 252,
+  "finite_forward": true,
+  "parameters_verified": 651,
+  "predicted_adapter_parameters": 11298960,
+  "source_id": "qwen3/swiftsvd_r070",
+  "source_json": "/home/zs89/Basis/outputs/source_delivery_20260917/sources/qwen3/swiftsvd_r070/source.json",
+  "status": "passed",
+  "suggested_adapter_rank": 15
+}