Reorganize repo: <method>/<stage>/<model> (tr3, svd_llm, svd_llm2 x heal/unheal)

README.md

@@ -1,95 +1,39 @@
 ---
 license: other
 license_name: see-each-subdirectory
-tags:
-  - tensor-decomposition
-  - tensor-ring
-  - model-compression
+tags: [tensor-decomposition, svd, low-rank, model-compression, knowledge-distillation]
 ---
 
-# TR-3: language models compressed with a 3-site Tensor Ring
-
-Every attention and MLP projection replaced by a **3-site Tensor Ring** (cyclic — the ring
-closes with a non-trivial boundary bond, so it is not a train/TT) plus a 5% sparse residual,
-then healed with pure forward-KL distillation from the corresponding dense teacher.
-
-| Directory | Base model | Params | Reduction | Licence |
-|---|---|---:|---:|---|
-| [`llama2-7b/`](./llama2-7b) | Llama-2-7b-chat | 6.74B → 4.02B | 40.30% | `llama2` |
-| [`llama2-13b/`](./llama2-13b) | Llama-2-13b-chat | 13.02B → 7.74B | 40.53% | `llama2` |
-| [`qwen2.5-3b/`](./qwen2.5-3b) | Qwen2.5-3B-Instruct | 3.09B → 2.08B | 32.65% | `qwen-research` — **non-commercial** |
-| [`qwen2.5-7b/`](./qwen2.5-7b) | Qwen2.5-7B-Instruct | 7.62B → 5.19B | 31.89% | `apache-2.0` |
-
-### Controls
-
-| Directory | What it isolates |
-|---|---|
-| [`tt3-llama2-7b/`](./tt3-llama2-7b) | Open-chain **Tensor Train** at the same per-layer budget as `llama2-7b/` |
-| [`tt3-qwen2.5-3b/`](./tt3-qwen2.5-3b) | Same, against `qwen2.5-3b/` |
-| [`svd-llm/`](./svd-llm), [`svd-llm-v2/`](./svd-llm-v2) | SVD-LLM baselines |
-
-Each TT-3 directory ships **both** `state_dict_fp16_unhealed.pt` and
-`state_dict_fp16_healed.pt`; the TR-3 directories ship the healed weights only.
-
-## Does the ring topology matter?
-
-The TT-3 controls are budget-matched to their TR-3 counterparts to within 0.1%, so the only
-structural difference is the closed ring versus the open chain:
-
-| Model | Arm | KL compressed | KL healed | MMLU healed |
-|---|---|---:|---:|---:|
-| LLaMA-2-7B | TT-3 | 6.6868 | 0.4077 | 33.38% |
-| LLaMA-2-7B | **TR-3** | **0.6932** | **0.1822** | **43.82%** |
-| Qwen2.5-3B | TT-3 | 6.3507 | 0.3955 | 43.33% |
-| Qwen2.5-3B | **TR-3** | **0.8888** | **0.1729** | **57.41%** |
-
-Roughly **9x lower KL before healing** and **~2.3x after**, worth 10-14 MMLU points. Each
-control's README states the two confounds that qualify this (a batch-size difference in the
-healed rows, and uneven ALS coverage) and the per-layer evidence that the ALS gap is an order
-of magnitude too small to explain the result.
-
-## Why the compression rates differ
-
-Rates are matched by the **damage compression inflicts**, not by the nominal percentage. Both
-Qwen2.5 sizes collapse to chance at the ~40% the LLaMA-2 models tolerate:
-
-| Model | Reduction | pre-heal KL |
-|---|---:|---:|
-| LLaMA-2-13B | 40.53% | 0.3616 |
-| Qwen2.5-7B | 31.89% | 0.4965 |
-| LLaMA-2-7B | 40.30% | 0.6915 |
-| Qwen2.5-3B | 32.65% | 0.8882 |
-| Qwen2.5-7B | 40.60% | 2.0204 — collapses |
-| Qwen2.5-3B | 40.94% | 3.5810 — collapses |
-
-Two effects are separable and both real: damage falls with model size, and at equal size and
-equal rate the Qwen2.5 architecture is harder to compress than LLaMA-2 (2.0204 vs 0.6915, both
-~7B at ~40%). Scale mitigates the architecture effect without removing it.
-
-## ⚠️ Licences differ per subdirectory
-
-The Hub shows one `license` field, so it is `other`; **the binding terms are per directory**.
-`qwen2.5-3b/` and `tt3-qwen2.5-3b/` are **Qwen RESEARCH LICENSE — research/evaluation only**,
-while `qwen2.5-7b/` is Apache 2.0: Alibaba released the two Qwen sizes under different terms,
-so the permissive licence does not carry across. LLaMA directories are Llama 2 Community
-License. Do not assume one subdirectory's terms apply to another.
+# TR-3 study models: main compressed models and matched-budget baselines
+
+Layout: **<method> / <stage> / <model>**. Method families are `tr3` (our 3-site tensor ring +
+sparse residual), `svd_llm` and `svd_llm_v2` (matched-budget baselines). Stages are `heal`
+(KD-distilled, deployable) and `unheal` (raw post-compression, for measuring the healing delta).
+
+| method | stage | model | params (B) | licence | KD result (probe) |
+|---|---|---|---:|---|---|
+| tr3 | heal | [`llama2-7b`](tr3/heal/llama2-7b) | 4.023 | llama2 | MMLU 43.21* |
+| tr3 | heal | [`llama2-13b`](tr3/heal/llama2-13b) | 7.740 | llama2 | see dir |
+| tr3 | heal | [`qwen2.5-3b`](tr3/heal/qwen2.5-3b) | 2.078 | qwen-research (non-commercial) | see dir |
+| tr3 | unheal | [`llama2-13b`](tr3/unheal/llama2-13b) | 7.740 | llama2 | — |
+| tr3 | unheal | [`llama2-7b`](tr3/unheal/llama2-7b) | (off-cluster archive) | llama2 | — |
+| svd_llm | heal | [`llama2-7b`](svd_llm/heal/llama2-7b) | 4.022 | llama2 | MMLU 39.47 |
+| svd_llm | heal | [`llama2-13b`](svd_llm/heal/llama2-13b) | 7.738 | llama2 | MMLU 46.54 |
+| svd_llm | unheal | [`llama2-7b`](svd_llm/unheal/llama2-7b) | 4.022 | llama2 | — |
+| svd_llm | unheal | [`llama2-13b`](svd_llm/unheal/llama2-13b) | 7.738 | llama2 | — |
+| svd_llm_v2 ‡ | heal | [`llama2-7b`](svd_llm2/heal/llama2-7b) | 4.023 | llama2 | MMLU 40.22 |
+| svd_llm_v2 ‡ | heal | [`llama2-13b`](svd_llm2/heal/llama2-13b) | 7.740 | llama2 | MMLU 44.96 |
+| svd_llm_v2 ‡ | unheal | [`llama2-7b`](svd_llm2/unheal/llama2-7b) | 4.023 | llama2 | — |
+| svd_llm_v2 ‡ | unheal | [`llama2-13b`](svd_llm2/unheal/llama2-13b) | 7.740 | llama2 | — |
+
+\* full-benchmark lm-eval 0.4.12 result; SVD probe columns are limit-40 healing probes here —
+each dir's README/provenance carries the precise protocol.
+
+‡ Official SVD-LLM V2 code was never released; `svd_llm_v2` artifacts are our faithful
+implementation of the published algorithms (per-layer ranks document the allocation).
 
 ## ⚠️ These are NOT standard `transformers` checkpoints
 
-`state_dict_fp16_*.pt` holds **factorisation cores**, not dense `nn.Linear` weights. Loading
-requires rebuilding the architecture with the layer classes shipped in each directory, using
-that directory's `ranks.json`. `AutoModelForCausalLM.from_pretrained()` will **not** work.
-Each subdirectory's README carries a complete loading recipe.
-
-## LLaMA-2-13B SVD baselines (healed vs unhealed)
-
-13B counterparts of the 7B SVD baselines at the matched 13B budget (7.7403B, 59.47% kept —
-same as TR-3 13B), KD-healed with the identical 60M-token protocol. `*-unhealed` dirs hold the
-raw compression weights (no distillation) so the healing delta is directly measurable:
-
-| Directory | Stage | Method | Params |
-|---|---|---|---:|
-| [`svd-llm-13b/`](./svd-llm-13b) | KD-healed | SVD-LLM | 7.738B |
-| [`svd-llm-13b-unhealed/`](./svd-llm-13b-unhealed) | raw compression | SVD-LLM | 7.738B |
-| [`svd-llm-v2-13b/`](./svd-llm-v2-13b) | KD-healed | SVD-LLM V2 ‡ | 7.7403B |
-| [`svd-llm-v2-13b-unhealed/`](./svd-llm-v2-13b-unhealed) | raw compression | SVD-LLM V2 ‡ | 7.7403B |
+`tr3/**` stores **ring cores** (load with each dir's `tensor_formats.py`); `svd_llm*/**` stores
+**low-rank factor pairs** (load with each dir's `lowrank_linear.py`). Weight files are
+uniformly named `state_dict_fp16.pt`. `from_pretrained()` will not work on this repo.

svd_llm/heal/llama2-13b/README.md

@@ -0,0 +1,29 @@
+---
+license: llama2
+base_model: NousResearch/Llama-2-13b-chat-hf
+tags: [svd, low-rank, model-compression, llama2, knowledge-distillation]
+---
+
+# SVD-LLM @ 7.74B (59.47% kept of LLaMA-2-13B-chat), KD-healed 60M tokens
+
+SVD-LLM (ICLR 2025): truncation-aware data whitening + closed-form parameter update, official code (@ 7538cca), uniform 41.58% removal across all 7 projection types of all 40 layers.
+
+- **HEALED checkpoint** (this dir). The matching **UNHEALED** pre-distillation weights are in
+  `svd-llm-13b-unhealed/` — do not mix them up: same `ranks.json`, different `state_dict_fp16.pt` (see
+  `provenance.json` SHA256 here vs the unhealed dir's record).
+- Healing: pure forward-KL distillation from frozen NousResearch/Llama-2-13b-chat-hf teacher,
+  60M tokens, held-out KL **2.0904 -> 0.3007**, probe MMLU (5-shot, limit 40) **24.56% -> 46.54%**.
+- Identical recipe/protocol to the TR-3 13B and the 7B SVD baselines (see their dirs).
+
+## Load
+
+```python
+import json
+from lowrank_linear import build, materialize_all
+model = build("NousResearch/Llama-2-13b-chat-hf", json.load(open("ranks.json")),
+              "state_dict_fp16.pt", device="cuda")
+materialize_all(model)   # inference-only fusion U@V
+```
+
+Full provenance (data mixture + revisions, decontamination, SHA256 of every input and
+checkpoint, 12 x 5M milestone trajectory) in `provenance.json`.

svd_llm/heal/llama2-13b/config.json

@@ -0,0 +1,26 @@
+{
+  "_name_or_path": null,
+  "architectures": [
+    "LlamaForCausalLM"
+  ],
+  "bos_token_id": 1,
+  "eos_token_id": 2,
+  "hidden_act": "silu",
+  "hidden_size": 5120,
+  "initializer_range": 0.02,
+  "intermediate_size": 13824,
+  "max_position_embeddings": 4096,
+  "model_type": "llama",
+  "num_attention_heads": 40,
+  "num_hidden_layers": 40,
+  "num_key_value_heads": 40,
+  "pad_token_id": 0,
+  "pretraining_tp": 1,
+  "rms_norm_eps": 1e-05,
+  "rope_scaling": null,
+  "tie_word_embeddings": false,
+  "torch_dtype": "float16",
+  "transformers_version": "4.31.0.dev0",
+  "use_cache": true,
+  "vocab_size": 32000
+}

svd_llm/heal/llama2-13b/lowrank_linear.py

@@ -0,0 +1,83 @@
+"""Minimal standalone loader for the SVD-baseline compressed LLaMA-2-7B checkpoints.
+
+No dependency beyond torch + transformers. The state_dict stores every compressed
+projection as two factors of a LowRankLinear (A: out x k, B: k x in); ranks.json
+holds the per-layer integer rank of each projection. Untouched parameters
+(embeddings, norms) keep their stock transformers names.
+
+Usage:
+    import json, torch
+    from lowrank_linear import build
+    model = build(
+        base_id="NousResearch/Llama-2-7b-chat-hf",
+        ranks=json.load(open("ranks.json")),
+        state_dict_path="state_dict_fp16.pt",
+        device="cuda", dtype=torch.float16,
+    )
+    tokenizer = AutoTokenizer.from_pretrained("NousResearch/Llama-2-7b-chat-hf")
+    out = model.generate(tokenizer("2+2=", return_tensors="pt").to("cuda").input_ids,
+                         max_new_tokens=16, do_sample=False)
+    print(tokenizer.decode(out[0]))
+"""
+
+from __future__ import annotations
+
+import torch
+import torch.nn as nn
+import torch.nn.functional as F
+
+
+class LowRankLinear(nn.Module):
+    """y = A (B x).  A: out x r, B: r x in.  (LLaMA projections carry no bias.)"""
+
+    def __init__(self, A: torch.Tensor, B: torch.Tensor):
+        super().__init__()
+        self.A = nn.Parameter(A)
+        self.B = nn.Parameter(B)
+
+    def forward(self, x):
+        return F.linear(F.linear(x, self.B), self.A)
+
+    @torch.no_grad()
+    def materialize_(self) -> nn.Linear:
+        """Replace this module with the dense W = A @ B (inference only; never call
+        while training -- it cuts the gradient path)."""
+        W = (self.A @ self.B).to(self.A.dtype)
+        dense = nn.Linear(self.B.shape[1], self.A.shape[0], bias=False,
+                          device=W.device, dtype=W.dtype)
+        dense.weight.data = W
+        return dense
+
+
+def build(base_id: str, ranks: dict, state_dict_path: str,
+          device: str = "cuda", dtype=torch.float16):
+    from transformers import AutoModelForCausalLM
+
+    model = AutoModelForCausalLM.from_pretrained(base_id, torch_dtype=dtype)
+    search = {}
+    for name, mod in model.named_modules():
+        if isinstance(mod, nn.Linear):
+            search[name] = mod
+    for path, k in ranks.items():
+        out_f, in_f = search[path].weight.shape
+        model_mod = model
+        for p in path.split(".")[:-1]:
+            model_mod = getattr(model_mod, p)
+        setattr(model_mod, path.split(".")[-1],
+                LowRankLinear(torch.zeros(out_f, int(k), dtype=dtype),
+                              torch.zeros(int(k), in_f, dtype=dtype)))
+    sd = torch.load(state_dict_path, map_location="cpu", weights_only=True)
+    model.load_state_dict(sd, strict=True)
+    del sd
+    return model.to(device)
+
+
+def materialize_all(model):
+    """Fuse every LowRankLinear back into a dense nn.Linear in place."""
+    for name, mod in list(model.named_modules()):
+        if isinstance(mod, LowRankLinear):
+            parent = model
+            for p in name.split(".")[:-1]:
+                parent = getattr(parent, p)
+            setattr(parent, name.split(".")[-1], mod.materialize_())
+    return model

svd_llm/heal/llama2-13b/provenance.json

@@ -0,0 +1,199 @@
+{
+  "method": "KD healing: alpha*T^2*KL(teacher||student) + (1-alpha)*CE",
+  "student_init": "/data1/rkp00022/zixuan/projects/TR3_main/checkpoints/svdllm1_r04158_tr3fmt/state_dict_fp16.pt",
+  "teacher": "NousResearch/Llama-2-13b-chat-hf",
+  "alpha": 1.0,
+  "temperature": 1.0,
+  "lr": 2e-05,
+  "tokens_seen": 59998208,
+  "epochs": 0.228,
+  "global_batch_tokens": 32768,
+  "dataset_mixture_pre_exclusion": {
+    "alpaca": 50000,
+    "openhermes": 200000,
+    "ultrachat": 100000
+  },
+  "dataset_revisions": {
+    "tatsu-lab/alpaca": "dce01c9b08f87459cf36a430d809084718273017",
+    "teknium/OpenHermes-2.5": "b82037821055c377bed0d495e72e46de3bc72e84",
+    "HuggingFaceH4/ultrachat_200k": "8049631c405ae6576f93f445c6b8166f76f5505a"
+  },
+  "canonical_pool_excluded_texts": 10000,
+  "train_texts_after_exclusion": 356000,
+  "chat_texts": 340000,
+  "mmlu_aux_texts_kept": 16000,
+  "mmlu_aux_split": "cais/mmlu auxiliary_train (official TRAIN split, disjoint from the evaluated test split; audit verified 0/99842 exact overlap)",
+  "mmlu_aux_stats": {
+    "requested_groups": 16000,
+    "rendered_groups": 16000,
+    "shots": 5,
+    "rows_total": 99842,
+    "rows_dropped_by_decontam": 71,
+    "rows_clean": 99771,
+    "revision": "c30699e8356da336a370243923dbaf21066bb9fe",
+    "format": "MMLU-shaped (NOT byte-identical to lm-eval; auxiliary_train has no subject)"
+  },
+  "gold_label_supervision": false,
+  "attribution_note": "alpha=1.0 => pure KD, no gold-label CE, so any MMLU change is attributable to distillation on in-domain text. alpha<1 adds CE on text containing answers = labeled MCQ training, which is a DIFFERENT and weaker claim.",
+  "train_tokens_total": 263112704,
+  "decontamination": {
+    "removed": {
+      "alpaca": 11,
+      "openhermes": 120,
+      "ultrachat": 33
+    }
+  },
+  "start_kl": 2.0904040336608887,
+  "final_kl": 0.30065654776990414,
+  "start_mmlu": 0.24561403508771928,
+  "final_mmlu": 0.46535087719298246,
+  "history": [
+    {
+      "tokens_m": 5.01,
+      "step": 153,
+      "held_out_kl": 0.5967434793710709,
+      "mmlu": 0.3929824561403509
+    },
+    {
+      "tokens_m": 10.03,
+      "step": 306,
+      "held_out_kl": 0.5406020767986774,
+      "mmlu": 0.3956140350877193
+    },
+    {
+      "tokens_m": 15.01,
+      "step": 458,
+      "held_out_kl": 0.5206542909145355,
+      "mmlu": 0.4083333333333333
+    },
+    {
+      "tokens_m": 20.02,
+      "step": 611,
+      "held_out_kl": 0.4988052695989609,
+      "mmlu": 0.40087719298245617
+    },
+    {
+      "tokens_m": 25.0,
+      "step": 763,
+      "held_out_kl": 0.4575178399682045,
+      "mmlu": 0.40350877192982454
+    },
+    {
+      "tokens_m": 30.02,
+      "step": 916,
+      "held_out_kl": 0.4198819752782583,
+      "mmlu": 0.4298245614035088
+    },
+    {
+      "tokens_m": 35.03,
+      "step": 1069,
+      "held_out_kl": 0.38331497088074684,
+      "mmlu": 0.43157894736842106
+    },
+    {
+      "tokens_m": 40.01,
+      "step": 1221,
+      "held_out_kl": 0.34914770163595676,
+      "mmlu": 0.4425438596491228
+    },
+    {
+      "tokens_m": 45.02,
+      "step": 1374,
+      "held_out_kl": 0.32529739663004875,
+      "mmlu": 0.44956140350877194
+    },
+    {
+      "tokens_m": 50.0,
+      "step": 1526,
+      "held_out_kl": 0.31298027001321316,
+      "mmlu": 0.4587719298245614
+    },
+    {
+      "tokens_m": 55.02,
+      "step": 1679,
+      "held_out_kl": 0.30493565648794174,
+      "mmlu": 0.46403508771929824
+    },
+    {
+      "tokens_m": 60.0,
+      "step": 1831,
+      "held_out_kl": 0.30065654776990414,
+      "mmlu": 0.46535087719298246,
+      "final": true
+    }
+  ],
+  "teacher_mmlu_same_limit": 0.5442982456140351,
+  "checkpoint": "/data1/rkp00022/zixuan/projects/TR3_main/checkpoints/kdheal_svdllm1r04158_h1/state_dict_fp16.pt",
+  "sha256": "42b2f5c959e8fdd07ef526e6d182cab2f7685354859e1262b035eef39ba98387",
+  "seed": 1234,
+  "mmlu_limit_per_subtask": 40,
+  "teacher_revision": "d73f5fa9c4bc135502e04c27b39660747172d76b",
+  "student_init_sha256": "5625041bed874f21664b6a2bff3ce4722612302630028c9b60ac7f2cb8d7c605",
+  "ranks_file": "/data1/rkp00022/zixuan/projects/TR3_main/checkpoints/svdllm1_r04158_tr3fmt/ranks.json",
+  "ranks_sha256": "5c9ed0db91c9c81c51e62e6f82c7c2772478cc353c6f28040164843e64219b55",
+  "input_identity_gate": {
+    "student_ckpt": {
+      "key": "svdllm1_r04158_tr3fmt",
+      "sha256": "5625041bed874f21664b6a2bff3ce4722612302630028c9b60ac7f2cb8d7c605",
+      "status": "unregistered_allowed"
+    },
+    "ranks": {
+      "key": "svdllm1_r04158_tr3fmt",
+      "sha256": "5c9ed0db91c9c81c51e62e6f82c7c2772478cc353c6f28040164843e64219b55",
+      "status": "unregistered_allowed"
+    }
+  },
+  "train_seed": 1234,
+  "mmlu_seed": 1234,
+  "dynamic_batch": false,
+  "dbl": null,
+  "snapshot_milestones": true,
+  "snapshots": [
+    "(PosixPath('/data1/rkp00022/zixuan/projects/TR3_main/checkpoints/kdheal_svdllm1r04158_h1_snap15m'), 15.007744)",
+    "(PosixPath('/data1/rkp00022/zixuan/projects/TR3_main/checkpoints/kdheal_svdllm1r04158_h1_snap30m'), 30.015488)",
+    "(PosixPath('/data1/rkp00022/zixuan/projects/TR3_main/checkpoints/kdheal_svdllm1r04158_h1_snap45m'), 45.023232)",
+    "(PosixPath('/data1/rkp00022/zixuan/projects/TR3_main/checkpoints/kdheal_svdllm1r04158_h1_snap50m'), 50.003968)",
+    "(PosixPath('/data1/rkp00022/zixuan/projects/TR3_main/checkpoints/kdheal_svdllm1r04158_h1_snap55m'), 55.017472)"
+  ],
+  "stream_skip_tokens_m": 0.0,
+  "stream_blocks_consumed": [
+    0,
+    58592
+  ],
+  "stream_note": "the block order is a seeded permutation consumed from index 0; stages sharing a --seed with stream_skip=0 re-train on the SAME blocks. [start, end] block indices are recorded so any continuation can be checked for overlap with what earlier stages consumed.",
+  "student_master_dtype": "bf16",
+  "fake_quant": null,
+  "rebalance_report": null,
+  "student_param_count": 7737799680,
+  "micro": 8,
+  "grad_accum": 4,
+  "seq_len": 1024,
+  "held_out_source": "canonical: mixture 2000/4000/4000, seed 1234, seq 1024, blocks 256:264 (identical to the compression gates)",
+  "decontam_stats": {
+    "mmlu": {
+      "items": 14042,
+      "new_ngrams": 764261,
+      "revision": "c30699e8356da336a370243923dbaf21066bb9fe"
+    },
+    "hellaswag": {
+      "items": 10042,
+      "new_ngrams": 1334302,
+      "revision": "218ec52e09a7e7462a5400043bb9a69a41d06b76"
+    },
+    "boolq": {
+      "items": 3270,
+      "new_ngrams": 276806,
+      "revision": "3de24cf8022e94f4ee4b9d55a6f539891524d646"
+    },
+    "triviaqa": {
+      "items": 17944,
+      "new_ngrams": 32608,
+      "revision": "0f7faf33a3908546c6fd5b73a660e0f8ff173c2f"
+    },
+    "gsm8k": {
+      "items": 1319,
+      "new_ngrams": 46281,
+      "revision": "740312add88f781978c0658806c59bc2815b9866"
+    }
+  }
+}

svd_llm/heal/llama2-13b/ranks.json

@@ -0,0 +1,282 @@
+{
+ "model.layers.0.self_attn.q_proj": 1495,
+ "model.layers.0.self_attn.k_proj": 1495,
+ "model.layers.0.self_attn.v_proj": 1495,
+ "model.layers.0.self_attn.o_proj": 1495,
+ "model.layers.0.mlp.gate_proj": 2182,
+ "model.layers.0.mlp.up_proj": 2182,
+ "model.layers.0.mlp.down_proj": 2182,
+ "model.layers.1.self_attn.q_proj": 1495,
+ "model.layers.1.self_attn.k_proj": 1495,
+ "model.layers.1.self_attn.v_proj": 1495,
+ "model.layers.1.self_attn.o_proj": 1495,
+ "model.layers.1.mlp.gate_proj": 2182,
+ "model.layers.1.mlp.up_proj": 2182,
+ "model.layers.1.mlp.down_proj": 2182,
+ "model.layers.2.self_attn.q_proj": 1495,
+ "model.layers.2.self_attn.k_proj": 1495,
+ "model.layers.2.self_attn.v_proj": 1495,
+ "model.layers.2.self_attn.o_proj": 1495,
+ "model.layers.2.mlp.gate_proj": 2182,
+ "model.layers.2.mlp.up_proj": 2182,
+ "model.layers.2.mlp.down_proj": 2182,
+ "model.layers.3.self_attn.q_proj": 1495,
+ "model.layers.3.self_attn.k_proj": 1495,
+ "model.layers.3.self_attn.v_proj": 1495,
+ "model.layers.3.self_attn.o_proj": 1495,
+ "model.layers.3.mlp.gate_proj": 2182,
+ "model.layers.3.mlp.up_proj": 2182,
+ "model.layers.3.mlp.down_proj": 2182,
+ "model.layers.4.self_attn.q_proj": 1495,
+ "model.layers.4.self_attn.k_proj": 1495,
+ "model.layers.4.self_attn.v_proj": 1495,
+ "model.layers.4.self_attn.o_proj": 1495,
+ "model.layers.4.mlp.gate_proj": 2182,
+ "model.layers.4.mlp.up_proj": 2182,
+ "model.layers.4.mlp.down_proj": 2182,
+ "model.layers.5.self_attn.q_proj": 1495,
+ "model.layers.5.self_attn.k_proj": 1495,
+ "model.layers.5.self_attn.v_proj": 1495,
+ "model.layers.5.self_attn.o_proj": 1495,
+ "model.layers.5.mlp.gate_proj": 2182,
+ "model.layers.5.mlp.up_proj": 2182,
+ "model.layers.5.mlp.down_proj": 2182,
+ "model.layers.6.self_attn.q_proj": 1495,
+ "model.layers.6.self_attn.k_proj": 1495,
+ "model.layers.6.self_attn.v_proj": 1495,
+ "model.layers.6.self_attn.o_proj": 1495,
+ "model.layers.6.mlp.gate_proj": 2182,
+ "model.layers.6.mlp.up_proj": 2182,
+ "model.layers.6.mlp.down_proj": 2182,
+ "model.layers.7.self_attn.q_proj": 1495,
+ "model.layers.7.self_attn.k_proj": 1495,
+ "model.layers.7.self_attn.v_proj": 1495,
+ "model.layers.7.self_attn.o_proj": 1495,
+ "model.layers.7.mlp.gate_proj": 2182,
+ "model.layers.7.mlp.up_proj": 2182,
+ "model.layers.7.mlp.down_proj": 2182,
+ "model.layers.8.self_attn.q_proj": 1495,
+ "model.layers.8.self_attn.k_proj": 1495,
+ "model.layers.8.self_attn.v_proj": 1495,
+ "model.layers.8.self_attn.o_proj": 1495,
+ "model.layers.8.mlp.gate_proj": 2182,
+ "model.layers.8.mlp.up_proj": 2182,
+ "model.layers.8.mlp.down_proj": 2182,
+ "model.layers.9.self_attn.q_proj": 1495,
+ "model.layers.9.self_attn.k_proj": 1495,
+ "model.layers.9.self_attn.v_proj": 1495,
+ "model.layers.9.self_attn.o_proj": 1495,
+ "model.layers.9.mlp.gate_proj": 2182,
+ "model.layers.9.mlp.up_proj": 2182,
+ "model.layers.9.mlp.down_proj": 2182,
+ "model.layers.10.self_attn.q_proj": 1495,
+ "model.layers.10.self_attn.k_proj": 1495,
+ "model.layers.10.self_attn.v_proj": 1495,
+ "model.layers.10.self_attn.o_proj": 1495,
+ "model.layers.10.mlp.gate_proj": 2182,
+ "model.layers.10.mlp.up_proj": 2182,
+ "model.layers.10.mlp.down_proj": 2182,
+ "model.layers.11.self_attn.q_proj": 1495,
+ "model.layers.11.self_attn.k_proj": 1495,
+ "model.layers.11.self_attn.v_proj": 1495,
+ "model.layers.11.self_attn.o_proj": 1495,
+ "model.layers.11.mlp.gate_proj": 2182,
+ "model.layers.11.mlp.up_proj": 2182,
+ "model.layers.11.mlp.down_proj": 2182,
+ "model.layers.12.self_attn.q_proj": 1495,
+ "model.layers.12.self_attn.k_proj": 1495,
+ "model.layers.12.self_attn.v_proj": 1495,
+ "model.layers.12.self_attn.o_proj": 1495,
+ "model.layers.12.mlp.gate_proj": 2182,
+ "model.layers.12.mlp.up_proj": 2182,
+ "model.layers.12.mlp.down_proj": 2182,
+ "model.layers.13.self_attn.q_proj": 1495,
+ "model.layers.13.self_attn.k_proj": 1495,
+ "model.layers.13.self_attn.v_proj": 1495,
+ "model.layers.13.self_attn.o_proj": 1495,
+ "model.layers.13.mlp.gate_proj": 2182,
+ "model.layers.13.mlp.up_proj": 2182,
+ "model.layers.13.mlp.down_proj": 2182,
+ "model.layers.14.self_attn.q_proj": 1495,
+ "model.layers.14.self_attn.k_proj": 1495,
+ "model.layers.14.self_attn.v_proj": 1495,
+ "model.layers.14.self_attn.o_proj": 1495,
+ "model.layers.14.mlp.gate_proj": 2182,
+ "model.layers.14.mlp.up_proj": 2182,
+ "model.layers.14.mlp.down_proj": 2182,
+ "model.layers.15.self_attn.q_proj": 1495,
+ "model.layers.15.self_attn.k_proj": 1495,
+ "model.layers.15.self_attn.v_proj": 1495,
+ "model.layers.15.self_attn.o_proj": 1495,
+ "model.layers.15.mlp.gate_proj": 2182,
+ "model.layers.15.mlp.up_proj": 2182,
+ "model.layers.15.mlp.down_proj": 2182,
+ "model.layers.16.self_attn.q_proj": 1495,
+ "model.layers.16.self_attn.k_proj": 1495,
+ "model.layers.16.self_attn.v_proj": 1495,
+ "model.layers.16.self_attn.o_proj": 1495,
+ "model.layers.16.mlp.gate_proj": 2182,
+ "model.layers.16.mlp.up_proj": 2182,
+ "model.layers.16.mlp.down_proj": 2182,
+ "model.layers.17.self_attn.q_proj": 1495,
+ "model.layers.17.self_attn.k_proj": 1495,
+ "model.layers.17.self_attn.v_proj": 1495,
+ "model.layers.17.self_attn.o_proj": 1495,
+ "model.layers.17.mlp.gate_proj": 2182,
+ "model.layers.17.mlp.up_proj": 2182,
+ "model.layers.17.mlp.down_proj": 2182,
+ "model.layers.18.self_attn.q_proj": 1495,
+ "model.layers.18.self_attn.k_proj": 1495,
+ "model.layers.18.self_attn.v_proj": 1495,
+ "model.layers.18.self_attn.o_proj": 1495,
+ "model.layers.18.mlp.gate_proj": 2182,
+ "model.layers.18.mlp.up_proj": 2182,
+ "model.layers.18.mlp.down_proj": 2182,
+ "model.layers.19.self_attn.q_proj": 1495,
+ "model.layers.19.self_attn.k_proj": 1495,
+ "model.layers.19.self_attn.v_proj": 1495,
+ "model.layers.19.self_attn.o_proj": 1495,
+ "model.layers.19.mlp.gate_proj": 2182,
+ "model.layers.19.mlp.up_proj": 2182,
+ "model.layers.19.mlp.down_proj": 2182,
+ "model.layers.20.self_attn.q_proj": 1495,
+ "model.layers.20.self_attn.k_proj": 1495,
+ "model.layers.20.self_attn.v_proj": 1495,
+ "model.layers.20.self_attn.o_proj": 1495,
+ "model.layers.20.mlp.gate_proj": 2182,
+ "model.layers.20.mlp.up_proj": 2182,
+ "model.layers.20.mlp.down_proj": 2182,
+ "model.layers.21.self_attn.q_proj": 1495,
+ "model.layers.21.self_attn.k_proj": 1495,
+ "model.layers.21.self_attn.v_proj": 1495,
+ "model.layers.21.self_attn.o_proj": 1495,
+ "model.layers.21.mlp.gate_proj": 2182,
+ "model.layers.21.mlp.up_proj": 2182,
+ "model.layers.21.mlp.down_proj": 2182,
+ "model.layers.22.self_attn.q_proj": 1495,
+ "model.layers.22.self_attn.k_proj": 1495,
+ "model.layers.22.self_attn.v_proj": 1495,
+ "model.layers.22.self_attn.o_proj": 1495,
+ "model.layers.22.mlp.gate_proj": 2182,
+ "model.layers.22.mlp.up_proj": 2182,
+ "model.layers.22.mlp.down_proj": 2182,
+ "model.layers.23.self_attn.q_proj": 1495,
+ "model.layers.23.self_attn.k_proj": 1495,
+ "model.layers.23.self_attn.v_proj": 1495,
+ "model.layers.23.self_attn.o_proj": 1495,
+ "model.layers.23.mlp.gate_proj": 2182,
+ "model.layers.23.mlp.up_proj": 2182,
+ "model.layers.23.mlp.down_proj": 2182,
+ "model.layers.24.self_attn.q_proj": 1495,
+ "model.layers.24.self_attn.k_proj": 1495,
+ "model.layers.24.self_attn.v_proj": 1495,
+ "model.layers.24.self_attn.o_proj": 1495,
+ "model.layers.24.mlp.gate_proj": 2182,
+ "model.layers.24.mlp.up_proj": 2182,
+ "model.layers.24.mlp.down_proj": 2182,
+ "model.layers.25.self_attn.q_proj": 1495,
+ "model.layers.25.self_attn.k_proj": 1495,
+ "model.layers.25.self_attn.v_proj": 1495,
+ "model.layers.25.self_attn.o_proj": 1495,
+ "model.layers.25.mlp.gate_proj": 2182,
+ "model.layers.25.mlp.up_proj": 2182,
+ "model.layers.25.mlp.down_proj": 2182,
+ "model.layers.26.self_attn.q_proj": 1495,
+ "model.layers.26.self_attn.k_proj": 1495,
+ "model.layers.26.self_attn.v_proj": 1495,
+ "model.layers.26.self_attn.o_proj": 1495,
+ "model.layers.26.mlp.gate_proj": 2182,
+ "model.layers.26.mlp.up_proj": 2182,
+ "model.layers.26.mlp.down_proj": 2182,
+ "model.layers.27.self_attn.q_proj": 1495,
+ "model.layers.27.self_attn.k_proj": 1495,
+ "model.layers.27.self_attn.v_proj": 1495,
+ "model.layers.27.self_attn.o_proj": 1495,
+ "model.layers.27.mlp.gate_proj": 2182,
+ "model.layers.27.mlp.up_proj": 2182,
+ "model.layers.27.mlp.down_proj": 2182,
+ "model.layers.28.self_attn.q_proj": 1495,
+ "model.layers.28.self_attn.k_proj": 1495,
+ "model.layers.28.self_attn.v_proj": 1495,
+ "model.layers.28.self_attn.o_proj": 1495,
+ "model.layers.28.mlp.gate_proj": 2182,
+ "model.layers.28.mlp.up_proj": 2182,
+ "model.layers.28.mlp.down_proj": 2182,
+ "model.layers.29.self_attn.q_proj": 1495,
+ "model.layers.29.self_attn.k_proj": 1495,
+ "model.layers.29.self_attn.v_proj": 1495,
+ "model.layers.29.self_attn.o_proj": 1495,
+ "model.layers.29.mlp.gate_proj": 2182,
+ "model.layers.29.mlp.up_proj": 2182,
+ "model.layers.29.mlp.down_proj": 2182,
+ "model.layers.30.self_attn.q_proj": 1495,
+ "model.layers.30.self_attn.k_proj": 1495,
+ "model.layers.30.self_attn.v_proj": 1495,
+ "model.layers.30.self_attn.o_proj": 1495,
+ "model.layers.30.mlp.gate_proj": 2182,
+ "model.layers.30.mlp.up_proj": 2182,
+ "model.layers.30.mlp.down_proj": 2182,
+ "model.layers.31.self_attn.q_proj": 1495,
+ "model.layers.31.self_attn.k_proj": 1495,
+ "model.layers.31.self_attn.v_proj": 1495,
+ "model.layers.31.self_attn.o_proj": 1495,
+ "model.layers.31.mlp.gate_proj": 2182,
+ "model.layers.31.mlp.up_proj": 2182,
+ "model.layers.31.mlp.down_proj": 2182,
+ "model.layers.32.self_attn.q_proj": 1495,
+ "model.layers.32.self_attn.k_proj": 1495,
+ "model.layers.32.self_attn.v_proj": 1495,
+ "model.layers.32.self_attn.o_proj": 1495,
+ "model.layers.32.mlp.gate_proj": 2182,
+ "model.layers.32.mlp.up_proj": 2182,
+ "model.layers.32.mlp.down_proj": 2182,
+ "model.layers.33.self_attn.q_proj": 1495,
+ "model.layers.33.self_attn.k_proj": 1495,
+ "model.layers.33.self_attn.v_proj": 1495,
+ "model.layers.33.self_attn.o_proj": 1495,
+ "model.layers.33.mlp.gate_proj": 2182,
+ "model.layers.33.mlp.up_proj": 2182,
+ "model.layers.33.mlp.down_proj": 2182,
+ "model.layers.34.self_attn.q_proj": 1495,
+ "model.layers.34.self_attn.k_proj": 1495,
+ "model.layers.34.self_attn.v_proj": 1495,
+ "model.layers.34.self_attn.o_proj": 1495,
+ "model.layers.34.mlp.gate_proj": 2182,
+ "model.layers.34.mlp.up_proj": 2182,
+ "model.layers.34.mlp.down_proj": 2182,
+ "model.layers.35.self_attn.q_proj": 1495,
+ "model.layers.35.self_attn.k_proj": 1495,
+ "model.layers.35.self_attn.v_proj": 1495,
+ "model.layers.35.self_attn.o_proj": 1495,
+ "model.layers.35.mlp.gate_proj": 2182,
+ "model.layers.35.mlp.up_proj": 2182,
+ "model.layers.35.mlp.down_proj": 2182,
+ "model.layers.36.self_attn.q_proj": 1495,
+ "model.layers.36.self_attn.k_proj": 1495,
+ "model.layers.36.self_attn.v_proj": 1495,
+ "model.layers.36.self_attn.o_proj": 1495,
+ "model.layers.36.mlp.gate_proj": 2182,
+ "model.layers.36.mlp.up_proj": 2182,
+ "model.layers.36.mlp.down_proj": 2182,
+ "model.layers.37.self_attn.q_proj": 1495,
+ "model.layers.37.self_attn.k_proj": 1495,
+ "model.layers.37.self_attn.v_proj": 1495,
+ "model.layers.37.self_attn.o_proj": 1495,
+ "model.layers.37.mlp.gate_proj": 2182,
+ "model.layers.37.mlp.up_proj": 2182,
+ "model.layers.37.mlp.down_proj": 2182,
+ "model.layers.38.self_attn.q_proj": 1495,
+ "model.layers.38.self_attn.k_proj": 1495,
+ "model.layers.38.self_attn.v_proj": 1495,
+ "model.layers.38.self_attn.o_proj": 1495,
+ "model.layers.38.mlp.gate_proj": 2182,
+ "model.layers.38.mlp.up_proj": 2182,
+ "model.layers.38.mlp.down_proj": 2182,
+ "model.layers.39.self_attn.q_proj": 1495,
+ "model.layers.39.self_attn.k_proj": 1495,
+ "model.layers.39.self_attn.v_proj": 1495,
+ "model.layers.39.self_attn.o_proj": 1495,
+ "model.layers.39.mlp.gate_proj": 2182,
+ "model.layers.39.mlp.up_proj": 2182,
+ "model.layers.39.mlp.down_proj": 2182
+}

svd_llm/heal/llama2-13b/state_dict_fp16.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:42b2f5c959e8fdd07ef526e6d182cab2f7685354859e1262b035eef39ba98387
+size 15475814672

svd_llm/heal/llama2-7b/README.md

@@ -0,0 +1,43 @@
+---
+license: llama2
+base_model: NousResearch/Llama-2-7b-chat-hf
+tags:
+  - svd
+  - low-rank
+  - model-compression
+  - llama2
+---
+
+# SVD-LLM (ICLR 2025): LLaMA-2-7B-chat @ 4.02B, KD-healed
+
+SVD-LLM: truncation-aware data whitening (256 wikitext2 samples) + closed-form parameter update, official code ([AIoT-MLSys-Lab/SVD-LLM @ 7538cca](https://github.com/AIoT-MLSys-Lab/SVD-LLM)), uniform 41.93% removal across all 7 projection types of all 32 layers.
+
+- **4.022B params** (down from 6.74B, ~40% reduction)
+- Healed with pure forward-KL distillation from the dense 7B teacher: **60M tokens**,
+  held-out KL **2.5712 -> 0.3552**, probe MMLU (5-shot, limit 40/subtask) **25.92% -> 40.83%**
+- Formal full-benchmark evaluation under the locked protocol (lm-eval 0.4.12, fp16, seed
+  1234, 5/10/0/5/5-shot MMLU/HS/BQ/TQA/GSM): **[being finalized; see
+  the TR-3 paper's Table 2]**
+
+| Reference | MMLU | HellaSwag | BoolQ | TriviaQA | GSM8K |
+|---|---:|---:|---:|---:|---:|
+| unhealed 4.02B (same structure) | 26.29 | 34.22 | 38.35 | 0.96 | 0.00 |
+| TR-3 (same budget + same healing) | 43.21 | 70.75 | 77.09 | 36.93 | 9.40 |
+| FP16 dense 6.74B | 47.22 | 78.59 | 80.58 | 57.17 | 22.67 |
+
+## How to load
+
+```python
+import json
+from lowrank_linear import build, materialize_all
+
+model = build("NousResearch/Llama-2-7b-chat-hf",
+              json.load(open("ranks.json")), "state_dict_fp16.pt",
+              device="cuda")
+# for generation/eval: fuse the factors first (inference-only)
+materialize_all(model)
+```
+
+`ranks.json` maps each compressed projection path to its integer rank; untouched weights are
+stock LLaMA-2-7B-chat tensors. The full healing-provenance record (data mixture & revisions,
+decontamination, SHA256 of inputs/checkpoint, 5M milestone trajectory) is in `provenance.json`.

svd_llm/heal/llama2-7b/config.json

@@ -0,0 +1,28 @@
+{
+  "_name_or_path": "NousResearch/Llama-2-7b-chat-hf",
+  "architectures": [
+    "LlamaForCausalLM"
+  ],
+  "attention_bias": false,
+  "bos_token_id": 1,
+  "eos_token_id": 2,
+  "hidden_act": "silu",
+  "hidden_size": 4096,
+  "initializer_range": 0.02,
+  "intermediate_size": 11008,
+  "max_position_embeddings": 4096,
+  "model_type": "llama",
+  "num_attention_heads": 32,
+  "num_hidden_layers": 32,
+  "num_key_value_heads": 32,
+  "pad_token_id": 0,
+  "pretraining_tp": 1,
+  "rms_norm_eps": 1e-05,
+  "rope_scaling": null,
+  "rope_theta": 10000.0,
+  "tie_word_embeddings": false,
+  "torch_dtype": "float16",
+  "transformers_version": "4.35.2",
+  "use_cache": true,
+  "vocab_size": 32000
+}

svd_llm/heal/llama2-7b/lowrank_linear.py

@@ -0,0 +1,83 @@
+"""Minimal standalone loader for the SVD-baseline compressed LLaMA-2-7B checkpoints.
+
+No dependency beyond torch + transformers. The state_dict stores every compressed
+projection as two factors of a LowRankLinear (A: out x k, B: k x in); ranks.json
+holds the per-layer integer rank of each projection. Untouched parameters
+(embeddings, norms) keep their stock transformers names.
+
+Usage:
+    import json, torch
+    from lowrank_linear import build
+    model = build(
+        base_id="NousResearch/Llama-2-7b-chat-hf",
+        ranks=json.load(open("ranks.json")),
+        state_dict_path="state_dict_fp16.pt",
+        device="cuda", dtype=torch.float16,
+    )
+    tokenizer = AutoTokenizer.from_pretrained("NousResearch/Llama-2-7b-chat-hf")
+    out = model.generate(tokenizer("2+2=", return_tensors="pt").to("cuda").input_ids,
+                         max_new_tokens=16, do_sample=False)
+    print(tokenizer.decode(out[0]))
+"""
+
+from __future__ import annotations
+
+import torch
+import torch.nn as nn
+import torch.nn.functional as F
+
+
+class LowRankLinear(nn.Module):
+    """y = A (B x).  A: out x r, B: r x in.  (LLaMA projections carry no bias.)"""
+
+    def __init__(self, A: torch.Tensor, B: torch.Tensor):
+        super().__init__()
+        self.A = nn.Parameter(A)
+        self.B = nn.Parameter(B)
+
+    def forward(self, x):
+        return F.linear(F.linear(x, self.B), self.A)
+
+    @torch.no_grad()
+    def materialize_(self) -> nn.Linear:
+        """Replace this module with the dense W = A @ B (inference only; never call
+        while training -- it cuts the gradient path)."""
+        W = (self.A @ self.B).to(self.A.dtype)
+        dense = nn.Linear(self.B.shape[1], self.A.shape[0], bias=False,
+                          device=W.device, dtype=W.dtype)
+        dense.weight.data = W
+        return dense
+
+
+def build(base_id: str, ranks: dict, state_dict_path: str,
+          device: str = "cuda", dtype=torch.float16):
+    from transformers import AutoModelForCausalLM
+
+    model = AutoModelForCausalLM.from_pretrained(base_id, torch_dtype=dtype)
+    search = {}
+    for name, mod in model.named_modules():
+        if isinstance(mod, nn.Linear):
+            search[name] = mod
+    for path, k in ranks.items():
+        out_f, in_f = search[path].weight.shape
+        model_mod = model
+        for p in path.split(".")[:-1]:
+            model_mod = getattr(model_mod, p)
+        setattr(model_mod, path.split(".")[-1],
+                LowRankLinear(torch.zeros(out_f, int(k), dtype=dtype),
+                              torch.zeros(int(k), in_f, dtype=dtype)))
+    sd = torch.load(state_dict_path, map_location="cpu", weights_only=True)
+    model.load_state_dict(sd, strict=True)
+    del sd
+    return model.to(device)
+
+
+def materialize_all(model):
+    """Fuse every LowRankLinear back into a dense nn.Linear in place."""
+    for name, mod in list(model.named_modules()):
+        if isinstance(mod, LowRankLinear):
+            parent = model
+            for p in name.split(".")[:-1]:
+                parent = getattr(parent, p)
+            setattr(parent, name.split(".")[-1], mod.materialize_())
+    return model

svd_llm/heal/llama2-7b/provenance.json

@@ -0,0 +1,196 @@
+{
+  "method": "KD healing: alpha*T^2*KL(teacher||student) + (1-alpha)*CE",
+  "student_init": "/data1/rkp00022/zixuan/projects/TR3_main/checkpoints/svdllm1_r04193_tr3fmt/state_dict_fp16.pt",
+  "teacher": "NousResearch/Llama-2-7b-chat-hf",
+  "alpha": 1.0,
+  "temperature": 1.0,
+  "lr": 2e-05,
+  "tokens_seen": 59998208,
+  "epochs": 0.228,
+  "global_batch_tokens": 32768,
+  "dataset_mixture_pre_exclusion": {
+    "alpaca": 50000,
+    "openhermes": 200000,
+    "ultrachat": 100000
+  },
+  "dataset_revisions": {
+    "tatsu-lab/alpaca": "dce01c9b08f87459cf36a430d809084718273017",
+    "teknium/OpenHermes-2.5": "b82037821055c377bed0d495e72e46de3bc72e84",
+    "HuggingFaceH4/ultrachat_200k": "8049631c405ae6576f93f445c6b8166f76f5505a"
+  },
+  "canonical_pool_excluded_texts": 10000,
+  "train_texts_after_exclusion": 356000,
+  "chat_texts": 340000,
+  "mmlu_aux_texts_kept": 16000,
+  "mmlu_aux_split": "cais/mmlu auxiliary_train (official TRAIN split, disjoint from the evaluated test split; audit verified 0/99842 exact overlap)",
+  "mmlu_aux_stats": {
+    "requested_groups": 16000,
+    "rendered_groups": 16000,
+    "shots": 5,
+    "rows_total": 99842,
+    "rows_dropped_by_decontam": 71,
+    "rows_clean": 99771,
+    "revision": "c30699e8356da336a370243923dbaf21066bb9fe",
+    "format": "MMLU-shaped (NOT byte-identical to lm-eval; auxiliary_train has no subject)"
+  },
+  "gold_label_supervision": false,
+  "attribution_note": "alpha=1.0 => pure KD, no gold-label CE, so any MMLU change is attributable to distillation on in-domain text. alpha<1 adds CE on text containing answers = labeled MCQ training, which is a DIFFERENT and weaker claim.",
+  "train_tokens_total": 263112704,
+  "decontamination": {
+    "removed": {
+      "alpaca": 11,
+      "openhermes": 120,
+      "ultrachat": 33
+    }
+  },
+  "start_kl": 2.571214348077774,
+  "final_kl": 0.3551874812692404,
+  "start_mmlu": 0.25921052631578945,
+  "final_mmlu": 0.4083333333333333,
+  "history": [
+    {
+      "tokens_m": 5.01,
+      "step": 153,
+      "held_out_kl": 0.6783921979367733,
+      "mmlu": 0.3219298245614035
+    },
+    {
+      "tokens_m": 10.03,
+      "step": 306,
+      "held_out_kl": 0.6254855506122112,
+      "mmlu": 0.3425438596491228
+    },
+    {
+      "tokens_m": 15.01,
+      "step": 458,
+      "held_out_kl": 0.6108568087220192,
+      "mmlu": 0.3557017543859649
+    },
+    {
+      "tokens_m": 20.02,
+      "step": 611,
+      "held_out_kl": 0.5468266606330872,
+      "mmlu": 0.34692982456140353
+    },
+    {
+      "tokens_m": 25.0,
+      "step": 763,
+      "held_out_kl": 0.5192997306585312,
+      "mmlu": 0.3719298245614035
+    },
+    {
+      "tokens_m": 30.02,
+      "step": 916,
+      "held_out_kl": 0.4662874862551689,
+      "mmlu": 0.37456140350877193
+    },
+    {
+      "tokens_m": 35.03,
+      "step": 1069,
+      "held_out_kl": 0.43436891213059425,
+      "mmlu": 0.38026315789473686
+    },
+    {
+      "tokens_m": 40.01,
+      "step": 1221,
+      "held_out_kl": 0.3999592885375023,
+      "mmlu": 0.39166666666666666
+    },
+    {
+      "tokens_m": 45.02,
+      "step": 1374,
+      "held_out_kl": 0.3796225134283304,
+      "mmlu": 0.40789473684210525
+    },
+    {
+      "tokens_m": 50.0,
+      "step": 1526,
+      "held_out_kl": 0.367245651781559,
+      "mmlu": 0.4
+    },
+    {
+      "tokens_m": 55.02,
+      "step": 1679,
+      "held_out_kl": 0.36011041700839996,
+      "mmlu": 0.40614035087719297
+    },
+    {
+      "tokens_m": 60.0,
+      "step": 1831,
+      "held_out_kl": 0.3551874812692404,
+      "mmlu": 0.4083333333333333,
+      "final": true
+    }
+  ],
+  "teacher_mmlu_same_limit": 0.47543859649122805,
+  "checkpoint": "/data1/rkp00022/zixuan/projects/TR3_main/checkpoints/kdheal_svdllm1r04193_h1/state_dict_fp16.pt",
+  "sha256": "0d6cb91e781a8daaca00bef920965ad84d13fd103405e975611fd89150886923",
+  "seed": 1234,
+  "mmlu_limit_per_subtask": 40,
+  "teacher_revision": "351844e75ed0bcbbe3f10671b3c808d2b83894ee",
+  "student_init_sha256": "f244b45c40d688d4a4e285026ac4272a3db892364d91963564b87b69cd21b8bd",
+  "ranks_file": "/data1/rkp00022/zixuan/projects/TR3_main/checkpoints/svdllm1_r04193_tr3fmt/ranks.json",
+  "ranks_sha256": "715d09b26b58bbc4052eb425dc041e5d8b3acb5cc6cef729e2ca34c76c0e2943",
+  "input_identity_gate": {
+    "student_ckpt": {
+      "key": "svdllm1_r04193_tr3fmt",
+      "sha256": "f244b45c40d688d4a4e285026ac4272a3db892364d91963564b87b69cd21b8bd",
+      "status": "unregistered_allowed"
+    },
+    "ranks": {
+      "key": "svdllm1_r04193_tr3fmt",
+      "sha256": "715d09b26b58bbc4052eb425dc041e5d8b3acb5cc6cef729e2ca34c76c0e2943",
+      "status": "unregistered_allowed"
+    }
+  },
+  "train_seed": 1234,
+  "mmlu_seed": 1234,
+  "dynamic_batch": false,
+  "dbl": null,
+  "snapshot_milestones": true,
+  "snapshots": [
+    "(PosixPath('/data1/rkp00022/zixuan/projects/TR3_main/checkpoints/kdheal_svdllm1r04193_h1_snap50m'), 50.003968)",
+    "(PosixPath('/data1/rkp00022/zixuan/projects/TR3_main/checkpoints/kdheal_svdllm1r04193_h1_snap55m'), 55.017472)"
+  ],
+  "stream_skip_tokens_m": 0.0,
+  "stream_blocks_consumed": [
+    0,
+    58592
+  ],
+  "stream_note": "the block order is a seeded permutation consumed from index 0; stages sharing a --seed with stream_skip=0 re-train on the SAME blocks. [start, end] block indices are recorded so any continuation can be checked for overlap with what earlier stages consumed.",
+  "student_master_dtype": "bf16",
+  "fake_quant": null,
+  "rebalance_report": null,
+  "student_param_count": 4021989376,
+  "micro": 16,
+  "grad_accum": 2,
+  "seq_len": 1024,
+  "held_out_source": "canonical: mixture 2000/4000/4000, seed 1234, seq 1024, blocks 256:264 (identical to the compression gates)",
+  "decontam_stats": {
+    "mmlu": {
+      "items": 14042,
+      "new_ngrams": 764261,
+      "revision": "c30699e8356da336a370243923dbaf21066bb9fe"
+    },
+    "hellaswag": {
+      "items": 10042,
+      "new_ngrams": 1334302,
+      "revision": "218ec52e09a7e7462a5400043bb9a69a41d06b76"
+    },
+    "boolq": {
+      "items": 3270,
+      "new_ngrams": 276806,
+      "revision": "3de24cf8022e94f4ee4b9d55a6f539891524d646"
+    },
+    "triviaqa": {
+      "items": 17944,
+      "new_ngrams": 32608,
+      "revision": "0f7faf33a3908546c6fd5b73a660e0f8ff173c2f"
+    },
+    "gsm8k": {
+      "items": 1319,
+      "new_ngrams": 46281,
+      "revision": "740312add88f781978c0658806c59bc2815b9866"
+    }
+  }
+}

svd_llm/heal/llama2-7b/ranks.json

@@ -0,0 +1,226 @@
+{
+ "model.layers.0.self_attn.q_proj": 1189,
+ "model.layers.0.self_attn.k_proj": 1189,
+ "model.layers.0.self_attn.v_proj": 1189,
+ "model.layers.0.self_attn.o_proj": 1189,
+ "model.layers.0.mlp.gate_proj": 1733,
+ "model.layers.0.mlp.up_proj": 1733,
+ "model.layers.0.mlp.down_proj": 1733,
+ "model.layers.1.self_attn.q_proj": 1189,
+ "model.layers.1.self_attn.k_proj": 1189,
+ "model.layers.1.self_attn.v_proj": 1189,
+ "model.layers.1.self_attn.o_proj": 1189,
+ "model.layers.1.mlp.gate_proj": 1733,
+ "model.layers.1.mlp.up_proj": 1733,
+ "model.layers.1.mlp.down_proj": 1733,
+ "model.layers.2.self_attn.q_proj": 1189,
+ "model.layers.2.self_attn.k_proj": 1189,
+ "model.layers.2.self_attn.v_proj": 1189,
+ "model.layers.2.self_attn.o_proj": 1189,
+ "model.layers.2.mlp.gate_proj": 1733,
+ "model.layers.2.mlp.up_proj": 1733,
+ "model.layers.2.mlp.down_proj": 1733,
+ "model.layers.3.self_attn.q_proj": 1189,
+ "model.layers.3.self_attn.k_proj": 1189,
+ "model.layers.3.self_attn.v_proj": 1189,
+ "model.layers.3.self_attn.o_proj": 1189,
+ "model.layers.3.mlp.gate_proj": 1733,
+ "model.layers.3.mlp.up_proj": 1733,
+ "model.layers.3.mlp.down_proj": 1733,
+ "model.layers.4.self_attn.q_proj": 1189,
+ "model.layers.4.self_attn.k_proj": 1189,
+ "model.layers.4.self_attn.v_proj": 1189,
+ "model.layers.4.self_attn.o_proj": 1189,
+ "model.layers.4.mlp.gate_proj": 1733,
+ "model.layers.4.mlp.up_proj": 1733,
+ "model.layers.4.mlp.down_proj": 1733,
+ "model.layers.5.self_attn.q_proj": 1189,
+ "model.layers.5.self_attn.k_proj": 1189,
+ "model.layers.5.self_attn.v_proj": 1189,
+ "model.layers.5.self_attn.o_proj": 1189,
+ "model.layers.5.mlp.gate_proj": 1733,
+ "model.layers.5.mlp.up_proj": 1733,
+ "model.layers.5.mlp.down_proj": 1733,
+ "model.layers.6.self_attn.q_proj": 1189,
+ "model.layers.6.self_attn.k_proj": 1189,
+ "model.layers.6.self_attn.v_proj": 1189,
+ "model.layers.6.self_attn.o_proj": 1189,
+ "model.layers.6.mlp.gate_proj": 1733,
+ "model.layers.6.mlp.up_proj": 1733,
+ "model.layers.6.mlp.down_proj": 1733,
+ "model.layers.7.self_attn.q_proj": 1189,
+ "model.layers.7.self_attn.k_proj": 1189,
+ "model.layers.7.self_attn.v_proj": 1189,
+ "model.layers.7.self_attn.o_proj": 1189,
+ "model.layers.7.mlp.gate_proj": 1733,
+ "model.layers.7.mlp.up_proj": 1733,
+ "model.layers.7.mlp.down_proj": 1733,
+ "model.layers.8.self_attn.q_proj": 1189,
+ "model.layers.8.self_attn.k_proj": 1189,
+ "model.layers.8.self_attn.v_proj": 1189,
+ "model.layers.8.self_attn.o_proj": 1189,
+ "model.layers.8.mlp.gate_proj": 1733,
+ "model.layers.8.mlp.up_proj": 1733,
+ "model.layers.8.mlp.down_proj": 1733,
+ "model.layers.9.self_attn.q_proj": 1189,
+ "model.layers.9.self_attn.k_proj": 1189,
+ "model.layers.9.self_attn.v_proj": 1189,
+ "model.layers.9.self_attn.o_proj": 1189,
+ "model.layers.9.mlp.gate_proj": 1733,
+ "model.layers.9.mlp.up_proj": 1733,
+ "model.layers.9.mlp.down_proj": 1733,
+ "model.layers.10.self_attn.q_proj": 1189,
+ "model.layers.10.self_attn.k_proj": 1189,
+ "model.layers.10.self_attn.v_proj": 1189,
+ "model.layers.10.self_attn.o_proj": 1189,
+ "model.layers.10.mlp.gate_proj": 1733,
+ "model.layers.10.mlp.up_proj": 1733,
+ "model.layers.10.mlp.down_proj": 1733,
+ "model.layers.11.self_attn.q_proj": 1189,
+ "model.layers.11.self_attn.k_proj": 1189,
+ "model.layers.11.self_attn.v_proj": 1189,
+ "model.layers.11.self_attn.o_proj": 1189,
+ "model.layers.11.mlp.gate_proj": 1733,
+ "model.layers.11.mlp.up_proj": 1733,
+ "model.layers.11.mlp.down_proj": 1733,
+ "model.layers.12.self_attn.q_proj": 1189,
+ "model.layers.12.self_attn.k_proj": 1189,
+ "model.layers.12.self_attn.v_proj": 1189,
+ "model.layers.12.self_attn.o_proj": 1189,
+ "model.layers.12.mlp.gate_proj": 1733,
+ "model.layers.12.mlp.up_proj": 1733,
+ "model.layers.12.mlp.down_proj": 1733,
+ "model.layers.13.self_attn.q_proj": 1189,
+ "model.layers.13.self_attn.k_proj": 1189,
+ "model.layers.13.self_attn.v_proj": 1189,
+ "model.layers.13.self_attn.o_proj": 1189,
+ "model.layers.13.mlp.gate_proj": 1733,
+ "model.layers.13.mlp.up_proj": 1733,
+ "model.layers.13.mlp.down_proj": 1733,
+ "model.layers.14.self_attn.q_proj": 1189,
+ "model.layers.14.self_attn.k_proj": 1189,
+ "model.layers.14.self_attn.v_proj": 1189,
+ "model.layers.14.self_attn.o_proj": 1189,
+ "model.layers.14.mlp.gate_proj": 1733,
+ "model.layers.14.mlp.up_proj": 1733,
+ "model.layers.14.mlp.down_proj": 1733,
+ "model.layers.15.self_attn.q_proj": 1189,
+ "model.layers.15.self_attn.k_proj": 1189,
+ "model.layers.15.self_attn.v_proj": 1189,
+ "model.layers.15.self_attn.o_proj": 1189,
+ "model.layers.15.mlp.gate_proj": 1733,
+ "model.layers.15.mlp.up_proj": 1733,
+ "model.layers.15.mlp.down_proj": 1733,
+ "model.layers.16.self_attn.q_proj": 1189,
+ "model.layers.16.self_attn.k_proj": 1189,
+ "model.layers.16.self_attn.v_proj": 1189,
+ "model.layers.16.self_attn.o_proj": 1189,
+ "model.layers.16.mlp.gate_proj": 1733,
+ "model.layers.16.mlp.up_proj": 1733,
+ "model.layers.16.mlp.down_proj": 1733,
+ "model.layers.17.self_attn.q_proj": 1189,
+ "model.layers.17.self_attn.k_proj": 1189,
+ "model.layers.17.self_attn.v_proj": 1189,
+ "model.layers.17.self_attn.o_proj": 1189,
+ "model.layers.17.mlp.gate_proj": 1733,
+ "model.layers.17.mlp.up_proj": 1733,
+ "model.layers.17.mlp.down_proj": 1733,
+ "model.layers.18.self_attn.q_proj": 1189,
+ "model.layers.18.self_attn.k_proj": 1189,
+ "model.layers.18.self_attn.v_proj": 1189,
+ "model.layers.18.self_attn.o_proj": 1189,
+ "model.layers.18.mlp.gate_proj": 1733,
+ "model.layers.18.mlp.up_proj": 1733,
+ "model.layers.18.mlp.down_proj": 1733,
+ "model.layers.19.self_attn.q_proj": 1189,
+ "model.layers.19.self_attn.k_proj": 1189,
+ "model.layers.19.self_attn.v_proj": 1189,
+ "model.layers.19.self_attn.o_proj": 1189,
+ "model.layers.19.mlp.gate_proj": 1733,
+ "model.layers.19.mlp.up_proj": 1733,
+ "model.layers.19.mlp.down_proj": 1733,
+ "model.layers.20.self_attn.q_proj": 1189,
+ "model.layers.20.self_attn.k_proj": 1189,
+ "model.layers.20.self_attn.v_proj": 1189,
+ "model.layers.20.self_attn.o_proj": 1189,
+ "model.layers.20.mlp.gate_proj": 1733,
+ "model.layers.20.mlp.up_proj": 1733,
+ "model.layers.20.mlp.down_proj": 1733,
+ "model.layers.21.self_attn.q_proj": 1189,
+ "model.layers.21.self_attn.k_proj": 1189,
+ "model.layers.21.self_attn.v_proj": 1189,
+ "model.layers.21.self_attn.o_proj": 1189,
+ "model.layers.21.mlp.gate_proj": 1733,
+ "model.layers.21.mlp.up_proj": 1733,
+ "model.layers.21.mlp.down_proj": 1733,
+ "model.layers.22.self_attn.q_proj": 1189,
+ "model.layers.22.self_attn.k_proj": 1189,
+ "model.layers.22.self_attn.v_proj": 1189,
+ "model.layers.22.self_attn.o_proj": 1189,
+ "model.layers.22.mlp.gate_proj": 1733,
+ "model.layers.22.mlp.up_proj": 1733,
+ "model.layers.22.mlp.down_proj": 1733,
+ "model.layers.23.self_attn.q_proj": 1189,
+ "model.layers.23.self_attn.k_proj": 1189,
+ "model.layers.23.self_attn.v_proj": 1189,
+ "model.layers.23.self_attn.o_proj": 1189,
+ "model.layers.23.mlp.gate_proj": 1733,
+ "model.layers.23.mlp.up_proj": 1733,
+ "model.layers.23.mlp.down_proj": 1733,
+ "model.layers.24.self_attn.q_proj": 1189,
+ "model.layers.24.self_attn.k_proj": 1189,
+ "model.layers.24.self_attn.v_proj": 1189,
+ "model.layers.24.self_attn.o_proj": 1189,
+ "model.layers.24.mlp.gate_proj": 1733,
+ "model.layers.24.mlp.up_proj": 1733,
+ "model.layers.24.mlp.down_proj": 1733,
+ "model.layers.25.self_attn.q_proj": 1189,
+ "model.layers.25.self_attn.k_proj": 1189,
+ "model.layers.25.self_attn.v_proj": 1189,
+ "model.layers.25.self_attn.o_proj": 1189,
+ "model.layers.25.mlp.gate_proj": 1733,
+ "model.layers.25.mlp.up_proj": 1733,
+ "model.layers.25.mlp.down_proj": 1733,
+ "model.layers.26.self_attn.q_proj": 1189,
+ "model.layers.26.self_attn.k_proj": 1189,
+ "model.layers.26.self_attn.v_proj": 1189,
+ "model.layers.26.self_attn.o_proj": 1189,
+ "model.layers.26.mlp.gate_proj": 1733,
+ "model.layers.26.mlp.up_proj": 1733,
+ "model.layers.26.mlp.down_proj": 1733,
+ "model.layers.27.self_attn.q_proj": 1189,
+ "model.layers.27.self_attn.k_proj": 1189,
+ "model.layers.27.self_attn.v_proj": 1189,
+ "model.layers.27.self_attn.o_proj": 1189,
+ "model.layers.27.mlp.gate_proj": 1733,
+ "model.layers.27.mlp.up_proj": 1733,
+ "model.layers.27.mlp.down_proj": 1733,
+ "model.layers.28.self_attn.q_proj": 1189,
+ "model.layers.28.self_attn.k_proj": 1189,
+ "model.layers.28.self_attn.v_proj": 1189,
+ "model.layers.28.self_attn.o_proj": 1189,
+ "model.layers.28.mlp.gate_proj": 1733,
+ "model.layers.28.mlp.up_proj": 1733,
+ "model.layers.28.mlp.down_proj": 1733,
+ "model.layers.29.self_attn.q_proj": 1189,
+ "model.layers.29.self_attn.k_proj": 1189,
+ "model.layers.29.self_attn.v_proj": 1189,
+ "model.layers.29.self_attn.o_proj": 1189,
+ "model.layers.29.mlp.gate_proj": 1733,
+ "model.layers.29.mlp.up_proj": 1733,
+ "model.layers.29.mlp.down_proj": 1733,
+ "model.layers.30.self_attn.q_proj": 1189,
+ "model.layers.30.self_attn.k_proj": 1189,
+ "model.layers.30.self_attn.v_proj": 1189,
+ "model.layers.30.self_attn.o_proj": 1189,
+ "model.layers.30.mlp.gate_proj": 1733,
+ "model.layers.30.mlp.up_proj": 1733,
+ "model.layers.30.mlp.down_proj": 1733,
+ "model.layers.31.self_attn.q_proj": 1189,
+ "model.layers.31.self_attn.k_proj": 1189,
+ "model.layers.31.self_attn.v_proj": 1189,
+ "model.layers.31.self_attn.o_proj": 1189,
+ "model.layers.31.mlp.gate_proj": 1733,
+ "model.layers.31.mlp.up_proj": 1733,
+ "model.layers.31.mlp.down_proj": 1733
+}

svd_llm/heal/llama2-7b/state_dict_fp16.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:0d6cb91e781a8daaca00bef920965ad84d13fd103405e975611fd89150886923
+size 8044149604

svd_llm/unheal/llama2-13b/README.md

@@ -0,0 +1,16 @@
+---
+license: llama2
+base_model: NousResearch/Llama-2-13b-chat-hf
+tags: [svd, low-rank, model-compression, llama2]
+---
+
+# SVD-LLM @ 7.74B (59.47% kept of LLaMA-2-13B-chat), UNHEALED
+
+SVD-LLM (ICLR 2025): truncation-aware data whitening + closed-form parameter update, official code (@ 7538cca), uniform 41.58% removal across all 7 projection types of all 40 layers.
+
+- **UNHEALED checkpoint**: raw post-training compression weights, NO distillation. The KD-healed
+  counterpart is in `svd-llm-13b/`. This dir exists so the healing delta is
+  measurable; use it as the compression-stage artifact, not as a deployable model.
+- Same `ranks.json` as the healed dir; weights differ (see `provenance.json` SHA256).
+
+Load exactly as in the sibling dir (`lowrank_linear.py`).

svd_llm/unheal/llama2-13b/config.json

@@ -0,0 +1,26 @@
+{
+  "_name_or_path": null,
+  "architectures": [
+    "LlamaForCausalLM"
+  ],
+  "bos_token_id": 1,
+  "eos_token_id": 2,
+  "hidden_act": "silu",
+  "hidden_size": 5120,
+  "initializer_range": 0.02,
+  "intermediate_size": 13824,
+  "max_position_embeddings": 4096,
+  "model_type": "llama",
+  "num_attention_heads": 40,
+  "num_hidden_layers": 40,
+  "num_key_value_heads": 40,
+  "pad_token_id": 0,
+  "pretraining_tp": 1,
+  "rms_norm_eps": 1e-05,
+  "rope_scaling": null,
+  "tie_word_embeddings": false,
+  "torch_dtype": "float16",
+  "transformers_version": "4.31.0.dev0",
+  "use_cache": true,
+  "vocab_size": 32000
+}

svd_llm/unheal/llama2-13b/lowrank_linear.py

@@ -0,0 +1,83 @@
+"""Minimal standalone loader for the SVD-baseline compressed LLaMA-2-7B checkpoints.
+
+No dependency beyond torch + transformers. The state_dict stores every compressed
+projection as two factors of a LowRankLinear (A: out x k, B: k x in); ranks.json
+holds the per-layer integer rank of each projection. Untouched parameters
+(embeddings, norms) keep their stock transformers names.
+
+Usage:
+    import json, torch
+    from lowrank_linear import build
+    model = build(
+        base_id="NousResearch/Llama-2-7b-chat-hf",
+        ranks=json.load(open("ranks.json")),
+        state_dict_path="state_dict_fp16.pt",
+        device="cuda", dtype=torch.float16,
+    )
+    tokenizer = AutoTokenizer.from_pretrained("NousResearch/Llama-2-7b-chat-hf")
+    out = model.generate(tokenizer("2+2=", return_tensors="pt").to("cuda").input_ids,
+                         max_new_tokens=16, do_sample=False)
+    print(tokenizer.decode(out[0]))
+"""
+
+from __future__ import annotations
+
+import torch
+import torch.nn as nn
+import torch.nn.functional as F
+
+
+class LowRankLinear(nn.Module):
+    """y = A (B x).  A: out x r, B: r x in.  (LLaMA projections carry no bias.)"""
+
+    def __init__(self, A: torch.Tensor, B: torch.Tensor):
+        super().__init__()
+        self.A = nn.Parameter(A)
+        self.B = nn.Parameter(B)
+
+    def forward(self, x):
+        return F.linear(F.linear(x, self.B), self.A)
+
+    @torch.no_grad()
+    def materialize_(self) -> nn.Linear:
+        """Replace this module with the dense W = A @ B (inference only; never call
+        while training -- it cuts the gradient path)."""
+        W = (self.A @ self.B).to(self.A.dtype)
+        dense = nn.Linear(self.B.shape[1], self.A.shape[0], bias=False,
+                          device=W.device, dtype=W.dtype)
+        dense.weight.data = W
+        return dense
+
+
+def build(base_id: str, ranks: dict, state_dict_path: str,
+          device: str = "cuda", dtype=torch.float16):
+    from transformers import AutoModelForCausalLM
+
+    model = AutoModelForCausalLM.from_pretrained(base_id, torch_dtype=dtype)
+    search = {}
+    for name, mod in model.named_modules():
+        if isinstance(mod, nn.Linear):
+            search[name] = mod
+    for path, k in ranks.items():
+        out_f, in_f = search[path].weight.shape
+        model_mod = model
+        for p in path.split(".")[:-1]:
+            model_mod = getattr(model_mod, p)
+        setattr(model_mod, path.split(".")[-1],
+                LowRankLinear(torch.zeros(out_f, int(k), dtype=dtype),
+                              torch.zeros(int(k), in_f, dtype=dtype)))
+    sd = torch.load(state_dict_path, map_location="cpu", weights_only=True)
+    model.load_state_dict(sd, strict=True)
+    del sd
+    return model.to(device)
+
+
+def materialize_all(model):
+    """Fuse every LowRankLinear back into a dense nn.Linear in place."""
+    for name, mod in list(model.named_modules()):
+        if isinstance(mod, LowRankLinear):
+            parent = model
+            for p in name.split(".")[:-1]:
+                parent = getattr(parent, p)
+            setattr(parent, name.split(".")[-1], mod.materialize_())
+    return model

svd_llm/unheal/llama2-13b/provenance.json

@@ -0,0 +1,7 @@
+{
+ "kind": "unhealed SVD compression (TR3-format conversion, verified pure re-keying)",
+ "source_ckpt": "/data1/rkp00022/zixuan/projects/TR3_main/checkpoints/svdllm1_llama2_13b_chat_r04158/NousResearch_Llama_2_13b_chat_hf_whitening_then_update_0.5842.pt",
+ "converted_state_dict": "/data1/rkp00022/zixuan/projects/TR3_main/scratch/hf_push_13b/svd-llm-13b-unhealed/state_dict_fp16.pt",
+ "state_dict_sha256": "5625041bed874f21664b6a2bff3ce4722612302630028c9b60ac7f2cb8d7c605",
+ "healed_counterpart": "svd-llm-13b/state_dict_fp16.pt"
+}

svd_llm/unheal/llama2-13b/ranks.json

@@ -0,0 +1,282 @@
+{
+ "model.layers.0.self_attn.q_proj": 1495,
+ "model.layers.0.self_attn.k_proj": 1495,
+ "model.layers.0.self_attn.v_proj": 1495,
+ "model.layers.0.self_attn.o_proj": 1495,
+ "model.layers.0.mlp.gate_proj": 2182,
+ "model.layers.0.mlp.up_proj": 2182,
+ "model.layers.0.mlp.down_proj": 2182,
+ "model.layers.1.self_attn.q_proj": 1495,
+ "model.layers.1.self_attn.k_proj": 1495,
+ "model.layers.1.self_attn.v_proj": 1495,
+ "model.layers.1.self_attn.o_proj": 1495,
+ "model.layers.1.mlp.gate_proj": 2182,
+ "model.layers.1.mlp.up_proj": 2182,
+ "model.layers.1.mlp.down_proj": 2182,
+ "model.layers.2.self_attn.q_proj": 1495,
+ "model.layers.2.self_attn.k_proj": 1495,
+ "model.layers.2.self_attn.v_proj": 1495,
+ "model.layers.2.self_attn.o_proj": 1495,
+ "model.layers.2.mlp.gate_proj": 2182,
+ "model.layers.2.mlp.up_proj": 2182,
+ "model.layers.2.mlp.down_proj": 2182,
+ "model.layers.3.self_attn.q_proj": 1495,
+ "model.layers.3.self_attn.k_proj": 1495,
+ "model.layers.3.self_attn.v_proj": 1495,
+ "model.layers.3.self_attn.o_proj": 1495,
+ "model.layers.3.mlp.gate_proj": 2182,
+ "model.layers.3.mlp.up_proj": 2182,
+ "model.layers.3.mlp.down_proj": 2182,
+ "model.layers.4.self_attn.q_proj": 1495,
+ "model.layers.4.self_attn.k_proj": 1495,
+ "model.layers.4.self_attn.v_proj": 1495,
+ "model.layers.4.self_attn.o_proj": 1495,
+ "model.layers.4.mlp.gate_proj": 2182,
+ "model.layers.4.mlp.up_proj": 2182,
+ "model.layers.4.mlp.down_proj": 2182,
+ "model.layers.5.self_attn.q_proj": 1495,
+ "model.layers.5.self_attn.k_proj": 1495,
+ "model.layers.5.self_attn.v_proj": 1495,
+ "model.layers.5.self_attn.o_proj": 1495,
+ "model.layers.5.mlp.gate_proj": 2182,
+ "model.layers.5.mlp.up_proj": 2182,
+ "model.layers.5.mlp.down_proj": 2182,
+ "model.layers.6.self_attn.q_proj": 1495,
+ "model.layers.6.self_attn.k_proj": 1495,
+ "model.layers.6.self_attn.v_proj": 1495,
+ "model.layers.6.self_attn.o_proj": 1495,
+ "model.layers.6.mlp.gate_proj": 2182,
+ "model.layers.6.mlp.up_proj": 2182,
+ "model.layers.6.mlp.down_proj": 2182,
+ "model.layers.7.self_attn.q_proj": 1495,
+ "model.layers.7.self_attn.k_proj": 1495,
+ "model.layers.7.self_attn.v_proj": 1495,
+ "model.layers.7.self_attn.o_proj": 1495,
+ "model.layers.7.mlp.gate_proj": 2182,
+ "model.layers.7.mlp.up_proj": 2182,
+ "model.layers.7.mlp.down_proj": 2182,
+ "model.layers.8.self_attn.q_proj": 1495,
+ "model.layers.8.self_attn.k_proj": 1495,
+ "model.layers.8.self_attn.v_proj": 1495,
+ "model.layers.8.self_attn.o_proj": 1495,
+ "model.layers.8.mlp.gate_proj": 2182,
+ "model.layers.8.mlp.up_proj": 2182,
+ "model.layers.8.mlp.down_proj": 2182,
+ "model.layers.9.self_attn.q_proj": 1495,
+ "model.layers.9.self_attn.k_proj": 1495,
+ "model.layers.9.self_attn.v_proj": 1495,
+ "model.layers.9.self_attn.o_proj": 1495,
+ "model.layers.9.mlp.gate_proj": 2182,
+ "model.layers.9.mlp.up_proj": 2182,
+ "model.layers.9.mlp.down_proj": 2182,
+ "model.layers.10.self_attn.q_proj": 1495,
+ "model.layers.10.self_attn.k_proj": 1495,
+ "model.layers.10.self_attn.v_proj": 1495,
+ "model.layers.10.self_attn.o_proj": 1495,
+ "model.layers.10.mlp.gate_proj": 2182,
+ "model.layers.10.mlp.up_proj": 2182,
+ "model.layers.10.mlp.down_proj": 2182,
+ "model.layers.11.self_attn.q_proj": 1495,
+ "model.layers.11.self_attn.k_proj": 1495,
+ "model.layers.11.self_attn.v_proj": 1495,
+ "model.layers.11.self_attn.o_proj": 1495,
+ "model.layers.11.mlp.gate_proj": 2182,
+ "model.layers.11.mlp.up_proj": 2182,
+ "model.layers.11.mlp.down_proj": 2182,
+ "model.layers.12.self_attn.q_proj": 1495,
+ "model.layers.12.self_attn.k_proj": 1495,
+ "model.layers.12.self_attn.v_proj": 1495,
+ "model.layers.12.self_attn.o_proj": 1495,
+ "model.layers.12.mlp.gate_proj": 2182,
+ "model.layers.12.mlp.up_proj": 2182,
+ "model.layers.12.mlp.down_proj": 2182,
+ "model.layers.13.self_attn.q_proj": 1495,
+ "model.layers.13.self_attn.k_proj": 1495,
+ "model.layers.13.self_attn.v_proj": 1495,
+ "model.layers.13.self_attn.o_proj": 1495,
+ "model.layers.13.mlp.gate_proj": 2182,
+ "model.layers.13.mlp.up_proj": 2182,
+ "model.layers.13.mlp.down_proj": 2182,
+ "model.layers.14.self_attn.q_proj": 1495,
+ "model.layers.14.self_attn.k_proj": 1495,
+ "model.layers.14.self_attn.v_proj": 1495,
+ "model.layers.14.self_attn.o_proj": 1495,
+ "model.layers.14.mlp.gate_proj": 2182,
+ "model.layers.14.mlp.up_proj": 2182,
+ "model.layers.14.mlp.down_proj": 2182,
+ "model.layers.15.self_attn.q_proj": 1495,
+ "model.layers.15.self_attn.k_proj": 1495,
+ "model.layers.15.self_attn.v_proj": 1495,
+ "model.layers.15.self_attn.o_proj": 1495,
+ "model.layers.15.mlp.gate_proj": 2182,
+ "model.layers.15.mlp.up_proj": 2182,
+ "model.layers.15.mlp.down_proj": 2182,
+ "model.layers.16.self_attn.q_proj": 1495,
+ "model.layers.16.self_attn.k_proj": 1495,
+ "model.layers.16.self_attn.v_proj": 1495,
+ "model.layers.16.self_attn.o_proj": 1495,
+ "model.layers.16.mlp.gate_proj": 2182,
+ "model.layers.16.mlp.up_proj": 2182,
+ "model.layers.16.mlp.down_proj": 2182,
+ "model.layers.17.self_attn.q_proj": 1495,
+ "model.layers.17.self_attn.k_proj": 1495,
+ "model.layers.17.self_attn.v_proj": 1495,
+ "model.layers.17.self_attn.o_proj": 1495,
+ "model.layers.17.mlp.gate_proj": 2182,
+ "model.layers.17.mlp.up_proj": 2182,
+ "model.layers.17.mlp.down_proj": 2182,
+ "model.layers.18.self_attn.q_proj": 1495,
+ "model.layers.18.self_attn.k_proj": 1495,
+ "model.layers.18.self_attn.v_proj": 1495,
+ "model.layers.18.self_attn.o_proj": 1495,
+ "model.layers.18.mlp.gate_proj": 2182,
+ "model.layers.18.mlp.up_proj": 2182,
+ "model.layers.18.mlp.down_proj": 2182,
+ "model.layers.19.self_attn.q_proj": 1495,
+ "model.layers.19.self_attn.k_proj": 1495,
+ "model.layers.19.self_attn.v_proj": 1495,
+ "model.layers.19.self_attn.o_proj": 1495,
+ "model.layers.19.mlp.gate_proj": 2182,
+ "model.layers.19.mlp.up_proj": 2182,
+ "model.layers.19.mlp.down_proj": 2182,
+ "model.layers.20.self_attn.q_proj": 1495,
+ "model.layers.20.self_attn.k_proj": 1495,
+ "model.layers.20.self_attn.v_proj": 1495,
+ "model.layers.20.self_attn.o_proj": 1495,
+ "model.layers.20.mlp.gate_proj": 2182,
+ "model.layers.20.mlp.up_proj": 2182,
+ "model.layers.20.mlp.down_proj": 2182,
+ "model.layers.21.self_attn.q_proj": 1495,
+ "model.layers.21.self_attn.k_proj": 1495,
+ "model.layers.21.self_attn.v_proj": 1495,
+ "model.layers.21.self_attn.o_proj": 1495,
+ "model.layers.21.mlp.gate_proj": 2182,
+ "model.layers.21.mlp.up_proj": 2182,
+ "model.layers.21.mlp.down_proj": 2182,
+ "model.layers.22.self_attn.q_proj": 1495,
+ "model.layers.22.self_attn.k_proj": 1495,
+ "model.layers.22.self_attn.v_proj": 1495,
+ "model.layers.22.self_attn.o_proj": 1495,
+ "model.layers.22.mlp.gate_proj": 2182,
+ "model.layers.22.mlp.up_proj": 2182,
+ "model.layers.22.mlp.down_proj": 2182,
+ "model.layers.23.self_attn.q_proj": 1495,
+ "model.layers.23.self_attn.k_proj": 1495,
+ "model.layers.23.self_attn.v_proj": 1495,
+ "model.layers.23.self_attn.o_proj": 1495,
+ "model.layers.23.mlp.gate_proj": 2182,
+ "model.layers.23.mlp.up_proj": 2182,
+ "model.layers.23.mlp.down_proj": 2182,
+ "model.layers.24.self_attn.q_proj": 1495,
+ "model.layers.24.self_attn.k_proj": 1495,
+ "model.layers.24.self_attn.v_proj": 1495,
+ "model.layers.24.self_attn.o_proj": 1495,
+ "model.layers.24.mlp.gate_proj": 2182,
+ "model.layers.24.mlp.up_proj": 2182,
+ "model.layers.24.mlp.down_proj": 2182,
+ "model.layers.25.self_attn.q_proj": 1495,
+ "model.layers.25.self_attn.k_proj": 1495,
+ "model.layers.25.self_attn.v_proj": 1495,
+ "model.layers.25.self_attn.o_proj": 1495,
+ "model.layers.25.mlp.gate_proj": 2182,
+ "model.layers.25.mlp.up_proj": 2182,
+ "model.layers.25.mlp.down_proj": 2182,
+ "model.layers.26.self_attn.q_proj": 1495,
+ "model.layers.26.self_attn.k_proj": 1495,
+ "model.layers.26.self_attn.v_proj": 1495,
+ "model.layers.26.self_attn.o_proj": 1495,
+ "model.layers.26.mlp.gate_proj": 2182,
+ "model.layers.26.mlp.up_proj": 2182,
+ "model.layers.26.mlp.down_proj": 2182,
+ "model.layers.27.self_attn.q_proj": 1495,
+ "model.layers.27.self_attn.k_proj": 1495,
+ "model.layers.27.self_attn.v_proj": 1495,
+ "model.layers.27.self_attn.o_proj": 1495,
+ "model.layers.27.mlp.gate_proj": 2182,
+ "model.layers.27.mlp.up_proj": 2182,
+ "model.layers.27.mlp.down_proj": 2182,
+ "model.layers.28.self_attn.q_proj": 1495,
+ "model.layers.28.self_attn.k_proj": 1495,
+ "model.layers.28.self_attn.v_proj": 1495,
+ "model.layers.28.self_attn.o_proj": 1495,
+ "model.layers.28.mlp.gate_proj": 2182,
+ "model.layers.28.mlp.up_proj": 2182,
+ "model.layers.28.mlp.down_proj": 2182,
+ "model.layers.29.self_attn.q_proj": 1495,
+ "model.layers.29.self_attn.k_proj": 1495,
+ "model.layers.29.self_attn.v_proj": 1495,
+ "model.layers.29.self_attn.o_proj": 1495,
+ "model.layers.29.mlp.gate_proj": 2182,
+ "model.layers.29.mlp.up_proj": 2182,
+ "model.layers.29.mlp.down_proj": 2182,
+ "model.layers.30.self_attn.q_proj": 1495,
+ "model.layers.30.self_attn.k_proj": 1495,
+ "model.layers.30.self_attn.v_proj": 1495,
+ "model.layers.30.self_attn.o_proj": 1495,
+ "model.layers.30.mlp.gate_proj": 2182,
+ "model.layers.30.mlp.up_proj": 2182,
+ "model.layers.30.mlp.down_proj": 2182,
+ "model.layers.31.self_attn.q_proj": 1495,
+ "model.layers.31.self_attn.k_proj": 1495,
+ "model.layers.31.self_attn.v_proj": 1495,
+ "model.layers.31.self_attn.o_proj": 1495,
+ "model.layers.31.mlp.gate_proj": 2182,
+ "model.layers.31.mlp.up_proj": 2182,
+ "model.layers.31.mlp.down_proj": 2182,
+ "model.layers.32.self_attn.q_proj": 1495,
+ "model.layers.32.self_attn.k_proj": 1495,
+ "model.layers.32.self_attn.v_proj": 1495,
+ "model.layers.32.self_attn.o_proj": 1495,
+ "model.layers.32.mlp.gate_proj": 2182,
+ "model.layers.32.mlp.up_proj": 2182,
+ "model.layers.32.mlp.down_proj": 2182,
+ "model.layers.33.self_attn.q_proj": 1495,
+ "model.layers.33.self_attn.k_proj": 1495,
+ "model.layers.33.self_attn.v_proj": 1495,
+ "model.layers.33.self_attn.o_proj": 1495,
+ "model.layers.33.mlp.gate_proj": 2182,
+ "model.layers.33.mlp.up_proj": 2182,
+ "model.layers.33.mlp.down_proj": 2182,
+ "model.layers.34.self_attn.q_proj": 1495,
+ "model.layers.34.self_attn.k_proj": 1495,
+ "model.layers.34.self_attn.v_proj": 1495,
+ "model.layers.34.self_attn.o_proj": 1495,
+ "model.layers.34.mlp.gate_proj": 2182,
+ "model.layers.34.mlp.up_proj": 2182,
+ "model.layers.34.mlp.down_proj": 2182,
+ "model.layers.35.self_attn.q_proj": 1495,
+ "model.layers.35.self_attn.k_proj": 1495,
+ "model.layers.35.self_attn.v_proj": 1495,
+ "model.layers.35.self_attn.o_proj": 1495,
+ "model.layers.35.mlp.gate_proj": 2182,
+ "model.layers.35.mlp.up_proj": 2182,
+ "model.layers.35.mlp.down_proj": 2182,
+ "model.layers.36.self_attn.q_proj": 1495,
+ "model.layers.36.self_attn.k_proj": 1495,
+ "model.layers.36.self_attn.v_proj": 1495,
+ "model.layers.36.self_attn.o_proj": 1495,
+ "model.layers.36.mlp.gate_proj": 2182,
+ "model.layers.36.mlp.up_proj": 2182,
+ "model.layers.36.mlp.down_proj": 2182,
+ "model.layers.37.self_attn.q_proj": 1495,
+ "model.layers.37.self_attn.k_proj": 1495,
+ "model.layers.37.self_attn.v_proj": 1495,
+ "model.layers.37.self_attn.o_proj": 1495,
+ "model.layers.37.mlp.gate_proj": 2182,
+ "model.layers.37.mlp.up_proj": 2182,
+ "model.layers.37.mlp.down_proj": 2182,
+ "model.layers.38.self_attn.q_proj": 1495,
+ "model.layers.38.self_attn.k_proj": 1495,
+ "model.layers.38.self_attn.v_proj": 1495,
+ "model.layers.38.self_attn.o_proj": 1495,
+ "model.layers.38.mlp.gate_proj": 2182,
+ "model.layers.38.mlp.up_proj": 2182,
+ "model.layers.38.mlp.down_proj": 2182,
+ "model.layers.39.self_attn.q_proj": 1495,
+ "model.layers.39.self_attn.k_proj": 1495,
+ "model.layers.39.self_attn.v_proj": 1495,
+ "model.layers.39.self_attn.o_proj": 1495,
+ "model.layers.39.mlp.gate_proj": 2182,
+ "model.layers.39.mlp.up_proj": 2182,
+ "model.layers.39.mlp.down_proj": 2182
+}

svd_llm/unheal/llama2-13b/state_dict_fp16.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5625041bed874f21664b6a2bff3ce4722612302630028c9b60ac7f2cb8d7c605
+size 15475812809

svd_llm/unheal/llama2-7b/README.md

@@ -0,0 +1,13 @@
+---
+license: llama2
+base_model: NousResearch/Llama-2-7b-chat-hf
+tags: [svd, low-rank, model-compression, llama2]
+---
+
+# SVD-LLM (ICLR 2025, official code): whitening + closed-form update, uniform 41.93% removal @ 4.02B, UNHEALED (LLaMA-2-7B-chat)
+
+- **UNHEALED checkpoint**: raw post-training compression weights, NO distillation. The KD-healed
+  counterpart is at `../heal/llama2-7b/`. Same `ranks.json`; weights differ.
+- Factored params: **4,021,989,376** (budget 4.0231B, matched to the TR-3 llama2-7b row).
+
+Load with `lowrank_linear.py` in this dir (identical recipe to the sibling dirs).

svd_llm/unheal/llama2-7b/config.json

@@ -0,0 +1,28 @@
+{
+  "_name_or_path": "NousResearch/Llama-2-7b-chat-hf",
+  "architectures": [
+    "LlamaForCausalLM"
+  ],
+  "attention_bias": false,
+  "bos_token_id": 1,
+  "eos_token_id": 2,
+  "hidden_act": "silu",
+  "hidden_size": 4096,
+  "initializer_range": 0.02,
+  "intermediate_size": 11008,
+  "max_position_embeddings": 4096,
+  "model_type": "llama",
+  "num_attention_heads": 32,
+  "num_hidden_layers": 32,
+  "num_key_value_heads": 32,
+  "pad_token_id": 0,
+  "pretraining_tp": 1,
+  "rms_norm_eps": 1e-05,
+  "rope_scaling": null,
+  "rope_theta": 10000.0,
+  "tie_word_embeddings": false,
+  "torch_dtype": "float16",
+  "transformers_version": "4.35.2",
+  "use_cache": true,
+  "vocab_size": 32000
+}

svd_llm/unheal/llama2-7b/lowrank_linear.py

@@ -0,0 +1,83 @@
+"""Minimal standalone loader for the SVD-baseline compressed LLaMA-2-7B checkpoints.
+
+No dependency beyond torch + transformers. The state_dict stores every compressed
+projection as two factors of a LowRankLinear (A: out x k, B: k x in); ranks.json
+holds the per-layer integer rank of each projection. Untouched parameters
+(embeddings, norms) keep their stock transformers names.
+
+Usage:
+    import json, torch
+    from lowrank_linear import build
+    model = build(
+        base_id="NousResearch/Llama-2-7b-chat-hf",
+        ranks=json.load(open("ranks.json")),
+        state_dict_path="state_dict_fp16.pt",
+        device="cuda", dtype=torch.float16,
+    )
+    tokenizer = AutoTokenizer.from_pretrained("NousResearch/Llama-2-7b-chat-hf")
+    out = model.generate(tokenizer("2+2=", return_tensors="pt").to("cuda").input_ids,
+                         max_new_tokens=16, do_sample=False)
+    print(tokenizer.decode(out[0]))
+"""
+
+from __future__ import annotations
+
+import torch
+import torch.nn as nn
+import torch.nn.functional as F
+
+
+class LowRankLinear(nn.Module):
+    """y = A (B x).  A: out x r, B: r x in.  (LLaMA projections carry no bias.)"""
+
+    def __init__(self, A: torch.Tensor, B: torch.Tensor):
+        super().__init__()
+        self.A = nn.Parameter(A)
+        self.B = nn.Parameter(B)
+
+    def forward(self, x):
+        return F.linear(F.linear(x, self.B), self.A)
+
+    @torch.no_grad()
+    def materialize_(self) -> nn.Linear:
+        """Replace this module with the dense W = A @ B (inference only; never call
+        while training -- it cuts the gradient path)."""
+        W = (self.A @ self.B).to(self.A.dtype)
+        dense = nn.Linear(self.B.shape[1], self.A.shape[0], bias=False,
+                          device=W.device, dtype=W.dtype)
+        dense.weight.data = W
+        return dense
+
+
+def build(base_id: str, ranks: dict, state_dict_path: str,
+          device: str = "cuda", dtype=torch.float16):
+    from transformers import AutoModelForCausalLM
+
+    model = AutoModelForCausalLM.from_pretrained(base_id, torch_dtype=dtype)
+    search = {}
+    for name, mod in model.named_modules():
+        if isinstance(mod, nn.Linear):
+            search[name] = mod
+    for path, k in ranks.items():
+        out_f, in_f = search[path].weight.shape
+        model_mod = model
+        for p in path.split(".")[:-1]:
+            model_mod = getattr(model_mod, p)
+        setattr(model_mod, path.split(".")[-1],
+                LowRankLinear(torch.zeros(out_f, int(k), dtype=dtype),
+                              torch.zeros(int(k), in_f, dtype=dtype)))
+    sd = torch.load(state_dict_path, map_location="cpu", weights_only=True)
+    model.load_state_dict(sd, strict=True)
+    del sd
+    return model.to(device)
+
+
+def materialize_all(model):
+    """Fuse every LowRankLinear back into a dense nn.Linear in place."""
+    for name, mod in list(model.named_modules()):
+        if isinstance(mod, LowRankLinear):
+            parent = model
+            for p in name.split(".")[:-1]:
+                parent = getattr(parent, p)
+            setattr(parent, name.split(".")[-1], mod.materialize_())
+    return model

svd_llm/unheal/llama2-7b/ranks.json

@@ -0,0 +1,226 @@
+{
+ "model.layers.0.self_attn.q_proj": 1189,
+ "model.layers.0.self_attn.k_proj": 1189,
+ "model.layers.0.self_attn.v_proj": 1189,
+ "model.layers.0.self_attn.o_proj": 1189,
+ "model.layers.0.mlp.gate_proj": 1733,
+ "model.layers.0.mlp.up_proj": 1733,
+ "model.layers.0.mlp.down_proj": 1733,
+ "model.layers.1.self_attn.q_proj": 1189,
+ "model.layers.1.self_attn.k_proj": 1189,
+ "model.layers.1.self_attn.v_proj": 1189,
+ "model.layers.1.self_attn.o_proj": 1189,
+ "model.layers.1.mlp.gate_proj": 1733,
+ "model.layers.1.mlp.up_proj": 1733,
+ "model.layers.1.mlp.down_proj": 1733,
+ "model.layers.2.self_attn.q_proj": 1189,
+ "model.layers.2.self_attn.k_proj": 1189,
+ "model.layers.2.self_attn.v_proj": 1189,
+ "model.layers.2.self_attn.o_proj": 1189,
+ "model.layers.2.mlp.gate_proj": 1733,
+ "model.layers.2.mlp.up_proj": 1733,
+ "model.layers.2.mlp.down_proj": 1733,
+ "model.layers.3.self_attn.q_proj": 1189,
+ "model.layers.3.self_attn.k_proj": 1189,
+ "model.layers.3.self_attn.v_proj": 1189,
+ "model.layers.3.self_attn.o_proj": 1189,
+ "model.layers.3.mlp.gate_proj": 1733,
+ "model.layers.3.mlp.up_proj": 1733,
+ "model.layers.3.mlp.down_proj": 1733,
+ "model.layers.4.self_attn.q_proj": 1189,
+ "model.layers.4.self_attn.k_proj": 1189,
+ "model.layers.4.self_attn.v_proj": 1189,
+ "model.layers.4.self_attn.o_proj": 1189,
+ "model.layers.4.mlp.gate_proj": 1733,
+ "model.layers.4.mlp.up_proj": 1733,
+ "model.layers.4.mlp.down_proj": 1733,
+ "model.layers.5.self_attn.q_proj": 1189,
+ "model.layers.5.self_attn.k_proj": 1189,
+ "model.layers.5.self_attn.v_proj": 1189,
+ "model.layers.5.self_attn.o_proj": 1189,
+ "model.layers.5.mlp.gate_proj": 1733,
+ "model.layers.5.mlp.up_proj": 1733,
+ "model.layers.5.mlp.down_proj": 1733,
+ "model.layers.6.self_attn.q_proj": 1189,
+ "model.layers.6.self_attn.k_proj": 1189,
+ "model.layers.6.self_attn.v_proj": 1189,
+ "model.layers.6.self_attn.o_proj": 1189,
+ "model.layers.6.mlp.gate_proj": 1733,
+ "model.layers.6.mlp.up_proj": 1733,
+ "model.layers.6.mlp.down_proj": 1733,
+ "model.layers.7.self_attn.q_proj": 1189,
+ "model.layers.7.self_attn.k_proj": 1189,
+ "model.layers.7.self_attn.v_proj": 1189,
+ "model.layers.7.self_attn.o_proj": 1189,
+ "model.layers.7.mlp.gate_proj": 1733,
+ "model.layers.7.mlp.up_proj": 1733,
+ "model.layers.7.mlp.down_proj": 1733,
+ "model.layers.8.self_attn.q_proj": 1189,
+ "model.layers.8.self_attn.k_proj": 1189,
+ "model.layers.8.self_attn.v_proj": 1189,
+ "model.layers.8.self_attn.o_proj": 1189,
+ "model.layers.8.mlp.gate_proj": 1733,
+ "model.layers.8.mlp.up_proj": 1733,
+ "model.layers.8.mlp.down_proj": 1733,
+ "model.layers.9.self_attn.q_proj": 1189,
+ "model.layers.9.self_attn.k_proj": 1189,
+ "model.layers.9.self_attn.v_proj": 1189,
+ "model.layers.9.self_attn.o_proj": 1189,
+ "model.layers.9.mlp.gate_proj": 1733,
+ "model.layers.9.mlp.up_proj": 1733,
+ "model.layers.9.mlp.down_proj": 1733,
+ "model.layers.10.self_attn.q_proj": 1189,
+ "model.layers.10.self_attn.k_proj": 1189,
+ "model.layers.10.self_attn.v_proj": 1189,
+ "model.layers.10.self_attn.o_proj": 1189,
+ "model.layers.10.mlp.gate_proj": 1733,
+ "model.layers.10.mlp.up_proj": 1733,
+ "model.layers.10.mlp.down_proj": 1733,
+ "model.layers.11.self_attn.q_proj": 1189,
+ "model.layers.11.self_attn.k_proj": 1189,
+ "model.layers.11.self_attn.v_proj": 1189,
+ "model.layers.11.self_attn.o_proj": 1189,
+ "model.layers.11.mlp.gate_proj": 1733,
+ "model.layers.11.mlp.up_proj": 1733,
+ "model.layers.11.mlp.down_proj": 1733,
+ "model.layers.12.self_attn.q_proj": 1189,
+ "model.layers.12.self_attn.k_proj": 1189,
+ "model.layers.12.self_attn.v_proj": 1189,
+ "model.layers.12.self_attn.o_proj": 1189,
+ "model.layers.12.mlp.gate_proj": 1733,
+ "model.layers.12.mlp.up_proj": 1733,
+ "model.layers.12.mlp.down_proj": 1733,
+ "model.layers.13.self_attn.q_proj": 1189,
+ "model.layers.13.self_attn.k_proj": 1189,
+ "model.layers.13.self_attn.v_proj": 1189,
+ "model.layers.13.self_attn.o_proj": 1189,
+ "model.layers.13.mlp.gate_proj": 1733,
+ "model.layers.13.mlp.up_proj": 1733,
+ "model.layers.13.mlp.down_proj": 1733,
+ "model.layers.14.self_attn.q_proj": 1189,
+ "model.layers.14.self_attn.k_proj": 1189,
+ "model.layers.14.self_attn.v_proj": 1189,
+ "model.layers.14.self_attn.o_proj": 1189,
+ "model.layers.14.mlp.gate_proj": 1733,
+ "model.layers.14.mlp.up_proj": 1733,
+ "model.layers.14.mlp.down_proj": 1733,
+ "model.layers.15.self_attn.q_proj": 1189,
+ "model.layers.15.self_attn.k_proj": 1189,
+ "model.layers.15.self_attn.v_proj": 1189,
+ "model.layers.15.self_attn.o_proj": 1189,
+ "model.layers.15.mlp.gate_proj": 1733,
+ "model.layers.15.mlp.up_proj": 1733,
+ "model.layers.15.mlp.down_proj": 1733,
+ "model.layers.16.self_attn.q_proj": 1189,
+ "model.layers.16.self_attn.k_proj": 1189,
+ "model.layers.16.self_attn.v_proj": 1189,
+ "model.layers.16.self_attn.o_proj": 1189,
+ "model.layers.16.mlp.gate_proj": 1733,
+ "model.layers.16.mlp.up_proj": 1733,
+ "model.layers.16.mlp.down_proj": 1733,
+ "model.layers.17.self_attn.q_proj": 1189,
+ "model.layers.17.self_attn.k_proj": 1189,
+ "model.layers.17.self_attn.v_proj": 1189,
+ "model.layers.17.self_attn.o_proj": 1189,
+ "model.layers.17.mlp.gate_proj": 1733,
+ "model.layers.17.mlp.up_proj": 1733,
+ "model.layers.17.mlp.down_proj": 1733,
+ "model.layers.18.self_attn.q_proj": 1189,
+ "model.layers.18.self_attn.k_proj": 1189,
+ "model.layers.18.self_attn.v_proj": 1189,
+ "model.layers.18.self_attn.o_proj": 1189,
+ "model.layers.18.mlp.gate_proj": 1733,
+ "model.layers.18.mlp.up_proj": 1733,
+ "model.layers.18.mlp.down_proj": 1733,
+ "model.layers.19.self_attn.q_proj": 1189,
+ "model.layers.19.self_attn.k_proj": 1189,
+ "model.layers.19.self_attn.v_proj": 1189,
+ "model.layers.19.self_attn.o_proj": 1189,
+ "model.layers.19.mlp.gate_proj": 1733,
+ "model.layers.19.mlp.up_proj": 1733,
+ "model.layers.19.mlp.down_proj": 1733,
+ "model.layers.20.self_attn.q_proj": 1189,
+ "model.layers.20.self_attn.k_proj": 1189,
+ "model.layers.20.self_attn.v_proj": 1189,
+ "model.layers.20.self_attn.o_proj": 1189,
+ "model.layers.20.mlp.gate_proj": 1733,
+ "model.layers.20.mlp.up_proj": 1733,
+ "model.layers.20.mlp.down_proj": 1733,
+ "model.layers.21.self_attn.q_proj": 1189,
+ "model.layers.21.self_attn.k_proj": 1189,
+ "model.layers.21.self_attn.v_proj": 1189,
+ "model.layers.21.self_attn.o_proj": 1189,
+ "model.layers.21.mlp.gate_proj": 1733,
+ "model.layers.21.mlp.up_proj": 1733,
+ "model.layers.21.mlp.down_proj": 1733,
+ "model.layers.22.self_attn.q_proj": 1189,
+ "model.layers.22.self_attn.k_proj": 1189,
+ "model.layers.22.self_attn.v_proj": 1189,
+ "model.layers.22.self_attn.o_proj": 1189,
+ "model.layers.22.mlp.gate_proj": 1733,
+ "model.layers.22.mlp.up_proj": 1733,
+ "model.layers.22.mlp.down_proj": 1733,
+ "model.layers.23.self_attn.q_proj": 1189,
+ "model.layers.23.self_attn.k_proj": 1189,
+ "model.layers.23.self_attn.v_proj": 1189,
+ "model.layers.23.self_attn.o_proj": 1189,
+ "model.layers.23.mlp.gate_proj": 1733,
+ "model.layers.23.mlp.up_proj": 1733,
+ "model.layers.23.mlp.down_proj": 1733,
+ "model.layers.24.self_attn.q_proj": 1189,
+ "model.layers.24.self_attn.k_proj": 1189,
+ "model.layers.24.self_attn.v_proj": 1189,
+ "model.layers.24.self_attn.o_proj": 1189,
+ "model.layers.24.mlp.gate_proj": 1733,
+ "model.layers.24.mlp.up_proj": 1733,
+ "model.layers.24.mlp.down_proj": 1733,
+ "model.layers.25.self_attn.q_proj": 1189,
+ "model.layers.25.self_attn.k_proj": 1189,
+ "model.layers.25.self_attn.v_proj": 1189,
+ "model.layers.25.self_attn.o_proj": 1189,
+ "model.layers.25.mlp.gate_proj": 1733,
+ "model.layers.25.mlp.up_proj": 1733,
+ "model.layers.25.mlp.down_proj": 1733,
+ "model.layers.26.self_attn.q_proj": 1189,
+ "model.layers.26.self_attn.k_proj": 1189,
+ "model.layers.26.self_attn.v_proj": 1189,
+ "model.layers.26.self_attn.o_proj": 1189,
+ "model.layers.26.mlp.gate_proj": 1733,
+ "model.layers.26.mlp.up_proj": 1733,
+ "model.layers.26.mlp.down_proj": 1733,
+ "model.layers.27.self_attn.q_proj": 1189,
+ "model.layers.27.self_attn.k_proj": 1189,
+ "model.layers.27.self_attn.v_proj": 1189,
+ "model.layers.27.self_attn.o_proj": 1189,
+ "model.layers.27.mlp.gate_proj": 1733,
+ "model.layers.27.mlp.up_proj": 1733,
+ "model.layers.27.mlp.down_proj": 1733,
+ "model.layers.28.self_attn.q_proj": 1189,
+ "model.layers.28.self_attn.k_proj": 1189,
+ "model.layers.28.self_attn.v_proj": 1189,
+ "model.layers.28.self_attn.o_proj": 1189,
+ "model.layers.28.mlp.gate_proj": 1733,
+ "model.layers.28.mlp.up_proj": 1733,
+ "model.layers.28.mlp.down_proj": 1733,
+ "model.layers.29.self_attn.q_proj": 1189,
+ "model.layers.29.self_attn.k_proj": 1189,
+ "model.layers.29.self_attn.v_proj": 1189,
+ "model.layers.29.self_attn.o_proj": 1189,
+ "model.layers.29.mlp.gate_proj": 1733,
+ "model.layers.29.mlp.up_proj": 1733,
+ "model.layers.29.mlp.down_proj": 1733,
+ "model.layers.30.self_attn.q_proj": 1189,
+ "model.layers.30.self_attn.k_proj": 1189,
+ "model.layers.30.self_attn.v_proj": 1189,
+ "model.layers.30.self_attn.o_proj": 1189,
+ "model.layers.30.mlp.gate_proj": 1733,
+ "model.layers.30.mlp.up_proj": 1733,
+ "model.layers.30.mlp.down_proj": 1733,
+ "model.layers.31.self_attn.q_proj": 1189,
+ "model.layers.31.self_attn.k_proj": 1189,
+ "model.layers.31.self_attn.v_proj": 1189,
+ "model.layers.31.self_attn.o_proj": 1189,
+ "model.layers.31.mlp.gate_proj": 1733,
+ "model.layers.31.mlp.up_proj": 1733,
+ "model.layers.31.mlp.down_proj": 1733
+}

svd_llm/unheal/llama2-7b/state_dict_fp16.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f244b45c40d688d4a4e285026ac4272a3db892364d91963564b87b69cd21b8bd
+size 8044148257

svd_llm2/heal/llama2-13b/README.md

@@ -0,0 +1,29 @@
+---
+license: llama2
+base_model: NousResearch/Llama-2-13b-chat-hf
+tags: [svd, low-rank, model-compression, llama2, knowledge-distillation]
+---
+
+# SVD-LLM V2 @ 7.74B (59.47% kept of LLaMA-2-13B-chat), KD-healed 60M tokens
+
+SVD-LLM V2 (NAACL 2025): heterogeneous ratio allocation (Alg. 1) + loss-optimized truncation (Alg. 2). Official code never released; our faithful implementation (see per-layer ranks in ranks.json). Exact budget match: 7,740,286,464 vs target 7,740,282,880.
+
+- **HEALED checkpoint** (this dir). The matching **UNHEALED** pre-distillation weights are in
+  `svd-llm-v2-13b-unhealed/` — do not mix them up: same `ranks.json`, different `state_dict_fp16.pt` (see
+  `provenance.json` SHA256 here vs the unhealed dir's record).
+- Healing: pure forward-KL distillation from frozen NousResearch/Llama-2-13b-chat-hf teacher,
+  60M tokens, held-out KL **2.0268 -> 0.2858**, probe MMLU (5-shot, limit 40) **26.89% -> 44.96%**.
+- Identical recipe/protocol to the TR-3 13B and the 7B SVD baselines (see their dirs).
+
+## Load
+
+```python
+import json
+from lowrank_linear import build, materialize_all
+model = build("NousResearch/Llama-2-13b-chat-hf", json.load(open("ranks.json")),
+              "state_dict_fp16.pt", device="cuda")
+materialize_all(model)   # inference-only fusion U@V
+```
+
+Full provenance (data mixture + revisions, decontamination, SHA256 of every input and
+checkpoint, 12 x 5M milestone trajectory) in `provenance.json`.

svd_llm2/heal/llama2-13b/config.json

@@ -0,0 +1,26 @@
+{
+  "_name_or_path": null,
+  "architectures": [
+    "LlamaForCausalLM"
+  ],
+  "bos_token_id": 1,
+  "eos_token_id": 2,
+  "hidden_act": "silu",
+  "hidden_size": 5120,
+  "initializer_range": 0.02,
+  "intermediate_size": 13824,
+  "max_position_embeddings": 4096,
+  "model_type": "llama",
+  "num_attention_heads": 40,
+  "num_hidden_layers": 40,
+  "num_key_value_heads": 40,
+  "pad_token_id": 0,
+  "pretraining_tp": 1,
+  "rms_norm_eps": 1e-05,
+  "rope_scaling": null,
+  "tie_word_embeddings": false,
+  "torch_dtype": "float16",
+  "transformers_version": "4.31.0.dev0",
+  "use_cache": true,
+  "vocab_size": 32000
+}

svd_llm2/heal/llama2-13b/lowrank_linear.py

@@ -0,0 +1,83 @@
+"""Minimal standalone loader for the SVD-baseline compressed LLaMA-2-7B checkpoints.
+
+No dependency beyond torch + transformers. The state_dict stores every compressed
+projection as two factors of a LowRankLinear (A: out x k, B: k x in); ranks.json
+holds the per-layer integer rank of each projection. Untouched parameters
+(embeddings, norms) keep their stock transformers names.
+
+Usage:
+    import json, torch
+    from lowrank_linear import build
+    model = build(
+        base_id="NousResearch/Llama-2-7b-chat-hf",
+        ranks=json.load(open("ranks.json")),
+        state_dict_path="state_dict_fp16.pt",
+        device="cuda", dtype=torch.float16,
+    )
+    tokenizer = AutoTokenizer.from_pretrained("NousResearch/Llama-2-7b-chat-hf")
+    out = model.generate(tokenizer("2+2=", return_tensors="pt").to("cuda").input_ids,
+                         max_new_tokens=16, do_sample=False)
+    print(tokenizer.decode(out[0]))
+"""
+
+from __future__ import annotations
+
+import torch
+import torch.nn as nn
+import torch.nn.functional as F
+
+
+class LowRankLinear(nn.Module):
+    """y = A (B x).  A: out x r, B: r x in.  (LLaMA projections carry no bias.)"""
+
+    def __init__(self, A: torch.Tensor, B: torch.Tensor):
+        super().__init__()
+        self.A = nn.Parameter(A)
+        self.B = nn.Parameter(B)
+
+    def forward(self, x):
+        return F.linear(F.linear(x, self.B), self.A)
+
+    @torch.no_grad()
+    def materialize_(self) -> nn.Linear:
+        """Replace this module with the dense W = A @ B (inference only; never call
+        while training -- it cuts the gradient path)."""
+        W = (self.A @ self.B).to(self.A.dtype)
+        dense = nn.Linear(self.B.shape[1], self.A.shape[0], bias=False,
+                          device=W.device, dtype=W.dtype)
+        dense.weight.data = W
+        return dense
+
+
+def build(base_id: str, ranks: dict, state_dict_path: str,
+          device: str = "cuda", dtype=torch.float16):
+    from transformers import AutoModelForCausalLM
+
+    model = AutoModelForCausalLM.from_pretrained(base_id, torch_dtype=dtype)
+    search = {}
+    for name, mod in model.named_modules():
+        if isinstance(mod, nn.Linear):
+            search[name] = mod
+    for path, k in ranks.items():
+        out_f, in_f = search[path].weight.shape
+        model_mod = model
+        for p in path.split(".")[:-1]:
+            model_mod = getattr(model_mod, p)
+        setattr(model_mod, path.split(".")[-1],
+                LowRankLinear(torch.zeros(out_f, int(k), dtype=dtype),
+                              torch.zeros(int(k), in_f, dtype=dtype)))
+    sd = torch.load(state_dict_path, map_location="cpu", weights_only=True)
+    model.load_state_dict(sd, strict=True)
+    del sd
+    return model.to(device)
+
+
+def materialize_all(model):
+    """Fuse every LowRankLinear back into a dense nn.Linear in place."""
+    for name, mod in list(model.named_modules()):
+        if isinstance(mod, LowRankLinear):
+            parent = model
+            for p in name.split(".")[:-1]:
+                parent = getattr(parent, p)
+            setattr(parent, name.split(".")[-1], mod.materialize_())
+    return model

svd_llm2/heal/llama2-13b/provenance.json

@@ -0,0 +1,199 @@
+{
+  "method": "KD healing: alpha*T^2*KL(teacher||student) + (1-alpha)*CE",
+  "student_init": "/data1/rkp00022/zixuan/projects/TR3_main/checkpoints/svdllm2_r04158_tr3fmt/state_dict_fp16.pt",
+  "teacher": "NousResearch/Llama-2-13b-chat-hf",
+  "alpha": 1.0,
+  "temperature": 1.0,
+  "lr": 2e-05,
+  "tokens_seen": 59998208,
+  "epochs": 0.228,
+  "global_batch_tokens": 32768,
+  "dataset_mixture_pre_exclusion": {
+    "alpaca": 50000,
+    "openhermes": 200000,
+    "ultrachat": 100000
+  },
+  "dataset_revisions": {
+    "tatsu-lab/alpaca": "dce01c9b08f87459cf36a430d809084718273017",
+    "teknium/OpenHermes-2.5": "b82037821055c377bed0d495e72e46de3bc72e84",
+    "HuggingFaceH4/ultrachat_200k": "8049631c405ae6576f93f445c6b8166f76f5505a"
+  },
+  "canonical_pool_excluded_texts": 10000,
+  "train_texts_after_exclusion": 356000,
+  "chat_texts": 340000,
+  "mmlu_aux_texts_kept": 16000,
+  "mmlu_aux_split": "cais/mmlu auxiliary_train (official TRAIN split, disjoint from the evaluated test split; audit verified 0/99842 exact overlap)",
+  "mmlu_aux_stats": {
+    "requested_groups": 16000,
+    "rendered_groups": 16000,
+    "shots": 5,
+    "rows_total": 99842,
+    "rows_dropped_by_decontam": 71,
+    "rows_clean": 99771,
+    "revision": "c30699e8356da336a370243923dbaf21066bb9fe",
+    "format": "MMLU-shaped (NOT byte-identical to lm-eval; auxiliary_train has no subject)"
+  },
+  "gold_label_supervision": false,
+  "attribution_note": "alpha=1.0 => pure KD, no gold-label CE, so any MMLU change is attributable to distillation on in-domain text. alpha<1 adds CE on text containing answers = labeled MCQ training, which is a DIFFERENT and weaker claim.",
+  "train_tokens_total": 263112704,
+  "decontamination": {
+    "removed": {
+      "alpaca": 11,
+      "openhermes": 120,
+      "ultrachat": 33
+    }
+  },
+  "start_kl": 2.026848241686821,
+  "final_kl": 0.28583861887454987,
+  "start_mmlu": 0.268859649122807,
+  "final_mmlu": 0.44956140350877194,
+  "history": [
+    {
+      "tokens_m": 5.01,
+      "step": 153,
+      "held_out_kl": 0.5490630865097046,
+      "mmlu": 0.42105263157894735
+    },
+    {
+      "tokens_m": 10.03,
+      "step": 306,
+      "held_out_kl": 0.5244443118572235,
+      "mmlu": 0.4337719298245614
+    },
+    {
+      "tokens_m": 15.01,
+      "step": 458,
+      "held_out_kl": 0.49302633106708527,
+      "mmlu": 0.4337719298245614
+    },
+    {
+      "tokens_m": 20.02,
+      "step": 611,
+      "held_out_kl": 0.47258753702044487,
+      "mmlu": 0.40964912280701754
+    },
+    {
+      "tokens_m": 25.0,
+      "step": 763,
+      "held_out_kl": 0.4351510964334011,
+      "mmlu": 0.4236842105263158
+    },
+    {
+      "tokens_m": 30.02,
+      "step": 916,
+      "held_out_kl": 0.39489416405558586,
+      "mmlu": 0.42412280701754385
+    },
+    {
+      "tokens_m": 35.03,
+      "step": 1069,
+      "held_out_kl": 0.36299458146095276,
+      "mmlu": 0.4359649122807018
+    },
+    {
+      "tokens_m": 40.01,
+      "step": 1221,
+      "held_out_kl": 0.33124174922704697,
+      "mmlu": 0.4219298245614035
+    },
+    {
+      "tokens_m": 45.02,
+      "step": 1374,
+      "held_out_kl": 0.30896387435495853,
+      "mmlu": 0.4355263157894737
+    },
+    {
+      "tokens_m": 50.0,
+      "step": 1526,
+      "held_out_kl": 0.2978489622473717,
+      "mmlu": 0.44605263157894737
+    },
+    {
+      "tokens_m": 55.02,
+      "step": 1679,
+      "held_out_kl": 0.28977251797914505,
+      "mmlu": 0.44605263157894737
+    },
+    {
+      "tokens_m": 60.0,
+      "step": 1831,
+      "held_out_kl": 0.28583861887454987,
+      "mmlu": 0.44956140350877194,
+      "final": true
+    }
+  ],
+  "teacher_mmlu_same_limit": 0.5442982456140351,
+  "checkpoint": "/data1/rkp00022/zixuan/projects/TR3_main/checkpoints/kdheal_svdllm2r04158_h1/state_dict_fp16.pt",
+  "sha256": "1d7cee642e764d6041c21aa1c9177635d9dcc48da03de3d0869c834348a4a676",
+  "seed": 1234,
+  "mmlu_limit_per_subtask": 40,
+  "teacher_revision": "d73f5fa9c4bc135502e04c27b39660747172d76b",
+  "student_init_sha256": "f0dbe9c89696d52b78344dc7564c8c5bc6bb57ecf7cd95edaa26e6738db87e23",
+  "ranks_file": "/data1/rkp00022/zixuan/projects/TR3_main/checkpoints/svdllm2_r04158_tr3fmt/ranks.json",
+  "ranks_sha256": "deedef0f4a8e33a06a5584b6e5417b5b8262342a1cfbcf797afe12e2fbfb9cc5",
+  "input_identity_gate": {
+    "student_ckpt": {
+      "key": "svdllm2_r04158_tr3fmt",
+      "sha256": "f0dbe9c89696d52b78344dc7564c8c5bc6bb57ecf7cd95edaa26e6738db87e23",
+      "status": "unregistered_allowed"
+    },
+    "ranks": {
+      "key": "svdllm2_r04158_tr3fmt",
+      "sha256": "deedef0f4a8e33a06a5584b6e5417b5b8262342a1cfbcf797afe12e2fbfb9cc5",
+      "status": "unregistered_allowed"
+    }
+  },
+  "train_seed": 1234,
+  "mmlu_seed": 1234,
+  "dynamic_batch": false,
+  "dbl": null,
+  "snapshot_milestones": true,
+  "snapshots": [
+    "(PosixPath('/data1/rkp00022/zixuan/projects/TR3_main/checkpoints/kdheal_svdllm2r04158_h1_snap15m'), 15.007744)",
+    "(PosixPath('/data1/rkp00022/zixuan/projects/TR3_main/checkpoints/kdheal_svdllm2r04158_h1_snap30m'), 30.015488)",
+    "(PosixPath('/data1/rkp00022/zixuan/projects/TR3_main/checkpoints/kdheal_svdllm2r04158_h1_snap45m'), 45.023232)",
+    "(PosixPath('/data1/rkp00022/zixuan/projects/TR3_main/checkpoints/kdheal_svdllm2r04158_h1_snap50m'), 50.003968)",
+    "(PosixPath('/data1/rkp00022/zixuan/projects/TR3_main/checkpoints/kdheal_svdllm2r04158_h1_snap55m'), 55.017472)"
+  ],
+  "stream_skip_tokens_m": 0.0,
+  "stream_blocks_consumed": [
+    0,
+    58592
+  ],
+  "stream_note": "the block order is a seeded permutation consumed from index 0; stages sharing a --seed with stream_skip=0 re-train on the SAME blocks. [start, end] block indices are recorded so any continuation can be checked for overlap with what earlier stages consumed.",
+  "student_master_dtype": "bf16",
+  "fake_quant": null,
+  "rebalance_report": null,
+  "student_param_count": 7740286464,
+  "micro": 8,
+  "grad_accum": 4,
+  "seq_len": 1024,
+  "held_out_source": "canonical: mixture 2000/4000/4000, seed 1234, seq 1024, blocks 256:264 (identical to the compression gates)",
+  "decontam_stats": {
+    "mmlu": {
+      "items": 14042,
+      "new_ngrams": 764261,
+      "revision": "c30699e8356da336a370243923dbaf21066bb9fe"
+    },
+    "hellaswag": {
+      "items": 10042,
+      "new_ngrams": 1334302,
+      "revision": "218ec52e09a7e7462a5400043bb9a69a41d06b76"
+    },
+    "boolq": {
+      "items": 3270,
+      "new_ngrams": 276806,
+      "revision": "3de24cf8022e94f4ee4b9d55a6f539891524d646"
+    },
+    "triviaqa": {
+      "items": 17944,
+      "new_ngrams": 32608,
+      "revision": "0f7faf33a3908546c6fd5b73a660e0f8ff173c2f"
+    },
+    "gsm8k": {
+      "items": 1319,
+      "new_ngrams": 46281,
+      "revision": "740312add88f781978c0658806c59bc2815b9866"
+    }
+  }
+}

svd_llm2/heal/llama2-13b/ranks.json

@@ -0,0 +1,282 @@
+{
+ "model.layers.0.self_attn.q_proj": 2560,
+ "model.layers.0.self_attn.k_proj": 2560,
+ "model.layers.0.self_attn.v_proj": 2560,
+ "model.layers.0.self_attn.o_proj": 2560,
+ "model.layers.0.mlp.gate_proj": 3237,
+ "model.layers.0.mlp.up_proj": 3231,
+ "model.layers.0.mlp.down_proj": 3674,
+ "model.layers.1.self_attn.q_proj": 2514,
+ "model.layers.1.self_attn.k_proj": 2553,
+ "model.layers.1.self_attn.v_proj": 2288,
+ "model.layers.1.self_attn.o_proj": 2560,
+ "model.layers.1.mlp.gate_proj": 2862,
+ "model.layers.1.mlp.up_proj": 2816,
+ "model.layers.1.mlp.down_proj": 2909,
+ "model.layers.2.self_attn.q_proj": 1842,
+ "model.layers.2.self_attn.k_proj": 1847,
+ "model.layers.2.self_attn.v_proj": 1839,
+ "model.layers.2.self_attn.o_proj": 2126,
+ "model.layers.2.mlp.gate_proj": 2506,
+ "model.layers.2.mlp.up_proj": 2500,
+ "model.layers.2.mlp.down_proj": 2665,
+ "model.layers.3.self_attn.q_proj": 1735,
+ "model.layers.3.self_attn.k_proj": 1735,
+ "model.layers.3.self_attn.v_proj": 1742,
+ "model.layers.3.self_attn.o_proj": 2020,
+ "model.layers.3.mlp.gate_proj": 2363,
+ "model.layers.3.mlp.up_proj": 2369,
+ "model.layers.3.mlp.down_proj": 2569,
+ "model.layers.4.self_attn.q_proj": 1618,
+ "model.layers.4.self_attn.k_proj": 1618,
+ "model.layers.4.self_attn.v_proj": 1651,
+ "model.layers.4.self_attn.o_proj": 1888,
+ "model.layers.4.mlp.gate_proj": 2302,
+ "model.layers.4.mlp.up_proj": 2310,
+ "model.layers.4.mlp.down_proj": 2471,
+ "model.layers.5.self_attn.q_proj": 1603,
+ "model.layers.5.self_attn.k_proj": 1601,
+ "model.layers.5.self_attn.v_proj": 1624,
+ "model.layers.5.self_attn.o_proj": 1818,
+ "model.layers.5.mlp.gate_proj": 2253,
+ "model.layers.5.mlp.up_proj": 2262,
+ "model.layers.5.mlp.down_proj": 2388,
+ "model.layers.6.self_attn.q_proj": 1555,
+ "model.layers.6.self_attn.k_proj": 1551,
+ "model.layers.6.self_attn.v_proj": 1588,
+ "model.layers.6.self_attn.o_proj": 1778,
+ "model.layers.6.mlp.gate_proj": 2218,
+ "model.layers.6.mlp.up_proj": 2227,
+ "model.layers.6.mlp.down_proj": 2327,
+ "model.layers.7.self_attn.q_proj": 1556,
+ "model.layers.7.self_attn.k_proj": 1552,
+ "model.layers.7.self_attn.v_proj": 1590,
+ "model.layers.7.self_attn.o_proj": 1701,
+ "model.layers.7.mlp.gate_proj": 2199,
+ "model.layers.7.mlp.up_proj": 2208,
+ "model.layers.7.mlp.down_proj": 2277,
+ "model.layers.8.self_attn.q_proj": 1535,
+ "model.layers.8.self_attn.k_proj": 1531,
+ "model.layers.8.self_attn.v_proj": 1565,
+ "model.layers.8.self_attn.o_proj": 1663,
+ "model.layers.8.mlp.gate_proj": 2184,
+ "model.layers.8.mlp.up_proj": 2190,
+ "model.layers.8.mlp.down_proj": 2244,
+ "model.layers.9.self_attn.q_proj": 1504,
+ "model.layers.9.self_attn.k_proj": 1501,
+ "model.layers.9.self_attn.v_proj": 1544,
+ "model.layers.9.self_attn.o_proj": 1630,
+ "model.layers.9.mlp.gate_proj": 2169,
+ "model.layers.9.mlp.up_proj": 2170,
+ "model.layers.9.mlp.down_proj": 2219,
+ "model.layers.10.self_attn.q_proj": 1507,
+ "model.layers.10.self_attn.k_proj": 1506,
+ "model.layers.10.self_attn.v_proj": 1551,
+ "model.layers.10.self_attn.o_proj": 1583,
+ "model.layers.10.mlp.gate_proj": 2166,
+ "model.layers.10.mlp.up_proj": 2164,
+ "model.layers.10.mlp.down_proj": 2197,
+ "model.layers.11.self_attn.q_proj": 1509,
+ "model.layers.11.self_attn.k_proj": 1503,
+ "model.layers.11.self_attn.v_proj": 1554,
+ "model.layers.11.self_attn.o_proj": 1577,
+ "model.layers.11.mlp.gate_proj": 2165,
+ "model.layers.11.mlp.up_proj": 2159,
+ "model.layers.11.mlp.down_proj": 2180,
+ "model.layers.12.self_attn.q_proj": 1501,
+ "model.layers.12.self_attn.k_proj": 1495,
+ "model.layers.12.self_attn.v_proj": 1534,
+ "model.layers.12.self_attn.o_proj": 1552,
+ "model.layers.12.mlp.gate_proj": 2158,
+ "model.layers.12.mlp.up_proj": 2148,
+ "model.layers.12.mlp.down_proj": 2164,
+ "model.layers.13.self_attn.q_proj": 1494,
+ "model.layers.13.self_attn.k_proj": 1494,
+ "model.layers.13.self_attn.v_proj": 1514,
+ "model.layers.13.self_attn.o_proj": 1534,
+ "model.layers.13.mlp.gate_proj": 2149,
+ "model.layers.13.mlp.up_proj": 2138,
+ "model.layers.13.mlp.down_proj": 2142,
+ "model.layers.14.self_attn.q_proj": 1482,
+ "model.layers.14.self_attn.k_proj": 1480,
+ "model.layers.14.self_attn.v_proj": 1512,
+ "model.layers.14.self_attn.o_proj": 1499,
+ "model.layers.14.mlp.gate_proj": 2141,
+ "model.layers.14.mlp.up_proj": 2129,
+ "model.layers.14.mlp.down_proj": 2122,
+ "model.layers.15.self_attn.q_proj": 1486,
+ "model.layers.15.self_attn.k_proj": 1482,
+ "model.layers.15.self_attn.v_proj": 1507,
+ "model.layers.15.self_attn.o_proj": 1512,
+ "model.layers.15.mlp.gate_proj": 2131,
+ "model.layers.15.mlp.up_proj": 2117,
+ "model.layers.15.mlp.down_proj": 2103,
+ "model.layers.16.self_attn.q_proj": 1476,
+ "model.layers.16.self_attn.k_proj": 1471,
+ "model.layers.16.self_attn.v_proj": 1490,
+ "model.layers.16.self_attn.o_proj": 1504,
+ "model.layers.16.mlp.gate_proj": 2121,
+ "model.layers.16.mlp.up_proj": 2107,
+ "model.layers.16.mlp.down_proj": 2077,
+ "model.layers.17.self_attn.q_proj": 1468,
+ "model.layers.17.self_attn.k_proj": 1466,
+ "model.layers.17.self_attn.v_proj": 1482,
+ "model.layers.17.self_attn.o_proj": 1491,
+ "model.layers.17.mlp.gate_proj": 2108,
+ "model.layers.17.mlp.up_proj": 2098,
+ "model.layers.17.mlp.down_proj": 2066,
+ "model.layers.18.self_attn.q_proj": 1455,
+ "model.layers.18.self_attn.k_proj": 1453,
+ "model.layers.18.self_attn.v_proj": 1469,
+ "model.layers.18.self_attn.o_proj": 1498,
+ "model.layers.18.mlp.gate_proj": 2094,
+ "model.layers.18.mlp.up_proj": 2088,
+ "model.layers.18.mlp.down_proj": 2058,
+ "model.layers.19.self_attn.q_proj": 1461,
+ "model.layers.19.self_attn.k_proj": 1460,
+ "model.layers.19.self_attn.v_proj": 1468,
+ "model.layers.19.self_attn.o_proj": 1499,
+ "model.layers.19.mlp.gate_proj": 2083,
+ "model.layers.19.mlp.up_proj": 2079,
+ "model.layers.19.mlp.down_proj": 2044,
+ "model.layers.20.self_attn.q_proj": 1462,
+ "model.layers.20.self_attn.k_proj": 1462,
+ "model.layers.20.self_attn.v_proj": 1474,
+ "model.layers.20.self_attn.o_proj": 1480,
+ "model.layers.20.mlp.gate_proj": 2077,
+ "model.layers.20.mlp.up_proj": 2074,
+ "model.layers.20.mlp.down_proj": 2030,
+ "model.layers.21.self_attn.q_proj": 1464,
+ "model.layers.21.self_attn.k_proj": 1464,
+ "model.layers.21.self_attn.v_proj": 1463,
+ "model.layers.21.self_attn.o_proj": 1460,
+ "model.layers.21.mlp.gate_proj": 2072,
+ "model.layers.21.mlp.up_proj": 2072,
+ "model.layers.21.mlp.down_proj": 2010,
+ "model.layers.22.self_attn.q_proj": 1436,
+ "model.layers.22.self_attn.k_proj": 1437,
+ "model.layers.22.self_attn.v_proj": 1442,
+ "model.layers.22.self_attn.o_proj": 1491,
+ "model.layers.22.mlp.gate_proj": 2061,
+ "model.layers.22.mlp.up_proj": 2063,
+ "model.layers.22.mlp.down_proj": 2003,
+ "model.layers.23.self_attn.q_proj": 1442,
+ "model.layers.23.self_attn.k_proj": 1443,
+ "model.layers.23.self_attn.v_proj": 1442,
+ "model.layers.23.self_attn.o_proj": 1488,
+ "model.layers.23.mlp.gate_proj": 2051,
+ "model.layers.23.mlp.up_proj": 2056,
+ "model.layers.23.mlp.down_proj": 1994,
+ "model.layers.24.self_attn.q_proj": 1433,
+ "model.layers.24.self_attn.k_proj": 1435,
+ "model.layers.24.self_attn.v_proj": 1436,
+ "model.layers.24.self_attn.o_proj": 1495,
+ "model.layers.24.mlp.gate_proj": 2044,
+ "model.layers.24.mlp.up_proj": 2050,
+ "model.layers.24.mlp.down_proj": 1988,
+ "model.layers.25.self_attn.q_proj": 1428,
+ "model.layers.25.self_attn.k_proj": 1428,
+ "model.layers.25.self_attn.v_proj": 1425,
+ "model.layers.25.self_attn.o_proj": 1475,
+ "model.layers.25.mlp.gate_proj": 2035,
+ "model.layers.25.mlp.up_proj": 2042,
+ "model.layers.25.mlp.down_proj": 1978,
+ "model.layers.26.self_attn.q_proj": 1424,
+ "model.layers.26.self_attn.k_proj": 1424,
+ "model.layers.26.self_attn.v_proj": 1421,
+ "model.layers.26.self_attn.o_proj": 1474,
+ "model.layers.26.mlp.gate_proj": 2027,
+ "model.layers.26.mlp.up_proj": 2035,
+ "model.layers.26.mlp.down_proj": 1969,
+ "model.layers.27.self_attn.q_proj": 1408,
+ "model.layers.27.self_attn.k_proj": 1409,
+ "model.layers.27.self_attn.v_proj": 1410,
+ "model.layers.27.self_attn.o_proj": 1474,
+ "model.layers.27.mlp.gate_proj": 2019,
+ "model.layers.27.mlp.up_proj": 2027,
+ "model.layers.27.mlp.down_proj": 1963,
+ "model.layers.28.self_attn.q_proj": 1409,
+ "model.layers.28.self_attn.k_proj": 1410,
+ "model.layers.28.self_attn.v_proj": 1408,
+ "model.layers.28.self_attn.o_proj": 1473,
+ "model.layers.28.mlp.gate_proj": 2010,
+ "model.layers.28.mlp.up_proj": 2017,
+ "model.layers.28.mlp.down_proj": 1955,
+ "model.layers.29.self_attn.q_proj": 1397,
+ "model.layers.29.self_attn.k_proj": 1398,
+ "model.layers.29.self_attn.v_proj": 1399,
+ "model.layers.29.self_attn.o_proj": 1464,
+ "model.layers.29.mlp.gate_proj": 2002,
+ "model.layers.29.mlp.up_proj": 2009,
+ "model.layers.29.mlp.down_proj": 1948,
+ "model.layers.30.self_attn.q_proj": 1389,
+ "model.layers.30.self_attn.k_proj": 1391,
+ "model.layers.30.self_attn.v_proj": 1385,
+ "model.layers.30.self_attn.o_proj": 1436,
+ "model.layers.30.mlp.gate_proj": 1993,
+ "model.layers.30.mlp.up_proj": 2001,
+ "model.layers.30.mlp.down_proj": 1941,
+ "model.layers.31.self_attn.q_proj": 1396,
+ "model.layers.31.self_attn.k_proj": 1396,
+ "model.layers.31.self_attn.v_proj": 1395,
+ "model.layers.31.self_attn.o_proj": 1446,
+ "model.layers.31.mlp.gate_proj": 1985,
+ "model.layers.31.mlp.up_proj": 1993,
+ "model.layers.31.mlp.down_proj": 1936,
+ "model.layers.32.self_attn.q_proj": 1393,
+ "model.layers.32.self_attn.k_proj": 1395,
+ "model.layers.32.self_attn.v_proj": 1383,
+ "model.layers.32.self_attn.o_proj": 1429,
+ "model.layers.32.mlp.gate_proj": 1979,
+ "model.layers.32.mlp.up_proj": 1985,
+ "model.layers.32.mlp.down_proj": 1929,
+ "model.layers.33.self_attn.q_proj": 1399,
+ "model.layers.33.self_attn.k_proj": 1400,
+ "model.layers.33.self_attn.v_proj": 1390,
+ "model.layers.33.self_attn.o_proj": 1430,
+ "model.layers.33.mlp.gate_proj": 1973,
+ "model.layers.33.mlp.up_proj": 1978,
+ "model.layers.33.mlp.down_proj": 1921,
+ "model.layers.34.self_attn.q_proj": 1401,
+ "model.layers.34.self_attn.k_proj": 1401,
+ "model.layers.34.self_attn.v_proj": 1383,
+ "model.layers.34.self_attn.o_proj": 1424,
+ "model.layers.34.mlp.gate_proj": 1968,
+ "model.layers.34.mlp.up_proj": 1972,
+ "model.layers.34.mlp.down_proj": 1910,
+ "model.layers.35.self_attn.q_proj": 1407,
+ "model.layers.35.self_attn.k_proj": 1407,
+ "model.layers.35.self_attn.v_proj": 1388,
+ "model.layers.35.self_attn.o_proj": 1412,
+ "model.layers.35.mlp.gate_proj": 1964,
+ "model.layers.35.mlp.up_proj": 1966,
+ "model.layers.35.mlp.down_proj": 1897,
+ "model.layers.36.self_attn.q_proj": 1417,
+ "model.layers.36.self_attn.k_proj": 1416,
+ "model.layers.36.self_attn.v_proj": 1392,
+ "model.layers.36.self_attn.o_proj": 1380,
+ "model.layers.36.mlp.gate_proj": 1962,
+ "model.layers.36.mlp.up_proj": 1963,
+ "model.layers.36.mlp.down_proj": 1882,
+ "model.layers.37.self_attn.q_proj": 1438,
+ "model.layers.37.self_attn.k_proj": 1436,
+ "model.layers.37.self_attn.v_proj": 1398,
+ "model.layers.37.self_attn.o_proj": 1391,
+ "model.layers.37.mlp.gate_proj": 1964,
+ "model.layers.37.mlp.up_proj": 1966,
+ "model.layers.37.mlp.down_proj": 1877,
+ "model.layers.38.self_attn.q_proj": 1446,
+ "model.layers.38.self_attn.k_proj": 1442,
+ "model.layers.38.self_attn.v_proj": 1392,
+ "model.layers.38.self_attn.o_proj": 1376,
+ "model.layers.38.mlp.gate_proj": 1971,
+ "model.layers.38.mlp.up_proj": 1978,
+ "model.layers.38.mlp.down_proj": 1877,
+ "model.layers.39.self_attn.q_proj": 1470,
+ "model.layers.39.self_attn.k_proj": 1466,
+ "model.layers.39.self_attn.v_proj": 1423,
+ "model.layers.39.self_attn.o_proj": 1389,
+ "model.layers.39.mlp.gate_proj": 1998,
+ "model.layers.39.mlp.up_proj": 2011,
+ "model.layers.39.mlp.down_proj": 1865
+}

svd_llm2/heal/llama2-13b/state_dict_fp16.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:1d7cee642e764d6041c21aa1c9177635d9dcc48da03de3d0869c834348a4a676
+size 15480788552

svd_llm2/heal/llama2-7b/README.md

@@ -0,0 +1,43 @@
+---
+license: llama2
+base_model: NousResearch/Llama-2-7b-chat-hf
+tags:
+  - svd
+  - low-rank
+  - model-compression
+  - llama2
+---
+
+# SVD-LLM V2 (NAACL 2025): LLaMA-2-7B-chat @ 4.02B, KD-healed
+
+SVD-LLM V2: heterogeneous compression-ratio allocation from theoretical truncation loss (Algorithm 1) + loss-optimized weight truncation (Algorithm 2). Official code was never released; this is our faithful implementation of the published algorithms (per-layer ranks in ranks.json show the allocation).
+
+- **4.023B params** (down from 6.74B, ~40% reduction)
+- Healed with pure forward-KL distillation from the dense 7B teacher: **60M tokens**,
+  held-out KL **2.5806 -> 0.3365**, probe MMLU (5-shot, limit 40/subtask) **25.70% -> 40.83%**
+- Formal full-benchmark evaluation under the locked protocol (lm-eval 0.4.12, fp16, seed
+  1234, 5/10/0/5/5-shot MMLU/HS/BQ/TQA/GSM): **[being finalized; see
+  the TR-3 paper's Table 2]**
+
+| Reference | MMLU | HellaSwag | BoolQ | TriviaQA | GSM8K |
+|---|---:|---:|---:|---:|---:|
+| unhealed 4.02B (same structure) | 26.85 | 34.58 | 37.83 | 1.81 | 0.00 |
+| TR-3 (same budget + same healing) | 43.21 | 70.75 | 77.09 | 36.93 | 9.40 |
+| FP16 dense 6.74B | 47.22 | 78.59 | 80.58 | 57.17 | 22.67 |
+
+## How to load
+
+```python
+import json
+from lowrank_linear import build, materialize_all
+
+model = build("NousResearch/Llama-2-7b-chat-hf",
+              json.load(open("ranks.json")), "state_dict_fp16.pt",
+              device="cuda")
+# for generation/eval: fuse the factors first (inference-only)
+materialize_all(model)
+```
+
+`ranks.json` maps each compressed projection path to its integer rank; untouched weights are
+stock LLaMA-2-7B-chat tensors. The full healing-provenance record (data mixture & revisions,
+decontamination, SHA256 of inputs/checkpoint, 5M milestone trajectory) is in `provenance.json`.

svd_llm2/heal/llama2-7b/config.json

@@ -0,0 +1,28 @@
+{
+  "_name_or_path": "NousResearch/Llama-2-7b-chat-hf",
+  "architectures": [
+    "LlamaForCausalLM"
+  ],
+  "attention_bias": false,
+  "bos_token_id": 1,
+  "eos_token_id": 2,
+  "hidden_act": "silu",
+  "hidden_size": 4096,
+  "initializer_range": 0.02,
+  "intermediate_size": 11008,
+  "max_position_embeddings": 4096,
+  "model_type": "llama",
+  "num_attention_heads": 32,
+  "num_hidden_layers": 32,
+  "num_key_value_heads": 32,
+  "pad_token_id": 0,
+  "pretraining_tp": 1,
+  "rms_norm_eps": 1e-05,
+  "rope_scaling": null,
+  "rope_theta": 10000.0,
+  "tie_word_embeddings": false,
+  "torch_dtype": "float16",
+  "transformers_version": "4.35.2",
+  "use_cache": true,
+  "vocab_size": 32000
+}

svd_llm2/heal/llama2-7b/lowrank_linear.py

@@ -0,0 +1,83 @@
+"""Minimal standalone loader for the SVD-baseline compressed LLaMA-2-7B checkpoints.
+
+No dependency beyond torch + transformers. The state_dict stores every compressed
+projection as two factors of a LowRankLinear (A: out x k, B: k x in); ranks.json
+holds the per-layer integer rank of each projection. Untouched parameters
+(embeddings, norms) keep their stock transformers names.
+
+Usage:
+    import json, torch
+    from lowrank_linear import build
+    model = build(
+        base_id="NousResearch/Llama-2-7b-chat-hf",
+        ranks=json.load(open("ranks.json")),
+        state_dict_path="state_dict_fp16.pt",
+        device="cuda", dtype=torch.float16,
+    )
+    tokenizer = AutoTokenizer.from_pretrained("NousResearch/Llama-2-7b-chat-hf")
+    out = model.generate(tokenizer("2+2=", return_tensors="pt").to("cuda").input_ids,
+                         max_new_tokens=16, do_sample=False)
+    print(tokenizer.decode(out[0]))
+"""
+
+from __future__ import annotations
+
+import torch
+import torch.nn as nn
+import torch.nn.functional as F
+
+
+class LowRankLinear(nn.Module):
+    """y = A (B x).  A: out x r, B: r x in.  (LLaMA projections carry no bias.)"""
+
+    def __init__(self, A: torch.Tensor, B: torch.Tensor):
+        super().__init__()
+        self.A = nn.Parameter(A)
+        self.B = nn.Parameter(B)
+
+    def forward(self, x):
+        return F.linear(F.linear(x, self.B), self.A)
+
+    @torch.no_grad()
+    def materialize_(self) -> nn.Linear:
+        """Replace this module with the dense W = A @ B (inference only; never call
+        while training -- it cuts the gradient path)."""
+        W = (self.A @ self.B).to(self.A.dtype)
+        dense = nn.Linear(self.B.shape[1], self.A.shape[0], bias=False,
+                          device=W.device, dtype=W.dtype)
+        dense.weight.data = W
+        return dense
+
+
+def build(base_id: str, ranks: dict, state_dict_path: str,
+          device: str = "cuda", dtype=torch.float16):
+    from transformers import AutoModelForCausalLM
+
+    model = AutoModelForCausalLM.from_pretrained(base_id, torch_dtype=dtype)
+    search = {}
+    for name, mod in model.named_modules():
+        if isinstance(mod, nn.Linear):
+            search[name] = mod
+    for path, k in ranks.items():
+        out_f, in_f = search[path].weight.shape
+        model_mod = model
+        for p in path.split(".")[:-1]:
+            model_mod = getattr(model_mod, p)
+        setattr(model_mod, path.split(".")[-1],
+                LowRankLinear(torch.zeros(out_f, int(k), dtype=dtype),
+                              torch.zeros(int(k), in_f, dtype=dtype)))
+    sd = torch.load(state_dict_path, map_location="cpu", weights_only=True)
+    model.load_state_dict(sd, strict=True)
+    del sd
+    return model.to(device)
+
+
+def materialize_all(model):
+    """Fuse every LowRankLinear back into a dense nn.Linear in place."""
+    for name, mod in list(model.named_modules()):
+        if isinstance(mod, LowRankLinear):
+            parent = model
+            for p in name.split(".")[:-1]:
+                parent = getattr(parent, p)
+            setattr(parent, name.split(".")[-1], mod.materialize_())
+    return model

svd_llm2/heal/llama2-7b/provenance.json

@@ -0,0 +1,196 @@
+{
+  "method": "KD healing: alpha*T^2*KL(teacher||student) + (1-alpha)*CE",
+  "student_init": "/data1/rkp00022/zixuan/projects/TR3_main/checkpoints/svdllm2_r04193_tr3fmt/state_dict_fp16.pt",
+  "teacher": "NousResearch/Llama-2-7b-chat-hf",
+  "alpha": 1.0,
+  "temperature": 1.0,
+  "lr": 2e-05,
+  "tokens_seen": 59998208,
+  "epochs": 0.228,
+  "global_batch_tokens": 32768,
+  "dataset_mixture_pre_exclusion": {
+    "alpaca": 50000,
+    "openhermes": 200000,
+    "ultrachat": 100000
+  },
+  "dataset_revisions": {
+    "tatsu-lab/alpaca": "dce01c9b08f87459cf36a430d809084718273017",
+    "teknium/OpenHermes-2.5": "b82037821055c377bed0d495e72e46de3bc72e84",
+    "HuggingFaceH4/ultrachat_200k": "8049631c405ae6576f93f445c6b8166f76f5505a"
+  },
+  "canonical_pool_excluded_texts": 10000,
+  "train_texts_after_exclusion": 356000,
+  "chat_texts": 340000,
+  "mmlu_aux_texts_kept": 16000,
+  "mmlu_aux_split": "cais/mmlu auxiliary_train (official TRAIN split, disjoint from the evaluated test split; audit verified 0/99842 exact overlap)",
+  "mmlu_aux_stats": {
+    "requested_groups": 16000,
+    "rendered_groups": 16000,
+    "shots": 5,
+    "rows_total": 99842,
+    "rows_dropped_by_decontam": 71,
+    "rows_clean": 99771,
+    "revision": "c30699e8356da336a370243923dbaf21066bb9fe",
+    "format": "MMLU-shaped (NOT byte-identical to lm-eval; auxiliary_train has no subject)"
+  },
+  "gold_label_supervision": false,
+  "attribution_note": "alpha=1.0 => pure KD, no gold-label CE, so any MMLU change is attributable to distillation on in-domain text. alpha<1 adds CE on text containing answers = labeled MCQ training, which is a DIFFERENT and weaker claim.",
+  "train_tokens_total": 263112704,
+  "decontamination": {
+    "removed": {
+      "alpaca": 11,
+      "openhermes": 120,
+      "ultrachat": 33
+    }
+  },
+  "start_kl": 2.5805691182613373,
+  "final_kl": 0.33650884963572025,
+  "start_mmlu": 0.25701754385964914,
+  "final_mmlu": 0.4083333333333333,
+  "history": [
+    {
+      "tokens_m": 5.01,
+      "step": 153,
+      "held_out_kl": 0.647051740437746,
+      "mmlu": 0.2894736842105263
+    },
+    {
+      "tokens_m": 10.03,
+      "step": 306,
+      "held_out_kl": 0.594494417309761,
+      "mmlu": 0.368859649122807
+    },
+    {
+      "tokens_m": 15.01,
+      "step": 458,
+      "held_out_kl": 0.5670272633433342,
+      "mmlu": 0.3592105263157895
+    },
+    {
+      "tokens_m": 20.02,
+      "step": 611,
+      "held_out_kl": 0.5243954174220562,
+      "mmlu": 0.36622807017543857
+    },
+    {
+      "tokens_m": 25.0,
+      "step": 763,
+      "held_out_kl": 0.5039057768881321,
+      "mmlu": 0.38859649122807016
+    },
+    {
+      "tokens_m": 30.02,
+      "step": 916,
+      "held_out_kl": 0.44497134163975716,
+      "mmlu": 0.37543859649122807
+    },
+    {
+      "tokens_m": 35.03,
+      "step": 1069,
+      "held_out_kl": 0.402653768658638,
+      "mmlu": 0.39342105263157895
+    },
+    {
+      "tokens_m": 40.01,
+      "step": 1221,
+      "held_out_kl": 0.380719942972064,
+      "mmlu": 0.393859649122807
+    },
+    {
+      "tokens_m": 45.02,
+      "step": 1374,
+      "held_out_kl": 0.3613550402224064,
+      "mmlu": 0.4105263157894737
+    },
+    {
+      "tokens_m": 50.0,
+      "step": 1526,
+      "held_out_kl": 0.3475096616894007,
+      "mmlu": 0.39780701754385966
+    },
+    {
+      "tokens_m": 55.02,
+      "step": 1679,
+      "held_out_kl": 0.34086795896291733,
+      "mmlu": 0.40482456140350875
+    },
+    {
+      "tokens_m": 60.0,
+      "step": 1831,
+      "held_out_kl": 0.33650884963572025,
+      "mmlu": 0.4083333333333333,
+      "final": true
+    }
+  ],
+  "teacher_mmlu_same_limit": 0.47543859649122805,
+  "checkpoint": "/data1/rkp00022/zixuan/projects/TR3_main/checkpoints/kdheal_svdllm2r04193_h1/state_dict_fp16.pt",
+  "sha256": "2c48578d7ddbe5f937c2436f35ad8ffbda01d3be45b4c4eab62e1e1c6104501c",
+  "seed": 1234,
+  "mmlu_limit_per_subtask": 40,
+  "teacher_revision": "351844e75ed0bcbbe3f10671b3c808d2b83894ee",
+  "student_init_sha256": "33e0dba497c0bfe3b4532b4c6c0e07274a2ca85219b475fb2c26978d5ae6dfbd",
+  "ranks_file": "/data1/rkp00022/zixuan/projects/TR3_main/checkpoints/svdllm2_r04193_tr3fmt/ranks.json",
+  "ranks_sha256": "90ba50885421afe8d0d19024aaf52bfbbee39433f99b1e934c4bf15a20f556a3",
+  "input_identity_gate": {
+    "student_ckpt": {
+      "key": "svdllm2_r04193_tr3fmt",
+      "sha256": "33e0dba497c0bfe3b4532b4c6c0e07274a2ca85219b475fb2c26978d5ae6dfbd",
+      "status": "unregistered_allowed"
+    },
+    "ranks": {
+      "key": "svdllm2_r04193_tr3fmt",
+      "sha256": "90ba50885421afe8d0d19024aaf52bfbbee39433f99b1e934c4bf15a20f556a3",
+      "status": "unregistered_allowed"
+    }
+  },
+  "train_seed": 1234,
+  "mmlu_seed": 1234,
+  "dynamic_batch": false,
+  "dbl": null,
+  "snapshot_milestones": true,
+  "snapshots": [
+    "(PosixPath('/data1/rkp00022/zixuan/projects/TR3_main/checkpoints/kdheal_svdllm2r04193_h1_snap50m'), 50.003968)",
+    "(PosixPath('/data1/rkp00022/zixuan/projects/TR3_main/checkpoints/kdheal_svdllm2r04193_h1_snap55m'), 55.017472)"
+  ],
+  "stream_skip_tokens_m": 0.0,
+  "stream_blocks_consumed": [
+    0,
+    58592
+  ],
+  "stream_note": "the block order is a seeded permutation consumed from index 0; stages sharing a --seed with stream_skip=0 re-train on the SAME blocks. [start, end] block indices are recorded so any continuation can be checked for overlap with what earlier stages consumed.",
+  "student_master_dtype": "bf16",
+  "fake_quant": null,
+  "rebalance_report": null,
+  "student_param_count": 4023119360,
+  "micro": 16,
+  "grad_accum": 2,
+  "seq_len": 1024,
+  "held_out_source": "canonical: mixture 2000/4000/4000, seed 1234, seq 1024, blocks 256:264 (identical to the compression gates)",
+  "decontam_stats": {
+    "mmlu": {
+      "items": 14042,
+      "new_ngrams": 764261,
+      "revision": "c30699e8356da336a370243923dbaf21066bb9fe"
+    },
+    "hellaswag": {
+      "items": 10042,
+      "new_ngrams": 1334302,
+      "revision": "218ec52e09a7e7462a5400043bb9a69a41d06b76"
+    },
+    "boolq": {
+      "items": 3270,
+      "new_ngrams": 276806,
+      "revision": "3de24cf8022e94f4ee4b9d55a6f539891524d646"
+    },
+    "triviaqa": {
+      "items": 17944,
+      "new_ngrams": 32608,
+      "revision": "0f7faf33a3908546c6fd5b73a660e0f8ff173c2f"
+    },
+    "gsm8k": {
+      "items": 1319,
+      "new_ngrams": 46281,
+      "revision": "740312add88f781978c0658806c59bc2815b9866"
+    }
+  }
+}

svd_llm2/heal/llama2-7b/ranks.json

@@ -0,0 +1,226 @@
+{
+ "model.layers.0.self_attn.q_proj": 2048,
+ "model.layers.0.self_attn.k_proj": 2048,
+ "model.layers.0.self_attn.v_proj": 2048,
+ "model.layers.0.self_attn.o_proj": 2048,
+ "model.layers.0.mlp.gate_proj": 2341,
+ "model.layers.0.mlp.up_proj": 2332,
+ "model.layers.0.mlp.down_proj": 2887,
+ "model.layers.1.self_attn.q_proj": 1911,
+ "model.layers.1.self_attn.k_proj": 1960,
+ "model.layers.1.self_attn.v_proj": 1826,
+ "model.layers.1.self_attn.o_proj": 2048,
+ "model.layers.1.mlp.gate_proj": 2115,
+ "model.layers.1.mlp.up_proj": 2094,
+ "model.layers.1.mlp.down_proj": 2280,
+ "model.layers.2.self_attn.q_proj": 1404,
+ "model.layers.2.self_attn.k_proj": 1405,
+ "model.layers.2.self_attn.v_proj": 1400,
+ "model.layers.2.self_attn.o_proj": 1662,
+ "model.layers.2.mlp.gate_proj": 1931,
+ "model.layers.2.mlp.up_proj": 1928,
+ "model.layers.2.mlp.down_proj": 2094,
+ "model.layers.3.self_attn.q_proj": 1252,
+ "model.layers.3.self_attn.k_proj": 1247,
+ "model.layers.3.self_attn.v_proj": 1285,
+ "model.layers.3.self_attn.o_proj": 1598,
+ "model.layers.3.mlp.gate_proj": 1821,
+ "model.layers.3.mlp.up_proj": 1826,
+ "model.layers.3.mlp.down_proj": 2002,
+ "model.layers.4.self_attn.q_proj": 1255,
+ "model.layers.4.self_attn.k_proj": 1252,
+ "model.layers.4.self_attn.v_proj": 1282,
+ "model.layers.4.self_attn.o_proj": 1490,
+ "model.layers.4.mlp.gate_proj": 1780,
+ "model.layers.4.mlp.up_proj": 1789,
+ "model.layers.4.mlp.down_proj": 1911,
+ "model.layers.5.self_attn.q_proj": 1239,
+ "model.layers.5.self_attn.k_proj": 1236,
+ "model.layers.5.self_attn.v_proj": 1266,
+ "model.layers.5.self_attn.o_proj": 1456,
+ "model.layers.5.mlp.gate_proj": 1757,
+ "model.layers.5.mlp.up_proj": 1765,
+ "model.layers.5.mlp.down_proj": 1855,
+ "model.layers.6.self_attn.q_proj": 1218,
+ "model.layers.6.self_attn.k_proj": 1214,
+ "model.layers.6.self_attn.v_proj": 1244,
+ "model.layers.6.self_attn.o_proj": 1393,
+ "model.layers.6.mlp.gate_proj": 1733,
+ "model.layers.6.mlp.up_proj": 1743,
+ "model.layers.6.mlp.down_proj": 1807,
+ "model.layers.7.self_attn.q_proj": 1209,
+ "model.layers.7.self_attn.k_proj": 1208,
+ "model.layers.7.self_attn.v_proj": 1237,
+ "model.layers.7.self_attn.o_proj": 1356,
+ "model.layers.7.mlp.gate_proj": 1718,
+ "model.layers.7.mlp.up_proj": 1726,
+ "model.layers.7.mlp.down_proj": 1770,
+ "model.layers.8.self_attn.q_proj": 1205,
+ "model.layers.8.self_attn.k_proj": 1201,
+ "model.layers.8.self_attn.v_proj": 1238,
+ "model.layers.8.self_attn.o_proj": 1314,
+ "model.layers.8.mlp.gate_proj": 1715,
+ "model.layers.8.mlp.up_proj": 1718,
+ "model.layers.8.mlp.down_proj": 1747,
+ "model.layers.9.self_attn.q_proj": 1193,
+ "model.layers.9.self_attn.k_proj": 1190,
+ "model.layers.9.self_attn.v_proj": 1229,
+ "model.layers.9.self_attn.o_proj": 1276,
+ "model.layers.9.mlp.gate_proj": 1711,
+ "model.layers.9.mlp.up_proj": 1711,
+ "model.layers.9.mlp.down_proj": 1725,
+ "model.layers.10.self_attn.q_proj": 1201,
+ "model.layers.10.self_attn.k_proj": 1194,
+ "model.layers.10.self_attn.v_proj": 1230,
+ "model.layers.10.self_attn.o_proj": 1252,
+ "model.layers.10.mlp.gate_proj": 1708,
+ "model.layers.10.mlp.up_proj": 1706,
+ "model.layers.10.mlp.down_proj": 1713,
+ "model.layers.11.self_attn.q_proj": 1199,
+ "model.layers.11.self_attn.k_proj": 1196,
+ "model.layers.11.self_attn.v_proj": 1213,
+ "model.layers.11.self_attn.o_proj": 1258,
+ "model.layers.11.mlp.gate_proj": 1699,
+ "model.layers.11.mlp.up_proj": 1694,
+ "model.layers.11.mlp.down_proj": 1699,
+ "model.layers.12.self_attn.q_proj": 1180,
+ "model.layers.12.self_attn.k_proj": 1179,
+ "model.layers.12.self_attn.v_proj": 1206,
+ "model.layers.12.self_attn.o_proj": 1234,
+ "model.layers.12.mlp.gate_proj": 1692,
+ "model.layers.12.mlp.up_proj": 1686,
+ "model.layers.12.mlp.down_proj": 1683,
+ "model.layers.13.self_attn.q_proj": 1176,
+ "model.layers.13.self_attn.k_proj": 1170,
+ "model.layers.13.self_attn.v_proj": 1190,
+ "model.layers.13.self_attn.o_proj": 1231,
+ "model.layers.13.mlp.gate_proj": 1688,
+ "model.layers.13.mlp.up_proj": 1678,
+ "model.layers.13.mlp.down_proj": 1661,
+ "model.layers.14.self_attn.q_proj": 1181,
+ "model.layers.14.self_attn.k_proj": 1179,
+ "model.layers.14.self_attn.v_proj": 1192,
+ "model.layers.14.self_attn.o_proj": 1213,
+ "model.layers.14.mlp.gate_proj": 1679,
+ "model.layers.14.mlp.up_proj": 1669,
+ "model.layers.14.mlp.down_proj": 1644,
+ "model.layers.15.self_attn.q_proj": 1177,
+ "model.layers.15.self_attn.k_proj": 1174,
+ "model.layers.15.self_attn.v_proj": 1187,
+ "model.layers.15.self_attn.o_proj": 1198,
+ "model.layers.15.mlp.gate_proj": 1670,
+ "model.layers.15.mlp.up_proj": 1661,
+ "model.layers.15.mlp.down_proj": 1621,
+ "model.layers.16.self_attn.q_proj": 1176,
+ "model.layers.16.self_attn.k_proj": 1174,
+ "model.layers.16.self_attn.v_proj": 1173,
+ "model.layers.16.self_attn.o_proj": 1175,
+ "model.layers.16.mlp.gate_proj": 1662,
+ "model.layers.16.mlp.up_proj": 1655,
+ "model.layers.16.mlp.down_proj": 1595,
+ "model.layers.17.self_attn.q_proj": 1163,
+ "model.layers.17.self_attn.k_proj": 1163,
+ "model.layers.17.self_attn.v_proj": 1167,
+ "model.layers.17.self_attn.o_proj": 1193,
+ "model.layers.17.mlp.gate_proj": 1647,
+ "model.layers.17.mlp.up_proj": 1643,
+ "model.layers.17.mlp.down_proj": 1584,
+ "model.layers.18.self_attn.q_proj": 1150,
+ "model.layers.18.self_attn.k_proj": 1149,
+ "model.layers.18.self_attn.v_proj": 1151,
+ "model.layers.18.self_attn.o_proj": 1190,
+ "model.layers.18.mlp.gate_proj": 1632,
+ "model.layers.18.mlp.up_proj": 1631,
+ "model.layers.18.mlp.down_proj": 1568,
+ "model.layers.19.self_attn.q_proj": 1156,
+ "model.layers.19.self_attn.k_proj": 1157,
+ "model.layers.19.self_attn.v_proj": 1152,
+ "model.layers.19.self_attn.o_proj": 1184,
+ "model.layers.19.mlp.gate_proj": 1623,
+ "model.layers.19.mlp.up_proj": 1623,
+ "model.layers.19.mlp.down_proj": 1557,
+ "model.layers.20.self_attn.q_proj": 1155,
+ "model.layers.20.self_attn.k_proj": 1155,
+ "model.layers.20.self_attn.v_proj": 1146,
+ "model.layers.20.self_attn.o_proj": 1172,
+ "model.layers.20.mlp.gate_proj": 1617,
+ "model.layers.20.mlp.up_proj": 1619,
+ "model.layers.20.mlp.down_proj": 1540,
+ "model.layers.21.self_attn.q_proj": 1147,
+ "model.layers.21.self_attn.k_proj": 1147,
+ "model.layers.21.self_attn.v_proj": 1136,
+ "model.layers.21.self_attn.o_proj": 1174,
+ "model.layers.21.mlp.gate_proj": 1607,
+ "model.layers.21.mlp.up_proj": 1611,
+ "model.layers.21.mlp.down_proj": 1531,
+ "model.layers.22.self_attn.q_proj": 1138,
+ "model.layers.22.self_attn.k_proj": 1138,
+ "model.layers.22.self_attn.v_proj": 1129,
+ "model.layers.22.self_attn.o_proj": 1149,
+ "model.layers.22.mlp.gate_proj": 1599,
+ "model.layers.22.mlp.up_proj": 1604,
+ "model.layers.22.mlp.down_proj": 1517,
+ "model.layers.23.self_attn.q_proj": 1122,
+ "model.layers.23.self_attn.k_proj": 1123,
+ "model.layers.23.self_attn.v_proj": 1115,
+ "model.layers.23.self_attn.o_proj": 1139,
+ "model.layers.23.mlp.gate_proj": 1589,
+ "model.layers.23.mlp.up_proj": 1593,
+ "model.layers.23.mlp.down_proj": 1508,
+ "model.layers.24.self_attn.q_proj": 1138,
+ "model.layers.24.self_attn.k_proj": 1139,
+ "model.layers.24.self_attn.v_proj": 1118,
+ "model.layers.24.self_attn.o_proj": 1135,
+ "model.layers.24.mlp.gate_proj": 1580,
+ "model.layers.24.mlp.up_proj": 1585,
+ "model.layers.24.mlp.down_proj": 1501,
+ "model.layers.25.self_attn.q_proj": 1116,
+ "model.layers.25.self_attn.k_proj": 1116,
+ "model.layers.25.self_attn.v_proj": 1100,
+ "model.layers.25.self_attn.o_proj": 1141,
+ "model.layers.25.mlp.gate_proj": 1573,
+ "model.layers.25.mlp.up_proj": 1577,
+ "model.layers.25.mlp.down_proj": 1496,
+ "model.layers.26.self_attn.q_proj": 1127,
+ "model.layers.26.self_attn.k_proj": 1126,
+ "model.layers.26.self_attn.v_proj": 1103,
+ "model.layers.26.self_attn.o_proj": 1121,
+ "model.layers.26.mlp.gate_proj": 1565,
+ "model.layers.26.mlp.up_proj": 1569,
+ "model.layers.26.mlp.down_proj": 1487,
+ "model.layers.27.self_attn.q_proj": 1108,
+ "model.layers.27.self_attn.k_proj": 1107,
+ "model.layers.27.self_attn.v_proj": 1095,
+ "model.layers.27.self_attn.o_proj": 1109,
+ "model.layers.27.mlp.gate_proj": 1557,
+ "model.layers.27.mlp.up_proj": 1561,
+ "model.layers.27.mlp.down_proj": 1475,
+ "model.layers.28.self_attn.q_proj": 1113,
+ "model.layers.28.self_attn.k_proj": 1111,
+ "model.layers.28.self_attn.v_proj": 1090,
+ "model.layers.28.self_attn.o_proj": 1105,
+ "model.layers.28.mlp.gate_proj": 1555,
+ "model.layers.28.mlp.up_proj": 1556,
+ "model.layers.28.mlp.down_proj": 1462,
+ "model.layers.29.self_attn.q_proj": 1128,
+ "model.layers.29.self_attn.k_proj": 1127,
+ "model.layers.29.self_attn.v_proj": 1094,
+ "model.layers.29.self_attn.o_proj": 1094,
+ "model.layers.29.mlp.gate_proj": 1552,
+ "model.layers.29.mlp.up_proj": 1553,
+ "model.layers.29.mlp.down_proj": 1450,
+ "model.layers.30.self_attn.q_proj": 1116,
+ "model.layers.30.self_attn.k_proj": 1115,
+ "model.layers.30.self_attn.v_proj": 1087,
+ "model.layers.30.self_attn.o_proj": 1093,
+ "model.layers.30.mlp.gate_proj": 1552,
+ "model.layers.30.mlp.up_proj": 1556,
+ "model.layers.30.mlp.down_proj": 1439,
+ "model.layers.31.self_attn.q_proj": 1156,
+ "model.layers.31.self_attn.k_proj": 1152,
+ "model.layers.31.self_attn.v_proj": 1126,
+ "model.layers.31.self_attn.o_proj": 1102,
+ "model.layers.31.mlp.gate_proj": 1570,
+ "model.layers.31.mlp.up_proj": 1576,
+ "model.layers.31.mlp.down_proj": 1429
+}

svd_llm2/heal/llama2-7b/state_dict_fp16.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:2c48578d7ddbe5f937c2436f35ad8ffbda01d3be45b4c4eab62e1e1c6104501c
+size 8046409836

svd_llm2/unheal/llama2-13b/README.md

@@ -0,0 +1,16 @@
+---
+license: llama2
+base_model: NousResearch/Llama-2-13b-chat-hf
+tags: [svd, low-rank, model-compression, llama2]
+---
+
+# SVD-LLM V2 @ 7.74B (59.47% kept of LLaMA-2-13B-chat), UNHEALED
+
+SVD-LLM V2 (NAACL 2025): heterogeneous ratio allocation (Alg. 1) + loss-optimized truncation (Alg. 2). Official code never released; our faithful implementation (see per-layer ranks in ranks.json). Exact budget match: 7,740,286,464 vs target 7,740,282,880.
+
+- **UNHEALED checkpoint**: raw post-training compression weights, NO distillation. The KD-healed
+  counterpart is in `svd-llm-v2-13b/`. This dir exists so the healing delta is
+  measurable; use it as the compression-stage artifact, not as a deployable model.
+- Same `ranks.json` as the healed dir; weights differ (see `provenance.json` SHA256).
+
+Load exactly as in the sibling dir (`lowrank_linear.py`).

svd_llm2/unheal/llama2-13b/config.json

@@ -0,0 +1,26 @@
+{
+  "_name_or_path": null,
+  "architectures": [
+    "LlamaForCausalLM"
+  ],
+  "bos_token_id": 1,
+  "eos_token_id": 2,
+  "hidden_act": "silu",
+  "hidden_size": 5120,
+  "initializer_range": 0.02,
+  "intermediate_size": 13824,
+  "max_position_embeddings": 4096,
+  "model_type": "llama",
+  "num_attention_heads": 40,
+  "num_hidden_layers": 40,
+  "num_key_value_heads": 40,
+  "pad_token_id": 0,
+  "pretraining_tp": 1,
+  "rms_norm_eps": 1e-05,
+  "rope_scaling": null,
+  "tie_word_embeddings": false,
+  "torch_dtype": "float16",
+  "transformers_version": "4.31.0.dev0",
+  "use_cache": true,
+  "vocab_size": 32000
+}

svd_llm2/unheal/llama2-13b/lowrank_linear.py

@@ -0,0 +1,83 @@
+"""Minimal standalone loader for the SVD-baseline compressed LLaMA-2-7B checkpoints.
+
+No dependency beyond torch + transformers. The state_dict stores every compressed
+projection as two factors of a LowRankLinear (A: out x k, B: k x in); ranks.json
+holds the per-layer integer rank of each projection. Untouched parameters
+(embeddings, norms) keep their stock transformers names.
+
+Usage:
+    import json, torch
+    from lowrank_linear import build
+    model = build(
+        base_id="NousResearch/Llama-2-7b-chat-hf",
+        ranks=json.load(open("ranks.json")),
+        state_dict_path="state_dict_fp16.pt",
+        device="cuda", dtype=torch.float16,
+    )
+    tokenizer = AutoTokenizer.from_pretrained("NousResearch/Llama-2-7b-chat-hf")
+    out = model.generate(tokenizer("2+2=", return_tensors="pt").to("cuda").input_ids,
+                         max_new_tokens=16, do_sample=False)
+    print(tokenizer.decode(out[0]))
+"""
+
+from __future__ import annotations
+
+import torch
+import torch.nn as nn
+import torch.nn.functional as F
+
+
+class LowRankLinear(nn.Module):
+    """y = A (B x).  A: out x r, B: r x in.  (LLaMA projections carry no bias.)"""
+
+    def __init__(self, A: torch.Tensor, B: torch.Tensor):
+        super().__init__()
+        self.A = nn.Parameter(A)
+        self.B = nn.Parameter(B)
+
+    def forward(self, x):
+        return F.linear(F.linear(x, self.B), self.A)
+
+    @torch.no_grad()
+    def materialize_(self) -> nn.Linear:
+        """Replace this module with the dense W = A @ B (inference only; never call
+        while training -- it cuts the gradient path)."""
+        W = (self.A @ self.B).to(self.A.dtype)
+        dense = nn.Linear(self.B.shape[1], self.A.shape[0], bias=False,
+                          device=W.device, dtype=W.dtype)
+        dense.weight.data = W
+        return dense
+
+
+def build(base_id: str, ranks: dict, state_dict_path: str,
+          device: str = "cuda", dtype=torch.float16):
+    from transformers import AutoModelForCausalLM
+
+    model = AutoModelForCausalLM.from_pretrained(base_id, torch_dtype=dtype)
+    search = {}
+    for name, mod in model.named_modules():
+        if isinstance(mod, nn.Linear):
+            search[name] = mod
+    for path, k in ranks.items():
+        out_f, in_f = search[path].weight.shape
+        model_mod = model
+        for p in path.split(".")[:-1]:
+            model_mod = getattr(model_mod, p)
+        setattr(model_mod, path.split(".")[-1],
+                LowRankLinear(torch.zeros(out_f, int(k), dtype=dtype),
+                              torch.zeros(int(k), in_f, dtype=dtype)))
+    sd = torch.load(state_dict_path, map_location="cpu", weights_only=True)
+    model.load_state_dict(sd, strict=True)
+    del sd
+    return model.to(device)
+
+
+def materialize_all(model):
+    """Fuse every LowRankLinear back into a dense nn.Linear in place."""
+    for name, mod in list(model.named_modules()):
+        if isinstance(mod, LowRankLinear):
+            parent = model
+            for p in name.split(".")[:-1]:
+                parent = getattr(parent, p)
+            setattr(parent, name.split(".")[-1], mod.materialize_())
+    return model

svd_llm2/unheal/llama2-13b/provenance.json

@@ -0,0 +1,7 @@
+{
+ "kind": "unhealed SVD compression (TR3-format conversion, verified pure re-keying)",
+ "source_ckpt": "/data1/rkp00022/zixuan/projects/TR3_main/checkpoints/svdllm2_llama2_13b_chat_r04158/NousResearch_Llama_2_13b_chat_hf_svdllmv2_keep_0.5740.pt",
+ "converted_state_dict": "/data1/rkp00022/zixuan/projects/TR3_main/scratch/hf_push_13b/svd-llm-v2-13b-unhealed/state_dict_fp16.pt",
+ "state_dict_sha256": "f0dbe9c89696d52b78344dc7564c8c5bc6bb57ecf7cd95edaa26e6738db87e23",
+ "healed_counterpart": "svd-llm-v2-13b/state_dict_fp16.pt"
+}

svd_llm2/unheal/llama2-13b/ranks.json

@@ -0,0 +1,282 @@
+{
+ "model.layers.0.self_attn.q_proj": 2560,
+ "model.layers.0.self_attn.k_proj": 2560,
+ "model.layers.0.self_attn.v_proj": 2560,
+ "model.layers.0.self_attn.o_proj": 2560,
+ "model.layers.0.mlp.gate_proj": 3237,
+ "model.layers.0.mlp.up_proj": 3231,
+ "model.layers.0.mlp.down_proj": 3674,
+ "model.layers.1.self_attn.q_proj": 2514,
+ "model.layers.1.self_attn.k_proj": 2553,
+ "model.layers.1.self_attn.v_proj": 2288,
+ "model.layers.1.self_attn.o_proj": 2560,
+ "model.layers.1.mlp.gate_proj": 2862,
+ "model.layers.1.mlp.up_proj": 2816,
+ "model.layers.1.mlp.down_proj": 2909,
+ "model.layers.2.self_attn.q_proj": 1842,
+ "model.layers.2.self_attn.k_proj": 1847,
+ "model.layers.2.self_attn.v_proj": 1839,
+ "model.layers.2.self_attn.o_proj": 2126,
+ "model.layers.2.mlp.gate_proj": 2506,
+ "model.layers.2.mlp.up_proj": 2500,
+ "model.layers.2.mlp.down_proj": 2665,
+ "model.layers.3.self_attn.q_proj": 1735,
+ "model.layers.3.self_attn.k_proj": 1735,
+ "model.layers.3.self_attn.v_proj": 1742,
+ "model.layers.3.self_attn.o_proj": 2020,
+ "model.layers.3.mlp.gate_proj": 2363,
+ "model.layers.3.mlp.up_proj": 2369,
+ "model.layers.3.mlp.down_proj": 2569,
+ "model.layers.4.self_attn.q_proj": 1618,
+ "model.layers.4.self_attn.k_proj": 1618,
+ "model.layers.4.self_attn.v_proj": 1651,
+ "model.layers.4.self_attn.o_proj": 1888,
+ "model.layers.4.mlp.gate_proj": 2302,
+ "model.layers.4.mlp.up_proj": 2310,
+ "model.layers.4.mlp.down_proj": 2471,
+ "model.layers.5.self_attn.q_proj": 1603,
+ "model.layers.5.self_attn.k_proj": 1601,
+ "model.layers.5.self_attn.v_proj": 1624,
+ "model.layers.5.self_attn.o_proj": 1818,
+ "model.layers.5.mlp.gate_proj": 2253,
+ "model.layers.5.mlp.up_proj": 2262,
+ "model.layers.5.mlp.down_proj": 2388,
+ "model.layers.6.self_attn.q_proj": 1555,
+ "model.layers.6.self_attn.k_proj": 1551,
+ "model.layers.6.self_attn.v_proj": 1588,
+ "model.layers.6.self_attn.o_proj": 1778,
+ "model.layers.6.mlp.gate_proj": 2218,
+ "model.layers.6.mlp.up_proj": 2227,
+ "model.layers.6.mlp.down_proj": 2327,
+ "model.layers.7.self_attn.q_proj": 1556,
+ "model.layers.7.self_attn.k_proj": 1552,
+ "model.layers.7.self_attn.v_proj": 1590,
+ "model.layers.7.self_attn.o_proj": 1701,
+ "model.layers.7.mlp.gate_proj": 2199,
+ "model.layers.7.mlp.up_proj": 2208,
+ "model.layers.7.mlp.down_proj": 2277,
+ "model.layers.8.self_attn.q_proj": 1535,
+ "model.layers.8.self_attn.k_proj": 1531,
+ "model.layers.8.self_attn.v_proj": 1565,
+ "model.layers.8.self_attn.o_proj": 1663,
+ "model.layers.8.mlp.gate_proj": 2184,
+ "model.layers.8.mlp.up_proj": 2190,
+ "model.layers.8.mlp.down_proj": 2244,
+ "model.layers.9.self_attn.q_proj": 1504,
+ "model.layers.9.self_attn.k_proj": 1501,
+ "model.layers.9.self_attn.v_proj": 1544,
+ "model.layers.9.self_attn.o_proj": 1630,
+ "model.layers.9.mlp.gate_proj": 2169,
+ "model.layers.9.mlp.up_proj": 2170,
+ "model.layers.9.mlp.down_proj": 2219,
+ "model.layers.10.self_attn.q_proj": 1507,
+ "model.layers.10.self_attn.k_proj": 1506,
+ "model.layers.10.self_attn.v_proj": 1551,
+ "model.layers.10.self_attn.o_proj": 1583,
+ "model.layers.10.mlp.gate_proj": 2166,
+ "model.layers.10.mlp.up_proj": 2164,
+ "model.layers.10.mlp.down_proj": 2197,
+ "model.layers.11.self_attn.q_proj": 1509,
+ "model.layers.11.self_attn.k_proj": 1503,
+ "model.layers.11.self_attn.v_proj": 1554,
+ "model.layers.11.self_attn.o_proj": 1577,
+ "model.layers.11.mlp.gate_proj": 2165,
+ "model.layers.11.mlp.up_proj": 2159,
+ "model.layers.11.mlp.down_proj": 2180,
+ "model.layers.12.self_attn.q_proj": 1501,
+ "model.layers.12.self_attn.k_proj": 1495,
+ "model.layers.12.self_attn.v_proj": 1534,
+ "model.layers.12.self_attn.o_proj": 1552,
+ "model.layers.12.mlp.gate_proj": 2158,
+ "model.layers.12.mlp.up_proj": 2148,
+ "model.layers.12.mlp.down_proj": 2164,
+ "model.layers.13.self_attn.q_proj": 1494,
+ "model.layers.13.self_attn.k_proj": 1494,
+ "model.layers.13.self_attn.v_proj": 1514,
+ "model.layers.13.self_attn.o_proj": 1534,
+ "model.layers.13.mlp.gate_proj": 2149,
+ "model.layers.13.mlp.up_proj": 2138,
+ "model.layers.13.mlp.down_proj": 2142,
+ "model.layers.14.self_attn.q_proj": 1482,
+ "model.layers.14.self_attn.k_proj": 1480,
+ "model.layers.14.self_attn.v_proj": 1512,
+ "model.layers.14.self_attn.o_proj": 1499,
+ "model.layers.14.mlp.gate_proj": 2141,
+ "model.layers.14.mlp.up_proj": 2129,
+ "model.layers.14.mlp.down_proj": 2122,
+ "model.layers.15.self_attn.q_proj": 1486,
+ "model.layers.15.self_attn.k_proj": 1482,
+ "model.layers.15.self_attn.v_proj": 1507,
+ "model.layers.15.self_attn.o_proj": 1512,
+ "model.layers.15.mlp.gate_proj": 2131,
+ "model.layers.15.mlp.up_proj": 2117,
+ "model.layers.15.mlp.down_proj": 2103,
+ "model.layers.16.self_attn.q_proj": 1476,
+ "model.layers.16.self_attn.k_proj": 1471,
+ "model.layers.16.self_attn.v_proj": 1490,
+ "model.layers.16.self_attn.o_proj": 1504,
+ "model.layers.16.mlp.gate_proj": 2121,
+ "model.layers.16.mlp.up_proj": 2107,
+ "model.layers.16.mlp.down_proj": 2077,
+ "model.layers.17.self_attn.q_proj": 1468,
+ "model.layers.17.self_attn.k_proj": 1466,
+ "model.layers.17.self_attn.v_proj": 1482,
+ "model.layers.17.self_attn.o_proj": 1491,
+ "model.layers.17.mlp.gate_proj": 2108,
+ "model.layers.17.mlp.up_proj": 2098,
+ "model.layers.17.mlp.down_proj": 2066,
+ "model.layers.18.self_attn.q_proj": 1455,
+ "model.layers.18.self_attn.k_proj": 1453,
+ "model.layers.18.self_attn.v_proj": 1469,
+ "model.layers.18.self_attn.o_proj": 1498,
+ "model.layers.18.mlp.gate_proj": 2094,
+ "model.layers.18.mlp.up_proj": 2088,
+ "model.layers.18.mlp.down_proj": 2058,
+ "model.layers.19.self_attn.q_proj": 1461,
+ "model.layers.19.self_attn.k_proj": 1460,
+ "model.layers.19.self_attn.v_proj": 1468,
+ "model.layers.19.self_attn.o_proj": 1499,
+ "model.layers.19.mlp.gate_proj": 2083,
+ "model.layers.19.mlp.up_proj": 2079,
+ "model.layers.19.mlp.down_proj": 2044,
+ "model.layers.20.self_attn.q_proj": 1462,
+ "model.layers.20.self_attn.k_proj": 1462,
+ "model.layers.20.self_attn.v_proj": 1474,
+ "model.layers.20.self_attn.o_proj": 1480,
+ "model.layers.20.mlp.gate_proj": 2077,
+ "model.layers.20.mlp.up_proj": 2074,
+ "model.layers.20.mlp.down_proj": 2030,
+ "model.layers.21.self_attn.q_proj": 1464,
+ "model.layers.21.self_attn.k_proj": 1464,
+ "model.layers.21.self_attn.v_proj": 1463,
+ "model.layers.21.self_attn.o_proj": 1460,
+ "model.layers.21.mlp.gate_proj": 2072,
+ "model.layers.21.mlp.up_proj": 2072,
+ "model.layers.21.mlp.down_proj": 2010,
+ "model.layers.22.self_attn.q_proj": 1436,
+ "model.layers.22.self_attn.k_proj": 1437,
+ "model.layers.22.self_attn.v_proj": 1442,
+ "model.layers.22.self_attn.o_proj": 1491,
+ "model.layers.22.mlp.gate_proj": 2061,
+ "model.layers.22.mlp.up_proj": 2063,
+ "model.layers.22.mlp.down_proj": 2003,
+ "model.layers.23.self_attn.q_proj": 1442,
+ "model.layers.23.self_attn.k_proj": 1443,
+ "model.layers.23.self_attn.v_proj": 1442,
+ "model.layers.23.self_attn.o_proj": 1488,
+ "model.layers.23.mlp.gate_proj": 2051,
+ "model.layers.23.mlp.up_proj": 2056,
+ "model.layers.23.mlp.down_proj": 1994,
+ "model.layers.24.self_attn.q_proj": 1433,
+ "model.layers.24.self_attn.k_proj": 1435,
+ "model.layers.24.self_attn.v_proj": 1436,
+ "model.layers.24.self_attn.o_proj": 1495,
+ "model.layers.24.mlp.gate_proj": 2044,
+ "model.layers.24.mlp.up_proj": 2050,
+ "model.layers.24.mlp.down_proj": 1988,
+ "model.layers.25.self_attn.q_proj": 1428,
+ "model.layers.25.self_attn.k_proj": 1428,
+ "model.layers.25.self_attn.v_proj": 1425,
+ "model.layers.25.self_attn.o_proj": 1475,
+ "model.layers.25.mlp.gate_proj": 2035,
+ "model.layers.25.mlp.up_proj": 2042,
+ "model.layers.25.mlp.down_proj": 1978,
+ "model.layers.26.self_attn.q_proj": 1424,
+ "model.layers.26.self_attn.k_proj": 1424,
+ "model.layers.26.self_attn.v_proj": 1421,
+ "model.layers.26.self_attn.o_proj": 1474,
+ "model.layers.26.mlp.gate_proj": 2027,
+ "model.layers.26.mlp.up_proj": 2035,
+ "model.layers.26.mlp.down_proj": 1969,
+ "model.layers.27.self_attn.q_proj": 1408,
+ "model.layers.27.self_attn.k_proj": 1409,
+ "model.layers.27.self_attn.v_proj": 1410,
+ "model.layers.27.self_attn.o_proj": 1474,
+ "model.layers.27.mlp.gate_proj": 2019,
+ "model.layers.27.mlp.up_proj": 2027,
+ "model.layers.27.mlp.down_proj": 1963,
+ "model.layers.28.self_attn.q_proj": 1409,
+ "model.layers.28.self_attn.k_proj": 1410,
+ "model.layers.28.self_attn.v_proj": 1408,
+ "model.layers.28.self_attn.o_proj": 1473,
+ "model.layers.28.mlp.gate_proj": 2010,
+ "model.layers.28.mlp.up_proj": 2017,
+ "model.layers.28.mlp.down_proj": 1955,
+ "model.layers.29.self_attn.q_proj": 1397,
+ "model.layers.29.self_attn.k_proj": 1398,
+ "model.layers.29.self_attn.v_proj": 1399,
+ "model.layers.29.self_attn.o_proj": 1464,
+ "model.layers.29.mlp.gate_proj": 2002,
+ "model.layers.29.mlp.up_proj": 2009,
+ "model.layers.29.mlp.down_proj": 1948,
+ "model.layers.30.self_attn.q_proj": 1389,
+ "model.layers.30.self_attn.k_proj": 1391,
+ "model.layers.30.self_attn.v_proj": 1385,
+ "model.layers.30.self_attn.o_proj": 1436,
+ "model.layers.30.mlp.gate_proj": 1993,
+ "model.layers.30.mlp.up_proj": 2001,
+ "model.layers.30.mlp.down_proj": 1941,
+ "model.layers.31.self_attn.q_proj": 1396,
+ "model.layers.31.self_attn.k_proj": 1396,
+ "model.layers.31.self_attn.v_proj": 1395,
+ "model.layers.31.self_attn.o_proj": 1446,
+ "model.layers.31.mlp.gate_proj": 1985,
+ "model.layers.31.mlp.up_proj": 1993,
+ "model.layers.31.mlp.down_proj": 1936,
+ "model.layers.32.self_attn.q_proj": 1393,
+ "model.layers.32.self_attn.k_proj": 1395,
+ "model.layers.32.self_attn.v_proj": 1383,
+ "model.layers.32.self_attn.o_proj": 1429,
+ "model.layers.32.mlp.gate_proj": 1979,
+ "model.layers.32.mlp.up_proj": 1985,
+ "model.layers.32.mlp.down_proj": 1929,
+ "model.layers.33.self_attn.q_proj": 1399,
+ "model.layers.33.self_attn.k_proj": 1400,
+ "model.layers.33.self_attn.v_proj": 1390,
+ "model.layers.33.self_attn.o_proj": 1430,
+ "model.layers.33.mlp.gate_proj": 1973,
+ "model.layers.33.mlp.up_proj": 1978,
+ "model.layers.33.mlp.down_proj": 1921,
+ "model.layers.34.self_attn.q_proj": 1401,
+ "model.layers.34.self_attn.k_proj": 1401,
+ "model.layers.34.self_attn.v_proj": 1383,
+ "model.layers.34.self_attn.o_proj": 1424,
+ "model.layers.34.mlp.gate_proj": 1968,
+ "model.layers.34.mlp.up_proj": 1972,
+ "model.layers.34.mlp.down_proj": 1910,
+ "model.layers.35.self_attn.q_proj": 1407,
+ "model.layers.35.self_attn.k_proj": 1407,
+ "model.layers.35.self_attn.v_proj": 1388,
+ "model.layers.35.self_attn.o_proj": 1412,
+ "model.layers.35.mlp.gate_proj": 1964,
+ "model.layers.35.mlp.up_proj": 1966,
+ "model.layers.35.mlp.down_proj": 1897,
+ "model.layers.36.self_attn.q_proj": 1417,
+ "model.layers.36.self_attn.k_proj": 1416,
+ "model.layers.36.self_attn.v_proj": 1392,
+ "model.layers.36.self_attn.o_proj": 1380,
+ "model.layers.36.mlp.gate_proj": 1962,
+ "model.layers.36.mlp.up_proj": 1963,
+ "model.layers.36.mlp.down_proj": 1882,
+ "model.layers.37.self_attn.q_proj": 1438,
+ "model.layers.37.self_attn.k_proj": 1436,
+ "model.layers.37.self_attn.v_proj": 1398,
+ "model.layers.37.self_attn.o_proj": 1391,
+ "model.layers.37.mlp.gate_proj": 1964,
+ "model.layers.37.mlp.up_proj": 1966,
+ "model.layers.37.mlp.down_proj": 1877,
+ "model.layers.38.self_attn.q_proj": 1446,
+ "model.layers.38.self_attn.k_proj": 1442,
+ "model.layers.38.self_attn.v_proj": 1392,
+ "model.layers.38.self_attn.o_proj": 1376,
+ "model.layers.38.mlp.gate_proj": 1971,
+ "model.layers.38.mlp.up_proj": 1978,
+ "model.layers.38.mlp.down_proj": 1877,
+ "model.layers.39.self_attn.q_proj": 1470,
+ "model.layers.39.self_attn.k_proj": 1466,
+ "model.layers.39.self_attn.v_proj": 1423,
+ "model.layers.39.self_attn.o_proj": 1389,
+ "model.layers.39.mlp.gate_proj": 1998,
+ "model.layers.39.mlp.up_proj": 2011,
+ "model.layers.39.mlp.down_proj": 1865
+}

svd_llm2/unheal/llama2-13b/state_dict_fp16.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f0dbe9c89696d52b78344dc7564c8c5bc6bb57ecf7cd95edaa26e6738db87e23
+size 15480786677

svd_llm2/unheal/llama2-7b/README.md

@@ -0,0 +1,13 @@
+---
+license: llama2
+base_model: NousResearch/Llama-2-7b-chat-hf
+tags: [svd, low-rank, model-compression, llama2]
+---
+
+# SVD-LLM V2 (NAACL 2025, official code unreleased; our implementation of Alg. 1+2) @ 4.02B, UNHEALED (LLaMA-2-7B-chat)
+
+- **UNHEALED checkpoint**: raw post-training compression weights, NO distillation. The KD-healed
+  counterpart is at `../heal/llama2-7b/`. Same `ranks.json`; weights differ.
+- Factored params: **4,023,119,360** (budget 4.0231B, matched to the TR-3 llama2-7b row).
+
+Load with `lowrank_linear.py` in this dir (identical recipe to the sibling dirs).

svd_llm2/unheal/llama2-7b/config.json

@@ -0,0 +1,28 @@
+{
+  "_name_or_path": "NousResearch/Llama-2-7b-chat-hf",
+  "architectures": [
+    "LlamaForCausalLM"
+  ],
+  "attention_bias": false,
+  "bos_token_id": 1,
+  "eos_token_id": 2,
+  "hidden_act": "silu",
+  "hidden_size": 4096,
+  "initializer_range": 0.02,
+  "intermediate_size": 11008,
+  "max_position_embeddings": 4096,
+  "model_type": "llama",
+  "num_attention_heads": 32,
+  "num_hidden_layers": 32,
+  "num_key_value_heads": 32,
+  "pad_token_id": 0,
+  "pretraining_tp": 1,
+  "rms_norm_eps": 1e-05,
+  "rope_scaling": null,
+  "rope_theta": 10000.0,
+  "tie_word_embeddings": false,
+  "torch_dtype": "float16",
+  "transformers_version": "4.35.2",
+  "use_cache": true,
+  "vocab_size": 32000
+}

svd_llm2/unheal/llama2-7b/lowrank_linear.py

@@ -0,0 +1,83 @@
+"""Minimal standalone loader for the SVD-baseline compressed LLaMA-2-7B checkpoints.
+
+No dependency beyond torch + transformers. The state_dict stores every compressed
+projection as two factors of a LowRankLinear (A: out x k, B: k x in); ranks.json
+holds the per-layer integer rank of each projection. Untouched parameters
+(embeddings, norms) keep their stock transformers names.
+
+Usage:
+    import json, torch
+    from lowrank_linear import build
+    model = build(
+        base_id="NousResearch/Llama-2-7b-chat-hf",
+        ranks=json.load(open("ranks.json")),
+        state_dict_path="state_dict_fp16.pt",
+        device="cuda", dtype=torch.float16,
+    )
+    tokenizer = AutoTokenizer.from_pretrained("NousResearch/Llama-2-7b-chat-hf")
+    out = model.generate(tokenizer("2+2=", return_tensors="pt").to("cuda").input_ids,
+                         max_new_tokens=16, do_sample=False)
+    print(tokenizer.decode(out[0]))
+"""
+
+from __future__ import annotations
+
+import torch
+import torch.nn as nn
+import torch.nn.functional as F
+
+
+class LowRankLinear(nn.Module):
+    """y = A (B x).  A: out x r, B: r x in.  (LLaMA projections carry no bias.)"""
+
+    def __init__(self, A: torch.Tensor, B: torch.Tensor):
+        super().__init__()
+        self.A = nn.Parameter(A)
+        self.B = nn.Parameter(B)
+
+    def forward(self, x):
+        return F.linear(F.linear(x, self.B), self.A)
+
+    @torch.no_grad()
+    def materialize_(self) -> nn.Linear:
+        """Replace this module with the dense W = A @ B (inference only; never call
+        while training -- it cuts the gradient path)."""
+        W = (self.A @ self.B).to(self.A.dtype)
+        dense = nn.Linear(self.B.shape[1], self.A.shape[0], bias=False,
+                          device=W.device, dtype=W.dtype)
+        dense.weight.data = W
+        return dense
+
+
+def build(base_id: str, ranks: dict, state_dict_path: str,
+          device: str = "cuda", dtype=torch.float16):
+    from transformers import AutoModelForCausalLM
+
+    model = AutoModelForCausalLM.from_pretrained(base_id, torch_dtype=dtype)
+    search = {}
+    for name, mod in model.named_modules():
+        if isinstance(mod, nn.Linear):
+            search[name] = mod
+    for path, k in ranks.items():
+        out_f, in_f = search[path].weight.shape
+        model_mod = model
+        for p in path.split(".")[:-1]:
+            model_mod = getattr(model_mod, p)
+        setattr(model_mod, path.split(".")[-1],
+                LowRankLinear(torch.zeros(out_f, int(k), dtype=dtype),
+                              torch.zeros(int(k), in_f, dtype=dtype)))
+    sd = torch.load(state_dict_path, map_location="cpu", weights_only=True)
+    model.load_state_dict(sd, strict=True)
+    del sd
+    return model.to(device)
+
+
+def materialize_all(model):
+    """Fuse every LowRankLinear back into a dense nn.Linear in place."""
+    for name, mod in list(model.named_modules()):
+        if isinstance(mod, LowRankLinear):
+            parent = model
+            for p in name.split(".")[:-1]:
+                parent = getattr(parent, p)
+            setattr(parent, name.split(".")[-1], mod.materialize_())
+    return model

svd_llm2/unheal/llama2-7b/ranks.json

@@ -0,0 +1,226 @@
+{
+ "model.layers.0.self_attn.q_proj": 2048,
+ "model.layers.0.self_attn.k_proj": 2048,
+ "model.layers.0.self_attn.v_proj": 2048,
+ "model.layers.0.self_attn.o_proj": 2048,
+ "model.layers.0.mlp.gate_proj": 2341,
+ "model.layers.0.mlp.up_proj": 2332,
+ "model.layers.0.mlp.down_proj": 2887,
+ "model.layers.1.self_attn.q_proj": 1911,
+ "model.layers.1.self_attn.k_proj": 1960,
+ "model.layers.1.self_attn.v_proj": 1826,
+ "model.layers.1.self_attn.o_proj": 2048,
+ "model.layers.1.mlp.gate_proj": 2115,
+ "model.layers.1.mlp.up_proj": 2094,
+ "model.layers.1.mlp.down_proj": 2280,
+ "model.layers.2.self_attn.q_proj": 1404,
+ "model.layers.2.self_attn.k_proj": 1405,
+ "model.layers.2.self_attn.v_proj": 1400,
+ "model.layers.2.self_attn.o_proj": 1662,
+ "model.layers.2.mlp.gate_proj": 1931,
+ "model.layers.2.mlp.up_proj": 1928,
+ "model.layers.2.mlp.down_proj": 2094,
+ "model.layers.3.self_attn.q_proj": 1252,
+ "model.layers.3.self_attn.k_proj": 1247,
+ "model.layers.3.self_attn.v_proj": 1285,
+ "model.layers.3.self_attn.o_proj": 1598,
+ "model.layers.3.mlp.gate_proj": 1821,
+ "model.layers.3.mlp.up_proj": 1826,
+ "model.layers.3.mlp.down_proj": 2002,
+ "model.layers.4.self_attn.q_proj": 1255,
+ "model.layers.4.self_attn.k_proj": 1252,
+ "model.layers.4.self_attn.v_proj": 1282,
+ "model.layers.4.self_attn.o_proj": 1490,
+ "model.layers.4.mlp.gate_proj": 1780,
+ "model.layers.4.mlp.up_proj": 1789,
+ "model.layers.4.mlp.down_proj": 1911,
+ "model.layers.5.self_attn.q_proj": 1239,
+ "model.layers.5.self_attn.k_proj": 1236,
+ "model.layers.5.self_attn.v_proj": 1266,
+ "model.layers.5.self_attn.o_proj": 1456,
+ "model.layers.5.mlp.gate_proj": 1757,
+ "model.layers.5.mlp.up_proj": 1765,
+ "model.layers.5.mlp.down_proj": 1855,
+ "model.layers.6.self_attn.q_proj": 1218,
+ "model.layers.6.self_attn.k_proj": 1214,
+ "model.layers.6.self_attn.v_proj": 1244,
+ "model.layers.6.self_attn.o_proj": 1393,
+ "model.layers.6.mlp.gate_proj": 1733,
+ "model.layers.6.mlp.up_proj": 1743,
+ "model.layers.6.mlp.down_proj": 1807,
+ "model.layers.7.self_attn.q_proj": 1209,
+ "model.layers.7.self_attn.k_proj": 1208,
+ "model.layers.7.self_attn.v_proj": 1237,
+ "model.layers.7.self_attn.o_proj": 1356,
+ "model.layers.7.mlp.gate_proj": 1718,
+ "model.layers.7.mlp.up_proj": 1726,
+ "model.layers.7.mlp.down_proj": 1770,
+ "model.layers.8.self_attn.q_proj": 1205,
+ "model.layers.8.self_attn.k_proj": 1201,
+ "model.layers.8.self_attn.v_proj": 1238,
+ "model.layers.8.self_attn.o_proj": 1314,
+ "model.layers.8.mlp.gate_proj": 1715,
+ "model.layers.8.mlp.up_proj": 1718,
+ "model.layers.8.mlp.down_proj": 1747,
+ "model.layers.9.self_attn.q_proj": 1193,
+ "model.layers.9.self_attn.k_proj": 1190,
+ "model.layers.9.self_attn.v_proj": 1229,
+ "model.layers.9.self_attn.o_proj": 1276,
+ "model.layers.9.mlp.gate_proj": 1711,
+ "model.layers.9.mlp.up_proj": 1711,
+ "model.layers.9.mlp.down_proj": 1725,
+ "model.layers.10.self_attn.q_proj": 1201,
+ "model.layers.10.self_attn.k_proj": 1194,
+ "model.layers.10.self_attn.v_proj": 1230,
+ "model.layers.10.self_attn.o_proj": 1252,
+ "model.layers.10.mlp.gate_proj": 1708,
+ "model.layers.10.mlp.up_proj": 1706,
+ "model.layers.10.mlp.down_proj": 1713,
+ "model.layers.11.self_attn.q_proj": 1199,
+ "model.layers.11.self_attn.k_proj": 1196,
+ "model.layers.11.self_attn.v_proj": 1213,
+ "model.layers.11.self_attn.o_proj": 1258,
+ "model.layers.11.mlp.gate_proj": 1699,
+ "model.layers.11.mlp.up_proj": 1694,
+ "model.layers.11.mlp.down_proj": 1699,
+ "model.layers.12.self_attn.q_proj": 1180,
+ "model.layers.12.self_attn.k_proj": 1179,
+ "model.layers.12.self_attn.v_proj": 1206,
+ "model.layers.12.self_attn.o_proj": 1234,
+ "model.layers.12.mlp.gate_proj": 1692,
+ "model.layers.12.mlp.up_proj": 1686,
+ "model.layers.12.mlp.down_proj": 1683,
+ "model.layers.13.self_attn.q_proj": 1176,
+ "model.layers.13.self_attn.k_proj": 1170,
+ "model.layers.13.self_attn.v_proj": 1190,
+ "model.layers.13.self_attn.o_proj": 1231,
+ "model.layers.13.mlp.gate_proj": 1688,
+ "model.layers.13.mlp.up_proj": 1678,
+ "model.layers.13.mlp.down_proj": 1661,
+ "model.layers.14.self_attn.q_proj": 1181,
+ "model.layers.14.self_attn.k_proj": 1179,
+ "model.layers.14.self_attn.v_proj": 1192,
+ "model.layers.14.self_attn.o_proj": 1213,
+ "model.layers.14.mlp.gate_proj": 1679,
+ "model.layers.14.mlp.up_proj": 1669,
+ "model.layers.14.mlp.down_proj": 1644,
+ "model.layers.15.self_attn.q_proj": 1177,
+ "model.layers.15.self_attn.k_proj": 1174,
+ "model.layers.15.self_attn.v_proj": 1187,
+ "model.layers.15.self_attn.o_proj": 1198,
+ "model.layers.15.mlp.gate_proj": 1670,
+ "model.layers.15.mlp.up_proj": 1661,
+ "model.layers.15.mlp.down_proj": 1621,
+ "model.layers.16.self_attn.q_proj": 1176,
+ "model.layers.16.self_attn.k_proj": 1174,
+ "model.layers.16.self_attn.v_proj": 1173,
+ "model.layers.16.self_attn.o_proj": 1175,
+ "model.layers.16.mlp.gate_proj": 1662,
+ "model.layers.16.mlp.up_proj": 1655,
+ "model.layers.16.mlp.down_proj": 1595,
+ "model.layers.17.self_attn.q_proj": 1163,
+ "model.layers.17.self_attn.k_proj": 1163,
+ "model.layers.17.self_attn.v_proj": 1167,
+ "model.layers.17.self_attn.o_proj": 1193,
+ "model.layers.17.mlp.gate_proj": 1647,
+ "model.layers.17.mlp.up_proj": 1643,
+ "model.layers.17.mlp.down_proj": 1584,
+ "model.layers.18.self_attn.q_proj": 1150,
+ "model.layers.18.self_attn.k_proj": 1149,
+ "model.layers.18.self_attn.v_proj": 1151,
+ "model.layers.18.self_attn.o_proj": 1190,
+ "model.layers.18.mlp.gate_proj": 1632,
+ "model.layers.18.mlp.up_proj": 1631,
+ "model.layers.18.mlp.down_proj": 1568,
+ "model.layers.19.self_attn.q_proj": 1156,
+ "model.layers.19.self_attn.k_proj": 1157,
+ "model.layers.19.self_attn.v_proj": 1152,
+ "model.layers.19.self_attn.o_proj": 1184,
+ "model.layers.19.mlp.gate_proj": 1623,
+ "model.layers.19.mlp.up_proj": 1623,
+ "model.layers.19.mlp.down_proj": 1557,
+ "model.layers.20.self_attn.q_proj": 1155,
+ "model.layers.20.self_attn.k_proj": 1155,
+ "model.layers.20.self_attn.v_proj": 1146,
+ "model.layers.20.self_attn.o_proj": 1172,
+ "model.layers.20.mlp.gate_proj": 1617,
+ "model.layers.20.mlp.up_proj": 1619,
+ "model.layers.20.mlp.down_proj": 1540,
+ "model.layers.21.self_attn.q_proj": 1147,
+ "model.layers.21.self_attn.k_proj": 1147,
+ "model.layers.21.self_attn.v_proj": 1136,
+ "model.layers.21.self_attn.o_proj": 1174,
+ "model.layers.21.mlp.gate_proj": 1607,
+ "model.layers.21.mlp.up_proj": 1611,
+ "model.layers.21.mlp.down_proj": 1531,
+ "model.layers.22.self_attn.q_proj": 1138,
+ "model.layers.22.self_attn.k_proj": 1138,
+ "model.layers.22.self_attn.v_proj": 1129,
+ "model.layers.22.self_attn.o_proj": 1149,
+ "model.layers.22.mlp.gate_proj": 1599,
+ "model.layers.22.mlp.up_proj": 1604,
+ "model.layers.22.mlp.down_proj": 1517,
+ "model.layers.23.self_attn.q_proj": 1122,
+ "model.layers.23.self_attn.k_proj": 1123,
+ "model.layers.23.self_attn.v_proj": 1115,
+ "model.layers.23.self_attn.o_proj": 1139,
+ "model.layers.23.mlp.gate_proj": 1589,
+ "model.layers.23.mlp.up_proj": 1593,
+ "model.layers.23.mlp.down_proj": 1508,
+ "model.layers.24.self_attn.q_proj": 1138,
+ "model.layers.24.self_attn.k_proj": 1139,
+ "model.layers.24.self_attn.v_proj": 1118,
+ "model.layers.24.self_attn.o_proj": 1135,
+ "model.layers.24.mlp.gate_proj": 1580,
+ "model.layers.24.mlp.up_proj": 1585,
+ "model.layers.24.mlp.down_proj": 1501,
+ "model.layers.25.self_attn.q_proj": 1116,
+ "model.layers.25.self_attn.k_proj": 1116,
+ "model.layers.25.self_attn.v_proj": 1100,
+ "model.layers.25.self_attn.o_proj": 1141,
+ "model.layers.25.mlp.gate_proj": 1573,
+ "model.layers.25.mlp.up_proj": 1577,
+ "model.layers.25.mlp.down_proj": 1496,
+ "model.layers.26.self_attn.q_proj": 1127,
+ "model.layers.26.self_attn.k_proj": 1126,
+ "model.layers.26.self_attn.v_proj": 1103,
+ "model.layers.26.self_attn.o_proj": 1121,
+ "model.layers.26.mlp.gate_proj": 1565,
+ "model.layers.26.mlp.up_proj": 1569,
+ "model.layers.26.mlp.down_proj": 1487,
+ "model.layers.27.self_attn.q_proj": 1108,
+ "model.layers.27.self_attn.k_proj": 1107,
+ "model.layers.27.self_attn.v_proj": 1095,
+ "model.layers.27.self_attn.o_proj": 1109,
+ "model.layers.27.mlp.gate_proj": 1557,
+ "model.layers.27.mlp.up_proj": 1561,
+ "model.layers.27.mlp.down_proj": 1475,
+ "model.layers.28.self_attn.q_proj": 1113,
+ "model.layers.28.self_attn.k_proj": 1111,
+ "model.layers.28.self_attn.v_proj": 1090,
+ "model.layers.28.self_attn.o_proj": 1105,
+ "model.layers.28.mlp.gate_proj": 1555,
+ "model.layers.28.mlp.up_proj": 1556,
+ "model.layers.28.mlp.down_proj": 1462,
+ "model.layers.29.self_attn.q_proj": 1128,
+ "model.layers.29.self_attn.k_proj": 1127,
+ "model.layers.29.self_attn.v_proj": 1094,
+ "model.layers.29.self_attn.o_proj": 1094,
+ "model.layers.29.mlp.gate_proj": 1552,
+ "model.layers.29.mlp.up_proj": 1553,
+ "model.layers.29.mlp.down_proj": 1450,
+ "model.layers.30.self_attn.q_proj": 1116,
+ "model.layers.30.self_attn.k_proj": 1115,
+ "model.layers.30.self_attn.v_proj": 1087,
+ "model.layers.30.self_attn.o_proj": 1093,
+ "model.layers.30.mlp.gate_proj": 1552,
+ "model.layers.30.mlp.up_proj": 1556,
+ "model.layers.30.mlp.down_proj": 1439,
+ "model.layers.31.self_attn.q_proj": 1156,
+ "model.layers.31.self_attn.k_proj": 1152,
+ "model.layers.31.self_attn.v_proj": 1126,
+ "model.layers.31.self_attn.o_proj": 1102,
+ "model.layers.31.mlp.gate_proj": 1570,
+ "model.layers.31.mlp.up_proj": 1576,
+ "model.layers.31.mlp.down_proj": 1429
+}

svd_llm2/unheal/llama2-7b/state_dict_fp16.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:33e0dba497c0bfe3b4532b4c6c0e07274a2ca85219b475fb2c26978d5ae6dfbd
+size 8046408465

tr3/unheal/llama2-13b/README.md

@@ -0,0 +1,12 @@
+---
+license: llama2
+base_model: NousResearch/Llama-2-13b-chat-hf
+tags: [tensor-decomposition, tensor-ring, model-compression, llama2]
+---
+
+# TR-3 LLaMA-2-13B-chat, UNHEALED (ring cores, pre-healing)
+
+- **UNHEALED checkpoint**: 3-site Tensor Ring cores + 5% sparse residual as produced by the
+  compression stage, before any distillation. The KD-healed counterpart is at `../../heal/llama2-13b/`.
+- Params: **7,740,282,880** (59.47% kept of 13.02B). Same `ranks.json` as the healed dir.
+- Loading: identical to the heal sibling (`tensor_formats.py`, ring-core TRLinear modules).

tr3/unheal/llama2-13b/ranks.json

@@ -0,0 +1,3362 @@
+{
+ "model.layers.0.self_attn.q_proj": {
+  "o1": 16,
+  "o2": 16,
+  "o3": 20,
+  "i1": 16,
+  "i2": 16,
+  "i3": 20,
+  "r0": 124,
+  "r1": 124,
+  "r2": 124,
+  "nnz": 1310720
+ },
+ "model.layers.0.self_attn.k_proj": {
+  "o1": 16,
+  "o2": 16,
+  "o3": 20,
+  "i1": 16,
+  "i2": 16,
+  "i3": 20,
+  "r0": 124,
+  "r1": 124,
+  "r2": 124,
+  "nnz": 1310720
+ },
+ "model.layers.0.self_attn.v_proj": {
+  "o1": 16,
+  "o2": 16,
+  "o3": 20,
+  "i1": 16,
+  "i2": 16,
+  "i3": 20,
+  "r0": 124,
+  "r1": 124,
+  "r2": 124,
+  "nnz": 1310720
+ },
+ "model.layers.0.self_attn.o_proj": {
+  "o1": 16,
+  "o2": 16,
+  "o3": 20,
+  "i1": 16,
+  "i2": 16,
+  "i3": 20,
+  "r0": 124,
+  "r1": 124,
+  "r2": 124,
+  "nnz": 1310720
+ },
+ "model.layers.0.mlp.gate_proj": {
+  "o1": 24,
+  "o2": 24,
+  "o3": 24,
+  "i1": 16,
+  "i2": 16,
+  "i3": 20,
+  "r0": 174,
+  "r1": 174,
+  "r2": 174,
+  "nnz": 3538944
+ },
+ "model.layers.0.mlp.up_proj": {
+  "o1": 24,
+  "o2": 24,
+  "o3": 24,
+  "i1": 16,
+  "i2": 16,
+  "i3": 20,
+  "r0": 174,
+  "r1": 174,
+  "r2": 174,
+  "nnz": 3538944
+ },
+ "model.layers.0.mlp.down_proj": {
+  "o1": 16,
+  "o2": 16,
+  "o3": 20,
+  "i1": 24,
+  "i2": 24,
+  "i3": 24,
+  "r0": 174,
+  "r1": 174,
+  "r2": 174,
+  "nnz": 3538944
+ },
+ "model.layers.1.self_attn.q_proj": {
+  "o1": 16,
+  "o2": 16,
+  "o3": 20,
+  "i1": 16,
+  "i2": 16,
+  "i3": 20,
+  "r0": 124,
+  "r1": 124,
+  "r2": 124,
+  "nnz": 1310720
+ },
+ "model.layers.1.self_attn.k_proj": {
+  "o1": 16,
+  "o2": 16,
+  "o3": 20,
+  "i1": 16,
+  "i2": 16,
+  "i3": 20,
+  "r0": 124,
+  "r1": 124,
+  "r2": 124,
+  "nnz": 1310720
+ },
+ "model.layers.1.self_attn.v_proj": {
+  "o1": 16,
+  "o2": 16,
+  "o3": 20,
+  "i1": 16,
+  "i2": 16,
+  "i3": 20,
+  "r0": 124,
+  "r1": 124,
+  "r2": 124,
+  "nnz": 1310720
+ },
+ "model.layers.1.self_attn.o_proj": {
+  "o1": 16,
+  "o2": 16,
+  "o3": 20,
+  "i1": 16,
+  "i2": 16,
+  "i3": 20,
+  "r0": 124,
+  "r1": 124,
+  "r2": 124,
+  "nnz": 1310720
+ },
+ "model.layers.1.mlp.gate_proj": {
+  "o1": 24,
+  "o2": 24,
+  "o3": 24,
+  "i1": 16,
+  "i2": 16,
+  "i3": 20,
+  "r0": 174,
+  "r1": 174,
+  "r2": 174,
+  "nnz": 3538944
+ },
+ "model.layers.1.mlp.up_proj": {
+  "o1": 24,
+  "o2": 24,
+  "o3": 24,
+  "i1": 16,
+  "i2": 16,
+  "i3": 20,
+  "r0": 174,
+  "r1": 174,
+  "r2": 174,
+  "nnz": 3538944
+ },
+ "model.layers.1.mlp.down_proj": {
+  "o1": 16,
+  "o2": 16,
+  "o3": 20,
+  "i1": 24,
+  "i2": 24,
+  "i3": 24,
+  "r0": 174,
+  "r1": 174,
+  "r2": 174,
+  "nnz": 3538944
+ },
+ "model.layers.2.self_attn.q_proj": {
+  "o1": 16,
+  "o2": 16,
+  "o3": 20,
+  "i1": 16,
+  "i2": 16,
+  "i3": 20,
+  "r0": 124,
+  "r1": 124,
+  "r2": 124,
+  "nnz": 1310720
+ },
+ "model.layers.2.self_attn.k_proj": {
+  "o1": 16,
+  "o2": 16,
+  "o3": 20,
+  "i1": 16,
+  "i2": 16,
+  "i3": 20,
+  "r0": 124,
+  "r1": 124,
+  "r2": 124,
+  "nnz": 1310720
+ },
+ "model.layers.2.self_attn.v_proj": {
+  "o1": 16,
+  "o2": 16,
+  "o3": 20,
+  "i1": 16,
+  "i2": 16,
+  "i3": 20,
+  "r0": 124,
+  "r1": 124,
+  "r2": 124,
+  "nnz": 1310720
+ },
+ "model.layers.2.self_attn.o_proj": {
+  "o1": 16,
+  "o2": 16,
+  "o3": 20,
+  "i1": 16,
+  "i2": 16,
+  "i3": 20,
+  "r0": 124,
+  "r1": 124,
+  "r2": 124,
+  "nnz": 1310720
+ },
+ "model.layers.2.mlp.gate_proj": {
+  "o1": 24,
+  "o2": 24,
+  "o3": 24,
+  "i1": 16,
+  "i2": 16,
+  "i3": 20,
+  "r0": 174,
+  "r1": 174,
+  "r2": 174,
+  "nnz": 3538944
+ },
+ "model.layers.2.mlp.up_proj": {
+  "o1": 24,
+  "o2": 24,
+  "o3": 24,
+  "i1": 16,
+  "i2": 16,
+  "i3": 20,
+  "r0": 174,
+  "r1": 174,
+  "r2": 174,
+  "nnz": 3538944
+ },
+ "model.layers.2.mlp.down_proj": {
+  "o1": 16,
+  "o2": 16,
+  "o3": 20,
+  "i1": 24,
+  "i2": 24,
+  "i3": 24,
+  "r0": 174,
+  "r1": 174,
+  "r2": 174,
+  "nnz": 3538944
+ },
+ "model.layers.3.self_attn.q_proj": {
+  "o1": 16,
+  "o2": 16,
+  "o3": 20,
+  "i1": 16,
+  "i2": 16,
+  "i3": 20,
+  "r0": 124,
+  "r1": 124,
+  "r2": 124,
+  "nnz": 1310720
+ },
+ "model.layers.3.self_attn.k_proj": {
+  "o1": 16,
+  "o2": 16,
+  "o3": 20,
+  "i1": 16,
+  "i2": 16,
+  "i3": 20,
+  "r0": 124,
+  "r1": 124,
+  "r2": 124,
+  "nnz": 1310720
+ },
+ "model.layers.3.self_attn.v_proj": {
+  "o1": 16,
+  "o2": 16,
+  "o3": 20,
+  "i1": 16,
+  "i2": 16,
+  "i3": 20,
+  "r0": 124,
+  "r1": 124,
+  "r2": 124,
+  "nnz": 1310720
+ },
+ "model.layers.3.self_attn.o_proj": {
+  "o1": 16,
+  "o2": 16,
+  "o3": 20,
+  "i1": 16,
+  "i2": 16,
+  "i3": 20,
+  "r0": 124,
+  "r1": 124,
+  "r2": 124,
+  "nnz": 1310720
+ },
+ "model.layers.3.mlp.gate_proj": {
+  "o1": 24,
+  "o2": 24,
+  "o3": 24,
+  "i1": 16,
+  "i2": 16,
+  "i3": 20,
+  "r0": 174,
+  "r1": 174,
+  "r2": 174,
+  "nnz": 3538944
+ },
+ "model.layers.3.mlp.up_proj": {
+  "o1": 24,
+  "o2": 24,
+  "o3": 24,
+  "i1": 16,
+  "i2": 16,
+  "i3": 20,
+  "r0": 174,
+  "r1": 174,
+  "r2": 174,
+  "nnz": 3538944
+ },
+ "model.layers.3.mlp.down_proj": {
+  "o1": 16,
+  "o2": 16,
+  "o3": 20,
+  "i1": 24,
+  "i2": 24,
+  "i3": 24,
+  "r0": 174,
+  "r1": 174,
+  "r2": 174,
+  "nnz": 3538944
+ },
+ "model.layers.4.self_attn.q_proj": {
+  "o1": 16,
+  "o2": 16,
+  "o3": 20,
+  "i1": 16,
+  "i2": 16,
+  "i3": 20,
+  "r0": 124,
+  "r1": 124,
+  "r2": 124,
+  "nnz": 1310720
+ },
+ "model.layers.4.self_attn.k_proj": {
+  "o1": 16,
+  "o2": 16,
+  "o3": 20,
+  "i1": 16,
+  "i2": 16,
+  "i3": 20,
+  "r0": 124,
+  "r1": 124,
+  "r2": 124,
+  "nnz": 1310720
+ },
+ "model.layers.4.self_attn.v_proj": {
+  "o1": 16,
+  "o2": 16,
+  "o3": 20,
+  "i1": 16,
+  "i2": 16,
+  "i3": 20,
+  "r0": 124,
+  "r1": 124,
+  "r2": 124,
+  "nnz": 1310720
+ },
+ "model.layers.4.self_attn.o_proj": {
+  "o1": 16,
+  "o2": 16,
+  "o3": 20,
+  "i1": 16,
+  "i2": 16,
+  "i3": 20,
+  "r0": 124,
+  "r1": 124,
+  "r2": 124,
+  "nnz": 1310720
+ },
+ "model.layers.4.mlp.gate_proj": {
+  "o1": 24,
+  "o2": 24,
+  "o3": 24,
+  "i1": 16,
+  "i2": 16,
+  "i3": 20,
+  "r0": 174,
+  "r1": 174,
+  "r2": 174,
+  "nnz": 3538944
+ },
+ "model.layers.4.mlp.up_proj": {
+  "o1": 24,
+  "o2": 24,
+  "o3": 24,
+  "i1": 16,
+  "i2": 16,
+  "i3": 20,
+  "r0": 174,
+  "r1": 174,
+  "r2": 174,
+  "nnz": 3538944
+ },
+ "model.layers.4.mlp.down_proj": {
+  "o1": 16,
+  "o2": 16,
+  "o3": 20,
+  "i1": 24,
+  "i2": 24,
+  "i3": 24,
+  "r0": 174,
+  "r1": 174,
+  "r2": 174,
+  "nnz": 3538944
+ },
+ "model.layers.5.self_attn.q_proj": {
+  "o1": 16,
+  "o2": 16,
+  "o3": 20,
+  "i1": 16,
+  "i2": 16,
+  "i3": 20,
+  "r0": 124,
+  "r1": 124,
+  "r2": 124,
+  "nnz": 1310720
+ },
+ "model.layers.5.self_attn.k_proj": {
+  "o1": 16,
+  "o2": 16,
+  "o3": 20,
+  "i1": 16,
+  "i2": 16,
+  "i3": 20,
+  "r0": 124,
+  "r1": 124,
+  "r2": 124,
+  "nnz": 1310720
+ },
+ "model.layers.5.self_attn.v_proj": {
+  "o1": 16,
+  "o2": 16,
+  "o3": 20,
+  "i1": 16,
+  "i2": 16,
+  "i3": 20,
+  "r0": 124,
+  "r1": 124,
+  "r2": 124,
+  "nnz": 1310720
+ },
+ "model.layers.5.self_attn.o_proj": {
+  "o1": 16,
+  "o2": 16,
+  "o3": 20,
+  "i1": 16,
+  "i2": 16,
+  "i3": 20,
+  "r0": 124,
+  "r1": 124,
+  "r2": 124,
+  "nnz": 1310720
+ },
+ "model.layers.5.mlp.gate_proj": {
+  "o1": 24,
+  "o2": 24,
+  "o3": 24,
+  "i1": 16,
+  "i2": 16,
+  "i3": 20,
+  "r0": 174,
+  "r1": 174,
+  "r2": 174,
+  "nnz": 3538944
+ },
+ "model.layers.5.mlp.up_proj": {
+  "o1": 24,
+  "o2": 24,
+  "o3": 24,
+  "i1": 16,
+  "i2": 16,
+  "i3": 20,
+  "r0": 174,
+  "r1": 174,
+  "r2": 174,
+  "nnz": 3538944
+ },
+ "model.layers.5.mlp.down_proj": {
+  "o1": 16,
+  "o2": 16,
+  "o3": 20,
+  "i1": 24,
+  "i2": 24,
+  "i3": 24,
+  "r0": 174,
+  "r1": 174,
+  "r2": 174,
+  "nnz": 3538944
+ },
+ "model.layers.6.self_attn.q_proj": {
+  "o1": 16,
+  "o2": 16,
+  "o3": 20,
+  "i1": 16,
+  "i2": 16,
+  "i3": 20,
+  "r0": 124,
+  "r1": 124,
+  "r2": 124,
+  "nnz": 1310720
+ },
+ "model.layers.6.self_attn.k_proj": {
+  "o1": 16,
+  "o2": 16,
+  "o3": 20,
+  "i1": 16,
+  "i2": 16,
+  "i3": 20,
+  "r0": 124,
+  "r1": 124,
+  "r2": 124,
+  "nnz": 1310720
+ },
+ "model.layers.6.self_attn.v_proj": {
+  "o1": 16,
+  "o2": 16,
+  "o3": 20,
+  "i1": 16,
+  "i2": 16,
+  "i3": 20,
+  "r0": 124,
+  "r1": 124,
+  "r2": 124,
+  "nnz": 1310720
+ },
+ "model.layers.6.self_attn.o_proj": {
+  "o1": 16,
+  "o2": 16,
+  "o3": 20,
+  "i1": 16,
+  "i2": 16,
+  "i3": 20,
+  "r0": 124,
+  "r1": 124,
+  "r2": 124,
+  "nnz": 1310720
+ },
+ "model.layers.6.mlp.gate_proj": {
+  "o1": 24,
+  "o2": 24,
+  "o3": 24,
+  "i1": 16,
+  "i2": 16,
+  "i3": 20,
+  "r0": 174,
+  "r1": 174,
+  "r2": 174,
+  "nnz": 3538944
+ },
+ "model.layers.6.mlp.up_proj": {
+  "o1": 24,
+  "o2": 24,
+  "o3": 24,
+  "i1": 16,
+  "i2": 16,
+  "i3": 20,
+  "r0": 174,
+  "r1": 174,
+  "r2": 174,
+  "nnz": 3538944
+ },
+ "model.layers.6.mlp.down_proj": {
+  "o1": 16,
+  "o2": 16,
+  "o3": 20,
+  "i1": 24,
+  "i2": 24,
+  "i3": 24,
+  "r0": 174,
+  "r1": 174,
+  "r2": 174,
+  "nnz": 3538944
+ },
+ "model.layers.7.self_attn.q_proj": {
+  "o1": 16,
+  "o2": 16,
+  "o3": 20,
+  "i1": 16,
+  "i2": 16,
+  "i3": 20,
+  "r0": 124,
+  "r1": 124,
+  "r2": 124,
+  "nnz": 1310720
+ },
+ "model.layers.7.self_attn.k_proj": {
+  "o1": 16,
+  "o2": 16,
+  "o3": 20,
+  "i1": 16,
+  "i2": 16,
+  "i3": 20,
+  "r0": 124,
+  "r1": 124,
+  "r2": 124,
+  "nnz": 1310720
+ },
+ "model.layers.7.self_attn.v_proj": {
+  "o1": 16,
+  "o2": 16,
+  "o3": 20,
+  "i1": 16,
+  "i2": 16,
+  "i3": 20,
+  "r0": 124,
+  "r1": 124,
+  "r2": 124,
+  "nnz": 1310720
+ },
+ "model.layers.7.self_attn.o_proj": {
+  "o1": 16,
+  "o2": 16,
+  "o3": 20,
+  "i1": 16,
+  "i2": 16,
+  "i3": 20,
+  "r0": 124,
+  "r1": 124,
+  "r2": 124,
+  "nnz": 1310720
+ },
+ "model.layers.7.mlp.gate_proj": {
+  "o1": 24,
+  "o2": 24,
+  "o3": 24,
+  "i1": 16,
+  "i2": 16,
+  "i3": 20,
+  "r0": 174,
+  "r1": 174,
+  "r2": 174,
+  "nnz": 3538944
+ },
+ "model.layers.7.mlp.up_proj": {
+  "o1": 24,
+  "o2": 24,
+  "o3": 24,
+  "i1": 16,
+  "i2": 16,
+  "i3": 20,
+  "r0": 174,
+  "r1": 174,
+  "r2": 174,
+  "nnz": 3538944
+ },
+ "model.layers.7.mlp.down_proj": {
+  "o1": 16,
+  "o2": 16,
+  "o3": 20,
+  "i1": 24,
+  "i2": 24,
+  "i3": 24,
+  "r0": 174,
+  "r1": 174,
+  "r2": 174,
+  "nnz": 3538944
+ },
+ "model.layers.8.self_attn.q_proj": {
+  "o1": 16,
+  "o2": 16,
+  "o3": 20,
+  "i1": 16,
+  "i2": 16,
+  "i3": 20,
+  "r0": 124,
+  "r1": 124,
+  "r2": 124,
+  "nnz": 1310720
+ },
+ "model.layers.8.self_attn.k_proj": {
+  "o1": 16,
+  "o2": 16,
+  "o3": 20,
+  "i1": 16,
+  "i2": 16,
+  "i3": 20,
+  "r0": 124,
+  "r1": 124,
+  "r2": 124,
+  "nnz": 1310720
+ },
+ "model.layers.8.self_attn.v_proj": {
+  "o1": 16,
+  "o2": 16,
+  "o3": 20,
+  "i1": 16,
+  "i2": 16,
+  "i3": 20,
+  "r0": 124,
+  "r1": 124,
+  "r2": 124,
+  "nnz": 1310720
+ },
+ "model.layers.8.self_attn.o_proj": {
+  "o1": 16,
+  "o2": 16,
+  "o3": 20,
+  "i1": 16,
+  "i2": 16,
+  "i3": 20,
+  "r0": 124,
+  "r1": 124,
+  "r2": 124,
+  "nnz": 1310720
+ },
+ "model.layers.8.mlp.gate_proj": {
+  "o1": 24,
+  "o2": 24,
+  "o3": 24,
+  "i1": 16,
+  "i2": 16,
+  "i3": 20,
+  "r0": 174,
+  "r1": 174,
+  "r2": 174,
+  "nnz": 3538944
+ },
+ "model.layers.8.mlp.up_proj": {
+  "o1": 24,
+  "o2": 24,
+  "o3": 24,
+  "i1": 16,
+  "i2": 16,
+  "i3": 20,
+  "r0": 174,
+  "r1": 174,
+  "r2": 174,
+  "nnz": 3538944
+ },
+ "model.layers.8.mlp.down_proj": {
+  "o1": 16,
+  "o2": 16,
+  "o3": 20,
+  "i1": 24,
+  "i2": 24,
+  "i3": 24,
+  "r0": 174,
+  "r1": 174,
+  "r2": 174,
+  "nnz": 3538944
+ },
+ "model.layers.9.self_attn.q_proj": {
+  "o1": 16,
+  "o2": 16,
+  "o3": 20,
+  "i1": 16,
+  "i2": 16,
+  "i3": 20,
+  "r0": 124,
+  "r1": 124,
+  "r2": 124,
+  "nnz": 1310720
+ },
+ "model.layers.9.self_attn.k_proj": {
+  "o1": 16,
+  "o2": 16,
+  "o3": 20,
+  "i1": 16,
+  "i2": 16,
+  "i3": 20,
+  "r0": 124,
+  "r1": 124,
+  "r2": 124,
+  "nnz": 1310720
+ },
+ "model.layers.9.self_attn.v_proj": {
+  "o1": 16,
+  "o2": 16,
+  "o3": 20,
+  "i1": 16,
+  "i2": 16,
+  "i3": 20,
+  "r0": 124,
+  "r1": 124,
+  "r2": 124,
+  "nnz": 1310720
+ },
+ "model.layers.9.self_attn.o_proj": {
+  "o1": 16,
+  "o2": 16,
+  "o3": 20,
+  "i1": 16,
+  "i2": 16,
+  "i3": 20,
+  "r0": 124,
+  "r1": 124,
+  "r2": 124,
+  "nnz": 1310720
+ },
+ "model.layers.9.mlp.gate_proj": {
+  "o1": 24,
+  "o2": 24,
+  "o3": 24,
+  "i1": 16,
+  "i2": 16,
+  "i3": 20,
+  "r0": 174,
+  "r1": 174,
+  "r2": 174,
+  "nnz": 3538944
+ },
+ "model.layers.9.mlp.up_proj": {
+  "o1": 24,
+  "o2": 24,
+  "o3": 24,
+  "i1": 16,
+  "i2": 16,
+  "i3": 20,
+  "r0": 174,
+  "r1": 174,
+  "r2": 174,
+  "nnz": 3538944
+ },
+ "model.layers.9.mlp.down_proj": {
+  "o1": 16,
+  "o2": 16,
+  "o3": 20,
+  "i1": 24,
+  "i2": 24,
+  "i3": 24,
+  "r0": 174,
+  "r1": 174,
+  "r2": 174,
+  "nnz": 3538944
+ },
+ "model.layers.10.self_attn.q_proj": {
+  "o1": 16,
+  "o2": 16,
+  "o3": 20,
+  "i1": 16,
+  "i2": 16,
+  "i3": 20,
+  "r0": 124,
+  "r1": 124,
+  "r2": 124,
+  "nnz": 1310720
+ },
+ "model.layers.10.self_attn.k_proj": {
+  "o1": 16,
+  "o2": 16,
+  "o3": 20,
+  "i1": 16,
+  "i2": 16,
+  "i3": 20,
+  "r0": 124,
+  "r1": 124,
+  "r2": 124,
+  "nnz": 1310720
+ },
+ "model.layers.10.self_attn.v_proj": {
+  "o1": 16,
+  "o2": 16,
+  "o3": 20,
+  "i1": 16,
+  "i2": 16,
+  "i3": 20,
+  "r0": 124,
+  "r1": 124,
+  "r2": 124,
+  "nnz": 1310720
+ },
+ "model.layers.10.self_attn.o_proj": {
+  "o1": 16,
+  "o2": 16,
+  "o3": 20,
+  "i1": 16,
+  "i2": 16,
+  "i3": 20,
+  "r0": 124,
+  "r1": 124,
+  "r2": 124,
+  "nnz": 1310720
+ },
+ "model.layers.10.mlp.gate_proj": {
+  "o1": 24,
+  "o2": 24,
+  "o3": 24,
+  "i1": 16,
+  "i2": 16,
+  "i3": 20,
+  "r0": 174,
+  "r1": 174,
+  "r2": 174,
+  "nnz": 3538944
+ },
+ "model.layers.10.mlp.up_proj": {
+  "o1": 24,
+  "o2": 24,
+  "o3": 24,
+  "i1": 16,
+  "i2": 16,
+  "i3": 20,
+  "r0": 174,
+  "r1": 174,
+  "r2": 174,
+  "nnz": 3538944
+ },
+ "model.layers.10.mlp.down_proj": {
+  "o1": 16,
+  "o2": 16,
+  "o3": 20,
+  "i1": 24,
+  "i2": 24,
+  "i3": 24,
+  "r0": 174,
+  "r1": 174,
+  "r2": 174,
+  "nnz": 3538944
+ },
+ "model.layers.11.self_attn.q_proj": {
+  "o1": 16,
+  "o2": 16,
+  "o3": 20,
+  "i1": 16,
+  "i2": 16,
+  "i3": 20,
+  "r0": 124,
+  "r1": 124,
+  "r2": 124,
+  "nnz": 1310720
+ },
+ "model.layers.11.self_attn.k_proj": {
+  "o1": 16,
+  "o2": 16,
+  "o3": 20,
+  "i1": 16,
+  "i2": 16,
+  "i3": 20,
+  "r0": 124,
+  "r1": 124,
+  "r2": 124,
+  "nnz": 1310720
+ },
+ "model.layers.11.self_attn.v_proj": {
+  "o1": 16,
+  "o2": 16,
+  "o3": 20,
+  "i1": 16,
+  "i2": 16,
+  "i3": 20,
+  "r0": 124,
+  "r1": 124,
+  "r2": 124,
+  "nnz": 1310720
+ },
+ "model.layers.11.self_attn.o_proj": {
+  "o1": 16,
+  "o2": 16,
+  "o3": 20,
+  "i1": 16,
+  "i2": 16,
+  "i3": 20,
+  "r0": 124,
+  "r1": 124,
+  "r2": 124,
+  "nnz": 1310720
+ },
+ "model.layers.11.mlp.gate_proj": {
+  "o1": 24,
+  "o2": 24,
+  "o3": 24,
+  "i1": 16,
+  "i2": 16,
+  "i3": 20,
+  "r0": 174,
+  "r1": 174,
+  "r2": 174,
+  "nnz": 3538944
+ },
+ "model.layers.11.mlp.up_proj": {
+  "o1": 24,
+  "o2": 24,
+  "o3": 24,
+  "i1": 16,
+  "i2": 16,
+  "i3": 20,
+  "r0": 174,
+  "r1": 174,
+  "r2": 174,
+  "nnz": 3538944
+ },
+ "model.layers.11.mlp.down_proj": {
+  "o1": 16,
+  "o2": 16,
+  "o3": 20,
+  "i1": 24,
+  "i2": 24,
+  "i3": 24,
+  "r0": 174,
+  "r1": 174,
+  "r2": 174,
+  "nnz": 3538944
+ },
+ "model.layers.12.self_attn.q_proj": {
+  "o1": 16,
+  "o2": 16,
+  "o3": 20,
+  "i1": 16,
+  "i2": 16,
+  "i3": 20,
+  "r0": 124,
+  "r1": 124,
+  "r2": 124,
+  "nnz": 1310720
+ },
+ "model.layers.12.self_attn.k_proj": {
+  "o1": 16,
+  "o2": 16,
+  "o3": 20,
+  "i1": 16,
+  "i2": 16,
+  "i3": 20,
+  "r0": 124,
+  "r1": 124,
+  "r2": 124,
+  "nnz": 1310720
+ },
+ "model.layers.12.self_attn.v_proj": {
+  "o1": 16,
+  "o2": 16,
+  "o3": 20,
+  "i1": 16,
+  "i2": 16,
+  "i3": 20,
+  "r0": 124,
+  "r1": 124,
+  "r2": 124,
+  "nnz": 1310720
+ },
+ "model.layers.12.self_attn.o_proj": {
+  "o1": 16,
+  "o2": 16,
+  "o3": 20,
+  "i1": 16,
+  "i2": 16,
+  "i3": 20,
+  "r0": 124,
+  "r1": 124,
+  "r2": 124,
+  "nnz": 1310720
+ },
+ "model.layers.12.mlp.gate_proj": {
+  "o1": 24,
+  "o2": 24,
+  "o3": 24,
+  "i1": 16,
+  "i2": 16,
+  "i3": 20,
+  "r0": 174,
+  "r1": 174,
+  "r2": 174,
+  "nnz": 3538944
+ },
+ "model.layers.12.mlp.up_proj": {
+  "o1": 24,
+  "o2": 24,
+  "o3": 24,
+  "i1": 16,
+  "i2": 16,
+  "i3": 20,
+  "r0": 174,
+  "r1": 174,
+  "r2": 174,
+  "nnz": 3538944
+ },
+ "model.layers.12.mlp.down_proj": {
+  "o1": 16,
+  "o2": 16,
+  "o3": 20,
+  "i1": 24,
+  "i2": 24,
+  "i3": 24,
+  "r0": 174,
+  "r1": 174,
+  "r2": 174,
+  "nnz": 3538944
+ },
+ "model.layers.13.self_attn.q_proj": {
+  "o1": 16,
+  "o2": 16,
+  "o3": 20,
+  "i1": 16,
+  "i2": 16,
+  "i3": 20,
+  "r0": 124,
+  "r1": 124,
+  "r2": 124,
+  "nnz": 1310720
+ },
+ "model.layers.13.self_attn.k_proj": {
+  "o1": 16,
+  "o2": 16,
+  "o3": 20,
+  "i1": 16,
+  "i2": 16,
+  "i3": 20,
+  "r0": 124,
+  "r1": 124,
+  "r2": 124,
+  "nnz": 1310720
+ },
+ "model.layers.13.self_attn.v_proj": {
+  "o1": 16,
+  "o2": 16,
+  "o3": 20,
+  "i1": 16,
+  "i2": 16,
+  "i3": 20,
+  "r0": 124,
+  "r1": 124,
+  "r2": 124,
+  "nnz": 1310720
+ },
+ "model.layers.13.self_attn.o_proj": {
+  "o1": 16,
+  "o2": 16,
+  "o3": 20,
+  "i1": 16,
+  "i2": 16,
+  "i3": 20,
+  "r0": 124,
+  "r1": 124,
+  "r2": 124,
+  "nnz": 1310720
+ },
+ "model.layers.13.mlp.gate_proj": {
+  "o1": 24,
+  "o2": 24,
+  "o3": 24,
+  "i1": 16,
+  "i2": 16,
+  "i3": 20,
+  "r0": 174,
+  "r1": 174,
+  "r2": 174,
+  "nnz": 3538944
+ },
+ "model.layers.13.mlp.up_proj": {
+  "o1": 24,
+  "o2": 24,
+  "o3": 24,
+  "i1": 16,
+  "i2": 16,
+  "i3": 20,
+  "r0": 174,
+  "r1": 174,
+  "r2": 174,
+  "nnz": 3538944
+ },
+ "model.layers.13.mlp.down_proj": {
+  "o1": 16,
+  "o2": 16,
+  "o3": 20,
+  "i1": 24,
+  "i2": 24,
+  "i3": 24,
+  "r0": 174,
+  "r1": 174,
+  "r2": 174,
+  "nnz": 3538944
+ },
+ "model.layers.14.self_attn.q_proj": {
+  "o1": 16,
+  "o2": 16,
+  "o3": 20,
+  "i1": 16,
+  "i2": 16,
+  "i3": 20,
+  "r0": 124,
+  "r1": 124,
+  "r2": 124,
+  "nnz": 1310720
+ },
+ "model.layers.14.self_attn.k_proj": {
+  "o1": 16,
+  "o2": 16,
+  "o3": 20,
+  "i1": 16,
+  "i2": 16,
+  "i3": 20,
+  "r0": 124,
+  "r1": 124,
+  "r2": 124,
+  "nnz": 1310720
+ },
+ "model.layers.14.self_attn.v_proj": {
+  "o1": 16,
+  "o2": 16,
+  "o3": 20,
+  "i1": 16,
+  "i2": 16,
+  "i3": 20,
+  "r0": 124,
+  "r1": 124,
+  "r2": 124,
+  "nnz": 1310720
+ },
+ "model.layers.14.self_attn.o_proj": {
+  "o1": 16,
+  "o2": 16,
+  "o3": 20,
+  "i1": 16,
+  "i2": 16,
+  "i3": 20,
+  "r0": 124,
+  "r1": 124,
+  "r2": 124,
+  "nnz": 1310720
+ },
+ "model.layers.14.mlp.gate_proj": {
+  "o1": 24,
+  "o2": 24,
+  "o3": 24,
+  "i1": 16,
+  "i2": 16,
+  "i3": 20,
+  "r0": 174,
+  "r1": 174,
+  "r2": 174,
+  "nnz": 3538944
+ },
+ "model.layers.14.mlp.up_proj": {
+  "o1": 24,
+  "o2": 24,
+  "o3": 24,
+  "i1": 16,
+  "i2": 16,
+  "i3": 20,
+  "r0": 174,
+  "r1": 174,
+  "r2": 174,
+  "nnz": 3538944
+ },
+ "model.layers.14.mlp.down_proj": {
+  "o1": 16,
+  "o2": 16,
+  "o3": 20,
+  "i1": 24,
+  "i2": 24,
+  "i3": 24,
+  "r0": 174,
+  "r1": 174,
+  "r2": 174,
+  "nnz": 3538944
+ },
+ "model.layers.15.self_attn.q_proj": {
+  "o1": 16,
+  "o2": 16,
+  "o3": 20,
+  "i1": 16,
+  "i2": 16,
+  "i3": 20,
+  "r0": 124,
+  "r1": 124,
+  "r2": 124,
+  "nnz": 1310720
+ },
+ "model.layers.15.self_attn.k_proj": {
+  "o1": 16,
+  "o2": 16,
+  "o3": 20,
+  "i1": 16,
+  "i2": 16,
+  "i3": 20,
+  "r0": 124,
+  "r1": 124,
+  "r2": 124,
+  "nnz": 1310720
+ },
+ "model.layers.15.self_attn.v_proj": {
+  "o1": 16,
+  "o2": 16,
+  "o3": 20,
+  "i1": 16,
+  "i2": 16,
+  "i3": 20,
+  "r0": 124,
+  "r1": 124,
+  "r2": 124,
+  "nnz": 1310720
+ },
+ "model.layers.15.self_attn.o_proj": {
+  "o1": 16,
+  "o2": 16,
+  "o3": 20,
+  "i1": 16,
+  "i2": 16,
+  "i3": 20,
+  "r0": 124,
+  "r1": 124,
+  "r2": 124,
+  "nnz": 1310720
+ },
+ "model.layers.15.mlp.gate_proj": {
+  "o1": 24,
+  "o2": 24,
+  "o3": 24,
+  "i1": 16,
+  "i2": 16,
+  "i3": 20,
+  "r0": 174,
+  "r1": 174,
+  "r2": 174,
+  "nnz": 3538944
+ },
+ "model.layers.15.mlp.up_proj": {
+  "o1": 24,
+  "o2": 24,
+  "o3": 24,
+  "i1": 16,
+  "i2": 16,
+  "i3": 20,
+  "r0": 174,
+  "r1": 174,
+  "r2": 174,
+  "nnz": 3538944
+ },
+ "model.layers.15.mlp.down_proj": {
+  "o1": 16,
+  "o2": 16,
+  "o3": 20,
+  "i1": 24,
+  "i2": 24,
+  "i3": 24,
+  "r0": 174,
+  "r1": 174,
+  "r2": 174,
+  "nnz": 3538944
+ },
+ "model.layers.16.self_attn.q_proj": {
+  "o1": 16,
+  "o2": 16,
+  "o3": 20,
+  "i1": 16,
+  "i2": 16,
+  "i3": 20,
+  "r0": 124,
+  "r1": 124,
+  "r2": 124,
+  "nnz": 1310720
+ },
+ "model.layers.16.self_attn.k_proj": {
+  "o1": 16,
+  "o2": 16,
+  "o3": 20,
+  "i1": 16,
+  "i2": 16,
+  "i3": 20,
+  "r0": 124,
+  "r1": 124,
+  "r2": 124,
+  "nnz": 1310720
+ },
+ "model.layers.16.self_attn.v_proj": {
+  "o1": 16,
+  "o2": 16,
+  "o3": 20,
+  "i1": 16,
+  "i2": 16,
+  "i3": 20,
+  "r0": 124,
+  "r1": 124,
+  "r2": 124,
+  "nnz": 1310720
+ },
+ "model.layers.16.self_attn.o_proj": {
+  "o1": 16,
+  "o2": 16,
+  "o3": 20,
+  "i1": 16,
+  "i2": 16,
+  "i3": 20,
+  "r0": 124,
+  "r1": 124,
+  "r2": 124,
+  "nnz": 1310720
+ },
+ "model.layers.16.mlp.gate_proj": {
+  "o1": 24,
+  "o2": 24,
+  "o3": 24,
+  "i1": 16,
+  "i2": 16,
+  "i3": 20,
+  "r0": 174,
+  "r1": 174,
+  "r2": 174,
+  "nnz": 3538944
+ },
+ "model.layers.16.mlp.up_proj": {
+  "o1": 24,
+  "o2": 24,
+  "o3": 24,
+  "i1": 16,
+  "i2": 16,
+  "i3": 20,
+  "r0": 174,
+  "r1": 174,
+  "r2": 174,
+  "nnz": 3538944
+ },
+ "model.layers.16.mlp.down_proj": {
+  "o1": 16,
+  "o2": 16,
+  "o3": 20,
+  "i1": 24,
+  "i2": 24,
+  "i3": 24,
+  "r0": 174,
+  "r1": 174,
+  "r2": 174,
+  "nnz": 3538944
+ },
+ "model.layers.17.self_attn.q_proj": {
+  "o1": 16,
+  "o2": 16,
+  "o3": 20,
+  "i1": 16,
+  "i2": 16,
+  "i3": 20,
+  "r0": 124,
+  "r1": 124,
+  "r2": 124,
+  "nnz": 1310720
+ },
+ "model.layers.17.self_attn.k_proj": {
+  "o1": 16,
+  "o2": 16,
+  "o3": 20,
+  "i1": 16,
+  "i2": 16,
+  "i3": 20,
+  "r0": 124,
+  "r1": 124,
+  "r2": 124,
+  "nnz": 1310720
+ },
+ "model.layers.17.self_attn.v_proj": {
+  "o1": 16,
+  "o2": 16,
+  "o3": 20,
+  "i1": 16,
+  "i2": 16,
+  "i3": 20,
+  "r0": 124,
+  "r1": 124,
+  "r2": 124,
+  "nnz": 1310720
+ },
+ "model.layers.17.self_attn.o_proj": {
+  "o1": 16,
+  "o2": 16,
+  "o3": 20,
+  "i1": 16,
+  "i2": 16,
+  "i3": 20,
+  "r0": 124,
+  "r1": 124,
+  "r2": 124,
+  "nnz": 1310720
+ },
+ "model.layers.17.mlp.gate_proj": {
+  "o1": 24,
+  "o2": 24,
+  "o3": 24,
+  "i1": 16,
+  "i2": 16,
+  "i3": 20,
+  "r0": 174,
+  "r1": 174,
+  "r2": 174,
+  "nnz": 3538944
+ },
+ "model.layers.17.mlp.up_proj": {
+  "o1": 24,
+  "o2": 24,
+  "o3": 24,
+  "i1": 16,
+  "i2": 16,
+  "i3": 20,
+  "r0": 174,
+  "r1": 174,
+  "r2": 174,
+  "nnz": 3538944
+ },
+ "model.layers.17.mlp.down_proj": {
+  "o1": 16,
+  "o2": 16,
+  "o3": 20,
+  "i1": 24,
+  "i2": 24,
+  "i3": 24,
+  "r0": 174,
+  "r1": 174,
+  "r2": 174,
+  "nnz": 3538944
+ },
+ "model.layers.18.self_attn.q_proj": {
+  "o1": 16,
+  "o2": 16,
+  "o3": 20,
+  "i1": 16,
+  "i2": 16,
+  "i3": 20,
+  "r0": 124,
+  "r1": 124,
+  "r2": 124,
+  "nnz": 1310720
+ },
+ "model.layers.18.self_attn.k_proj": {
+  "o1": 16,
+  "o2": 16,
+  "o3": 20,
+  "i1": 16,
+  "i2": 16,
+  "i3": 20,
+  "r0": 124,
+  "r1": 124,
+  "r2": 124,
+  "nnz": 1310720
+ },
+ "model.layers.18.self_attn.v_proj": {
+  "o1": 16,
+  "o2": 16,
+  "o3": 20,
+  "i1": 16,
+  "i2": 16,
+  "i3": 20,
+  "r0": 124,
+  "r1": 124,
+  "r2": 124,
+  "nnz": 1310720
+ },
+ "model.layers.18.self_attn.o_proj": {
+  "o1": 16,
+  "o2": 16,
+  "o3": 20,
+  "i1": 16,
+  "i2": 16,
+  "i3": 20,
+  "r0": 124,
+  "r1": 124,
+  "r2": 124,
+  "nnz": 1310720
+ },
+ "model.layers.18.mlp.gate_proj": {
+  "o1": 24,
+  "o2": 24,
+  "o3": 24,
+  "i1": 16,
+  "i2": 16,
+  "i3": 20,
+  "r0": 174,
+  "r1": 174,
+  "r2": 174,
+  "nnz": 3538944
+ },
+ "model.layers.18.mlp.up_proj": {
+  "o1": 24,
+  "o2": 24,
+  "o3": 24,
+  "i1": 16,
+  "i2": 16,
+  "i3": 20,
+  "r0": 174,
+  "r1": 174,
+  "r2": 174,
+  "nnz": 3538944
+ },
+ "model.layers.18.mlp.down_proj": {
+  "o1": 16,
+  "o2": 16,
+  "o3": 20,
+  "i1": 24,
+  "i2": 24,
+  "i3": 24,
+  "r0": 174,
+  "r1": 174,
+  "r2": 174,
+  "nnz": 3538944
+ },
+ "model.layers.19.self_attn.q_proj": {
+  "o1": 16,
+  "o2": 16,
+  "o3": 20,
+  "i1": 16,
+  "i2": 16,
+  "i3": 20,
+  "r0": 124,
+  "r1": 124,
+  "r2": 124,
+  "nnz": 1310720
+ },
+ "model.layers.19.self_attn.k_proj": {
+  "o1": 16,
+  "o2": 16,
+  "o3": 20,
+  "i1": 16,
+  "i2": 16,
+  "i3": 20,
+  "r0": 124,
+  "r1": 124,
+  "r2": 124,
+  "nnz": 1310720
+ },
+ "model.layers.19.self_attn.v_proj": {
+  "o1": 16,
+  "o2": 16,
+  "o3": 20,
+  "i1": 16,
+  "i2": 16,
+  "i3": 20,
+  "r0": 124,
+  "r1": 124,
+  "r2": 124,
+  "nnz": 1310720
+ },
+ "model.layers.19.self_attn.o_proj": {
+  "o1": 16,
+  "o2": 16,
+  "o3": 20,
+  "i1": 16,
+  "i2": 16,
+  "i3": 20,
+  "r0": 124,
+  "r1": 124,
+  "r2": 124,
+  "nnz": 1310720
+ },
+ "model.layers.19.mlp.gate_proj": {
+  "o1": 24,
+  "o2": 24,
+  "o3": 24,
+  "i1": 16,
+  "i2": 16,
+  "i3": 20,
+  "r0": 174,
+  "r1": 174,
+  "r2": 174,
+  "nnz": 3538944
+ },
+ "model.layers.19.mlp.up_proj": {
+  "o1": 24,
+  "o2": 24,
+  "o3": 24,
+  "i1": 16,
+  "i2": 16,
+  "i3": 20,
+  "r0": 174,
+  "r1": 174,
+  "r2": 174,
+  "nnz": 3538944
+ },
+ "model.layers.19.mlp.down_proj": {
+  "o1": 16,
+  "o2": 16,
+  "o3": 20,
+  "i1": 24,
+  "i2": 24,
+  "i3": 24,
+  "r0": 174,
+  "r1": 174,
+  "r2": 174,
+  "nnz": 3538944
+ },
+ "model.layers.20.self_attn.q_proj": {
+  "o1": 16,
+  "o2": 16,
+  "o3": 20,
+  "i1": 16,
+  "i2": 16,
+  "i3": 20,
+  "r0": 124,
+  "r1": 124,
+  "r2": 124,
+  "nnz": 1310720
+ },
+ "model.layers.20.self_attn.k_proj": {
+  "o1": 16,
+  "o2": 16,
+  "o3": 20,
+  "i1": 16,
+  "i2": 16,
+  "i3": 20,
+  "r0": 124,
+  "r1": 124,
+  "r2": 124,
+  "nnz": 1310720
+ },
+ "model.layers.20.self_attn.v_proj": {
+  "o1": 16,
+  "o2": 16,
+  "o3": 20,
+  "i1": 16,
+  "i2": 16,
+  "i3": 20,
+  "r0": 124,
+  "r1": 124,
+  "r2": 124,
+  "nnz": 1310720
+ },
+ "model.layers.20.self_attn.o_proj": {
+  "o1": 16,
+  "o2": 16,
+  "o3": 20,
+  "i1": 16,
+  "i2": 16,
+  "i3": 20,
+  "r0": 124,
+  "r1": 124,
+  "r2": 124,
+  "nnz": 1310720
+ },
+ "model.layers.20.mlp.gate_proj": {
+  "o1": 24,
+  "o2": 24,
+  "o3": 24,
+  "i1": 16,
+  "i2": 16,
+  "i3": 20,
+  "r0": 174,
+  "r1": 174,
+  "r2": 174,
+  "nnz": 3538944
+ },
+ "model.layers.20.mlp.up_proj": {
+  "o1": 24,
+  "o2": 24,
+  "o3": 24,
+  "i1": 16,
+  "i2": 16,
+  "i3": 20,
+  "r0": 174,
+  "r1": 174,
+  "r2": 174,
+  "nnz": 3538944
+ },
+ "model.layers.20.mlp.down_proj": {
+  "o1": 16,
+  "o2": 16,
+  "o3": 20,
+  "i1": 24,
+  "i2": 24,
+  "i3": 24,
+  "r0": 174,
+  "r1": 174,
+  "r2": 174,
+  "nnz": 3538944
+ },
+ "model.layers.21.self_attn.q_proj": {
+  "o1": 16,
+  "o2": 16,
+  "o3": 20,
+  "i1": 16,
+  "i2": 16,
+  "i3": 20,
+  "r0": 124,
+  "r1": 124,
+  "r2": 124,
+  "nnz": 1310720
+ },
+ "model.layers.21.self_attn.k_proj": {
+  "o1": 16,
+  "o2": 16,
+  "o3": 20,
+  "i1": 16,
+  "i2": 16,
+  "i3": 20,
+  "r0": 124,
+  "r1": 124,
+  "r2": 124,
+  "nnz": 1310720
+ },
+ "model.layers.21.self_attn.v_proj": {
+  "o1": 16,
+  "o2": 16,
+  "o3": 20,
+  "i1": 16,
+  "i2": 16,
+  "i3": 20,
+  "r0": 124,
+  "r1": 124,
+  "r2": 124,
+  "nnz": 1310720
+ },
+ "model.layers.21.self_attn.o_proj": {
+  "o1": 16,
+  "o2": 16,
+  "o3": 20,
+  "i1": 16,
+  "i2": 16,
+  "i3": 20,
+  "r0": 124,
+  "r1": 124,
+  "r2": 124,
+  "nnz": 1310720
+ },
+ "model.layers.21.mlp.gate_proj": {
+  "o1": 24,
+  "o2": 24,
+  "o3": 24,
+  "i1": 16,
+  "i2": 16,
+  "i3": 20,
+  "r0": 174,
+  "r1": 174,
+  "r2": 174,
+  "nnz": 3538944
+ },
+ "model.layers.21.mlp.up_proj": {
+  "o1": 24,
+  "o2": 24,
+  "o3": 24,
+  "i1": 16,
+  "i2": 16,
+  "i3": 20,
+  "r0": 174,
+  "r1": 174,
+  "r2": 174,
+  "nnz": 3538944
+ },
+ "model.layers.21.mlp.down_proj": {
+  "o1": 16,
+  "o2": 16,
+  "o3": 20,
+  "i1": 24,
+  "i2": 24,
+  "i3": 24,
+  "r0": 174,
+  "r1": 174,
+  "r2": 174,
+  "nnz": 3538944
+ },
+ "model.layers.22.self_attn.q_proj": {
+  "o1": 16,
+  "o2": 16,
+  "o3": 20,
+  "i1": 16,
+  "i2": 16,
+  "i3": 20,
+  "r0": 124,
+  "r1": 124,
+  "r2": 124,
+  "nnz": 1310720
+ },
+ "model.layers.22.self_attn.k_proj": {
+  "o1": 16,
+  "o2": 16,
+  "o3": 20,
+  "i1": 16,
+  "i2": 16,
+  "i3": 20,
+  "r0": 124,
+  "r1": 124,
+  "r2": 124,
+  "nnz": 1310720
+ },
+ "model.layers.22.self_attn.v_proj": {
+  "o1": 16,
+  "o2": 16,
+  "o3": 20,
+  "i1": 16,
+  "i2": 16,
+  "i3": 20,
+  "r0": 124,
+  "r1": 124,
+  "r2": 124,
+  "nnz": 1310720
+ },
+ "model.layers.22.self_attn.o_proj": {
+  "o1": 16,
+  "o2": 16,
+  "o3": 20,
+  "i1": 16,
+  "i2": 16,
+  "i3": 20,
+  "r0": 124,
+  "r1": 124,
+  "r2": 124,
+  "nnz": 1310720
+ },
+ "model.layers.22.mlp.gate_proj": {
+  "o1": 24,
+  "o2": 24,
+  "o3": 24,
+  "i1": 16,
+  "i2": 16,
+  "i3": 20,
+  "r0": 174,
+  "r1": 174,
+  "r2": 174,
+  "nnz": 3538944
+ },
+ "model.layers.22.mlp.up_proj": {
+  "o1": 24,
+  "o2": 24,
+  "o3": 24,
+  "i1": 16,
+  "i2": 16,
+  "i3": 20,
+  "r0": 174,
+  "r1": 174,
+  "r2": 174,
+  "nnz": 3538944
+ },
+ "model.layers.22.mlp.down_proj": {
+  "o1": 16,
+  "o2": 16,
+  "o3": 20,
+  "i1": 24,
+  "i2": 24,
+  "i3": 24,
+  "r0": 174,
+  "r1": 174,
+  "r2": 174,
+  "nnz": 3538944
+ },
+ "model.layers.23.self_attn.q_proj": {
+  "o1": 16,
+  "o2": 16,
+  "o3": 20,
+  "i1": 16,
+  "i2": 16,
+  "i3": 20,
+  "r0": 124,
+  "r1": 124,
+  "r2": 124,
+  "nnz": 1310720
+ },
+ "model.layers.23.self_attn.k_proj": {
+  "o1": 16,
+  "o2": 16,
+  "o3": 20,
+  "i1": 16,
+  "i2": 16,
+  "i3": 20,
+  "r0": 124,
+  "r1": 124,
+  "r2": 124,
+  "nnz": 1310720
+ },
+ "model.layers.23.self_attn.v_proj": {
+  "o1": 16,
+  "o2": 16,
+  "o3": 20,
+  "i1": 16,
+  "i2": 16,
+  "i3": 20,
+  "r0": 124,
+  "r1": 124,
+  "r2": 124,
+  "nnz": 1310720
+ },
+ "model.layers.23.self_attn.o_proj": {
+  "o1": 16,
+  "o2": 16,
+  "o3": 20,
+  "i1": 16,
+  "i2": 16,
+  "i3": 20,
+  "r0": 124,
+  "r1": 124,
+  "r2": 124,
+  "nnz": 1310720
+ },
+ "model.layers.23.mlp.gate_proj": {
+  "o1": 24,
+  "o2": 24,
+  "o3": 24,
+  "i1": 16,
+  "i2": 16,
+  "i3": 20,
+  "r0": 174,
+  "r1": 174,
+  "r2": 174,
+  "nnz": 3538944
+ },
+ "model.layers.23.mlp.up_proj": {
+  "o1": 24,
+  "o2": 24,
+  "o3": 24,
+  "i1": 16,
+  "i2": 16,
+  "i3": 20,
+  "r0": 174,
+  "r1": 174,
+  "r2": 174,
+  "nnz": 3538944
+ },
+ "model.layers.23.mlp.down_proj": {
+  "o1": 16,
+  "o2": 16,
+  "o3": 20,
+  "i1": 24,
+  "i2": 24,
+  "i3": 24,
+  "r0": 174,
+  "r1": 174,
+  "r2": 174,
+  "nnz": 3538944
+ },
+ "model.layers.24.self_attn.q_proj": {
+  "o1": 16,
+  "o2": 16,
+  "o3": 20,
+  "i1": 16,
+  "i2": 16,
+  "i3": 20,
+  "r0": 124,
+  "r1": 124,
+  "r2": 124,
+  "nnz": 1310720
+ },
+ "model.layers.24.self_attn.k_proj": {
+  "o1": 16,
+  "o2": 16,
+  "o3": 20,
+  "i1": 16,
+  "i2": 16,
+  "i3": 20,
+  "r0": 124,
+  "r1": 124,
+  "r2": 124,
+  "nnz": 1310720
+ },
+ "model.layers.24.self_attn.v_proj": {
+  "o1": 16,
+  "o2": 16,
+  "o3": 20,
+  "i1": 16,
+  "i2": 16,
+  "i3": 20,
+  "r0": 124,
+  "r1": 124,
+  "r2": 124,
+  "nnz": 1310720
+ },
+ "model.layers.24.self_attn.o_proj": {
+  "o1": 16,
+  "o2": 16,
+  "o3": 20,
+  "i1": 16,
+  "i2": 16,
+  "i3": 20,
+  "r0": 124,
+  "r1": 124,
+  "r2": 124,
+  "nnz": 1310720
+ },
+ "model.layers.24.mlp.gate_proj": {
+  "o1": 24,
+  "o2": 24,
+  "o3": 24,
+  "i1": 16,
+  "i2": 16,
+  "i3": 20,
+  "r0": 174,
+  "r1": 174,
+  "r2": 174,
+  "nnz": 3538944
+ },
+ "model.layers.24.mlp.up_proj": {
+  "o1": 24,
+  "o2": 24,
+  "o3": 24,
+  "i1": 16,
+  "i2": 16,
+  "i3": 20,
+  "r0": 174,
+  "r1": 174,
+  "r2": 174,
+  "nnz": 3538944
+ },
+ "model.layers.24.mlp.down_proj": {
+  "o1": 16,
+  "o2": 16,
+  "o3": 20,
+  "i1": 24,
+  "i2": 24,
+  "i3": 24,
+  "r0": 174,
+  "r1": 174,
+  "r2": 174,
+  "nnz": 3538944
+ },
+ "model.layers.25.self_attn.q_proj": {
+  "o1": 16,
+  "o2": 16,
+  "o3": 20,
+  "i1": 16,
+  "i2": 16,
+  "i3": 20,
+  "r0": 124,
+  "r1": 124,
+  "r2": 124,
+  "nnz": 1310720
+ },
+ "model.layers.25.self_attn.k_proj": {
+  "o1": 16,
+  "o2": 16,
+  "o3": 20,
+  "i1": 16,
+  "i2": 16,
+  "i3": 20,
+  "r0": 124,
+  "r1": 124,
+  "r2": 124,
+  "nnz": 1310720
+ },
+ "model.layers.25.self_attn.v_proj": {
+  "o1": 16,
+  "o2": 16,
+  "o3": 20,
+  "i1": 16,
+  "i2": 16,
+  "i3": 20,
+  "r0": 124,
+  "r1": 124,
+  "r2": 124,
+  "nnz": 1310720
+ },
+ "model.layers.25.self_attn.o_proj": {
+  "o1": 16,
+  "o2": 16,
+  "o3": 20,
+  "i1": 16,
+  "i2": 16,
+  "i3": 20,
+  "r0": 124,
+  "r1": 124,
+  "r2": 124,
+  "nnz": 1310720
+ },
+ "model.layers.25.mlp.gate_proj": {
+  "o1": 24,
+  "o2": 24,
+  "o3": 24,
+  "i1": 16,
+  "i2": 16,
+  "i3": 20,
+  "r0": 174,
+  "r1": 174,
+  "r2": 174,
+  "nnz": 3538944
+ },
+ "model.layers.25.mlp.up_proj": {
+  "o1": 24,
+  "o2": 24,
+  "o3": 24,
+  "i1": 16,
+  "i2": 16,
+  "i3": 20,
+  "r0": 174,
+  "r1": 174,
+  "r2": 174,
+  "nnz": 3538944
+ },
+ "model.layers.25.mlp.down_proj": {
+  "o1": 16,
+  "o2": 16,
+  "o3": 20,
+  "i1": 24,
+  "i2": 24,
+  "i3": 24,
+  "r0": 174,
+  "r1": 174,
+  "r2": 174,
+  "nnz": 3538944
+ },
+ "model.layers.26.self_attn.q_proj": {
+  "o1": 16,
+  "o2": 16,
+  "o3": 20,
+  "i1": 16,
+  "i2": 16,
+  "i3": 20,
+  "r0": 124,
+  "r1": 124,
+  "r2": 124,
+  "nnz": 1310720
+ },
+ "model.layers.26.self_attn.k_proj": {
+  "o1": 16,
+  "o2": 16,
+  "o3": 20,
+  "i1": 16,
+  "i2": 16,
+  "i3": 20,
+  "r0": 124,
+  "r1": 124,
+  "r2": 124,
+  "nnz": 1310720
+ },
+ "model.layers.26.self_attn.v_proj": {
+  "o1": 16,
+  "o2": 16,
+  "o3": 20,
+  "i1": 16,
+  "i2": 16,
+  "i3": 20,
+  "r0": 124,
+  "r1": 124,
+  "r2": 124,
+  "nnz": 1310720
+ },
+ "model.layers.26.self_attn.o_proj": {
+  "o1": 16,
+  "o2": 16,
+  "o3": 20,
+  "i1": 16,
+  "i2": 16,
+  "i3": 20,
+  "r0": 124,
+  "r1": 124,
+  "r2": 124,
+  "nnz": 1310720
+ },
+ "model.layers.26.mlp.gate_proj": {
+  "o1": 24,
+  "o2": 24,
+  "o3": 24,
+  "i1": 16,
+  "i2": 16,
+  "i3": 20,
+  "r0": 174,
+  "r1": 174,
+  "r2": 174,
+  "nnz": 3538944
+ },
+ "model.layers.26.mlp.up_proj": {
+  "o1": 24,
+  "o2": 24,
+  "o3": 24,
+  "i1": 16,
+  "i2": 16,
+  "i3": 20,
+  "r0": 174,
+  "r1": 174,
+  "r2": 174,
+  "nnz": 3538944
+ },
+ "model.layers.26.mlp.down_proj": {
+  "o1": 16,
+  "o2": 16,
+  "o3": 20,
+  "i1": 24,
+  "i2": 24,
+  "i3": 24,
+  "r0": 174,
+  "r1": 174,
+  "r2": 174,
+  "nnz": 3538944
+ },
+ "model.layers.27.self_attn.q_proj": {
+  "o1": 16,
+  "o2": 16,
+  "o3": 20,
+  "i1": 16,
+  "i2": 16,
+  "i3": 20,
+  "r0": 124,
+  "r1": 124,
+  "r2": 124,
+  "nnz": 1310720
+ },
+ "model.layers.27.self_attn.k_proj": {
+  "o1": 16,
+  "o2": 16,
+  "o3": 20,
+  "i1": 16,
+  "i2": 16,
+  "i3": 20,
+  "r0": 124,
+  "r1": 124,
+  "r2": 124,
+  "nnz": 1310720
+ },
+ "model.layers.27.self_attn.v_proj": {
+  "o1": 16,
+  "o2": 16,
+  "o3": 20,
+  "i1": 16,
+  "i2": 16,
+  "i3": 20,
+  "r0": 124,
+  "r1": 124,
+  "r2": 124,
+  "nnz": 1310720
+ },
+ "model.layers.27.self_attn.o_proj": {
+  "o1": 16,
+  "o2": 16,
+  "o3": 20,
+  "i1": 16,
+  "i2": 16,
+  "i3": 20,
+  "r0": 124,
+  "r1": 124,
+  "r2": 124,
+  "nnz": 1310720
+ },
+ "model.layers.27.mlp.gate_proj": {
+  "o1": 24,
+  "o2": 24,
+  "o3": 24,
+  "i1": 16,
+  "i2": 16,
+  "i3": 20,
+  "r0": 174,
+  "r1": 174,
+  "r2": 174,
+  "nnz": 3538944
+ },
+ "model.layers.27.mlp.up_proj": {
+  "o1": 24,
+  "o2": 24,
+  "o3": 24,
+  "i1": 16,
+  "i2": 16,
+  "i3": 20,
+  "r0": 174,
+  "r1": 174,
+  "r2": 174,
+  "nnz": 3538944
+ },
+ "model.layers.27.mlp.down_proj": {
+  "o1": 16,
+  "o2": 16,
+  "o3": 20,
+  "i1": 24,
+  "i2": 24,
+  "i3": 24,
+  "r0": 174,
+  "r1": 174,
+  "r2": 174,
+  "nnz": 3538944
+ },
+ "model.layers.28.self_attn.q_proj": {
+  "o1": 16,
+  "o2": 16,
+  "o3": 20,
+  "i1": 16,
+  "i2": 16,
+  "i3": 20,
+  "r0": 124,
+  "r1": 124,
+  "r2": 124,
+  "nnz": 1310720
+ },
+ "model.layers.28.self_attn.k_proj": {
+  "o1": 16,
+  "o2": 16,
+  "o3": 20,
+  "i1": 16,
+  "i2": 16,
+  "i3": 20,
+  "r0": 124,
+  "r1": 124,
+  "r2": 124,
+  "nnz": 1310720
+ },
+ "model.layers.28.self_attn.v_proj": {
+  "o1": 16,
+  "o2": 16,
+  "o3": 20,
+  "i1": 16,
+  "i2": 16,
+  "i3": 20,
+  "r0": 124,
+  "r1": 124,
+  "r2": 124,
+  "nnz": 1310720
+ },
+ "model.layers.28.self_attn.o_proj": {
+  "o1": 16,
+  "o2": 16,
+  "o3": 20,
+  "i1": 16,
+  "i2": 16,
+  "i3": 20,
+  "r0": 124,
+  "r1": 124,
+  "r2": 124,
+  "nnz": 1310720
+ },
+ "model.layers.28.mlp.gate_proj": {
+  "o1": 24,
+  "o2": 24,
+  "o3": 24,
+  "i1": 16,
+  "i2": 16,
+  "i3": 20,
+  "r0": 174,
+  "r1": 174,
+  "r2": 174,
+  "nnz": 3538944
+ },
+ "model.layers.28.mlp.up_proj": {
+  "o1": 24,
+  "o2": 24,
+  "o3": 24,
+  "i1": 16,
+  "i2": 16,
+  "i3": 20,
+  "r0": 174,
+  "r1": 174,
+  "r2": 174,
+  "nnz": 3538944
+ },
+ "model.layers.28.mlp.down_proj": {
+  "o1": 16,
+  "o2": 16,
+  "o3": 20,
+  "i1": 24,
+  "i2": 24,
+  "i3": 24,
+  "r0": 174,
+  "r1": 174,
+  "r2": 174,
+  "nnz": 3538944
+ },
+ "model.layers.29.self_attn.q_proj": {
+  "o1": 16,
+  "o2": 16,
+  "o3": 20,
+  "i1": 16,
+  "i2": 16,
+  "i3": 20,
+  "r0": 124,
+  "r1": 124,
+  "r2": 124,
+  "nnz": 1310720
+ },
+ "model.layers.29.self_attn.k_proj": {
+  "o1": 16,
+  "o2": 16,
+  "o3": 20,
+  "i1": 16,
+  "i2": 16,
+  "i3": 20,
+  "r0": 124,
+  "r1": 124,
+  "r2": 124,
+  "nnz": 1310720
+ },
+ "model.layers.29.self_attn.v_proj": {
+  "o1": 16,
+  "o2": 16,
+  "o3": 20,
+  "i1": 16,
+  "i2": 16,
+  "i3": 20,
+  "r0": 124,
+  "r1": 124,
+  "r2": 124,
+  "nnz": 1310720
+ },
+ "model.layers.29.self_attn.o_proj": {
+  "o1": 16,
+  "o2": 16,
+  "o3": 20,
+  "i1": 16,
+  "i2": 16,
+  "i3": 20,
+  "r0": 124,
+  "r1": 124,
+  "r2": 124,
+  "nnz": 1310720
+ },
+ "model.layers.29.mlp.gate_proj": {
+  "o1": 24,
+  "o2": 24,
+  "o3": 24,
+  "i1": 16,
+  "i2": 16,
+  "i3": 20,
+  "r0": 174,
+  "r1": 174,
+  "r2": 174,
+  "nnz": 3538944
+ },
+ "model.layers.29.mlp.up_proj": {
+  "o1": 24,
+  "o2": 24,
+  "o3": 24,
+  "i1": 16,
+  "i2": 16,
+  "i3": 20,
+  "r0": 174,
+  "r1": 174,
+  "r2": 174,
+  "nnz": 3538944
+ },
+ "model.layers.29.mlp.down_proj": {
+  "o1": 16,
+  "o2": 16,
+  "o3": 20,
+  "i1": 24,
+  "i2": 24,
+  "i3": 24,
+  "r0": 174,
+  "r1": 174,
+  "r2": 174,
+  "nnz": 3538944
+ },
+ "model.layers.30.self_attn.q_proj": {
+  "o1": 16,
+  "o2": 16,
+  "o3": 20,
+  "i1": 16,
+  "i2": 16,
+  "i3": 20,
+  "r0": 124,
+  "r1": 124,
+  "r2": 124,
+  "nnz": 1310720
+ },
+ "model.layers.30.self_attn.k_proj": {
+  "o1": 16,
+  "o2": 16,
+  "o3": 20,
+  "i1": 16,
+  "i2": 16,
+  "i3": 20,
+  "r0": 124,
+  "r1": 124,
+  "r2": 124,
+  "nnz": 1310720
+ },
+ "model.layers.30.self_attn.v_proj": {
+  "o1": 16,
+  "o2": 16,
+  "o3": 20,
+  "i1": 16,
+  "i2": 16,
+  "i3": 20,
+  "r0": 124,
+  "r1": 124,
+  "r2": 124,
+  "nnz": 1310720
+ },
+ "model.layers.30.self_attn.o_proj": {
+  "o1": 16,
+  "o2": 16,
+  "o3": 20,
+  "i1": 16,
+  "i2": 16,
+  "i3": 20,
+  "r0": 124,
+  "r1": 124,
+  "r2": 124,
+  "nnz": 1310720
+ },
+ "model.layers.30.mlp.gate_proj": {
+  "o1": 24,
+  "o2": 24,
+  "o3": 24,
+  "i1": 16,
+  "i2": 16,
+  "i3": 20,
+  "r0": 174,
+  "r1": 174,
+  "r2": 174,
+  "nnz": 3538944
+ },
+ "model.layers.30.mlp.up_proj": {
+  "o1": 24,
+  "o2": 24,
+  "o3": 24,
+  "i1": 16,
+  "i2": 16,
+  "i3": 20,
+  "r0": 174,
+  "r1": 174,
+  "r2": 174,
+  "nnz": 3538944
+ },
+ "model.layers.30.mlp.down_proj": {
+  "o1": 16,
+  "o2": 16,
+  "o3": 20,
+  "i1": 24,
+  "i2": 24,
+  "i3": 24,
+  "r0": 174,
+  "r1": 174,
+  "r2": 174,
+  "nnz": 3538944
+ },
+ "model.layers.31.self_attn.q_proj": {
+  "o1": 16,
+  "o2": 16,
+  "o3": 20,
+  "i1": 16,
+  "i2": 16,
+  "i3": 20,
+  "r0": 124,
+  "r1": 124,
+  "r2": 124,
+  "nnz": 1310720
+ },
+ "model.layers.31.self_attn.k_proj": {
+  "o1": 16,
+  "o2": 16,
+  "o3": 20,
+  "i1": 16,
+  "i2": 16,
+  "i3": 20,
+  "r0": 124,
+  "r1": 124,
+  "r2": 124,
+  "nnz": 1310720
+ },
+ "model.layers.31.self_attn.v_proj": {
+  "o1": 16,
+  "o2": 16,
+  "o3": 20,
+  "i1": 16,
+  "i2": 16,
+  "i3": 20,
+  "r0": 124,
+  "r1": 124,
+  "r2": 124,
+  "nnz": 1310720
+ },
+ "model.layers.31.self_attn.o_proj": {
+  "o1": 16,
+  "o2": 16,
+  "o3": 20,
+  "i1": 16,
+  "i2": 16,
+  "i3": 20,
+  "r0": 124,
+  "r1": 124,
+  "r2": 124,
+  "nnz": 1310720
+ },
+ "model.layers.31.mlp.gate_proj": {
+  "o1": 24,
+  "o2": 24,
+  "o3": 24,
+  "i1": 16,
+  "i2": 16,
+  "i3": 20,
+  "r0": 174,
+  "r1": 174,
+  "r2": 174,
+  "nnz": 3538944
+ },
+ "model.layers.31.mlp.up_proj": {
+  "o1": 24,
+  "o2": 24,
+  "o3": 24,
+  "i1": 16,
+  "i2": 16,
+  "i3": 20,
+  "r0": 174,
+  "r1": 174,
+  "r2": 174,
+  "nnz": 3538944
+ },
+ "model.layers.31.mlp.down_proj": {
+  "o1": 16,
+  "o2": 16,
+  "o3": 20,
+  "i1": 24,
+  "i2": 24,
+  "i3": 24,
+  "r0": 174,
+  "r1": 174,
+  "r2": 174,
+  "nnz": 3538944
+ },
+ "model.layers.32.self_attn.q_proj": {
+  "o1": 16,
+  "o2": 16,
+  "o3": 20,
+  "i1": 16,
+  "i2": 16,
+  "i3": 20,
+  "r0": 124,
+  "r1": 124,
+  "r2": 124,
+  "nnz": 1310720
+ },
+ "model.layers.32.self_attn.k_proj": {
+  "o1": 16,
+  "o2": 16,
+  "o3": 20,
+  "i1": 16,
+  "i2": 16,
+  "i3": 20,
+  "r0": 124,
+  "r1": 124,
+  "r2": 124,
+  "nnz": 1310720
+ },
+ "model.layers.32.self_attn.v_proj": {
+  "o1": 16,
+  "o2": 16,
+  "o3": 20,
+  "i1": 16,
+  "i2": 16,
+  "i3": 20,
+  "r0": 124,
+  "r1": 124,
+  "r2": 124,
+  "nnz": 1310720
+ },
+ "model.layers.32.self_attn.o_proj": {
+  "o1": 16,
+  "o2": 16,
+  "o3": 20,
+  "i1": 16,
+  "i2": 16,
+  "i3": 20,
+  "r0": 124,
+  "r1": 124,
+  "r2": 124,
+  "nnz": 1310720
+ },
+ "model.layers.32.mlp.gate_proj": {
+  "o1": 24,
+  "o2": 24,
+  "o3": 24,
+  "i1": 16,
+  "i2": 16,
+  "i3": 20,
+  "r0": 174,
+  "r1": 174,
+  "r2": 174,
+  "nnz": 3538944
+ },
+ "model.layers.32.mlp.up_proj": {
+  "o1": 24,
+  "o2": 24,
+  "o3": 24,
+  "i1": 16,
+  "i2": 16,
+  "i3": 20,
+  "r0": 174,
+  "r1": 174,
+  "r2": 174,
+  "nnz": 3538944
+ },
+ "model.layers.32.mlp.down_proj": {
+  "o1": 16,
+  "o2": 16,
+  "o3": 20,
+  "i1": 24,
+  "i2": 24,
+  "i3": 24,
+  "r0": 174,
+  "r1": 174,
+  "r2": 174,
+  "nnz": 3538944
+ },
+ "model.layers.33.self_attn.q_proj": {
+  "o1": 16,
+  "o2": 16,
+  "o3": 20,
+  "i1": 16,
+  "i2": 16,
+  "i3": 20,
+  "r0": 124,
+  "r1": 124,
+  "r2": 124,
+  "nnz": 1310720
+ },
+ "model.layers.33.self_attn.k_proj": {
+  "o1": 16,
+  "o2": 16,
+  "o3": 20,
+  "i1": 16,
+  "i2": 16,
+  "i3": 20,
+  "r0": 124,
+  "r1": 124,
+  "r2": 124,
+  "nnz": 1310720
+ },
+ "model.layers.33.self_attn.v_proj": {
+  "o1": 16,
+  "o2": 16,
+  "o3": 20,
+  "i1": 16,
+  "i2": 16,
+  "i3": 20,
+  "r0": 124,
+  "r1": 124,
+  "r2": 124,
+  "nnz": 1310720
+ },
+ "model.layers.33.self_attn.o_proj": {
+  "o1": 16,
+  "o2": 16,
+  "o3": 20,
+  "i1": 16,
+  "i2": 16,
+  "i3": 20,
+  "r0": 124,
+  "r1": 124,
+  "r2": 124,
+  "nnz": 1310720
+ },
+ "model.layers.33.mlp.gate_proj": {
+  "o1": 24,
+  "o2": 24,
+  "o3": 24,
+  "i1": 16,
+  "i2": 16,
+  "i3": 20,
+  "r0": 174,
+  "r1": 174,
+  "r2": 174,
+  "nnz": 3538944
+ },
+ "model.layers.33.mlp.up_proj": {
+  "o1": 24,
+  "o2": 24,
+  "o3": 24,
+  "i1": 16,
+  "i2": 16,
+  "i3": 20,
+  "r0": 174,
+  "r1": 174,
+  "r2": 174,
+  "nnz": 3538944
+ },
+ "model.layers.33.mlp.down_proj": {
+  "o1": 16,
+  "o2": 16,
+  "o3": 20,
+  "i1": 24,
+  "i2": 24,
+  "i3": 24,
+  "r0": 174,
+  "r1": 174,
+  "r2": 174,
+  "nnz": 3538944
+ },
+ "model.layers.34.self_attn.q_proj": {
+  "o1": 16,
+  "o2": 16,
+  "o3": 20,
+  "i1": 16,
+  "i2": 16,
+  "i3": 20,
+  "r0": 124,
+  "r1": 124,
+  "r2": 124,
+  "nnz": 1310720
+ },
+ "model.layers.34.self_attn.k_proj": {
+  "o1": 16,
+  "o2": 16,
+  "o3": 20,
+  "i1": 16,
+  "i2": 16,
+  "i3": 20,
+  "r0": 124,
+  "r1": 124,
+  "r2": 124,
+  "nnz": 1310720
+ },
+ "model.layers.34.self_attn.v_proj": {
+  "o1": 16,
+  "o2": 16,
+  "o3": 20,
+  "i1": 16,
+  "i2": 16,
+  "i3": 20,
+  "r0": 124,
+  "r1": 124,
+  "r2": 124,
+  "nnz": 1310720
+ },
+ "model.layers.34.self_attn.o_proj": {
+  "o1": 16,
+  "o2": 16,
+  "o3": 20,
+  "i1": 16,
+  "i2": 16,
+  "i3": 20,
+  "r0": 124,
+  "r1": 124,
+  "r2": 124,
+  "nnz": 1310720
+ },
+ "model.layers.34.mlp.gate_proj": {
+  "o1": 24,
+  "o2": 24,
+  "o3": 24,
+  "i1": 16,
+  "i2": 16,
+  "i3": 20,
+  "r0": 174,
+  "r1": 174,
+  "r2": 174,
+  "nnz": 3538944
+ },
+ "model.layers.34.mlp.up_proj": {
+  "o1": 24,
+  "o2": 24,
+  "o3": 24,
+  "i1": 16,
+  "i2": 16,
+  "i3": 20,
+  "r0": 174,
+  "r1": 174,
+  "r2": 174,
+  "nnz": 3538944
+ },
+ "model.layers.34.mlp.down_proj": {
+  "o1": 16,
+  "o2": 16,
+  "o3": 20,
+  "i1": 24,
+  "i2": 24,
+  "i3": 24,
+  "r0": 174,
+  "r1": 174,
+  "r2": 174,
+  "nnz": 3538944
+ },
+ "model.layers.35.self_attn.q_proj": {
+  "o1": 16,
+  "o2": 16,
+  "o3": 20,
+  "i1": 16,
+  "i2": 16,
+  "i3": 20,
+  "r0": 124,
+  "r1": 124,
+  "r2": 124,
+  "nnz": 1310720
+ },
+ "model.layers.35.self_attn.k_proj": {
+  "o1": 16,
+  "o2": 16,
+  "o3": 20,
+  "i1": 16,
+  "i2": 16,
+  "i3": 20,
+  "r0": 124,
+  "r1": 124,
+  "r2": 124,
+  "nnz": 1310720
+ },
+ "model.layers.35.self_attn.v_proj": {
+  "o1": 16,
+  "o2": 16,
+  "o3": 20,
+  "i1": 16,
+  "i2": 16,
+  "i3": 20,
+  "r0": 124,
+  "r1": 124,
+  "r2": 124,
+  "nnz": 1310720
+ },
+ "model.layers.35.self_attn.o_proj": {
+  "o1": 16,
+  "o2": 16,
+  "o3": 20,
+  "i1": 16,
+  "i2": 16,
+  "i3": 20,
+  "r0": 124,
+  "r1": 124,
+  "r2": 124,
+  "nnz": 1310720
+ },
+ "model.layers.35.mlp.gate_proj": {
+  "o1": 24,
+  "o2": 24,
+  "o3": 24,
+  "i1": 16,
+  "i2": 16,
+  "i3": 20,
+  "r0": 174,
+  "r1": 174,
+  "r2": 174,
+  "nnz": 3538944
+ },
+ "model.layers.35.mlp.up_proj": {
+  "o1": 24,
+  "o2": 24,
+  "o3": 24,
+  "i1": 16,
+  "i2": 16,
+  "i3": 20,
+  "r0": 174,
+  "r1": 174,
+  "r2": 174,
+  "nnz": 3538944
+ },
+ "model.layers.35.mlp.down_proj": {
+  "o1": 16,
+  "o2": 16,
+  "o3": 20,
+  "i1": 24,
+  "i2": 24,
+  "i3": 24,
+  "r0": 174,
+  "r1": 174,
+  "r2": 174,
+  "nnz": 3538944
+ },
+ "model.layers.36.self_attn.q_proj": {
+  "o1": 16,
+  "o2": 16,
+  "o3": 20,
+  "i1": 16,
+  "i2": 16,
+  "i3": 20,
+  "r0": 124,
+  "r1": 124,
+  "r2": 124,
+  "nnz": 1310720
+ },
+ "model.layers.36.self_attn.k_proj": {
+  "o1": 16,
+  "o2": 16,
+  "o3": 20,
+  "i1": 16,
+  "i2": 16,
+  "i3": 20,
+  "r0": 124,
+  "r1": 124,
+  "r2": 124,
+  "nnz": 1310720
+ },
+ "model.layers.36.self_attn.v_proj": {
+  "o1": 16,
+  "o2": 16,
+  "o3": 20,
+  "i1": 16,
+  "i2": 16,
+  "i3": 20,
+  "r0": 124,
+  "r1": 124,
+  "r2": 124,
+  "nnz": 1310720
+ },
+ "model.layers.36.self_attn.o_proj": {
+  "o1": 16,
+  "o2": 16,
+  "o3": 20,
+  "i1": 16,
+  "i2": 16,
+  "i3": 20,
+  "r0": 124,
+  "r1": 124,
+  "r2": 124,
+  "nnz": 1310720
+ },
+ "model.layers.36.mlp.gate_proj": {
+  "o1": 24,
+  "o2": 24,
+  "o3": 24,
+  "i1": 16,
+  "i2": 16,
+  "i3": 20,
+  "r0": 174,
+  "r1": 174,
+  "r2": 174,
+  "nnz": 3538944
+ },
+ "model.layers.36.mlp.up_proj": {
+  "o1": 24,
+  "o2": 24,
+  "o3": 24,
+  "i1": 16,
+  "i2": 16,
+  "i3": 20,
+  "r0": 174,
+  "r1": 174,
+  "r2": 174,
+  "nnz": 3538944
+ },
+ "model.layers.36.mlp.down_proj": {
+  "o1": 16,
+  "o2": 16,
+  "o3": 20,
+  "i1": 24,
+  "i2": 24,
+  "i3": 24,
+  "r0": 174,
+  "r1": 174,
+  "r2": 174,
+  "nnz": 3538944
+ },
+ "model.layers.37.self_attn.q_proj": {
+  "o1": 16,
+  "o2": 16,
+  "o3": 20,
+  "i1": 16,
+  "i2": 16,
+  "i3": 20,
+  "r0": 124,
+  "r1": 124,
+  "r2": 124,
+  "nnz": 1310720
+ },
+ "model.layers.37.self_attn.k_proj": {
+  "o1": 16,
+  "o2": 16,
+  "o3": 20,
+  "i1": 16,
+  "i2": 16,
+  "i3": 20,
+  "r0": 124,
+  "r1": 124,
+  "r2": 124,
+  "nnz": 1310720
+ },
+ "model.layers.37.self_attn.v_proj": {
+  "o1": 16,
+  "o2": 16,
+  "o3": 20,
+  "i1": 16,
+  "i2": 16,
+  "i3": 20,
+  "r0": 124,
+  "r1": 124,
+  "r2": 124,
+  "nnz": 1310720
+ },
+ "model.layers.37.self_attn.o_proj": {
+  "o1": 16,
+  "o2": 16,
+  "o3": 20,
+  "i1": 16,
+  "i2": 16,
+  "i3": 20,
+  "r0": 124,
+  "r1": 124,
+  "r2": 124,
+  "nnz": 1310720
+ },
+ "model.layers.37.mlp.gate_proj": {
+  "o1": 24,
+  "o2": 24,
+  "o3": 24,
+  "i1": 16,
+  "i2": 16,
+  "i3": 20,
+  "r0": 174,
+  "r1": 174,
+  "r2": 174,
+  "nnz": 3538944
+ },
+ "model.layers.37.mlp.up_proj": {
+  "o1": 24,
+  "o2": 24,
+  "o3": 24,
+  "i1": 16,
+  "i2": 16,
+  "i3": 20,
+  "r0": 174,
+  "r1": 174,
+  "r2": 174,
+  "nnz": 3538944
+ },
+ "model.layers.37.mlp.down_proj": {
+  "o1": 16,
+  "o2": 16,
+  "o3": 20,
+  "i1": 24,
+  "i2": 24,
+  "i3": 24,
+  "r0": 174,
+  "r1": 174,
+  "r2": 174,
+  "nnz": 3538944
+ },
+ "model.layers.38.self_attn.q_proj": {
+  "o1": 16,
+  "o2": 16,
+  "o3": 20,
+  "i1": 16,
+  "i2": 16,
+  "i3": 20,
+  "r0": 124,
+  "r1": 124,
+  "r2": 124,
+  "nnz": 1310720
+ },
+ "model.layers.38.self_attn.k_proj": {
+  "o1": 16,
+  "o2": 16,
+  "o3": 20,
+  "i1": 16,
+  "i2": 16,
+  "i3": 20,
+  "r0": 124,
+  "r1": 124,
+  "r2": 124,
+  "nnz": 1310720
+ },
+ "model.layers.38.self_attn.v_proj": {
+  "o1": 16,
+  "o2": 16,
+  "o3": 20,
+  "i1": 16,
+  "i2": 16,
+  "i3": 20,
+  "r0": 124,
+  "r1": 124,
+  "r2": 124,
+  "nnz": 1310720
+ },
+ "model.layers.38.self_attn.o_proj": {
+  "o1": 16,
+  "o2": 16,
+  "o3": 20,
+  "i1": 16,
+  "i2": 16,
+  "i3": 20,
+  "r0": 124,
+  "r1": 124,
+  "r2": 124,
+  "nnz": 1310720
+ },
+ "model.layers.38.mlp.gate_proj": {
+  "o1": 24,
+  "o2": 24,
+  "o3": 24,
+  "i1": 16,
+  "i2": 16,
+  "i3": 20,
+  "r0": 174,
+  "r1": 174,
+  "r2": 174,
+  "nnz": 3538944
+ },
+ "model.layers.38.mlp.up_proj": {
+  "o1": 24,
+  "o2": 24,
+  "o3": 24,
+  "i1": 16,
+  "i2": 16,
+  "i3": 20,
+  "r0": 174,
+  "r1": 174,
+  "r2": 174,
+  "nnz": 3538944
+ },
+ "model.layers.38.mlp.down_proj": {
+  "o1": 16,
+  "o2": 16,
+  "o3": 20,
+  "i1": 24,
+  "i2": 24,
+  "i3": 24,
+  "r0": 174,
+  "r1": 174,
+  "r2": 174,
+  "nnz": 3538944
+ },
+ "model.layers.39.self_attn.q_proj": {
+  "o1": 16,
+  "o2": 16,
+  "o3": 20,
+  "i1": 16,
+  "i2": 16,
+  "i3": 20,
+  "r0": 124,
+  "r1": 124,
+  "r2": 124,
+  "nnz": 1310720
+ },
+ "model.layers.39.self_attn.k_proj": {
+  "o1": 16,
+  "o2": 16,
+  "o3": 20,
+  "i1": 16,
+  "i2": 16,
+  "i3": 20,
+  "r0": 124,
+  "r1": 124,
+  "r2": 124,
+  "nnz": 1310720
+ },
+ "model.layers.39.self_attn.v_proj": {
+  "o1": 16,
+  "o2": 16,
+  "o3": 20,
+  "i1": 16,
+  "i2": 16,
+  "i3": 20,
+  "r0": 124,
+  "r1": 124,
+  "r2": 124,
+  "nnz": 1310720
+ },
+ "model.layers.39.self_attn.o_proj": {
+  "o1": 16,
+  "o2": 16,
+  "o3": 20,
+  "i1": 16,
+  "i2": 16,
+  "i3": 20,
+  "r0": 124,
+  "r1": 124,
+  "r2": 124,
+  "nnz": 1310720
+ },
+ "model.layers.39.mlp.gate_proj": {
+  "o1": 24,
+  "o2": 24,
+  "o3": 24,
+  "i1": 16,
+  "i2": 16,
+  "i3": 20,
+  "r0": 174,
+  "r1": 174,
+  "r2": 174,
+  "nnz": 3538944
+ },
+ "model.layers.39.mlp.up_proj": {
+  "o1": 24,
+  "o2": 24,
+  "o3": 24,
+  "i1": 16,
+  "i2": 16,
+  "i3": 20,
+  "r0": 174,
+  "r1": 174,
+  "r2": 174,
+  "nnz": 3538944
+ },
+ "model.layers.39.mlp.down_proj": {
+  "o1": 16,
+  "o2": 16,
+  "o3": 20,
+  "i1": 24,
+  "i2": 24,
+  "i3": 24,
+  "r0": 174,
+  "r1": 174,
+  "r2": 174,
+  "nnz": 3538944
+ }
+}

tr3/unheal/llama2-13b/state_dict_fp16.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ac278282a318af346fd66b2a1e6f9a85893628b2c81ec0beefb0891da16699a2
+size 20556180503