Upload svd_llm2/heal/llama2-13b-base/README.md with huggingface_hub

svd_llm2/heal/llama2-13b-base/README.md

@@ -0,0 +1,51 @@
+---
+license: llama2
+base_model: NousResearch/Llama-2-13b-hf
+tags:
+  - svd
+  - low-rank
+  - model-compression
+  - llama2
+---
+
+# SVD-LLM V2 (NAACL 2025): LLaMA-2-13B base @ 7.74B, KD-healed
+
+SVD-LLM V2: heterogeneous compression-ratio allocation from theoretical truncation loss (Algorithm 1) + loss-optimized weight truncation (Algorithm 2). Official code was never released; this is our faithful implementation of the published algorithms (per-layer ranks in ranks.json show the allocation).
+
+- **7.740B params** (7,740,277,760; down from 13.02B, ~40.5% reduction)
+- Healed with pure forward-KL distillation from the dense 13B base teacher: **60M tokens**,
+  held-out KL **1.5569 -> 0.1788**, probe MMLU (limit 40/subtask) **26.14% -> 45.48%**
+- ⚠️ **Probe-only evaluation**: full-benchmark evaluation (full MMLU / HellaSwag / BoolQ /
+  TriviaQA / GSM8K) was intentionally NOT run for the base-model series. All numbers below
+  come from the same probe harness (lm-eval 0.4.12, MMLU limit=40 questions/subtask,
+  n=2280, seed 1234, fp16) and are mutually comparable.
+
+| Reference (same probe harness) | probe MMLU |
+|---|---:|
+| unhealed 7.74B (same structure, unhealed sibling dir) | 26.14% |
+| **this checkpoint (healed 60M)** | **45.48%** |
+| SVD-LLM V1 base 13B (same budget + same healing recipe) | 43.33% |
+| FP16 dense 13B base teacher | 54.25% |
+
+## How to load
+
+```python
+import json
+from lowrank_linear import build, materialize_all
+
+model = build("NousResearch/Llama-2-13b-hf",
+              json.load(open("ranks.json")), "state_dict_fp16.pt",
+              device="cuda")
+# for generation/eval: fuse the factors first (inference-only)
+materialize_all(model)
+```
+
+`ranks.json` maps each compressed projection path to its integer rank; untouched weights are
+stock LLaMA-2-13B base tensors. The full healing-provenance record (data mixture & revisions,
+decontamination, SHA256 of inputs/checkpoint, 5M milestone trajectory) is in `provenance.json`.
+
+## License
+
+Derived from Meta's LLaMA 2 (`NousResearch/Llama-2-13b-hf`, an unofficial redistribution of
+the same weights). Subject to the **LLaMA 2 Community License** — see
+https://ai.meta.com/llama/license/.