Upload svd_llm/heal/llama2-13b-base/README.md with huggingface_hub

svd_llm/heal/llama2-13b-base/README.md

@@ -0,0 +1,53 @@
+---
+license: llama2
+base_model: NousResearch/Llama-2-13b-hf
+tags:
+  - svd
+  - low-rank
+  - model-compression
+  - llama2
+---
+
+# SVD-LLM (ICLR 2025): LLaMA-2-13B base @ 7.74B, KD-healed
+
+SVD-LLM: truncation-aware data whitening (256 wikitext2 samples) + closed-form parameter update, official code ([AIoT-MLSys-Lab/SVD-LLM @ 7538cca](https://github.com/AIoT-MLSys-Lab/SVD-LLM)), uniform ~41.6% removal across all 7 projection types of all 40 layers.
+
+- **7.738B params** (7,737,799,680; down from 13.02B, ~40.6% reduction)
+- Healed with pure forward-KL distillation from the dense 13B base teacher: **60M tokens**,
+  held-out KL **1.6616 -> 0.1912**, probe MMLU (limit 40/subtask) **25.75% -> 43.33%**
+- ⚠️ **Probe-only evaluation**: full-benchmark evaluation (full MMLU / HellaSwag / BoolQ /
+  TriviaQA / GSM8K) was intentionally NOT run for the base-model series. All numbers below
+  come from the same probe harness (lm-eval 0.4.12, MMLU limit=40 questions/subtask,
+  n=2280, seed 1234, fp16) and are mutually comparable.
+
+| Reference (same probe harness) | probe MMLU |
+|---|---:|
+| unhealed 7.74B (same structure, unhealed sibling dir) | 25.75% |
+| **this checkpoint (healed 60M)** | **43.33%** |
+| FP16 dense 13B base teacher | 54.25% |
+
+(SVD-LLM V2 / TR-3 base 13B siblings are still being produced; this table will gain those
+rows when they land.)
+
+## How to load
+
+```python
+import json
+from lowrank_linear import build, materialize_all
+
+model = build("NousResearch/Llama-2-13b-hf",
+              json.load(open("ranks.json")), "state_dict_fp16.pt",
+              device="cuda")
+# for generation/eval: fuse the factors first (inference-only)
+materialize_all(model)
+```
+
+`ranks.json` maps each compressed projection path to its integer rank; untouched weights are
+stock LLaMA-2-13B base tensors. The full healing-provenance record (data mixture & revisions,
+decontamination, SHA256 of inputs/checkpoint, 5M milestone trajectory) is in `provenance.json`.
+
+## License
+
+Derived from Meta's LLaMA 2 (`NousResearch/Llama-2-13b-hf`, an unofficial redistribution of
+the same weights). Subject to the **LLaMA 2 Community License** — see
+https://ai.meta.com/llama/license/.