Upload asvd_llm/heal/llama2-13b-base/README.md with huggingface_hub

asvd_llm/heal/llama2-13b-base/README.md

@@ -0,0 +1,57 @@
+---
+license: llama2
+base_model: NousResearch/Llama-2-13b-hf
+tags:
+  - asvd
+  - low-rank
+  - model-compression
+  - llama2
+---
+
+# ASVD: LLaMA-2-13B base @ 7.75B, KD-healed
+
+ASVD (official-style: activation-aware SVD with abs_mean scaling, ppl-based sensitivity
+allocation, alpha=0.5, seed=233, 256 calibration samples, ratio target 0.5897).
+
+- **7.747B params** (7,746,790,400 = factored 6,994,022,400 + untouched 752,768,000;
+  down from 13.02B, ~40.5% reduction)
+- Healed with pure forward-KL distillation from the dense 13B base teacher: **60M tokens**,
+  held-out KL **3.9041 -> 0.1624**, probe MMLU (limit 40/subtask) **25.22% -> 43.99%**
+- ⚠️ **Probe-only evaluation**: full-benchmark evaluation (full MMLU / HellaSwag / BoolQ /
+  TriviaQA / GSM8K) was intentionally NOT run for the base-model series. All numbers below
+  come from the same probe harness (lm-eval 0.4.12, MMLU limit=40 questions/subtask,
+  n=2280, seed 1234, fp16) and are mutually comparable.
+
+| Reference (same probe harness) | probe MMLU |
+|---|---:|
+| unhealed 7.75B (same structure, unhealed sibling dir) | 25.22% |
+| **this checkpoint (healed 60M)** | **43.99%** |
+| SVD-LLM V1 base 13B (same budget + same healing recipe) | 43.33% |
+| SVD-LLM V2 base 13B (same budget + same healing recipe) | 45.48% |
+| FP16 dense 13B base teacher | 54.25% |
+
+Note: the LLaMA-2-7B-base ASVD sibling (`asvd_llm/*/llama2-7b-base/`) is archived as a
+negative result (capability not restored by healing); this 13B instance healed normally.
+
+## How to load
+
+```python
+import json
+from lowrank_linear import build, materialize_all
+
+model = build("NousResearch/Llama-2-13b-hf",
+              json.load(open("ranks.json")), "state_dict_fp16.pt",
+              device="cuda")
+# for generation/eval: fuse the factors first (inference-only)
+materialize_all(model)
+```
+
+`ranks.json` maps each compressed projection path to its integer rank; untouched weights are
+stock LLaMA-2-13B base tensors. The full healing-provenance record (data mixture & revisions,
+decontamination, SHA256 of inputs/checkpoint, 5M milestone trajectory) is in `provenance.json`.
+
+## License
+
+Derived from Meta's LLaMA 2 (`NousResearch/Llama-2-13b-hf`, an unofficial redistribution of
+the same weights). Subject to the **LLaMA 2 Community License** — see
+https://ai.meta.com/llama/license/.