File size: 4,579 Bytes
49d95a5
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
# JOBS 台账

每提交一个 Slurm job 追加一行;job 完成后回填【状态】与【实际耗时】。

**主实验 Slurm 依赖链**(2026-07-23 提交;全部 sbatch,`--dependency=afterok` 硬门禁)。
_ssh 直跑的前一轮 4 个压缩 job 被上一 session teardown 杀掉(nohup 未存活),改用 sbatch 重跑(Slurm job 存活于 teardown,且提供 job-id 供 afterok 链使用)。_

| Slurm id | 依赖 | 脚本 | 作用 | 输入 | 输出 | 状态 | 耗时 |
|---|---|---|---|---|---|---|---|
| 152959 | — | compress.sbatch | **BASE all-linear r=0.7** (SVD-LLM t=0) | base calib | weights/base/all_r07 | PENDING | — |
| 152960 | — | compress.sbatch | **OURS all-linear r=0.7** (Traj-MC) | ours calib | weights/ours/all_r07 | PENDING | — |
| 152961 | — | compress.sbatch | BASE attn r=0.8 — **parked, no eval** | base calib | weights/base/attn_r08 | PENDING | — |
| 152962 | — | compress.sbatch | OURS attn r=0.8 — **parked, no eval** | ours calib | weights/ours/attn_r08 | PENDING | — |
| 152963 | afterok:152959:152960 | diff_gate.sbatch | **HARD GATE**: BASE/OURS all_r07 summary diff; fail → cancels 152966-969 | 2×summary.json | results/compress/ | PENDING | — |
| 152964 | — | eval_lmeval.sbatch | REF 5×ppl MCQ (arc_c/arc_e/piqa/wino/mmlu) | dense | results/eval/ref/ | PENDING | — |
| 152965 | — | eval_lmeval.sbatch | REF GSM8K (accelerate DP×4) | dense | results/eval/ref/ | PENDING | — |
| 152966 | afterok:152963 | eval_lmeval.sbatch | **BASE** 5×ppl MCQ | weights/base/all_r07 | results/eval/base/ | PENDING | — |
| 152967 | afterok:152963 | eval_lmeval.sbatch | **BASE** GSM8K (DP×4) | weights/base/all_r07 | results/eval/base/ | PENDING | — |
| 152968 | afterok:152963 | eval_lmeval.sbatch | **OURS** 5×ppl MCQ | weights/ours/all_r07 | results/eval/ours/ | PENDING | — |
| 152969 | afterok:152963 | eval_lmeval.sbatch | **OURS** GSM8K (DP×4) | weights/ours/all_r07 | results/eval/ours/ | PENDING | — |

后续:全部 eval 落地 → `analysis/mcnemar.py`(主指标列) + `analysis/holm.py` → results/stats/。

**GSM8K 并行说明**:官方 harness 用 accelerate 数据并行(docs[rank::world] 分发 + 原子 gather)——即 strided 分片的 harness 原生形式,单 job/单 seed/单 git-hash,合并一致性由构造保证,无需 merge_shards 的跨分片一致性门禁。

_前置 gate:REF harness 验证(piqa 74.33 / winogrande 74.33 / arc_c 43.0 @ n=300)复现官方 EVAL.md,已闭。_

---
### 2026-07-24 现状(前一链 152959-969 因压缩 4h TIMEOUT 级联取消,已重来)

| Slurm id | 依赖 | 作用 | 输入 | 输出 | 状态 |
|---|---|---|---|---|---|
| 153565 | — | BASE all-linear r=0.7 压缩 (nsamples=256, 旧慢代码) | base calib | weights/base/all_r07 | RUNNING(SVD) node3 |
| 153566 | — | OURS all-linear r=0.7 压缩 (nsamples=256) | ours calib | weights/ours/all_r07 | RUNNING(SVD) node3 |
| 153709 | — | **REF GSM8K** (accelerate DP×4) | dense | results/eval/ref/lmeval/gsm8k | PENDING |
| 153710 | — | **REF HellaSwag** (accelerate DP×4) | dense | results/eval/ref/lmeval/hellaswag | PENDING |

REF 其余 5 项(arc_c/arc_e/piqa/wino/mmlu)已 COMPLETED(152964)并复现官方 EVAL.md(MMLU 65.99≈65.9)。
待 153565/566 完成 → diff_gate → BASE/OURS 全 7 任务评测。
_压缩提速修复(GPU 常驻 XtX + 显存分箱)已进 compress.py,供后续/消融加速;153565/566 用的是旧代码但已近完成,不重跑。_

### 主表 7-task 评测批(2026-07-24,压缩 gate PASS 后提交)
gate: summary_diff base vs ours all_r07 = PASS(ratio=0.7 n_compressed=224 kept=0.6998 逐字段一致)。

| arm | gsm8k(DP×4) | hellaswag(DP×4) | 5×MCQ(1gpu) |
|---|---|---|---|
| REF | 153709 | 153710 | ✅152964(done) |
| BASE | 153738 | 153739 | 153740 |
| OURS | 153741 | 153742 | 153743 |

全部落地 → mcnemar(主指标, m=7) + holm → results/stats/。

### 多语种重做(2026-07-25):校准 en → multilingual(16-lang interleave),照 spec "不限语种"
REF 不用校准 → 结果复用,不重跑。C4-en base/ours 已归档 _c4en_archive/。
多语种 calib 已建(isolation OK),recompress base/ours all_r07 完成,diff-gate PASS(kept=0.6998)。

| arm | mcq5(1gpu) | gsm8k(1gpu) | hella(1gpu) |
|---|---|---|---|
| BASE | 154648 | 154649 | 154650 |
| OURS | 154651 | 154652 | 154653 |
| REF | ✅done(152964) | 154315(run) | 154317(run) |

single-process(避 DP 24GB prep OOM);broad partition;OOM 则改 pin genai。全落地 → McNemar+Holm(m=7)。