snf_experiments / method_v2 /export_report.md
CharlesZhang-USTC's picture
Add Control v2 incremental experiment results (evidence, visuals, 42 fixed samples)
a64600b verified
|
Raw History Blame Contribute Delete
1.97 kB
# Control v2 export
Source: /jizhi/jizhi2/worker/trainer/snf/runtime/outputs/method-v2
Code actually run: /jizhi/jizhi2/worker/trainer/snf/snf-method-v2 (includes the numpy legacy-alias env fix in suite/momask_loading.py)
## Phases (gate: planned == completed, failed == 0)
{
"smoke": {
"completed": 63,
"curves": 21,
"gallery": "/jizhi/jizhi2/worker/trainer/snf/runtime/outputs/method-v2/smoke/summary/index.html",
"worker_failures": [],
"planned": 63,
"incomplete": 0
},
"dev": {
"completed": 2772,
"curves": 84,
"gallery": "/jizhi/jizhi2/worker/trainer/snf/runtime/outputs/method-v2/dev/summary/index.html",
"worker_failures": [],
"planned": 2772,
"incomplete": 0
},
"mechanism": {
"completed": 216,
"curves": 72,
"gallery": "/jizhi/jizhi2/worker/trainer/snf/runtime/outputs/method-v2/mechanism/summary/index.html",
"worker_failures": [],
"planned": 216,
"incomplete": 0
},
"test": {
"completed": 15012,
"curves": 2268,
"gallery": "/jizhi/jizhi2/worker/trainer/snf/runtime/outputs/method-v2/test/summary/index.html",
"worker_failures": [],
"planned": 15012,
"incomplete": 0
}
}
## Contents
- evidence/: numeric evidence for all phases, calibration, test-analysis, test-plan, plans,
actual running code (suite/packaging/docs), locks (weights, env freeze, nvidia-smi, tasks sha)
- paper-visuals/: fixed qualitative figures produced by the pipeline
- samples/: 42 pre-fixed samples (first test task per model, seed=3, requested effect 0.5,
raw+mapped x 7 methods). Selection ignores outcomes.
- selected_jobs.csv, missing_files.csv, SHA256SUMS.txt
## Caveats
- calibration_evaluation.json denominator is complete_locked_manifest: unreachable slots are kept
in the denominator and were not inferred.
- interpretation field: requested-effect evaluation; does not establish intrinsic generation
capacity or causal mechanism.
- MoMask did not improve after calibration; retained as a negative result.