| # Artifact Backup Index — where every experiment's outputs live |
|
|
| **One-glance index of what is backed up, from which experiment, and where.** |
| Chronological transfer log: [`artifact_backup_log.md`](artifact_backup_log.md). |
| HF private repos: **dataset** `cycloevan/ransom-ml`, **model** `cycloevan/ransom-ml-models`. |
| Rule of thumb: model weights (`*.pt`, `*.safetensors`) + tokenizers → **model** repo; |
| metrics / predictions / reports / manifests → **dataset** repo. Raw PE is NEVER backed up (§0). |
|
|
| To fetch a tagged snapshot: |
| `huggingface_hub.HfApi().list_repo_files("<repo>", repo_type="<model|dataset>", revision="<tag>")`. |
|
|
| --- |
|
|
| ## Backup units (each = one experiment stage → its artifacts) |
|
|
| ### 1. Phase B — feature-representation selection · tag `phase-b-feature-select` · 2026-07-23 |
| **Experiment.** Validation-only selection of the ASM feature representation for the whole |
| ladder: **normalized opcode+operand vs raw opcode-only**, function filter FIXED at `[5,250]` |
| (not a variable). Two sub-stages: (a) *feature-pretrain* — one DistilBERT MLM per variant |
| (6L/768/12H, max_len 256, 3-epoch MLM on the **official-TRAIN 2,123** corpus, official TEST |
| excluded; vocab normalized=2,007 / opcode=978); (b) *feature-select* — M1 (`--mode asm_only`) |
| supervised, **2 variants × 3 model seeds {42,1337,2024} = 6 runs**, split seed 42, uncapped |
| functions, 5 epochs, `--no-test` (Test-509 never opened; metrics are Validation). |
| **Result / gate.** Winner = **normalized_5_250** (Valid F1 0.9569±0.0028 vs opcode 0.9530±0.0016; |
| higher AP, tighter variance) → Phase C input locked (filter_5_250 / token-field=tokens / |
| normalized_5_250 tokenizer). Full write-up: `reports/validation_feature_selection/PHASE_B_MANIFEST.md`. |
|
|
| | artifact | what it is | repo | path (revision = tag `phase-b-feature-select`) | |
| |---|---|---|---| |
| | development MLM ×2 | DistilBERT MLM weights per variant (`final/model.safetensors` + `checkpoint-45120`) + embedded tokenizer | **model** | `checkpoints/pretrain_development_variants/{normalized_5_250,opcode_5_250}/distilbert_len256/` | |
| | feature-select M1 ×6 | `best.pt` per (variant × seed) — asm_only Validation-selected weights | **model** | `reports/validation_feature_selection/{variant}/poolattention_fusegated_seed{S}_asm_only/best.pt` | |
| | custom WordPiece ×3 | tokenizers used by MLM/feature-select (+ `.meta.json`) | **model** | `tokenizer/development_variants/{normalized_5_250,opcode_5_250,normalized_1_inf}.json` | |
| | Validation metrics/config | `valid_metrics.json` (×6), `config.json`, `leaderboard.json` (×2), preprocessing audits, `PHASE_B_MANIFEST.md` | **dataset** | `reports/validation_feature_selection/` | |
|
|
| Verified present (HfApi list): model — best.pt 6/6, MLM final 2/2, tokenizers 6/6; |
| dataset — 28 files incl. 6 valid_metrics + manifest. |
| |
| --- |
| |
| ### 2. Phase C-1 — backbone × length (FULL 6/6) · tag `phase-c1` · 2026-07-31 |
| **Experiment.** Validation-only M1 (`--mode asm_only`) full cross of **{distilbert, bert, roberta} × |
| {max_len 128, 256}**, 3 model seeds {42,1337,2024}, split seed 42, uncapped, `--no-test` (Test-509 |
| never opened — runner log "screening complete; Test remained unopened"). All backbones share the |
| SAME winner feature from Phase B (`normalized_5_250`, token-field=tokens, filter [5,250], WordPiece |
| vocab 2,007). Each config = 1 development MLM (per-backbone encoder over the WordPiece vocab, |
| max_len 128/256, 3-epoch MLM on official-TRAIN 2,123) + 3 supervised M1 runs. |
| **Result / gate. Winner = bert_len256** (Valid F1 **0.9649 ± 0.0019**, recall 0.9713, tightest |
| variance). Full standings (Valid F1): bert_len256 0.9649 > roberta_len256 0.9571 > |
| distilbert_len256 0.9544 > bert_len128 0.9503 > roberta_len128 0.9502 > distilbert_len128 0.9492. |
| Clear pattern: **len256 ≫ len128** (all three len256 beat all three len128). → Phase C-2/D/E/F use |
| backbone=bert, max_len=256; **Phase F reuses the bert_len256 development MLM** (no MLM retrain). |
| (Supersedes `phase-c1-partial` = the earlier 4/6 GPU-drop recovery snapshot.) |
| |
| | artifact | what it is | repo | path (revision = tag `phase-c1`) | |
| |---|---|---|---| |
| | development MLM ×6 | per-config encoder MLM (`final/model.safetensors` + `checkpoint-45120`) over WordPiece normalized_5_250 | **model** | `checkpoints/pretrain_development_backbone_grid/{distilbert,bert,roberta}_len{128,256}/` | |
| | M1 best.pt ×18 | `best.pt` per (config × seed) — asm_only Validation-selected weights (swept, `reports/…` path preserved) | **model** | `reports/validation_backbone_length/{config}/poolattention_fusegated_seed{S}_asm_only/best.pt` | |
| | Validation metrics/config | `valid_metrics.json` (×18), `selection.json`, `leaderboard.json` (×6), `config.json`, `environment.json` | **dataset** | `reports/validation_backbone_length/` | |
|
|
| Verified present (HfApi list): model — swept best.pt 18/18, MLM safetensors 12 (6 final + 6 ckpt); |
| dataset — 6/6 leaderboards. |
|
|
| --- |
|
|
| ### 3. Phase D Block 1 — GNN × node selection screen · tag `phase-d-block1` · 2026-08-09 |
| **Experiment.** Validation-only (`--no-test`) single-seed(42) DUAL screen: `{gcn,sage,gat} × |
| {bbfe,ldp,opcode_hist} × edge=real × mode=dual` = 9 cells. Each cell trains the FULL dual model |
| (per-function BERT ASM branch + GNN graph branch, gated fusion), reusing the Phase C winner |
| bert_len256 dev MLM; only the graph-branch (GNN type × node feature) varies. Uncapped functions. |
| **Result / gate. Winner = gat × bbfe** (Valid F1 **0.9834**, AUC 0.9987). Ranking: bbfe 3-way |
| (gat 0.9834 / gcn 0.9804 / sage 0.9804) ≫ opcode_hist (0.9577–0.9603) ≥ ldp (0.9505–0.9577). |
| Mechanistic signal: within bbfe the GNN choice barely moves F1 (~0.3pp) → supports "graph |
| re-aggregates the ASM representation (redundant)". Deployed dual = gat+bbfe+real+bert_len256 → |
| Phase E freeze → F (3-seed Test). Note: one GPU-drop (NVML) mid cell-4; recovered by pod |
| Stop→Start (port 8961→8977) + resume-safe skip; single-seed is a deliberate screen (final numbers |
| come from Phase F 3-seed). |
|
|
| | artifact | what it is | repo | path (revision = tag `phase-d-block1`) | |
| |---|---|---|---| |
| | dual best.pt ×9 | `best.pt` per (GNN × node) cell — full dual (ASM+GNN) Validation-selected weights (swept) | **model** | `reports/validation_block1/{gnn}_{node}/pool…/best.pt` | |
| | Validation metrics/config | `valid_metrics.json` (×9), `selection.json`, `config.json`, **`block1_comparison.json`** (9-cell ranking) | **dataset** | `reports/validation_block1/` | |
| |
| Verified present (HfApi list): model — swept best.pt 9/9; dataset — 29 files incl. block1_comparison. |
| Local backup 2.9G verified (9/9). No new dev MLM (reused bert_len256). |
| |
| --- |
| |
| ## Earlier snapshots (superseded / historical — see log for detail) |
| | tag | repos | what | |
| |---|---|---| |
| | `protocol-lock-20260714` | model + dataset | Full processed-data archive, splits, manifest, exclusions, reports, checkpoints, models, tokenizer at the protocol lock. | |
| | `base-test509-3seed` | model | Capped (128-func) base run 3-seed Test-509 weights (§4b) — historical, capped-artifact. | |
| | `r0shuffle-gate-seed42` | dataset | Uncapped LDP ladder + R0-shuffle gate + uncapped Pulse (seed42, exploratory/selected-on-test — prior only, NOT paper-valid). | |
| | `ldp-ladder-seed42` | dataset | Uncapped content-vs-topology ladder (seed42 exploratory). | |
| |
| --- |
| |
| ## Maintenance |
| `scripts/backup_to_hf.sh` routes by leading path / extension and, since 2026-07-23, **sweeps |
| any `*.pt`/`*.safetensors` found under a dataset-routed dir to the model repo** (fixes the |
| Phase-B `best.pt` silent-drop). Re-run per stage: `bash scripts/backup_to_hf.sh <targets…>`, |
| then tag both repos with the stage id and append a row here + in `artifact_backup_log.md`. |
|
|
