Hugging Face's logo

cycloevan
/
ransom-ml-models

Safetensors
Model card Files
xet
 Community
ransom-ml-models / ARTIFACT_BACKUP_INDEX.md
cycloevan's picture
cycloevan
index: add Phase D Block 1 (winner gat×bbfe)
d0e7b6d verified
preview code
|
Raw History Blame Contribute Delete
7.72 kB

Artifact Backup Index — where every experiment's outputs live

One-glance index of what is backed up, from which experiment, and where. Chronological transfer log: artifact_backup_log.md. HF private repos: dataset cycloevan/ransom-ml, model cycloevan/ransom-ml-models. Rule of thumb: model weights (*.pt, *.safetensors) + tokenizers → model repo; metrics / predictions / reports / manifests → dataset repo. Raw PE is NEVER backed up (§0).

To fetch a tagged snapshot: huggingface_hub.HfApi().list_repo_files("<repo>", repo_type="<model|dataset>", revision="<tag>").

Backup units (each = one experiment stage → its artifacts)

1. Phase B — feature-representation selection · tag phase-b-feature-select · 2026-07-23

Experiment. Validation-only selection of the ASM feature representation for the whole ladder: normalized opcode+operand vs raw opcode-only, function filter FIXED at [5,250] (not a variable). Two sub-stages: (a) feature-pretrain — one DistilBERT MLM per variant (6L/768/12H, max_len 256, 3-epoch MLM on the official-TRAIN 2,123 corpus, official TEST excluded; vocab normalized=2,007 / opcode=978); (b) feature-select — M1 (--mode asm_only) supervised, 2 variants × 3 model seeds {42,1337,2024} = 6 runs, split seed 42, uncapped functions, 5 epochs, --no-test (Test-509 never opened; metrics are Validation). Result / gate. Winner = normalized_5_250 (Valid F1 0.9569±0.0028 vs opcode 0.9530±0.0016; higher AP, tighter variance) → Phase C input locked (filter_5_250 / token-field=tokens / normalized_5_250 tokenizer). Full write-up: reports/validation_feature_selection/PHASE_B_MANIFEST.md.

artifact what it is repo path (revision = tag phase-b-feature-select)
development MLM ×2 DistilBERT MLM weights per variant (final/model.safetensors + checkpoint-45120) + embedded tokenizer model checkpoints/pretrain_development_variants/{normalized_5_250,opcode_5_250}/distilbert_len256/
feature-select M1 ×6 best.pt per (variant × seed) — asm_only Validation-selected weights model reports/validation_feature_selection/{variant}/poolattention_fusegated_seed{S}_asm_only/best.pt
custom WordPiece ×3 tokenizers used by MLM/feature-select (+ .meta.json) model tokenizer/development_variants/{normalized_5_250,opcode_5_250,normalized_1_inf}.json
Validation metrics/config valid_metrics.json (×6), config.json, leaderboard.json (×2), preprocessing audits, PHASE_B_MANIFEST.md dataset reports/validation_feature_selection/

Verified present (HfApi list): model — best.pt 6/6, MLM final 2/2, tokenizers 6/6; dataset — 28 files incl. 6 valid_metrics + manifest.

2. Phase C-1 — backbone × length (FULL 6/6) · tag phase-c1 · 2026-07-31

Experiment. Validation-only M1 (--mode asm_only) full cross of {distilbert, bert, roberta} × {max_len 128, 256}, 3 model seeds {42,1337,2024}, split seed 42, uncapped, --no-test (Test-509 never opened — runner log "screening complete; Test remained unopened"). All backbones share the SAME winner feature from Phase B (normalized_5_250, token-field=tokens, filter [5,250], WordPiece vocab 2,007). Each config = 1 development MLM (per-backbone encoder over the WordPiece vocab, max_len 128/256, 3-epoch MLM on official-TRAIN 2,123) + 3 supervised M1 runs. Result / gate. Winner = bert_len256 (Valid F1 0.9649 ± 0.0019, recall 0.9713, tightest variance). Full standings (Valid F1): bert_len256 0.9649 > roberta_len256 0.9571 > distilbert_len256 0.9544 > bert_len128 0.9503 > roberta_len128 0.9502 > distilbert_len128 0.9492. Clear pattern: len256 ≫ len128 (all three len256 beat all three len128). → Phase C-2/D/E/F use backbone=bert, max_len=256; Phase F reuses the bert_len256 development MLM (no MLM retrain). (Supersedes phase-c1-partial = the earlier 4/6 GPU-drop recovery snapshot.)

artifact what it is repo path (revision = tag phase-c1)
development MLM ×6 per-config encoder MLM (final/model.safetensors + checkpoint-45120) over WordPiece normalized_5_250 model checkpoints/pretrain_development_backbone_grid/{distilbert,bert,roberta}_len{128,256}/
M1 best.pt ×18 best.pt per (config × seed) — asm_only Validation-selected weights (swept, reports/… path preserved) model reports/validation_backbone_length/{config}/poolattention_fusegated_seed{S}_asm_only/best.pt
Validation metrics/config valid_metrics.json (×18), selection.json, leaderboard.json (×6), config.json, environment.json dataset reports/validation_backbone_length/

Verified present (HfApi list): model — swept best.pt 18/18, MLM safetensors 12 (6 final + 6 ckpt); dataset — 6/6 leaderboards.

3. Phase D Block 1 — GNN × node selection screen · tag phase-d-block1 · 2026-08-09

Experiment. Validation-only (--no-test) single-seed(42) DUAL screen: {gcn,sage,gat} × {bbfe,ldp,opcode_hist} × edge=real × mode=dual = 9 cells. Each cell trains the FULL dual model (per-function BERT ASM branch + GNN graph branch, gated fusion), reusing the Phase C winner bert_len256 dev MLM; only the graph-branch (GNN type × node feature) varies. Uncapped functions. Result / gate. Winner = gat × bbfe (Valid F1 0.9834, AUC 0.9987). Ranking: bbfe 3-way (gat 0.9834 / gcn 0.9804 / sage 0.9804) ≫ opcode_hist (0.9577–0.9603) ≥ ldp (0.9505–0.9577). Mechanistic signal: within bbfe the GNN choice barely moves F1 (~0.3pp) → supports "graph re-aggregates the ASM representation (redundant)". Deployed dual = gat+bbfe+real+bert_len256 → Phase E freeze → F (3-seed Test). Note: one GPU-drop (NVML) mid cell-4; recovered by pod Stop→Start (port 8961→8977) + resume-safe skip; single-seed is a deliberate screen (final numbers come from Phase F 3-seed).

artifact what it is repo path (revision = tag phase-d-block1)
dual best.pt ×9 best.pt per (GNN × node) cell — full dual (ASM+GNN) Validation-selected weights (swept) model reports/validation_block1/{gnn}_{node}/pool…/best.pt
Validation metrics/config valid_metrics.json (×9), selection.json, config.json, block1_comparison.json (9-cell ranking) dataset reports/validation_block1/

Verified present (HfApi list): model — swept best.pt 9/9; dataset — 29 files incl. block1_comparison. Local backup 2.9G verified (9/9). No new dev MLM (reused bert_len256).

Earlier snapshots (superseded / historical — see log for detail)

tag repos what
protocol-lock-20260714 model + dataset Full processed-data archive, splits, manifest, exclusions, reports, checkpoints, models, tokenizer at the protocol lock.
base-test509-3seed model Capped (128-func) base run 3-seed Test-509 weights (§4b) — historical, capped-artifact.
r0shuffle-gate-seed42 dataset Uncapped LDP ladder + R0-shuffle gate + uncapped Pulse (seed42, exploratory/selected-on-test — prior only, NOT paper-valid).
ldp-ladder-seed42 dataset Uncapped content-vs-topology ladder (seed42 exploratory).

Maintenance

scripts/backup_to_hf.sh routes by leading path / extension and, since 2026-07-23, sweeps any *.pt/*.safetensors found under a dataset-routed dir to the model repo (fixes the Phase-B best.pt silent-drop). Re-run per stage: bash scripts/backup_to_hf.sh <targets…>, then tag both repos with the stage id and append a row here + in artifact_backup_log.md.