Spaces:
Sleeping
Sleeping
Commit History
-
Namespace all outputs under experiment_dir, add MLflow run metadata
All pipeline outputs and sentinels now live under {output_dir}/{mlflow_experiment}/ instead of scattered across the project root and output_dir. Prevents clobbering across experiments and mirrors the MLflow hierarchy. Changes: - Snakefile: derive config["experiment_dir"], pre-create it, override eval.out dynamically; update all sentinel expand() paths - rules/*: sentinel touch paths and inter-rule deps use experiment_dir; correlations/figures dirs moved under experiment_dir - config.yaml: remove hardcoded eval.out (now derived in Snakefile) - scripts/run_*.py: add mlflow.note.content description, model tag, and DatasetInput (local source → output path) to every run Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>0bf8fb1
-
Add docstrings to all static rules for snakemake --list
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>fbffedd -
Add drop_columns option to transform stage for refined dataset reduction
- reprocess_metrics(): drop_columns param strips named columns before saving the refined dataset; primary is never touched so columns are always recoverable - run_transform.py: --drop-columns passthrough, logged to MLflow - transform.smk: reads transform.drop_columns from config, builds --drop-columns flag - config.yaml: transform.drop_columns: [] with note on common candidates (P_w, w_bins) Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>2f68beb -
Write transform output to separate _refined directory
- reprocess_metrics(): rename out_dir→in_dir, add out_dir for refined output. Defaults to {dataset_path}_refined. Copies metadata.json into refined dir. - run_transform.py: add --out-dir arg (default {dataset-dir}_refined); logs out_dir to MLflow. - transform.smk: passes --out-dir {output_dir}/{run_key}_refined explicitly. - dashboard_utils.py: prefer {ds_name}_refined path when loading locally, fall back to {ds_name}. - run_merge.py: cross_model_merge prefers {dir}_refined over primary dir. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>aaaf91c -
feat: add Snakemake pipeline, MLflow tracking, metrics refactor, tests
Pipeline / workflow: - Snakefile + config.yaml orchestrate primary → transform → merge stages - rules/primary.smk, transform.smk, correlations.smk, merge.smk - Sentinel-file pattern (done/*.done) avoids directory-output conflicts - Incremental cross-model merge: appends new models without clobbering; --refresh flag for explicit replacement after re-transform - Per-model checkpoint merge rule for Pythia step-collapse CLI wrappers (scripts/): - run_primary.py — process_model() + MLflow logging - run_transform.py — reprocess_metrics() + MLflow logging (git sha tagged) - run_correlations.py — thin wrapper for run_multi_circuit_analysis() - run_merge.py — cross-model incremental merge + checkpoint merge Metrics refactor: - src/transformer_analysis/metrics.py: canonical home for all stat functions, bin configs, normality_metrics, singular_value_metrics, and adaptive binning helpers (make_weight_bins, make_sv_bins) - histogram_utils.py: backward-compat re-export shim; no logic c…24e1932