Spaces:
Sleeping
Sleeping
Commit History
-
Fix W_QK computation and dashboard data loading
head_analyzer.py: restore original W_Q^T @ W_K formula giving (d_model, d_model) per head — the true bilinear form in token space. Commit 8ff6453 had changed this to W_Q @ W_K^T → (head_dim, head_dim) for memory savings, but that changed the mathematical object entirely and corrupted statistics for all models. run_merge.py: fix incremental merge to read existing metadata before rmtree/move so Pythia entries aren't dropped when adding new models; also fix atomic save via temp-path to avoid HF datasets self-overwrite error. dashboards: remove campaign concept throughout; all pages now load from OUTPUT_DIR via load_dataset_with_metadata(ds_name=...) with no campaign param. get_available_campaigns() removed. cross_model, sandbox, animations, step_evolution, weights_dashboard, singular_values all updated. config.yaml: drop 'all' from targets (target_all already composes families). Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>9c80428 -
Write transform output to separate _refined directory
- reprocess_metrics(): rename out_dir→in_dir, add out_dir for refined output. Defaults to {dataset_path}_refined. Copies metadata.json into refined dir. - run_transform.py: add --out-dir arg (default {dataset-dir}_refined); logs out_dir to MLflow. - transform.smk: passes --out-dir {output_dir}/{run_key}_refined explicitly. - dashboard_utils.py: prefer {ds_name}_refined path when loading locally, fall back to {ds_name}. - run_merge.py: cross_model_merge prefers {dir}_refined over primary dir. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>aaaf91c -
Rename src modules for head/pair symmetry; add pair_figures.smk; split eval_metrics
src/transformer_analysis/: weight_analysis → head_pipeline attn_head_analysis → head_analyzer metrics → head_metrics correlation_analysis → pair_pipeline head_correlations → pair_analyzer eval_metrics.py (new — implementation extracted from compute_perplexity.py) scripts/: plot_correlations.py → plot_pair_correlations.py plot_corr_figures.py → plot_pair_correlations_batch.py compute_perplexity.py → deleted (impl in eval_metrics; thin wrapper run_eval.py) run_eval.py (new thin wrapper) rules/: correlations.smk — switched to sentinel output (consistent with other rules) pair_figures.smk (new — regenerate pair figures from .npz without re-running correlations) eval.smk — updated to call run_eval.py Snakefile: added correlations_all, pair_figures_all convenience targets Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>a81fd74 -
feat: add Snakemake pipeline, MLflow tracking, metrics refactor, tests
Pipeline / workflow: - Snakefile + config.yaml orchestrate primary → transform → merge stages - rules/primary.smk, transform.smk, correlations.smk, merge.smk - Sentinel-file pattern (done/*.done) avoids directory-output conflicts - Incremental cross-model merge: appends new models without clobbering; --refresh flag for explicit replacement after re-transform - Per-model checkpoint merge rule for Pythia step-collapse CLI wrappers (scripts/): - run_primary.py — process_model() + MLflow logging - run_transform.py — reprocess_metrics() + MLflow logging (git sha tagged) - run_correlations.py — thin wrapper for run_multi_circuit_analysis() - run_merge.py — cross-model incremental merge + checkpoint merge Metrics refactor: - src/transformer_analysis/metrics.py: canonical home for all stat functions, bin configs, normality_metrics, singular_value_metrics, and adaptive binning helpers (make_weight_bins, make_sv_bins) - histogram_utils.py: backward-compat re-export shim; no logic c…24e1932