arxiv:2601.09173

Geometric Stability: The Missing Axis of Representations

Published on Jul 6

· Submitted by

Prashant Raju on Jan 15

Upvote

Authors:

Prashant C. Raju

Abstract

Geometric stability measures representational robustness under perturbation, offering complementary insights to similarity metrics in analyzing learned representations across diverse domains.

Generated by Qwen/Qwen2.5-Coder-32B-Instruct

Representational similarity analysis and related methods compare the internal geometries of neural networks, but they measure only alignment between spaces, leaving a blind spot -- whether a representation's structure is reliably recoverable, not merely similar. We introduce geometric stability, a distinct axis, and Shesha, a metric that quantifies it from a single representation by correlating dissimilarity matrices built from complementary random halves of the feature dimensions. Unlike CKA and Procrustes distance, Shesha is provably non-invariant to orthogonal rotations of the feature basis. This is by design: the basis is privileged for learned models, since probes, patching, and steering act on coordinates, and a rotation-invariant metric cannot see whether the targeted structure survives them. A double dissociation isolates the mechanism -- removing the top principal component collapses CKA while Shesha holds, whereas rotating a representation into its eigenbasis, which preserves the spectrum and CKA exactly, collapses Shesha. Across 2,463 encoder configurations in seven domains, the metrics are redundant under geometry-preserving transforms and anti-correlate under compression (ρ=-0.47). Across 170 vision models spanning 6 clean and 38 corruption-shifted datasets, DINOv2 ranks first or second in transferability on three of six clean datasets yet bottom-quartile in stability on five, an isolated dissociation rather than a trade-off.

View arXiv page View PDF GitHub 3 auto Add to collection

Community

pcr2120

Paper author Paper submitter Jan 15

•

edited Jan 15

DeepSeek got it half right with their mHC paper: stability matters for scaling. But they only measure stability DURING training.

What about the stability of what models LEARN?

I built Shesha to measure this - a geometric stability metric with SOTA results across AI Safety, Constitutional AI, Model selection, and CRISPR perturbation analysis.

The core insight: Most evals check external similarity (does output X match Y?). But imagine a massive library where someone reshuffled all the books. A content-based audit would say that nothing's wrong since the inventory is identical. But the library is useless since nothing can be found. That's the gap Shesha fills.

The implications are broad with SOTA results across 4 domains:

AI Safety - Shesha is the best canary in the coal mine. Shesha outperforms CKA and Procrustes on drift detection. Detects 2x more drift than CKA, triggers earlier 73% of the time, catches subtle LoRA shifts at 90% sensitivity (5% FPR) - with only 7% false alarms vs Procrustes' 44%.
Constitutional AI - Shesha provides the best steering prediction. Constitutional AI needs models you can actually steer. Most metrics ask: "Are classes separable?" Wrong question! Shesha asks: "Is that separation STABLE under perturbation?" Tested on 35-69 embedding models across 3 experiments. Shesha outperforms Fisher discriminant, silhouette score, Procrustes, and anisotropy. The correlations with intervention success are rho=0.89-0.96, and the partial correlations after controlling for separability are rho=0.67-0.76. Stability ≠ separability.
Model selection - Shesha exposes what LogME misses. The DINO Paradox - best transfer scores, worst geometric stability. Tested 94 vision models on 6 datasets: DINOv2 ranked #1 in LogME on 4/6 datasets but last or near last in stability on 5/6. SOTA transfer incurs a "geometric tax."
CRISPR perturbations - Shesha serves as a new filter for target selection. CRISPR screens find hits by magnitude, but magnitude alone can't distinguish clean lineage drivers from promiscuous regulators. Shesha adds precision. Tested on 811 perturbations, Shesha showed uniformly positive magnitude-stability correlations ranging from rho=0.746 in high-variance screens to rho=0.963 in cleaner activation settings. Notably, in discordant cases, Shesha separates KLF1 (stable, specific) from CEBPA (strong but messy) purely from geometry.

Additional validation in neuroscience: Geometric stability predicted neural-behavioral coupling (rho=0.18, p=0.005) in Neuropixels data. Centroid drift showed no relationship (rho=0.00). Stability ≠ consistency.

Try it yourself:
PyPI: pip install shesha-geometry
Tutorials: https://github.com/prashantcraju/shesha?tab=readme-ov-file#tutorials

Preprint: https://arxiv.org/abs/2601.09173
Code: https://github.com/prashantcraju/geometric-stability