Method Guides
Each guide presents the API and parameter defaults first, followed by a complete
example: imports, model loading, input/prompt preparation, calibration or target
construction when needed, method execution, result inspection and visualization.
Examples can be copied without first running a hidden shared setup cell.
Availability depends on the model adapter;
check probe.describe() or the model matrix.
Residual and head lenses support language-model and VLM representations;
EmbedLens uses the input-embedding space; its visual-analysis mode needs projected
visual tokens, while tokens="all" also permits text-only inputs. Spatial attention maps
can analyze vision classifiers or decoder visual tokens when the adapter
exposes the required sites. Original EAP-IG currently uses a text-only GPT-2
graph. These modality boundaries are documented on each method page.
Lens
| Method | Public factory | What it does |
|---|---|---|
| Logit Lens | probe.lens.logit(...) |
Decode intermediate residuals with the model's final normalization and vocabulary head. |
| Tuned Lens | probe.lens.tuned(...) |
Fit one affine translator per residual layer before applying the model's frozen readout. |
| Attention Lens | probe.lens.attention(...) |
Train separate vocabulary decoders for individual attention heads. |
| Jacobian Lens | probe.lens.jacobian(...) |
Estimate a layer-to-final-residual Jacobian, transport representations, and decode them. |
| MJ-Lens | probe.lens.mj(...) |
Separate source modalities and output channels, with optional discrete visual-codebook readout and controlled modality index. |
| EmbedLens | probe.lens.embed(...) |
Find vocabulary embeddings nearest to projected visual tokens. |
| Patchscope | probe.lens.patchscope(...) |
Insert a source representation into a target prompt and inspect its output logits. |
Attention
| Method | Public factory | What it does |
|---|---|---|
| Attention Profile | probe.attention.profile(...) |
Measure attention entropy, concentration, and mass assigned to token groups. |
| Head Logit Attribution | probe.attention.head_logits(...) |
Project head contributions to vocabulary logits with a fixed final normalization scale. |
| Attention Rollout | probe.attention.rollout(...) |
Compose normalized self-attention transitions across consecutive layers. |
| Attention Relevance | probe.attention.relevance(...) |
Propagate positive gradient-weighted self-attention for a selected output score. |
| Attention Reweighting | probe.attention.reweight(...) |
Rescale attention on selected key tokens and measure the output change. |
| Attention Temperature | probe.attention.temperature(...) |
Change the sharpness of attention at selected query rows. |
| Attention Grad-CAM | probe.attention.grad_cam(...) |
Target-conditioned CAM using attention heads as channels. |
| Self-Attention Attribution | probe.attention.attribution(...) |
Integrate gradients along a single layer's zero-to-original probability path. |
| TAM | probe.attention.tam(...) |
Combine backward Markov transitions with input-path final-attention feedback. |
| Beyond Intuition | probe.attention.beyond_intuition(...) |
Headwise or tokenwise perception with integrated reasoning feedback. |
| Chefer DTD/LRP | probe.attention.dtd_lrp(...) |
Copy native ViT weights into the pinned author's actual relevance-propagation backend. |
Profile, head logit attribution, reweighting, and temperature are reusable statistics/readout/intervention tools. They do not reproduce VAR, OPERA, or PAI. The source index identifies the paper-backed attribution algorithms and their implemented scope.
Causal
| Method | Public factory | What it does |
|---|---|---|
| Activation Patching | probe.causal.patch(...) |
Replace selected receiver residuals with source residuals and measure the causal effect. |
| Path Patching | probe.causal.path(...) |
Freeze other head outputs, transfer sender outputs, then replay only recomputed receiver Q/K/V or final residual. |
| Attribution Patching | probe.causal.attribute(...) |
Approximate a patch effect using activation differences and a receiver-side gradient. |
| Attention Knockout | probe.causal.knockout(...) |
Block selected query-to-key attention paths before softmax. |
| EAP / EAP-IG | probe.causal.eap_ig(graph="transformer", ...) |
Attribute independently replaceable GPT-2 edges along an input-embedding path, then evaluate retained circuits with actual forwards. |
| Residual Steering | probe.causal.steer(...) |
Add a supplied direction to selected residuals and measure the output change. |
| Visual Steering Vector (VSV) | probe.causal.vsv(positive, negative, ...) |
Build per-image residual directions from image and text-only inputs; configure a joint steering run. |
Sparse Autoencoders (SAE)
Learn sparse feature dictionaries from frozen-model activations, inspect their activity, and measure feature interventions with reconstruction controls.
| Method | Public factory | What it does |
|---|---|---|
| ReLU | probe.sae.relu(...) |
Nonnegative features with L1 sparsity. |
| TopK | probe.sae.topk(...) |
Per-vector positive TopK selection. |
| Gated | probe.sae.gated(...) |
Separate feature selection from activation magnitude. |
| JumpReLU | probe.sae.jumprelu(...) |
Learn feature thresholds with a surrogate-gradient sparsity objective. |
| AbsTopK | probe.sae.abstopk(...) |
Keep the largest absolute activations with their signs. |
| BatchTopK | probe.sae.batchtopk(...) |
Allocate a sparsity budget across a training minibatch. |
| Matryoshka BatchTopK | probe.sae.matryoshka_batchtopk(...) |
Train nested dictionary prefixes with shared sparse codes. |
| Archetypal TopK | probe.sae.archetypal_topk(...) |
Constrain TopK dictionary atoms with training-data anchors. |
| Archetypal JumpReLU | probe.sae.archetypal_jumprelu(...) |
Combine learned thresholds and an archetypal dictionary. |
| Matching Pursuit | probe.sae.matching_pursuit(...) |
Greedily encode residuals with nonnegative coefficients. |
| Signed MP | probe.sae.signed_matching_pursuit(...) |
Greedily encode residuals with signed coefficients. |
| Nonnegative OMP | probe.sae.orthogonal_matching_pursuit(...) |
Refit nonnegative coefficients jointly on selected support. |
| Signed OMP | probe.sae.signed_orthogonal_matching_pursuit(...) |
Refit signed coefficients by least squares. |
| QSAE | probe.sae.qsae(...) |
Use learned scalar quantization levels; an experimental recipe. |