neuronpedia/icalens-gemma-2-2b

ICA Lens for google/gemma-2-2b, in SAELens format

Converted from sida/icalens-gemma-2-2b-pile10k at revision 28825189b207db2afd235726e8ef2e23002a2e6a. ICA Lens is by Liu & Han et al., Non-Gaussianity Reveals Interpretable Directions in Language Models.

Each folder blocks.L.hook_resid_post/ is a SAELens StandardSAE with d_sae = d_in = 2304. Layers: 0-25 (26 layers), zero-based block outputs.

• W_enc = (s * n * R).T, b_enc = 0, W_dec = (W * s / n).T, b_dec = center.
• R, W and center are the lens reading matrix, writing matrix and center.
• n is the norm of each writing column, so W_dec rows are unit norm.
• s is +1 or -1 from the profile tail_direction, so ReLU keeps the interpretable tail.
• An activation is |s * n * score| on that tail. Divide by n to get the standardized ICA score.
• icalens_components.json keeps sign, writing_norm, strength, skewness and excess_kurtosis per component.

Components are in the order of the source lens: component 0 has the largest non-Gaussianity.