| # Implementation spec: SAPLMA, HaMI, LLM-Check, ICR Probe |
|
|
| All four are **implemented**. Every architecture detail |
| below was checked against the source paper or the authors' released code, not |
| recalled. Fidelity argumentation lives in `docs/baseline_fidelity_audit.md` |
| (§4 SAPLMA, §5 HaMI); this file is the build record and the remaining work. |
|
|
| ## Decisions already made (do not relitigate) |
|
|
| 1. **Pipeline:** all three go into the baseline runner |
| (`scripts/train_faithful_baselines_dgx.py`) as new `--detector` values, under |
| the same suite/protocol contract as EPR/Lookback/MVA. NOT scored ad-hoc from |
| the LatticeSpike signal cache. Protocol consistency is what makes the |
| fidelity argument defensible, and the same hidden-state extraction pass also |
| unblocks WEPR's missing causal number. |
| 2. **HaMI keeps sequence-level supervision.** Learning token scores from |
| bag-level labels IS its contribution. This belongs in the "preserved" column |
| of the adaptation table, not "adapted". |
| 3. **What we equalise is the evaluation** — same token set, same causal horizon, |
| same metrics — never the methods themselves. |
| 4. **The default detector set is frozen.** `STRICT_DETECTORS` still resolves to |
| EPR/WEPR/Lookback/MVA-CRF, so a stored command that reproduced a published |
| run keeps reproducing it. Both new methods are opt-in via `--detector`. |
|
|
| ## 1. SAPLMA — implemented |
|
|
| `baselines/saplma/`, detector name `saplma`. |
|
|
| Azaria & Mitchell, *"The Internal State of an LLM Knows When It's Lying"*, |
| **EMNLP Findings 2023**, [aclanthology.org/2023.findings-emnlp.68](https://aclanthology.org/2023.findings-emnlp.68/), |
| arXiv:2304.13734. |
|
|
| Independently validated: PARALLAX (2026) found SAPLMA one of only **two** |
| reliable internal-state probes out of 22 evaluated (0.911 AUROC on HaluBench), |
| with MIND, HaloScope, ACT, SEPs and HalluShift all near-chance (0.49-0.62). |
|
|
| **Verified from paper section 4, quoted:** "three hidden layers with decreasing |
| numbers of hidden units (256, 128, 64), all utilizing ReLU activations. The |
| final layer is a sigmoid output. We use the Adam optimizer. We do not fine-tune |
| any of these hyper-parameters for this task. The classifier is trained for 5 |
| epochs." |
|
|
| **Correction to an earlier draft of this file:** SAPLMA does **not** average |
| activations across tokens. It reads the activations at the **last token of the |
| statement** — one 4096-vector per statement. The earlier note here said |
| "averages across all tokens" and was wrong. |
|
|
| **Layer:** the paper builds five probes (last, 28th, 24th, 20th, middle of 32) |
| and finds 20/32 best for OPT-6.7b, the **middle layer** best for LLaMA2-7b. We |
| probe the middle layer of a 16-layer model, matching the LLaMA finding. |
|
|
| **Unstated in the paper:** learning rate and batch size. The reference is Keras, |
| so its defaults (1e-3, 32) are used — the only reading consistent with the |
| paper's claim that nothing was tuned. |
|
|
| **Supervision:** `--saplma-supervision response` (default, faithful) fits one |
| row per answer — last-token activations against the response label — and |
| localises per token at scoring time, exactly as EPR is handled here. |
| `--saplma-supervision token` fits every labelled token and reports under |
| `saplma_token_supervised`, so the more generous configuration is on the table |
| rather than assumed away. |
|
|
| **NOTE:** `baselines/hidden_state_probe/` is **NOT** SAPLMA — it is logistic |
| regression on the last layer with `class_weight="balanced"`, correctly classed a |
| control in `fidelity_matrix.md`. Do not relabel it. |
|
|
| ## 2. HaMI — implemented |
|
|
| `baselines/hami/`, detector name `hami`. |
|
|
| *"Robust Hallucination Detection in LLMs via Adaptive Token Selection"*, |
| **NeurIPS 2025**, [arXiv:2504.07863](https://arxiv.org/abs/2504.07863), |
| code [github.com/mala-lab/HaMI](https://github.com/mala-lab/HaMI). |
|
|
| **Architecture (`hami.py`):** |
|
|
| ``` |
| Linear(n_features, 256) -> BatchNorm1d -> ReLU -> Dropout(0.3) -> Linear(256, 1) -> Sigmoid |
| ``` |
|
|
| **Loss (`hami.py:MIL_loss`), transcribed exactly:** |
| |
| ```python |
| k = int(length * k_ratio) + 1 # k_ratio = 0.1 |
| loss += (1. - s_topk_mean**2 + nor_max[i]**2) # index-matched bag pairs |
| smooth += torch.sum((s[:-1] - s[1:])**2) * 0.0008 # bags longer than 8 only |
| loss = loss / len(n_len_list) + smooth / len(n_len_list) |
| ``` |
| |
| The abnormal term is index-matched against the normal term, which is why the |
| reference draws equal-sized batches from two loaders. Note `1. - m**2 + n**2`, |
| not `(1 - m)**2`. |
|
|
| **Training (`train.py`, `main.py`):** Adam lr 1e-3, weight decay 5e-4; |
| `ReduceLROnPlateau(mode='min', factor=0.1, patience=20, min_lr=1e-6)` stepped on |
| the training loss; batch 128 bags per polarity with `drop_last=True`; **100 |
| optimizer steps, not 100 passes**; validation after every step, keeping the best |
| model; validation metric is bag-level AUROC where the bag score is the mean of |
| its top-k token scores (`validate.py`). |
|
|
| **Layer selection:** `main.py` loops over every layer, trains a separate model |
| for each, and records validation results per layer — the layer is a |
| validation-selected hyperparameter. We run one layer per invocation |
| (`--hidden-layer`), so a layer sweep is a sweep of runs. |
|
|
| **Two things the reference does that we cannot:** |
| - Its default `--mode se` scales hidden states by a probability from clustering |
| several sampled responses per question. Our dataset has one response per |
| prompt, so we use the reference's `ori` (unweighted) mode. HaMI is therefore |
| evaluated below its published capability, and that must be stated. |
| - Bag labels come from answer correctness (`acc`), with ambiguous `acc == 2.0` |
| bags dropped under `is_refined`. Ours: abnormal if the response contains any |
| hallucinated token, normal if none, uncertain-only responses dropped. |
|
|
| **Step budget caution.** 100 steps is tied to their dataset size. On our larger |
| training split it sees a fraction of the data. `--hami-steps` exists for this; |
| any run that raises it must say so in the results, and the reference's own |
| best-on-validation rule is what makes a longer budget defensible. |
|
|
| ## 3. LLM-Check — implemented |
|
|
| `baselines/llm_check/`, detector names `token_logprob`, |
| `llm_check_logit_entropy`, `llm_check_window_entropy`, `llm_check_hidden`, |
| `llm_check_attention`. |
|
|
| Sriramanan et al., **NeurIPS 2024**, code |
| [GaurangSriramanan/LLM_Check_Hallucination_Detection](https://github.com/GaurangSriramanan/LLM_Check_Hallucination_Detection). |
| Five training-free scores from one forward pass, all verified against |
| `common_utils.py`: |
|
|
| ```python |
| perplexity: log_softmax(logits)[t-1, tok[t]] # mean then exp |
| logit_entropy: softmax(topk(logits, 50)); (-p*log p).mean() # own row |
| window_entropy: softmax(logits); (-p*log p).mean() # own row, w=1 |
| hidden: log(svdvals(Z.T @ J @ Z + 1e-3*I)).mean() # J centres features |
| attention: sum_h log(diag(A_h)) # over the answer block |
| ``` |
|
|
| Note `.mean()` not `.sum()` in both entropies, and note that the reference reads |
| the realized-token log-probability at row `t-1` but both entropies at row `t`. |
| Extraction stores rows `prompt_len-1 .. prompt_len+T-1` so index `t` is the |
| predicting row and index `t+1` is the own row; both alignments are then served |
| from one array. |
|
|
| **Three things that are easy to get wrong:** |
|
|
| 1. **Orientation is fitted, not fixed.** The authors' notebook negates |
| `perplexity` and the attention score, negates neither entropy, and ships no |
| orientation for the hidden score. An earlier draft here hard-coded the sign |
| from the paper's prose and put both eigen scores at ~0.20 AUROC; the fitted |
| bit puts them at ~0.80. One bit per score, fitted on **train**. |
| 2. **`centered_svd_val` centres over the FEATURE axis.** The reference passes |
| `(dim, n_tokens)` and builds a `dim x dim` centering matrix, so `Sigma` comes |
| out `n_tokens x n_tokens`. Easy to transpose by mistake. |
| 3. **The attention diagonal changes meaning with the row.** Own row = the |
| token's self-attention (post-emission). Predicting row = attention to the |
| most recent prefix token (causal). Not the same quantity; say so. |
|
|
| Hidden score is pooled per token over the **expanding causal prefix** |
| (`--llm-check-hidden-window 0`, the default), which is the direct causal |
| restriction of the reference's whole-response score and adds no hyperparameter. |
|
|
| `token_logprob` is LLM-Check's per-token perplexity term. It gets its own row |
| because it is the floor every method must clear, but it is computed once, not |
| double-counted under two names. |
|
|
| ## 4. ICR Probe — implemented |
|
|
| `baselines/icr_probe/`, detector name `icr_probe`. |
|
|
| Zhang et al., **ACL 2025 Main**, code |
| [XavierZhang2002/ICR_Probe](https://github.com/XavierZhang2002/ICR_Probe) |
| (`src/icr_score.py`, `src/icr_probe.py`, `src/utils.py`, `src/config.py`). |
| An earlier draft of this file said there was no code release. There is. |
|
|
| **ICR Score, per layer `a` and scored row `r`** (`compute_icr`): |
|
|
| ```python |
| dx = H[a + 1][r] - H[a][r] # consecutive layers |
| targets = H[a][topk_idx] # the LOWER layer, easy to get wrong |
| w = (dx * targets).sum(1) / (targets.norm(1) + 1e-8) |
| ICR = js_divergence(w, attn_row[topk_idx]) # standardise -> softmax -> JSD |
| ``` |
|
|
| **Five details that a paper-only implementation gets wrong:** |
|
|
| 1. **Attention is the softmaxed probability, not the pre-softmax score.** |
| Equation 3 defines `(Q.K)/sqrt(d)`; the code uses HF `output_attentions`. |
| 2. **`top_p` silently overrides `top_k`.** `top_k = top_k if top_p is None else |
| int(top_p * N)`, and the README passes both, so `top_k=20` is dead code and |
| k is a *fraction* of the sequence. |
| 3. **Projection targets come from the lower layer**, `origin_hidden_states[a]`, |
| not the layer whose update is being measured. |
| 4. **The block mask drops response-to-prompt attention entirely**, keeping only |
| prompt-to-prompt and response-to-response. |
| 5. **The induction-head fallback fires on a count test**, not a threshold: if |
| fewer than `n_heads // 8` heads pass, the verdict is discarded and the top |
| `n_heads // 8` by skewness are taken instead. |
|
|
| **Probe** (`src/utils.py`, `src/config.py`): `L -> 128 -> 64 -> 32 -> 1`, each |
| hidden layer BatchNorm1d + LeakyReLU(0.01) + Dropout(0.3), sigmoid; Kaiming |
| uniform `a=0.01`; BCE, Adam(1e-3, wd 1e-5), ReduceLROnPlateau(min, 0.5, 5), 100 |
| epochs, batch 16, best validation loss. |
|
|
| **The featurisation is ours, unavoidably.** `_load_data` is referenced but never |
| defined in the release, and the README says the training code must be adapted. |
| Default `--icr-supervision response` mean-pools ICR over the answer's tokens and |
| fits the response label; scoring feeds each token's own layer-vector to the |
| fitted probe. `--icr-supervision token` is the token-supervised variant. |
|
|
| **Cost — the planning error worth remembering.** ICR needs consecutive-layer |
| hidden states *and* per-layer attention, which looked like it demanded a |
| 12-24 GB multi-layer cache. It does not: the score reduces to one scalar per |
| (token, layer) at extraction time, so the cached feature is `(T, L)` — about |
| **15 MB for the whole dataset**. Always check what the feature reduces to before |
| budgeting for what the computation reads. |
|
|
| **The AWS L4 signal cache cannot serve ICR**, for three independent reasons: |
| it samples layers 4/8/12/16 so no consecutive difference exists; it stores no |
| attention, only the collapsed `attn_ent` scalar; and it covers answer prediction |
| rows only, while the projection needs all N tokens. Checked before spending |
| instance time. |
|
|
| ## What was wired into the runner |
|
|
| - `DETECTOR_SAPLMA` / `DETECTOR_HAMI`, plus `HIDDEN_STATE_DETECTORS` so all |
| three hidden-state methods share one extraction path and one feature array. |
| - `--hidden-alignment {generated_token_position, generation_prediction_position}`. |
| The causal suite sets the predicting row automatically; native suites keep the |
| token's own row. Both new detectors are in `CAUSAL_CAPABLE_DETECTORS`. |
| - `hidden_sequence_training_example()` carries the whole answer's hidden states |
| plus **both** label granularities, so each detector picks the one its paper |
| uses. U rows stay in the matrix and are excluded by the detector, because both |
| methods index into the sequence (last token, adjacent-token smoothing) and |
| deleting rows would silently change what "last" and "adjacent" mean. |
| - `--saplma-supervision/-epochs/-lr/-batch-size`, |
| `--hami-steps/-batch-size/-k-ratio/-lr/-weight-decay`. |
| - Summary JSON gains a `hidden_state_detectors` block recording layer, |
| alignment, and each method's fitted-run facts (bag counts, selected step, |
| validation AUROC, training positive rate). |
|
|
| ## Cross-cutting cautions from this codebase |
|
|
| - **Never add a key to a cache spec unconditionally.** The fingerprint is a hash |
| of the whole spec, so a new key invalidates every cache built before it |
| existed. This broke the 5.1 GB LatticeSpike base cache in r6 and cost a night. |
| `hidden_alignment` is therefore added to `feature_cache_spec` **only when |
| non-default**, and there is a test asserting its absence from legacy specs. |
| - **Never bump `EXTRACTION_CODE_VERSION`** unless extraction genuinely changes |
| for existing configurations; it invalidates every cache on every box. |
| - **Note that `selected_features` is already part of the fingerprint.** Running |
| `--detector saplma hami` will not reuse a cache built for the four-detector |
| set. Extract the two hidden-state methods in their own pass — they need no |
| attention output, so it is the cheap pass, not the expensive one. |
| - **Probe before long runs.** A 5-record extraction probe caught two shape bugs |
| in r6 that would each have surfaced hours in. |
| - **Verify the training path, not just extraction.** r6's failures were both in |
| training/cache-validation, which the extraction probe did not cover. |
| `tests/test_train_faithful_baselines_dgx.py` now fits and scores both new |
| detectors end to end on synthetic rows for exactly this reason. |
| - **Compare single-seed arms against the same seed's control.** Per-seed |
| validation AP for the current best LatticeSpike config: seed 42 = 0.2785, |
| seed 43 = 0.2632, seed 44 = 0.2633; 3-seed mean 0.2683 +/- 0.0088. Comparing a |
| seed-42 arm to the mean inflates it by ~0.010 and turned three nulls into |
| apparent passes in r5. |
| |
| ## Where things stand |
| |
| - Current best documented in `ablations/BEST_MODEL.md` |
| (TEST AP 0.2741 +/- 0.0238, AUROC 0.9283 +/- 0.0040, 3 seeds). |
| - Baseline audit in `docs/baseline_fidelity_audit.md`; SAPLMA is §4, HaMI §5. |
| - Adaptation table rows for both are in `scripts/gen_results_tables.py` |
| (`tab:adaptation`). The fair results table has **no rows for them yet** — |
| add rows only once runs exist. |
| - Rounds archived in `ablations/r0`, `r1`, `r2`, `r3_5`, `r4`, `r5_stage1`, |
| `multilayer`, each with `ANALYSIS.md`. |
| - **r6b results are still unfetched and unarchived** (4 arms: lookback s42/s43, |
| head128 s43/s44) in `outputs/sweep_r6` on the 5090. |
| - WEPR still has no causal-matched number; the same hidden-state extraction pass |
| should fix that. |
| - Planned external evaluation: RAGTruth and Mu-SHROOM (team already working on |
| both). FAVA parked but `hallucination_labelling/span_align.py` already |
| supports it. |
|
|
