# Strict Baseline Paper Notes

These notes summarize the detector designs used by the strict LLaMA baseline
branch.

## Sources

- EPR: "Learned Hallucination Detection in Black-Box LLMs using Token-level
  Entropy Production Rate" (`arXiv:2509.04492`), plus the Artefactual public
  implementation: https://github.com/artefactory/artefactual
- Lookback Lens: "Lookback Lens: Detecting and Mitigating Contextual
  Hallucinations in Large Language Models Using Only Attention Maps"
  (`arXiv:2407.07071`), plus the public implementation:
  https://github.com/voidism/Lookback-Lens
- Multi-View Attention: "Hallucinated Span Detection with Multi-View Attention
  Features" (`arXiv:2504.04335`, ACL Anthology `2025.starsem-1.31`), plus the
  public implementation: https://github.com/Ogamon958/mva_hal_det

## Strict Mapping

| Method | Paper/reference approach | Our implementation |
| --- | --- | --- |
| EPR | Computes top-k per-token entropy contribution `-p log2(p)` and sums rank contributions into a token score. | Teacher-forces the saved generated token IDs, extracts top-15 next-token log-probabilities, and computes the same `-exp(logp) * log2(exp(logp))` contribution in `float32`. |
| Lookback Lens | Computes each layer/head's `context_attention / (context_attention + generated_attention)`, then averages complete trailing windows before logistic regression. | Computes the same ratio from Hugging Face attention tensors, uses full trailing windows of size 8 for the strict run, max-pools hallucination-positive labels, and trains logistic regression with `max_iter=1000`. |
| Multi-View Attention | Uses three feature families per token: average incoming attention, incoming attention entropy, and outgoing attention entropy; feeds standardized features into a Transformer encoder with CRF decoding and selects hyperparameters by validation F1 search. | Extracts `key_avg`, `key_entropy`, and `query_entropy` by default; `key_avg` and `key_entropy` use the paper's adjusted incoming attention; standardizes the train split; runs a 200-trial Optuna search over the paper/public hyperparameter ranges; trains a Transformer-CRF with AdamW, ReduceLROnPlateau validation-loss scheduling, validation F1 early stopping, batch size 64, and up to 150 epochs. |

## Scope

The strict run evaluates these detector methods on our LLaMA 3.2
token-labelled dataset. It does not claim to reproduce the papers' benchmark
numbers, because those depend on the original datasets, model sizes, prompt
collections, and train/validation/test partitions.

The safe claim is: We implemented and evaluated strict detector-method paths
for EPR, Lookback Lens, and Multi-View Attention on our LLaMA 3.2
token-labelled dataset.