# Hallucination Detection Probes
This repository contains hallucination detection probes for various large language models. These probes are trained to detect factual inaccuracies in model outputs.
## Probe Types
We provide three types of probes for each model:
### 1. **Linear Probes** (`*_linear`)
Simple linear classifiers trained on model hidden states to detect hallucinations.
### 2. **LoRA Probes with KL Regularization** (`*_lora_lambda_kl_0_05`)
LoRA adapters trained with KL divergence regularization (λ=0.05) to maintain proximity to the base model while learning to detect hallucinations.
### 3. **LoRA Probes with LM Regularization** (`*_lora_lambda_lm_0_01`)
LoRA adapters trained with cross-entropy loss regularization (λ=0.01) to preserve language modeling capabilities while detecting hallucinations.
## Supported Models
- Llama 3.3 70B
- Llama 3.1 8B
- Gemma 2 9B
- Mistral Small 24B
- Qwen 2.5 7B
## Usage
For loading and using these probes, see the reference implementat…
