|
Download examples/README.md from Anonymous1611/Lattice_Spike: direct link, hf CLI and curl.
- Browser
- Download file 970 Bytes
-
https://huggingface.co/Anonymous1611/Lattice_Spike/resolve/main/examples/README.md
- Command line
-
hf download hf://Anonymous1611/Lattice_Spike/examples/README.md
-
curl -L -o README.md https://huggingface.co/Anonymous1611/Lattice_Spike/resolve/main/examples/README.md
970 Bytes
Tiny Token Evaluation Demo
tiny_token_eval.jsonl is a tiny fake evaluation set for smoke-testing the
token-level baseline harness. It is intentionally small and not a benchmark.
Each row has:
prompt: the question/context.answer: the generated answer to score.hallucinated_spans: answer substrings that should be labeled hallucinated.metadata: notes explaining the toy case.
The DGX evaluator converts hallucinated_spans into tokenizer-aligned 0/1
labels at runtime. That avoids hard-coding labels that only work for one
tokenizer.
Run it with:
python scripts/eval_tiny_baselines_dgx.py \
--model Qwen/Qwen2.5-0.5B-Instruct \
--detector epr \
--detector lookback \
--device cuda \
--dtype bfloat16
The wrapper writes token-level predictions to:
outputs/tiny_token_eval_predictions.jsonl
Use these numbers only as an end-to-end sanity check. Formal detector quality still needs a real labeled dataset.