|
Download examples/README.md from Anonymous1611/Lattice_Spike: direct link, hf CLI and curl.
- Browser
- Download file 970 Bytes
-
https://huggingface.co/Anonymous1611/Lattice_Spike/resolve/main/examples/README.md
- Command line
-
hf download hf://Anonymous1611/Lattice_Spike/examples/README.md
-
curl -L -o README.md https://huggingface.co/Anonymous1611/Lattice_Spike/resolve/main/examples/README.md
970 Bytes
| # Tiny Token Evaluation Demo | |
| `tiny_token_eval.jsonl` is a tiny fake evaluation set for smoke-testing the | |
| token-level baseline harness. It is intentionally small and not a benchmark. | |
| Each row has: | |
| - `prompt`: the question/context. | |
| - `answer`: the generated answer to score. | |
| - `hallucinated_spans`: answer substrings that should be labeled hallucinated. | |
| - `metadata`: notes explaining the toy case. | |
| The DGX evaluator converts `hallucinated_spans` into tokenizer-aligned `0/1` | |
| labels at runtime. That avoids hard-coding labels that only work for one | |
| tokenizer. | |
| Run it with: | |
| ```bash | |
| python scripts/eval_tiny_baselines_dgx.py \ | |
| --model Qwen/Qwen2.5-0.5B-Instruct \ | |
| --detector epr \ | |
| --detector lookback \ | |
| --device cuda \ | |
| --dtype bfloat16 | |
| ``` | |
| The wrapper writes token-level predictions to: | |
| ```text | |
| outputs/tiny_token_eval_predictions.jsonl | |
| ``` | |
| Use these numbers only as an end-to-end sanity check. Formal detector quality | |
| still needs a real labeled dataset. | |