This is an experimental research artifact, it does actually run but I can't verify acceptance rates because my 12gb of vram apparently doesnt cut it because kv caches and all that
But yeah it does work- this was trained on a laptop gpu which is kinda crazy ngl
if you want to try it- use ik_llama.cpp
heres the self eval:
python -m toffee xspark eval --head xspark_out/dflash2/dflash2_head.pt \
--target-model ./VeriLoop-E2 --data-dir xspark_data/harvest --anchors 512
{"name": "NVIDIA GeForce RTX 4080 Laptop GPU", "vram_gb": 11.67608642578125, "event": "gpu_detected", "level": "info", "timestamp": "2026-11-06T14:22:03.355667Z"}
{
"variant": "dflash2",
"n_blocks": 512,
"block_size": 8,
"accept_len": 1.171875,
"oracle_len": 2.068359375,
"accept@0": 0.15625,
"recall@0": 0.49609375,
"accept@1": 0.015625,
"recall@1": 0.26171875,
"accept@2": 0.0,
"recall@2": 0.13671875,
"accept@3": 0.0,
"recall@3": 0.078125,
"accept@4": 0.0,
"recall@4": 0.048828125,
"accept@5": 0.0,
"recall@5": 0.029296875,
"accept@6": 0.0,
"recall@6": 0.017578125
}
What the numbas say:
| metric | value | plain english |
|---|---|---|
recall@0 |
254/512 = 49.6% | the true next token is in your top-16 half the time β the head knows it |
accept@0 |
80/512 = 15.6% | the walk picks it 16% of the time |
| gap | 174 anchors | truth was sitting in the candidate list and the selector walked past it |
oracle_len |
2.068 | accept length if the walk always picked the truth when available β the ceiling of this head as-is |
accept_len |
1.172 | you're capturing (1.17β1)/(2.07β1) β 16% of the available headroom |
if you want to recreate this abomination use my llm toolkit
- Downloads last month
- 220
Hardware compatibility
Log In to add your hardware
32-bit