poolside

/

Laguna-S-2.1

like 684

Follow

Poolside 1.13k

Running the README's DFlash command, draft acceptance is ~10% at greedy (temperature 0) — well below usable, and the result is slower than no speculation (~8 t/s vs ~20 t/s baseline).

Measurements

• draft acceptance = 0.108 (158 / 2053), mean len 2.1 — temp 0.8
• draft acceptance = 0.106 (181 / 1713), mean len 2.5 — temp 0 (greedy)

Setup

• Single RTX 4090, build from poolsideai/llama.cpp@laguna (commit 04b2b72, 07-21)
• Files = current HF HEAD of this repo:
  • laguna-s-2.1-Q4_K_M.gguf — the 07-23 "Q4_K_M: correct yarn_attn_factor to 1.0 (llama.cpp derives mscale)" re-upload
  • laguna-s-2.1-DFlash-BF16.gguf — 07-22 "Correct DFlash config (#12)"

Command — verified config-independent (same ~10% with or without KV quant, -kvu, --no-mmap, thinking mode). Below is the exact greedy run:

llama-server \
  -m  laguna-s-2.1-Q4_K_M.gguf \
  -md laguna-s-2.1-DFlash-BF16.gguf \
  --spec-type draft-dflash --spec-draft-n-max 15 \
  -fa on --jinja \
  -ngl all -ngld all -ncmoe 41 \
  -c 8192 --port 1234 --host 0.0.0.0

Load (draft parses cleanly — the ctx_other line is the benign memory-fitting warning the log itself flags as normal):

common_speculative_impl_draft_dflash: adding speculative implementation 'draft-dflash'
common_speculative_impl_draft_dflash: - n_max=15, n_min=0, p_min=0.00
common_speculative_impl_draft_dflash: - block_size=16, mask_token_id=12, n_extract=6
srv  llama_server: model loaded

Result (temperature 0, n_predict 300):

eval time =   35801.06 ms /   300 tokens (  119.34 ms per token,   8.38 tokens per second)
draft acceptance = 0.10566 (  181 accepted /  1713 generated), mean len =  2.53

Hypothesis
The DFlash draft (07-22) predates the 07-23 yarn_attn_factor correction to the main model. Since DFlash keys off the target's intermediate hidden states, a change to the target's YaRN/mscale scaling shifts exactly what the draft was aligned against. Does laguna-s-2.1-DFlash-BF16.gguf need regenerating against the corrected main model? Happy to re-test any updated draft.