poolside

/

Laguna-S-2.1

like 684

Follow

Poolside 1.13k

I am testing this, I been sick on the couch and running and improving my benchmark webui on it with claude and codex.
I never tried the --dry* params you used. How did you come up with those?

They were buried in the readme...

确实是解决循环问题，至少Q4不抽筋了！

Okay I cannot tell if it got better. Reasoning is triggered more but they just updated the gguf and that works better

It got better in some area but now is quite bad in tool calling. Did you changed something there?

Okay I cannot tell if it got better. Reasoning is triggered more but they just updated the gguf and that works better

Yeah I'm going to wait for unsloth to update their quants.

It got better in some area but now is quite bad in tool calling. Did you changed something there?

Hey! if you dont mind, can you check new dry settings and yarn settings in my repo? (from readme of the template), and the template, one by one, so we know what is causing this degradation? preferably, 3 passes + avg/mean error

Not possible as I'm using the NVFP4 weights in vllm. There is no dry or yarn settings. I just took your jinja template. But I guess you are using some other quant where you have a lot of problems, which is why you need those changes. For me the NVFP4 works fine (as can be seen in the screenshots).

Not possible as I'm using the NVFP4 weights in vllm. There is no dry or yarn settings. I just took your jinja template. But I guess you are using some other quant where you have a lot of problems, which is why you need those changes. For me the NVFP4 works fine (as can be seen in the screenshots).

Yeah, i use apex-i-compact which weights -20gb from nvfp4
0.00.559.878 I llama_model_loader: - type f32: 287 tensors
0.00.559.879 I llama_model_loader: - type q3_K: 114 tensors
0.00.559.879 I llama_model_loader: - type q4_K: 267 tensors
0.00.559.879 I llama_model_loader: - type q6_K: 146 tensors
0.00.559.880 I print_info: file format = GGUF V3 (latest)
0.00.559.881 I print_info: file type = Q6_K
0.00.559.883 I print_info: file size = 50.63 GiB (3.70 BPW)

For me all of those settings in combination fix insane overthinking per turn, instead of going "but wait../Actually" for 10k+ tokens, it just works, thinks more as turns get more complicated, less if its just a simple follow up toolcall.
i mean, it does also work with overthinking, but at 10t/s tg and 500pp its not what i would call a snappy experience.
what did hurt your tool calling is probably "optimizations" for claude code/harness, not sure what exactly though. ive been using this for last 2 days vibecoding and never saw a failed toolcall in CC. all edits\searches\mcp\agents calls are spot on, it even loads right skills by itself, which qwen 3.6 35b a3b never did (at nvfp4-fast by unsloth or apex-i-balanced)

anyways, if it works, dont break it

I tested the new q4km without any params it loops, with the recommended:
Temp: 0.7
TopK: 20
TopP: 0.95
MinP: 0
It is pretty bad but doesn’t loop.
But it makes mistakes left and right or refuses to do much of my prompts at all.

Temp: 0.3 made it more focused but worse.
Temp: 0.6 was better

But funnily TopK does not make much of a difference for me.

I will experiment with minP and topP.
But it takes a long time to test. At 15-20tps it takes 12h per Variation to test for me

I tested the new q4km without any params it loops, with the recommended:
Temp: 0.7
TopK: 20
TopP: 0.95
MinP: 0
It is pretty bad but doesn’t loop.
But it makes mistakes left and right or refuses to do much of my prompts at all.

Temp: 0.3 made it more focused but worse.
Temp: 0.6 was better

But funnily TopK does not make much of a difference for me.

I will experiment with minP and topP.
But it takes a long time to test. At 15-20tps it takes 12h per Variation to test for me

if you are using llama.cpp, try to add dry-sampling. also "refuses to do much of my prompts at all." for me was a "non-moe-aware" quant (IQ3_M in my case) issue. Apex-i-compact steers very well.

Left it overnight to cook a plan with /goal, over 1000 toolcalls, 10+ agents, no issues at all, and kimi k3 rated plan as "good to implement"

That solves something mysterious I found.
With iq4nl from unsloth it performed the best, but only sometimes.
Like 10 being shit and then my sims neighborhood prompt, it outperformed qwen, gemma and deepseekv4 flash.
But also only once and I am comfused.
Q4km never achieved that, but appears to be more stable.
Maybe its the params…
Also iq4nl seems to perform best with kv cache quant q8 instead of f16 or bf16.
The others I tested fail at q8:
Unsloth ud Mxfp4, unsloth ud q4kxl.
Apex one from Myril I think was their name was the fastest but worse, although stable throughout all kv quant sizes.

I will try the dry sampling thing with the no param and the recommended temp topk etc.

Thank you

there is also a matter of this as far as KV cache issues go. apparently, you gotta set yarn-attn-factor = 1.0 instead of stock 1.43xxxx since its wrong, or smth. i did and Myrics apex-i-compact works wonders for me, with dry as follows:
cmd = [
r".\llama-server.exe",
"-m", r"D:\Laguna-S-2.1-APEX-i-compact.gguf",
"--reasoning-preserve",
"--reasoning-budget", "16000",
"--reasoning-budget-message", "I've thought enough. Answering now with what I have.",
"-ngl", "999",
"--n-cpu-moe", "47",
"--rope-scaling", "yarn",
"--rope-scale", "32",
"--yarn-orig-ctx", "8192",
"--yarn-attn-factor", "1.0",
"--load-mode", "dio",
"-c", "170000",
"--cache-type-k", "q8_0",
"--cache-type-v", "q8_0",
"-np", "1",
"-fa", "on",
"-t", "8",
"-tb", "8",
"-b", "3072",
"-ub", "3072",
"--jinja",
"-kvu",
"--temp", "0.7",
"--top-p", "0.95",
"--top-k", "20",
"--min-p", "0.0",
"--dry-multiplier", "0.6",
"--dry-base", "1.75",
"--dry-allowed-length", "4",
"--dry-penalty-last-n", "-1",
"--dry-sequence-breaker", r'\n,:,",*,;,{,}',
"--samplers", "top_k;top_p;min_p;temperature;dry",
"--alias", "laguna-s-2.1",
"--cache-reuse", "256",
"--cache-ram", "1024",
"--host", "127.0.0.1",
"--port", "8080",
"--verbosity", "4",
"--chat-template-file", r"G:\xlam3\laguna\chat_template_laguna.jinja",
]