Instructions to use olka-fi/Laguna-S-2.1-MXFP4 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use olka-fi/Laguna-S-2.1-MXFP4 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="olka-fi/Laguna-S-2.1-MXFP4", trust_remote_code=True)
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("olka-fi/Laguna-S-2.1-MXFP4", trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained("olka-fi/Laguna-S-2.1-MXFP4", trust_remote_code=True, device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use olka-fi/Laguna-S-2.1-MXFP4 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "olka-fi/Laguna-S-2.1-MXFP4"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "olka-fi/Laguna-S-2.1-MXFP4",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/olka-fi/Laguna-S-2.1-MXFP4

SGLang

How to use olka-fi/Laguna-S-2.1-MXFP4 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "olka-fi/Laguna-S-2.1-MXFP4" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "olka-fi/Laguna-S-2.1-MXFP4",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "olka-fi/Laguna-S-2.1-MXFP4" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "olka-fi/Laguna-S-2.1-MXFP4",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use olka-fi/Laguna-S-2.1-MXFP4 with Docker Model Runner:
```
docker model run hf.co/olka-fi/Laguna-S-2.1-MXFP4
```

Laguna S 2.1 — MXFP4 (mixed precision)

A 4-bit MXFP4 quantization of poolside/Laguna-S-2.1, produced with qstream. The routed MoE experts (~96% of the weights) are quantized to MXFP4; everything quality-sensitive stays BF16.

The original model card follows in full below.


Size	64 GiB (down from 219 GiB BF16 source, ~29%)
Format	compressed-tensors `mxfp4-pack-quantized` (E2M1 4-bit + E8M0 group-32 scales)
Base	118B-A8B MoE for agentic coding: 256 routed experts top-10 + shared expert, sigmoid routing with score-correction bias, 48 layers (12 full-attention + 36 sliding-window-512), 1M context (YaRN)

What is quantized to what

Component	Precision	Why
Routed experts (`model.layers.{1..47}.mlp.experts.*`)	MXFP4 (4-bit)	~96% of the weights — the only place worth the size win
Attention (incl. `g_proj` output gating), shared experts, router gate, layer-0 dense MLP	BF16	sensitive / runs on every token — kept lossless from the source
Embeddings, lm_head, norms	BF16	unchanged

Quality & faithfulness

We lead with deterministic, reproducible faithfulness metrics. (Task evals served over vLLM with continuous batching are not bitwise-deterministic — the same prompt at temperature=0 can yield different reductions depending on batch composition — so task accuracies carry a little serving noise; we quote one full-test-set score, where that noise is small.)

Metric	Result	What it shows
Perplexity (wikitext-2 test, 309k tokens)	9.82	language modeling intact — a broken quant lands in the hundreds
Routed-expert SQNR	≈ 19 dB	reconstruction error is just the unavoidable 4-bit rounding (MXFP4 vs the BF16 source)
GSM8K (full test, 1319 questions, temp 0)	94.5%	end-to-end reasoning intact through the quantized experts

Why this is enough to trust the checkpoint:

Only the routed experts changed. ~96% of the weights are re-quantized to MXFP4; everything else is bit-identical BF16 to the source. So the model is the base model except for 4-bit rounding on the expert GEMMs.
The error is uniform and diffuse. All 36,096 expert tensors sit in a tight band (median relative error 0.1117, p95 0.1124, max 0.1356) with no pathological layers — pure 4-bit rounding, no outliers to hide.

PPL script: evals/eval_ppl_wikitext2.py in the qstream repo.

Serving with vLLM

Laguna is natively supported in vLLM ≥ 0.23, and this checkpoint uses the stock compressed-tensors MXFP4 path — no patches needed:

vllm serve olka-fi/Laguna-S-2.1-MXFP4 --served-model-name laguna --max-model-len 60000

Keep --max-model-len bounded: only the 12 full-attention layers grow KV with context (~48 KiB/token in BF16 KV), the other 36 layers hold a fixed 512-token sliding window. The native 1M context needs ~46 GiB of KV per sequence.

Observed throughput: ≈600 tok/s aggregate generation at batch size 16 on a single RTX PRO 6000 Blackwell (96 GB) — the whole checkpoint plus KV fits on one card.

FP8 KV cache — works uncalibrated

--kv-cache-dtype fp8 is a verified free lunch on this model: no calibration or static k/v scales needed. Measured against BF16 KV: wikitext-2 PPL 9.812 vs 9.816, passkey retrieval 20/20 vs 20/20 at all depths in ~30k-token contexts. (Laguna's per-head k_norm keeps keys inside e4m3 range at scale 1.0.) This halves KV to ~24 KiB/token — double the context or concurrency.

On SM120 (RTX PRO 6000 / Blackwell workstation) three extra knobs are required:

VLLM_ATTENTION_BACKEND=FLASHINFER \
VLLM_FLASHINFER_WORKSPACE_BUFFER_SIZE=1073741824 \
vllm serve olka-fi/Laguna-S-2.1-MXFP4 --served-model-name laguna --max-model-len 60000 \
  --kv-cache-dtype fp8 \
  --speculative-config '{"model":"olka-fi/Laguna-S-2.1-MXFP4-dspark","method":"dspark","num_speculative_tokens":4}' \
  --attention-config.disable_flashinfer_q_quantization=1

FlashInfer because FA2 can't read fp8 KV; disable_flashinfer_q_quantization because vLLM otherwise also quantizes queries to fp8, which flashinfer's fa2 prefill template rejects outside the SM100 trtllm-gen path; the 1 GiB workspace because the default 394 MiB overflows on Laguna's 72-head sliding-window layers.

How it was made

qstream-quantize \
  --model_dir <poolside/Laguna-S-2.1> \
  --output_dir ./Laguna-S-2.1-MXFP4 \
  --include_layers "mlp.experts." \
  --workers 8

Activation-aware (γ-weighted) MSE scale selection with per-block search over candidate E8M0 exponents; the BF16 remainder is passed through untouched and the mixed-precision config.json (ignore list + expert targets) is emitted for stock vLLM loading. Built with qstream @9f8cd0c.

License

Inherits the OpenMDW-1.1 license from the base model. This is a derivative (quantized) work of poolside's Laguna S 2.1.

Original model card

Use on OpenRouter · Use on Vercel AI Gateway · Release blog post

Laguna S 2.1

Laguna S 2.1 is a 118B total parameter Mixture-of-Experts model with 8B activated parameters per token, designed for agentic coding and long-horizon work. It sits between Laguna XS 2.1 (33B-A3B) and Laguna M.1 (225B-A23B) in the Laguna series and shares the family recipe: a token-choice router with softplus gating over 256 routed experts plus one shared expert, grouped-query attention, and interleaved full/sliding-window attention.

Highlights

Mixed SWA and global attention layout: 48 layers in a 1:3 global-to-SWA ratio (12 global attention layers, 36 sliding-window layers, window 512), with softplus attention gating and per-layer-type rotary scales
1M context: 1,048,576-token context window
Native reasoning support: interleaved thinking between tool calls, with per-request control via enable_thinking
Speculative decoding: a trained DFlash draft model is available for lower-latency serving
Quantized variants: FP8, NVFP4, INT4 and GGUF
OpenMDW-1.1 license: Use and modify the model and associated materials freely for commercial and non-commercial purposes (learn more about OpenMDW)

Model overview

Number of parameters: 118B total, ~8B activated per token
Layers: 48 (12 global attention, 36 sliding-window attention)
Experts: 256 routed (top-10) plus 1 shared expert
Attention: grouped-query, 8 KV heads, head dim 128; per-head softplus output gating
Sliding window: 512 tokens
Context window: 1,048,576 tokens
Vocabulary: 100,352 tokens (Laguna family tokenizer)
Modality: text-to-text
Reasoning: interleaved thinking with preserved thinking

Benchmark results

benchmarks

Model	Size	Terminal-Bench 2.1	SWE-bench Multilingual	SWE-Bench Pro (Public Dataset)	DeepSWE	SWE Atlas (Codebase QnA)	Toolathlon Verified
Laguna S 2.1	118B-A8B	70.2%	78.5%	59.4%	40.4%	46.2%	49.7%
Tencent Hy3	295B-A21B	71.7%	75.8%	57.9%	-	-	-
Inkling	975B-A41B	63.8%	-	54.3%	-	-	45.5%*
Nemotron 3 Ultra	550B-A55B	56.4%	67.7%	-	-	-	34.3%*
DeepSeek-V4-Pro Max	1.6T-A49B	64.0%*	76.2%	55.4%	9.0%*	27.2%*	55.9%*
Kimi K3	2800B-A50B	88.3%	-	-	69%	-	-
Qwen 3.7 Max	-	74.5%*	78.3%	60.6%	-	-	-
Muse Spark 1.1	-	80%	-	61.5%	53.3%	42.2%*	75.6%
Claude Fable 5	-	88%	-	80.3%	70%	-	-

Benchmarks as of 21 July 2026. Laguna S 2.1 in bold; a dash (-) marks a benchmark a model was not evaluated on. Scores marked * are as reported by third parties: Terminal-Bench 2.1 and DeepSWE via Artificial Analysis, SWE Atlas via Scale AI's official leaderboard, and Toolathlon Verified via its official leaderboard. Full evaluation trajectories: trajectories.poolside.ai.

Usage

Laguna S 2.1 uses the same laguna architecture as Laguna XS 2.1, so the same engine integrations apply (vLLM, SGLang, Transformers, TRT-LLM, llama.cpp). At 118B parameters the BF16 checkpoint needs multiple GPUs (roughly 236GB of weights); quantized variants reduce this substantially.

vLLM

vllm serve \
    --model poolside/Laguna-S-2.1 \
    --tensor-parallel-size 4 \
    --tool-call-parser poolside_v1 \
    --reasoning-parser poolside_v1 \
    --enable-auto-tool-choice \
    --served-model-name laguna \
    --default-chat-template-kwargs '{"enable_thinking": true}'

Optional: speculative decoding with DFlash. Pair with the Laguna S 2.1 DFlash draft model by adding --speculative-config '{"model":"poolside/Laguna-S-2.1-DFlash","num_speculative_tokens":7,"method":"dflash"}'.

SGLang

python -m sglang.launch_server \
  --model-path poolside/Laguna-S-2.1 \
  --tp-size 4 \
  --reasoning-parser poolside_v1 \
  --tool-call-parser poolside_v1 \
  --trust-remote-code

TRT-LLM

trtllm-serve poolside/Laguna-S-2.1 --trust-remote-code \
    --tool_parser poolside_v1 --reasoning_parser laguna

Note the flag names differ from vLLM's (--tool_parser, and the reasoning parser is laguna, not poolside_v1).

llama.cpp

GGUF conversions are available at poolside/Laguna-S-2.1-GGUF. Serve with poolside's llama.cpp fork, branch laguna, which carries full Laguna support including DFlash speculative decoding. (Base Laguna support is also in upstream review: ggml-org/llama.cpp#25165.)

git clone --branch laguna https://github.com/poolsideai/llama.cpp
cd llama.cpp && cmake -B build && cmake --build build -j

./build/bin/llama-server -m laguna-s-2.1-Q4_K_M.gguf --jinja --port 8000

# with DFlash speculative decoding:
./build/bin/llama-server -m laguna-s-2.1-Q4_K_M.gguf \
  -md laguna-s-2.1-DFlash-BF16.gguf \
  --spec-type draft-dflash --spec-draft-n-max 15 -fa on --jinja --port 8000

Controlling reasoning

Laguna S 2.1 has native reasoning support and works best with preserved thinking: keep reasoning_content from prior assistant messages in the message history. The model will generally reason before calling tools and between tool calls, and may stop reasoning in follow-up steps if prior thinking blocks are dropped.

Thinking is controlled per request via the chat template:

extra_body={"chat_template_kwargs": {"enable_thinking": False}}

or at the server level with --default-chat-template-kwargs '{"enable_thinking": true}'. For agentic coding use cases we recommend enabling thinking and preserving reasoning in the message history.

License

This model is licensed under the OpenMDW-1.1 License.

Intended and Responsible Use

Laguna S 2.1 is designed for software engineering and agentic coding use cases, and you are responsible for confirming that it is appropriate for your intended application. Laguna S 2.1 is subject to the OpenMDW-1.1 License, and should be used consistently with Poolside's Acceptable Use Policy. We advise against circumventing Laguna S 2.1 safety guardrails without implementing substantially equivalent mitigations appropriate for your use case.

Please report security vulnerabilities or safety concerns to security@poolside.ai.

Downloads last month: 386

Safetensors

Model size

64B params

Tensor type

BF16

Model tree for olka-fi/Laguna-S-2.1-MXFP4

Base model

poolside/Laguna-S-2.1

Quantized

(70)

this model

Adapters

1 model