OsaurusAI/Laguna-S-2.1-JANG_4M

JANG_4M JANG affine bundle of poolside/Laguna-S-2.1 — 118B-parameter / ~8B-active MoE for agentic coding and long-horizon work, quantized for Apple Silicon with AWQ-protected routed experts.

Everything quality-critical stays high precision: attention (including the softplus output gate), the shared expert path, router, and norms. The 256-routed-expert bulk (~97% of parameters) carries the low bits, protected by activation-aware (AWQ) scales selected by measured quantization error on real expert weights and real calibration activations — the search always includes a no-AWQ baseline, so scales ship only because they beat it (2.9% lower reconstruction error at this bit width).

Bundle

Field	Value
Source	poolside/Laguna-S-2.1 @ a50e85e (BF16, 235 GB)
Architecture	laguna / LagunaForCausalLM — 48 layers (12 global + 36 SWA w512), GQA 8 KV heads, 256 routed experts top-10 + 1 shared, dense layer 0, 1M context
On-disk size	68.1 GB (15 shards)
Routed experts	4-bit gate/up/down affine, group 64, AWQ input scales folded
Attention q/k/v/o + g_proj	8-bit affine
Shared expert / dense FFN	8-bit affine
Embeddings / lm_head	6-bit / 8-bit affine
Router, e_score bias, norms	fp16 passthrough (routing bit-identical to unfolded math)
Modality	text-only (verified from tensor index — no vision/audio/video weights)

Measured performance (M5 Max, 128 GB)

Metric	Value
Decode	33-44 tok/s greedy (cold-warm)
Load time	5.3 s
Long-context cache parity	teacher-forced top-1 agreement 0.996 / 1.000 across a 2,913-token pass (pre / post the 512 sliding window)
BF16 vs quantized	greedy smoke + chat-with-thinking verified coherent

Memory note: budget bundle size + ~10 GB. The reference Python runtime auto-sets the Metal wired limit (min(bundle x 1.2 + 8 GB, 118 GB)) — without a wired limit, decode throughput roughly halves on 64 GB+ working sets.

Chat protocol

• GLM-style think tags. Thinking is ON by default (vendor default_chat_template_kwargs.enable_thinking=true); pass enable_thinking=False to disable. The generation prompt ends <assistant><think> (thinking) or <assistant></think> (off).
• Stop tokens: eos_token_id = [2, 24] — id 24 is end-of-turn and MUST be in the stop set.
• The template emits its own leading 〈|EOS|〉 (= bos id 2): do not prepend another BOS.
• Tool calls: <tool_call>name<arg_key>k</arg_key><arg_value>v</arg_value></tool_call> (GLM-4.7-compatible parsing).
• Sampling defaults (vendor): temperature 1.0, top_p 1.0, top_k 20.

Run it

Osaurus loads this bundle natively. Python reference runtime (mixed-precision affine aware):

pip install mlx mlx-lm transformers
python -m jang_tools.laguna.runtime --src ./Laguna-S-2.1-JANG_4M \
    --prompt 'def fibonacci(n):' --max-new 64

config.json[quantization] carries per-module {bits, group_size, mode} overrides — loaders must honor per-module bits (a single top-level width mis-dequantizes the low-bit experts).

Source model quality (poolside, BF16 base)

Per the upstream model card: Terminal-Bench 2.1 70.2%, SWE-bench Multilingual 78.5%, SWE-Bench Pro 59.4% — competitive with much larger frontier MoEs. These figures are for the BF16 source model.

---

Quantized and verified by Jinho Jang (eric@osaurus.ai) with the JANG toolchain. License: OpenMDW-1.1 (inherited from the source model).