nbeerbower/Laguna-S-2.1-colibri-int4

Laguna-S-2.1 — colibri int4

An int4 conversion of poolside/Laguna-S-2.1 (118B total / 8B active MoE, coding) into the colibri container format, for the laguna.c standalone CPU/CUDA inference engine (part of the sabrewing fork of colibrì).

What this is

• Routed experts (>95% of the weights) are stored as packed int4 — per-row symmetric scales (.qs, f32), low-nibble/high-nibble packing, value = (nibble − 8) · scale. Gate and up projections are fused block-concatenated per expert (experts.gate_up_proj [E·2I, D/2]); down is experts.down_proj [E·D, I/2].
• Residents (attention q/k/v/o + per-head softplus gate, QK-norms, dense layer-0 MLP, shared experts, embeddings, lm_head) are bf16; norms, the router weight and the loss-free e_score_correction_bias stay f32.
• Total footprint ≈ 63 GB (vs 235 GB bf16), so the whole model is RAM-resident on a 128 GB+ machine — no expert-streaming disk wall.

Faithfulness

Produced by tools/convert_laguna_int4.py. The fuse+pack is bit-exact against the int4 round-trip, and the C engine reads the container within quantization noise (tiny-model teacher-forced perplexity within ~0.13% of the f32 forward, which is itself token-exact vs the transformers reference).

Architecture (for the curious)

GQA (8 KV heads, head_dim 128) with interleaved 512-token sliding / global attention (1:3), per-head QK-RMSNorm, a per-head softplus attention output gate, RoPE (YaRN partial-rotary on global layers, plain full-rotary on sliding), a dense layer 0 + sigmoid loss-free-routed MoE (256 experts, top-10, + 1 shared expert, routed scaling 2.5). 48 layers, hidden 3072, vocab 100,352.

Usage

git clone https://github.com/Schneewolf-Labs/sabrewing && cd sabrewing/c
make laguna
SNAP=/path/to/Laguna-S-2.1-colibri-int4 ./laguna   # Stage-A oracle / prompt mode

Original model © Poolside, licensed under OpenMDW-1.1 — this conversion inherits it.