System2-Online/Qwen3.8-27B-DualProcess-GGUF

System2: Quantized Edge Dual-Process Coding Models

Official repository for System2 tested quantized models optimized for sovereign, on-device agentic software engineering.

• Organization: System2-Online
• Engine: pi-system2
• Core Research: tj-doomer-it/System2
• Quantization Architecture: Unsloth Dynamic (UD) per-layer heterogeneous quantization
• Evaluated Code Quality: 8.5 / 10 (Independent adversarial audit by Claude 3.7)

---

1. Quantization Architecture: Unsloth Dynamic (UD) & Layer Rebalancing

Standard uniform quantization compresses every tensor uniformly, which disproportionately degrades sensitive attention projections, router gates, and critical language syntax layers.

These models utilize the Unsloth Dynamic (UD) architecture:

1. Heterogeneous Bit-Budget Rebalancing: Sensitive layers (attention heads, critical MLP projections, and gating tensors) are preserved at higher precision, while non-critical weights are aggressively compressed.
2. Selective Component Swapping & Pruning: Auxiliary components (such as multi-token prediction heads in smaller sub-8 GB tiers) are swapped or pruned to allocate the maximum available bit budget to core reasoning weights.
3. Calibrated Importance Matrix (imatrix): Calibrated against diverse code generation, multi-turn reasoning, and syntactic edge cases to maintain Pareto-frontier accuracy (+10–15% top-1 accuracy over naive uniform quants of identical file size).

---

2. The Breakthrough: The "Regime 2" Protocol

At 2-bit to 3-bit precision, compounding token-level quantization noise causes internal chain-of-thought (<think>) generation loops to hallucinate, repeat, and overflow context windows before producing executable code.

System2 solves this through Dual-Process Cognition:

1. System 1 (Perceptual Triage & Fast Syntax): The quantized model is run with thinking: off (or temperature-calibrated direct completion). Without noisy internal CoT loops, the model synthesizes clean, idiomatic code without catastrophic degradation.
2. System 2 (Deliberation & Symbolic Verification): Outer deterministic verification harnesses (pi-system2) provide test execution, traceback inspection, and AST invariant checking in an external closed feedback loop.

In benchmark testing, the 10.18 GB IQ3_XXS model running locally on an Apple Silicon M4 Mac autonomously architected an in-memory Pub/Sub event bus with 22 passing pytest suites in 0.05s, diagnosing and fixing its own test setup with zero human intervention.

---

3. Model Variants

File	Precision / Architecture	File Size	Recommended RAM / VRAM	Optimal Use Case
Qwen3.8-27B-UD-IQ3_XXS.gguf	UD-IQ3_XXS (~3.06 bpw)	10.4 GB	16 GB Unified Memory	Primary Recommended: Optimal frontier coding quality floor.
Qwen3.8-27B-UD-Q2_K_XL.gguf	UD-Q2_K_XL (~2.75 bpw)	9.4 GB	16 GB Unified Memory	Balanced performance and memory footprint.
Qwen3.8-27B-UD-IQ2_XXS.gguf	UD-IQ2_XXS (~2.06 bpw)	6.9 GB	8 GB – 12 GB RAM	Extreme low-memory tier for edge devices.

---

4. Quickstart: Running with LM Studio + pi-system2

1. Load Model in LM Studio

1. Load Qwen3.8-27B-UD-IQ3_XXS.gguf into LM Studio.
2. Start the local inference server at http://localhost:1234/v1.
3. In LM Studio presets, set Temperature: 0.2 and enable tool calling.

2. Run with pi-system2

Install and launch the System2 agent harness:

pi-system2 --model system2/local -p "Architect a thread-safe token bucket rate limiter in Python with tests"

The virtual model router will automatically route to your local model with the Regime 2 protocol enforced (thinkingLevel: "off"), running the external System 2 verification loop.

---

Citation & Contact

For research inquiries, commercial deployment, or enterprise air-gapped agent harnesses:

• Web: system2.online
• Hugging Face: huggingface.co/System2-Online
• GitHub: github.com/tj-doomer-it/System2