Swift-Qwen3.8-27B-Uncensored-W4A16-AutoRound

Update (2026-09-21) -- MTP tensor fix: model-00004-of-00004.safetensors still contained the 27 superseded int8 mtp.* tensors that the int4 GPTQ recalibration in model_extra_tensors.safetensors replaced, and model.safetensors.index.json only routed 3 of them (mtp.draft_lm_head.*) to the extras file. vLLM loads every key in each file it opens, so with SPEC=mtp it saw two widths of mtp.fc.weight_packed ((5120, 2560) int8 in shard 4, (5120, 1280) int4 in the extras file) and failed with a shape mismatch (The size of tensor a (640) must match the size of tensor b (1280)). DFlash2 (SPEC=dflash2) never loads those tensors, which is why it went unnoticed. Fix: the 27 duplicated tensors were removed from shard 4 (3.25 GB -> 2.61 GB) and all 27 index entries now point at model_extra_tensors.safetensors. No weights were changed: the 240 tensors left in shard 4 are byte-identical to before and shards 1-3 and the extras file are untouched. Verified: no key appears in more than one file, the index matches the file contents, and vLLM 0.28.0 boots and serves with SPEC=mtp (the previous upload crashes at load).

Update (2026-09-18): lm_head/embed_tokens and the MTP head were recalibrated with GPTQ (Hessian-based, from real captured activations) instead of the original RTN int8 shrink, now quantized to int4. Measured single-stream: 119-120 → 137-141 tok/s (+15-18%), direct before/after comparison on the same hardware. Weights below are the recalibrated version.

A W4A16 AutoRound quantization of d0xin/Swift-Qwen3.8-27B-Uncensored-BF16, reproducing dbirks/Qwen3.8-27B-W4A16-AutoRound's published recipe against this uncensored, reasoning-efficient base instead of the official checkpoint. Combines ukisai/Swift-Qwen3.8-27b's reasoning-token-efficiency adapter with d0xin's abliteration, quantized to run comfortably on a single 24GB GPU with speculative decoding intact.

Provenance

Qwen/Qwen3.8-27B (Apache 2.0)
  -> ukisai/Swift-Qwen3.8-27b (Swift Open License v1.0)
       reasoning-efficiency LoRA adapter, merged
  -> d0xin/Swift-Qwen3.8-27B-Uncensored-BF16
       rank-1 directional residual-stream ablation, layer 38
       (131 / 1,199 tensors modified, vision tensors unchanged)
  -> this repository
       AutoRound W4A16 g128 sym quantization + int8 head/embed/MTP shrink

See NOTICE for the full attribution chain and license obligations.

Tested with

Quantized and served on a single RTX 3090 (24GB) using syv-ai/HyperQwen's vLLM 0.28.0 container (DFlash2 MTP speculative decoding, --profile single). Both context tiers documented there (CTX=fast, 65,536 tokens; CTX=huge, 245,760 tokens via KVarN) boot and serve correctly with this checkpoint -- see the throughput numbers below.

Throughput

Single-stream decode, DFlash2 MTP speculative decoding, RTX 3090, from a real /v1/chat/completions request (not a synthetic benchmark harness -- we didn't run llama-bench here since that targets GGUF/llama.cpp and this checkpoint is a vLLM/compressed-tensors format, a different serving path than what's actually being measured below):

Context tier	Decode throughput
CTX=fast (65,536 tokens)	137-141 tok/s
CTX=huge (245,760 tokens, KVarN)	96 tok/s

CTX=fast reflects the GPTQ-recalibrated head/MTP weights (see Update above), measured single-stream with vllm bench serve against real prompts. Both the pre- and post-recalibration checkpoints were benchmarked back-to-back on the same hardware (two runs each, second run kept per the harness's own warmup note): pre-recalibration measured 119-120 tok/s, post-recalibration 137-141 tok/s. CTX=huge has not been re-measured since the recalibration. Both remain in line with the official checkpoint's own numbers on the same hardware and serving stack.

Why this exists

Two other third-party attempts to combine Swift + uncensoring into an AutoRound/GPTQ checkpoint for vLLM both fail to boot on a 24GB card (torch.OutOfMemoryError at KV cache init, regardless of context tier). In both cases the root cause was the same: lm_head and embed_tokens were left unquantized at full BF16 (untied embeddings on this 248,320-vocab / 5,120-hidden model means that's ~5GB right there), and the vision tower was similarly left unquantized. This checkpoint runs prepare/'s int8 shrink pass on lm_head, embed_tokens, and the MTP module on top of the AutoRound pass -- the step both of those checkpoints skipped -- which is what makes it actually fit.

Quantization recipe

• Tool: auto-round 0.15.0
• Scheme: W4A16, group_size=128, symmetric
• Calibration: 128 samples, 200 iterations (SignRound per-block early-stopping -- most blocks converged well before iter 200; observed range iter 67-197 in this run)
• Excluded from quantization (kept BF16, matching dbirks' original recipe): linear_attn.in_proj_a, linear_attn.in_proj_b (DeltaNet recurrence-control projections), visual.* (vision tower), mtp.* (draft head)
• Quantized on top, via this repo's own prepare/ scripts (not part of the AutoRound pass): lm_head and embed_tokens GPTQ-calibrated to int4 g128 (Hessian from real captured activations; KL divergence vs. bf16 improved from 0.0069 with a naive RTN int4 pass to 0.0029), the MTP module's linear layers GPTQ-calibrated to int4 similarly (round-trip relative error 0.14-0.17 across its 8 linears), plus a 40,960-token draft vocabulary built for speculative decoding
• Quantization tuning time: 1h48m on a single RTX 3090 (64/64 transformer blocks, 400/607 tensors actually quantized -- the rest are the excluded BF16 layers above)
• On-disk size: 16.4GB (down from 52GB BF16)

Reproduction script (quantize_abliterated.py) and this exact provenance writeup are included in this repository.

Correctness validation

Ran a 6-task battery (math, code generation, factual recall, a constraint-logic puzzle, a security-training explanation, and strict output-format compliance) against this checkpoint and against the official Qwen3.8-27B-W4A16-AutoRound baseline, both served via vLLM 0.28.0 with DFlash2 MTP speculative decoding on an RTX 3090:

Task	This checkpoint	Official baseline
Modular arithmetic	correct	correct
is_prime() implementation	correct	correct
Factual recall (capital city)	correct	correct
3-variable logic puzzle (unique solution)	correct	correct
Security-training explanation	answered, no refusal	answered, no refusal
Strict 3-item format compliance	correct	correct

6/6 correct on both -- no quality regression observed from this quantization relative to the official recipe applied to the official checkpoint.

Reasoning-token efficiency

UkisAI's own card for the Swift adapter reports "58.3% fewer thinking tokens" as a headline figure, with a more specific "mean thinking token reduction of 41.0% on general reasoning benchmarks." On our 6-task battery above, this checkpoint used 1,058 total reasoning tokens versus 1,539 for the official baseline on the same prompts -- 31.2% fewer, directionally consistent with UkisAI's own numbers but measured independently, on a small sample, after quantization.

The "no refusal" result on the security-training prompt is a weak signal for the uncensoring specifically: it's a fairly mainstream security-awareness topic that the aligned baseline also answered without refusing. This checkpoint's refusal-rate behavior under harder adversarial prompts has not been separately stress-tested here; see d0xin's own INTELLIGENCE_VALIDATION.json and 100-prompt refusal eval for that base model's own numbers (0/100 refusals reported there).

Known limitations

• Validated with a 6-prompt battery, not a full benchmark suite (no GPQA/MMLU-Pro/LiveCodeBench/AIME reproduction here).
• Ships with the same shared DFlash2 drafter and draft_vocab_ids.json used by the official-weights checkpoints -- that drafter is trained on the official model's output distribution, so expect a lower MTP acceptance rate on content this tune generates differently (~20-21% in this family's other uncensored variants, rejection sampling always falls back to this model's own distribution so it's a speed characteristic, not a correctness issue).
• CTX=huge (KVarN) tier tested and boots clean; both tiers were only exercised with short completions during validation, not long-context/long-generation stress testing.

Files

Model weights and tokenizer/config files are unmodified in structure from the AutoRound + prepare/ pipeline output -- see config.json's quantization_config for the exact compressed-tensors quantization metadata. chat_template.jinja is byte-identical to the official Qwen3.8-27B checkpoint's (sha256 c3cf9e34...81041).

License

Swift Open License v1.0 (LICENSE) governs the Swift Contribution this work incorporates; commercial use above $1M annual revenue requires a separate license from UkisAI (contact: https://ukisai.com/contact). The underlying Qwen3.8-27B base remains under Apache License 2.0 (LICENSE-APACHE-2.0), included per Section 4(e) of the Swift Open License v1.0. See NOTICE for the full chain.