Dragoy/Swift-Qwen3.8-27B-abliterated-NVFP4-NInfer

Updated 2026-09-18 — DFlash2 drafter added (re-converted in place with the official tools.convert @ 98dada0e, components text,vision,mtp,dflash2): text/vision/MTP weights byte-identical to the 2026-09-17 v3 build; new 5-layer DFlash2 draft model from z-lab/Qwen3.8-27B-DFlash2 @ 50307d4c; min runtime revision is now 98dada0e. Same filename — just re-download. See discussion #2.

Swift-Qwen3.8-27B · huihui-style abliterated · NVFP4 · NInfer

A 27.78B-parameter multimodal derivative of ukisai/Swift-Qwen3.8-27b, abliterated in the huihui-ai style, quantized to NVFP4 + FP8 for the NInfer engine on Blackwell (sm_120a).

Base	ukisai/Swift-Qwen3.8-27b @ 1b30aaaf753f
Abliteration	huihui-style refusal-direction removal, layers 17–51 (70 tensors), transferred by weight-difference from the Qwen/Qwen3.8-27B ↔ huihui pair
Quantization	NVFP4 (MLP gate/up/down) + FP8 (attention, GDN, lm_head) — allocation copied verbatim from unsloth/Qwen3.8-27B-NVFP4, 32 calibration samples
Engine	Neroued/ninfer @ a140e7ae82a1, built for sm_120a
Container	NInfer artifact container v3 (NINFER\x00\x03), upgraded in place from v2 by the official upgrade_ninfer_v2_to_v3.py @ f76e19c0 — weights byte-identical, maintained Qwen chat template installed
Artifact	qwen3_8_27b_swift_abliterated_nvfp4.ninfer — 1246 objects (incl. dflash2 drafter), 23,719,715,844 bytes
Published	2026-09-16
Updated	2026-09-18 — DFlash2 drafter added; text/vision/MTP weights byte-identical

Why this exists

ukisai/Swift-Qwen3.8-27b is a strong multimodal Qwen3.8-27B checkpoint, but it still carries the refusal direction of the base model. The huihui abliteration removes that direction (Arditi et al. 2024, refusal-projection family), and the NInfer NVFP4 build makes the result runnable on a single Blackwell card at ~20 GB. This repository is the end-to-end product of that chain: Swift → huihui-style abliteration → NVFP4 → NInfer, built and verified on one GPU box, with every step recorded in NOTICE.

Quickstart

The artifact runs with the NInfer CLI or ninfer-serve (requires the ninfer runtime at revision ≥ 98dada0e (DFlash2 support; f76e19c0 suffices for MTP-only runs), CUDA ≥ 13.1, an sm_120a build):

# one-shot
ninfer qwen3_8_27b_swift_abliterated_nvfp4.ninfer \
  --prompt "Tell me something interesting about the universe." \
  --max-context 16384 --max-new 512 --no-thinking

# with MTP speculative decoding
ninfer qwen3_8_27b_swift_abliterated_nvfp4.ninfer \
  --prompt "Write a haiku about a GPU cluster at 3am." \
  --max-context 16384 --max-new 256 \
  --spec mtp --draft-tokens 5 --lm-head-draft --no-thinking

# with DFlash2 speculative decoding (fastest: 1-15 token draft window)
ninfer qwen3_8_27b_swift_abliterated_nvfp4.ninfer \
  --prompt "Write a haiku about a GPU cluster at 3am." \
  --max-context 16384 --max-new 256 \
  --spec dflash2 --draft-tokens 7 --lm-head-draft --no-thinking

# serving
ninfer-serve qwen3_8_27b_swift_abliterated_nvfp4.ninfer --port 8088

It is a multimodal model (image-text-to-text); the full Qwen3.8-27B vision tower and MTP head are included in the artifact.

Measured on this artifact

Check	Result
Artifact inventory (contract)	1246 objects (incl. dflash2) — VERDICT: OK
MTP acceptance (smoke, 256 new tokens)	46.3% acceptance, 1 fallback step
AdvBench refusal probe (20 harmful prompts, greedy)	0.0% refusal

Refusal probe: standard refusal-prefix heuristic over the first 400 characters of greedy completions on the first 20 AdvBench harmful_behaviors prompts (no content is stored, only verdicts).

Provenance

Component	Source
Base weights	ukisai/Swift-Qwen3.8-27b @ 1b30aaaf753fe5c1cb51ada2ea0367a53445359c
Abliteration transform reference	Qwen/Qwen3.8-27B @ 1d4bf0f2ff60 and huihui-ai/Huihui-Qwen3.8-27B-abliterated @ 739e3c5b8984
Quantization recipe	unsloth/Qwen3.8-27B-NVFP4 quantization_config (verbatim, recipe/unsloth_qconfig.json)
Converter	Neroued/ninfer @ 98dada0e03cb073f — official tools.convert, recipe qwen3_8_27b_nvfp4, components text,vision,mtp,dflash2 (2026-09-18 re-conversion); original v2 build: @ a140e7ae82a1 + patches/0001-nvfp4-provenance.patch
Container v3 upgrade	official tools/upgrade_ninfer_v2_to_v3.py @ f76e19c0fbd0 (Neroued/ninfer) — v2→v3 reframe, weights preserved; maintained Qwen chat template installed
Conversion report	qwen3_8_27b_swift_abliterated_nvfp4.ninfer.conversion.json
Inventory contract	artifact-manifest.json, checksums in SHA256SUMS

Reproduction

1. scripts/fetch_source.sh (in the build log) downloaded the three BF16 checkpoints (Xet-free, size-verified, resumable).
2. The abliteration step applied W_swift + (W_huihui - W_base) to exactly the tensors the reference abliteration modified; all other tensors are byte-identical to Swift.
3. recipe/quantize_nvfp4.py ran llm-compressor oneshot (CPU-resident, ~8 GB VRAM peak).
4. The NInfer engine was built for sm_120a and convert_nvfp4.py produced the artifact; verify_artifact.py confirmed the 1124-object contract. On 2026-09-18 the artifact was re-converted in place by the official tools.convert @ 98dada0e (same sources + DFlash2 drafter): 1246-object contract, text/vision/MTP weights byte-identical (verified object-by-object).

Frontend disclosure

Frontend gate was satisfied without substitution (all six files already canonical).

Also available

The same checkpoint in standard Hugging Face safetensors format (NVFP4 + FP8, vLLM/transformers-compatible, with JSON-schema structured output) instead of the single-file NInfer artifact: Dragoy/Swift-Qwen3.8-27B-abliterated-NVFP4.

License

This repository is a derivative of the Swift checkpoint, whose license is the Swift Open License v1.0 — not Apache. The chain:

Component	Licence
Qwen/Qwen3.8-27B (base model)	Apache-2.0 — Copyright 2026 Alibaba Cloud (LICENSE-APACHE-2.0)
ukisai/Swift-Qwen3.8-27b (Swift Contribution)	Swift Open License v1.0 (LICENSE)
This repo (abliteration + quantization + packaging)	derivative work — the Swift Contribution contained in it stays under the Swift Open License v1.0

What that means in practice:

• Free use, including commercial, while your gross revenue (counting all controlled entities) is below the $1,000,000 per fiscal year threshold; qualified non-profits have no threshold for non-commercial or research use.
• Above the threshold: obtain a separate written licence from UkisAI (Swift Enterprise License).
• Redistribution: ship both licence files, keep the copyright and attribution notices, and mark files you modified (Swift licence §4–§5).

See also NOTICE for the derivation chain.

This is a description of what the licences say, not legal advice.

Intended use and limitations

This is an uncensored model: the upstream abliteration removes the refusal direction, so it will attempt requests a stock instruction-tuned model declines. It is published for research, evaluation and local deployment where that behaviour is understood and wanted.

Use at your own responsibility. Anyone deploying it is responsible for their own safeguards, output handling, and compliance with the licences above and applicable law. The publisher publishes this artifact as-is, without warranty of any kind, and is not liable for how it is used or for its outputs. The refusal behaviour — and any capability cost of the abliteration — originates entirely upstream; this repository changes precision and format, not behaviour.

Credit for the base model to Qwen (Alibaba Cloud); for the Swift training to UkisAI; for the abliteration style to huihui-ai; for the engine and artifact contract to Neroued; and for the published NVFP4 recipe to unsloth.