Opus4.7-GODs.Ghost.Codex-4B.GGuF

Opus4.7 – GODs.Ghost.Codex 4B is a full-parameter fine-tune of Alibaba's Qwen/Qwen3.5-4B (4B-parameter hybrid multimodal model) performed by WithinUsAI — the publisher's organization under its earlier name — on a sixteen-dataset mixture of frontier reasoning traces, real coding-agent session logs, multi-language code feedback, creative writing, and vision-language instruction data, followed by a directional-ablation (abliteration) pass that cuts measured refusals from 99/100 to 8/100. It is published here by In The Loop Labs (ITLL) as GGUF quantizations for local inference. The goal: a pocket-sized, code-first model that carries the working style of frontier "agentic coding" teachers — step-by-step reasoning, tool-aware thinking, clean refactors — on hardware as small as a laptop.

Fine-tune + ablation	WithinUsAI (full fine-tune, all weights updated — no LoRA or adapters)
Publisher	In The Loop Labs (ITLL, non-profit) — the same organization, rebranded; its dataset Organized_4k_Contex_WIUAI_1.3M still carries the old WIUAI initials
Base model	Qwen/Qwen3.5-4B (post-trained instruct release, Apache 2.0); ablation reference: unsloth/Qwen3.5-4B
Training data	16 listed datasets, ≈938K rows/sessions as listed (938,054 exact-counted across 15; the 16th — a gated <1K corpus — excluded; ≈937.5K unique — details in Section 4)
Decensoring	Directional ablation (abliteration) — KL divergence 0.0361 vs. the stated original; refusals 99/100 → 8/100
Teacher lineage (via data)	Claude Opus 4.6 / Opus 4.5 / Sonnet 4.6 / Haiku 4.5, GPT-5.4 / gpt-5.3-codex / GPT-5.5 / GPT-5.4-mini, GPT-5.1 / GPT-5.2 (prompt generation), Gemini 3 Pro (+ Gemini 3 Flash), "Pony Alpha" (the stealth model that turned out to be GLM-5), Step-3.5-Flash, GPT-4-0613 / GPT-3.5-turbo
Parameters	4B (language model), inherited from base
Architecture	Hybrid Gated DeltaNet + Gated Attention (GQA) — dense FFNs, no MoE at the 4B size
Context length	262,144 tokens native (base spec; extensible to ~1,010,000)
License	Apache 2.0 (inherited from the base model)
Format	GGUF — 3 quantizations (Q4_K_M, Q5_K_M, Q8_0), text-only
Languages	English-focused fine-tune on a base supporting 201

1. Model overview

The name comes from actual training data. GODs is WithinUsAI's own Python_GOD_Coder_50k; Ghost is Royal_Ghost_Coder_501k — together 551K rows, about 59% of the entire mixture; Codex points at the real agentic coding. Opus4.7 refers to the "Opus-style" distillation generation the trace data embodies (Claude Opus 4.6 and adjacent teachers); 4B is the parameter class; GGuF (sic) is the format. The base model is an unusual and strong choice for this size class. Qwen3.5-4B is a hybrid: in each of 32 layers the network mostly uses Gated DeltaNet linear-attention blocks (cheap, constant-memory sequence processing), with every fourth layer a full Gated Attention block using grouped-query attention for precise retrieval.

2. Fine-tuning methodology and ablation

WithinUsAI performed a full fine-tune: every weight of the base network was updated on the sixteen-dataset mixture described below. No LoRA, no adapters, no frozen layers. At 4B parameters this is a meaningful commitment —. The ablation pass. Before the fine-tune, the model went through directional ablation (abliteration): a post-hoc weight edit that finds the refusal direction in activation space and modifies selected projection matrices so the model stops projecting onto it. The repository publishes the full parameter set:

Parameter	Value
direction_index	19.23
attn.out_proj.max_weights (4 matrices)	0: 0.91 · 1: 1.37 · 2: 1.01 · 3: 0.96
attn.out_proj.max_weight_position	22.23
attn.out_proj.min_weights (4 matrices)	0: 0.38 · 1: 0.90 · 2: 0.35 · 3: 0.79
attn.out_proj.min_weight_distance	8.17
mlp.down_proj.max_weights (4 matrices)	0: 1.31 · 1: 0.82 · 2: 0.81 · 3: 1.11
mlp.down_proj.max_weight_position	26.68
mlp.down_proj.min_weights (4 matrices)	0: 0.07 · 1: 0.37 · 2: 0.09 · 3: 0.72
mlp.down_proj.min_weight_distance	4.13
attn.o_proj.max_weights (4 matrices)	0: 1.33 · 1: 1.27 · 2: 0.97 · 3: 1.04
attn.o_proj.max_weight_position	28.05
attn.o_proj.min_weights (4 matrices)	0: 1.31 · 1: 0.93 · 2: 0.24 · 3: 0.24
attn.o_proj.min_weight_distance	10.45

Metric	This model	Original model (unsloth/Qwen3.5-4B)
KL divergence	0.0361	0 (by definition)
Refusals (100-prompt harness)	8/100	99/100

Then trained checkpoint were converted to GGUF in three quantizations.