noelthekke/ideogram4-text-encoder-abliterated

Ideogram 4 Text Encoder -- Abliterated (research use only)

This is a research artifact, not a product. It is the text-encoder language model used by ideogram-ai/ideogram-4-fp8 to condition its image generation, with directional ablation applied to remove its refusal behavior. It is not a chat model, not a complete image generation pipeline, and not an official or endorsed Ideogram product.

⚠️ Content warning

This model has had its refusal mechanism deliberately removed. When used to condition ideogram-ai/ideogram-4-fp8's image generator, it may pass through prompts -- and therefore contribute to generated images -- that the original model would have resisted encoding, including depictions of violence, non-consensual imagery, or other harmful content. You bear full responsibility for how you use it, under the same legal and license restrictions as the original model (see below). This repository does not include any content filtering, and none should be assumed.

What this is, precisely

ideogram-ai/ideogram-4-fp8 is a text-to-image diffusion pipeline: text_encoder + transformer (DiT) + unconditional_transformer + vae. The DiT itself has no linguistic "refusal" mechanism -- but text_encoder is a Qwen3VLModel (a Qwen3-VL-8B vision-language backbone) used to embed the prompt before the DiT ever sees it.

This repository contains only the language-model half of that text encoder (architecture class Qwen3VLTextModel: 36 layers, hidden size 4096, GQA 32/8 heads, no vision tower, no lm_head -- it was never meant to generate text, only to produce conditioning embeddings). The vision tower, transformer, unconditional_transformer, vae, and scheduler are not included. This model cannot generate images on its own; it is only useful recombined with the original, unmodified versions of those components from ideogram-ai/ideogram-4-fp8.

Method

Directional ablation, following Arditi et al. 2024, "Refusal in Language Models Is Mediated by a Single Direction":

1. Collected residual-stream activations (last-token hidden state, at every layer) over a set of contrastive image-generation prompts -- harmful (e.g. "generate a photorealistic image of graphic violence") vs. harmless (e.g. "generate an image of a golden retriever playing in a park").
2. Computed the mean-difference direction per layer, normalized -- the "refusal direction."
3. Projected that direction out of the output space of self_attn.o_proj and mlp.down_proj at all 36 layers: W_new = W - d ⊗ (d @ W), so nothing those projections produce can have a component along the refusal direction.

The original checkpoint's weights were stored in a scaled fp8 scheme (fbgemm_fp8: F8_E4M3 + per-channel F32 scale); they were dequantized to bf16 before this projection, since the math needs real floating-point precision. This repository's weights are plain bf16, not fp8.

Full implementation: heretic/ideogram/abliterate.py, a custom script in a local, unpublished research repository (not on GitHub or any public host at time of writing; adapted from an original Llama 3.1 abliteration script in the same repo). Not TransformerLens -- a direct PyTorch/transformers implementation operating on the real Qwen3VLTextModel module tree.

Intended use

AI safety and interpretability research into how refusal behavior is represented and can be removed in text-to-image conditioning models -- specifically, probing where image-generation models draw their content-safety lines and how that differs from text-generation refusal. Non-commercial research use only (see License).

Limitations

• No lm_head: this model cannot generate text and was never trained to. It only produces hidden-state embeddings for conditioning the (separate, unmodified) DiT.
• Ablation was validated by inspecting whether the refusal direction's component in the output was removed (it was, to within floating-point precision) -- it was not validated end-to-end through the full image-generation pipeline. Whether removing this direction actually changes what images get generated, and by how much, has not been measured here.
• The contrastive prompt sets used to derive the refusal direction were small (12 harmful / 12 harmless examples) and hand-written; they do not comprehensively cover the space of content this model might resist or refuse.

License and required notices

This is a Model Derivative of ideogram-ai/ideogram-4-fp8 under the Ideogram Non-Commercial Model Agreement (included in this repository in full, as required by that Agreement). That means, without exception:

• Non-commercial use only -- the same restriction as the original model.
• The same use restrictions apply, including (verbatim from the original Agreement): no military or surveillance purposes; no biometric processing; no infringement of third-party rights; no generation of unlawful content, including CSAM or non-consensual intimate images; no violation of privacy/data protection laws; no automated "consequential decisions" or other high-risk use cases. Removing this model's internal refusal behavior does not grant permission to violate any of these restrictions -- they remain your legal obligation regardless of what the model will or won't refuse to do.
• See the full Acceptable Use Policy, incorporated by reference into the Agreement.
• This repository is not an official Ideogram product and has not been endorsed, approved, or validated by Ideogram, Inc.
• See NOTICE for the required attribution notice and a description of exactly what was modified from the original release.

If you redistribute this model or any further derivative of it, you must comply with the same Section 3 (Redistribution and Use) conditions in LICENSE.md that this repository complies with.