Nyx Image (NyxImage23M)
Sub-40M (23.19M) Discrete Masked Diffusion Transformer for Edge Pixel Art
Built by Sednium Technologies β’ Licensed under Apache-2.0
Nyx Image (NyxImage23M) is an ultra-compact 23.19 Million parameter discrete masked diffusion transformer engineered to synthesize authentic 16Γ16 retro game sprites directly from text prompts. By eliminating latent autoencoders and continuous VAE bottlenecks, Nyx Image generates discrete pixel palette tokens with precise alpha transparency borders and high-contrast pixel shading in 16 parallel steps.
Key Features
- Direct Discrete Palette Diffusion: Operates natively on 16Γ16 discrete pixel tokens (
vocab_size = 513) with built-in transparent background masking (Index 0). - Frozen CLIP Text Conditioning: Conditioned on frozen sequence embeddings (24 tokens Γ 512-dim) and pooled projections from CLIP ViT-B/32.
- Continuous Color Grounding: Infused with 6-frequency Fourier OKLab color representations, mapping discrete palette IDs into perceptual color space.
- Asymmetric Hourglass Backbone: 2 High-Res blocks (16Γ16) β 2Γ2 Patch Merge β 6 Core Bottleneck blocks (8Γ8 with Conv-SwiGLU) β 2Γ2 Patch Unmerge β 2 High-Res Decoder blocks (16Γ16).
- Edge-Ready Deployments: Available in both native Safetensors (
.safetensors) and universal ONNX (.onnx) formats for sub-millisecond edge and embedded deployment. - Commercial Apache-2.0 License: Complete freedom for commercial games, game jams, indie development, and personal projects.
Technical Comparison: How Nyx Image Compares
A transparent, grounded comparison of Nyx Image against the primary open and commercial approaches used for AI pixel art generation:
| Model / Approach | Architecture | Model Size | Target Output | Native Alpha Transparency | Sampling Steps | Open Weights & License |
|---|---|---|---|---|---|---|
| Nyx Image 23M (Ours) | Discrete Masked Diffusion (MDLM) | 23.19M (~46 MB fp16) | Native $16\times16$ Discrete Palette | Yes (Index 0 true alpha) | 16 parallel steps | Open Weights (Apache-2.0) |
Tessera-Small (thejagstudio) |
Discrete Masked Diffusion (MDLM) | 47.6M (~95 MB fp16) | Native $16\times16$ Discrete Palette | Yes (Index 0 true alpha) | 16β32 parallel steps | Open Weights (CC BY-NC 4.0 Non-commercial) |
SDXL Pixel Art (stablediffusionapi) |
Continuous Latent Diffusion (UNet+VAE) | ~2,600M (2.6B) | $1024\times1024$ Continuous RGB | No (Solid background, color bleed) | 30β50 steps | Open Weights (OpenRAIL++ restrictions) |
| SD 1.5 PixelArt SpriteSheet | Continuous Latent Diffusion (UNet+VAE) | ~860M | $512\times512$ Continuous RGB | No (Requires manual chroma keying) | 25β50 steps | Open Weights (CreativeML OpenRAIL-M) |
| Retro Diffusion (Astropulse) | Proprietary Diffusion + Downsampler | Proprietary (~hundreds of MB) | Multi-resolution Sprite Sheets | Varies (plugin post-processing) | Cloud / API Latency | Closed Source / Paid Commercial Service |
| Discrete Autoregressive (Pixel Transformers) | Autoregressive Sequence Model | ~10Mβ30M | Native $16\times16$ Discrete Tokens | Yes | 256 sequential steps ($O(N)$) | Research Checkpoints / Varies |
What Makes Nyx Image Different & Honest Trade-offs
- True Discrete Tokens vs. Continuous VAEs: Continuous models (SD 1.5, SDXL) generate continuous RGB pixels with anti-aliasing fuzziness, requiring post-process color reduction and downscaling. Nyx Image directly outputs palette indices from step 1, guaranteeing hard pixel boundaries and authentic retro shading.
- 2.05Γ Smaller Footprint than Tessera: By utilizing a 2D patch-merge bottleneck ($16\times16 \to 8\times8 \to 16\times16$) with Conv-SwiGLU, Nyx Image achieves dense semantic representation with only 23.19M parameters compared to Tessera's flat 47.6M architecture.
- Inference-Time Bilateral Symmetrizer: Unlike previous discrete diffusion models that suffer from lopsided limbs or distorted wings, Nyx Image features an inference-time logit mirror-blending mechanism (
symmetry_strength=0.5) that locks bilateral sprites into clean symmetry without retraining. - Universal Edge Deployment: Exported directly to Safetensors and ONNX (Opset 17) with verified sub-millisecond execution for native Rust game engines (
ort/tract) and mobile devices. - Commercial License: Released under permissive Apache-2.0 for complete freedom in indie and commercial game development.
Truthful Limitations: Nyx Image is strictly specialized for $16\times16$ retro game sprites. It cannot generate high-resolution illustrations or larger canvas sizes without multi-tile compositing, and complex multi-subject prompts may lose fine detail within a 256-pixel canvas budget.
Model Weights & Formats
The model weights are provided in three standardized formats for different runtime environments:
| File Name | Format | Size | Target Environment |
|---|---|---|---|
nyx-image-23m.safetensors |
Safetensors | 88.5 MB | PyTorch, Hugging Face safetensors, Python pipelines |
nyx-image-23m.onnx |
ONNX (Opset 17) | 90.4 MB | ONNX Runtime, Rust (ort), C/C++, WebAssembly, Mobile |
nyx-image-23m-fp16.pt |
PyTorch Checkpoint | 46.5 MB | Training resumption, PyTorch evaluation, EMA weights |
palette_513.npy |
NumPy Array | 1.6 KB | 513-entry RGB color lookup table for pixel rendering |
Quickstart & Inference Guide
1. Installation
pip install torch torchvision transformers pillow numpy safetensors onnxruntime
2. Python Inference (PyTorch & Safetensors)
import torch
import numpy as np
from PIL import Image
from transformers import CLIPTokenizer, CLIPTextModel
from safetensors.torch import load_file
from nyx_image import NyxImage23M, ModelConfig, sample_sprites, tokens_to_rgba_image
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
# 1. Load Architecture & Safetensors Weights
config = ModelConfig()
model = NyxImage23M(config).to(device)
weights = load_file("nyx-image-23m.safetensors")
model.load_state_dict(weights, strict=False)
model.eval()
# 2. Load Color Palette
palette = np.load("palette_513.npy")
model.set_palette(palette)
# 3. Encode Prompt with CLIP
prompt = "healing potion bottle with pink sparkling liquid"
tok = CLIPTokenizer.from_pretrained("openai/clip-vit-base-patch32")
text_model = CLIPTextModel.from_pretrained("openai/clip-vit-base-patch32").to(device)
text_model.eval()
inputs = tok([prompt], padding="max_length", max_length=24, truncation=True, return_tensors="pt").to(device)
with torch.no_grad():
text_out = text_model(inputs["input_ids"])
text_tokens = text_out.last_hidden_state # (1, 24, 512)
text_pooled = text_out.pooler_output # (1, 512)
# 4. Generate 16x16 Sprite (16 Steps)
# Set symmetry_strength=0.5 for bilateral symmetry (dragons, monsters, shields, armor)
sampled_tokens = sample_sprites(
model, text_tokens, text_pooled,
steps=16, cfg_scale=3.0, temp_start=0.8, temp_end=0.2, choice_temp=1.0,
symmetry_strength=0.0
)
# 5. Convert to RGBA and Save (Upscaled 8x to 128x128 for viewing)
img = tokens_to_rgba_image(sampled_tokens[0].cpu().numpy(), palette, upscale=8)
img.save("generated_potion.png")
print("Sprite generated successfully!")
3. Edge Inference with ONNX Runtime (Zero-PyTorch Pipeline)
Because Nyx Image is fully exported to ONNX, edge devices can execute the model with zero PyTorch training dependencies:
import onnxruntime as ort
import numpy as np
import math
# 1. Initialize ONNX Runtime Session
session = ort.InferenceSession("nyx-image-23m.onnx", providers=["CPUExecutionProvider"])
palette = np.load("palette_513.npy")
# Inputs expected by ONNX graph:
# - pixel_tokens: (B, 256) int64
# - timesteps: (B,) float32
# - text_tokens: (B, 24, 512) float32
# - text_pooled: (B, 512) float32
# - x0_self_cond: (B, 256, 512) float32
# - drop_text_mask: (B,) bool
def run_onnx_step(tokens, t, text_tokens, text_pooled, sc, drop):
ort_inputs = {
"pixel_tokens": tokens.astype(np.int64),
"timesteps": np.array([t], dtype=np.float32),
"text_tokens": text_tokens.astype(np.float32),
"text_pooled": text_pooled.astype(np.float32),
"x0_self_cond": sc.astype(np.float32),
"drop_text_mask": np.array([drop], dtype=bool)
}
logits = session.run(["logits"], ort_inputs)[0]
return logits
print("ONNX Runtime engine initialized.")
4. Running in Rust (Native Edge Game Engines)
The ONNX model can be embedded directly into native Rust game engines (Bevy, Macroquad, Fyrox) using ort or tract:
// Cargo.toml
// [dependencies]
// ort = "2.0"
use ort::session::Session;
fn main() -> Result<(), Box<dyn std::error::Error>> {
let session = Session::builder()?
.commit_from_file("nyx-image-23m.onnx")?;
println!("Nyx Image ONNX model loaded successfully in native Rust runtime!");
Ok(())
}
Technical Specifications
| Parameter | Specification | Technical Rationale |
|---|---|---|
| Model Architecture | NyxImage23M |
Discrete Masked Diffusion Transformer (MDLM) |
| Total Parameters | 23,191,824 | Optimal sub-40M capacity for detailed edge sprites |
| Canvas Resolution | $16 \times 16$ (256 tokens) | Standard retro game sprite dimensions |
| Vocabulary Size | 513 Tokens | Index 0 = Transparent, 1β511 = OKLab Palette, 512 = [MASK] |
| Hidden Dimension ($d_{\text{model}}$) | 384 | Balanced representational capacity and low FLOPs |
| Attention Heads | 6 heads ($d_{\text{head}} = 64$) | QK-RMSNorm stabilized multi-head attention |
| Bottleneck Structure | $8 \times 8$ Grid (Patch Merge/Unmerge) | Captures macro composition before fine detail decoding |
| Position Encoding | 2D Axial Rotary (RoPE) | Enforces strict Cartesian spatial relationships |
| Text Conditioning | Frozen CLIP ViT-B/32 | 24 token sequence cross-attention + pooled global vector |
| Color Embedding | Fourier OKLab (6 bands) | Continuous perceptual color geometry per token |
| Inference Budget | 16 Steps | Fast parallel unmasking with Spatial NMS |
Training Details
- Dataset:
Angais/pixel-art-atlas-1.4m(1.4M $16 \times 16$ pixel art game sprites) - Total Samples Processed: 6,400,000 sprites (4.57 full passes)
- Hardware: Kaggle Dual Tesla T4 GPUs (Distributed Data Parallel)
- Optimization: Fused AdamW ($\beta_1=0.9, \beta_2=0.95$) with cosine learning rate schedule
- Loss: Focal weighted discrete cross-entropy ($w_0 = 0.15$ on background, $w_k = 1.0$ on foreground)
- Final Convergence: 1.5569 nats (Perplexity: 4.74)
License
Nyx Image is licensed under the Apache-2.0 License.
Copyright 2026 Sednium Technologies
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
β€οΈ Made with Love by Bhoid
"A huge thank you to everyone in the open-source community for 1,000+ downloads on Charon v2! ππ Honestly, I really didn't expect this much love and support on my very first fine-tune. You guys inspire me to keep building, experimenting, and open-sourcing cool AI models like Nyx. Thank you all so much! β¨π"
Dataset used to train Bhoidy/nyx-image-23m
Space using Bhoidy/nyx-image-23m 1
Evaluation results
- Final Loss on Angais/pixel-art-atlas-1.4mself-reported1.5569 nats
- Perplexity on Angais/pixel-art-atlas-1.4mself-reported4.740
- Parameters on Angais/pixel-art-atlas-1.4mself-reported23191824.000
