Asuka-FM: 200M Dual-Stream SPRINT DiT (Flow Matching)

An ultra-efficient, 200M-parameter Dual-Stream Diffusion Transformer (DiT) trained from scratch on consumer hardware (1x NVIDIA RTX 4090 in ~200 hours) using Rectified Flow / Flow Matching, SPRINT token-dropping, and Continuous Aspect-Normalized RoPE.

Despite its compact size, the model generates high-fidelity anime art with accurate character reproduction, prompt alignment, and zero-shot camera control (panning and zoom) at inference time.

Developed by: aipracticecafe
Repository: deeplearningcafe/asuka-fm
Dataset: curated-danbooru-2026 (~337K samples)
Architecture: Dual-Stream DiT (SPRINT-aligned, SwiGLU, Sandwich RMSNorm)
Parameters: ~200M (Backbone) + 230M (Text Encoder)
Objective: Conditional Flow Matching (Logit-Normal timestep sampling)
License: OpenRAIL++

Prompt

1girl, souryuu asuka langley, neon genesis evangelion, sensitive, masterpiece, good score, absurdres, highres, best quality, amazing quality, great quality, very aesthetic, year 2025, year 2026, newest, blue eyes, blurry, bodysuit, breasts, contrapposto, depth of field, hand on own hip, interface headset $evangelion$, long hair, looking at viewer, mecha pilot suit, parted lips, plugsuit $evangelion$, red bodysuit, red hair, simple background, solo, standing, white background, cowboy shot, small breasts, plugsuit, interface headset

Prompt

1girl, hoshino ai, oshi no ko, ebifurya, sensitive, masterpiece, good score, absurdres, highres, best quality, amazing quality, great quality, dress, gloves, hair ornament, idol, idol clothes, long hair, looking at viewer, microphone, one eye closed, open mouth, pink dress, pink gloves, pointing, purple eyes, purple hair, solo, star-shaped pupils, star, star in eye, symbol-shaped pupils, symbol in eye, anime coloring, black shirt, brooch, frilled gloves, frilled shirt, frilled sleeves, frills, hair between eyes, heart, heart brooch, jewelry, messy hair, one side up, pink shirt, rabbit hair ornament, shiny clothes, shiny skin, shirt, short sleeves, solo, star $symbol$, turtleneck, wing brooch, wings, yellow shirt, upper body, simple background

Prompt

1girl, souryuu asuka langley, neon genesis evangelion, rebuild of evangelion, evangelion: 3.0 you can $not$ redo, sensitive, masterpiece, good score, absurdres, highres, best quality, amazing quality, great quality, very aesthetic, year 2025, year 2026, newest, long hair, blue eyes, orange hair, breasts, alternate costume, blush, parted lips, alternate hairstyle, high updo, pendant choker, black straps, white sleeveless, off-shoulder dress, grey plaid pleated microdress, belt, parted lips, arms under breasts, cleavage, crossed arms, crossed legs, expressionless, looking at viewer, simple background, sitting, thighhighs, thighs, white background, chair, throne

Prompt

1girl, souryuu asuka langley, neon genesis evangelion, rebuild of evangelion, evangelion: 3.0 you can $not$ redo, tomose shunsaku, fujima takuya, sensitive, masterpiece, good score, absurdres, highres, best quality, amazing quality, great quality, very aesthetic, year 2025, year 2026, newest, solo, alternate costume, looking at viewer, portrait, blush, bow, bowtie, closed mouth, collared shirt, flower, hair flower, hair ornament, jacket, looking at viewer, red bow, red bowtie, school uniform, shirt, simple background, tokiwadai school uniform, white shirt, wing collar, winter uniform, yellow jacket

Prompt

1girl, souryuu asuka langley, neon genesis evangelion, rebuild of evangelion, evangelion:3+1.0 thrice upon a time, b-ginga, sensitive, masterpiece, good score, absurdres, highres, best quality, amazing quality, great quality, very aesthetic, year 2025, year 2026, newest, long hair, blue eyes, orange hair, breasts, official alternate costume, blush, parted lips, black eyepatch, plugsuit, white bodysuit, gloves, interface headset $evangelion$, plugsuit $evangelion$, mecha pilot suit, white gloves, interface headset, multicolored clothes, one eye covered, bodysuit, covered navel, full body, eyepatch, from side, multicolored bodysuit, thighs, turtleneck bodysuit, floating hair, closed mouth, full moon, moon, night, outdoors, sitting, red rose, flower, dutch angle, perspective, virtual youtuber, wariza

Prompt

1girl, emilia $re:zero$, re:zero kara hajimeru isekai seikatsu, sensitive, masterpiece, good score, absurdres, highres, best quality, amazing quality, great quality, very aesthetic, year 2025, year 2026, newest, floating hair, flower, cowboy shot, hair flower, hair ornament, long hair, looking at viewer, miniskirt, parted lips, pleated skirt, purple eyes, purple hair, skirt, solo, thighhighs, very long hair, white flower, white skirt, white thighhighs, zettai ryouiki, breasts, ribbon, medium breasts, braid, hair ribbon, pointy ears, hand on own chest, purple ribbon, simple background

Prompt

1girl, frieren, sousou no frieren, tomose shunsaku, fujima takuya, sensitive, masterpiece, good score, absurdres, highres, best quality, amazing quality, great quality, very aesthetic, year 2025, year 2026, newest, belt, black belt, breasts, capelet, closed mouth, earrings, elf, green eyes, grey hair, jewelry, long hair, looking afar, parted bangs, pointy ears, shirt, small breasts, solo, twintails, upper body, white capelet, white shirt, white background, striped clothes, dangle earrings

Prompt

1girl, hozuki kaede, onii-chan wa oshimai!, sensitive, masterpiece, good score, absurdres, highres, best quality, amazing quality, great quality, very aesthetic, year 2025, year 2026, newest, bead necklace, beads, blue cardigan, blue hair, blue skirt, bracelet, breasts, bursting breasts, cardigan, cardigan around waist, cleavage, clenched hand, clothes around waist, collarbone, gyaru, hair between eyes, hair ribbon, heart, jewelry, kogal, large breasts, multicolored hair, necklace, open mouth, pink hair, ponytail, red eyes, ribbon, school uniform, scrunchie, shirt, sidelocks, simple background, skirt, smile, solo, standing, streaked hair, thighs, two-tone hair, white background, white shirt, wrist scrunchie, looking at viewer, blush, hair ornament, bow, short sleeves, :d, hair bow, cowboy shot, earrings, hand up, bowtie, miniskirt, sweater, side ponytail, pink bow, dot nose, bead bracelet

Pre-Release Notice & Research Documentation: This repository is a technical pre-release of the Asuka-FM anime foundation checkpoints.

Comprehensive Technical Article (Japanese): A complete, deep-dive engineering breakdown covering the mathematical foundations, SPRINT sequence sparsification, continuous RoPE derivations, and the real-world pitfalls of pixel diffusion is published on Zenn:
👉 Asuka-FM：RTX 4090 1枚でゼロから作る画像生成AI

Upcoming arXiv Paper: The formal systems-architectural paper detailing benchmarks (GenEval / DPG-Bench), ablation studies, and open-domain photorealistic checkpoints will be released shortly on arXiv.

Key Architecture & Features

1. SPRINT Dual-Stream Transformer Backbone

The base architecture is based on the results from (i1: A Simple and Fully Open Recipe for Strong Text-to-Image Models)[https://arxiv.org/abs/2606.11289]

Minimalist Conditioning: Uses prepended time tokens rather than heavy per-block AdaLN modulation, reducing parameter overhead.
Sandwich Normalization & SwiGLU: RMSNorm placed before and after attention and FFN operations ensures training stability in bfloat16.
SPRINT Acceleration: Employs structured group-wise 2x2 patch dropping (75% drop ratio during early pretraining), cutting compute demands by half while retaining global semantic context.

2. HDM Continuous Aspect-Normalized RoPE (Camera Control)

Based on the Patch Diffusion: Faster and More Data-Efficient implementation proposed in the HDM paper. Early experiments with discrete integer position IDs caused severe multi-head replication when deviating from trained aspect ratios. We resolve this by adopting continuous aspect-normalized coordinate mapping which was the original idea proposed in HDM:

$r_H = \sqrt{\frac{H}{W}}, \quad r_W = \sqrt{\frac{W}{H}}$

$y_{\text{pos}} = \left(\frac{y_{\text{center}}}{H}\right) \cdot (2 r_H) - r_H, \quad x_{\text{pos}} = \left(\frac{x_{\text{center}}}{W}\right) \cdot (2 r_W) - r_W$

Manipulating $(x_{\text{pos}}, y_{\text{pos}})$ directly controls the visual framing at inference time:

X-Shift / Y-Shift: Translates the virtual camera viewport horizontally or vertically.
Zoom: Scales spatial coordinates around the focal center.

3. Components

Text Encoder: LiquidAI/LFM2.5-230M (unidirectional causal LM hidden states passed via a 2-layer Transformer Text Adapter).
Autoencoder: kaiyuyue/FLUX.2-dev-vae (C=32, f=8, patch size p=2, effective image token size 16x16).

Figure: Cherry picked samples

Training Recipe & Multi-Stage Progression (1x RTX 4090, ~100h)

The model uses progressive multi-resolution pretraining on 337,000 curated Danbooru samples with shifted square cropping and in-RAM latent caching.

Optimizer: 8-bit AdamW (bitsandbytes), $\beta=(0.9, 0.95)$, WD=0.01
Learning Rate: 5e-4 with warmup and cosine annealing only for last stage
Effective Batch Size: 256 (Batch size 64x4 gradient accumulation)
Precision: Mixed Precision (torch.bfloat16 with torch.compile)

Latent Pretraining Progression (Stages 1 – 3)

Stage / Phase	Res.	Drop ($r$)	Samples	Throughput	MFU (Act/Eff)	Exec Compute	Time	Cost
1A: SPRINT Sparse	$256^2$	0.75	33.70M	334.3 img/s	74.4% / 86.8%	7.92 EFLOPs	28.0 h	$9.24
1B: Dense Refine	$256^2$	0.00	16.85M	235.1 img/s	70.4% / 70.4%	8.41 EFLOPs	20.0 h	$6.60
1C: WSD Cooldown	$256^2$	0.00	8.43M	235.1 img/s	70.4% / 70.4%	4.21 EFLOPs	10.0 h	$3.30
2A: High-Res Sparse	$512^2$	0.75	8.43M	108.5 img/s	58.5% / 68.3%	8.23 EFLOPs	33.0 h	$10.89
2B: Dense Finetune	$512^2$	0.00	3.37M	70.9 img/s	64.0% / 64.0%	5.02 EFLOPs	13.0 h	$4.29
2C: WSD Cooldown	$512^2$	0.00	1.69M	70.9 img/s	64.0% / 64.0%	2.34 EFLOPs	6.0 h	$1.98
3A: 1MP SPRINT Sparse	$1024^2$	0.75	1.01M	7.80 img/s	12.4% / 25.7%	2.61 EFLOPs	36.0 h	$11.88
3B: 1MP Dense Refine	$1024^2$	0.00	1.35M	7.11 img/s	24.6% / 24.6%	7.03 EFLOPs	53.0 h	$17.49
Cumulative Subtotal (512px)	—	—	72.47M	—	52.1% / 79.5%	36.13 EFLOPs	110.0 h	$36.30
Cumulative Grand Total (1MP)	—	—	74.83M	—	41.4% / 65.2%	45.77 EFLOPs	199.0 h	$65.67

Latent-to-Pixel Adaptation (DiP Head, Zero-VAE Decode)

To eliminate the autoencoder bottleneck during few-step sampling and downstream reinforcement learning (Flow-GRPO), the converged 512px latent foundation was adapted to RGB pixel space via a 10.09M Conv U-Net Patch Detailer Head (PDH) under native $v$-prediction:

Adaptation Phase	Data Target	Drop ($r$)	Samples	Throughput	MFU (Act)	Wall Time	Cost
Phase 1: PDH Warmup	Synth (Frozen DiT)	0.75	0.50M	61.8 img/s	72.8%	2.25 h	$0.74
Phase 2A: Joint Sparse	Synth (Unfrozen)	0.75	2.50M	57.7 img/s	67.8%	12.0 h	$3.98
Phase 2B: Dense Refine	Synth (Unfrozen)	0.00	1.50M	44.4 img/s	69.8%	9.4 h	$3.10
Phase 3: Real Data SFT	Curated Real 396k	0.00	5.16M	42.1 img/s	66.2%	34.0 h	$11.22
Total Pixel DiT Transfer	—	—	9.66M	46.5 img/s	67.5%	57.65 h	$19.04

Checkpoint Guide

Checkpoint Directory	Architecture	Target Res	Recommended Use Case
`asuka-1024-1mp`	Latent (f=8, C=32)	$1024 \times 1024$	Native megapixel generation. Eliminates upscale artifacts on intricate backgrounds.
`asuka-1024-pixel-dip`	Pixel DiT (+10M PDH)	$1024 \times 1024$	Zero VAE decode. Decisively faster in few-step regimes ($N \le 8$, up to $3.34\times$ speedup at 1 step) and optimal for online RL rollouts (Flow-GRPO).

The Pixel DiT model permanently eliminates the terminal VAE decode, but requires evaluating the convolutional PDH head at every sampling step. For multi-step trajectories ($N \ge 13$), latent ODE integration followed by a single terminal VAE decode is faster. Choose asuka-1024-pixel-dip specifically for few-step distilled sampling ($N \le 8$) or online RL.

Inference and WebUI

The training repository has a simple Gradio UI to perform sampling with camera controls. ui_app.py

Recommended Settings

Sampling Algorithm: Forward Euler ODE Solver (Rectified Flow)
Inference Steps: 75
CFG Scale: 9.0
Time Shift: 3.0 (higher values enhance contrast/saturation)

Tag Formatting Template

1girl/1boy, character name, series name, artists, rating, quality tags, actions/clothing

Negative prompt:

sketch, monochrome, english text, realistic, traditional media, very displeasing, displeasing, bad score, worse score, bad quality,worst quality, worst detail, watermark, signature, blurry, bad hands, bad anatomy, bad fingers, extra fingers, extra limbs, deformed limbs, comic, speech bubble, lowres

Python Sampling Script

import torch
from omegaconf import OmegaConf
from src.diffusion.sampling import generate_samples
from src.diffusion.schedules import LinearSchedule
from src.models.factory import load_trainable_model

# 1. Configuration & runtime parameters
cfg = OmegaConf.load("config.yaml")
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
dtype = torch.bfloat16

# Select mode: Set True for Zero-VAE Pixel DiT, False for Latent DiT
use_pixel_mode = False
checkpoint_path = (
    "checkpoints/asuka-512-pixel-dip"
    if use_pixel_mode
    else "checkpoints/asuka-512-latent"
)

# 2. Instantiate backbone and conditional encoders
model, text_encoder, vae, tokenizer, _ = load_trainable_model(
    models_path=cfg.paths.models,
    device=device,
    dtype=dtype,
    resume_from_checkpoint=checkpoint_path,
    model_type="sprint_dual_pixel" if use_pixel_mode else "sprint_dual",
    model_cfg=cfg.models,
    autocast_dtype=dtype,
)

schedule = LinearSchedule(device=device)

# 3. Generation parameters
# In few-step regimes (N <= 8), pixel mode is up to 3.34x faster.
# In multi-step regimes (N >= 13), latent mode is faster due to VAE amortization.
sample_configs = [
    {
        "prompt": (
            "1girl, souryuu asuka langley, neon genesis evangelion, "
            "masterpiece, absurdres"
        ),
        "negative_prompt": "very displeasing, displeasing, bad score",
        "height": 512,
        "width": 512,
        "sample_steps": 4 if use_pixel_mode else 30,
        "cfg_scale": 6.0,
        "shift": 1.0,
        "zoom": 1.0,
        "x_shift": 0.0,
        "y_shift": 0.0,
        "seed": 42,
    }
]

# 4. Execute sampling trajectory
images = generate_samples(
    unet=model,
    text_encoder=text_encoder,
    tokenizer=tokenizer,
    vae=None if use_pixel_mode else vae,
    schedule=schedule,
    sample_configs=sample_configs,
    global_batch_size=1,
    diffusion_type="flow_matching",
    device=device,
    dtype=dtype,
    autocast_dtype=dtype,
    use_unet_mult=False,
    vae_mean=cfg.models.vae_mean,
    vae_std=cfg.models.vae_std,
    in_channels=cfg.models.in_channels,
    coord_system="aspect_norm",
    pixel_space=use_pixel_mode,
)

images[0].save("output.png")

Content Advisory & Limitations

Complex Hands: The compact 200M parameter count prioritizes prompt faithfulness and character identity; complex hand interactions may still require multiple seed attempts.
NSFW & Sensitive Content: The model was trained on a dataset that contains uncensored anime-style art, including sensitive, questionable, and explicit material (g, s, q, e ratings). Use appropriate tag filtering during training if a SFW-only model is required.
Legal Disclaimer: Images are sourced from Danbooru for research, educational, and generative modeling purposes under fair use. Copyright of all underlying illustrations belongs to their respective creators.

References

i1: i1: A Simple and Fully Open Recipe for Strong Text-to-Image Models
Patch Diffusion: Patch Diffusion: Faster and More Data-Efficient
Home-made Diffusion Model from Scratch to Hatch: HDM
Danbooru Dataset Toolkit: Dataset processed and formatted using the Danbooru Dataset Toolkit.

License

This model is licensed under the CreativeML Open RAIL++-M License. You are free to use, modify, and distribute the model weights provided you include the license text and respect the behavioral use restrictions.

Downloads last month: -

Datasets used to train huangdffs/asuka-fm

Papers for huangdffs/asuka-fm