RAY-IMAGE DiT v5 MoE — Milestone checkpoint archive
Maintainer: Rishix500
Main v5 repository: ray-image-dit-v5-moe
Project documentation: RAY-IMAGE GitHub README
Visibility: Private repository
Purpose
This repository preserves selected RAY-IMAGE DiT v5 sparse-MoE training milestones. It is an archive of checkpoint snapshots, separate from the main v5 repository's rolling set of recent checkpoints.
Architecture reference
v5 is an experimental 512 × 512 text-to-image Diffusion Transformer trained with flow matching / velocity prediction. It uses a custom VAE v3 latent path (16 × 64 × 64), frozen Qwen3-4B text embeddings (128 × 2,560), and sparse MoE feed-forward layers with one shared expert plus eight routed SwiGLU experts using top-2 routing. The configuration is approximately 482M total parameters and 204M active parameters per token.
Files present
The authenticated repository inventory checked on 6 November 2026 contained both full and EMA-only files for these steps:
| Step | Full checkpoint | EMA checkpoint |
|---|---|---|
| 130,000 | Present | Present |
| 140,000 | Present | Present |
| 150,000 | Present | Present |
| 160,000 | Present | Present |
| 170,000 | Present | Present |
Full checkpoints are approximately 7.72 GB each; EMA-only checkpoints are approximately 1.93 GB each.
How to use this archive
Select a milestone that matches the exact model configuration and code revision used for that run. Full checkpoints are intended for training resumption and contain more state than an EMA-only inference candidate. EMA-only files are intended for sampling with the matching RAY-IMAGE model and VAE implementation.
These files are serialized PyTorch checkpoints, not standalone Diffusers pipelines. This repository does not contain the model source code, tokenizer, Qwen3-4B encoder, matching VAE weights, or a general-purpose inference interface.
Research status and limitations
The v5 run was warm-started from v4's 120k checkpoint. Training logs show modest loss improvement through the reported ~170k region, balanced routing diagnostics, and non-zero capacity drops. Those metrics do not prove improved image quality. Use identical prompts, seeds, sampling steps, CFG scale, and VAE when comparing milestones.
No formal FID/CLIP benchmark or verified sample gallery is supplied by this archive. Repository file presence was checked; checkpoint deserialization and inference were not performed as part of that check.