gatilin/illumex_vision_flow_io

ILLUME-X: vision, flow interfaces and shared pixel autoencoder

This bundle contains the source-checkpoint's 26-layer packed SigLIP vision tower, connector, visual and latent position embeddings, latent input/output projections and timestep embedder. Every selected tensor is unchanged. The separately included pixel autoencoder is a byte-identical upstream BAGEL dependency, not claimed to be a newly trained ILLUME-X VAE.

The root upstream model card has no full configuration. The native inference builder disables RoPE and removes the final SigLIP layer; the included 27-layer source configuration is therefore instantiated as 26 layers, exactly as the official builder does.

Load

from huggingface_hub import snapshot_download
import sys
folder=snapshot_download("gatilin/illumex_vision_flow_io")
sys.path.insert(0,folder)
from component import load_model,load_vae,vision_features
vision_io=load_model(folder)
pixel_codec=load_vae(folder)

vision_features accepts packed [patches,588] RGB patches (14x14x3, upstream per-patch HWC flattening), corresponding position IDs, and per-image patch counts. It returns 1152-dimensional visual features and 3584-dimensional connector-plus-position features. Preprocessing and task routing must follow the original model; these features are not complete LLM responses.

The pixel codec separately accepts NCHW RGB in [-1,1], downsamples by 8 to 16 channels, and uses the original scale=0.3611 / shift=0.1159 conventions. The native VAE samples the posterior; set the torch RNG seed for reproducibility. Semantic vision features and VAE latents have different roles and must not be interchanged.

Validation and boundaries

Install requirements.txt, then run python validate.py. Validation covers strict loading of all 435 vision/interface tensors and all 244 VAE tensors, independently packed images versus joint packing, latent projection/timestep shapes, position tables, attention segment isolation, 128px VAE reconstruction, and exact scale/shift parity under fixed noise. Complete measured results are in validation.json.

CPU attention uses segmented PyTorch SDPA and preserves input float32 instead of forcing a CUDA BF16 cast. CUDA FlashAttention numerical equivalence, VQA quality, dense-prediction quality and complete interleaved/text-to-image generation have not been benchmarked. The language/MoT backbone and its task-control logic are excluded. This is a reusable component bundle.

Source tensor mapping, per-tensor hashes, pinned source/code revisions, alias/provenance records and VAE cross-source SHA-256 evidence are included. Identical tensor names or shapes across BAGEL derivatives do not establish identical learned values.

License

Weight metadata: apache-2.0. Preserve original per-file code notices and the separate common-autoencoder provenance; see THIRD_PARTY_LICENSES.md. Original source revision: 1d467046cf56fbb33bda7e49619342c9b5b515a3.

Official weights: https://huggingface.co/ChonghuinanWang/ILLUME-X