|
Download README.md from DexagonAI/indepth-10M: direct link, hf CLI and curl.
- Browser
- Download file 8.92 kB
-
https://huggingface.co/DexagonAI/indepth-10M/resolve/main/README.md
- Command line
-
hf download hf://DexagonAI/indepth-10M/README.md
-
curl -L -o README.md https://huggingface.co/DexagonAI/indepth-10M/resolve/main/README.md
8.92 kB
| library_name: pytorch | |
| pipeline_tag: text-to-image | |
| tags: | |
| - pixel-art | |
| - text-to-image | |
| - sprite | |
| - rectified-flow | |
| - diffusion | |
| - game-assets | |
| - dit | |
| datasets: | |
| - Angais/pixel-art-atlas-1.4m | |
| # InDepth (indepth-10M) | |
| **InDepth** is a tiny text-to-sprite model: type a short caption like *"cat wearing green crown"* or *"ghost cat with glowing eyes"* and it paints a **16×16 RGBA pixel sprite** — fully offline, in ~10 seconds on CPU, in any browser. It is the engine behind the single-file [InDepth.html](https://huggingface.co/DexagonAI/indepth-10M/blob/main/InDepth.html) demo page (no build, no libraries, no network). | |
| > InDepth is absolutely not suitable for professional workflows and merely shows how small an image generation model can get. | |
|  | |
| *25 prompts, 16 steps, CFG 1.5, seed 13372000 — straight from the safetensors weights, no cherry-picking.* | |
| ## Model details | |
| | | | | |
| |---|---| | |
| | **Output** | 16×16 RGBA (premultiplied handled at decode; transparent background) | | |
| | **Parameters** | 10,040,336 total — 8,052,112 DiT + 1,463,936 text encoder (incl. 524,288 subword embedding) | | |
| | **Architecture** | DiT (patch 2×2 → 64 tokens, width 256, 8 blocks, 4 heads, adaLN-Zero, 2D sincos positions) | | |
| | **Text encoder** | 8,192-word vocab + hashed FNV-1a char-trigram subwords (4,096 buckets), 2-layer transformer width 128, masked-mean pooling | | |
| | **Flow** | Rectified flow (linear interpolation, uniform velocity target), Euler sampling | | |
| | **Conditioning** | Caption embedding into adaLN; caption dropout 10% during training → CFG at inference | | |
| | **Weights (shipped)** | int8 per-row 90th-percentile quantization + sparse fp32 outlier residuals (9.7 MB on disk, ~48 MB in memory as float32) | | |
| | **Training data** | Captions and sprites built on [Angais/pixel-art-atlas-1.4m](https://huggingface.co/datasets/Angais/pixel-art-atlas-1.4m) plus synthetic compositional prompts | | |
| ## Training | |
| - **Data**: 3,114,709 image–caption pairs — | |
| - 1,409,404 generated sprites for every caption row of the pixel-art-atlas-1.4m dataset, | |
| - 300,000 synthetic compositional prompts (grammar-built *subject + color + accessory* combos, e.g. "frog holding golden sword") with matching images, | |
| - 1,405,305 original dataset images with their captions. | |
| - **Recipe**: 150,000 steps, batch 256, AdamW (lr 2e-4, 2k warmup, cosine decay), bf16 autocast, EMA 0.9999, grad clip 1.0, caption dropout 10%, seed-pinned. | |
| - **Hardware**: single NVIDIA RTX PRO 6000 (Blackwell), ~6,800 samples/s. Wall time ≈ 1.6 h for train + eval. | |
| ## Evaluation | |
| Evaluated against the previous release on 2,000 fresh disjoint prompts and 1,000 held-out compositional prompts (reference images from the training generation pipeline): | |
| | Metric | v2 shipped (500K pairs) | **InDepth v4 (3.11M pairs)** | | |
| |---|---|---| | |
| | Flow loss — fresh prompts | 0.17221 | **0.16428** | | |
| | Flow loss — compositional | 0.20677 | **0.16323** | | |
| | NN-RMSE vs reference — compositional | 92.25 | **87.99** | | |
| | Mask IoU vs reference — compositional | 0.7072 | **0.7584** | | |
|  | |
| *Held-out compositional prompts — reference / v2 / v4. The 3.1M-pair mix closes most of the gap to the reference.* | |
|  | |
| *Attribute binding (v2 left, v4 right): improved, but color→accessory binding is still the weakest skill.* | |
| ## Side-by-side at 16 steps vs same-dataset models | |
| The same 15 prompts against three open pixel-art models trained on the same dataset, each at its shipped sampling settings — [tessera-small](https://huggingface.co/TheJagStudio/tessera-small) (47.6M, 16 steps / CFG 3.0), [UbaNova-16](https://huggingface.co/ubanova/UbaNova-16) (42.4M autoregressive, 256 sequential steps, temperature 1.0 / top-p 0.9 / CFG 3.0) and [nyx-image-23m](https://huggingface.co/Bhoidy/nyx-image-23m) (23.2M discrete masked diffusion, 16 steps / CFG 3.0). InDepth runs at 16 steps / CFG 1.5, seed 13372000: | |
|  | |
| InDepth holds its own against all three: it matches their composition on most prompts (crown, potion, rocket ship, cherry blossom tree, spellbook, crystal, ice cream, skull, cheeseburger, dragon), matches or beats them on color-to-object binding (crown gems, potion, crystal), and adds transparent backgrounds where tessera-small paints opaque fills. tessera-small keeps an edge on subtle shading as the 47.6M original; nyx's 513-color palette gives it richer shading on dense objects; UbaNova-16 draws the sharpest single-object edges. | |
| ### What that costs in parameters | |
| | Model | Image-generation params | Text encoder | Total | | |
| |---|---:|---|---:| | |
| | tessera-small | 47.6M | ~63M frozen CLIP ViT-B/32 | ~111M | | |
| | UbaNova-16 | 42.4M | ~63M frozen CLIP ViT-B/32 | ~105M | | |
| | nyx-image-23m | 23.2M | ~63M frozen CLIP ViT-B/32 | ~86M | | |
| | **InDepth (this model)** | **8.05M** | **1.46M trained from scratch** | **10.04M** | | |
| Every rival leans on a ~63M-parameter frozen CLIP text tower. InDepth's text encoder is trained from scratch in 1.46M parameters — its whole 10.04M budget is about 8.6× smaller than the next-smallest competitor's total (~86M), and its image-generation stack alone (8.05M) is 2.9× smaller than nyx's generator. InDepth is also the only one of the four that runs as a pure-JS in-browser engine with no install. | |
| ## Usage | |
| ### Python (PyTorch) | |
| ```python | |
| from run_indepth import main # see repo: CLI, loads weights + vocab automatically | |
| main() # or: | |
| ``` | |
| ```bash | |
| pip install torch safetensors pillow huggingface_hub | |
| python run_indepth.py "cat wearing green crown" --steps 4 --cfg 1.5 --seed 13372000 --out sprites/ | |
| ``` | |
| Or programmatically: | |
| ```python | |
| import torch | |
| from modeling_indepth import IndepthConfig, IndepthModel, generate, load_indepth, decode_to_rgba | |
| from tokenizer import IndepthTokenizer | |
| model, config, directory = load_indepth("DexagonAI/indepth-10M") # or a local path | |
| tokenizer = IndepthTokenizer(directory) | |
| tokens, subwords = [torch.tensor(r, dtype=torch.long) for r in tokenizer.batch(["cat wearing green crown"])] | |
| image = generate(model, config, tokens, subwords, seed=13372000, steps=4, guidance=1.5, device="cpu") | |
| sprite = decode_to_rgba(image)[0] # (16, 16, 4) uint8 — save with PIL, scale x8 NEAREST for viewing | |
| ``` | |
| - `steps`: 1–16 (4 is the sweet spot; 1-step is bit-exact with the embedded PyTorch reference). | |
| - `guidance`: 1.0 = off (wild), 1.5 = default, 2.0–2.5 = stronger prompt adherence for two-attribute prompts. | |
| - Prompts: lowercase, short works best; unknown words still paint something (character-trigram fallback). | |
| ### Browser (no install) | |
| Download **[InDepth.html](https://huggingface.co/DexagonAI/indepth-10M/blob/main/InDepth.html)** (24 MB, single file) and open it in any modern browser — it ships the same int8 weights, a pure-JS inference engine, a live step-by-step "watch it paint" view, and a PyTorch parity self-test that runs on load (1-step bit-exact, 4-step ≤ 12/255 float32-vs-float64 drift). Works offline; runs in ~48 MB of memory. | |
| ### File map | |
| | File | Purpose | | |
| |---|---| | |
| | `model.safetensors` | EMA weights, float32, 127 tensors, 10,040,336 params | | |
| | `config.json` | Architecture hyperparameters | | |
| | `vocab.json` | 8,192-word vocabulary (+ pad/unk) and tokenizer settings | | |
| | `modeling_indepth.py` | Self-contained model definition + CFG sampler | | |
| | `tokenizer.py` | Word + FNV-1a trigram tokenizer (bit-exact with the JS engine) | | |
| | `run_indepth.py` | CLI / library generation script | | |
| | `InDepth.html` | The complete offline browser demo (weights embedded) | | |
| | `samples/` | Sample grids used in this card | | |
| ## Limitations | |
| - **Attribute binding**: two-attribute prompts ("X wearing C Y") improved substantially over the previous release but are not solved — colors sometimes bind to the whole subject rather than the accessory. | |
| - **16×16 only**: no resolution flexibility; details are limited by 256 pixels per sprite. | |
| - **Short captions**: the 2-layer text encoder saturates beyond ~7–8 words. | |
| ## Credits | |
| Model and InDepth page by **Zwiq**. Caption dataset: [Angais/pixel-art-atlas-1.4m](https://huggingface.co/datasets/Angais/pixel-art-atlas-1.4m). | |
| Models compared on this card (both Apache-2.0, trained on the same dataset): | |
| - [nyx-image-23m](https://huggingface.co/Bhoidy/nyx-image-23m) by Sednium Technologies ([@Bhoidy](https://huggingface.co/Bhoidy)) | |
| - [UbaNova-16](https://huggingface.co/ubanova/UbaNova-16) by Muhammad Fahril Irkham ([@UbaNova](https://huggingface.co/ubanova)) | |
| ## Reproducibility | |
| Default seed 13372000. The `InDepth.html` page self-verifies on load: 1-step no-CFG sampling must be bit-exact (max diff 0/255) against a PyTorch float64 reference embedded at export; the 4-step CFG path documents its float32 drift (≤ 12/255). The browser engine and the PyTorch tokenizer share identical FNV-1a trigram hashing, verified bit-exact. | |