GlyphAtlas: Disentangling Stroke Detail from Layout for Scene Text Synthesis

Luozheng Qin^1*, Changhai Zhou^2*, Yiyi Zhou^1#

* Equal contribution   # Corresponding author

¹ Xiamen University, ² Fudan University

fr0zencrane@163.com

Overview

GlyphAtlas generates and edits scene text by separating where text belongs from how its strokes are drawn. A scene-aligned glyph image anchors layout and coarse structure, while a dedicated glyph atlas supplies detailed character and word references at an independent resolution. Together, they support small, dense and structurally complex text, multilingual editing, and joint synthesis across multiple regions.

Method

1. Keep layout in the glyph image. The masked scene, edit mask and scene-aligned glyph image provide appearance context, target locations and coarse glyph structure.
2. Put stroke detail in the glyph atlas. Characters and words are rendered in separate cells and encoded with the backbone's frozen VAE. Their reference resolution stays independent of their size in the scene.
3. Bind both spatial streams to the atlas. Atlas-anchored attention lets scene and glyph-image tokens retrieve the cells assigned to their regions. Atlas tokens attend to the prompt and their owning region, preserving correspondence throughout joint generation.
4. Control detail through cell resolution. Randomized atlas heights train the model to use different token budgets. Square cells at 48, 64 and 80 pixels contribute 9, 16 and 25 tokens, respectively.

Model

This repository provides the final GlyphAtlas checkpoint with random atlas heights. The model uses atlas-anchored attention to bind detailed glyph references to both scene and glyph-image tokens.

Setting	Value
Base model	FLUX.1-Fill-dev
Parameters	11.9B, including 593,536 Atlas parameters
Weight precision	BF16
Atlas cell heights	48, 64 or 80 pixels; 64 by default
Default sampling	10 steps, guidance 1
Input canvas	512 × 512, with 1–8 text regions
Weight download	5 safetensors shards, 23.81 GB

Download and Inference

1. Set up the code

Use Linux, Python 3.10 and a BF16-capable CUDA GPU. The setup script installs PyTorch 2.4.0 with CUDA 12.1 and the project dependencies.

git clone https://github.com/Fr0zenCrane/GlyphAtlas.git
cd GlyphAtlas
bash tools/setup_env.sh
source .venv/bin/activate
python tools/bootstrap_upstream.py

Enable access to FLUX.1-Fill-dev on Hugging Face, then authenticate and download the base components and fonts:

huggingface-cli login
python tools/download_assets.py --inference-assets

2. Download GlyphAtlas

Run this command from the code repository:

python - <<'PY_DOWNLOAD'
from huggingface_hub import snapshot_download

snapshot_download(
    repo_id="Fr0zencr4nE/GlyphAtlas",
    local_dir="checkpoints/GlyphAtlas",
)
PY_DOWNLOAD

3. Generate or edit scene text

The example specifies the source image, scene prompt, target text and polygon in examples/edit.json.

python runtime/infer.py \
  --input examples/edit.json \
  --checkpoint checkpoints/GlyphAtlas/transformer \
  --output runs/bookshop \
  --steps 10 --guidance 1 --atlas-mode fixed64

The generated image is saved to runs/bookshop/bookshop/result.png, alongside the glyph condition, edit mask and generation settings. Set --atlas-mode fixed48 for a smaller atlas or --atlas-mode fixed80 for more detailed glyph references.

For your own images, edit the image path, prompt, target strings and polygons in the request JSON. JSON lists and JSONL files support batch inference. See the input format and output guide.

Training

The released model is the final random-resolution distilled checkpoint, saved at refinement step 4,610. Refinement mixes OCR-filtered examples from AnyText-benchmark, TextOCR, Total-Text, ReCTS and STIPLAR with the general scene-text training pool in a 1:1 ratio. The final 994 updates sample atlas heights 48, 64 and 80 with probabilities 0.2, 0.7 and 0.1. The checkpoint and inference settings are recorded in model_info.json.

GlyphAtlas-Data-1.5M provides scene-text data organized by 16 sources. The code README connects data preparation to training, and the training guide covers distributed launch and checkpoint resume.

License

The model weights are distributed under the FLUX.1 [dev] Non-Commercial License. See NOTICE for attribution.

Acknowledgements

We thank the teams behind FLUX.1-Fill-dev, FLUX-Text and PaddleOCR for their work.