| --- |
| license: apache-2.0 |
| tags: |
| - image-tokenizer |
| - autoencoder |
| - latent-diffusion |
| - imagenet |
| datasets: |
| - ILSVRC/imagenet-1k |
| --- |
| |
| # InterpTok |
|
|
| Weights of **Image Tokenizers Should Be Trained Between Images**, by Ole Beißwenger, Jan-Niklas Dihlmann and Hendrik Lensch, University of Tübingen. |
|
|
| [📄 Paper](https://arxiv.org/abs/2610.16759) | [💻 Code](https://github.com/cgtuebingen/InterpTok) | [🌐 Project Page](https://graphics.cs.uni-tuebingen.de/papers/interptok/) |
|
|
| InterpTok fine-tunes the image tokenizer [MAETok](https://huggingface.co/MAETok/maetok-b-128) to decode mixtures of the codes of two images into natural images. |
| On ImageNet at 256 × 256, it keeps MAETok's reconstruction quality and lowers its interpolation FID from 14.42 to 4.30. |
| A SiT-B trained on its codes improves its generation FID without guidance from 11.61 to 8.48. |
|
|
| | File | Model | |
| |---|---| |
| | `interptok_41k.safetensors` | InterpTok, the model of the paper, 41,000 steps | |
| | `interptok_82k.safetensors` | InterpTok trained further, to 82,000 steps | |
| | `interptok_decoder_41k.safetensors` | InterpTok's decoder on MAETok's frozen encoder, which decodes MAETok's codes and the samples of generators trained on them | |
| | `sit-b_interptok-41k/` | SiT-B trained for 400,000 steps on the codes of `interptok_41k` | |
| | `sit-b_maetok/` | SiT-B trained for 400,000 steps on MAETok's codes, the baseline of the paper | |
|
|
| Each tokenizer holds 217M parameters in float32, MAETok-B-128's encoder and a decoder of 18 blocks, and encodes a 256 × 256 image into a code of 128 tokens with 32 values each. |
| Each SiT-B folder is an experiment of the code of [Xu et al.](https://github.com/tongdaxu/Making-rFID-Predictive-of-Diffusion-gFID): `args.json`, the moving average of the weights in `checkpoints/0400000.pt`, and the configuration of its tokenizer. |
| `SHA256SUMS.txt` lists the checksum of every file. |
|
|
| ## Usage |
|
|
| ```bash |
| git clone --recursive https://github.com/cgtuebingen/InterpTok |
| hf download CGTuebingen/InterpTok-Models --local-dir InterpTok/weights |
| cd InterpTok |
| python src/interpolate.py a.jpg b.jpg --weights weights/interptok_41k.safetensors --out a_to_b.png |
| ``` |
|
|
| The [code repository](https://github.com/cgtuebingen/InterpTok) explains installation, training and evaluation. |
|
|
| ## Citation |
|
|
| ```bibtex |
| @article{beisswenger2026interptok, |
| title = {Image Tokenizers Should Be Trained Between Images}, |
| author = {Bei{\ss}wenger, Ole and Dihlmann, Jan-Niklas and Lensch, Hendrik}, |
| journal = {arXiv preprint arXiv:2610.16759}, |
| year = {2026} |
| } |
| ``` |
|
|
| ## License |
|
|
| Apache License 2.0. |
|
|
