---
license: apache-2.0
library_name: pytorch
pipeline_tag: image-to-image
tags:
- image-tokenizer
- binary-tokenizer
- bitdance
- component-extraction
base_model: shallowdream204/BitDance-Tokenizer
---
# BitDance d32c128 binary image tokenizer

Standalone native encoder, sign quantizer and GAN decoder from the official BitDance tokenizer release. The safetensors file is byte-for-byte identical to `ae_d32c128.safetensors`; its original SHA-256 and pinned source revision are recorded in `extraction.json`. No generation backbone is included.

This is the published **BitDance** codec. It is not asserted to be the semantic-distilled UniWeTok checkpoint merely because both describe 128-bit representations.

## Load and reconstruct

```python
from huggingface_hub import snapshot_download
import sys, torch
folder = snapshot_download("gatilin/bitdance_binary_tokenizer_d32c128")
sys.path.insert(0, folder)
from component import load_model
codec = load_model(folder)
# RGB pixels: NCHW, [-1, 1], spatial dimensions divisible by 32.
pixels = torch.linspace(-1, 1, 3*256*256).reshape(1,3,256,256)
with torch.inference_mode():
    signs = codec.encode(pixels)  # [1,128,8,8], each value -1 or +1
    reconstructed = codec.decode(signs)
```

Install `requirements.txt` and run `python validate.py` in the downloaded directory. `component.py` also provides explicitly documented byte-packing helpers for storing 128 sign bits. Their byte convention is a portable bundle helper, **not an inferred mapping to upstream AR vocabulary IDs**.

## Validation

Strict native state-dict loading, official-file SHA-256 equality, 256px encode/decode, binary alphabet, exact bit roundtrip, fixed-seed bit-decoding parity, native stochastic decoding, and rectangular 256x320 encode/decode passed on CPU in float32. Exact results and measured fixture MSE are in `validation.json`. These are component correctness tests on deterministic synthetic inputs, not claims about perceptual quality or text-to-image benchmark scores.

The GAN decoder injects fresh Gaussian noise even in eval mode. Set the same torch RNG seed before decoding if you need reproducible output; this native behavior is preserved.

The 2^128 space is represented implicitly by 128 binary channels, not a materialized embedding table. The published model uses the original convolutional implementation in `autoencoder.py`, not the separate attention-augmented `autoencoder_uni.py`. The only native-file edit changes an absolute filesystem-helper import into a package-relative import; source and code revisions are in the provenance files.

## Source and license

- Weights: https://huggingface.co/shallowdream204/BitDance-Tokenizer
- Code: https://github.com/shallowdream204/BitDance
- Paper: https://arxiv.org/abs/2602.14041

Apache-2.0; original notices are retained. Extraction, tests, loader and documentation added for this standalone distribution.