| --- |
| license: mit |
| library_name: pytorch |
| pipeline_tag: image-to-image |
| base_model: ByteVisionLab/TokenFlow |
| tags: |
| - tokenflow |
| - image-tokenizer |
| - unified-multimodal |
| - component-extraction |
| --- |
| # TokenFlow CLIP-B 32K with enhanced RGB decoder |
|
|
| The complete official `tokenflow_clipb_32k_enhanced.pt` model mapping: trained CLIP-B/16 semantic encoder, pixel encoder, both codebooks, multi-scale residual quantizer, semantic decoder and enhanced RGB decoder. Source weights: [ByteVisionLab/TokenFlow](https://huggingface.co/ByteVisionLab/TokenFlow/tree/5924ad7f25fe030292de3995912b895ab9fd9321). All 649 tensor entries retain original names, dtypes and values. |
|
|
| ```python |
| from huggingface_hub import snapshot_download |
| import sys, torch |
| folder = snapshot_download('gatilin/tokenflow_clipb_32k_visual_codec') |
| sys.path.insert(0, folder) |
| from component import load_model |
| codec = load_model(folder) |
| x = torch.zeros(1, 3, 256, 256) # RGB NCHW in [-1, 1] |
| with torch.inference_mode(): |
| quantized, losses, ids = codec.encode(x) |
| semantic_features, reconstructed_rgb = codec.decode_code(ids) |
| ``` |
|
|
| This release follows the official enhanced-decoder reconstruction script with `teacher=clipb_224`, `infer_interpolate=True`, and 32768 codes. Scales are `[1,2,4,6,8,10,12,14,16]`; IDs concatenate the square grids in that order. Both semantic and pixel codebook vectors are required to reconstruct the residual representation. The trained CLIP visual weights are already included; the adapter constructs the exact original visual architecture and strictly loads them without an extra CLIP weight download. |
|
|
| Native classes are pinned to source revision `9812da01c0695f4989060cc4eea053715e9e4441`. The two adaptations are documented: parameter device/dtype helper compatibility, and construction of the already-checkpointed CLIP branch without its redundant network download. Original tensor computation classes are retained. See `source_reconstruction.sh`, `code_provenance.json` and `validation.json`. |
|
|
| Validation covers all source tensor payload hashes, strict loading, nine-scale ID geometry and bounds, residual lookup reconstruction, finite semantic/RGB outputs, and exact decode-code parity with canonical lookup decoding. Synthetic reconstruction does not establish image quality or full MLLM/VQA/text-to-image performance. Weight-card MIT terms and source-code Apache-2.0 terms are recorded separately in `LICENSE` and `CODE_LICENSE`; original copyright notices remain in the native files. |
|
|
| `CLIP_LICENSE` separately preserves the original OpenAI MIT license and attribution for the bundled CLIP-derived visual transformer. |
|
|
