| --- |
| library_name: diffusers |
| pipeline_tag: any-to-any |
| license: other |
| license_name: component-specific-licenses |
| license_link: https://huggingface.co/onkarsus13/Unifiedweights/blob/main/LICENSES/README.md |
| language: |
| - en |
| tags: |
| - unifiedflow |
| - text-to-image |
| - image-to-text |
| - text-generation |
| - safetensors |
| --- |
| |
| # UnifiedFlow weights |
|
|
| A self-contained UnifiedFlow pipeline for text-to-image and image-to-text |
| inference, with a separate text-pretraining variant in `pretrain/`. |
|
|
| The shared DiT models image and text embeddings. TextEncoder supplies text |
| embeddings, ColaDecoder produces tokens, and the frozen image codec converts |
| between image embeddings and pixels. The image output resolution is 224 × 224. |
|
|
| ## Download and install |
|
|
| The repository includes the Python implementation required to load these custom |
| components. Authenticate with `hf auth login` if the repository is private. |
|
|
| ```bash |
| hf download onkarsus13/Unifiedweights --local-dir ./Unifiedweights \ |
| --exclude "pretrain/*" |
| python -m pip install ./Unifiedweights |
| ``` |
|
|
| ## Generate an image and caption |
|
|
| ```python |
| from unifiedflow import UnifiedFlowPipeline |
| |
| pipe = UnifiedFlowPipeline.from_pretrained( |
| "onkarsus13/Unifiedweights", |
| load_image_encoder=True, |
| load_image_decoder=True, |
| ).to("cuda") |
| |
| image = pipe.generate_image("A red apple on a wooden table.").images[0] |
| image.save("apple.png") |
| result = pipe.generate_text( |
| image=image, |
| prompt="Describe this image.", |
| max_new_tokens=256, |
| ) |
| print(result.texts[0]) |
| print(result.finish_reason) |
| ``` |
|
|
| For an existing download, replace the repository ID with `"./Unifiedweights"`. |
| For captioning alone, load only the image encoder. For image generation alone, |
| load only the image decoder. Text stops at EOS or the configured token limit; |
| `finish_reason` reports `"eos"` or `"length"`. |
|
|
| ## Text continuation |
|
|
| ```python |
| pipe = UnifiedFlowPipeline.from_pretrained( |
| "onkarsus13/Unifiedweights", |
| subfolder="pretrain", |
| ).to("cuda") |
| result = pipe.generate_text( |
| prompt="The future of artificial intelligence is", |
| max_new_tokens=128, |
| ) |
| print(result.texts[0]) |
| ``` |
|
|
| ## Files |
|
|
| ```text |
| model_index.json |
| transformer/ config.json + diffusion_pytorch_model.safetensors |
| text_encoder/ config.json + diffusion_pytorch_model.safetensors |
| text_decoder/ config.json + diffusion_pytorch_model.safetensors |
| image_codec/ config.json + diffusion_pytorch_model.safetensors |
| tokenizer/ |
| pretrain/ self-contained text-pretraining variant |
| src/unifiedflow/ Python implementation |
| pyproject.toml |
| ``` |
|
|
| Each variant contains the complete model components. Loading from the Hub fetches |
| only the selected variant and requested image codec. All model tensors use |
| Safetensors; optimizer states and external filesystem references are excluded. |
|
|
| These are packaged pretrained inference checkpoints. No additional training or |
| new quality evaluation is claimed. |
|
|
| Component licenses are listed in [LICENSES/README.md](LICENSES/README.md). |
| Required copyrights, asset provenance, and source revisions are retained in |
| `LICENSE`, `THIRD_PARTY_NOTICES.md`, and `LICENSES/`. |
|
|
