--- license: apache-2.0 base_model: ByteDance-Seed/BAGEL-7B-MoT pipeline_tag: any-to-any language: - en tags: - unimore - bagel - multimodal - text-to-image datasets: - HuggingSelf/UniMore --- # UniMore-BAGEL The final UniMore checkpoint based on BAGEL-7B-MoT, supporting multimodal understanding and image generation. This repository contains the merged BF16 model, its VAE, tokenizer, and configuration files. ## Method Two branches are trained independently from the same pretrained BAGEL model: - Generation: grounded phrase-to-crop supervision with the native generation loss. - Understanding: full-image caption supervision with positive cross-entropy and full-caption ranking against detail-omission, incorrect-detail, and random negatives. The branches are combined using ```text W = W_base + ((W_t2i - W_base) + lambda * (W_i2t - W_base)) ``` Here, `lambda` controls the understanding branch's contribution. Merge arithmetic uses BF16, including intermediate differences and additions. ## Download and Use ```bash hf download HuggingSelf/UniMore --local-dir /data/models/UniMore-BAGEL ``` Use the BAGEL adapter in [TorchUMM](https://github.com/AIFrontierLab/TorchUMM), setting `model_path` to the downloaded directory. Installation, training, and inference instructions are in the [UniMore package](https://github.com/AIFrontierLab/TorchUMM/tree/main/src/umm/post_training/UniMore). These weights use BAGEL's native `ema.safetensors` format, not a Transformers `AutoModel` checkpoint. The generation evaluation configuration uses 512px images, 50 sampling steps, text/image CFG 4/1, timestep shift 3, and seed 42. ## Training | Setting | Generation | Understanding | |---|---:|---:| | Training records | 2,460 | 4,000 | | Optimizer updates | 1,000 | 600 | | Learning rate | 5e-5 | 1.8e-5 | | Warmup updates | 50 | 60 | | Training image size | 256 | 112 | | LoRA rank / alpha | 16 / 32 | 16 / 32 | Training data and a preparation script are available in the [UniMore dataset repository](https://huggingface.co/datasets/HuggingSelf/UniMore). ## Files and Attribution - `ema.safetensors`: final merged model. - `ae.safetensors`: the original BAGEL VAE. - Tokenizer and JSON files: the original BAGEL inference configuration. - `release.json`: file sizes and fixed source revisions. The base model is [BAGEL-7B-MoT](https://huggingface.co/ByteDance-Seed/BAGEL-7B-MoT). The Apache 2.0 license is included in `LICENSE`.