| --- |
| license: apache-2.0 |
| base_model: ThinkMorph/ThinkMorph-SSL-Base |
| base_model_relation: finetune |
| pipeline_tag: any-to-any |
| library_name: bagel |
| datasets: |
| - ThinkMorph/Pretext-270K |
| - ThinkMorph/Instruct-180K |
| --- |
| |
| # ThinkMorph-SSL-Instruct |
|
|
| <p> |
| <a href="https://huggingface.co/datasets/ThinkMorph/Pretext-270K"><img src="https://img.shields.io/badge/Pretext--270K-Dataset-yellow?logo=huggingface&logoColor=yellow" alt="Pretext-270K Dataset"/></a> |
| <a href="https://huggingface.co/datasets/ThinkMorph/Instruct-180K"><img src="https://img.shields.io/badge/Instruct--180K-Dataset-yellow?logo=huggingface&logoColor=yellow" alt="Instruct-180K Dataset"/></a> |
| <a href="https://huggingface.co/ThinkMorph/ThinkMorph-SSL-Base"><img src="https://img.shields.io/badge/ThinkMorph--SSL--Base-Model-orange?logo=huggingface&logoColor=orange" alt="ThinkMorph-SSL-Base Model"/></a> |
| </p> |
|
|
| ## About |
|
|
| ThinkMorph-SSL-Instruct reasons with interleaved text and images. On a visual question it thinks |
| in text, generates intermediate images when they help (a zoomed-in crop, a depth map, a |
| bird's-eye view, the view from another position), and then answers. |
|
|
| It is trained from [BAGEL-7B-MoT](https://huggingface.co/ByteDance-Seed/BAGEL-7B-MoT) in two stages: |
|
|
| | Stage | Data | Model | |
| | --- | --- | --- | |
| | 1. Self-supervised pretext reasoning | [Pretext-270K](https://huggingface.co/datasets/ThinkMorph/Pretext-270K) | [ThinkMorph-SSL-Base](https://huggingface.co/ThinkMorph/ThinkMorph-SSL-Base) | |
| | 2. Instruction tuning | [Instruct-180K](https://huggingface.co/datasets/ThinkMorph/Instruct-180K) | ThinkMorph-SSL-Instruct (this model) | |
|
|
| ## Usage |
|
|
| The checkpoint has the same layout as BAGEL-7B-MoT and loads with the |
| [BAGEL code](https://github.com/ByteDance-Seed/Bagel). Keep the VAE and tokenizer files shipped |
| here. Prompt the model with the system prompt it was trained with: |
|
|
| ```text |
| Use <think>...</think> for intermediate reasoning or planning. |
| |
| To generate an image, write <gen_image> followed by an image-size declaration: |
| <gen_image>[Image size: (height, width)] |
| Generated images become part of your context and can inform subsequent reasoning. You may generate more than one image if needed. |
| |
| If the task requires a final textual answer, wrap it in <answer>...</answer>. |
| If no textual answer is needed, the response may end after reasoning or image generation. |
| |
| Example: |
| <think>reasoning step</think> |
| <gen_image>[Image size: (512, 512)] |
| <think>observe and reason about the generated image</think> |
| <answer>final answer</answer> |
| ``` |
|
|
| ## Benchmarks |
|
|
| Results will be added with the paper. |
|
|
| ## Citation |
|
|
| The citation will be added with the paper. |
|
|
