File size: 2,644 Bytes
d1fc329 fa0c6a3 d1fc329 fa0c6a3 d1fc329 70df08c d1fc329 0523fce fa0c6a3 0523fce fa0c6a3 0523fce fa0c6a3 0523fce fa0c6a3 0523fce afd7a84 0523fce fa0c6a3 0523fce fa0c6a3 0523fce fa0c6a3 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 | ---
license: apache-2.0
base_model: ThinkMorph/ThinkMorph-SSL-Base
base_model_relation: finetune
pipeline_tag: any-to-any
library_name: bagel
datasets:
- ThinkMorph/Pretext-270K
- ThinkMorph/Instruct-180K
---
# ThinkMorph-SSL-Instruct
<p>
<a href="https://huggingface.co/datasets/ThinkMorph/Pretext-270K"><img src="https://img.shields.io/badge/Pretext--270K-Dataset-yellow?logo=huggingface&logoColor=yellow" alt="Pretext-270K Dataset"/></a>
<a href="https://huggingface.co/datasets/ThinkMorph/Instruct-180K"><img src="https://img.shields.io/badge/Instruct--180K-Dataset-yellow?logo=huggingface&logoColor=yellow" alt="Instruct-180K Dataset"/></a>
<a href="https://huggingface.co/ThinkMorph/ThinkMorph-SSL-Base"><img src="https://img.shields.io/badge/ThinkMorph--SSL--Base-Model-orange?logo=huggingface&logoColor=orange" alt="ThinkMorph-SSL-Base Model"/></a>
</p>
## About
ThinkMorph-SSL-Instruct reasons with interleaved text and images. On a visual question it thinks
in text, generates intermediate images when they help (a zoomed-in crop, a depth map, a
bird's-eye view, the view from another position), and then answers.
It is trained from [BAGEL-7B-MoT](https://huggingface.co/ByteDance-Seed/BAGEL-7B-MoT) in two stages:
| Stage | Data | Model |
| --- | --- | --- |
| 1. Self-supervised pretext reasoning | [Pretext-270K](https://huggingface.co/datasets/ThinkMorph/Pretext-270K) | [ThinkMorph-SSL-Base](https://huggingface.co/ThinkMorph/ThinkMorph-SSL-Base) |
| 2. Instruction tuning | [Instruct-180K](https://huggingface.co/datasets/ThinkMorph/Instruct-180K) | ThinkMorph-SSL-Instruct (this model) |
## Usage
The checkpoint has the same layout as BAGEL-7B-MoT and loads with the
[BAGEL code](https://github.com/ByteDance-Seed/Bagel). Keep the VAE and tokenizer files shipped
here. Prompt the model with the system prompt it was trained with:
```text
Use <think>...</think> for intermediate reasoning or planning.
To generate an image, write <gen_image> followed by an image-size declaration:
<gen_image>[Image size: (height, width)]
Generated images become part of your context and can inform subsequent reasoning. You may generate more than one image if needed.
If the task requires a final textual answer, wrap it in <answer>...</answer>.
If no textual answer is needed, the response may end after reasoning or image generation.
Example:
<think>reasoning step</think>
<gen_image>[Image size: (512, 512)]
<think>observe and reason about the generated image</think>
<answer>final answer</answer>
```
## Benchmarks
Results will be added with the paper.
## Citation
The citation will be added with the paper.
|