---
license: apache-2.0
base_model: ByteDance-Seed/BAGEL-7B-MoT
base_model_relation: finetune
pipeline_tag: any-to-any
library_name: bagel
datasets:
- ThinkMorph/Pretext-270K
---

# ThinkMorph-SSL-Base

<p>
  <a href="https://huggingface.co/datasets/ThinkMorph/Pretext-270K"><img src="https://img.shields.io/badge/Pretext--270K-Dataset-yellow?logo=huggingface&logoColor=yellow" alt="Pretext-270K Dataset"/></a>
  <a href="https://huggingface.co/ThinkMorph/ThinkMorph-SSL-Instruct"><img src="https://img.shields.io/badge/ThinkMorph--SSL--Instruct-Model-orange?logo=huggingface&logoColor=orange" alt="ThinkMorph-SSL-Instruct Model"/></a>
</p>

## About

ThinkMorph-SSL-Base is [BAGEL-7B-MoT](https://huggingface.co/ByteDance-Seed/BAGEL-7B-MoT) after self-supervised pretext
reasoning on [Pretext-270K](https://huggingface.co/datasets/ThinkMorph/Pretext-270K). Given a corrupted image, it
describes the image, generates the restored version, and describes the result. It was trained
without questions and gives no answers. It is the starting point of
[ThinkMorph-SSL-Instruct](https://huggingface.co/ThinkMorph/ThinkMorph-SSL-Instruct).

## Usage

The checkpoint has the same layout as BAGEL-7B-MoT and loads with the
[BAGEL code](https://github.com/ByteDance-Seed/Bagel). Its output is `<think>...</think>`, a
generated image, then `<think>...</think>`. It was trained with the same system prompt as
ThinkMorph-SSL-Instruct.

## Citation

The citation will be added with the paper.