Minimal card for now

README.md

@@ -3,84 +3,8 @@ license: apache-2.0
 base_model: ByteDance-Seed/BAGEL-7B-MoT
 pipeline_tag: any-to-any
 library_name: bagel
-tags:
-- bagel
-- bagel-mot
-- thinkmorph-ssl
-- primer
-- multimodal
-- pretext
-datasets:
-- ThinkMorph/Pretext-270K
 ---
 
 # ThinkMorph-SSL-Base
 
-This is the **intermediate Stage 1 model** of the PRIMER recipe. It is the EMA weights at
-step 5,500 of the pretext stage, trained from BAGEL-7B-MoT on
-[Pretext-270K](https://huggingface.co/datasets/ThinkMorph/Pretext-270K). The final model,
-[ThinkMorph-SSL-Instruct](https://huggingface.co/ThinkMorph/ThinkMorph-SSL-Instruct), starts its Stage 2 training from these weights.
-
-> **This model has only had pretext training, with no question-answer supervision.** Its
-> exercises take a corrupted input image and output a description, a restored image and a
-> second description. It is not meant to answer questions directly. The paper's appendix
-> reports roughly 40% on V\* and MMMU for this checkpoint. For question answering, use
-> ThinkMorph-SSL-Instruct.
-
-## Files
-
-| File | Bytes | SHA256 |
-| --- | ---: | --- |
-| `ema.safetensors` | 29,351,872,574 | `f1dd6a3fb88c9145ac8c3d69fd60877cb359ec3996f3a8218580f621e47943ae` |
-| `ae.safetensors` | 167,664,678 | `e49d48aa3474df744ad718d11736d87f38d52bf7ca57dee6b8417978bfbfc9ad` |
-| `config.json` | 5,175 | `4bc85a4daaef591f0b539216d09a1ee2294a1f3a3bda9f95885c13ed191ae701` |
-| `llm_config.json` | 2,136 | `cb884fc65d117b8e5ec3856090224cd739c511e01c5c1de488cede5a54c8e994` |
-| `vit_config.json` | 1,830 | `78664013bf6778ad4ecaebe78c153ef7fd46f7638fa78c0df1cb17d5bd7df0ab` |
-| `generation_config.json` | 243 | `3a8f9087e486054c8a4a08dae2e5a3ba62e23da212b5b8c08bc42cb983c3459f` |
-| `vocab.json` | 3,383,407 | `87a257b04b17642a0688c98cd1df89c398bda4fee532d6f88b38a659ecb4ac8d` |
-| `merges.txt` | 1,671,853 | `8831e4f1a044471340f7c0a83d7bd71306a5b867e95fd870f74d0c5308a904d5` |
-| `added_tokens.json` | 630 | `a0582f48ed8e795fda5b7b8eb2d98b492842dc229e78b66793bf545ce7ec52c8` |
-| `special_tokens_map.json` | 613 | `76862e765266b85aa9459767e33cbaf13970f327a0e88d1c65846c2ddd3a1ecd` |
-| `tokenizer_config.json` | 4,864 | `b7af0d4c9b917db05150006b466cfa2e8424b3915ff5fe2023a6f512a5e16dd5` |
-
-All files except `ema.safetensors` are byte-identical to the ThinkMorph-SSL-Instruct release. The
-checkpoint's own config and tokenizer files were compared with those of ThinkMorph-SSL-Instruct and
-are the same. `ema.safetensors` uses the same format as ThinkMorph-SSL-Instruct: 1,451 bfloat16 tensors
-with the same names and shapes. The unused 26th SigLIP layer is left out.
-
-## How the file was made
-
-1. The source is the FSDP2 checkpoint of run `0503_stage0_270k_32gpu` at step 5,500
-   (32 `.distcp` shards, 117,699,388,909 bytes). It holds both the regular weights and the
-   EMA weights, all 1,467 tensors of each, in fp32.
-2. `scripts/merge_checkpoint.py --checkpoint-type ema` from the PRIMER code release merged
-   the EMA state into a single fp32 `model.safetensors`. All 1,467 weights came from the EMA
-   state; none were filled in from a base model.
-3. The 1,451 tensors that ThinkMorph-SSL-Instruct ships were cast once to bfloat16, with no other
-   change, and saved as `ema.safetensors`. A spot check confirmed that the cast values match
-   exactly. The VAE weights are identical to ThinkMorph-SSL-Instruct's, as expected, because the VAE
-   is frozen in both stages.
-
-## Training
-
-| Item | Value |
-| --- | --- |
-| Run | `0503_stage0_270k_32gpu` |
-| Initialization | BAGEL-7B-MoT (`lmms-lab/BAGEL-7B-MoT-ver.LE` layout and VAE) |
-| Data | Pretext-270K, 270,896 train rows; the val files are not used |
-| GPUs | 32 (4 nodes × 8) |
-| Learning rate | 2.83e-5, warmup 1,000 |
-| Configured horizon | 10,000 steps; this release is the step-5,500 EMA save |
-| EMA | decay 0.9998 from step 1,000, fp32 shadow |
-| Losses | text cross-entropy and image flow matching, weights 1 and 1; VAE frozen |
-
-## Loading
-
-Loading works the same way as for ThinkMorph-SSL-Instruct. With the official BAGEL inference code or
-the PRIMER code release, point `model_path` at a local copy. The loader reads
-`ema.safetensors`.
-
-## License
-
-The weights are released under Apache-2.0, following BAGEL-7B-MoT. Pretext-270K has its own
-per-source terms; see its card.
+Model weights. More details will be added later.