| --- |
| license: apache-2.0 |
| base_model: openbmb/MiniCPM-o-4_5 |
| library_name: transformers |
| tags: |
| - multimodal |
| - audio |
| - video |
| - full-duplex |
| - gander |
| --- |
| |
| # Gander-v63-Unit8 Thinker |
|
|
| This directory contains the materialized Thinker component of Gander v63. It combines the |
| fine-tuned language model and audio projection with the frozen vision encoder, audio encoder, |
| and visual resampler from `openbmb/MiniCPM-o-4_5`. It does not contain the Talker speech decoder. |
|
|
| ## Streaming contract |
|
|
| - 1 second input units |
| - up to 8 lexical text tokens in each speaking unit |
| - 50 S3 speech tokens in the paired Talker unit |
| - streaming-exact audio frontend |
| - vocabulary size 151749, including `<|backchannel|>` at token ID 151748 |
|
|
| ## Loading |
|
|
| ```python |
| import torch |
| from transformers import AutoModel, AutoTokenizer |
| |
| path = "/path/to/thinker" |
| tokenizer = AutoTokenizer.from_pretrained(path, trust_remote_code=True) |
| model = AutoModel.from_pretrained( |
| path, |
| trust_remote_code=True, |
| torch_dtype=torch.bfloat16, |
| init_vision=True, |
| init_audio=True, |
| init_tts=False, |
| ).eval() |
| ``` |
|
|
| Full-duplex streaming, task tools, task slate, and detached speech generation require the |
| matching Gander runtime. A Codex or other worker is not embedded in these weights. |
|
|
| ## Training summary |
|
|
| The Thinker was initialized from MiniCPM-o 4.5 and trained for one epoch (8407 optimizer |
| steps). The LLM and audio projection were trainable; the vision encoder, audio encoder, |
| resampler, and TTS modules were frozen. Training used BF16 and the Unit-8/50 contract above. |
|
|
