metadata
license: apache-2.0
base_model: openbmb/MiniCPM-o-4_5
library_name: transformers
tags:
- multimodal
- audio
- video
- full-duplex
- gander
Gander-v63-Unit8 Thinker
This directory contains the materialized Thinker component of Gander v63. It combines the
fine-tuned language model and audio projection with the frozen vision encoder, audio encoder,
and visual resampler from openbmb/MiniCPM-o-4_5. It does not contain the Talker speech decoder.
Streaming contract
- 1 second input units
- up to 8 lexical text tokens in each speaking unit
- 50 S3 speech tokens in the paired Talker unit
- streaming-exact audio frontend
- vocabulary size 151749, including
<|backchannel|>at token ID 151748
Loading
import torch
from transformers import AutoModel, AutoTokenizer
path = "/path/to/thinker"
tokenizer = AutoTokenizer.from_pretrained(path, trust_remote_code=True)
model = AutoModel.from_pretrained(
path,
trust_remote_code=True,
torch_dtype=torch.bfloat16,
init_vision=True,
init_audio=True,
init_tts=False,
).eval()
Full-duplex streaming, task tools, task slate, and detached speech generation require the matching Gander runtime. A Codex or other worker is not embedded in these weights.
Training summary
The Thinker was initialized from MiniCPM-o 4.5 and trained for one epoch (8407 optimizer steps). The LLM and audio projection were trainable; the vision encoder, audio encoder, resampler, and TTS modules were frozen. Training used BF16 and the Unit-8/50 contract above.