Text Generation
Transformers
Safetensors
Chinese
English
ynet32
custom_code
ymodel
ymodel32
conversational
Instructions to use SnifferCaptain/YModel3.2-fm-1B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use SnifferCaptain/YModel3.2-fm-1B with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="SnifferCaptain/YModel3.2-fm-1B", trust_remote_code=True) messages = [ {"role": "user", "content": "Who are you?"}, ] pipe(messages)# pip install -U transformers accelerate # Load model directly from transformers import AutoModelForCausalLM model = AutoModelForCausalLM.from_pretrained("SnifferCaptain/YModel3.2-fm-1B", trust_remote_code=True, device_map="auto") - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use SnifferCaptain/YModel3.2-fm-1B with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "SnifferCaptain/YModel3.2-fm-1B" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "SnifferCaptain/YModel3.2-fm-1B", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker
docker model run hf.co/SnifferCaptain/YModel3.2-fm-1B
- SGLang
How to use SnifferCaptain/YModel3.2-fm-1B with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "SnifferCaptain/YModel3.2-fm-1B" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "SnifferCaptain/YModel3.2-fm-1B", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "SnifferCaptain/YModel3.2-fm-1B" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "SnifferCaptain/YModel3.2-fm-1B", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }' - Docker Model Runner
How to use SnifferCaptain/YModel3.2-fm-1B with Docker Model Runner:
docker model run hf.co/SnifferCaptain/YModel3.2-fm-1B
|
Download README.md from SnifferCaptain/YModel3.2-fm-1B: direct link, hf CLI and curl.
- Browser
- Download file 8.25 kB
-
https://huggingface.co/SnifferCaptain/YModel3.2-fm-1B/resolve/main/README.md
- Command line
-
hf download hf://SnifferCaptain/YModel3.2-fm-1B/README.md
-
curl -L -o README.md https://huggingface.co/SnifferCaptain/YModel3.2-fm-1B/resolve/main/README.md
8.25 kB
| license: mit | |
| language: | |
| - zh | |
| - en | |
| pipeline_tag: text-generation | |
| library_name: transformers | |
| tags: | |
| - custom_code | |
| - safetensors | |
| - ymodel | |
| - ymodel32 | |
| *this file is generated by Qwen3.8-27B under human supervision* | |
| ## 模型描述 | |
| `YModel3.2-fm-1B` 是 YModel3.2 家族的 1B 参数**基础权重模型**(foundation weights)。 | |
| ## 模型细节 | |
| | 项目 | 值 | | |
| | --- | --- | | |
| | 架构 | YModel3.2 | | |
| | Scale level | 1 | | |
| | 参数量 | 1,120,343,553(1.12B,含嵌入)| | |
| | Hidden size | 1024 | | |
| | 块数 | 31(6 个 FullBlock + 25 个 GroupBlock)| | |
| | layer pattern | `fgggggfgggggfgggggfgggggfgggggf` | | |
| | 注意力头 / KV 头 | 8 / 2(GQA)| | |
| | 头维度 | 256 | | |
| | Group size | 8(固定分组)| | |
| | FFN | SwiGLU + SEBlock,intermediate 3072 | | |
| | DCP | dynamic compression probe(probe hidden 1024);use_dcp=false,分组固定,probe 权重保留 | | |
| | Sengram | bucket 16384 / topk 12 | | |
| | 词表 | 151,669(Qwen3 tokenizer,tied embeddings)| | |
| | 上下文 | 8192 | | |
| | 精度 | bfloat16 | | |
| ### Group Attn 与 Token Superposition | |
| YModel3.2-fm 的 group attn 主要借鉴了论文 **Efficient Pre-Training with Token Superposition**(Bowen Peng, Théo Gigant, Jeffrey Quesnelle, arXiv:2605.06546)的思想:将连续 token 分组(superpose)为一个整体进行高效处理。论文提出的 Token-Superposition Training(TST)把多个连续 token 合并为一个 bag、以 multi-hot 交叉熵(MCE)目标训练,再经 recovery 阶段回到标准训练,在 10B A1B 规模上报告了至多 2.5x 的预训练时间缩减。本模型的 group attn 是其架构侧的延伸:GroupBlock 以 group_size=8 的分组处理 token,FullBlock / GroupBlock 按 f/g pattern 交错排布,并用 F2G / G2F 层完成组与完整 token 流之间的信息转换。 | |
| ## 训练细节 | |
| 训练分四个阶段完成(bf16,AdamW 与 Muon 双优化器): | |
| 1. **预训练**:causal LM,1024 上下文,**8.52B Tokens**,冻结嵌入与 lm_head。 | |
| 2. **SFT**:UltraData SFT + Instant 混合语料,8192 上下文(预训练为 1024,此阶段首次扩展到 8192),Muon 5e-6 / AdamW 5e-7,嵌入与 lm_head 冻结,**4.14B Tokens**。 | |
| 3. **蒸馏**:teacher 为 Qwen3-0.6B(CE=0.5 / KL=0.5,temperature=1.0),固定 group_size=4,DCP probe 冻结,哈希辅助监督加强至 1e-1,Muon 1e-4 / AdamW 1e-5,**0.98B Tokens**。 | |
| ## Usage | |
| ```python | |
| from transformers import AutoTokenizer, AutoModelForCausalLM | |
| model_id = "SnifferCaptain/YModel3.2-fm-1B" | |
| tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True) | |
| model = AutoModelForCausalLM.from_pretrained( | |
| model_id, | |
| trust_remote_code=True, | |
| torch_dtype="auto", | |
| ) | |
| ``` | |
| ### 对话格式 | |
| 模型使用 ChatML 风格的对话边界。完整的输入可以写成: | |
| ```text | |
| <|im_start|>system | |
| {system prompt}<|im_end|> | |
| <|im_start|>user | |
| {user content}<|im_end|> | |
| <|im_start|>assistant | |
| <think>{thinking_definitions} | |
| {reasoning content}</think> | |
| {assistant content}<|im_end|> | |
| ``` | |
| 其中,`{thinking_definitions}` 是紧跟在 `<think>` 后面的**第一行控制信息**,不是思考正文。它可以是: | |
| - `juice = 1.23`:思考预算提示,可以使用小数。 | |
| - `structured = true`:表示结构化思考。 | |
| - `juice = 1.23; structured = true`:同时指定预算和结构化思考。 | |
| 控制行结束后换行,再写入 `{reasoning content}`;思考结束使用 `</think>`,之后空一行写最终的 `{assistant content}`。自由形式思考可以省略 `structured`,也可以显式写成 `structured = false`。如果没有控制字段,保留空的控制行即可: | |
| ```text | |
| <think> | |
| {reasoning content} | |
| </think> | |
| ``` | |
| 每个角色都使用 `<|im_start|>role\n...<|im_end|>` 边界;`system` 和 `user` 内容不会放入 `<think>`,思考与最终回答只属于 `assistant` 消息。 | |
| 注意:该架构的 `generate()` 为模型自定义实现,不使用 KV cache(每步重算完整前缀),长序列生成会相应变慢。 | |
| ## Notes | |
| - 包含自定义代码:`modeling_ymodel32.py`(自包含:配置类 + 完整架构)、`ymodel32_eval.py`(参考实现),加载需 `trust_remote_code=True`。 | |
| - 权重为单个 `model.safetensors`(bfloat16,约 2.55GB)。 | |
| - 基准分数与示例输出待补充。 | |
| --- | |
| ## Model Description | |
| `YModel3.2-fm-1B` is the 1B-parameter **foundation-weights model** of the YModel3.2 family. | |
| ## Model Details | |
| | Item | Value | | |
| | --- | --- | | |
| | Architecture | YModel3.2 | | |
| | Scale level | 1 | | |
| | Parameters | 1,120,343,553 (1.12B, incl. embeddings) | | |
| | Hidden size | 1024 | | |
| | Blocks | 31 (6 FullBlocks + 25 GroupBlocks) | | |
| | Layer pattern | `fgggggfgggggfgggggfgggggfgggggf` | | |
| | Attention heads / KV heads | 8 / 2 (GQA) | | |
| | Head dim | 256 | | |
| | Group size | 8 (fixed grouping) | | |
| | FFN | SwiGLU + SEBlock, intermediate 3072 | | |
| | DCP | dynamic compression probe (probe hidden 1024); use_dcp=false, fixed grouping, probe weights retained | | |
| | Sengram | bucket 16384 / topk 12 | | |
| | Vocab | 151,669 (Qwen3 tokenizer, tied embeddings) | | |
| | Context | 8192 | | |
| | Dtype | bfloat16 | | |
| ### Group Attn and Token Superposition | |
| The group attn of YModel3.2-fm is inspired primarily by **Efficient Pre-Training with Token Superposition** (Bowen Peng, Théo Gigant, Jeffrey Quesnelle, arXiv:2605.06546): superposing contiguous tokens into a group and processing them as a single unit. The paper's Token-Superposition Training (TST) merges many contiguous tokens into one bag and trains with a multi-hot cross-entropy (MCE) objective, then recovers to standard training, reporting up to a 2.5x pre-training time reduction at 10B A1B scale. This model's group attn is the architectural extension of that idea: GroupBlocks process tokens in groups of 8, FullBlocks and GroupBlocks are interleaved per the f/g pattern, and F2G / G2F layers transfer information between the group and full-token streams. | |
| ## Training Details | |
| Training ran in four stages (bf16, AdamW + Muon dual optimizers): | |
| 1. **Pre-training**: causal LM, 1024 context, **8.52B Tokens**, embeddings and lm_head frozen. | |
| 2. **SFT**: mixed UltraData SFT + Instant corpus, 8192 context (pre-training used 1024, so 8192 is new context here), Muon 5e-6 / AdamW 5e-7, embeddings and lm_head frozen, **4.14B Tokens**. | |
| 3. **Distillation**: teacher Qwen3-0.6B (CE=0.5 / KL=0.5, temperature 1.0), fixed group_size=4, DCP probe frozen, hash auxiliary supervision raised to 1e-1, Muon 1e-4 / AdamW 1e-5, **0.98B Tokens**. | |
| ## Usage | |
| ```python | |
| from transformers import AutoTokenizer, AutoModelForCausalLM | |
| model_id = "SnifferCaptain/YModel3.2-fm-1B" | |
| tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True) | |
| model = AutoModelForCausalLM.from_pretrained( | |
| model_id, | |
| trust_remote_code=True, | |
| torch_dtype="auto", | |
| ) | |
| ``` | |
| ### Chat Format | |
| The model uses ChatML-style turn boundaries. A complete input can be written as: | |
| ```text | |
| <|im_start|>system | |
| {system prompt}<|im_end|> | |
| <|im_start|>user | |
| {user content}<|im_end|> | |
| <|im_start|>assistant | |
| <think>{thinking_definitions} | |
| {reasoning content}</think> | |
| {assistant content}<|im_end|> | |
| ``` | |
| `{thinking_definitions}` is the **first control line** immediately after `<think>`, not the reasoning body. It may be: | |
| - `juice = 1.23` for a numeric reasoning-budget hint; | |
| - `structured = true` to mark structured reasoning; or | |
| - `juice = 1.23; structured = true` to provide both fields. | |
| Terminate the control line with a newline, then place the reasoning in `{reasoning content}`. Close it with `</think>`, leave one blank line, and put the final answer in `{assistant content}`. For free-form reasoning, `structured` may be omitted or written as `structured = false`. If no control field is needed, keep an empty control line: | |
| ```text | |
| <think> | |
| {reasoning content} | |
| </think> | |
| ``` | |
| Note: this architecture ships a custom `generate()` implementation without KV caching (the full prefix is recomputed each step), so long-sequence generation is correspondingly slower. | |
| ## Notes | |
| - Custom code included: `modeling_ymodel32.py` (self-contained: config class + full architecture) and `ymodel32_eval.py` (reference implementation); load with `trust_remote_code=True`. | |
| - Weights: single `model.safetensors` (bfloat16, ~2.55GB). | |
| - Benchmark scores and sample outputs: to be added. | |