Instructions to use dokoomae/LFM2.5-230M-Base-CMoE-e4s2a2 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use dokoomae/LFM2.5-230M-Base-CMoE-e4s2a2 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="dokoomae/LFM2.5-230M-Base-CMoE-e4s2a2")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("dokoomae/LFM2.5-230M-Base-CMoE-e4s2a2")
model = AutoModelForCausalLM.from_pretrained("dokoomae/LFM2.5-230M-Base-CMoE-e4s2a2", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=256)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use dokoomae/LFM2.5-230M-Base-CMoE-e4s2a2 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "dokoomae/LFM2.5-230M-Base-CMoE-e4s2a2"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "dokoomae/LFM2.5-230M-Base-CMoE-e4s2a2",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/dokoomae/LFM2.5-230M-Base-CMoE-e4s2a2

SGLang

How to use dokoomae/LFM2.5-230M-Base-CMoE-e4s2a2 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "dokoomae/LFM2.5-230M-Base-CMoE-e4s2a2" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "dokoomae/LFM2.5-230M-Base-CMoE-e4s2a2",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "dokoomae/LFM2.5-230M-Base-CMoE-e4s2a2" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "dokoomae/LFM2.5-230M-Base-CMoE-e4s2a2",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use dokoomae/LFM2.5-230M-Base-CMoE-e4s2a2 with Docker Model Runner:
```
docker model run hf.co/dokoomae/LFM2.5-230M-Base-CMoE-e4s2a2
```

LFM2.5-230M-Base-CMoE-e4s2a2

本模型是 LiquidAI/LFM2.5-230M-Base 的 CMoE (Carved Mixture-of-Experts,arXiv:2502.04416) 转换版本。

采用 CMoE 论文中的 Carving by Sharing (无冗余切分) 策略:把每一层 (≥ num_dense_layers) 的 dense SwiGLU FFN 沿 intermediate 维度等分成 4 块,直接映射到 4 个 expert,没有任何冗余拷贝,参数量与 dense 原模型几乎相同。

未做任何 fine-tune。init 时 4 个 active expert 严格还原 dense FFN (各层 hidden state 相对误差 < 1.5%,argmax 一致),后续可通过 fine-tune 让 router 学到非平凡路由。

配置:e4s2a2

符号	含义	值
`e4`	总 expert 数	4
`s2`	shared expert 数 (通过 `expert_bias` 始终路由)	2
`a2`	每 token 激活的 routed expert 数	2

每个 token 共激活 4 个 expert (2 shared + 2 routed)。配合 moe_intermediate_size = 640 = 2560/4,每 token 的激活参数量与 dense FFN 完全相同,即 MoE 在不增加激活计算量的前提下,把 FFN 路由方式从"全部用"变成"按需用一部分"。

加载方式 (LFM 官方模板,无需 `trust_remote_code`)

from transformers import AutoModelForCausalLM, AutoTokenizer, TextStreamer
import torch

model_id = "dokoomae/LFM2.5-230M-Base-CMoE-e4s2a2"
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    device_map="auto",
    dtype=torch.bfloat16,
    # attn_implementation="flash_attention_2"  # 在支持的 GPU 上取消注释
)
tokenizer = AutoTokenizer.from_pretrained(model_id)
streamer = TextStreamer(tokenizer, skip_prompt=True, skip_special_tokens=True)

prompt = "What is C. elegans?"
input_ids = tokenizer.apply_chat_template(
    [{"role": "user", "content": prompt}],
    add_generation_prompt=True,
    return_tensors="pt",
    tokenize=True,
)["input_ids"].to(model.device)

output = model.generate(
    input_ids,
    do_sample=True,
    temperature=0.1,
    top_k=50,
    repetition_penalty=1.05,
    max_new_tokens=512,
    streamer=streamer,
)

转换细节

架构

源模型: Lfm2ForCausalLM (dense,14 层混合 conv/attention,hidden=1024,FFN intermediate=2560)
目标模型: Lfm2MoeForCausalLM (transformers 原生类,无需 trust_remote_code=True)
MoE 层: 第 2–13 层 (前 2 层保持 dense,遵循 LFM2-MoE 惯例)
每个 expert 的 intermediate size: 640 (= 2560 / 4 激活 expert)
参数量: 229,742,336 (dense 原模型 229,693,184,膨胀率仅 1.0002×,多出的 49K 是 router 权重和 expert_bias)

Carving 策略:Carving by Sharing (无冗余)

把 dense SwiGLU FFN (w1, w2, w3,intermediate dim=2560) 沿 intermediate 维度等分成 4 块 (每块 640),直接映射到 4 个 expert,没有任何复制或噪声:

expert id	来源	角色	expert_bias
0	chunk 0 (intermediate 0–640)	shared (始终路由)	100.0
1	chunk 1 (intermediate 640–1280)	shared (始终路由)	100.0
2	chunk 2 (intermediate 1280–1920)	routed (bias 倾向被选)	50.0
3	chunk 3 (intermediate 1920–2560)	routed (bias 倾向被选)	50.0

init 时为何能严格等价 dense

Router 权重零初始化 → 所有 expert 的 sigmoid score 都是 0.5
expert_bias = [100, 100, 50, 50] 让 top-4 选择固定为 [0, 1, 2, 3] (即 4 个唯一 chunk)
norm_topk_prob=True 把 top-K sigmoid 权重归一化到和为 1 (每个 0.25)
设 routed_scaling_factor = 4.0,让有效 per-expert 权重 = 0.25 × 4 = 1.0
4 个被选 expert 的输出 = chunk_0 + chunk_1 + chunk_2 + chunk_3 = dense FFN 完整输出 ✓

与 e8s2a2 (Splitting + Replicate) 的对比

维度	e4s2a2 Sharing (本模型)	e8s2a2 Splitting+Replicate
expert 总数	4	8
参数量	229M (= dense,无膨胀)	324M (1.41× dense)
每 token 激活 expert	4	4
每 token 激活参数量	2560 (= dense)	2560 (= dense)
init 时等价 dense	严格等价	严格等价
fine-tune 分化空间	4 个 expert 各自负责 640 维子空间	8 个 expert (4 unique + 4 noisy replicate),replicate 可分化出新功能
适用场景	小模型 + 中等 fine-tune 量,不浪费参数	大模型 + 大量 fine-tune + 多任务,需要更多 expert 容量

对 230M 这种小模型,e4s2a2 Sharing 更合适:不浪费参数,每个 expert (640 维) 比 e8s2a2 的 replicate 版 (320 维) 表达力更强。

init 等价性验证 (CPU,bfloat16)

对 prompt "What is C. elegans?" 的逐层 hidden state 比对:

层	dense 均值	MoE 均值	相对误差
0–2	(完全相同)	(完全相同)	0.000
3	+0.000022	+0.000022	0.001
7	-0.006387	-0.006410	0.007
13	-0.007979	-0.008013	0.012
输出 logits	-0.048054	-0.048511	0.015

dense 和 MoE 的 argmax next token 都是 'C',功能等价。

速度对比 (CPU,bfloat16,max_new_tokens=128)

指标	Dense (原模型)	CMoE-e4s2a2 (本模型)	比例
参数量	229,693,184	229,742,336	1.0002×
forward 平均 (ms)	61.2	77.9	1.27×
generate 平均 (s)	7.05	8.71	1.24×
tokens / sec	18.16	14.70	0.81×

说明:CPU 上 MoE 会因 router + scatter/gather 开销略慢 (~20%)。在 CUDA GPU 上 (尤其是配合 grouped-GEMM kernel),MoE 在相同激活参数量下通常能达到与 dense 相当甚至更快的速度,同时为后续 fine-tune 提供 expert 分化的潜力。

文件清单

文件	说明
`config.json`	Lfm2MoeConfig (model_type=`lfm2_moe`)
`model.safetensors`	Carved 权重 (229M 参数,bfloat16)
`generation_config.json`	生成默认配置 (从源模型拷贝)
`tokenizer.json`	Fast tokenizer (从源模型拷贝)
`tokenizer_config.json`	Tokenizer 配置 (从源模型拷贝)
`chat_template.jinja`	Chat template (从源模型拷贝)
`LICENSE`	LFM Open License v1.0 (继承自源模型)

许可证

LFM Open License v1.0 (继承自源模型 LiquidAI/LFM2.5-230M-Base)。

重要条款:

年营收低于 1000 万美元的实体可免费用于商业用途
年营收超过 1000 万美元的实体需另行与 Liquid AI 签订商业许可
合格的非营利组织 (501(c)(3) 或同等机构) 用于非商业/研究目的不受营收门槛限制

完整许可证文本见同仓库的 LICENSE 文件。

Downloads last month: -

Safetensors

Model size

0.2B params

Tensor type

BF16

Model tree for dokoomae/LFM2.5-230M-Base-CMoE-e4s2a2

Base model

LiquidAI/LFM2.5-230M-Base

Finetuned

(15)

this model

Paper for dokoomae/LFM2.5-230M-Base-CMoE-e4s2a2

CMoE: Fast Carving of Mixture-of-Experts for Efficient LLM Inference

Paper • 2502.04416 • Published Feb 6, 2025 • 12