beyoru commited on
Commit
07281ce
·
verified ·
1 Parent(s): 66f07ee

Fix transformers compatibility note

Browse files
Files changed (1) hide show
  1. README.md +15 -8
README.md CHANGED
@@ -128,14 +128,21 @@ text/language-model-only flag that the upstream release requires.
128
 
129
  ### Transformers
130
 
131
- ⚠️ This checkpoint stores MoE experts in the **per-expert (split)** layout
132
- (`model.language_model.layers.N.mlp.experts.{0..255}.{gate,up,down}_proj.weight`), inherited unchanged
133
- from upstream. `transformers` 5.x expects a **fused** layout (`experts.gate_up_proj`,
134
- `experts.down_proj`) and ships no conversion mapping for this architecture, so loading with
135
- transformers 5.x reports 80 missing parameters. Use `transformers==4.57.1` (the version the upstream
136
- model declares), or serve with vLLM / SGLang, which have their own MoE loaders and handle the split
137
- layout correctly. **This is an upstream property, not something introduced here** — the original
138
- release behaves identically.
 
 
 
 
 
 
 
139
 
140
  ---
141
 
 
128
 
129
  ### Transformers
130
 
131
+ Works out of the box — verified on `transformers==5.10.1`.
132
+
133
+ ```python
134
+ import torch
135
+ from transformers import AutoModelForImageTextToText, AutoProcessor
136
+
137
+ model = AutoModelForImageTextToText.from_pretrained(
138
+ "beyoru/KAT-Coder-V2.5-Dev-VL", dtype=torch.bfloat16, device_map="auto")
139
+ processor = AutoProcessor.from_pretrained("beyoru/KAT-Coder-V2.5-Dev-VL")
140
+ ```
141
+
142
+ Note: this checkpoint stores MoE experts in the **per-expert (split)** layout
143
+ (`...mlp.experts.{0..255}.{gate,up,down}_proj.weight`), inherited unchanged from upstream, while
144
+ transformers 5.x uses a **fused** layout internally. The conversion is automatic at load time — no
145
+ manual step is needed, and no weights are lost.
146
 
147
  ---
148