Instructions to use inclusionAI/Ling-3.1-flash with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Inference
- HuggingChat
- Notebooks
- Google Colab
- Kaggle
Fix expert_swiglu_limit_list in modeling
#10
by ChangxinTian - opened
Fixes the missing SwiGLU clamps in the HF modeling code to align with SGLang, addressing discussion #9.
Thanks for the detailed investigation and helpful feedback
Smoke Test Results: Using the full Ling-3.0-flash checkpoint, we compared the original implementation, the new implementation, and the new implementation with clipping disabled.
- Backward compatibility: With clipping disabled, generated token IDs, prefill logits, and fixed-input cached-decode logits matched the original implementation exactly (maximum difference: 0).
- Clipping takes effect: Clipping was triggered in 189 of 1,981 sampled MLPs during inference. The maximum same-input prefill-logit difference was 7.84375.
- Numerical correctness: Captured MLP outputs matched the expected SwiGLU clipping formula with maximum error: 0. Per-layer thresholds, shared-expert configuration, and out-of-range MTP behavior passed validation.
- Inference coverage: All 9 full-model inference runs and 13 component checks passed. All three variants returned
579for the arithmetic case.
michael-qiu changed pull request status to open
michael-qiu changed pull request status to merged