| --- |
| license: other |
| license_name: swift-open-license-v1.0-and-qwen-community-license |
| license_link: LICENSE |
| base_model: ukisai/Swift1.5-Qwen3.8-Flash-Next |
| tags: [moe, pruning, expert-pruning, distillation, qwen4_exp] |
| --- |
| <img src="https://huggingface.co/ohmychemo/Wren/resolve/main/WrenLogo.svg" width="100%" alt="Wren logo"> |
|
|
| # Wren |
|
|
| **Wren** is a compact derivative of [Swift 1.5 Qwen3.8-Flash-Next](https://huggingface.co/ukisai/Swift1.5-Qwen3.8-Flash-Next) by UkisAI, which is itself a derivative of [Qwen3.8-Flash-Next](https://huggingface.co/Qwen/Qwen3.8-Flash-Next). This repository holds the bf16 weights with **50% of the routed experts removed** and the routers and norms healed. "Swift" and "UkisAI" are trademarks of UkisAI and are used here only to describe the model's origin. |
|
|
| - 256 of 512 routed experts are kept per layer, chosen by maximin over REAP saliency measured on 13 areas, including agentic data. The MTP head was removed. |
| - Routers and norms were healed by logit distillation from Swift's top-20 log-probs (reasoning effort xhigh), on 3.6M tokens. |
| - Evaluation, on the same 300 questions: all-domain quiz 83.3 vs 88.0 for intact Swift; synthetic logic 70.0 vs 74.7. |
| - Training logs, masks and evaluations are in `training/`. |
|
|
| **Quantised versions** (GGUF, including the per-expert mixed-precision Wren_T3 for a 12 GB GPU) and **the research paper** that documents every phase are in [ohmychemo/Wren-GGUF](https://huggingface.co/ohmychemo/Wren-GGUF). |
| |
| ## License |
| The licenses of the base models apply: |
| - Swift Open License v1.0 for the UkisAI contribution, see [LICENSE](LICENSE); |
| - Qwen Community License 1.0 for Qwen3.8-Flash-Next, see [LICENSE-QWEN](LICENSE-QWEN). |
| |
| Attribution and the list of changes are in [NOTICE](NOTICE). Commercial use above the thresholds stated in the licenses needs a separate license. |
| |
