--- license: other license_name: swift-open-license-v1.0-and-qwen-community-license license_link: LICENSE base_model: ukisai/Swift1.5-Qwen3.8-Flash-Next tags: [moe, pruning, expert-pruning, distillation, qwen4_exp] --- Wren logo # Wren **Wren** is a compact derivative of [Swift 1.5 Qwen3.8-Flash-Next](https://huggingface.co/ukisai/Swift1.5-Qwen3.8-Flash-Next) by UkisAI, which is itself a derivative of [Qwen3.8-Flash-Next](https://huggingface.co/Qwen/Qwen3.8-Flash-Next). This repository holds the bf16 weights with **50% of the routed experts removed** and the routers and norms healed. "Swift" and "UkisAI" are trademarks of UkisAI and are used here only to describe the model's origin. - 256 of 512 routed experts are kept per layer, chosen by maximin over REAP saliency measured on 13 areas, including agentic data. The MTP head was removed. - Routers and norms were healed by logit distillation from Swift's top-20 log-probs (reasoning effort xhigh), on 3.6M tokens. - Evaluation, on the same 300 questions: all-domain quiz 83.3 vs 88.0 for intact Swift; synthetic logic 70.0 vs 74.7. - Training logs, masks and evaluations are in `training/`. **Quantised versions** (GGUF, including the per-expert mixed-precision Wren_T3 for a 12 GB GPU) and **the research paper** that documents every phase are in [ohmychemo/Wren-GGUF](https://huggingface.co/ohmychemo/Wren-GGUF). ## License The licenses of the base models apply: - Swift Open License v1.0 for the UkisAI contribution, see [LICENSE](LICENSE); - Qwen Community License 1.0 for Qwen3.8-Flash-Next, see [LICENSE-QWEN](LICENSE-QWEN). Attribution and the list of changes are in [NOTICE](NOTICE). Commercial use above the thresholds stated in the licenses needs a separate license.