README.md

---
license: other
license_name: swift-open-license-v1.0-and-qwen-community-license
license_link: LICENSE
base_model: ukisai/Swift1.5-Qwen3.8-Flash-Next
tags: [moe, pruning, expert-pruning, distillation, qwen4_exp]
---
<img src="https://huggingface.co/ohmychemo/Wren/resolve/main/WrenLogo.svg" width="100%" alt="Wren logo">

# Wren

**Wren** is a compact derivative of [Swift 1.5 Qwen3.8-Flash-Next](https://huggingface.co/ukisai/Swift1.5-Qwen3.8-Flash-Next) by UkisAI, which is itself a derivative of [Qwen3.8-Flash-Next](https://huggingface.co/Qwen/Qwen3.8-Flash-Next). This repository holds the bf16 weights with **50% of the routed experts removed** and the routers and norms healed. "Swift" and "UkisAI" are trademarks of UkisAI and are used here only to describe the model's origin.

- 256 of 512 routed experts are kept per layer, chosen by maximin over REAP saliency measured on 13 areas, including agentic data. The MTP head was removed.
- Routers and norms were healed by logit distillation from Swift's top-20 log-probs (reasoning effort xhigh), on 3.6M tokens.
- Evaluation, on the same 300 questions: all-domain quiz 83.3 vs 88.0 for intact Swift; synthetic logic 70.0 vs 74.7.
- Training logs, masks and evaluations are in `training/`.

**Quantised versions** (GGUF, including the per-expert mixed-precision Wren_T3 for a 12 GB GPU) and **the research paper** that documents every phase are in [ohmychemo/Wren-GGUF](https://huggingface.co/ohmychemo/Wren-GGUF).

## License
The licenses of the base models apply:
- Swift Open License v1.0 for the UkisAI contribution, see [LICENSE](LICENSE);
- Qwen Community License 1.0 for Qwen3.8-Flash-Next, see [LICENSE-QWEN](LICENSE-QWEN).

Attribution and the list of changes are in [NOTICE](NOTICE). Commercial use above the thresholds stated in the licenses needs a separate license.