fantastisk's picture
Add files using upload-large-folder tool
da4e5c6 verified
|
Raw History Blame Contribute Delete
2.59 kB
---
license: apache-2.0
base_model:
- Qwen/Qwen3.8-27B
pipeline_tag: text-generation
tags:
- slipstream
- dgx-spark
- nvfp4
- qwen
---
# Qwen3.8-27B-Slipstream
Qwen3.8-27B (dense) for the Slipstream inference engine: our own NVFP4 quantization of the official BF16 weights with MSE-chosen block scales, MXFP8 prefill copies and the MTP head. Runs on one Spark (TP1) or two (TP2).
These are **Slipstream weight files** (`weights.vbin` + `index.json` per directory), the engine's own pre-packed format. They run only with Slipstream; they are not safetensors and do not load in other runtimes. Built with Slipstream's converters at commit `20c2dd9` (2026-10-31).
Base model: [Qwen/Qwen3.8-27B](https://huggingface.co/Qwen/Qwen3.8-27B). Engine: [Slipstream](https://github.com/Interactify/slipstream) (engine code release coming soon). Hardware: two NVIDIA DGX Spark (GB10, 128 GB each) connected over their 200 GbE link.
## Contents
| Directory | What | Node |
|---|---|---|
| `qwen38_27b8` | model weights for one Spark (TP1) | single Spark |
| `qwen38_27b8_tp2_r0` | model weights, rank 0 (TP2) | first |
| `qwen38_27b8_tp2_r1` | model weights, rank 1 (TP2) | second |
## Download
On the first Spark (rank 0, runs the HTTP server):
```bash
hf download Interactify/Qwen3.8-27B-Slipstream --local-dir ~/vmodels --include "qwen38_27b8_tp2_r0/*"
```
On the second Spark (rank 1):
```bash
hf download Interactify/Qwen3.8-27B-Slipstream --local-dir ~/vmodels --include "qwen38_27b8_tp2_r1/*"
```
On a single Spark, download `qwen38_27b8` instead:
```bash
hf download Interactify/Qwen3.8-27B-Slipstream --local-dir ~/vmodels --include "qwen38_27b8/*"
```
The tokenizer, chat template and generation config:
```bash
hf download Qwen/Qwen3.8-27B --exclude "*.safetensors" # on the first Spark
```
The directory names match the defaults in Slipstream's `server/models.json`, so no configuration is needed.
## Run
```bash
python server/slipstream.py serve qwen3.8-27b
```
The server speaks the OpenAI and Anthropic APIs on port 8080. See the Slipstream README for profiles, the router and engine settings.
## Measured on Slipstream
- Decode, one stream with MTP drafts: structured 119 / prose 71 / code 93 / JSON 103 tokens/s (serial 26).
- Prefill (MXFP8 W8A8): 1541 tokens/s at 16k.
- Quality: NLL +1.0 % against BF16 (512 tokens of English text), the same top-1 accuracy.
## Notes
- Context up to 262,144 tokens.
- Slipstream runs this model from its `build-q35` build (see `server/models.json`).
## License
Apache 2.0, as the base model. See `LICENSE` and `NOTICE`.