Qwopus3.8-27B-Flash-V2-RBLN-0.11.2

Jackrong/Qwopus3.8-27B-Flash-V2, pre-compiled for 16 Rebellions RBLN-CA25 NPUs with optimum-rbln. This repository contains compiled artifacts, tokenizer and processor configuration; RBLN NPUs and the RBLN SDK are required.

Full 64-layer prefill, batch-1 decode, and vision compilation completed successfully. Runtime loading, generation, long-context accuracy, and device memory fit have not been tested. This is a compile-validated release, not an inference benchmark.

Hardware and software

Component	Configuration
NPU target	16× RBLN-CA25; language model and vision each compiled across 16 devices
Compiler	rebel-compiler==0.11.2
Wrapper	optimum-rbln==0.11.2
Installed serving stack (not exercised)	vllm-rbln==0.11.2.post1, vllm==0.24.0+cpu
Framework	transformers==5.8.1, torch==2.11.0+cpu
Weights	Original BF16 checkpoint, loaded exactly into FP32 host tensors
Host graph dtype	float32 host interface; compiler lowering determines NPU arithmetic precision

Compile configuration

Option	Value
max_seq_len	262,144 (source model's native configured maximum)
batch_size	1
decoder_batch_sizes	[1]
attn_impl / kvcache_partition_len	flash_attn / 16,384
kvcache_num_blocks	16 (262,144 shared KV token positions)
prefill_chunk_size / gdn_chunk_size	128 / 128
Layers	64: 48 GatedDeltaNet + 16 full-attention
Vision max_seq_len / batch_size	1,024 / 1
Phases	prefill, decode
MTP	Excluded; no speculative decoding artifact

Only a batch-1 decoder is included. There are no larger dynamic-batch decoder variants in this release. Input and generated tokens share the context budget. The compiled context length does not establish long-context quality or runtime fit.

Serving

The following is an untested starting configuration for the installed vLLM-RBLN stack; runtime compatibility and memory usage still need validation.

export VLLM_PLUGINS=rbln
export VLLM_RBLN_USE_VLLM_MODEL=0
export VLLM_RBLN_NUM_DEVICES_PER_LOCAL_RANK=16
export VLLM_RBLN_SUB_BLOCK_CACHE=false

vllm serve iAcloud/Qwopus3.8-27B-Flash-V2-RBLN-0.11.2 \
  --dtype float32 \
  --max-model-len 262144 \
  --max-num-seqs 1 \
  --max-num-batched-tokens 128 \
  --enable-chunked-prefill \
  --no-enable-prefix-caching

Do not pass --tensor-parallel-size 16: the 16-device partition is baked into the RBLN artifacts; the vLLM process uses its default tensor parallel size of 1. Use a compatible runtime and retain the compiled prefill and batch dimensions. Vision support is included in the compiled files but has not been exercised.

Provenance and validation

• Source: Jackrong/Qwopus3.8-27B-Flash-V2
• Source revision: 13f92e09a46fa364f8de1edb85684d57bda01126.
• Loaded all 1,184 main-model parameter tensors; omitted the 15 auxiliary MTP tensors.
• Applied a process-local GDN prefix-sum patch: replace cumsum with the equivalent triangular matrix multiplication to work around SDK TP lowering.
• Source BF16 values are exactly representable in the FP32 host model. Compiler lowering controls device precision.
• Full compilation completed in 1,150.3 seconds. Artifact payload is approximately 55.8 GiB; this file size is not a device-memory budget.
• No NPU inference or runtime creation was performed.

compile_status.json records the local compilation result. Source model training, post-training claims, and intended usage are documented by the original author.

License

Apache-2.0, as declared by the source repository. Credit for the model belongs to Jackrong and the Qwen contributors. This repository provides an RBLN compilation.