Qwen3.6-35B-A3B-REAP20

Qwen/Qwen3.6-35B-A3B with 20% of its MoE experts structurally pruned with REAP (PRUNE method), produced with llmcompressor.

  • Every one of the 40 MoE layers keeps its 205 most salient experts of the original 256 (51 pruned per layer, 8200 of 10240 retained)
  • Expert selection is uniform across layers: each layer's 51 lowest-REAP-saliency experts were removed, and the routers were sliced to match
  • config.json records the new expert count (num_experts: 205); expert indices are remapped contiguously
  • The MTP layer (mtp.layers.0) was not calibrated and is left untouched
  • All remaining weights (including attention, vision tower, embeddings and the shared experts) are untouched BF16

REAP scores each expert by the router-weighted mean of its output activation norms over the calibration tokens routed to it, S_j = mean(g_j * ||f_j||_2), using forward computation only (no gradients or recovery training). This model was calibrated on all 2,048 samples of the RazorCal multi-domain calibration corpus; see the REAP expert pruning example for the calibration and pruning scripts.

Usage

from transformers import AutoProcessor, Qwen3_5MoeForConditionalGeneration

model = Qwen3_5MoeForConditionalGeneration.from_pretrained(
    "inference-optimization/Qwen3.6-35B-A3B-REAP20", torch_dtype="bfloat16"
)
processor = AutoProcessor.from_pretrained(
    "inference-optimization/Qwen3.6-35B-A3B-REAP20"
)
Downloads last month
49
Safetensors
Model size
30B params
Tensor type
BF16
ยท
Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐Ÿ™‹ Ask for provider support

Model tree for inference-optimization/Qwen3.6-35B-A3B-REAP20

Finetuned
(533)
this model

Collection including inference-optimization/Qwen3.6-35B-A3B-REAP20