UW-SyFI/servingstudio-corpora

ServingStudio token corpora

Recorded per-token MoE routes for routing: corpus in ServingStudio Sim. Each directory is one capture: manifest.json (dimensions + FNV-1a checksum), routes.u16 (little-endian u16, C order [token, layer, top_k]), and provenance.json. Reference a file by commit, never by branch:

token_corpus_file: hf://UW-SyFI/servingstudio-corpora@<commit-sha>/glm52_nvfp4_mtp5/manifest.json

glm52_nvfp4_mtp5

• Model: nvidia/GLM-5.2-NVFP4 @ aec724e8c7b8ee9db3b48c01c320f63f9cdaf8aa
• Deployment: vLLM, B200, TP4 EP4 DP1, MTP num_speculative_tokens=5, fp8 KV
• Workload: alignment pack glm52_nvfp4_b200_spec5, case 07_quadrant_interference_c48 (192 requests, enwik9 prompts, concurrency 48)
• Captured by the profile_corpus (profile_kind: token_corpus) pass, Slurm job 952, 2026-09-23
• 307,008 accepted generated tokens x 76 layers (75 body MoE layers + the MTP layer) x top-8 of 256 experts
• Scope: accepted generated tokens only; no prompt or rejected-draft routes

glm53_nvfp4_dflash2

• Model: incoai/GLM-5.3-NVFP4 @ 54e52520606f96b3d9fc84088ad22882a61648ac, proposer incoai/GLM-5.3-DFlash2 @ 425aa615ce320caac34400208b30808c8f14f76c
• Deployment: vLLM (fork 79838a5a, v0.28 base), B200, TP4 EP4 DP1, DFlash2 num_speculative_tokens=7, fp8 KV, max_num_seqs 64
• Workload: trace/diverse_100.csv (100 requests, enwik9 prompts, concurrency 64)
• Captured by a one-off capture_corpus.py (non-streaming /v1/completions routed_experts), Slurm job 823, 2026-09-22
• 135,144 accepted generated tokens x 75 layers (the body MoE layers; the target runs no MTP layer) x top-8 of 256 experts
• Scope: accepted generated tokens only; no prompt or rejected-draft routes

glm53_flash_fp8_tp4_ep4

• Model: zai-org/GLM-5.3-Flash @ 03eb5366286afd40d2221b1d9c63a6dd1ba4832e (FP8 block checkpoint)
• Deployment: vLLM (fork 3f667d7e, v0.28 base), B200, TP4 EP4 DP1, no speculative decoding, fp8 KV, max-model-len 8192
• Workload: 256 independent requests (input 256/1024/2048/4096 x output 128/256, 32 each), enwik9 prompts, saturated at concurrency 32
• Captured by the profile_kind: token_corpus pass, Slurm job 1071, 2026-09-24
• 48,896 accepted generated tokens x 42 layers (the MoE layers 3-44) x top-8 of 288 experts
• Scope: accepted generated tokens only; no prompt routes