File size: 807 Bytes
d07994b | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 | """vLLM general plugin: install the cheap Qwen3.8-Flash-Next MTP drafter (q38_draft.py) in every vLLM process.
Inactive unless Q38_DRAFT_VOCAB is set, so the same environment still runs unpatched servers.
Q38_DRAFT_HOME directory holding q38_draft.py / nvfp4_ops.py (this repo's mtp_draft/)
Q38_DRAFT_VOCAB draft vocabulary mask (.pt bool[248320]); empty -> plugin inactive
Q38_DRAFT_HEAD_Q marlin (default) | cutlass | bf16 draft lm_head
Q38_DRAFT_MTP_Q marlin (default) | cutlass | bf16 MTP block BF16 linears
"""
import os
import sys
def main():
if not os.environ.get("Q38_DRAFT_VOCAB"):
return
home = os.environ.get("Q38_DRAFT_HOME")
if home and home not in sys.path:
sys.path.insert(0, home)
import q38_draft
q38_draft.install_from_env()
|