vLLM 0.10.x ignores rope_theta under rope_parameters and falls back to 10000
#1
by un-seen - opened
Thanks for the quant. We used it on 2x RTX 3090 for AIME, GPQA Diamond and IFEval runs, and for a neuron-steering study.
One thing to flag: config.json keeps rope_theta only under rope_parameters (the transformers 5.x layout). vLLM 0.10.x does not read it there and falls back to rope_theta = 10000, so output turns to garbage after about 4K tokens. Adding a top-level "rope_theta": 120000000 to config.json fixed it for us. Anyone on an older vLLM may hit the same thing.