SaylorTwift HF Staff
vLLM: --enable-auto-tool-choice --tool-call-parser lfm2 (OpenCode sends tool_choice=auto)
f4740f8 verified | # Policy server: vLLM serving LiquidAI/LFM2.5-2.6B on GPU 1. | |
| # | |
| # Flag provenance: | |
| # --return-tokens-as-token-ids capture proxy needs the exact sampled token ids | |
| # --logprobs-mode processed_logprobs proxy needs per-token logprobs that match the policy | |
| # --weight-transfer-config TRL AsyncGRPOTrainer pushes merged weights over NCCL | |
| # VLLM_SERVER_DEV_MODE=1 exposes the weight-update endpoint TRL posts to | |
| # | |
| # Deliberately NOT passed: | |
| # --enable-lora adapter sync rotates adapter names, which the | |
| # OpenEnv capture proxy cannot address; the trainer | |
| # merges the LoRA into the base and syncs full weights | |
| set -euo pipefail | |
| MODEL="${MODEL:-LiquidAI/LFM2.5-2.6B}" | |
| PORT="${PORT:-8001}" | |
| CUDA_VISIBLE_DEVICES=1 VLLM_SERVER_DEV_MODE=1 \ | |
| vllm serve "$MODEL" \ | |
| --port "$PORT" \ | |
| --max-model-len 32768 \ | |
| --return-tokens-as-token-ids \ | |
| --logprobs-mode processed_logprobs \ | |
| --enable-auto-tool-choice \ | |
| --tool-call-parser lfm2 \ | |
| --weight-transfer-config '{"backend":"nccl"}' | |