reference/code/Nemotron-Labs-Diffusion/xp/dlm_api/README.md

dlm_api

OpenAI-compatible HTTP server that wraps the Nemotron-Labs diffusion-LM inference paths. This package backs the inference workers spawned by xp/examples/run_dlm_eval_pipeline_gpu_only.sh and is the runtime that xp/nemo-skills/eval_dlm.py talks to over HTTP.

Files

• dlm_batch_server.py — FastAPI worker. Owns model loading, request batching, NFE logging, and dispatching into one of the registered generation algorithms.
• dlm_openai_server.py — Pydantic request/response models and chat template helpers (no networking).
• dlm_load_balancer.py — Multi-GPU load balancer that fans requests out to N worker processes (one per GPU).
• dlm_generate/ — Generation algorithm registry. The three algorithms used by eval.sh:
  • nemotron — diffusion sampling (used by --mode dlm and, with LINEAR_SPECULATION=true, --mode linear_spec).
  • nemotron_mixed — mixed AR/dLM (loaded alongside nemotron from the same engine).
  • ar_native — pure autoregressive via the model's own ar_generate method (--mode ar).

The third-party fast_dllm / dinfer / dllm_eval / huggingface algorithm packages from the upstream LLaDA-API tree have been removed in this slim build because they target LLaDA-family models (e.g. GSAI-ML/LLaDA-8B-Instruct), not the Nemotron diffusion family.

How eval.sh uses this

Each SLURM job runs dlm_batch_server.py per GPU and a single dlm_load_balancer.py at the front. The eval client (eval_dlm.py) hits the load balancer over http://localhost:$LOAD_BALANCER_PORT/v1. All flags relevant to the four modes — --engine, --linear-speculation, --draft-lora-only, --lora-path, --max-thinking-tokens, --eos-early-stop, etc. — are documented in dlm_batch_server.py --help.