# Serve Nemotron 3.5 Parse

## Native DFlash2

Download both models together. Use vLLM with native Nemotron Parse and
`DFlash2DraftModel` support; no model-specific plugin is needed.

```bash
hf download nvidia/Nemotron-3.5-Parse --local-dir ./parse35
vllm serve ./parse35 \
  --served-model-name nemotron-parse \
  --trust-remote-code \
  --speculative-config '{"method":"dflash","model":"./parse35/dflash2","num_speculative_tokens":32}'
```

The speculator path is a local subdirectory, not a second Hub model ID.
Use 8, 16, or 32 speculative tokens to tune for your workload; the length stays
fixed for that server. See [performance](../dflash2/README.md#performance).
For base inference, omit `--speculative-config`.

Use the [model card's HTTP or Transformers examples](../README.md#run-parse).
Transformers does not need the serving plugin and uses normal decoding.
For overlapping document/crop requests, use the downloaded
[`DocumentParser` helpers](postprocessing.md).

## Prompts

```python
TASK_PROMPT = "</s><s><predict_bbox><predict_classes><output_markdown><predict_text_in_pic>"
```

Use this task string in Transformers and HTTP requests. Preserve the class and
coordinate tokens in generated text for postprocessing. For offline vLLM,
prepend the image placeholder directly: `"<pad>" + TASK_PROMPT`, with
`add_special_tokens=False`. Do not insert a newline between them.

The model takes RGB page images; render PDFs to pages first. Coordinates refer
to the evaluated image content. The [prompt controls](../README.md#task-prompts)
request boxes, classes, or plain text; table export formats are chosen after inference.

## Throughput tuning

Concurrency is the number of outstanding requests across all stages, not the
number of documents. Increase it while observing memory, latency and completed
page throughput. Double and triple pass reuse page/crop images, so retain the
multimodal processor cache. Prefix caching, batching budgets and draft length
can affect different workloads differently.

Use [the measured profiles and protocols](../reference/dflash2-passes/README.md)
when reproducing a number. The basic launch command above prioritizes usability;
it does not set every benchmarking option. A page that hits its token limit is
partial output, even if the benchmark itself finishes successfully.

## One-token MTP

MTP weights are included in the target weights. The established MTP serving path
uses the repository's serving plugin. Follow the
[reference installation](serving-reference.md#installation-reference-mtp-plugin-environment)
and [MTP command](serving-reference.md#one-token-mtp) in a separate environment.
That guide preserves its pinned reproduction setup. Do not install its serving
extras into the native DFlash2 environment. Choose one speculator per server.