Image-Text-to-Text
Transformers
Safetensors
nemotron_parse
text-generation
VLM
OCR
Parse
conversational
custom_code
Eval Results
Instructions to use nvidia/Nemotron-3.5-Parse with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use nvidia/Nemotron-3.5-Parse with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("image-text-to-text", model="nvidia/Nemotron-3.5-Parse", trust_remote_code=True) messages = [ { "role": "user", "content": [ {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"}, {"type": "text", "text": "What animal is on the candy?"} ] }, ] pipe(text=messages)# pip install -U transformers accelerate # Load model directly from transformers import AutoModelForCausalLM model = AutoModelForCausalLM.from_pretrained("nvidia/Nemotron-3.5-Parse", trust_remote_code=True, device_map="auto") - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use nvidia/Nemotron-3.5-Parse with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "nvidia/Nemotron-3.5-Parse" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "nvidia/Nemotron-3.5-Parse", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }'Use Docker
docker model run hf.co/nvidia/Nemotron-3.5-Parse
- SGLang
How to use nvidia/Nemotron-3.5-Parse with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "nvidia/Nemotron-3.5-Parse" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "nvidia/Nemotron-3.5-Parse", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "nvidia/Nemotron-3.5-Parse" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "nvidia/Nemotron-3.5-Parse", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }' - Docker Model Runner
How to use nvidia/Nemotron-3.5-Parse with Docker Model Runner:
docker model run hf.co/nvidia/Nemotron-3.5-Parse
Download reference/parsebench/pipeline/engine/caption_relocation.py from nvidia/Nemotron-3.5-Parse: direct link, hf CLI and curl.
- Browser
- Download file 2.59 kB
-
https://huggingface.co/nvidia/Nemotron-3.5-Parse/resolve/main/reference/parsebench/pipeline/engine/caption_relocation.py
- Command line
-
hf download hf://nvidia/Nemotron-3.5-Parse/reference/parsebench/pipeline/engine/caption_relocation.py
-
curl -L -o caption_relocation.py https://huggingface.co/nvidia/Nemotron-3.5-Parse/resolve/main/reference/parsebench/pipeline/engine/caption_relocation.py
2.59 kB
| # SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. | |
| # SPDX-License-Identifier: OpenMDW-1.1 | |
| """Relocate only plain above-chart captions with geometric/preservation guards.""" | |
| import re | |
| from bs4 import BeautifulSoup | |
| BLOCK=re.compile(r'(<x_([\d.]+)><y_([\d.]+)>)(.*?)(<x_([\d.]+)><y_([\d.]+)><class_([^>]+)>)',re.S) | |
| def plain(text): | |
| text=text.replace('<tbc>','').strip() | |
| if any(marker in text for marker in ['**','__','~~','`']):return None | |
| if re.search(r'(?<!\w)_[^_\n]+_(?!\w)|<[/a-zA-Z!]|\\[A-Za-z&%_]|\]\(',text):return None | |
| return text if text and '\n' not in text else None | |
| def relocate_above_captions(raw,markdown,render_base): | |
| matches=list(BLOCK.finditer(raw));eligible={};targets={};moved=set() | |
| allowed={'Chart'} | |
| boxes=[tuple(map(float,[m[2],m[3],m[6],m[7]])) for m in matches] | |
| for i,m in enumerate(matches): | |
| if m[8]!='Caption' or plain(m[4]) is None:continue | |
| box=boxes[i];candidates=[] | |
| for j in [i-1,i+1]: | |
| if not 0<=j<len(matches) or matches[j][8] not in allowed:continue | |
| other=boxes[j] | |
| if box[3]>other[1]:continue # Attach above-object captions only; retain notes below as standalone. | |
| width=min(box[2]-box[0],other[2]-other[0]);overlap=max(0,min(box[2],other[2])-max(box[0],other[0]));gap=max(other[1]-box[3],box[1]-other[3]) | |
| if width>0 and overlap/width>=.8 and 0<=gap<=.03:candidates.append((gap,j)) | |
| candidates.sort() | |
| if not candidates or (len(candidates)>1 and abs(candidates[0][0]-candidates[1][0])<1e-9):continue | |
| gap,j=candidates[0];eligible.setdefault(j,[]).append((gap,i)) | |
| for j,caps in eligible.items(): | |
| caps.sort() | |
| if len(caps)>1 and abs(caps[0][0]-caps[1][0])<1e-9:continue | |
| caption_index=caps[0][1];label=plain(matches[caption_index][4]);m=matches[j];body=m[4] | |
| if len(re.findall(r'<chart\b',body))!=1:continue | |
| title=re.search(r'(?m)^title:[ \t]*([^\n]*)',body) | |
| if title: | |
| if label not in title[1]:body=body[:title.start()]+'title: '+title[1].strip()+' — '+label+body[title.end():] | |
| else:body=re.sub(r'(<chart\b[^>]*>)',lambda q:q[0]+'\ntitle: '+label,body,count=1) | |
| derived=m[1]+body+m[5] | |
| rendered=render_base(derived) | |
| # Do not move text unless it survives inside an actual table caption. | |
| captions=BeautifulSoup(rendered,'html.parser').find_all('caption') | |
| if not any(label in cap.get_text(' ',strip=True) for cap in captions):continue | |
| targets[m.start()]=derived | |
| moved.add(matches[caption_index].start()) | |
| if not targets:return markdown | |
| derived=BLOCK.sub(lambda m:'' if m.start() in moved else targets.get(m.start(),m[0]),raw) | |
| return render_base(derived) | |