Instructions to use pei2333/SuperGlance-9B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use pei2333/SuperGlance-9B with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("image-text-to-text", model="pei2333/SuperGlance-9B") messages = [ { "role": "user", "content": [ {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"}, {"type": "text", "text": "What animal is on the candy?"} ] }, ] pipe(text=messages)# pip install -U transformers accelerate # Load model directly from transformers import AutoProcessor, AutoModelForMultimodalLM processor = AutoProcessor.from_pretrained("pei2333/SuperGlance-9B") model = AutoModelForMultimodalLM.from_pretrained("pei2333/SuperGlance-9B", device_map="auto") messages = [ { "role": "user", "content": [ {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"}, {"type": "text", "text": "What animal is on the candy?"} ] }, ] inputs = processor.apply_chat_template( messages, add_generation_prompt=True, tokenize=True, return_dict=True, return_tensors="pt", ).to(model.device) outputs = model.generate(**inputs, max_new_tokens=256) print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:])) - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use pei2333/SuperGlance-9B with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "pei2333/SuperGlance-9B" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "pei2333/SuperGlance-9B", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }'Use Docker
docker model run hf.co/pei2333/SuperGlance-9B
- SGLang
How to use pei2333/SuperGlance-9B with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "pei2333/SuperGlance-9B" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "pei2333/SuperGlance-9B", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "pei2333/SuperGlance-9B" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "pei2333/SuperGlance-9B", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }' - Docker Model Runner
How to use pei2333/SuperGlance-9B with Docker Model Runner:
docker model run hf.co/pei2333/SuperGlance-9B
SuperGlance-9B
Paper: arXiv:2611.19138 ยท PDF ยท Project page: guobapei.github.io/SuperGlance ยท Code: GuobaPei/SuperGlance
Other checkpoints: SuperGlance-9B-Stage1 ยท SuperGlance-9B-Stage2
Waymo Open Dataset notice. This model was made using the Waymo Open Dataset, provided by Waymo LLC under the Waymo Dataset License Agreement for Non-Commercial Use, available at waymo.com/open/terms, and your access and use of such work are governed by the terms and conditions therein. This model is released under that Agreement. Any further downstream use or modification of it is subject to the Terms of that Agreement, including its non-commercial restrictions. It may not be used in the operation of a vehicle, in production systems or for primarily commercial purposes.
The release terms of this repository are not decided yet; see Licence below.
SuperGlance is an end-to-end driving model. It reads the current front-camera frame and, when all three are available, the frames 1.5, 1.0 and 0.5 s earlier. The prompt gives the ego speed, acceleration, past 1.5 s of positions, the navigation command and any instruction. The model then scores a fixed library of 5,273 candidate trajectories. Each candidate is 8 future poses (x, y, heading). This repository holds the final model after the reinforcement-learning stage (RL), with its LoRA merged into the stage-2 model, SuperGlance-9B-Stage2. That model is Qwen3.5-9B after SuperGlance's two question-answering stages.
Files
| file | content |
|---|---|
model.safetensors-*-of-00004.safetensors, model.safetensors.index.json, config.json, tokenizer and processor files |
the merged model (bf16), loads with AutoModelForImageTextToText |
choice_head.safetensors |
the Policy Head: weight [5273, 4096], bias [5273] |
candidates_k5273.npy |
the candidate library, [5273, 8, 3]: x, y (m) and heading (rad) of 8 future poses |
SuperGlance.pdf |
the paper (the arXiv version) |
extend_head.json |
how the head was extended from 5,121 to 5,273 candidates before RL |
adapter/ |
RL's LoRA and Policy Head before the merge (the path the paper's numbers were read with). Its config names Qwen/Qwen3.5-9B as the architecture, but it applies to SuperGlance-9B-Stage2: load that model first and use it as the base |
MERGE_INFO.json |
merge arithmetic: Wโฒ = round_bf16(W + 2.0 ยท BยทA) for 200 matrices |
LICENSE-Qwen3.5-9B |
the base model's licence (Apache-2.0), kept as its terms require |
MERGE_INFO.json and extend_head.json use internal names: S1 is the last phase, RL' is RL, and the choice head is the Policy Head.
Readout
- h = the final hidden state at the last prompt position.
- Logits = hยทWแต + b over the 5,273 candidates.
- Plan = the softmax (ฯ = 0.2) over the 64 highest logits, as weights on those candidates' poses.
The code release has the prompt builder and the readout.
The merged weights and the adapter path are not bit-identical. On 16 navtest scenes:
- the head is identical;
- the top candidate agrees on 15 of 16 scenes;
- the largest plan difference is 0.16 m in position and 0.009 rad in heading.
These differences come from rounding the merged weights to bf16.
Results (paper)
| benchmark | metric | value |
|---|---|---|
| NAVSIM v1.1 navtest | PDMS | 91.51 |
| WOD-E2E test | RFS | 7.987 |
Training
The base model is Qwen/Qwen3.5-9B (Apache-2.0). The training ran in this order:
- Stage 1. Multiple-choice QA, 1,158,150 questions.
- Stage 2. The stage-1 questions except
text_notaandopenimages_vertical(1,115,662), plus 124,378 DriveLM-nuScenes questions, giving SuperGlance-9B-Stage2. - SFT. Supervised driving on NAVSIM navtrain (OpenScene / nuPlan), PhysicalAI-AV and WOD-E2E rows, with the evaluation splits removed.
- Last phase. A continuation on the same pool, with the WOD-E2E rows replaced by train-split frames 135โ165.
- RL. Reinforcement learning on the candidate library.
Licence
The release terms are pending. These are the terms of the training data, taken from each source's own page:
| data | terms |
|---|---|
| Qwen3.5-9B (base) | Apache-2.0 |
| WOD-E2E | Waymo non-commercial licence; trained models may be shared under the conditions above |
| NAVSIM navtrain (OpenScene / nuPlan) | CC BY-NC-SA 4.0 (OpenScene) plus the nuPlan / nuScenes terms of use |
| DriveLM-nuScenes | CC BY-NC-SA 4.0, plus the nuScenes terms of use |
| PhysicalAI-AV | NVIDIA Autonomous Vehicle Dataset License Agreement (internal development use; no derivative works or redistribution) |
| stage-1 QA sources | see the SuperGlance-QA dataset card |
- Downloads last month
- 73
Model tree for pei2333/SuperGlance-9B
Base model
Qwen/Qwen3.5-9B-Base