emb-ai/traffic-sign-bench-models

TrafficSignBench Models

Planner checkpoints for TrafficSignBench — closed-loop evaluation of traffic-sign compliance. Drop these weights into the benchmark repo as checkpoints/ and the eval CLI picks them up automatically.

---

What's inside

Total ≈ 576 MB. IDM and PPO ship with the code — no download needed.

Path	Planner	Role	Size
carl/nuplan_51479_1B/model_best.pth	CaRL	nuPlan-trained baseline	7.6 MB
plant2_pretrain/epoch=029_final_3.ckpt	PlanT-2	pretrained baseline	426 MB
plant2_finetuned/plant2_supervised_2nd_final.pt	PlanT-2-FT	rule-supervised fine-tune	142 MB

---

Quick start

From a checkout of traffic-sign-bench:

# All checkpoints → ./checkpoints  (paths match eval defaults)
hf download emb-ai/traffic-sign-bench-models --local-dir checkpoints

# Or pull only what you need
hf download emb-ai/traffic-sign-bench-models \
  --include "carl/**" \
  --local-dir checkpoints

Then evaluate (scenes from the dataset must already be under data/scenes/):

python -m traffic_bench.eval manifest sign=yield paths.split=test
python -m traffic_bench.eval run policy=carl sign=yield
python -m traffic_bench.eval run policy=plant2 sign=yield
python -m traffic_bench.eval run policy=plant2_ft sign=yield

Override a weight explicitly with model_path=/path/to/file.

---

Layout

checkpoints/                          ← --local-dir target
├── carl/
│   └── nuplan_51479_1B/
│       └── model_best.pth            ← policy=carl, carl_rule
├── plant2_pretrain/
│   └── epoch=029_final_3.ckpt        ← policy=plant2, plant2_rule
└── plant2_finetuned/
    └── plant2_supervised_2nd_final.pt ← policy=plant2_ft

Keep this tree under the repo root. Renaming folders breaks the defaults in traffic_bench/eval/engine/sim/checkpoints.py.

---

Policy → checkpoint

policy=	Planner	Sign access	Uses
carl	CaRL	none	carl/.../model_best.pth
carl_rule	CaRL + rule overlay	privileged	same file
plant2	PlanT-2	none	plant2_pretrain/...ckpt
plant2_rule	PlanT-2 + rule overlay	privileged	same file
plant2_ft	PlanT-2-FT	learned from signs	newest *.pt / *.ckpt / *.pth in plant2_finetuned/

*_rule planners read the active rule directly — useful as oracle upper bounds, not as fair baselines.

---

License

Released under CC BY 4.0. Please credit the original planner authors when redistributing CaRL or PlanT-2 derivatives.

---

Citation

@misc{trafficsignbench2026,
  title     = {TrafficSignBench: Evaluating Traffic Rule Compliance
               in Autonomous Driving},
  year      = {2026},
  publisher = {GitHub},
  url       = {https://github.com/emb-ai/traffic-sign-bench}
}