S4VY: Segment Anything in Feed-Forward 4D Visual Geometry

Model weights for S4VY, a promptable instance segmentation model built on feed-forward 4D visual geometry. S4VY turns RGB observations into class-agnostic 4D instance masks, where each persistent object query tracks one entity across every view, and grounds them in natural language through an agentic system.

This is a research project from NVIDIA Research.

Jingdong Zhang, Xin Li, Jan Kautz, Wenping Wang, Chris Choy · arXiv:2609.36875

Files

Path	What it is
segmenter/s4vy.pt	the segmenter behind every headline number
segmenter/s4vy_prompt.pt	prompt encoder for point and box prompts
segmenter/s4vy_joint.pt	the from-scratch joint variant
grounder/full_model.pt	dual-stream grounder
critic/full_model.pt	critic that picks between the two mask candidates

Licensing — read before use

These checkpoints bundle upstream model weights. They are not a single-license release, and the terms differ per file. NOTICE gives the full breakdown; the summary is:

• segmenter/* contain VGGT-Omega weights. Under the FAIR Noncommercial Research License (LICENSE), derivative works may be distributed only under that same Agreement, and a copy must reach every recipient. Downloading these files means accepting it. Noncommercial research use only.
• grounder/* and critic/* contain Qwen3-VL-4B-Instruct and Qwen3-VL-2B-Instruct weights, licensed Apache-2.0 (LICENSE.Apache-2.0). Our modifications are described in NOTICE.
• Training data terms apply on top. Several sources are noncommercial-only, and KITTI-STEP carries CC BY-NC-SA. See NOTICE §4 and docs/DATA.md in the code repository.

The binding constraint across the whole repository is therefore noncommercial research use.

Citation

@article{zhang2026s4vy,
  title   = {S4VY: Segment Anything in Feed-Forward 4D Visual Geometry},
  author  = {Zhang, Jingdong and Li, Xin and Kautz, Jan and Wang, Wenping and Choy, Chris},
  journal = {arXiv preprint arXiv:2609.36875},
  year    = {2026}
}