--- license: mit language: - en pipeline_tag: robotics tags: - vision-and-language-navigation - vln-ce - world-model - embodied-ai - habitat - navigation - robotics --- # SC²-WM Official checkpoints for **SC²-WM: A Self-Correcting World Model with Closed-Loop Feedback for Vision-and-Language Navigation in Continuous Environments** (ICML 2026). 📄 Paper: https://arxiv.org/abs/2608.07548 💻 Code: https://github.com/sunrise-yuze/SC2_WM Authors: Xuan Yao, Yuze Zhu, Junyu Gao, Zongmeng Wang, Changsheng Xu ## Overview Vision-and-Language Navigation in Continuous Environments (VLN-CE) requires agents to make fine-grained navigation decisions under partial observability. However, most existing methods rely on open-loop execution, lacking mechanisms to detect and correct internal state drift during inference. We propose **SC²-WM, a self-correcting world model framework that introduces internal feedback for closed-loop decision making** in VLN-CE. Our method derives feedback from world-model foresight to perform state-level plan refinement before action execution. To handle challenging scenarios, we further introduce conditional world-aware adaptation, which enables model-level correction by selectively updating the world model at test time when feedback indicates model capacity insufficiency. The framework is built on Habitat-Sim and is evaluated on the R2R-CE and RxR-CE benchmarks. ## Checkpoints This repository hosts the released weights for the **R2R-CE** setting. | File | Description | |---|---| | `ckpt.46600.pth` | R2R-CE main checkpoint (fine-tuned SC²-WM model) | | `ViT-B-16.pt` | CLIP ViT-B/16 vision encoder | | `pretrained/model_step_100000.pt` | VLN-BERT (R2R variant) | | `pretrained/model_step_82500.pt` | VLN-BERT (default) | | `pretrained/cwp_predictor.pth` | Candidate waypoint predictor | | `pretrained/NeRF_p16_8x8.pth` | NeRF rendering module | | `pretrained/segm.pt` | Image segmentation module | | `pretrained/resnet18-f37072fd.pth` | ResNet-18 backbone | ## Results ### R2R-CE (val_unseen) | Method | SR | SPL | NE ↓ | |---|---|---|---| | Baseline (VLN-3DFF) | 44.9 | 30.4 | 5.95 | | **SC²-WM (Ours)** | **50.9** | **37.2** | **5.37** | ### RxR-CE (val_unseen) | Method | SR | SPL | NE ↓ | |---|---|---|---| | Baseline (VLN-3DFF) | 25.5 | 18.1 | 8.79 | | **SC²-WM (Ours)** | **35.8** | **27.2** | **8.36** | ## Usage Please refer to our [GitHub repository](https://github.com/sunrise-yuze/SC2_WM) for installation, data preparation, and evaluation instructions. Download the checkpoints into the directory layout described in the repository README. You can download the weights with the `huggingface_hub` library: ```python from huggingface_hub import snapshot_download snapshot_download(repo_id="zhuyuze/SC2_WM", local_dir="SC2_WM") ``` ## Citation If you find this project useful in your research, please consider citing: ```bibtex @inproceedings{yao2026sc2wm, title = {{$SC^2$-WM}: A Self-Correcting World Model with Closed-Loop Feedback for Vision-and-Language Navigation in Continuous Environments}, author = {Yao, Xuan and Zhu, Yuze and Gao, Junyu and Wang, Zongmeng and Xu, Changsheng}, booktitle = {Proceedings of the 43rd International Conference on Machine Learning (ICML)}, year = {2026} } ``` ## License Released under the [MIT License](https://github.com/sunrise-yuze/SC2_WM/blob/main/LICENSE).