hzxllll commited on
Commit
ee20c02
·
verified ·
1 Parent(s): 5118a68

Describe the navigation model, training method and visual capabilities

Browse files
Files changed (1) hide show
  1. README.md +3 -3
README.md CHANGED
@@ -12,9 +12,9 @@ tags:
12
 
13
  **ZeroVLN: Learning to Navigate without Supervised Fine-Tuning of the VLM Brain**
14
 
15
- This repository contains the **step-152 checkpoint** used for the main ZeroVLN results. Given a navigation instruction and a camera view, the VLM brain can request a movement in natural language. A continuous-action expert executes the request, using fresh camera views to adjust its motion. The brain then observes the result and chooses its next action, including when to stop.
16
 
17
- Training begins with the base brain frozen. A trajectory planner supplies reference trajectories for the expert to learn how to execute the brain’s requests. During closed-loop, multi-turn reinforcement learning, Trajectory-Level RLOO uses the return from each episode to update both the brain and the action expert. The brain receives no navigation-domain supervised fine-tuning.
18
 
19
  ## Checkpoint contents
20
 
@@ -51,7 +51,7 @@ Keep the directory name `global_step_000152` so the loader can match it to the s
51
 
52
  ## Evaluation
53
 
54
- The table reports success rate (SR) and success weighted by path length (SPL), both in percent, for the main checkpoint in closed-loop simulation.
55
 
56
  | Benchmark | SR ↑ | SPL ↑ |
57
  | --- | ---: | ---: |
 
12
 
13
  **ZeroVLN: Learning to Navigate without Supervised Fine-Tuning of the VLM Brain**
14
 
15
+ ZeroVLN is a vision-and-language navigation model that follows natural-language instructions in continuous environments without navigation-domain supervised fine-tuning of its vision-language model (VLM) brain. At each interaction round, the brain describes the intended movement, a continuous-action expert executes it, and the brain uses the resulting observation to decide whether to move again or stop.
16
 
17
+ The action expert first learns from reference trajectories while the base brain remains frozen. Closed-loop, multi-turn reinforcement learning then updates both policies from trajectory returns. After joint training, the brain largely retains its general visual capabilities. Its mean accuracy across 12 visual benchmarks with free-form answers is **59.16%**, compared with **59.47%** for the base brain.
18
 
19
  ## Checkpoint contents
20
 
 
51
 
52
  ## Evaluation
53
 
54
+ The table reports success rate (SR) and success weighted by path length (SPL), both in percent, for ZeroVLN in closed-loop simulation.
55
 
56
  | Benchmark | SR ↑ | SPL ↑ |
57
  | --- | ---: | ---: |