Describe the navigation model, training method and visual capabilities
Browse files
README.md
CHANGED
|
@@ -12,9 +12,9 @@ tags:
|
|
| 12 |
|
| 13 |
**ZeroVLN: Learning to Navigate without Supervised Fine-Tuning of the VLM Brain**
|
| 14 |
|
| 15 |
-
|
| 16 |
|
| 17 |
-
|
| 18 |
|
| 19 |
## Checkpoint contents
|
| 20 |
|
|
@@ -51,7 +51,7 @@ Keep the directory name `global_step_000152` so the loader can match it to the s
|
|
| 51 |
|
| 52 |
## Evaluation
|
| 53 |
|
| 54 |
-
The table reports success rate (SR) and success weighted by path length (SPL), both in percent, for
|
| 55 |
|
| 56 |
| Benchmark | SR ↑ | SPL ↑ |
|
| 57 |
| --- | ---: | ---: |
|
|
|
|
| 12 |
|
| 13 |
**ZeroVLN: Learning to Navigate without Supervised Fine-Tuning of the VLM Brain**
|
| 14 |
|
| 15 |
+
ZeroVLN is a vision-and-language navigation model that follows natural-language instructions in continuous environments without navigation-domain supervised fine-tuning of its vision-language model (VLM) brain. At each interaction round, the brain describes the intended movement, a continuous-action expert executes it, and the brain uses the resulting observation to decide whether to move again or stop.
|
| 16 |
|
| 17 |
+
The action expert first learns from reference trajectories while the base brain remains frozen. Closed-loop, multi-turn reinforcement learning then updates both policies from trajectory returns. After joint training, the brain largely retains its general visual capabilities. Its mean accuracy across 12 visual benchmarks with free-form answers is **59.16%**, compared with **59.47%** for the base brain.
|
| 18 |
|
| 19 |
## Checkpoint contents
|
| 20 |
|
|
|
|
| 51 |
|
| 52 |
## Evaluation
|
| 53 |
|
| 54 |
+
The table reports success rate (SR) and success weighted by path length (SPL), both in percent, for ZeroVLN in closed-loop simulation.
|
| 55 |
|
| 56 |
| Benchmark | SR ↑ | SPL ↑ |
|
| 57 |
| --- | ---: | ---: |
|