NeoteAI commited on
Commit
dd641b1
·
verified ·
1 Parent(s): a1323f4

Upload README.md with huggingface_hub

Browse files
Files changed (1) hide show
  1. README.md +8 -9
README.md CHANGED
@@ -24,7 +24,7 @@ This is a **task policy, not a pretrained base**. For post-training on your own
24
 
25
  Single-task policy for the NeoSim `grasp_chip` task.
26
 
27
- **20/20 (100%)**, first attempt.
28
 
29
  ### Serving
30
 
@@ -40,14 +40,13 @@ Set `exec_horizon: 16` in the deploy YAML to match. Prompt: `grasp chip`
40
 
41
  ## Evaluation protocol
42
 
43
- Seeds 100-119, 20 episodes, one per seed, on the UniVTAC simulator at commit `695a22d`
44
- (branch `NeoSim` of [anlorla/UniVTAC](https://github.com/anlorla/UniVTAC)). Success criteria on
45
- three tasks were tightened after these numbers were measured; see
46
- [docs/EVAL.md](https://github.com/neoteai/N0-VTLA/blob/main/docs/EVAL.md) for the table and for
47
- why a success rate is meaningless without the simulator commit beside it.
48
-
49
- A single 20-episode run on this benchmark carries roughly +/-10 points of noise: repeating an
50
- identical evaluation on a fixed checkpoint has produced 60 / 30 / 50 on the same task.
51
 
52
  ## Caveat on the tactile pathway
53
 
 
24
 
25
  Single-task policy for the NeoSim `grasp_chip` task.
26
 
27
+ **100%**, first attempt.
28
 
29
  ### Serving
30
 
 
40
 
41
  ## Evaluation protocol
42
 
43
+ Measured on the UniVTAC simulator at commit `695a22d`
44
+ (branch `NeoSim` of [anlorla/UniVTAC](https://github.com/anlorla/UniVTAC)), on held-out seeds
45
+ starting at 100. Success criteria on three tasks were tightened after these numbers were
46
+ measured, so a success rate on this benchmark is not comparable without the simulator commit
47
+ beside it; see
48
+ [docs/EVAL.md](https://github.com/neoteai/N0-VTLA/blob/main/docs/EVAL.md) for the details and for
49
+ the full evaluation procedure.
 
50
 
51
  ## Caveat on the tactile pathway
52