File size: 11,891 Bytes
7d45f7c | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 348 349 350 351 352 353 354 355 356 357 358 359 360 361 362 363 364 365 366 367 368 369 370 371 372 373 374 375 376 377 378 379 380 381 382 383 384 385 | ### 1.1 Core Example (`billboard_id=3`, default config is upper yellow frame — use ID as reference)
```powershell
& C:/Users/ydook/anaconda3/envs/xy-vla/python.exe d:/XY-VLA/run_xyVLA.py `
--cmd "Fly through only the rectangular portal marked billboard_id=18" `
--sim-steps 200 `
--infer-every 6 `
--xvla-steps 4 `
--log-every 10 `
--speed 5 `
--cmd-motion-amplify 2.5 `
--cmd-min-step 0.045 `
--infer-displacement-scale 1.6
```
### 1.2 Other IDs (change the number in `--cmd` only)
Lower ring example (default config **`billboard_id=12`** ≈ orange frame; still use ID as reference):
```powershell
& C:/Users/ydook/anaconda3/envs/xy-vla/python.exe d:/XY-VLA/run_xyVLA.py `
--cmd "Pass through portal billboard number 12 only; stay parallel to the long edge of billboard_id=12." `
--sim-steps 220 `
--infer-every 6 `
--xvla-steps 4 `
--log-every 10 `
--speed 5 `
--cmd-motion-amplify 2.5 `
--cmd-min-step 0.045 `
--infer-displacement-scale 1.6
```
Short phrase (suitable for batch scripts changing the last digit):
```powershell
& C:/Users/ydook/anaconda3/envs/xy-vla/python.exe d:/XY-VLA/run_xyVLA.py `
--cmd "Navigate through rectangular frame billboard_id=16; long-opening alignment; no other portals." `
--sim-steps 220 `
--infer-every 6 `
--xvla-steps 4 `
--log-every 10 `
--speed 5 `
--cmd-motion-amplify 2.5 `
--cmd-min-step 0.045 `
--infer-displacement-scale 1.6
```
### 1.3 Alternative: Color Commands (legacy compatibility, lower precision)
Easy to confuse when 2 portals share the same color on upper/lower layers; use only when **ambiguity is acceptable** or for **temporary comparison**.
```powershell
& C:/Users/ydook/anaconda3/envs/xy-vla/python.exe d:/XY-VLA/run_xyVLA.py `
--cmd "Fly through the yellow rectangular frame safely, align with the long opening" `
--sim-steps 220 `
--infer-every 6 `
--xvla-steps 4 `
--log-every 10 `
--speed 5 `
--cmd-motion-amplify 2.5 `
--cmd-min-step 0.045 `
--infer-displacement-scale 1.6
```
---
## 2. Capability Test B — Obstacle Avoidance & Safety Semantics (still targeting portal traversal)
```powershell
& C:/Users/ydook/anaconda3/envs/xy-vla/python.exe d:/XY-VLA/run_xyVLA.py `
--cmd "Fly through only billboard_id=3 safely; keep clearance from all other frames and obstacles" `
--sim-steps 280 `
--infer-every 5 `
--xvla-steps 4 `
--log-every 12 `
--speed 5 `
--cmd-motion-amplify 2.5 `
--cmd-min-step 0.045 `
--infer-displacement-scale 1.5
```
**Control**: Disable near-field repulsion (comparison experiment only):
```powershell
& C:/Users/ydook/anaconda3/envs/xy-vla/python.exe d:/XY-VLA/run_xyVLA.py `
--cmd "Fly through billboard_id=3 only" `
--no-local-avoidance `
--sim-steps 220 `
--infer-every 6 `
--xvla-steps 4 `
--log-every 10 `
--speed 5 `
--cmd-motion-amplify 2.5 `
--cmd-min-step 0.045 `
--infer-displacement-scale 1.6
```
---
## 3. Capability Test C — Traverse Specified ID Only Under Multi-Frame Interference
**Test objective**: Whether **`only billboard_id=N`** / **`ignore other billboard markers`** in the instruction suppresses neighboring distractor frames.
```powershell
& C:/Users/ydook/anaconda3/envs/xy-vla/python.exe d:/XY-VLA/run_xyVLA.py `
--cmd "Fly through ONLY billboard_id=7; numbered markers 6 and 8 are distractors — do not enter them." `
--sim-steps 260 `
--infer-every 6 `
--xvla-steps 4 `
--log-every 10 `
--speed 5 `
--cmd-motion-amplify 2.5 `
--cmd-min-step 0.045 `
--infer-displacement-scale 1.6
```
```powershell
& C:/Users/ydook/anaconda3/envs/xy-vla/python.exe d:/XY-VLA/run_xyVLA.py `
--cmd "Pass through portal billboard 16 only; all other billboard_id values must be skipped." `
--sim-steps 280 `
--infer-every 6 `
--xvla-steps 4 `
--log-every 10 `
--speed 5 `
--cmd-motion-amplify 2.5 `
--cmd-min-step 0.045 `
--infer-displacement-scale 1.6
```
---
## 4. Capability Test D — Visual Alignment + Traversal (Two-Stage Semantics)
**Test objective**: **Align/center** first, then **traverse** — evaluate whether temporal language is interpreted by the model as pose/position first, then crossing.
```powershell
& C:/Users/ydook/anaconda3/envs/xy-vla/python.exe d:/XY-VLA/run_xyVLA.py `
--cmd "Center billboard_id=5 opening in view, then fly through it along the long opening only." `
--sim-steps 300 `
--infer-every 5 `
--xvla-steps 4 `
--log-every 12 `
--speed 5 `
--cmd-motion-amplify 2.5 `
--cmd-min-step 0.045 `
--infer-displacement-scale 1.5
```
Near-range gate pose estimation relies on **`fpv_slot_align`**, **`gate_pose_estimator`** (`opencv` / `xvla`); see `config.json`.
---
## 5. Capability Test E — Tilt / Narrow Slot Geometry Hints
Mid-layer frames in the scene have **random tilt/yaw**; explicitly requiring **tilted / narrow slot** in the instruction tests the model's response to geometric language (still combined with real simulation poses).
```powershell
& C:/Users/ydook/anaconda3/envs/xy-vla/python.exe d:/XY-VLA/run_xyVLA.py `
--cmd "Fly through the tilted rectangular portal billboard_id=1; stay parallel to the long edge to avoid hitting the rails." `
--sim-steps 280 `
--infer-every 6 `
--xvla-steps 4 `
--log-every 10 `
--speed 5 `
--cmd-motion-amplify 2.5 `
--cmd-min-step 0.045 `
--infer-displacement-scale 1.5
```
---
## 6. Capability Test F — Motion Scale Trio (Smoke / Robust / Aggressive)
Same `--cmd`, only change dynamics-related parameters; for **video comparison**, best use **`--gui`** or increase `recording_scene_margin`.
**Smoke (small steps, easy to stabilize)**
```powershell
& C:/Users/ydook/anaconda3/envs/xy-vla/python.exe d:/XY-VLA/run_xyVLA.py `
--cmd "Fly through billboard_id=3 only" `
--sim-steps 240 `
--infer-every 8 `
--xvla-steps 2 `
--log-every 12 `
--speed 4 `
--cmd-motion-amplify 2.0 `
--cmd-min-step 0.04 `
--infer-displacement-scale 1.2
```
**Robust (default)** — same as **§1.1**.
**Aggressive (faster approach, prone to going off-screen or overshoot)**
```powershell
& C:/Users/ydook/anaconda3/envs/xy-vla/python.exe d:/XY-VLA/run_xyVLA.py `
--cmd "Fly through billboard_id=3 only" `
--sim-steps 200 `
--infer-every 4 `
--xvla-steps 4 `
--log-every 8 `
--speed 6 `
--cmd-motion-amplify 3.0 `
--cmd-min-step 0.05 `
--infer-displacement-scale 2.2
```
---
## 7. Capability Test G — Full Scene Language Catalog Ablation
**Catalog off**: Test pure vision + short instruction; compare the effect of **having vs. not having a numeric object list** on frame selection / path.
```powershell
& C:/Users/ydook/anaconda3/envs/xy-vla/python.exe d:/XY-VLA/run_xyVLA.py `
--cmd "Fly through billboard_id=3 only" `
--no-xvla-scene-catalog `
--sim-steps 220 `
--infer-every 6 `
--xvla-steps 4 `
--log-every 10 `
--speed 5 `
--cmd-motion-amplify 2.5 `
--cmd-min-step 0.045 `
--infer-displacement-scale 1.6
```
---
## 8. Full 20-Frame Sequential Traversal vs `--cmd`
Code convention: when **`--cmd`** is present, **non-empty `task_sequence` in JSON is ignored**; language instruction only. For **20-frame sequential endurance**, **do not pass `--cmd`**, e.g.:
```powershell
& C:/Users/ydook/anaconda3/envs/xy-vla/python.exe d:/XY-VLA/run_xyVLA.py `
--config e:/X-VLA/config.json `
--scheme widowx_ee6d `
--instruction "Sequential portal flight demo"
```
---
## 9. Top-Down Coarse Planning Phase1+2+3 (`cmd_coarse_plan_once`)
### 9.1 Copy-Paste Command — Single Frame Traversal (default Phase3 action)
Same `--cmd` as **§1.1**; with **`cmd_coarse_plan_once: true`** runs Phase3 instead of main simulation:
```powershell
& C:/Users/ydook/anaconda3/envs/xy-vla/python.exe d:/XY-VLA/run_xyVLA.py `
--cmd "Fly through only the rectangular portal marked billboard_id=3" `
--sim-steps 200 `
--infer-every 6 `
--xvla-steps 4 `
--log-every 10 `
--speed 5 `
--cmd-motion-amplify 2.5 `
--cmd-min-step 0.045 `
--infer-displacement-scale 1.6
```
Expected log: `[Phase 3][Refine] object '…' base_action=pass_through`.
### 9.2 Copy-Paste Commands — Explicit Five Actions (single-frame debug)
**Fly by** (does not emphasize center through):
```powershell
& C:/Users/ydook/anaconda3/envs/xy-vla/python.exe d:/XY-VLA/run_xyVLA.py `
--cmd "Fly by portal billboard number 3 without entering the opening center" `
--sim-steps 200 `
--infer-every 6 `
--xvla-steps 4 `
--log-every 10 `
--speed 5 `
--cmd-motion-amplify 2.5 `
--cmd-min-step 0.045 `
--infer-displacement-scale 1.6
```
**Orbit 2 laps**:
```powershell
& C:/Users/ydook/anaconda3/envs/xy-vla/python.exe d:/XY-VLA/run_xyVLA.py `
--cmd "Orbit billboard_id=3 twice for inspection, then pass through billboard_id=3" `
--sim-steps 200 `
--infer-every 6 `
--xvla-steps 4 `
--log-every 10 `
--speed 5 `
--cmd-motion-amplify 2.5 `
--cmd-min-step 0.045 `
--infer-displacement-scale 1.6
```
**Hover**:
```powershell
& C:/Users/ydook/anaconda3/envs/xy-vla/python.exe d:/XY-VLA/run_xyVLA.py `
--cmd "Hover over portal billboard number 5" `
--sim-steps 200 `
--infer-every 6 `
--xvla-steps 4 `
--log-every 10 `
--speed 5 `
--cmd-motion-amplify 2.5 `
--cmd-min-step 0.045 `
--infer-displacement-scale 1.6
```
**Collision** (geometry debug, not safe flight):
```powershell
& C:/Users/ydook/anaconda3/envs/xy-vla/python.exe d:/XY-VLA/run_xyVLA.py `
--cmd "Collide with rectangular portal billboard_id=7; ram the frame directly" `
--sim-steps 200 `
--infer-every 6 `
--xvla-steps 4 `
--log-every 10 `
--speed 5 `
--cmd-motion-amplify 2.5 `
--cmd-min-step 0.045 `
--infer-displacement-scale 1.6
```
### 9.3 Copy-Paste Command — Multi-Frame Chain + Phase3 Refinement
Same as **`cmd_multi_1.md` §5.1**; Phase3 turns each **feedback sphere contacted along the global trajectory** red sequentially and refines:
```powershell
& C:/Users/ydook/anaconda3/envs/xy-vla/python.exe d:/XY-VLA/run_xyVLA.py `
--cmd "Pass through portal billboard number 2 then pass through portal billboard number 3, then pass through portal billboard number 5" `
--sim-steps 200 `
--infer-every 6 `
--xvla-steps 4 `
--log-every 10 `
--speed 5 `
--cmd-motion-amplify 2.5 `
--cmd-min-step 0.045 `
--infer-displacement-scale 1.6
```
### 9.4 Disable Coarse Planning to Restore Closed-Loop Portal Traversal
Set **`"cmd_coarse_plan_once": false`** in **`config.json`** → `schemes.widowx_ee6d`, then run **§2–§7** commands to enter the X‑VLA main simulation loop.
## 10. Appendix: Full Command Template with `config` / `scheme` / `QS`
```powershell
& C:/Users/ydook/anaconda3/envs/xy-vla/python.exe d:/XY-VLA/run_xyVLA.py `
--config e:/X-VLA/config.json `
--scheme widowx_ee6d `
--cmd "Fly through only billboard_id=3; align with the long opening; ignore other portals." `
--sim-steps 220 `
--infer-every 6 `
--xvla-steps 4 `
--log-every 10 `
--speed 5 `
--cmd-motion-amplify 2.5 `
--cmd-min-step 0.045 `
--infer-displacement-scale 1.6 `
--qs-path e:/X-VLA/QS.json
```
If v‑vla is also needed: add `` ` `` at the end of the **`--qs-path`** line above, and **start a new line** for `--vvla-url` (no continuation on the last line); full example:
```powershell
& C:/Users/ydook/anaconda3/envs/xy-vla/python.exe d:/XY-VLA/run_xyVLA.py `
--config e:/X-VLA/config.json `
--scheme widowx_ee6d `
--cmd "Fly through only billboard_id=3; align with the long opening; ignore other portals." `
--sim-steps 220 `
--infer-every 6 `
--xvla-steps 4 `
--log-every 10 `
--speed 5 `
--cmd-motion-amplify 2.5 `
--cmd-min-step 0.045 `
--infer-displacement-scale 1.6 `
--qs-path e:/X-VLA/QS.json `
--vvla-url http://127.0.0.1:9000/vvla
```
(`--vvla-url` should match your actual service address.) Seeing `[===> TransCMD]: ...` when multiple `Q` entries match is normal.
|