File size: 11,891 Bytes
7d45f7c
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
### 1.1 Core Example (`billboard_id=3`, default config is upper yellow frame — use ID as reference)

```powershell
& C:/Users/ydook/anaconda3/envs/xy-vla/python.exe d:/XY-VLA/run_xyVLA.py `
  --cmd "Fly through only the rectangular portal marked billboard_id=18" `
  --sim-steps 200 `
  --infer-every 6 `
  --xvla-steps 4 `
  --log-every 10 `
  --speed 5 `
  --cmd-motion-amplify 2.5 `
  --cmd-min-step 0.045 `
  --infer-displacement-scale 1.6
```

### 1.2 Other IDs (change the number in `--cmd` only)

Lower ring example (default config **`billboard_id=12`** ≈ orange frame; still use ID as reference):

```powershell
& C:/Users/ydook/anaconda3/envs/xy-vla/python.exe d:/XY-VLA/run_xyVLA.py `
  --cmd "Pass through portal billboard number 12 only; stay parallel to the long edge of billboard_id=12." `
  --sim-steps 220 `
  --infer-every 6 `
  --xvla-steps 4 `
  --log-every 10 `
  --speed 5 `
  --cmd-motion-amplify 2.5 `
  --cmd-min-step 0.045 `
  --infer-displacement-scale 1.6
```

Short phrase (suitable for batch scripts changing the last digit):

```powershell
& C:/Users/ydook/anaconda3/envs/xy-vla/python.exe d:/XY-VLA/run_xyVLA.py `
  --cmd "Navigate through rectangular frame billboard_id=16; long-opening alignment; no other portals." `
  --sim-steps 220 `
  --infer-every 6 `
  --xvla-steps 4 `
  --log-every 10 `
  --speed 5 `
  --cmd-motion-amplify 2.5 `
  --cmd-min-step 0.045 `
  --infer-displacement-scale 1.6
```

### 1.3 Alternative: Color Commands (legacy compatibility, lower precision)

Easy to confuse when 2 portals share the same color on upper/lower layers; use only when **ambiguity is acceptable** or for **temporary comparison**.

```powershell
& C:/Users/ydook/anaconda3/envs/xy-vla/python.exe d:/XY-VLA/run_xyVLA.py `
  --cmd "Fly through the yellow rectangular frame safely, align with the long opening" `
  --sim-steps 220 `
  --infer-every 6 `
  --xvla-steps 4 `
  --log-every 10 `
  --speed 5 `
  --cmd-motion-amplify 2.5 `
  --cmd-min-step 0.045 `
  --infer-displacement-scale 1.6
```

---

## 2. Capability Test B — Obstacle Avoidance & Safety Semantics (still targeting portal traversal)

```powershell
& C:/Users/ydook/anaconda3/envs/xy-vla/python.exe d:/XY-VLA/run_xyVLA.py `
  --cmd "Fly through only billboard_id=3 safely; keep clearance from all other frames and obstacles" `
  --sim-steps 280 `
  --infer-every 5 `
  --xvla-steps 4 `
  --log-every 12 `
  --speed 5 `
  --cmd-motion-amplify 2.5 `
  --cmd-min-step 0.045 `
  --infer-displacement-scale 1.5
```

**Control**: Disable near-field repulsion (comparison experiment only):

```powershell
& C:/Users/ydook/anaconda3/envs/xy-vla/python.exe d:/XY-VLA/run_xyVLA.py `
  --cmd "Fly through billboard_id=3 only" `
  --no-local-avoidance `
  --sim-steps 220 `
  --infer-every 6 `
  --xvla-steps 4 `
  --log-every 10 `
  --speed 5 `
  --cmd-motion-amplify 2.5 `
  --cmd-min-step 0.045 `
  --infer-displacement-scale 1.6
```

---

## 3. Capability Test C — Traverse Specified ID Only Under Multi-Frame Interference

**Test objective**: Whether **`only billboard_id=N`** / **`ignore other billboard markers`** in the instruction suppresses neighboring distractor frames.

```powershell
& C:/Users/ydook/anaconda3/envs/xy-vla/python.exe d:/XY-VLA/run_xyVLA.py `
  --cmd "Fly through ONLY billboard_id=7; numbered markers 6 and 8 are distractors — do not enter them." `
  --sim-steps 260 `
  --infer-every 6 `
  --xvla-steps 4 `
  --log-every 10 `
  --speed 5 `
  --cmd-motion-amplify 2.5 `
  --cmd-min-step 0.045 `
  --infer-displacement-scale 1.6
```

```powershell
& C:/Users/ydook/anaconda3/envs/xy-vla/python.exe d:/XY-VLA/run_xyVLA.py `
  --cmd "Pass through portal billboard 16 only; all other billboard_id values must be skipped." `
  --sim-steps 280 `
  --infer-every 6 `
  --xvla-steps 4 `
  --log-every 10 `
  --speed 5 `
  --cmd-motion-amplify 2.5 `
  --cmd-min-step 0.045 `
  --infer-displacement-scale 1.6
```
---

## 4. Capability Test D — Visual Alignment + Traversal (Two-Stage Semantics)

**Test objective**: **Align/center** first, then **traverse** — evaluate whether temporal language is interpreted by the model as pose/position first, then crossing.

```powershell
& C:/Users/ydook/anaconda3/envs/xy-vla/python.exe d:/XY-VLA/run_xyVLA.py `
  --cmd "Center billboard_id=5 opening in view, then fly through it along the long opening only." `
  --sim-steps 300 `
  --infer-every 5 `
  --xvla-steps 4 `
  --log-every 12 `
  --speed 5 `
  --cmd-motion-amplify 2.5 `
  --cmd-min-step 0.045 `
  --infer-displacement-scale 1.5
```

Near-range gate pose estimation relies on **`fpv_slot_align`**, **`gate_pose_estimator`** (`opencv` / `xvla`); see `config.json`.

---

## 5. Capability Test E — Tilt / Narrow Slot Geometry Hints

Mid-layer frames in the scene have **random tilt/yaw**; explicitly requiring **tilted / narrow slot** in the instruction tests the model's response to geometric language (still combined with real simulation poses).

```powershell
& C:/Users/ydook/anaconda3/envs/xy-vla/python.exe d:/XY-VLA/run_xyVLA.py `
  --cmd "Fly through the tilted rectangular portal billboard_id=1; stay parallel to the long edge to avoid hitting the rails." `
  --sim-steps 280 `
  --infer-every 6 `
  --xvla-steps 4 `
  --log-every 10 `
  --speed 5 `
  --cmd-motion-amplify 2.5 `
  --cmd-min-step 0.045 `
  --infer-displacement-scale 1.5
```

---

## 6. Capability Test F — Motion Scale Trio (Smoke / Robust / Aggressive)

Same `--cmd`, only change dynamics-related parameters; for **video comparison**, best use **`--gui`** or increase `recording_scene_margin`.

**Smoke (small steps, easy to stabilize)**

```powershell
& C:/Users/ydook/anaconda3/envs/xy-vla/python.exe d:/XY-VLA/run_xyVLA.py `
  --cmd "Fly through billboard_id=3 only" `
  --sim-steps 240 `
  --infer-every 8 `
  --xvla-steps 2 `
  --log-every 12 `
  --speed 4 `
  --cmd-motion-amplify 2.0 `
  --cmd-min-step 0.04 `
  --infer-displacement-scale 1.2
```

**Robust (default)** — same as **§1.1**.

**Aggressive (faster approach, prone to going off-screen or overshoot)**

```powershell
& C:/Users/ydook/anaconda3/envs/xy-vla/python.exe d:/XY-VLA/run_xyVLA.py `
  --cmd "Fly through billboard_id=3 only" `
  --sim-steps 200 `
  --infer-every 4 `
  --xvla-steps 4 `
  --log-every 8 `
  --speed 6 `
  --cmd-motion-amplify 3.0 `
  --cmd-min-step 0.05 `
  --infer-displacement-scale 2.2
```

---

## 7. Capability Test G — Full Scene Language Catalog Ablation

**Catalog off**: Test pure vision + short instruction; compare the effect of **having vs. not having a numeric object list** on frame selection / path.

```powershell
& C:/Users/ydook/anaconda3/envs/xy-vla/python.exe d:/XY-VLA/run_xyVLA.py `
  --cmd "Fly through billboard_id=3 only" `
  --no-xvla-scene-catalog `
  --sim-steps 220 `
  --infer-every 6 `
  --xvla-steps 4 `
  --log-every 10 `
  --speed 5 `
  --cmd-motion-amplify 2.5 `
  --cmd-min-step 0.045 `
  --infer-displacement-scale 1.6
```

---

## 8. Full 20-Frame Sequential Traversal vs `--cmd`

Code convention: when **`--cmd`** is present, **non-empty `task_sequence` in JSON is ignored**; language instruction only. For **20-frame sequential endurance**, **do not pass `--cmd`**, e.g.:

```powershell
& C:/Users/ydook/anaconda3/envs/xy-vla/python.exe d:/XY-VLA/run_xyVLA.py `
  --config e:/X-VLA/config.json `
  --scheme widowx_ee6d `
  --instruction "Sequential portal flight demo"
```

---

## 9. Top-Down Coarse Planning Phase1+2+3 (`cmd_coarse_plan_once`)

### 9.1 Copy-Paste Command — Single Frame Traversal (default Phase3 action)

Same `--cmd` as **§1.1**; with **`cmd_coarse_plan_once: true`** runs Phase3 instead of main simulation:

```powershell
& C:/Users/ydook/anaconda3/envs/xy-vla/python.exe d:/XY-VLA/run_xyVLA.py `
  --cmd "Fly through only the rectangular portal marked billboard_id=3" `
  --sim-steps 200 `
  --infer-every 6 `
  --xvla-steps 4 `
  --log-every 10 `
  --speed 5 `
  --cmd-motion-amplify 2.5 `
  --cmd-min-step 0.045 `
  --infer-displacement-scale 1.6
```

Expected log: `[Phase 3][Refine] object '…' base_action=pass_through`.

### 9.2 Copy-Paste Commands — Explicit Five Actions (single-frame debug)

**Fly by** (does not emphasize center through):

```powershell
& C:/Users/ydook/anaconda3/envs/xy-vla/python.exe d:/XY-VLA/run_xyVLA.py `
  --cmd "Fly by portal billboard number 3 without entering the opening center" `
  --sim-steps 200 `
  --infer-every 6 `
  --xvla-steps 4 `
  --log-every 10 `
  --speed 5 `
  --cmd-motion-amplify 2.5 `
  --cmd-min-step 0.045 `
  --infer-displacement-scale 1.6
```

**Orbit 2 laps**:

```powershell
& C:/Users/ydook/anaconda3/envs/xy-vla/python.exe d:/XY-VLA/run_xyVLA.py `
  --cmd "Orbit billboard_id=3 twice for inspection, then pass through billboard_id=3" `
  --sim-steps 200 `
  --infer-every 6 `
  --xvla-steps 4 `
  --log-every 10 `
  --speed 5 `
  --cmd-motion-amplify 2.5 `
  --cmd-min-step 0.045 `
  --infer-displacement-scale 1.6
```

**Hover**:

```powershell
& C:/Users/ydook/anaconda3/envs/xy-vla/python.exe d:/XY-VLA/run_xyVLA.py `
  --cmd "Hover over portal billboard number 5" `
  --sim-steps 200 `
  --infer-every 6 `
  --xvla-steps 4 `
  --log-every 10 `
  --speed 5 `
  --cmd-motion-amplify 2.5 `
  --cmd-min-step 0.045 `
  --infer-displacement-scale 1.6
```

**Collision** (geometry debug, not safe flight):

```powershell
& C:/Users/ydook/anaconda3/envs/xy-vla/python.exe d:/XY-VLA/run_xyVLA.py `
  --cmd "Collide with rectangular portal billboard_id=7; ram the frame directly" `
  --sim-steps 200 `
  --infer-every 6 `
  --xvla-steps 4 `
  --log-every 10 `
  --speed 5 `
  --cmd-motion-amplify 2.5 `
  --cmd-min-step 0.045 `
  --infer-displacement-scale 1.6
```

### 9.3 Copy-Paste Command — Multi-Frame Chain + Phase3 Refinement

Same as **`cmd_multi_1.md` §5.1**; Phase3 turns each **feedback sphere contacted along the global trajectory** red sequentially and refines:

```powershell
& C:/Users/ydook/anaconda3/envs/xy-vla/python.exe d:/XY-VLA/run_xyVLA.py `
  --cmd "Pass through portal billboard number 2 then pass through portal billboard number 3, then pass through portal billboard number 5" `
  --sim-steps 200 `
  --infer-every 6 `
  --xvla-steps 4 `
  --log-every 10 `
  --speed 5 `
  --cmd-motion-amplify 2.5 `
  --cmd-min-step 0.045 `
  --infer-displacement-scale 1.6
```

### 9.4 Disable Coarse Planning to Restore Closed-Loop Portal Traversal

Set **`"cmd_coarse_plan_once": false`** in **`config.json`** → `schemes.widowx_ee6d`, then run **§2–§7** commands to enter the X‑VLA main simulation loop.


## 10. Appendix: Full Command Template with `config` / `scheme` / `QS`

```powershell
& C:/Users/ydook/anaconda3/envs/xy-vla/python.exe d:/XY-VLA/run_xyVLA.py `
  --config e:/X-VLA/config.json `
  --scheme widowx_ee6d `
  --cmd "Fly through only billboard_id=3; align with the long opening; ignore other portals." `
  --sim-steps 220 `
  --infer-every 6 `
  --xvla-steps 4 `
  --log-every 10 `
  --speed 5 `
  --cmd-motion-amplify 2.5 `
  --cmd-min-step 0.045 `
  --infer-displacement-scale 1.6 `
  --qs-path e:/X-VLA/QS.json
```

If v‑vla is also needed: add `` ` `` at the end of the **`--qs-path`** line above, and **start a new line** for `--vvla-url` (no continuation on the last line); full example:

```powershell
& C:/Users/ydook/anaconda3/envs/xy-vla/python.exe d:/XY-VLA/run_xyVLA.py `
  --config e:/X-VLA/config.json `
  --scheme widowx_ee6d `
  --cmd "Fly through only billboard_id=3; align with the long opening; ignore other portals." `
  --sim-steps 220 `
  --infer-every 6 `
  --xvla-steps 4 `
  --log-every 10 `
  --speed 5 `
  --cmd-motion-amplify 2.5 `
  --cmd-min-step 0.045 `
  --infer-displacement-scale 1.6 `
  --qs-path e:/X-VLA/QS.json `
  --vvla-url http://127.0.0.1:9000/vvla
```

(`--vvla-url` should match your actual service address.) Seeing `[===> TransCMD]: ...` when multiple `Q` entries match is normal.