Spaces:
Running
Running
| <html lang="en"> | |
| <head> | |
| <meta charset="UTF-8"> | |
| <meta name="viewport" content="width=device-width, initial-scale=1.0"> | |
| <title>Minecraft Agent — Qwen3.5-9B</title> | |
| <style> | |
| :root { | |
| --bg: #0d1117; --card: #161b22; --border: #30363d; | |
| --text: #e6edf3; --muted: #8b949e; --green: #3fb950; --blue: #58a6ff; --gold: #d29922; | |
| } | |
| * { margin: 0; padding: 0; box-sizing: border-box; } | |
| body { background: var(--bg); color: var(--text); font-family: -apple-system, "Segoe UI", Roboto, Helvetica, Arial, sans-serif; line-height: 1.6; } | |
| .container { max-width: 1080px; margin: 0 auto; padding: 0 24px 80px; } | |
| header { text-align: center; padding: 64px 0 40px; } | |
| header .emoji { font-size: 56px; } | |
| h1 { font-size: 36px; margin: 12px 0 8px; } | |
| header p { color: var(--muted); font-size: 17px; max-width: 640px; margin: 0 auto; } | |
| .badges { margin-top: 18px; } | |
| .badge { display: inline-block; padding: 4px 12px; border-radius: 999px; font-size: 13px; margin: 3px; border: 1px solid var(--border); color: var(--muted); } | |
| .badge.green { color: var(--green); border-color: var(--green); } | |
| .badge.blue { color: var(--blue); border-color: var(--blue); } | |
| h2 { font-size: 24px; margin: 56px 0 20px; padding-bottom: 10px; border-bottom: 1px solid var(--border); } | |
| h2 .num { color: var(--blue); margin-right: 10px; font-family: ui-monospace, monospace; } | |
| table { width: 100%; border-collapse: collapse; margin: 16px 0; font-size: 15px; } | |
| th, td { padding: 10px 14px; text-align: left; border: 1px solid var(--border); } | |
| th { background: var(--card); color: var(--muted); font-weight: 600; } | |
| tr:nth-child(even) td { background: rgba(255,255,255,0.02); } | |
| .hl { color: var(--green); font-weight: 700; } | |
| .cards { display: grid; grid-template-columns: repeat(auto-fit, minmax(220px, 1fr)); gap: 14px; margin: 18px 0; } | |
| .card { background: var(--card); border: 1px solid var(--border); border-radius: 10px; padding: 18px; } | |
| .card .k { color: var(--muted); font-size: 13px; text-transform: uppercase; letter-spacing: .05em; } | |
| .card .v { font-size: 21px; font-weight: 700; margin-top: 4px; } | |
| .card .s { color: var(--muted); font-size: 13px; margin-top: 4px; } | |
| .traj { display: grid; grid-template-columns: repeat(auto-fit, minmax(300px, 1fr)); gap: 16px; margin: 20px 0; } | |
| .step { background: var(--card); border: 1px solid var(--border); border-radius: 10px; overflow: hidden; } | |
| .step img { width: 100%; display: block; aspect-ratio: 16/9; object-fit: cover; } | |
| .step .meta { padding: 12px 14px; } | |
| .step .t { color: var(--muted); font-size: 12px; } | |
| .step .a { font-family: ui-monospace, "SF Mono", monospace; font-size: 13px; color: var(--gold); margin-top: 4px; word-break: break-all; } | |
| pre { background: var(--card); border: 1px solid var(--border); border-radius: 10px; padding: 16px 18px; overflow-x: auto; font-size: 13.5px; line-height: 1.7; } | |
| code { font-family: ui-monospace, "SF Mono", monospace; } | |
| p code, li code { background: rgba(255,255,255,0.07); padding: 2px 6px; border-radius: 5px; font-size: 13px; color: var(--gold); } | |
| .note { background: rgba(88,166,255,0.08); border: 1px solid rgba(88,166,255,0.3); border-radius: 10px; padding: 14px 18px; margin: 18px 0; font-size: 14.5px; } | |
| ul { margin: 10px 0 10px 22px; } | |
| li { margin: 6px 0; } | |
| a { color: var(--blue); text-decoration: none; } | |
| a:hover { text-decoration: underline; } | |
| .instr { background: rgba(63,185,80,0.08); border: 1px solid rgba(63,185,80,0.35); border-radius: 10px; padding: 14px 18px; margin: 18px 0; font-size: 16px; } | |
| footer { margin-top: 64px; text-align: center; color: var(--muted); font-size: 13px; } | |
| </style> | |
| </head> | |
| <body> | |
| <div class="container"> | |
| <header> | |
| <div class="emoji">⛏️</div> | |
| <h1>Minecraft Agent — Qwen3.5-9B</h1> | |
| <p>A vision-language agent that plays Minecraft from raw first-person pixels (640×360 @ 20 FPS) and emits structured text actions. Stage-3 SFT on 200k+ human gameplay trajectories.</p> | |
| <div class="badges"> | |
| <span class="badge green">24.9% success (easy-ng, 29-frame context)</span> | |
| <span class="badge blue">Base: Qwen3.5-9B</span> | |
| <span class="badge">6M frames · 20 FPS</span> | |
| <span class="badge">Text-action space</span> | |
| <a href="cot.html" class="badge green" style="text-decoration:none">CoT Demo Dataset →</a> | |
| </div> | |
| </header> | |
| <h2><span class="num">01</span>Model</h2> | |
| <div class="cards"> | |
| <div class="card"><div class="k">Architecture</div><div class="v">Qwen3.5-9B</div><div class="s">Hybrid linear/full attention + ViT tower</div></div> | |
| <div class="card"><div class="k">Weights</div><div class="v">17.5 GB</div><div class="s">bf16 · single safetensors</div></div> | |
| <div class="card"><div class="k">Checkpoint</div><div class="v">step 3000</div><div class="s">run nf-noop2 (keep_p=0.2 + focal 0.75)</div></div> | |
| <div class="card"><div class="k">License</div><div class="v">Apache-2.0</div><div class="s">Model repo on the Hub</div></div> | |
| </div> | |
| <p>➡ <a href="https://huggingface.co/EmberJin/Minecraft-Agent-Qwen3.5-9B-Stage3" target="_blank"><strong>EmberJin/Minecraft-Agent-Qwen3.5-9B-Stage3</strong></a></p> | |
| <h2><span class="num">02</span>Evaluation</h2> | |
| <p style="color:var(--muted)">Benchmark <b>easy-ng</b>: 202 non-GUI tasks (153 Embodied <code>mine_block</code> + 49 Combat <code>kill_entity</code>) × 3 rollouts, difficulty=easy, 200-step episodes.</p> | |
| <table> | |
| <tr><th>Inference history window</th><th>Success</th><th>Embodied</th><th>Combat</th></tr> | |
| <tr><td>3 frames (naive default)</td><td>15.0%</td><td>17.9%</td><td>6.2%</td></tr> | |
| <tr><td><b>29 frames (matches training context)</b></td><td class="hl">24.9%</td><td class="hl">24.8%</td><td class="hl">25.2%</td></tr> | |
| </table> | |
| <div class="note">💡 <b>Key finding</b>: aligning the inference context length with the 30-step training segments yields <b>+9.9pp</b> over the naive 3-frame window — memory truncation was a hidden bottleneck. Combat more than quadruples (6.2% → 25.2%): tracking an enemy that leaves the field of view requires long-term memory.</div> | |
| <h2><span class="num">03</span>Training Recipe</h2> | |
| <table> | |
| <tr><th>Component</th><th>Value</th></tr> | |
| <tr><td>Base model</td><td>Qwen3.5-9B (stage-2)</td></tr> | |
| <tr><td>Data</td><td><code>minecraft-text-action-dataset-noop-filtered</code> — 92.85% of raw dataset after removing 7.15% pure no-op collapse trajectories</td></tr> | |
| <tr><td>No-op frame drop (<code>KEEP_NO_OP_P</code>)</td><td><b>0.2</b> (drop 80% of no-op frames at data level)</td></tr> | |
| <tr><td>Focal loss</td><td>decay = 0.75 (down-weights repeated consecutive actions)</td></tr> | |
| <tr><td>Sample format</td><td>30-step trajectory segments: <code>user(prompt+instruction+img) → assistant(action) → user(img) → …</code></td></tr> | |
| </table> | |
| <h2><span class="num">04</span>Action Space</h2> | |
| <p>The model outputs a single line per step:</p> | |
| <pre>Action: move(dx, dy) and press(keys) and click(button)</pre> | |
| <ul> | |
| <li><code>move(dx, dy)</code> — mouse movement (horizontal, vertical)</li> | |
| <li><code>press(keys)</code> — <code>w/a/s/d</code> movement, <code>space</code> jump, <code>e</code> inventory, <code>1</code>–<code>9</code> hotbar, <code>left.control</code> sprint, <code>left.shift</code> careful move</li> | |
| <li><code>click('left')</code> attack / <code>click('right')</code> place or use</li> | |
| <li><code>no_op</code> — wait</li> | |
| </ul> | |
| <h2><span class="num">05</span>A Real Trajectory from Training Data</h2> | |
| <div class="instr">🎯 Task instruction: <b>Mine the cobweb to clear the path.</b></div> | |
| <p style="color:var(--muted)">Six frames (evenly sampled from a 30-step segment) with the ground-truth actions the model learns to imitate:</p> | |
| <div class="traj"> | |
| <div class="step"><img src="assets/frame_0.jpg" alt="frame 0"><div class="meta"><div class="t">step 0</div><div class="a">Action: move(-53, 0) and press(a)</div></div></div> | |
| <div class="step"><img src="assets/frame_1.jpg" alt="frame 6"><div class="meta"><div class="t">step 6</div><div class="a">Action: move(0, 0) and press(w, a)</div></div></div> | |
| <div class="step"><img src="assets/frame_2.jpg" alt="frame 12"><div class="meta"><div class="t">step 12</div><div class="a">Action: move(13, 13) and press(w)</div></div></div> | |
| <div class="step"><img src="assets/frame_3.jpg" alt="frame 18"><div class="meta"><div class="t">step 18</div><div class="a">Action: move(80, 0) and press(d)</div></div></div> | |
| <div class="step"><img src="assets/frame_4.jpg" alt="frame 24"><div class="meta"><div class="t">step 24</div><div class="a">Action: move(7, -7) and press(w, d, left.control)</div></div></div> | |
| <div class="step"><img src="assets/frame_5.jpg" alt="frame 29"><div class="meta"><div class="t">step 29</div><div class="a">Action: move(0, 0) and press(w, left.control)</div></div></div> | |
| </div> | |
| <h2><span class="num">06</span>Inference</h2> | |
| <pre># vLLM | |
| vllm serve EmberJin/Minecraft-Agent-Qwen3.5-9B-Stage3 \ | |
| --limit-mm-per-prompt '{"image": 30}' \ | |
| --enable-prefix-caching \ | |
| --max-model-len 32768</pre> | |
| <div class="note">⚠️ GUI tasks (crafting / smelting / chest interaction) are out of scope: all text-action models score 0% on them — the action space has no interface-coordinate grounding.</div> | |
| <footer> | |
| Minecraft Agent · Qwen3.5-9B Stage3 · easy-ng benchmark · single seed<br> | |
| <a href="cot.html" style="color: var(--blue); text-decoration: none;">CoT Demo Dataset</a> · 531 trajectories | |
| <a href="https://huggingface.co/EmberJin/Minecraft-Agent-Qwen3.5-9B-Stage3">Model repository</a> | |
| </footer> | |
| </div> | |
| </body> | |
| </html> | |