index.html

<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>Minecraft Agent — Qwen3.5-9B</title>
<style>
  :root {
    --bg: #0d1117; --card: #161b22; --border: #30363d;
    --text: #e6edf3; --muted: #8b949e; --green: #3fb950; --blue: #58a6ff; --gold: #d29922;
  }
  * { margin: 0; padding: 0; box-sizing: border-box; }
  body { background: var(--bg); color: var(--text); font-family: -apple-system, "Segoe UI", Roboto, Helvetica, Arial, sans-serif; line-height: 1.6; }
  .container { max-width: 1080px; margin: 0 auto; padding: 0 24px 80px; }
  header { text-align: center; padding: 64px 0 40px; }
  header .emoji { font-size: 56px; }
  h1 { font-size: 36px; margin: 12px 0 8px; }
  header p { color: var(--muted); font-size: 17px; max-width: 640px; margin: 0 auto; }
  .badges { margin-top: 18px; }
  .badge { display: inline-block; padding: 4px 12px; border-radius: 999px; font-size: 13px; margin: 3px; border: 1px solid var(--border); color: var(--muted); }
  .badge.green { color: var(--green); border-color: var(--green); }
  .badge.blue { color: var(--blue); border-color: var(--blue); }
  h2 { font-size: 24px; margin: 56px 0 20px; padding-bottom: 10px; border-bottom: 1px solid var(--border); }
  h2 .num { color: var(--blue); margin-right: 10px; font-family: ui-monospace, monospace; }
  table { width: 100%; border-collapse: collapse; margin: 16px 0; font-size: 15px; }
  th, td { padding: 10px 14px; text-align: left; border: 1px solid var(--border); }
  th { background: var(--card); color: var(--muted); font-weight: 600; }
  tr:nth-child(even) td { background: rgba(255,255,255,0.02); }
  .hl { color: var(--green); font-weight: 700; }
  .cards { display: grid; grid-template-columns: repeat(auto-fit, minmax(220px, 1fr)); gap: 14px; margin: 18px 0; }
  .card { background: var(--card); border: 1px solid var(--border); border-radius: 10px; padding: 18px; }
  .card .k { color: var(--muted); font-size: 13px; text-transform: uppercase; letter-spacing: .05em; }
  .card .v { font-size: 21px; font-weight: 700; margin-top: 4px; }
  .card .s { color: var(--muted); font-size: 13px; margin-top: 4px; }
  .traj { display: grid; grid-template-columns: repeat(auto-fit, minmax(300px, 1fr)); gap: 16px; margin: 20px 0; }
  .step { background: var(--card); border: 1px solid var(--border); border-radius: 10px; overflow: hidden; }
  .step img { width: 100%; display: block; aspect-ratio: 16/9; object-fit: cover; }
  .step .meta { padding: 12px 14px; }
  .step .t { color: var(--muted); font-size: 12px; }
  .step .a { font-family: ui-monospace, "SF Mono", monospace; font-size: 13px; color: var(--gold); margin-top: 4px; word-break: break-all; }
  pre { background: var(--card); border: 1px solid var(--border); border-radius: 10px; padding: 16px 18px; overflow-x: auto; font-size: 13.5px; line-height: 1.7; }
  code { font-family: ui-monospace, "SF Mono", monospace; }
  p code, li code { background: rgba(255,255,255,0.07); padding: 2px 6px; border-radius: 5px; font-size: 13px; color: var(--gold); }
  .note { background: rgba(88,166,255,0.08); border: 1px solid rgba(88,166,255,0.3); border-radius: 10px; padding: 14px 18px; margin: 18px 0; font-size: 14.5px; }
  ul { margin: 10px 0 10px 22px; }
  li { margin: 6px 0; }
  a { color: var(--blue); text-decoration: none; }
  a:hover { text-decoration: underline; }
  .instr { background: rgba(63,185,80,0.08); border: 1px solid rgba(63,185,80,0.35); border-radius: 10px; padding: 14px 18px; margin: 18px 0; font-size: 16px; }
  footer { margin-top: 64px; text-align: center; color: var(--muted); font-size: 13px; }
</style>
</head>
<body>
<div class="container">

<header>
  <div class="emoji">⛏️</div>
  <h1>Minecraft Agent — Qwen3.5-9B</h1>
  <p>A vision-language agent that plays Minecraft from raw first-person pixels (640×360 @ 20 FPS) and emits structured text actions. Stage-3 SFT on 200k+ human gameplay trajectories.</p>
  <div class="badges">
    <span class="badge green">24.9% success (easy-ng, 29-frame context)</span>
    <span class="badge blue">Base: Qwen3.5-9B</span>
    <span class="badge">6M frames · 20 FPS</span>
    <span class="badge">Text-action space</span>
    <a href="cot.html" class="badge green" style="text-decoration:none">CoT Demo Dataset →</a>
  </div>
</header>

<h2><span class="num">01</span>Model</h2>
<div class="cards">
  <div class="card"><div class="k">Architecture</div><div class="v">Qwen3.5-9B</div><div class="s">Hybrid linear/full attention + ViT tower</div></div>
  <div class="card"><div class="k">Weights</div><div class="v">17.5 GB</div><div class="s">bf16 · single safetensors</div></div>
  <div class="card"><div class="k">Checkpoint</div><div class="v">step 3000</div><div class="s">run nf-noop2 (keep_p=0.2 + focal 0.75)</div></div>
  <div class="card"><div class="k">License</div><div class="v">Apache-2.0</div><div class="s">Model repo on the Hub</div></div>
</div>
<p>➡ &nbsp;<a href="https://huggingface.co/EmberJin/Minecraft-Agent-Qwen3.5-9B-Stage3" target="_blank"><strong>EmberJin/Minecraft-Agent-Qwen3.5-9B-Stage3</strong></a></p>

<h2><span class="num">02</span>Evaluation</h2>
<p style="color:var(--muted)">Benchmark <b>easy-ng</b>: 202 non-GUI tasks (153 Embodied <code>mine_block</code> + 49 Combat <code>kill_entity</code>) × 3 rollouts, difficulty=easy, 200-step episodes.</p>
<table>
  <tr><th>Inference history window</th><th>Success</th><th>Embodied</th><th>Combat</th></tr>
  <tr><td>3 frames (naive default)</td><td>15.0%</td><td>17.9%</td><td>6.2%</td></tr>
  <tr><td><b>29 frames (matches training context)</b></td><td class="hl">24.9%</td><td class="hl">24.8%</td><td class="hl">25.2%</td></tr>
</table>
<div class="note">💡 <b>Key finding</b>: aligning the inference context length with the 30-step training segments yields <b>+9.9pp</b> over the naive 3-frame window — memory truncation was a hidden bottleneck. Combat more than quadruples (6.2% → 25.2%): tracking an enemy that leaves the field of view requires long-term memory.</div>

<h2><span class="num">03</span>Training Recipe</h2>
<table>
  <tr><th>Component</th><th>Value</th></tr>
  <tr><td>Base model</td><td>Qwen3.5-9B (stage-2)</td></tr>
  <tr><td>Data</td><td><code>minecraft-text-action-dataset-noop-filtered</code> — 92.85% of raw dataset after removing 7.15% pure no-op collapse trajectories</td></tr>
  <tr><td>No-op frame drop (<code>KEEP_NO_OP_P</code>)</td><td><b>0.2</b> (drop 80% of no-op frames at data level)</td></tr>
  <tr><td>Focal loss</td><td>decay = 0.75 (down-weights repeated consecutive actions)</td></tr>
  <tr><td>Sample format</td><td>30-step trajectory segments: <code>user(prompt+instruction+img) → assistant(action) → user(img) → …</code></td></tr>
</table>

<h2><span class="num">04</span>Action Space</h2>
<p>The model outputs a single line per step:</p>
<pre>Action: move(dx, dy) and press(keys) and click(button)</pre>
<ul>
  <li><code>move(dx, dy)</code> — mouse movement (horizontal, vertical)</li>
  <li><code>press(keys)</code> — <code>w/a/s/d</code> movement, <code>space</code> jump, <code>e</code> inventory, <code>1</code>–<code>9</code> hotbar, <code>left.control</code> sprint, <code>left.shift</code> careful move</li>
  <li><code>click('left')</code> attack / <code>click('right')</code> place or use</li>
  <li><code>no_op</code> — wait</li>
</ul>

<h2><span class="num">05</span>A Real Trajectory from Training Data</h2>
<div class="instr">🎯 Task instruction: <b>Mine the cobweb to clear the path.</b></div>
<p style="color:var(--muted)">Six frames (evenly sampled from a 30-step segment) with the ground-truth actions the model learns to imitate:</p>
<div class="traj">
  <div class="step"><img src="assets/frame_0.jpg" alt="frame 0"><div class="meta"><div class="t">step 0</div><div class="a">Action: move(-53, 0) and press(a)</div></div></div>
  <div class="step"><img src="assets/frame_1.jpg" alt="frame 6"><div class="meta"><div class="t">step 6</div><div class="a">Action: move(0, 0) and press(w, a)</div></div></div>
  <div class="step"><img src="assets/frame_2.jpg" alt="frame 12"><div class="meta"><div class="t">step 12</div><div class="a">Action: move(13, 13) and press(w)</div></div></div>
  <div class="step"><img src="assets/frame_3.jpg" alt="frame 18"><div class="meta"><div class="t">step 18</div><div class="a">Action: move(80, 0) and press(d)</div></div></div>
  <div class="step"><img src="assets/frame_4.jpg" alt="frame 24"><div class="meta"><div class="t">step 24</div><div class="a">Action: move(7, -7) and press(w, d, left.control)</div></div></div>
  <div class="step"><img src="assets/frame_5.jpg" alt="frame 29"><div class="meta"><div class="t">step 29</div><div class="a">Action: move(0, 0) and press(w, left.control)</div></div></div>
</div>

<h2><span class="num">06</span>Inference</h2>
<pre># vLLM
vllm serve EmberJin/Minecraft-Agent-Qwen3.5-9B-Stage3 \
  --limit-mm-per-prompt '{"image": 30}' \
  --enable-prefix-caching \
  --max-model-len 32768</pre>

<div class="note">⚠️ GUI tasks (crafting / smelting / chest interaction) are out of scope: all text-action models score 0% on them — the action space has no interface-coordinate grounding.</div>

<footer>
  Minecraft Agent · Qwen3.5-9B Stage3 · easy-ng benchmark · single seed<br>
  <a href="cot.html" style="color: var(--blue); text-decoration: none;">CoT Demo Dataset</a> · 531 trajectories
  <a href="https://huggingface.co/EmberJin/Minecraft-Agent-Qwen3.5-9B-Stage3">Model repository</a>
</footer>

</div>
</body>
</html>