Bind Creative placement to task IDs and retain Gemini failure diagnostics

README.md

@@ -27,7 +27,7 @@ Endpoint OpenAI-compatible dedicado ao projeto original MineDojo/Voyager.
 - Alias principal: `granite-4.0-h-tiny`
 - CPU: 16 threads de decode e 16 threads de prefill/batch
 
-Action, Curriculum, Critic e Skill Manager usam o mesmo Granite H-Tiny. O
+No Naldo_Bot, Action, Curriculum, Critic e Skill Manager usam o Granite H-Tiny. O
 Curriculum em Creative usa contexto determinístico em vez de consultar QA
 Survival para cada tarefa, reduzindo latência e evitando receitas/pré-requisitos
 incompatíveis com o modo Creative.
@@ -52,9 +52,9 @@ O Voyager original usa embeddings no SkillManager e no cache de QA do Curriculum
 Referência:
 https://huggingface.co/Qwen/Qwen3-Embedding-0.6B-GGUF
 
-## CPU-only estrito
+## Inferência local somente em CPU
 
-A inferência não usa GPU:
+A inferência Granite e os embeddings locais não usam GPU:
 
 - `CUDA_VISIBLE_DEVICES=""`
 - `HIP_VISIBLE_DEVICES=""`
@@ -73,16 +73,15 @@ a função não é chamada pela API.
 
 ## Compatibilidade Voyager
 
-Por padrão `PLAYER_MODE=0`. Portanto o servidor não injeta o antigo prompt Bedrock e
-não força JSON. As adições aos prompts exigem inventário real, contagem por `item.count`,
-aquisição/crafting dos pré-requisitos e verificação do resultado observado.
+Por padrão `PLAYER_MODE=0`. O servidor não injeta o antigo prompt Bedrock.
+Os dois Voyagers jogam em Creative: mineração, colocação, equipamento e inventário
+são verificados a partir da observação real. O `placeBlocks` recebe o ID exato
+da tarefa canônica; o agente não pode substituir `bricks` por `brick`.
 
 O endpoint aceita temperatura 0.0, que coincide com os defaults do Voyager para
 ActionAgent, CurriculumAgent, CriticAgent e SkillManager.
 
-Para clientes OpenAI que usam `tools`, a camada HTTP converte o formato XML nativo do
-Qwen3-Coder (`<tool_call>`, `<function=...>`, `<parameter=...>`) para `tool_calls` no
-formato OpenAI. Isso não altera os prompts nem o loop do Voyager.
+O endpoint mantém a compatibilidade HTTP de chat e embeddings para os clientes.
 
 ## API
 
@@ -92,22 +91,23 @@ formato OpenAI. Isso não altera os prompts nem o loop do Voyager.
 - `GET /v1/models`
 - `POST /v1/chat/completions`
 - `POST /v1/embeddings`
+- `GET /minecraft/status`
+- `GET /voyager/status`
 - `GET /gemini-bot/status`
 
 ## Segundo jogador: Gemini_Bot
 
-`Gemini_Bot` entra no mesmo servidor Java 1.19 em Creative, com processo,
-reconexão e estado independentes do Voyager/Naldo_Bot. A cada turno autônomo
-ele observa posição, jogadores próximos e blocos visíveis. O ADM
-`IffyBinkie6134` também pode escrever `Gemini_Bot, segue o Naldo` no chat.
-O bot aceita uma decisão JSON do Gemini CLI e executa somente ações pequenas
-e validadas: explorar, seguir, minerar um bloco próximo, colocar um bloco,
-responder no chat ou aguardar. Mineração e colocação verificam o bloco real.
+`Gemini_Bot` executa uma segunda instância completa do Voyager no mesmo servidor
+Java 1.19, com processo Mineflayer e checkpoint próprios. Curriculum, Action,
+Critic e Skill Manager consultam o Gemini CLI por um proxy local exclusivo;
+somente os embeddings usam o modelo local. `Naldo_Bot` continua usando Granite.
+Ambos validam tarefas Creative pela mudança real no mundo ou pelo inventário.
 
 - CLI fixado em `@google/gemini-cli@0.61.0`, modelo efetivo `gemini-3.5-flash-lite`.
 - Segredos privados do Space: `GEMINI_BOT_KEY_1` a `GEMINI_BOT_KEY_6`.
 - Nenhuma chave deve ser salva no Git, no checkpoint ou nas saídas de status.
-- Status: `/gemini-bot/status` (conexão, decisões e contadores; sem chaves).
+- Status: `/gemini-bot/status` (conexão, tarefas, métricas e falhas por posição
+  da chave; nunca revela valores de chaves).
 - Controle de inicialização: `GEMINI_BOT_AUTOSTART=0` desativa somente o
   segundo bot. O Naldo_Bot e o mundo persistente seguem independentes.
 
@@ -129,8 +129,12 @@ O comportamento do agente deve continuar vindo do projeto original:
 https://github.com/MineDojo/Voyager
 
 Este Space executa o Voyager fixado em `55e45a880755d0c8c66ca7fb5fe7962ac8974f89`
-com correções de reconexão, observações nulas, movimento e mineração limitada.
-Mundo, autenticação e checkpoint permanecem em `/persist/minecraft-friends`.
+com correções de reconexão, observações nulas, exploração horizontal limitada e
+mineração que não cava o chão sob o jogador. O mundo plano tem superfície em Y=64:
+`/persist/minecraft-friends/world-20260927-flat-y64-v1`. Os checkpoints atuais
+são `/persist/minecraft-friends/voyager/ckpt-20260927-flat-y64-v1` e
+`/persist/minecraft-friends/gemini-voyager/ckpt-20260927-flat-y64-v1`.
+A autenticação Xbox/Friends também persiste sob `/persist/minecraft-friends`.
 
 Validação offline, sem iniciar modelos ou servidor de jogo:
 `python tests/test_voyager_patch.py /caminho/para/Voyager-oficial`

gemini_voyager_proxy.py

@@ -27,7 +27,9 @@ EMBED_BASE = os.environ.get("GEMINI_VOYAGER_EMBED_BASE", "http://127.0.0.1:7860/
 LOCK = threading.Lock()
 STATE_LOCK = threading.Lock()
 state = {"calls": 0, "failures": 0, "selected_key": 0, "last_elapsed_s": None,
-         "prompt_tokens": 0, "completion_tokens": 0, "last_error": None}
+         "prompt_tokens": 0, "completion_tokens": 0, "last_error": None,
+         "last_failure": None, "last_failure_at": None,
+         "last_failed_key_slot": None, "failures_by_key_slot": [0] * len(KEYS)}
 api = FastAPI(title="Gemini Voyager Adapter", version="1.0.0")
 def _content(value: Any) -> str:
     if isinstance(value, str):
@@ -108,11 +110,16 @@ def _call_cli(prompt: str) -> tuple[str, int, int]:
                     state["last_error"] = None
                 return envelope["response"], p_tokens, c_tokens
             code = envelope.get("error", {}).get("code") if isinstance(envelope.get("error"), dict) else None
-            last = f"Gemini CLI exit {proc.returncode}" + (f" API {code}" if code is not None else "")
+            safe_code = str(code) if isinstance(code, (str, int)) and str(code).isalnum() and len(str(code)) <= 32 else None
+            last = f"Gemini CLI exit {proc.returncode}" + (f" API {safe_code}" if safe_code else "")
             with STATE_LOCK:
                 state["selected_key"] = (slot + 1) % len(KEYS)
                 state["failures"] += 1
                 state["last_error"] = last
+                state["last_failure"] = last
+                state["last_failure_at"] = int(time.time())
+                state["last_failed_key_slot"] = slot + 1
+                state["failures_by_key_slot"][slot] += 1
         raise RuntimeError(last)
 
 

tests/test_gemini_bot_stack.py

@@ -1,9 +1,14 @@
 """Gemini_Bot runs the same Voyager architecture with isolated state and credentials."""
 import ast
+import json
 import os
 from pathlib import Path
+import subprocess
 import sys
 from tempfile import TemporaryDirectory
+import threading
+import time
+from types import SimpleNamespace
 from unittest.mock import patch
 
 ROOT = Path(__file__).resolve().parents[1]
@@ -40,7 +45,34 @@ def test_no_stored_credentials():
     assert tree is not None
 
 
+def test_proxy_preserves_safe_failure_diagnostics():
+    source = ast.parse((ROOT / "gemini_voyager_proxy.py").read_text())
+    func = next(node for node in source.body if isinstance(node, ast.FunctionDef) and node.name == "_call_cli")
+    state = {"calls": 0, "failures": 0, "selected_key": 0, "last_elapsed_s": None,
+             "prompt_tokens": 0, "completion_tokens": 0, "last_error": None,
+             "last_failure": None, "last_failure_at": None, "last_failed_key_slot": None,
+             "failures_by_key_slot": [0, 0]}
+    ns = {"KEYS": ["private-1", "private-2"], "LOCK": threading.Lock(),
+          "STATE_LOCK": threading.Lock(), "state": state, "os": os, "subprocess": subprocess,
+          "json": json, "time": time, "CLI": "gemini", "MODEL": "gemini-3.5-flash-lite",
+          "WORKSPACE": "/tmp", "HOME": "/tmp", "SYSTEM_MD": "/tmp/system.md",
+          "_usage": lambda stats: (12, 3)}
+    exec(compile(ast.Module(body=[func], type_ignores=[]), "proxy.py", "exec"), ns)
+    responses = [SimpleNamespace(returncode=1, stdout='{"error":{"code":429}}'),
+                 SimpleNamespace(returncode=0, stdout='{"response":"ok","stats":{}}')]
+    with patch.object(subprocess, "run", side_effect=responses) as run:
+        assert ns["_call_cli"]("hello") == ("ok", 12, 3)
+    assert run.call_args_list[0].kwargs["env"]["GEMINI_API_KEY"] == "private-1"
+    assert run.call_args_list[1].kwargs["env"]["GEMINI_API_KEY"] == "private-2"
+    assert state["failures"] == 1 and state["last_error"] is None
+    assert state["last_failure"] == "Gemini CLI exit 1 API 429"
+    assert state["last_failed_key_slot"] == 1
+    assert state["failures_by_key_slot"] == [1, 0]
+    assert "private-" not in str(state)
+
+
 if __name__ == "__main__":
     test_isolation()
     test_no_stored_credentials()
+    test_proxy_preserves_safe_failure_diagnostics()
     print("Gemini bot stack tests passed")

tests/test_voyager_patch.py

@@ -27,6 +27,7 @@ from voyager_verification import (
     verify_task, quarantine_invalid_skills, task_history,
 )
 from voyager_action_parse import extract_action_code
+from voyager_action_parse import canonical_place_code
 
 
 def functions(filename, names, namespace):
@@ -86,7 +87,7 @@ def test_resume():
             file.parent.mkdir(parents=True, exist_ok=True)
             file.write_text("{}")
         original_completed = ["Mine 1 stone block to upgrade tools", "Mine 1 stone", "Place 4 snow_block"]
-        original_failed = ["Mine 1 stone", "Craft a stone pickaxe", "Craft a stone pickaxe"]
+        original_failed = ["Mine 1 stone", "Craft a stone pickaxe", "Craft a stone pickaxe", "Place 1 bricks"]
         (ckpt / "curriculum/completed_tasks.json").write_text(json.dumps(original_completed))
         (ckpt / "curriculum/failed_tasks.json").write_text(json.dumps(original_failed))
         voyager = Mock()
@@ -132,6 +133,7 @@ def test_resume():
         voyager.return_value.env.mineflayer.process.terminate.assert_called_once_with()
         assert json.loads((ckpt / "curriculum/completed_tasks.json").read_text()) == original_completed[::2]
         assert json.loads((ckpt / "curriculum/failed_tasks.json").read_text()) == ["Craft a stone pickaxe"]
+        assert json.loads((ckpt / "quarantine/bricks_block_id_failure_v1.json").read_text()) == {"failed": original_failed}
         archive = json.loads((ckpt / "quarantine/curriculum_history_before_semantic_compaction_v1.json").read_text())
         assert archive == {"completed": original_completed, "failed": original_failed}
         voyager.return_value.learn.side_effect = RuntimeError("fatal curriculum error")
@@ -239,15 +241,17 @@ def check_action_behavior(source):
     def parse_js(code):
         subprocess.run(["node", "--check", "-"], input=code, text=True,
                        capture_output=True, check=True)
-        node = Node(type="FunctionDeclaration", id=SimpleNamespace(name="placeStone"),
+        name = code.split("function ", 1)[1].split("(", 1)[0]
+        node = Node(type="FunctionDeclaration", id=SimpleNamespace(name=name),
                     params=[SimpleNamespace(type="Identifier", name="bot")])
         node["async"] = True
+        node["source"] = code
         return SimpleNamespace(program=SimpleNamespace(body=[node]))
 
-    generator = lambda node: SimpleNamespace(code="async function placeStone(bot) { await placeBlocks(bot, 'stone', 1); }")
+    generator = lambda node: SimpleNamespace(code=node["source"])
     require = lambda package: (SimpleNamespace(parseSync=parse_js) if package == "@babel/core"
                                else SimpleNamespace(default=generator))
-    ns = {"extract_action_code": extract_action_code, "require": require}
+    ns = {"extract_action_code": extract_action_code, "canonical_place_code": canonical_place_code, "require": require}
     exec(compile(ast.Module(body=[method], type_ignores=[]), "action.py", "exec"), ns)
     process = MethodType(ns["process_ai_message"], SimpleNamespace())
     samples = [
@@ -271,6 +275,12 @@ def check_action_behavior(source):
             assert "await placeStone(bot)" not in extract_action_code(content)
         else:
             assert "before program execution" in result
+    wrong_id = "```js\nasync function placeBrick(bot) { await placeBlocks(bot, 'brick_block', 4); }\n```"
+    repaired = process(SimpleNamespace(content=wrong_id), task="Place 1 bricks")
+    assert repaired["program_name"] == "placeBrick"
+    assert 'placeBlocks(bot, "bricks", 1)' in repaired["program_code"]
+    assert "brick_block" not in repaired["program_code"]
+    assert "brick_block" in process(SimpleNamespace(content=wrong_id), task="Mine 1 stone")["program_code"]
     print("PASS Action parser normalizes prose, rejects ambiguity, truncation and extra functions")
 
 
@@ -415,6 +425,7 @@ const bot = {entity:{position:{x:0.5,y:64,z:0.5}},
         assert "Blocked invalid movement packet" in index
         core = (stack.SRC / "voyager/voyager.py").read_text()
         assert "Recovered Voyager observation without reconnecting Mineflayer" in core
+        assert "process_ai_message(message=ai_message, task=self.task)" in core
         assert "traceback.print_exc()" in core
 
 

voyager_action_parse.py

@@ -1,7 +1,10 @@
 """Extract one executable Action function from small-model responses."""
 
+import json
 import re
 
+from voyager_verification import SAFE_PLACE_BLOCKS, canonical_task, task_spec
+
 
 _FENCE = re.compile(r"```([^\n`]*)\n(.*?)```", re.DOTALL)
 _FUNCTION = re.compile(
@@ -79,3 +82,19 @@ def extract_action_code(response: str) -> str:
     if len(functions) != 1:
         raise ValueError(f"Expected exactly one complete Action function; found {len(functions)}")
     return functions[0]
+
+
+def canonical_place_code(task: str, function_name: str) -> str | None:
+    """Bind a Creative placement to the verified task, not a guessed block ID."""
+    if canonical_task(task) != task:
+        return None
+    spec = task_spec(task)
+    if not spec or spec[0] != "place" or spec[2] not in SAFE_PLACE_BLOCKS:
+        return None
+    if not re.fullmatch(r"[A-Za-z_$][\w$]*", function_name):
+        raise ValueError("Invalid Action function name")
+    _, count, name = spec
+    return (
+        f"async function {function_name}(bot) {{ "
+        f"await placeBlocks(bot, {json.dumps(name)}, {count}); }}"
+    )

voyager_runner.py

@@ -107,6 +107,20 @@ def main() -> None:
                 repair_marker.write_text("ok\n", encoding="utf-8")
                 print("[voyager] removed false starter-task failure from missing Creative context", flush=True)
 
+            # Granite called placeBlocks("brick") / ("brick_block") for the
+            # legitimate "Place 1 bricks" task. Both failed before changing
+            # the world. Archive the failure before allowing one fresh attempt
+            # with the exact task ID bound by the Action parser.
+            bricks_marker = ckpt / ".bricks_block_id_repair_v1"
+            if bot_username == "Naldo_Bot" and not bricks_marker.exists():
+                if "Place 1 bricks" in failed and "Place 1 bricks" not in completed:
+                    bricks_archive = ckpt / "quarantine" / "bricks_block_id_failure_v1.json"
+                    bricks_archive.parent.mkdir(parents=True, exist_ok=True)
+                    bricks_archive.write_text(json.dumps({"failed": original_failed}, indent=2) + "\n", encoding="utf-8")
+                    failed = [item for item in failed if item != "Place 1 bricks"]
+                    print("[voyager] archived invalid brick-ID failure for one retry", flush=True)
+                bricks_marker.write_text("ok\n", encoding="utf-8")
+
             for path, values, original in ((completed_path, completed, original_completed), (failed_path, failed, original_failed)):
                 if values != original:
                     temporary = path.with_suffix(".json.tmp")

voyager_stack.py

@@ -520,7 +520,7 @@ async function exploreUntil(bot, direction, maxTime = 60, callback) {
             '            observation += f"Critique: {critique}\\n\\n"',
             '            observation += f"Critique: {critique[:1200]}\\n\\n"',
         )
-        safe_action_parser = '''    def process_ai_message(self, message):
+        safe_action_parser = '''    def process_ai_message(self, message, task=None):
         # Do not concatenate fences or try the same malformed response 3 times.
         # The caller retries with fresh feedback while Mineflayer stays connected.
         try:
@@ -535,6 +535,12 @@ async function exploreUntil(bot, direction, maxTime = 60, callback) {
             params = list(node["params"])
             if len(params) != 1 or params[0].type != "Identifier" or params[0].name != "bot":
                 raise ValueError("Action function must take exactly one bot argument")
+            # A valid Place task supplies its exact Minecraft block ID. Small
+            # models sometimes change "bricks" into the non-block "brick";
+            # retain the function name while binding the primitive to the task.
+            place_code = canonical_place_code(task, node.id.name) if task else None
+            if place_code:
+                node = babel.parseSync(place_code).program.body[0]
             return {
                 "program_code": babel_generator(node).code,
                 "program_name": node.id.name,
@@ -552,7 +558,7 @@ async function exploreUntil(bot, direction, maxTime = 60, callback) {
         )
         if action_parser_count != 1:
             raise RuntimeError("Voyager Action parser marker changed")
-        action_agent_text = 'from voyager_action_parse import extract_action_code\n' + action_agent_text
+        action_agent_text = 'from voyager_action_parse import extract_action_code, canonical_place_code\n' + action_agent_text
         action_agent_path.write_text(action_agent_text, encoding="utf-8")
 
         # Bound curriculum memory rendered into every task proposal. The full
@@ -901,16 +907,24 @@ async function exploreUntil(bot, direction, maxTime = 60, callback) {
         # be parsed. EventRecorder immediately indexes events[0], so a truncated
         # LLM response turns a recoverable parse retry into IndexError. Record
         # the last valid observation instead and let Voyager retry the action.
-        voyager_src = voyager_src.replace(
-            '''            self.recorder.record([], self.task)
+        voyager_src = voyager_src.replace(
+            '''            self.recorder.record([], self.task)
             print(f"\\033[34m{parsed_result} Trying again!\\033[0m")
 ''',
             '''            if self.last_events:
                 self.recorder.record(self.last_events, self.task)
             print(f"\\033[34m{parsed_result} Trying again!\\033[0m")
 ''',
-            1,
-        )
+            1,
+        )
+        action_call = '        parsed_result = self.action_agent.process_ai_message(message=ai_message)'
+        if voyager_src.count(action_call) != 1:
+            raise RuntimeError("Voyager Action call marker changed")
+        voyager_src = voyager_src.replace(
+            action_call,
+            '        parsed_result = self.action_agent.process_ai_message(message=ai_message, task=self.task)',
+            1,
+        )
 
         old_recovery = '''            except Exception as e:\n                time.sleep(3)  # wait for mineflayer to exit\n                info = {\n                    "task": task,\n                    "success": False,\n                }\n                # reset bot status here\n                self.last_events = self.env.reset(\n                    options={\n                        "mode": "hard",\n                        "wait_ticks": self.env_wait_ticks,\n                        "inventory": self.last_events[-1][1]["inventory"],\n                        "equipment": self.last_events[-1][1]["status"]["equipment"],\n                        "position": self.last_events[-1][1]["status"]["position"],\n                    }\n                )\n                # use red color background to print the error\n                print("Your last round rollout terminated due to error:")\n                print(f"\\033[41m{e}\\033[0m")\n'''
         new_recovery = '''            except Exception as e:\n                info = {\n                    "task": task,\n                    "success": False,\n                }\n                print("Your last round rollout terminated due to error:")\n                traceback.print_exc()\n                print(f"\\033[41m{type(e).__name__}: {e}\\033[0m")\n                try:\n                    # Preserve the current Mineflayer connection whenever it is\n                    # still responsive. This prevents needless leave/rejoin.\n                    self.last_events = self.env.step("")\n                    print("Recovered Voyager observation without reconnecting Mineflayer")\n                except Exception as recovery_error:\n                    print("Live recovery failed; performing one hard Mineflayer reset")\n                    traceback.print_exc()\n                    previous = self.last_events[-1][1] if self.last_events else {}\n                    status = previous.get("status", {}) if isinstance(previous, dict) else {}\n                    self.last_events = self.env.reset(\n                        options={\n                            "mode": "hard",\n                            "wait_ticks": self.env_wait_ticks,\n                            "inventory": previous.get("inventory", {}) if isinstance(previous, dict) else {},\n                            "equipment": status.get("equipment", []),\n                            "position": status.get("position", None),\n                        }\n                    )\n'''