Remove files the package no longer has
Browse files- CHANGELOG.md +0 -44
- CHANGELOG.md.meta +0 -7
- Documentation~/SceneAgent.md +0 -98
- Editor/AgentE2ETest.cs +0 -94
- Editor/AgentE2ETest.cs.meta +0 -11
- Runtime/PyMath.cs +0 -35
- Runtime/PyMath.cs.meta +0 -11
- Runtime/Resources/SceneAgent/system_prompt_v3.txt +0 -16
- Runtime/Resources/SceneAgent/system_prompt_v3.txt.meta +0 -7
- Third Party Notices.md +0 -12
- Third Party Notices.md.meta +0 -7
CHANGELOG.md
DELETED
|
@@ -1,44 +0,0 @@
|
|
| 1 |
-
# Changelog
|
| 2 |
-
|
| 3 |
-
## [1.0.0-rc.2] - 2026-09-28 (not published)
|
| 4 |
-
### Changed
|
| 5 |
-
- Renamed to **Maverick XR Agent** (`com.mcbuxrlab.maverick`) for the Maverick-4B-Unity-XR-Agent model; the C# namespaces and
|
| 6 |
-
component names are unchanged.
|
| 7 |
-
- Install from Git: `https://huggingface.co/ErenAta00/Maverick-Unity.git`.
|
| 8 |
-
- Licence holder: Eren Ata, MCBU XRLab (MIT).
|
| 9 |
-
|
| 10 |
-
## [1.0.0-rc.1] - 2026-09-27 (not published)
|
| 11 |
-
### Added
|
| 12 |
-
- `LocalModelServer` in the package (was in the demo): finds llama-server and the `.gguf` in
|
| 13 |
-
`StreamingAssets/SceneAgent`, GPU first with CPU fallback, `capFrameRate` while the model runs on the GPU,
|
| 14 |
-
`ModeChanged` event.
|
| 15 |
-
- `SceneAgentSession`: commands, clarification answers with the trained follow-up, optional JSON-lines session log.
|
| 16 |
-
- `SceneAgentChatBox`: on-screen chat to try the agent without writing UI.
|
| 17 |
-
- `SceneAgentScriptRunner`: plays command scripts and writes a report (Editor, build, or command line).
|
| 18 |
-
- D5b answer checking (`SceneGrammar.Check`, `SceneGrammar.Schema`), on by default, verified against the evaluation's
|
| 19 |
-
Python check on 59,294 cases.
|
| 20 |
-
- Editor menu Tools → Scene Agent: Set Up Scene, Make Selected Interactive, Validate Scene, Open Model Folder.
|
| 21 |
-
- `SceneWorld.initFromTransforms` (positions read from the scene at Start), `SceneWorld.ToolExecuted` event.
|
| 22 |
-
- `SceneItem` affordance overrides for labels outside the catalogue.
|
| 23 |
-
- The v3 system prompt ships in `Resources/SceneAgent`.
|
| 24 |
-
- README, reference documentation, MIT license, third-party notices.
|
| 25 |
-
- `SceneAgentGaze`: "this / that" from the mouse pointer or the camera centre.
|
| 26 |
-
- Typed clarification answers ("the red one") in `SceneAgentSession`.
|
| 27 |
-
- Substitute guard (`SceneAgent.guardSubstitutes`).
|
| 28 |
-
- The model server is tied to the app's lifetime on Windows (a job object: no llama-server left after a crash) and
|
| 29 |
-
waits for the old server to exit on a scene reload.
|
| 30 |
-
- `-sceneAgentScript` works in any scene with a session.
|
| 31 |
-
- Readable tool lines in the chat (`place(hammer_1, toolbox_1, in) ok`); raw JSON with `SceneAgentSession.rawToolLines`.
|
| 32 |
-
- `SceneAgentScriptRunner` gives the gaze back to `SceneAgentGaze` when the script ends.
|
| 33 |
-
### Fixed
|
| 34 |
-
- Play mode in the Editor: an unassigned `systemPromptAsset` threw `UnassignedReferenceException` on the first command.
|
| 35 |
-
- Make Selected Interactive: duplicated objects (Ctrl+D) get a fresh id; nested objects move directly under the Scene
|
| 36 |
-
Agent object and their supports are inferred from their own renderers; devices get a default state.
|
| 37 |
-
- Validate Scene: a model path pointing nowhere is an error; no dialogs in batch mode.
|
| 38 |
-
- A context overflow gives a readable message instead of the raw HTTP 400.
|
| 39 |
-
### Changed
|
| 40 |
-
- `SceneAgent` defaults: D5b on, strict D5 off; waits for its `LocalModelServer` when one is set.
|
| 41 |
-
|
| 42 |
-
## [0.1.0]
|
| 43 |
-
- SceneWorld (tool semantics mirroring the training simulator), SceneAgent (chat loop), strict D5 grammar, the Unity
|
| 44 |
-
E2E harness.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
CHANGELOG.md.meta
DELETED
|
@@ -1,7 +0,0 @@
|
|
| 1 |
-
fileFormatVersion: 2
|
| 2 |
-
guid: 88b114c1c231c3b4d856afc169a4b9d9
|
| 3 |
-
TextScriptImporter:
|
| 4 |
-
externalObjects: {}
|
| 5 |
-
userData:
|
| 6 |
-
assetBundleName:
|
| 7 |
-
assetBundleVariant:
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Documentation~/SceneAgent.md
DELETED
|
@@ -1,98 +0,0 @@
|
|
| 1 |
-
# Scene Agent: reference
|
| 2 |
-
|
| 3 |
-
Start with the package `README.md` (install, quick start). This page lists what each part does and how the model sees
|
| 4 |
-
your scene.
|
| 5 |
-
|
| 6 |
-
## How a command runs
|
| 7 |
-
1. `SceneWorld.BuildSceneJson()` lists the player's room: every `SceneItem` there, the room names, the player's hands
|
| 8 |
-
and gaze.
|
| 9 |
-
2. `SceneAgent.Run` sends the system prompt, the scene and the request to the local model (OpenAI-compatible
|
| 10 |
-
`/v1/chat/completions` on llama.cpp, temperature 0).
|
| 11 |
-
3. The answer is either text (done) or tool calls. With D5b (default) each answer is checked first (below).
|
| 12 |
-
4. Each call runs through `SceneWorld.Execute`, which changes the logical state, moves the GameObject
|
| 13 |
-
(`SceneWorld.Sync`) and raises `ToolExecuted`. A call Unity refuses (`ToolError`) goes back to the model as
|
| 14 |
-
`{"ok": false, "error": "…"}`.
|
| 15 |
-
5. Results go back to the model; up to `maxTurns` (4) rounds. `ask_clarification` ends the turn with a question; the
|
| 16 |
-
user's pick comes back as the next request (`SceneAgentSession.Answer` builds it).
|
| 17 |
-
|
| 18 |
-
## What the model sees
|
| 19 |
-
```json
|
| 20 |
-
{"player": {"room": "kitchen", "gaze": null, "left_hand": null, "right_hand": "mug_1"},
|
| 21 |
-
"locations": ["kitchen", "living_room", "office"],
|
| 22 |
-
"visible_objects": [
|
| 23 |
-
{"id": "counter_1", "label": "counter", "on": "floor", "pos": [-1.6, 0.0, 1.2]},
|
| 24 |
-
{"id": "mug_2", "label": "mug", "color": "red", "on": "counter_1", "pos": [-1.6, 0.9, 1.2]},
|
| 25 |
-
{"id": "fridge_1", "label": "fridge", "color": "white", "on": "floor", "state": "closed", "pos": [1.9, 0.0, 2.4]}]}
|
| 26 |
-
```
|
| 27 |
-
Positions are room-local metres of the object's bottom, rounded to 0.1 m. Only the player's room is listed.
|
| 28 |
-
|
| 29 |
-
## Components
|
| 30 |
-
| component | fields you may set |
|
| 31 |
-
|---|---|
|
| 32 |
-
| `SceneWorld` | `rooms`, `playerRoom`, `gaze`, `leftHand` / `rightHand`, `leftHandAnchor` / `rightHandAnchor` (where held objects are shown), `initFromTransforms` (read positions from the scene at Start), `roomOrigin` (code: where each room sits in world space) |
|
| 33 |
-
| `SceneItem` | `itemId` (unique, e.g. `mug_1`), `label` (plain noun), `color`, `room`, `on` (support id or `floor`), `state` (e.g. `off`, `closed`), affordance override flags |
|
| 34 |
-
| `SceneAgent` | `world`, `server`, `endpoint`, `systemPromptAsset` (leave empty), `maxTurns`, `fallbackDecoding` (D5b, on), `constrainedDecoding` (strict D5, off) |
|
| 35 |
-
| `LocalModelServer` | `serverExe`, `modelPath`, `port`, `contextSize`, `preferGpu`, `cpuThreads`, `capFrameRate`, `launchOnStart`, `extraArgs` |
|
| 36 |
-
| `SceneAgentSession` | `agent`, `world`, `logDirectory`; `Command(text)`, `Answer(itemId)`, events `Line`, `Finished`, `PendingRequest`, `Candidates` |
|
| 37 |
-
| `SceneAgentChatBox` | `visibleLines`, `toggleKey` (F1), `examples` |
|
| 38 |
-
| `SceneAgentScriptRunner` | `script`, `outputDirectory`, `screenshots`, `quitWhenDone`; command line `-sceneAgentScript`, `-sceneAgentOut`, `-sceneAgentShots` |
|
| 39 |
-
|
| 40 |
-
## Tools
|
| 41 |
-
| tool | arguments | Unity refuses when |
|
| 42 |
-
|---|---|---|
|
| 43 |
-
| `grab` | object_id, hand (left/right/both) | not graspable, already held, hand not free |
|
| 44 |
-
| `release` | hand | nothing in that hand (the object drops to the floor) |
|
| 45 |
-
| `place` | object_id, target_id, relation (on/in), orientation (default/upright/horizontal/vertical/upside_down) | object not carryable; target not a surface (on) or not a container / closed (in) |
|
| 46 |
-
| `move_by` | object_id, direction (left/right/forward/backward/up/down), distance, unit (m/cm) | held or fixed object |
|
| 47 |
-
| `rotate` | object_id, degrees, axis (yaw/pitch/roll) | held or fixed object |
|
| 48 |
-
| `set_state` | object_id, state (on/off/open/closed/half_open/locked/unlocked) | the object has no such state |
|
| 49 |
-
| `press` | object_id, hold_seconds | not a button |
|
| 50 |
-
| `highlight` | object_ids | an id not in the room |
|
| 51 |
-
| `teleport` | location_id | not one of `rooms` (held objects come along) |
|
| 52 |
-
| `find_objects` | label, color, room | never; returns the matches in every room |
|
| 53 |
-
| `ask_clarification` | question, candidate_ids | never; sets `SceneWorld.clarify` and ends the turn |
|
| 54 |
-
|
| 55 |
-
Every tool acts on the player's room only: an object elsewhere is "not visible" until `teleport`.
|
| 56 |
-
|
| 57 |
-
## Affordances
|
| 58 |
-
`SceneCatalog` (generated from the training simulator) maps about 200 labels to what they afford: grasp, surface
|
| 59 |
-
height, container, fixed, states, button. `SceneItem.overrideAffordances` replaces the lookup for one item with its
|
| 60 |
-
own flags (`grabbable`, `isSurface` + `surfaceHeight`, `isContainer`, `isFixed`, `isButton`, `states`). The model
|
| 61 |
-
learned the catalogue's common sense; overrides that contradict it (a grabbable "table") will confuse it.
|
| 62 |
-
|
| 63 |
-
## D5b: checking the model's answer
|
| 64 |
-
With `fallbackDecoding` on, each answer is checked against the tool schema and the scene (`SceneGrammar.Check`,
|
| 65 |
-
byte-for-byte the same verdicts as the evaluation's Python check, 59,294 cases):
|
| 66 |
-
- `ok` → executed as it is.
|
| 67 |
-
- `invalid_tool` (a tool that does not exist) or `invalid_value` (an argument or value outside the schema, or a room
|
| 68 |
-
not in `rooms`) → replaced by the refusal "I can't do that; none of my tools can."
|
| 69 |
-
- `invalid_id` (an id not in the scene, the hands, the gaze or earlier tool results) or `malformed` (a call that does
|
| 70 |
-
not parse) → generated again with a grammar that only allows valid calls.
|
| 71 |
-
Each turn's verdict is in `AgentTurn.d5b`. Strict D5 (`constrainedDecoding`) applies the grammar to every answer; it
|
| 72 |
-
was measured to force actions where the model should refuse, so it is off.
|
| 73 |
-
|
| 74 |
-
## Substitute guard
|
| 75 |
-
`SceneAgent.guardSubstitutes` (on): before an acting call (grab, place, move_by, rotate, set_state, press) runs, if the
|
| 76 |
-
acted-on object is not named in the request while the request names an object that exists only in other rooms, the
|
| 77 |
-
call is skipped and the model receives `{"ok": false, "error": "not done: 'screwdriver_1' is a screwdriver, but the request
|
| 78 |
-
names a drill; the drill (drill_1) is in the storage, not in this room"}`. Naming uses the label or its last word
|
| 79 |
-
("desk lamp" is named by "lamp"), with plural endings. It never fires for pronouns ("put it there").
|
| 80 |
-
|
| 81 |
-
## Events
|
| 82 |
-
- `SceneWorld.ToolExecuted (string tool, Dictionary<string, object> args, object result)`: after every accepted call.
|
| 83 |
-
- `LocalModelServer.ModeChanged (Mode)`: Starting, Gpu, Cpu, External, Failed, Stopped.
|
| 84 |
-
- `SceneAgentSession.Line (string)` and `Finished (AgentResult)`.
|
| 85 |
-
|
| 86 |
-
## Script format (SceneAgentScriptRunner)
|
| 87 |
-
```
|
| 88 |
-
// comment
|
| 89 |
-
pick up the red mug a command
|
| 90 |
-
#answer mug_1 answer the pending "which one?" with this id
|
| 91 |
-
#look lamp_2 set the gaze ("#look none" clears it)
|
| 92 |
-
#wait 2.5 pause, in seconds (not counted in any step's time)
|
| 93 |
-
```
|
| 94 |
-
|
| 95 |
-
## Changing the model
|
| 96 |
-
Any GGUF of this model family works through `LocalModelServer.modelPath`. The system prompt in
|
| 97 |
-
`Resources/SceneAgent/system_prompt_v3.txt` must be the one the model was trained with; a different model needs its
|
| 98 |
-
own prompt and, for D5b, the same tool schema (`SceneGrammar.Schema.cs`, generated from `spec/tools_v0.json`).
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Editor/AgentE2ETest.cs
DELETED
|
@@ -1,94 +0,0 @@
|
|
| 1 |
-
// End-to-end test: build real GameObjects for each case, let SceneAgent serialise the scene, query the
|
| 2 |
-
// model server, execute the calls on the GameObjects, and compare the final state with gold.
|
| 3 |
-
// Unity -batchmode -nographics -projectPath <proj> -executeMethod AgentE2ETest.Run
|
| 4 |
-
// -e2eInput cases.json -e2eOutput report.json
|
| 5 |
-
using System;
|
| 6 |
-
using System.Collections.Generic;
|
| 7 |
-
using System.IO;
|
| 8 |
-
using System.Linq;
|
| 9 |
-
using UnityEditor;
|
| 10 |
-
using UnityEngine;
|
| 11 |
-
using SceneAgent;
|
| 12 |
-
|
| 13 |
-
[Serializable] public class E2EObj { public string id, label, color, room, on, relation, orientation, state; public bool has_pos; public double[] pos; public double yaw, pitch, roll; }
|
| 14 |
-
[Serializable] public class E2EWorld { public string[] rooms; public E2EObj[] objects; public string room, gaze, left_hand, right_hand; }
|
| 15 |
-
[Serializable] public class E2ECase { public string id; public string category; public string utterance; public E2EWorld world; public string gold_sig; public bool gold_text_end; }
|
| 16 |
-
[Serializable] public class E2ESuite { public string endpoint; public string model; public string prompt_path; public bool constrained; public bool fallback; public E2ECase[] cases; }
|
| 17 |
-
[Serializable] public class E2EResult { public string id, category, error, final_text, clarification, first_output, sig, scene, d5b; public bool ok; public int turns; public double seconds; }
|
| 18 |
-
[Serializable] public class E2EReport { public string unity; public int n; public double success_rate; public double mean_seconds; public List<E2EResult> results = new List<E2EResult>(); }
|
| 19 |
-
|
| 20 |
-
public static class AgentE2ETest
|
| 21 |
-
{
|
| 22 |
-
static Vector3 SizeFor(string label)
|
| 23 |
-
{
|
| 24 |
-
switch (label)
|
| 25 |
-
{
|
| 26 |
-
case "table": case "desk": return new Vector3(1.2f, 0.75f, 0.6f);
|
| 27 |
-
case "counter": return new Vector3(1.4f, 0.9f, 0.6f);
|
| 28 |
-
case "shelf": return new Vector3(1.0f, 1.2f, 0.35f);
|
| 29 |
-
case "door": return new Vector3(0.9f, 2.0f, 0.05f);
|
| 30 |
-
case "window": return new Vector3(1.0f, 1.0f, 0.05f);
|
| 31 |
-
case "chair": case "stool": case "crate": case "chest": return new Vector3(0.5f, 0.5f, 0.5f);
|
| 32 |
-
case "box": case "basket": case "vase": case "drawer": case "tv": case "fan": case "lamp": return new Vector3(0.4f, 0.4f, 0.4f);
|
| 33 |
-
default: return Vector3.one * 0.15f;
|
| 34 |
-
}
|
| 35 |
-
}
|
| 36 |
-
|
| 37 |
-
static string Nn(string s) => string.IsNullOrEmpty(s) ? null : s;
|
| 38 |
-
|
| 39 |
-
public static void Run()
|
| 40 |
-
{
|
| 41 |
-
var args = Environment.GetCommandLineArgs();
|
| 42 |
-
string input = null, output = null;
|
| 43 |
-
for (int i = 0; i < args.Length - 1; i++)
|
| 44 |
-
{
|
| 45 |
-
if (args[i] == "-e2eInput") input = args[i + 1];
|
| 46 |
-
if (args[i] == "-e2eOutput") output = args[i + 1];
|
| 47 |
-
}
|
| 48 |
-
var suite = JsonUtility.FromJson<E2ESuite>(File.ReadAllText(input));
|
| 49 |
-
string prompt = File.ReadAllText(suite.prompt_path).Replace("\r\n", "\n");
|
| 50 |
-
var report = new E2EReport { unity = Application.unityVersion, n = suite.cases.Length };
|
| 51 |
-
foreach (var cs in suite.cases)
|
| 52 |
-
{
|
| 53 |
-
var root = new GameObject("SceneRoot");
|
| 54 |
-
var world = root.AddComponent<SceneWorld>();
|
| 55 |
-
var agent = root.AddComponent<SceneAgent.SceneAgent>();
|
| 56 |
-
agent.endpoint = suite.endpoint; agent.model = suite.model; agent.systemPrompt = prompt; agent.world = world;
|
| 57 |
-
agent.constrainedDecoding = suite.constrained; // D5, off unless the runner asks (--grammar on)
|
| 58 |
-
agent.fallbackDecoding = suite.fallback; // D5b (--grammar d5b)
|
| 59 |
-
world.playerRoom = cs.world.room; world.gaze = Nn(cs.world.gaze);
|
| 60 |
-
world.leftHand = Nn(cs.world.left_hand); world.rightHand = Nn(cs.world.right_hand);
|
| 61 |
-
world.rooms = cs.world.rooms.ToList();
|
| 62 |
-
foreach (var o in cs.world.objects)
|
| 63 |
-
{
|
| 64 |
-
var go = GameObject.CreatePrimitive(PrimitiveType.Cube);
|
| 65 |
-
go.name = o.id;
|
| 66 |
-
go.transform.SetParent(root.transform, false);
|
| 67 |
-
go.transform.localScale = SizeFor(o.label);
|
| 68 |
-
var it = go.AddComponent<SceneItem>();
|
| 69 |
-
it.itemId = o.id; it.label = o.label; it.color = Nn(o.color); it.room = o.room; it.on = o.on;
|
| 70 |
-
it.relation = Nn(o.relation); it.orientation = Nn(o.orientation); it.state = Nn(o.state);
|
| 71 |
-
it.hasPos = o.has_pos; it.pos = (double[])o.pos.Clone(); it.yaw = o.yaw; it.pitch = o.pitch; it.roll = o.roll;
|
| 72 |
-
}
|
| 73 |
-
foreach (var it in world.Items) world.Sync(it);
|
| 74 |
-
|
| 75 |
-
string scene = world.BuildSceneJson(); // what the model is shown; compared with the offline prompt
|
| 76 |
-
var r = agent.RunBlocking(cs.utterance);
|
| 77 |
-
string sig = world.Signature();
|
| 78 |
-
var res = new E2EResult {
|
| 79 |
-
id = cs.id, category = cs.category, error = r.error, final_text = r.finalText, clarification = r.clarificationQuestion, sig = sig, turns = r.turns.Count,
|
| 80 |
-
seconds = r.seconds, first_output = r.turns.Count > 0 ? r.turns[0].modelOutput : null, scene = scene,
|
| 81 |
-
d5b = string.Join(",", r.turns.Select(t => t.d5b ?? "")) };
|
| 82 |
-
res.ok = r.error == null && sig == cs.gold_sig && (!cs.gold_text_end || r.finalText != null);
|
| 83 |
-
report.results.Add(res);
|
| 84 |
-
// one line per command, for the live dashboard (tools/dashboard/v3_view.py reads unity_e2e.log)
|
| 85 |
-
Debug.Log($"[E2E] case {report.results.Count}/{report.n} {res.id} ok={res.ok} turns={res.turns}");
|
| 86 |
-
UnityEngine.Object.DestroyImmediate(root);
|
| 87 |
-
}
|
| 88 |
-
report.success_rate = report.results.Count(x => x.ok) / (double)Math.Max(1, report.n);
|
| 89 |
-
report.mean_seconds = report.results.Average(x => x.seconds);
|
| 90 |
-
File.WriteAllText(output, JsonUtility.ToJson(report, true));
|
| 91 |
-
Debug.Log($"[E2E] n={report.n} success={report.success_rate:F3} mean_s={report.mean_seconds:F2}");
|
| 92 |
-
EditorApplication.Exit(0);
|
| 93 |
-
}
|
| 94 |
-
}
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Editor/AgentE2ETest.cs.meta
DELETED
|
@@ -1,11 +0,0 @@
|
|
| 1 |
-
fileFormatVersion: 2
|
| 2 |
-
guid: 96e9993592c204748af996724da83344
|
| 3 |
-
MonoImporter:
|
| 4 |
-
externalObjects: {}
|
| 5 |
-
serializedVersion: 2
|
| 6 |
-
defaultReferences: []
|
| 7 |
-
executionOrder: 0
|
| 8 |
-
icon: {instanceID: 0}
|
| 9 |
-
userData:
|
| 10 |
-
assetBundleName:
|
| 11 |
-
assetBundleVariant:
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Runtime/PyMath.cs
DELETED
|
@@ -1,35 +0,0 @@
|
|
| 1 |
-
// Python's round(x, n) and f"{x:.nf}" in C#, so the scene text and the tool effects match the training data byte for
|
| 2 |
-
// byte. Python rounds the exact binary value of x, ties to even; .NET's Math.Round(x, n) scales by 10^n first, so 0.45
|
| 3 |
-
// (binary 0.45000000000000001...) became 0.4 where the training scenes say 0.5, and -0.04 became 0.0 where they say
|
| 4 |
-
// -0.0 (v2 Unity reference on the v3 sample: 81 + 37 of 440 scenes differed from the offline prompt).
|
| 5 |
-
using System;
|
| 6 |
-
using System.Globalization;
|
| 7 |
-
using System.Numerics;
|
| 8 |
-
|
| 9 |
-
namespace SceneAgent
|
| 10 |
-
{
|
| 11 |
-
public static class PyMath
|
| 12 |
-
{
|
| 13 |
-
/// <summary>Python round(x, n) for n >= 0: exact, ties to even, sign kept (round(-0.04, 1) == -0.0).</summary>
|
| 14 |
-
public static double Round(double x, int n)
|
| 15 |
-
{
|
| 16 |
-
if (double.IsNaN(x) || double.IsInfinity(x)) return x;
|
| 17 |
-
long bits = BitConverter.DoubleToInt64Bits(x);
|
| 18 |
-
bool neg = bits < 0;
|
| 19 |
-
int e = (int)((bits >> 52) & 0x7FF);
|
| 20 |
-
long m = bits & 0xFFFFFFFFFFFFFL;
|
| 21 |
-
if (e == 0) e = 1; else m |= 1L << 52;
|
| 22 |
-
e -= 1075; // |x| = m * 2^e exactly
|
| 23 |
-
BigInteger num = new BigInteger(m) * BigInteger.Pow(10, n), den = BigInteger.One;
|
| 24 |
-
if (e > 0) num <<= e; else den <<= -e;
|
| 25 |
-
BigInteger q = BigInteger.DivRem(num, den, out BigInteger r);
|
| 26 |
-
int c = (r * 2).CompareTo(den);
|
| 27 |
-
if (c > 0 || (c == 0 && !q.IsEven)) q += 1;
|
| 28 |
-
double v = (double)q / Math.Pow(10, n); // q and 10^n are exact doubles: the division is correctly rounded
|
| 29 |
-
return neg ? -v : v;
|
| 30 |
-
}
|
| 31 |
-
|
| 32 |
-
/// <summary>Python f"{x:.{n}f}".</summary>
|
| 33 |
-
public static string Fixed(double x, int n) => Round(x, n).ToString("F" + n, CultureInfo.InvariantCulture);
|
| 34 |
-
}
|
| 35 |
-
}
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Runtime/PyMath.cs.meta
DELETED
|
@@ -1,11 +0,0 @@
|
|
| 1 |
-
fileFormatVersion: 2
|
| 2 |
-
guid: aa67273f40b6679448917c8a06ebe741
|
| 3 |
-
MonoImporter:
|
| 4 |
-
externalObjects: {}
|
| 5 |
-
serializedVersion: 2
|
| 6 |
-
defaultReferences: []
|
| 7 |
-
executionOrder: 0
|
| 8 |
-
icon: {instanceID: 0}
|
| 9 |
-
userData:
|
| 10 |
-
assetBundleName:
|
| 11 |
-
assetBundleVariant:
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Runtime/Resources/SceneAgent/system_prompt_v3.txt
DELETED
|
@@ -1,16 +0,0 @@
|
|
| 1 |
-
You are an assistant embedded in a 3D Unity scene. Turn the user's request into tool calls that use object ids exactly as listed in the scene; never invent an id or a tool. Only objects in the user's current room are listed. Match objects by meaning: a synonym or a more general word can name a listed object, and details the scene does not record (a colour when no colours are listed, or words like sliced, wooden, empty) do not rule an object out. If exactly one listed object fits, act on it. If several fit equally well, call ask_clarification with their ids. If none fits, use find_objects to look in the other rooms. place moves an object directly, so there is no need to grab it first. If a request is impossible or needs a tool you do not have, call no tool and briefly explain why. Answer questions about the scene in plain text.
|
| 2 |
-
|
| 3 |
-
# Tools
|
| 4 |
-
grab(object_id:str, hand:left|right|both)
|
| 5 |
-
release(hand:left|right|both)
|
| 6 |
-
place(object_id:str, target_id:str, relation?:on|in, orientation?:default|upright|horizontal|vertical|upside_down)
|
| 7 |
-
move_by(object_id:str, direction:left|right|forward|backward|up|down, distance:num, unit:m|cm)
|
| 8 |
-
rotate(object_id:str, degrees:num, axis?:yaw|pitch|roll)
|
| 9 |
-
set_state(object_id:str, state:on|off|open|closed|half_open|locked|unlocked)
|
| 10 |
-
press(object_id:str, hold_seconds?:num)
|
| 11 |
-
highlight(object_ids:list)
|
| 12 |
-
teleport(location_id:str)
|
| 13 |
-
find_objects(label:str, color?:str, room?:str)
|
| 14 |
-
ask_clarification(question:str, candidate_ids:list)
|
| 15 |
-
|
| 16 |
-
Reply with one <tool_call>{"name": ..., "arguments": {...}}</tool_call> block per call, or plain text.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Runtime/Resources/SceneAgent/system_prompt_v3.txt.meta
DELETED
|
@@ -1,7 +0,0 @@
|
|
| 1 |
-
fileFormatVersion: 2
|
| 2 |
-
guid: 6a38ee93718310f469efc52b22f6a1b9
|
| 3 |
-
TextScriptImporter:
|
| 4 |
-
externalObjects: {}
|
| 5 |
-
userData:
|
| 6 |
-
assetBundleName:
|
| 7 |
-
assetBundleVariant:
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Third Party Notices.md
DELETED
|
@@ -1,12 +0,0 @@
|
|
| 1 |
-
# Third Party Notices
|
| 2 |
-
|
| 3 |
-
This package's code is MIT (`LICENSE.md`). It does not contain the following; they are downloaded or placed next to
|
| 4 |
-
it by the user, under their own licenses:
|
| 5 |
-
|
| 6 |
-
| component | used for | license |
|
| 7 |
-
|---|---|---|
|
| 8 |
-
| llama.cpp (`llama-server` and its libraries), ggml authors | running the model locally | MIT |
|
| 9 |
-
| Qwen3 base models (Qwen team, Alibaba Cloud) | base of the fine-tuned Scene Agent model weights | Apache-2.0 |
|
| 10 |
-
| Maverick-4B-Unity-XR-Agent model weights (fine-tune of Qwen3-4B) | the model in the `.gguf` file | Apache-2.0, as a derivative of Qwen3 |
|
| 11 |
-
|
| 12 |
-
The CUDA build of llama.cpp needs NVIDIA's CUDA runtime libraries (distributed with that build under NVIDIA's terms).
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Third Party Notices.md.meta
DELETED
|
@@ -1,7 +0,0 @@
|
|
| 1 |
-
fileFormatVersion: 2
|
| 2 |
-
guid: 897bd16f1021def4ea23daa6d7637e56
|
| 3 |
-
TextScriptImporter:
|
| 4 |
-
externalObjects: {}
|
| 5 |
-
userData:
|
| 6 |
-
assetBundleName:
|
| 7 |
-
assetBundleVariant:
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|