Coding agents can now operate 3D engines: they write and run code, inspect the result and revise the scene. Code4Scene is a
+benchmark of 190 Unreal Engine cases built from human-assembled scenes that evaluates them on two complementary settings under a shared execution
+interface. Construction starts from an empty level and an open-ended text specification. Editing gives the agent a corrupted copy of a
+scene and reference images of the original, and asks it to restore the intended state while leaving everything else untouched.
+
Rather than scoring code or rendered views, Code4Scene evaluates the engine-native scene the agent saved, on task fulfilment, artifact integrity
+and static physical validity, and compares edits against withheld ground truth. Across 14 agent configurations, construction and editing rank the
+agents differently, spatial relations remain the weakest requirement family, and a third of complete repairs still change the surrounding scene.
1 · InputPrompt or referenceAn open-ended scene description, or a corrupted scene with reference images of the original.
2 · AgentCoding agentA frontier or open-weight model in its own CLI harness, with the Unreal Engine editor as a tool.
3 · CodeCode in the engineIt writes and runs Python against the editor: spawns, moves and edits actors, inspects, revises.
4 · SceneEngine-native sceneThe saved level (.umap): every actor, transform and material, as the engine holds it.
5 · EvaluatorScored in the engineRequirements located and judged, overview views judged, physics measured, edits diffed against the withheld ground truth.
+
Setting 1 · constructionText-to-Scene
+
From an empty level and the content pack's asset catalog, the agent builds the scene a prompt describes. Many realizations are valid, so the
+evaluator judges requirements rather than matching a reference. 20 public cases.
+
case score = 0.2 × Detailed Alignment + 0.6 × Overview Alignment + 0.2 × Physical Safety
+
Setting 2 · editingImage-to-Scene
+
The agent gets a corrupted copy of a human-assembled scene and reference views of the original. It must restore every target actor (within
+5 cm, 5° and 5%) and change nothing else. 75 public cases: 25 indoor, 50 outdoor.
+
case score = 0.8 × Repair F1 + 0.2 × Physical Safety