Code4Scene: Benchmarking Coding Agents for Constructing and Editing 3D Scenes
190 Unreal Engine cases built from human-assembled scenes. Coding agents write and run code that builds a scene from text, or
repairs one from reference images, and Code4Scene scores the engine-native scene they save, not their code or a rendered view.
The leaderboard, one scene through the evaluator, two scenes compared, and what the benchmark finds, in 90 seconds.
Overview
Score the scene, not the code or a render
Coding agents can now operate 3D engines: they write and run code, inspect the result and revise the scene. Code4Scene is a
benchmark of 190 Unreal Engine cases built from human-assembled scenes that evaluates them on two complementary settings under a shared execution
interface. Construction starts from an empty level and an open-ended text specification. Editing gives the agent a corrupted copy of a
scene and reference images of the original, and asks it to restore the intended state while leaving everything else untouched.
Rather than scoring code or rendered views, Code4Scene evaluates the engine-native scene the agent saved, on task fulfilment, artifact integrity
and static physical validity, and compares edits against withheld ground truth. Across 14 agent configurations, construction and editing rank the
agents differently, spatial relations remain the weakest requirement family, and a third of complete repairs still change the surrounding scene.
1 · InputPrompt or referenceAn open-ended scene description, or a corrupted scene with reference images of the original.
2 · AgentCoding agentA frontier or open-weight model in its own CLI harness, with the Unreal Engine editor as a tool.
3 · CodeCode in the engineIt writes and runs Python against the editor: spawns, moves and edits actors, inspects, revises.
4 · SceneEngine-native sceneThe saved level (.umap): every actor, transform and material, as the engine holds it.
5 · EvaluatorScored in the engineRequirements located and judged, overview views judged, physics measured, edits diffed against the withheld ground truth.
Setting 1 · construction
Text-to-Scene
From an empty level and the content pack's asset catalog, the agent builds the scene a prompt describes. Many realizations are valid, so the
evaluator judges requirements rather than matching a reference. 20 public cases.
The agent gets a corrupted copy of a human-assembled scene and reference views of the original. It must restore every target actor (within
5 cm, 5° and 5%) and change nothing else. 75 public cases: 25 indoor, 50 outdoor.
case score = 0.8 × Repair F1 + 0.2 × Physical Safety
Leaderboard
14 coding agents on the public set
Scores are from the paper's Table 2. Pick a setting; the chart and the table follow.
Score against cost
Each agent's score against what one case cost it. The line is the frontier: no agent left of it scores higher. Hover a point for its numbers.
OpenAIGoogleAnthropicMetaZ.aiAlibabaxAIThinking MachinesDeepSeekon the frontier
Cost is the mean reported cost of one run, estimated from its token usage at the provider's list price (paper Figure 1b; provisional).
Ranking
Overall = ½ Text-to-Scene + ½ Image-to-Scene over the 95 public cases; its cost is weighted the same way.
75 public cases (25 indoor, 50 outdoor). Case score = 0.8 × Repair F1 + 0.2 × Physical. Repair F1 counts target actors restored to the withheld ground truth.
Text-to-Scene (left) and Image-to-Scene (right) score per agent, in overall order (paper Table 2). Astra and Gemini tie at 0.619 overall.
Takeaway 2
Spatial Composition remains the weakest requirement family for all 14 agents.
Errors persist even when the required objects are present: generating the right objects does not ensure that their relationships satisfy the specification.
Each row is one agent's four family scores; the orange dot, Spatial Composition, is the lowest in every row (paper Figure 14b).
The objects are there; the relation is not
✓ slab roof 0.95✓ stone coping 0.82✗ slab roof enclosed by the coping 0.78
The judge's own frame of GPT-5.6 Sol on Egyptian Temple. It finds the slab roof and the stone coping, but not the roof enclosed by the coping: the coping is separate low wall segments with visible gaps (ringed).
Mismatch rate when the required objects are present
Across the benchmark (paper Figure 6a).
Takeaway 3
Editing ability varies substantially across repair types.
Astra performs best on Transform repairs, while Gemini is stronger on Lifecycle and Layout operations.
Mean Image-to-Scene score by repair type for four agents; the outlined cell is the best of the four in its row (paper Figure 6b).
Takeaway 4
Recovering the target does not guarantee precise editing.
Across all 14 configurations, 35.8% of complete recoveries still contain unintended changes to the surrounding scene.
@article{code4scene2026,
title = {Code4Scene: Benchmarking Coding Agents for Constructing and Editing 3D Scenes},
author = {Ye, Xiaokang and Mantri, Siddhant Hitesh and Chen, Zimeng and Zhang, Edward and Zheng, Zhaoxu and Li, Yuanheng and Chen, Yizhao and Huang, Tianyang and Qin, Lianhui},
year = {2026}
}
