BlenderWorld: Benchmarking Visual Agents for 3D Creation from Real-World References
Abstract
BlenderWorld is a benchmark for evaluating multimodal language models on realistic 3D creation tasks using rendered outputs and binary rubrics, revealing significant gaps in spatial coherence.
Recent advances in multimodal large language models (MLLMs) have enabled them to act as autonomous agents, creating and editing 3D models via code or direct software manipulation. In practical workflows, these models must infer object shapes, materials, spatial layouts, and lighting from diverse references while strictly following instructions to reconstruct, modify, or design 3D content. However, existing benchmarks provide limited coverage of such open-ended requests across real-world scenarios. To bridge this gap, we introduce BlenderWorld, the first comprehensive benchmark dedicated to evaluating frontier models on realistic 3D creation tasks. BlenderWorld comprises 223 tasks spanning architecture, products, natural environments, and scientific visualization, driven by text, image, video, and structured specification inputs. To rigorously assess these complex deliverables, we establish a unified evaluation protocol: we render the submitted Blender projects from multiple viewpoints and evaluate them against 2,651 task-specific binary rubrics alongside holistic visual quality scores. Benchmarking thirteen frontier models reveals that GPT-6-Astra achieves the highest overall fused score of 79.9, followed by Opus-5 (73.0) and GPT-5.6-Sol (67.7). Ultimately, our findings reveal a critical bottleneck: while current models can parse broad instructions, they still profoundly struggle with fine-grained 3D spatial coherence and accurate structural reproduction in Blender, leaving substantial headroom for future 3D-native agentics.
Community
🎉 Excited to share our latest work from the Qwen team! Visual agents can already build websites, design posters, and create slide decks. But how well can they handle 3D creation? This demands spatial reasoning, tool use, and iterative visual feedback—yet existing benchmarks offer limited coverage of realistic, open-ended 3D tasks.
🔥 Introducing BlenderWorld, the first comprehensive benchmark for systematically evaluating visual agents on real-world 3D creation tasks.
🔹 223 realistic tasks across 9 domains and 22 subcategories, including architecture, product design, nature, character animation, and scientific visualization. Inputs span text, images, videos, and structured specification files.
🔹 Deliverables are complete, editable Blender projects, going beyond a visually appealing render.
🔹 2,651 task-specific, fine-grained criteria cover 13 aspects, including geometry, spatial layout, materials, and functionality. An agent automatically selects and renders 8 key views: VLM-as-Judge evaluates the visual output, while Agent-as-Judge inspects the underlying project structure. Overall quality is also assessed across 8 dimensions, including realism and physical plausibility.
🔹 We evaluate 13 frontier models. GPT-6-Astra leads with a score of 79.9, but even the strongest model satisfies every fine-grained requirement on only 42.2% of tasks. The results are promising, with substantial room for improvement in interpreting multimodal references, meeting detailed requirements, and handling physical and spatial relationships.
🌍 We also built an interactive 3D gallery where you can explore, rotate, and compare creations from different models directly in your browser. Take a look!
📖 Paper: https://arxiv.org/abs/2610.23986
🚀 Project: https://blenderworld.github.io/
💻 GitHub: https://github.com/QwenLM/BlenderWorld
⭐ The code and dataset are publicly available! If you find BlenderWorld useful, consider giving us a star on GitHub. Thanks for your support!
Get this paper in your agent:
hf papers read 2610.23986 Don't have the latest CLI?
curl -LsSf https://hf.co/cli/install.sh | bash Models citing this paper 0
No model linking this paper
Datasets citing this paper 0
No dataset linking this paper
Spaces citing this paper 1
Collections including this paper 0
No Collection including this paper