Spaces:
Running
Running
| title: Real2Sim Explorer | |
| emoji: 🧱 | |
| colorFrom: blue | |
| colorTo: indigo | |
| sdk: static | |
| app_file: index.html | |
| pinned: false | |
| short_description: Explore Real2Sim pipelines for robotics and Physical AI | |
| # 🧱 Real2Sim Explorer | |
| **Explore how real environments are reconstructed, structured and converted into simulation-ready worlds for robotics and Physical AI.** | |
| The **Real2Sim Explorer** focuses on one of the most practical applications of World Compilation: | |
| > **How do we turn a real environment into a simulation that preserves the information needed for learning, planning and control?** | |
| A typical pipeline is: | |
| ```text | |
| REAL WORLD | |
| ↓ | |
| Capture | |
| ↓ | |
| Reconstruction | |
| ↓ | |
| Object Decomposition | |
| ↓ | |
| Semantics | |
| ↓ | |
| Physical Properties | |
| ↓ | |
| Dynamics | |
| ↓ | |
| Simulation Conversion | |
| ↓ | |
| Domain Randomization | |
| ↓ | |
| Synthetic Data | |
| ↓ | |
| Validation | |
| ``` | |
| ## Why Real2Sim matters | |
| Real-world data is valuable but expensive, slow and difficult to scale. | |
| Simulation is scalable but can diverge from reality. | |
| Real2Sim connects the two. | |
| The goal is not merely to create a visually similar 3D scene. A useful Real2Sim pipeline should preserve enough: | |
| - geometry | |
| - state | |
| - object identity | |
| - physical properties | |
| - dynamics | |
| - actions | |
| - semantics | |
| - uncertainty | |
| to support downstream AI systems. | |
| ## Real2Sim as World Compilation | |
| ```text | |
| REALITY | |
| ↓ | |
| WORLD COMPILER | |
| ↓ | |
| SIMULATION-READY WORLD | |
| ↓ | |
| SYNTHETIC EXPERIENCE | |
| ↓ | |
| WORLD MODEL / POLICY / PLANNER | |
| ``` | |
| ## Included practical examples | |
| - `examples/scene_schema.json` — a compact simulation-ready world schema | |
| - `examples/domain_randomization.json` — example randomization ranges | |
| - `examples/validation_config.json` — example Real2Sim validation protocol | |
| ## The reality gap | |
| A reconstructed scene can be visually impressive while still being poor for robotics. | |
| The reality gap may come from: | |
| - inaccurate geometry | |
| - wrong friction | |
| - incorrect mass | |
| - weak contact modeling | |
| - missing objects | |
| - unrealistic sensors | |
| - timing differences | |
| - imperfect dynamics | |
| Therefore Real2Sim quality should ultimately be evaluated by **downstream transfer**, not appearance alone. | |
| ## Relationship to `worldcompiler` | |
| `worldcompiler` covers the broader transformation: | |
| ```text | |
| Reality → Structured Representation | |
| ``` | |
| Real2Sim is one important branch: | |
| ```text | |
| Reality → Structured Representation → Simulation | |
| ``` | |
| ## Cooperation | |
| **Cooperation, research and ecosystem partnerships:** | |
| [agenten@magenta.de](mailto:agenten@magenta.de) | |
| --- | |
| **Compile the real world into environments AI can safely explore.** | |