---
title: Real2Sim Explorer
emoji: 🧱
colorFrom: blue
colorTo: indigo
sdk: static
app_file: index.html
pinned: false
short_description: Explore Real2Sim pipelines for robotics and Physical AI
---

# 🧱 Real2Sim Explorer

**Explore how real environments are reconstructed, structured and converted into simulation-ready worlds for robotics and Physical AI.**

The **Real2Sim Explorer** focuses on one of the most practical applications of World Compilation:

> **How do we turn a real environment into a simulation that preserves the information needed for learning, planning and control?**

A typical pipeline is:

```text
REAL WORLD
   ↓
Capture
   ↓
Reconstruction
   ↓
Object Decomposition
   ↓
Semantics
   ↓
Physical Properties
   ↓
Dynamics
   ↓
Simulation Conversion
   ↓
Domain Randomization
   ↓
Synthetic Data
   ↓
Validation
```

## Why Real2Sim matters

Real-world data is valuable but expensive, slow and difficult to scale.

Simulation is scalable but can diverge from reality.

Real2Sim connects the two.

The goal is not merely to create a visually similar 3D scene. A useful Real2Sim pipeline should preserve enough:

- geometry
- state
- object identity
- physical properties
- dynamics
- actions
- semantics
- uncertainty

to support downstream AI systems.

## Real2Sim as World Compilation

```text
REALITY
   ↓
WORLD COMPILER
   ↓
SIMULATION-READY WORLD
   ↓
SYNTHETIC EXPERIENCE
   ↓
WORLD MODEL / POLICY / PLANNER
```

## Included practical examples

- `examples/scene_schema.json` — a compact simulation-ready world schema
- `examples/domain_randomization.json` — example randomization ranges
- `examples/validation_config.json` — example Real2Sim validation protocol

## The reality gap

A reconstructed scene can be visually impressive while still being poor for robotics.

The reality gap may come from:

- inaccurate geometry
- wrong friction
- incorrect mass
- weak contact modeling
- missing objects
- unrealistic sensors
- timing differences
- imperfect dynamics

Therefore Real2Sim quality should ultimately be evaluated by **downstream transfer**, not appearance alone.

## Relationship to `worldcompiler`

`worldcompiler` covers the broader transformation:

```text
Reality → Structured Representation
```

Real2Sim is one important branch:

```text
Reality → Structured Representation → Simulation
```

## Cooperation

**Cooperation, research and ecosystem partnerships:**  
[agenten@magenta.de](mailto:agenten@magenta.de)

---

**Compile the real world into environments AI can safely explore.**