Hugging Face's logo

Spaces:
worldcompiler
/
real2sim-explorer
Running

App Files Community
real2sim-explorer
File size: 2,577 Bytes
e05985f
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
---
title: Real2Sim Explorer
emoji: 🧱
colorFrom: blue
colorTo: indigo
sdk: static
app_file: index.html
pinned: false
short_description: Explore Real2Sim pipelines for robotics and Physical AI
---

# 🧱 Real2Sim Explorer

**Explore how real environments are reconstructed, structured and converted into simulation-ready worlds for robotics and Physical AI.**

The **Real2Sim Explorer** focuses on one of the most practical applications of World Compilation:

> **How do we turn a real environment into a simulation that preserves the information needed for learning, planning and control?**

A typical pipeline is:

```text
REAL WORLD
   ↓
Capture
   ↓
Reconstruction
   ↓
Object Decomposition
   ↓
Semantics
   ↓
Physical Properties
   ↓
Dynamics
   ↓
Simulation Conversion
   ↓
Domain Randomization
   ↓
Synthetic Data
   ↓
Validation
```

## Why Real2Sim matters

Real-world data is valuable but expensive, slow and difficult to scale.

Simulation is scalable but can diverge from reality.

Real2Sim connects the two.

The goal is not merely to create a visually similar 3D scene. A useful Real2Sim pipeline should preserve enough:

- geometry
- state
- object identity
- physical properties
- dynamics
- actions
- semantics
- uncertainty

to support downstream AI systems.

## Real2Sim as World Compilation

```text
REALITY
   ↓
WORLD COMPILER
   ↓
SIMULATION-READY WORLD
   ↓
SYNTHETIC EXPERIENCE
   ↓
WORLD MODEL / POLICY / PLANNER
```

## Included practical examples

- `examples/scene_schema.json` — a compact simulation-ready world schema
- `examples/domain_randomization.json` — example randomization ranges
- `examples/validation_config.json` — example Real2Sim validation protocol

## The reality gap

A reconstructed scene can be visually impressive while still being poor for robotics.

The reality gap may come from:

- inaccurate geometry
- wrong friction
- incorrect mass
- weak contact modeling
- missing objects
- unrealistic sensors
- timing differences
- imperfect dynamics

Therefore Real2Sim quality should ultimately be evaluated by **downstream transfer**, not appearance alone.

## Relationship to `worldcompiler`

`worldcompiler` covers the broader transformation:

```text
Reality → Structured Representation
```

Real2Sim is one important branch:

```text
Reality → Structured Representation → Simulation
```

## Cooperation

**Cooperation, research and ecosystem partnerships:**  
[agenten@magenta.de](mailto:agenten@magenta.de)

---

**Compile the real world into environments AI can safely explore.**