File size: 1,513 Bytes
585c960 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 | # Architecture
```text
worldgen/worldgen_t2s.py prompt -> panorama -> depth -> scene
worldgen/worldgen_i2s.py image -> Fill panorama -> depth -> scene
worldgen/worldgen_i2s_prep.py 90-degree band cond/mask
worldgen/worldgen_da2.py ncnn Vulkan DA-2 bridge
tools/stable-diffusion.cpp FLUX.1-dev / FLUX.1-Fill-dev (C++/Vulkan)
tools/da2-ncnn/da2ncnn.exe DA-2 (C++/ncnn/Vulkan)
rust/ worldgen-vk (Rust): RGBD -> 3DGS/mesh PLY
da2/main.cpp ncnn runner source
patches/da2/ DA-2 export fixes
```
## FLUX stage
`stable-diffusion.cpp` with `SD_VULKAN=ON`. Diffusion, T5, CLIP and VAE all
run on Vulkan0 (the VAE uses `--vae-tiling --vae-tile-size 512x512` to avoid
a 9 s untiled decode).
## DA-2 stage
DA-2 is a DINOv2 ViT-L/14 + SphereViT head. The export path:
1. Apply `patches/da2/` (remove `quantile`, separate QKV projections).
2. Trace at 1x3x512x1024.
3. pnnx -> ncnn (0 custom layers).
4. `da2ncnn.exe` runs the graph on Vulkan.
The runner must be built with `-DNCNN_VULKAN` and call
`net.set_vulkan_device(0)`; otherwise ncnn silently uses CPU (8.7 s vs 3.05 s).
## Rust stage
`worldgen-vk both` converts pano+depth to:
- standard 17-float 3DGS PLY
- vertex-colored mesh PLY with a panorama UV seam fix
It is CPU geometry, 0.29 s for 524,288 splats / 1,046,528 triangles.
## Coordinate conventions
WorldGen uses the same normalized camera/center formula as the Python
reference; validation of center/scale/quaternion is ~1e-7.
|