RADIANT
RADIANT: Object-Centric Feed-Forward Global Illumination
RADIANT (RADiance-based Inter-Actor Neural Transformer) renders global illumination of unseen 3D scenes in a single forward pass from point inputs. Each object is represented by 32 surface-patch tokens, so the attention sequence length grows with the number of objects rather than with the number of polygons. In a single network, an inter-object scene transformer models radiative exchange and a view decoder renders the HDR image for a query camera. Invariant form-factor attention adds the distances and facing angles between surface patches to the query-key dot products as SE(3)-invariant inner products, weighted by gains predicted from each object's area and materials. For primary visibility without mesh connectivity, camera rays cross-attend the scene tokens, guided by a depth-tested splat of the point inputs.
This repository holds the trained model, 205.9 M parameters:
| Weights | Trained on | Configuration |
|---|---|---|
radiant_512.pth |
RADIANT-Objaverse, 512 × 512 | configs/radiant_512_300k-600k.yaml |
The model was trained from scratch for 600k steps with a global batch of 24: a 100k-step cosine schedule, followed by continuations on
the tails of 200k-, 300k- and 600k-step cosine schedules, one configuration each in configs/. The weights are those of step 596k,
the checkpoint with the highest per-image PSNR on the held-out split.
Results
Held-out split of RADIANT-Objaverse, 1,028 scenes and 4,112 images at 512 × 512:
| PSNR per image (dB) | PSNR pooled (dB) | LPIPS | â„“1 (log radiance) |
|---|---|---|---|
| 32.22 | 27.65 | 0.042 | 0.012 |
PSNR is computed after Reinhard tone mapping and sRGB conversion, per image and pooled over all pixels. LPIPS uses the AlexNet backbone on the same tone-mapped images. â„“1 is the mean absolute error of log(1 + L).
Usage
hf download frankzydou/RADIANT radiant_512.pth --local-dir checkpoints
import torch
ckpt = torch.load("checkpoints/radiant_512.pth", map_location="cpu", weights_only=True)
state_dict = ckpt["model_state_dict"] # 450 float32 tensors
step = ckpt["global_step"] # 596000
SHA-256 of radiant_512.pth: ac80109ae0c21ef9c16172ee9b68e77a1eed91cf2150369a137285ac194f8db2