---
license: mit
tags:
- novel-view-synthesis
- lact
- test-time-training
- dl3dv
---

# test_lact

A LaCT (Large-Chunk Test-Time Training) novel-view-synthesis model, trained from scratch on DL3DV at 256×256 with **sparse inputs (2–6 context views)**.

The architecture is unmodified `LaCTLVSM` from [a1600012888/LaCT](https://github.com/a1600012888/LaCT) (commit `a648340`), using config `lact_l24_d768_ttt2x`: 24 blocks, dim 768, 312.3M parameters.

This is **not** the official LaCT checkpoint. The official scene models use 32–64 input views, batch 768 and far more compute. This run used a small sparse-view recipe, so the numbers below are much lower than the paper's.

## Files

| file | content |
|---|---|
| `model.safetensors` | Model weights at iteration 65,000, fp32. Keys match `LaCTLVSM` exactly. No optimizer state. |
| `configs/lact_l24_d768_ttt2x.yaml` | Model config, copied from the LaCT repo. |
| `configs/train_abl_lact_l24.yaml` | Training config used for this run. |
| `code/encoder_decoder_lact.py` | Wrapper used in training. It maps the training code's camera inputs (w2c + K) to LaCT's `c2w` / `fxfycxcy`. |
| `eval_results.json` | PSNR / SSIM / LPIPS every 5k iterations, 0–65k. |
| `logs/train.log` | Full training log. |
| `LICENSE-LaCT` | MIT license of the LaCT code. |

## Usage

```python
import sys, torch
from omegaconf import OmegaConf
from safetensors.torch import load_file

# git clone https://github.com/a1600012888/LaCT && git -C LaCT checkout a648340
sys.path.insert(0, "LaCT/lact_nvs")
from model import LaCTLVSM

cfg = OmegaConf.to_container(OmegaConf.load("configs/lact_l24_d768_ttt2x.yaml"), resolve=True)
model = LaCTLVSM(**cfg)
model.load_state_dict(load_file("model.safetensors"), strict=True)
model = model.cuda().eval()

# input_data = {"image": [B, N, 3, 256, 256] in [0, 1],
#               "fxfycxcy": [B, N, 4] in pixels at 256x256,
#               "c2w": [B, N, 4, 4]}
# target_data = {"fxfycxcy": [B, T, 4], "c2w": [B, T, 4, 4]}
with torch.no_grad(), torch.autocast("cuda", dtype=torch.bfloat16):
    rendered = model(input_data, target_data)  # [B, T, 3, 256, 256], values in [0, 1]
```

LaCT's `lact_ttt.py` uses `torch.compile`, so the first call compiles.

### Camera convention

Cameras use the OpenCV convention for `c2w`. The training poses were normalized as follows:

1. Re-center the scene on the first context camera.
2. Divide translations by `1.35 × max(distance of the context cameras from the first camera)`.

After this, the farthest context camera is at distance 1/1.35. Use the same normalization at inference.

## Training

- **Data:** DL3DV, `images_8` (480×270, distorted, as released), resized and cropped to 256×256.
- **Views per sample:** 2–6 context views (weights 2:8, 3:2, 4:8, 5:2, 6:8) plus 8 target views.
  - Context views are evenly spaced 10–20 frames apart, with jitter.
  - Targets are drawn from the context span, extended by one spacing on each side.
- **Optimization:**
  - Batch size 16. The training code batches by view count: with c context views, each step has floor(2 × 16 / c) scenes.
  - AdamW, lr 4e-4, betas (0.9, 0.95), weight decay 0.05.
  - Warmup 3k iterations, then cosine decay to 0 at 65k.
  - Grad clip 3.0.
- **Loss:** L2 + 0.5 × perceptual.
- **Hardware:** one GPU (B300), bf16 autocast. Activation checkpointing on the first 12 blocks; per-block `torch.compile`. About 24 hours.

## Results

The test set is 140 DL3DV benchmark scenes at 256×256, with fixed context/target view indices. "Nv" means N context views.

| iter | 2v PSNR | 4v PSNR | 6v PSNR | 2v / 4v / 6v SSIM | 2v / 4v / 6v LPIPS |
|---|---|---|---|---|---|
| 5k | 16.14 | 17.67 | 18.24 | 0.416 / 0.440 / 0.448 | 0.665 / 0.642 / 0.637 |
| 25k | 16.86 | 19.05 | 20.02 | 0.464 / 0.526 / 0.556 | 0.533 / 0.453 / 0.421 |
| 45k | 17.42 | 20.20 | 21.24 | 0.493 / 0.584 / 0.619 | 0.491 / 0.381 / 0.345 |
| **65k** | **17.65** | **20.68** | **21.78** | **0.504 / 0.607 / 0.644** | **0.475 / 0.358 / 0.320** |

`eval_results.json` has every 5k step.