test_lact

A LaCT (Large-Chunk Test-Time Training) novel-view-synthesis model, trained from scratch on DL3DV at 256×256 with sparse inputs (2–6 context views).

The architecture is unmodified LaCTLVSM from a1600012888/LaCT (commit a648340), using config lact_l24_d768_ttt2x: 24 blocks, dim 768, 312.3M parameters.

This is not the official LaCT checkpoint. The official scene models use 32–64 input views, batch 768 and far more compute. This run used a small sparse-view recipe, so the numbers below are much lower than the paper's.

Files

file	content
model.safetensors	Model weights at iteration 65,000, fp32. Keys match LaCTLVSM exactly. No optimizer state.
configs/lact_l24_d768_ttt2x.yaml	Model config, copied from the LaCT repo.
configs/train_abl_lact_l24.yaml	Training config used for this run.
code/encoder_decoder_lact.py	Wrapper used in training. It maps the training code's camera inputs (w2c + K) to LaCT's c2w / fxfycxcy.
eval_results.json	PSNR / SSIM / LPIPS every 5k iterations, 0–65k.
logs/train.log	Full training log.
LICENSE-LaCT	MIT license of the LaCT code.

Usage

import sys, torch
from omegaconf import OmegaConf
from safetensors.torch import load_file

# git clone https://github.com/a1600012888/LaCT && git -C LaCT checkout a648340
sys.path.insert(0, "LaCT/lact_nvs")
from model import LaCTLVSM

cfg = OmegaConf.to_container(OmegaConf.load("configs/lact_l24_d768_ttt2x.yaml"), resolve=True)
model = LaCTLVSM(**cfg)
model.load_state_dict(load_file("model.safetensors"), strict=True)
model = model.cuda().eval()

# input_data = {"image": [B, N, 3, 256, 256] in [0, 1],
#               "fxfycxcy": [B, N, 4] in pixels at 256x256,
#               "c2w": [B, N, 4, 4]}
# target_data = {"fxfycxcy": [B, T, 4], "c2w": [B, T, 4, 4]}
with torch.no_grad(), torch.autocast("cuda", dtype=torch.bfloat16):
    rendered = model(input_data, target_data)  # [B, T, 3, 256, 256], values in [0, 1]

LaCT's lact_ttt.py uses torch.compile, so the first call compiles.

Camera convention

Cameras use the OpenCV convention for c2w. The training poses were normalized as follows:

1. Re-center the scene on the first context camera.
2. Divide translations by 1.35 × max(distance of the context cameras from the first camera).

After this, the farthest context camera is at distance 1/1.35. Use the same normalization at inference.

Training

• Data: DL3DV, images_8 (480×270, distorted, as released), resized and cropped to 256×256.
• Views per sample: 2–6 context views (weights 2:8, 3:2, 4:8, 5:2, 6:8) plus 8 target views.
  • Context views are evenly spaced 10–20 frames apart, with jitter.
  • Targets are drawn from the context span, extended by one spacing on each side.
• Optimization:
  • Batch size 16. The training code batches by view count: with c context views, each step has floor(2 × 16 / c) scenes.
  • AdamW, lr 4e-4, betas (0.9, 0.95), weight decay 0.05.
  • Warmup 3k iterations, then cosine decay to 0 at 65k.
  • Grad clip 3.0.
• Loss: L2 + 0.5 × perceptual.
• Hardware: one GPU (B300), bf16 autocast. Activation checkpointing on the first 12 blocks; per-block torch.compile. About 24 hours.

Results

The test set is 140 DL3DV benchmark scenes at 256×256, with fixed context/target view indices. "Nv" means N context views.

iter	2v PSNR	4v PSNR	6v PSNR	2v / 4v / 6v SSIM	2v / 4v / 6v LPIPS
5k	16.14	17.67	18.24	0.416 / 0.440 / 0.448	0.665 / 0.642 / 0.637
25k	16.86	19.05	20.02	0.464 / 0.526 / 0.556	0.533 / 0.453 / 0.421
45k	17.42	20.20	21.24	0.493 / 0.584 / 0.619	0.491 / 0.381 / 0.345
65k	17.65	20.68	21.78	0.504 / 0.607 / 0.644	0.475 / 0.358 / 0.320

eval_results.json has every 5k step.