| # RecGen — sparse-structure denoiser, finetuned (70k) |
|
|
| An updated **sparse-structure (SS) denoiser** for RecGen. It is a drop-in |
| replacement for the `stereo_denoiser_ema0.9999_step0055000.pt` at the repository |
| root; the SLAT denoiser is **unchanged** and should still be taken from the root. |
|
|
| ``` |
| stereo_denoiser_ema0.9999_step0070000.pt this checkpoint |
| stereo_config.json required — architecture + pose representation |
| stereo_pose_stats.json required — pose normalization statistics |
| ``` |
|
|
| ## Usage |
|
|
| Pair it with the SLAT denoiser at the repository root: |
|
|
| ```python |
| from recgen_utils import load_pipeline |
| pipeline = load_pipeline( |
| checkpoint_slat="slat_denoiser_ema0.9999_step0075000.pt", # repo root, unchanged |
| checkpoint_sparse="sparse-structure-ft-70k/stereo_denoiser_ema0.9999_step0070000.pt", |
| ) |
| ``` |
|
|
| ## What changed |
|
|
| Finetuned from the released SS checkpoint on a preprocessed multi-view dataset |
| with SAM2-mixed masks and per-sample color jitter. |
|
|
| | | | |
| |---|---| |
| | initialized from | released `stereo_denoiser_ema0.9999_step0055000.pt` | |
| | optimizer | AdamW, lr 1e-5, weight decay 0 | |
| | augmentation | color jitter (brightness/contrast/saturation 0.4, hue 0.1) | |
| | masks | mixed source, 50% SAM2 | |
| | EMA | 0.9999 | |
| | architecture | unchanged — 1024 channels, 2 pose tokens, `6d_translation_scale`, 2 views @ 518px | |
|
|
| ## Evaluation |
|
|
| Full HomebrewedDB evaluation, 539 samples, **single-view conditioning**, with the |
| SLAT denoiser held fixed at the released checkpoint so that differences are |
| attributable to the sparse-structure stage alone. |
|
|
| | metric | released SS @55k | this checkpoint | | |
| |---|---|---|---| |
| | ADD-S recall @2% of diameter | 0.048 | **0.093** | ~1.9× | |
| | Chamfer distance, median (cm) | 0.410 | **0.403** | | |
| | Chamfer distance, 90th pct (cm) | 0.800 | **0.740** | −7.5% | |
| | Chamfer distance, pre-ICP median (cm) | 0.669 | **0.627** | −6.3% | |
|
|
| ADD-S recall at the conventional 10%-of-diameter threshold is saturated for both |
| checkpoints (0.950 vs 0.961) and does not separate them; the tighter 2% threshold |
| is where the difference appears. The improvement is concentrated in pose accuracy |
| and in the hard tail rather than in median reconstruction accuracy. |
|
|
| ## License |
|
|
| CC-BY-NC-4.0, as the rest of this repository. |
|
|
