Hugging Face's logo

jhoonjwa
/
bert_simpson

Model card Files
xet
 Community
bert_simpson / forgebench /README.md
HI
claude fable 5.1
Squash history to reclaim storage
47e6e42
preview code
|
Raw History Blame Contribute Delete
24.1 kB

FORGE3D: final evaluation protocol, evaluator, inference code, and what still has to be generated

This folder (forgebench/ of Ronaldo-GOAT/bert_simpson) contains everything needed to (1) generate meshes for FORGE3D (ours) and the baselines on the three benchmarks and (2) score them with the final evaluator code/eval/eval_final.py. Every number in section 7 is written by that script (summary.json), never by hand.

Contents:

path what
code/eval/eval_final.py final evaluator (FORGE3DBench-150, OmniObject3D-300, Toys4K-300); writes per-object results.json, results.csv and summary.json
code/eval/appeval/ renderer, metrics and geometry used by the evaluator. render.py has the far-clip fix (section 2.5). geometry.py adds extent-relative keys and leaves the original keys unchanged
code/eval/heldout_index/ held-out (novel-view) camera lists: fb150.json, omni300.json, omni300_bad_frame_objects.txt
code/eval/heldout/ prepare_heldout.py (how the held-out views were materialised) and eval_heldout.py (reference implementation; eval_final.masked_view equals it)
code/eval/tests/ unit tests (section 8)
code/eval/align_v2.py, cammap_pixal3d.py, cammap_fb150_recenter.py placement of baseline meshes into the GT frame (section 3)
code/ours/ FORGE3D inference. Final path: ours_combined_cell_batched.sh
code/baselines/ baseline drivers (see code/SETUP.md for the official repos, commits and envs)
data/fb150_heldout.tar (new) FORGE3DBench-150 held-out views: fb150_heldout/{inputs,renders} (2867 views; NV uses the 2798 with visibility >= 0.20)
data/omni3d300_heldout_saved.tar (new) OmniObject3D-300 held-out saved views top, top2, bottom, bottom2 -> omni3d300_rand/{inputs,renders}; extract next to omni3d300_4v_inputs.tar
todo_final/ what must be generated: missing_<method>_<dataset>_<setting>.{json,txt} (+ .selection.json subsets) and SUMMARY.json
results/ summary.json and results.csv of every cell scored with the final protocol
MANIFEST.sha256 sha256 of every file in this folder
todo/, build_rest658.py, views_4v.json the other 658 FORGE3DBench objects (unchanged from the previous upload, see section 9)

Tars that were uploaded before and are still needed: data/fb150_exp_4v.tar (FB150 inputs, cameras, GT meshes and pointmaps; the 1v setting uses its front view, which is byte-identical to the old exp_1v), data/omni3d300_4v_inputs.tar, data/omni3d300_gt_canon.tar. The Toys4K-300 exp dir (toys4k300_rand: 8 views, cameras, GT) is not in this folder.

1. Datasets

benchmark objects source of images GT mesh frame input views held-out (NV) views
FORGE3DBench-150 (FB150) 150 Multi-view scene images from the PSL MVScenes benchmark: Factory (78) and Park (72) environments; ground, robot-table and table anchors; 120 scenes, 24 cameras per scene; lit, cluttered, with occlusion. Inputs are crops of the scene frame. The alpha channel is the object's visible (modal) mask, i.e. the instance mask with depth > 0 canonical mesh, centred, max extent 0.9 1v = the front camera (visibility >= 0.40). 4v = front + 3 random other cameras of the same scene with visibility >= 0.40 (random.Random(f"{obj}/4v/0")), tags front, side, back, oside the other scene cameras (20 of 24) with visibility >= 0.20 and >= 400 visible px: 2798 views, 13-20 per object (mean 18.7)
OmniObject3D-300 300 official OmniObject3D Blender renders (100 per object) of real scans, no occlusion, alpha = object alpha scan mesh, canonicalised, max extent 1.0 4v = 4 random official renders front, side, back, oside (selection.json -> view_idx) 4 saved random renders top, top2, bottom, bottom2 (1200 views). Optional pool = the remaining 92 official renders per object (--nv-set all)
Toys4K-300 300 our Blender renders of textured Toys4K meshes max extent 1.0, up = +Z 4v = front, side, back, oside (front + 3 random). 1v = front original protocol: 24-view orbit (section 2.3)

Visibility on FB150 = modal / amodal pixel area of the object in that camera (views_4v.json -> all_cam_visibility). It is copied into heldout_index/fb150.json as vis.

Camera convention (all datasets): renders/<obj>_<tag>.npz stores fx, fy, cx, cy (pixels, at res x res), c2w_cv (OpenCV camera-to-world: x right, y down, z forward, in the GT canonical frame), bbox = (y0, y1, x0, x1) (the crop of the full frame that inputs/<obj>_<tag>.png holds) and res (FB150 1440, Omni 800, Toys 768).

2. Evaluation protocol (final)

2.1 FORGE3DBench-150 and OmniObject3D-300: masked GT-image protocol (IV and NV)

One code path (eval_final.masked_view, identical to the held-out evaluator eval_heldout.heldout_view) for both input views and held-out views:

  1. Render the predicted mesh (already in the GT canonical frame, section 3) with the unlit nvdiffrast renderer (appeval/render.py: albedo or texture only, no lighting) at the dataset camera (K, c2w_cv, res), then crop it with bbox.
  2. Reference = the dataset's own GT image of that camera (inputs/<obj>_<tag>.png: the real scene-frame crop on FB150, the official render on Omni). The mask M = alpha > 0.5. On FB150 M is the visible mask, so occluders and occluded object parts are never scored. On Omni M is the object alpha.
  3. pred_alpha *= M, ref_alpha := M, both composited over white. Pixels outside M are white in both images and are ignored. Pixels in M that the prediction does not cover are compared as white vs GT and are penalised. coverage = fraction of M covered by the prediction (diagnostic).
  4. Metrics per view: LPIPS (AlexNet, primary), SSIM (gaussian, win 11), PSNR, CLIP-ViT-B/32 cosine (appeval/appearance.py, unchanged). Mean over the views of an object, then mean over objects.
  • IV = the input cameras: 4v -> front, side, back, oside, 1v -> front.
  • NV = held-out cameras never given to any model. The set is the same for 1v and 4v (it never contains the 4 input cameras of the 4v setting):
    • FB150: heldout_index/fb150.json, only views with vis >= 0.20 (--min-vis 0.2, the default). Views below 20% visibility are dropped.
    • Omni: top, top2, bottom, bottom2 (--nv-set saved, the default). The 92-view pool is optional (--nv-set all).
  • Omni bad-frame objects: 14 objects whose GT canon.glb is not in the render-camera frame (heldout_index/omni300_bad_frame_objects.txt). Seven are frame_offset (2.5-7.5 cm off); seven are broken_camera: insole_001-006 and ornaments_010, whose stored cameras sit at the origin. --exclude-bad-frame (used for the headline) drops them from means. They are still scored, listed under excluded_objects, and means_including_excluded is also written.
  • The protocol ceiling is NOT perfect. The GT images are lit (scene lighting, shading, tone mapping) and the render is unlit, so the GT mesh scored as the prediction does not reach LPIPS 0. The ceilings (GT mesh vs GT image, same code) are in section 7.4. A diagnostic --ref gtrender replaces the GT image with the GT-mesh render (same camera and mask). It is not the protocol and is only used for the pipeline-identity test.

2.2 Geometry (all datasets, unchanged)

appeval/geometry.py on the placed prediction vs the GT canonical mesh, with no alignment inside the evaluator. It samples 100k surface points per mesh (pytorch3d, seed 0) and reports:

  • CD-L1 = 0.5 (mean p->g + mean g->p) of the unsquared Euclidean NN distance;
  • F@tau with tau in {0.01, 0.02, 0.05} in GT-frame units (headline F@0.01);
  • normal consistency;
  • volume IoU @128^3.

It also reports cd_l1_rel and f01_rel (distances divided by the GT max extent), because the FB150 GT has extent 0.9 and Toys/Omni 1.0.

2.3 Toys4K-300: ORIGINAL protocol (unchanged)

Toys4K keeps appeval/evaluate_appforce.eval_object exactly:

  • IV = the prediction rendered at the input cameras vs the input crop, unmasked (object alpha of both);
  • NV = 24-view orbit (8 azimuths x elevations {-30, 0, 30}, radius 2.6, fov 40, 512^2), prediction render vs GT-mesh render;
  • geometry as in 2.2.

eval_final.py --dataset toys300 only calls eval_object and writes the same summary format. Its numbers equal evaluate_appforce.py (test in section 8). The far-clip fix does not affect Toys, because all its cameras are 2.6 units from the origin.

2.4 Mesh placement before scoring (per method)

method FB150 Omni / Toys
FORGE3D (ours) none (generated in the GT frame) none
ReconViaGen, Amodal3R, Hunyuan3D-2mv, Cupid align_v2.py: GT-extent scale (0.9 FB150 / 1.0 others), 24 cube + 2000 random rotations scored by 64^3 voxel F1, rigid ICP on the top 8, input-view silhouette tie-break same
Pixal3D 4v batch_pixal3d_mv_recenter.py --center origin + cammap_fb150_recenter.py (camera-only, no ICP or scale fit): X = C'_0 inv(F(d0)) A G + c batch_pixal3d_mv.py + cammap_pixal3d.py (camera-only): X = C0 inv(F(d)) A G, C0 = c2w_cv diag(1,-1,-1,1), d = |C0 t|
Pixal3D 1v single-view pipeline + camera-only cammap (pixal3dsv_1v_cammap) Toys 1v: MV pipeline with 1 view + cammap_pixal3d.py

In the formulas, G is the exported glb vertices and A = [[-1,0,0],[0,0,1],[0,1,0]] undoes the glb export axes. F(d) is the official canonical front camera (Blender c2w, camera at (0,-d,0)). For the re-centred FB150 run, C'_0 is the virtual (re-centred) front camera in the shifted GT frame and c is the object centre. Both are written by the driver to <obj>.cams.json.

2.5 Fixes relative to the pre-final code

  • Renderer far-clip (appeval/render.py, render_input_view): far was a fixed 20 units. FB150 tiny objects have cameras 19.2-21.6 units from the origin, so they rendered empty. The far plane is now max(20, |camera centre| + 10). This is identical for every camera within 10 units (all Toys/Omni). Example, GT mesh of PaGr_s000017_smallredcube0, front camera at 20.7 units: IV LPIPS 0.746 -> 0.075.
  • Masked IV on FB150: the old IV compared the unmasked prediction silhouette with the visible-mask crop, which penalised correctly predicted occluded parts. IV now uses the visible mask, exactly like NV.
  • Held-out NV replaces the 24-orbit GT-render NV on FB150 and Omni (the GT image is the reference).
  • The summary is produced by the evaluator (--summarize merges shards). When duplicate rows exist, the last complete row wins and conflicts are listed. Missing, failed and excluded objects are listed explicitly.

3. Running the evaluator

Environment: the mv-sam3d env (torch, nvdiffrast, pytorch3d, lpips, open_clip, torchmetrics, scikit-image, trimesh). One process needs about 2-3 GB of GPU memory. Rough speed on an A100: FB150 about 10 s/object (4 IV + about 19 NV views at 1440^2 + geometry), Omni and Toys about 4-8 s/object. Set --exp / --heldout-root to your extracted data (the defaults are the original server's paths).

cd forgebench/code/eval
# FB150 4v (placed meshes <obj>.glb in $M), 8 shards on 8 GPUs, then merge + summary
for i in 0 1 2 3 4 5 6 7; do
  CUDA_VISIBLE_DEVICES=$i python eval_final.py --dataset fb150 --setting 4v --meshes $M --method ours_fb150_4v \
     --exp $FB --heldout-root $DATA/fb150_heldout --min-vis 0.2 --shard $i --nshards 8 --out $OUT/ours_fb150_4v/s$i &
done; wait
python eval_final.py --dataset fb150 --setting 4v --method ours_fb150_4v --exp $FB --out $OUT/ours_fb150_4v \
     --summarize $OUT/ours_fb150_4v/s*          # -> results.json, results.csv, summary.json
# FB150 1v: same with --setting 1v (IV = front only; NV = same held-out set)
# Omni 4v (headline excludes the 14 bad-frame objects; they are still scored and listed)
python eval_final.py --dataset omni300 --setting 4v --meshes $M --exp $OMNI --nv-set saved --exclude-bad-frame --out ...
# Toys4K (original protocol)
python eval_final.py --dataset toys300 --setting 4v --meshes $M --exp $TOYS --out ...
# diagnostics: GT mesh as the prediction (protocol ceiling) / GT-render reference (identity test)
python eval_final.py --dataset fb150 --setting 4v --pred-is-gt --out ...  [--ref gtrender]

summary.json keys: means (headline, exclusions applied: iv, nv {lpips, ssim, psnr, clip, coverage, n_views_total, n_objects}, geometry {cd_l1, f01, f02, f05, normal_consistency, vol_iou, cd_l1_rel, f01_rel}), means_including_excluded, n_scored, n_expected, missing_objects, failed_objects, excluded_objects, duplicate_conflicts_last_wins, the protocol flags and the evaluator's sha256. A cell is final only if n_scored == n_expected (minus listed exclusions).

4. Reproducing FORGE3D (ours)

Checkpoints on this Hub repo:

  • SS-flow: ckpt/ssflow_seedcond_v3_20260916/step_0080000.pt (raw weights)
  • SLAT-flow: ckpt/slatflow_mv8_200k_4gpu_nockpt_4567_from21k_20260923/step_0032000.pt

Seed 42. Code: code/ours/ + the training repo (migrator/code/mv-sam3d-for-6d-v2-ssflow/ on this Hub, providing mvsam3d, tools/val_daemon.py, configs/train_slatflow_prod.yaml) + the SAM3D env (code/SETUP.md).

Final path = batched (code/ours/ours_combined_cell_batched.sh):

  1. pass 1 dump_seed.py (pointmap seed);
  2. pass 2 ssflow_coords_batched.py --batch $SSB (SS-flow ODE);
  3. pass 3 finish_cache.py (stage-1 cache);
  4. stage 2 val_daemon.py --once with mkconfig_batched.py (SLAT-flow, batch S2B, decode to glb).

Keep **S2B <= 4**: noisy seeds OOM the batched decoder at 8/16. Meshes land in $SCRATCH/runs/step_0032000/<DS>/seed/glb/<obj>.glb in the GT frame (no placement needed). Use a fresh $SCRATCH per cell: stage 2 decodes every object in the cache.

export CKPT=<dir with the two ckpt folders> SCRATCH=<work dir> GPUS="0 1 2 3 4 5 6 7"
export R=<data/env root> REPO=<.../mv-sam3d-for-6d-v2> PY=<mv-sam3d env python>     # see the script header
# FB150 4v (1v: NV=1, same exp dir; the 1v inputs are the front view of exp_4v)
OBJFILE=todo_final/missing_ours_fb150_4v.txt SSB=16 S2B=4 bash code/ours/ours_combined_cell_batched.sh "$GPUS" $FB fb150 4 \
  $CKPT/ssflow_seedcond_v3_20260916/step_0080000.pt \
  $CKPT/slatflow_mv8_200k_4gpu_nockpt_4567_from21k_20260923/step_0032000.pt $SCRATCH/ours_fb150_4v ours_fb150_4v
# Toys4K-300: EXP=$TOYS DS=toys4k300_rand NV=4|1 ; Omni-300: EXP=$OMNI DS=omni3d300_rand NV=4

Batched vs unbatched: code/ours/batch_equivalence.md (differences are at the level of 1 occupancy voxel, 1e-3 F@0.01).

5. Baselines

Drivers and exact argument lists: code/SETUP.md and code/baselines/commands_reference.py. All drivers skip existing outputs and take --shard i --nshards n; seed 42 everywhere. Hunyuan3D-2mv on the random 4v sets needs the slot-permuted inputs (code/baselines/omni_reference/hy_slots.py <ds> -> <ds>_rand_hyslots: side->right, back->back, oside->left). None of the previously uploaded baseline drivers changed; the only differences on disk are path constants. New files: baselines/pixal3d_ours/{batch_pixal3d_mv_recenter.py, recenter.py} and eval/cammap_fb150_recenter.py.

6. What needs to be generated (state of the original server, scanned 2026-09-26 04:56 KST)

Rule: baselines generate only the missing objects. FORGE3D (ours) regenerates every object with the final batched code. Per cell, todo_final/missing_<method>_<dataset>_<setting>.json holds the object list, the count that exists now, the placement and the exact command. The .txt file is the plain object list. RVG and Cupid take --selection, so they get missing_*.selection.json. Variables: $FB = exp_4v of FB150; $OMNI = omni3d300_rand; $OMNI_HY = omni3d300_rand_hyslots; $TOYS = toys4k300_rand; $GEN = output root; $G = GPU; $i/$N = shard index/count.

method Toys4K-300 4v Toys4K-300 1v Omni3D-300 4v FB150 4v FB150 1v
FORGE3D (ours) 300 (all; 300 exist) 300 (all; 288 exist) 300 (all; 300 exist) 150 (all; 150 exist) 150 (all; 150 exist)
ReconViaGen 0 0 182 (118 exist) 0 0
Amodal3R 0 0 177 (123 exist) 0 0
Hunyuan3D-2mv 0 0 210 (90 exist) 0 0
Pixal3D 0 0 147 (153 exist) 150 (all, re-centre driver) 0
Cupid 0 0 229 (71 exist) 0 0

Generation status on the original server at the time of the scan:

  • The Omni 4v baseline schedulers stopped at about 00:50 KST on 09-26 and nothing is running. The 4v baseline meshes that exist are not yet placed or scored.
  • FB150 Pixal3D 4v (re-centre, --center origin, 3 shards on GPUs 0-2) is queued behind a Toys 1v Pixal3D job that is running, with 217/300 done (after_toys1v.sh).

After generation:

  1. Place the baselines: align_v2.py for RVG / Amodal3R / Hy3D-2mv / Cupid, the camera maps for Pixal3D.
  2. Score every cell with eval_final.py (section 3).
  3. Replace the "pending rescore" rows in section 7.

7. Results (final protocol only)

Each row lists how it was produced. Rows not produced with the final protocol are marked pending rescore. Their old numbers are not reported here: the old FB150/Omni IV/NV used the unmasked IV, the 24-orbit NV and the far=20 renderer.

7.1 Toys4K-300 (original protocol, n = 300 unless noted)

Produced by appeval/evaluate_appforce.py (= the Toys path of eval_final.py) on the placed meshes; summary.json written by eval_final.py --summarize over those result files (results/toys4k300/<cell>/). The FORGE3D rows come from the unbatched run (SS 80k + SLAT 32k) and will be replaced by the batched regeneration.

Toys4K-300 4v

method n IV LPIPS IV SSIM IV PSNR NV LPIPS NV SSIM NV PSNR CD-L1 F@0.01 NC placement / source
FORGE3D (ours) 300 0.0875 0.9157 22.39 0.0512 0.9426 25.07 0.0085 0.8502 0.8969 none
ReconViaGen 300 0.1508 0.8577 18.14 0.0780 0.9246 21.12 0.0147 0.5161 0.8448 align_v2
Amodal3R 300 0.2105 0.8255 16.01 0.1077 0.9095 18.78 0.0221 0.4071 0.8034 align_v2
Hunyuan3D-2mv 300 0.1687 0.8678 17.94 0.0851 0.9203 20.88 0.0165 0.5139 0.8391 align_v2
Pixal3D 300 0.1141 0.8878 20.42 0.0569 0.9395 23.82 0.0095 0.7494 0.8761 cammap_pixal3d
Cupid 300 0.1899 0.8610 17.39 0.0990 0.9178 20.44 0.0227 0.4808 0.7934 align_v2

Toys4K-300 1v

method n IV LPIPS IV SSIM IV PSNR NV LPIPS NV SSIM NV PSNR CD-L1 F@0.01 NC placement / source
FORGE3D (ours) 288 0.1067 0.8954 20.60 0.0920 0.9268 20.85 0.0253 0.5881 0.7893 none pending: 12 missing (n=288/300)
ReconViaGen 300 0.2003 0.8338 16.69 0.1136 0.9136 19.01 0.0270 0.3752 0.7718 align_v2
Amodal3R 300 0.2260 0.8169 15.98 0.1237 0.9040 17.97 0.0289 0.3498 0.7661 align_v2 (2 retry rows replaced)
Hunyuan3D-2mv 300 0.1898 0.8572 17.81 0.1055 0.9155 19.87 0.0245 0.4354 0.7971 align_v2
Pixal3D 300 0.1081 0.8959 20.83 0.1361 0.9091 18.77 0.0444 0.3273 0.7247 cammap_pixal3d (MV weights, 1 view)
Cupid 300 0.1798 0.8633 17.73 0.1047 0.9173 19.90 0.0237 0.4447 0.7809 align_v2

7.2 FORGE3DBench-150 (masked GT-image protocol, IV + NV with visibility >= 0.20)

All cells are pending rescore. FB150 meshes exist for every method except Pixal3D 4v, which is being regenerated. Commands are in section 3.

method 4v 1v
FORGE3D (ours), RVG, Amodal3R, Hy3D-2mv, Cupid pending rescore pending rescore
Pixal3D pending (regeneration + rescore) pending rescore

Smoke test of the final evaluator (FORGE3D 4v meshes, first 10 objects of the selection; not a table row): n=10: IV LPIPS 0.0733 / SSIM 0.9049 / PSNR 23.26 (views 40); NV LPIPS 0.0818 / SSIM 0.9016 / PSNR 22.73 (views 185); CD-L1 0.0105, F@0.01 0.752, NC 0.8959 (code/eval/tests/TEST_RESULTS.json).

7.3 OmniObject3D-300 4v (masked GT-image protocol; headline excludes the 14 bad-frame objects)

All cells are pending rescore. FORGE3D meshes are complete (300/300, unbatched); every baseline is partial (section 6).

7.4 Protocol ceilings (GT mesh scored as the prediction, final code)

  • FB150 (all 150, 4v IV / NV vis>=0.2): IV LPIPS 0.1450 / SSIM 0.8604 / PSNR 18.51 (views 600); NV LPIPS 0.1444 / SSIM 0.8615 / PSNR 18.43 (views 2798); 1v IV (front only): LPIPS 0.1477 / SSIM 0.8565 / PSNR 18.35.
  • Omni (60 objects: every 5th; bad-frame excluded): IV LPIPS 0.0781 / SSIM 0.9191 / PSNR 23.01 (views 228); NV LPIPS 0.0821 / SSIM 0.9165 / PSNR 22.73 (views 228).
  • Toys4K (original protocol, 5 objects): IV LPIPS 0.0484 / PSNR 26.26; NV (orbit, render vs render) exactly perfect. Methods that bake scene lighting into their texture can score above the GT-mesh ceiling on IV (e.g. FORGE3D on FaGr_s000001_butternutsquash1: IV PSNR 25.0 vs GT-mesh 22.8), because the reference is the lit image.

8. Tests (run 2026-09-26 KST, GPU 7)

test n obj IV LPIPS / PSNR NV LPIPS / PSNR CD-L1 / F@0.01 / NC
FB150 GT mesh, --ref image (protocol ceiling) 5 0.1554 / 17.66 0.1558 / 17.69 0.0000 / 1.000 / 1.0000
FB150 GT mesh, --ref gtrender (identity) 5 0.0000 / inf 1.339e-11 / inf 0.0000 / 1.000 / 1.0000
Omni GT mesh, --ref image (ceiling, bad-frame excluded) 4 0.0667 / 22.58 0.07639 / 22.15 0.0000 / 1.000 / 0.9999
Omni GT mesh, --ref gtrender (identity, bad-frame excluded) 4 0.0000 / 186.37 1.548e-11 / 186.55 0.0000 / 1.000 / 0.9999
Toys GT mesh (original protocol) 5 0.0484 / 26.26 2.961e-19 / inf 0.0000 / 1.000 / 1.0000
  • tests/test_equiv_heldout.py (eval_final.masked_view vs eval_heldout.heldout_view with the original far=20 renderer, FB150 incl. far-camera cubes + Omni): EQUIV (cams < 10 units: 19 views, bit-identical renders 11, max metric diff 2.05e-08, max pix diff 5.96e-08): PASS ; far cams (>=10 units, far-clip fix may change render): 9 views, changed 4
  • Toys path vs the unchanged evaluate_appforce.py on 2 FORGE3D meshes: max |diff| appearance 3.1e-10, geometry 1.5e-05 (the same order as two runs of the original code: GPU point-sampling noise).
  • Real prediction (FORGE3D FB150 4v, first 10 objects): n=10: IV LPIPS 0.0733 / SSIM 0.9049 / PSNR 23.26 (views 40); NV LPIPS 0.0818 / SSIM 0.9016 / PSNR 22.73 (views 185); CD-L1 0.0105, F@0.01 0.752, NC 0.8959 (code/eval/tests/TEST_RESULTS.json).
  • GT-mesh scored as the prediction: geometry exactly perfect on all datasets; with --ref gtrender appearance is exactly perfect (LPIPS 0, SSIM 1, PSNR inf / >180 dB); with the protocol reference (GT image) it gives the ceiling, not 0, because the GT images are lit and the render is unlit.

9. Other material in this folder (unchanged from the previous upload)

  • todo/, build_rest658.py, views_4v.json: the remaining 658 objects of the full 808-object FORGE3DBench (same builder as FB150), meant for generation on other servers. Held-out views for those objects are not built. To score them with the final protocol, build them with code/eval/heldout/prepare_heldout.py.
  • code/omni_vggt/: the experimental OmniObject3D real-capture VGGT-Omega seed ablation. It is not part of the main protocol. The OpenDataLab keys are read only from the OPENXLAB_AK / OPENXLAB_SK environment variables and no credential is in this folder.
  • code/baselines/pixal3d_ours/batch_pixal3d_mv_cropK.py is superseded on FB150 by the re-centre driver.