File size: 4,330 Bytes
3dcd7c7 4c82bf4 3dcd7c7 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 | ---
license: cc-by-nc-sa-4.0
library_name: pytorch
language:
- en
tags:
- ditgs
- 3d-reconstruction
- gaussian-splatting
- multiview
base_model:
- Wan-AI/Wan2.2-TI2V-5B-Diffusers
- nightkiller/FlashWorldPretrain
- jiaweir/tokengs
base_model_relation: finetune
---
# DiTGS
Camera-fixed Residual Cross-Attention feature readout for 3D Gaussian reconstruction.
[Code, installation and training](https://github.com/Qingzhongyun/DiTGS)
## Checkpoint
`model.pt` is the **raw final main-model checkpoint after 10,000 warmup updates
and 20,000 additional joint updates**. It uses the corrected-camera pipeline.
It is not an EMA checkpoint or an attention-ablation checkpoint.
The file is approximately 22.055 GB in decimal units. It contains the complete
inference network: the fine-tuned FlashWorld/Wan transformer, frozen Wan VAE,
frozen TokenGS image encoder, fine-tuned GS decoder/head, new adapter, independent
DiT-memory attention/KV/LayerScale branches, cached empty prompt and all required
normalization/rotary buffers. Storage dtypes and tensor values are unchanged.
There is no optimizer, EMA, RNG state, dataset media or full UMT5 text encoder.
Original server paths have been replaced with portable relative configuration.
`release.json` records the exact size, SHA256, component sizes and model contract.
## Download and inference
Install the source repository's pinned dependencies first. This is a custom
PyTorch model; it is not a Diffusers `from_pretrained` pipeline.
```bash
hf download qyk21/DiTGS --local-dir weights/ditgs
bash scripts/infer.sh 1 --checkpoint weights/ditgs/model.pt --data-root /path/to/DL3DV/benchmark --output outputs/residual/dl3dv_1view
bash scripts/infer.sh 2 --checkpoint weights/ditgs/model.pt --data-root /path/to/DL3DV/benchmark --output outputs/residual/dl3dv_2view
```
Run commands from the DiTGS code repository. The full checkpoint includes the
pretrained components needed for model inference; separate Wan/FlashWorld/TokenGS
weight downloads are required for training from initialization, not for loading
this file. LPIPS/VGG evaluation weights are downloaded separately by their
libraries. Use a CUDA GPU with enough memory for the complete model and rendering;
this publication did not establish a minimum GPU-memory requirement.
The benchmark expects posed images and camera intrinsics/extrinsics, not unposed
image reconstruction. The released 2-view index contains 140 scenes and 50 target
frames per scene. The 1-view setting uses its first context frame. Training used
2 input views. Default inference uses all 8 DiT camera slots, Euler NFE=1,
seed=42, native 256x448 evaluation, and no test-time optimization.
See the source README for novel-target versus input-overlap scoring and ZIP layout.
## License and upstream attribution
The checkpoint is released under **CC BY-NC-SA 4.0** in inheritance of the
FlashWorld pretrained-weight terms, with the upstream notices retained.
It is intended for **noncommercial academic research**. Retain attribution,
identify modifications and comply with ShareAlike when distributing adaptations.
The code repository's Apache-2.0 license is separate from this weight release.
TokenGS's root release README states Apache-2.0, while its separate model card
at the same upstream revision names a restrictive NVIDIA internal-research
license. This release follows the root release declaration; the discrepancy is
preserved explicitly in [UPSTREAM_NOTICES.md](UPSTREAM_NOTICES.md), including
links to both statements. No separate written upstream clarification is claimed.
A research-use statement does not override applicable upstream restrictions.
See [LICENSE](LICENSE), [NOTICE](NOTICE) and [UPSTREAM_NOTICES.md](UPSTREAM_NOTICES.md).
Please cite and acknowledge TokenGS, FlashWorld, Wan and DL3DV in work using this
model. Original paper/project links and changes are recorded in NOTICE.
## Validation and limitations
Packaging used CPU serialization only. Every model tensor and buffer was compared
with the existing inference export and found identical; the model contract was
also preserved. No CUDA training, inference, speed or quality evaluation was rerun
for this upload. No new performance claims are made here. This checkpoint is for
the main Residual CA model; ablation training scripts are in the code repository.
|