Download docs/pretraining.md from gdslab/alma: direct link, hf CLI and curl.
- Browser
- Download file 1.86 kB
-
https://huggingface.co/gdslab/alma/resolve/main/docs/pretraining.md
- Command line
-
hf download hf://gdslab/alma/docs/pretraining.md
-
curl -L -o pretraining.md https://huggingface.co/gdslab/alma/resolve/main/docs/pretraining.md
Masked-autoencoder pretraining
Separate QL1 DataParallel and QL2 DistributedDataParallel loops are ported from the corresponding research repositories. Model, optimizer, masking, dataset, and scheduler settings are in the YAMLs. Use a GPU allocation with working PointNet++ and Chamfer extensions.
Inputs are cropped NumPy XYZ arrays in feet. Edit each split's data_root
and pattern in a copied config; data are not distributed. Dataset code
samples/pads 20,480 points, applies configured vertical clipping, centers at
the sampled bounding-box midpoint, and divides XYZ by [50,50,25] (QL1) or
[100,100,75] (QL2). Training applies quarter-turn augmentation.
This differs from downstream normalization.
QL2 stage 2 starts from stage 1's best checkpoint. Stage 1's original config
allowed 300 epochs, but the recorded run stopped around epoch 152; the config
maximum does not mean completed training. Final stage-2 best is epoch 80.
Re-running the full stage-1 maximum is a new run and need not reproduce that
lineage. --max-epoch sets a stop epoch while retaining the scheduler period.
Use --resume with the same output directory for a trusted full training
checkpoint. Release weights omit optimizer state; use a full mae artifact
with --start-checkpoint for initialization, not exact resumption.
Encoder-only weights cannot initialize the reconstruction decoder strictly.
Resume preserves best-checkpoint selection, but inherited loops do not
restore all RNG/scaler state; bitwise resumed-run equivalence is not claimed.
Global batch size must be divisible by DDP world size.
The release contains model training and downstream pipelines. It does not contain private split manifests or every historical data-generation and ablation script. Original split/crop/thinning provenance is necessary to reproduce the paper's specific experiments.