Hugging Face's logo

gdslab
/
alma

English
lidar
point-cloud
self-supervised-learning
masked-autoencoder
semantic-segmentation
Model card Files
xet
 Community
alma / docs /publishing.md
goel65's picture
goel65
Upload folder using huggingface_hub
48ee80f verified
preview code
|
Raw History Blame Contribute Delete
6.28 kB

Publishing ALMA code and weights

Current terminology update

The current release uses public segmentation terminology throughout. Four segmentation configs were renamed. The QL2 segmentation checkpoint now uses a public provenance identifier; all tensors are identical, but its file hash changed. Upload the refreshed folder and synchronize the config directory so obsolete config filenames are removed from the current hosted revision:

hf upload gdslab/alma ./release_artifacts . --delete="configs/*"

This follows the CLI's folder synchronization behavior: local configs are uploaded and obsolete remote configs are removed. Existing repository history is retained. After the upload, use its new commit hash with scripts/set_weight_urls.py, refresh/upload the manifest, and then commit/push the code changes. The user completed this upload at commit aa51bb453ec2aac74e74e1d84e2d0903625575e7; all six URLs are now pinned to that revision. Upload the refreshed manifest/docs before pushing GitHub code.

Six verified weights (about 1.44 GB) are staged in release_artifacts/. This directory and downloaded weights are excluded from Git. Originals remain in the research repositories. Run the commands below yourself; authentication and uploads are not performed by the preparation step.

1. Validate inside a GPU allocation 

cd /scratch/gautschi/goel65/ALMA
conda activate pointm2ae
ALMA_WEIGHTS_DIR=release_artifacts python -m unittest discover -s tests -v

Confirm the CUDA test passes rather than skips. Record the actual result in the audit/change log. Full training and paper-metric reproduction have not been rerun; do not infer those from the smoke test.

2. Refresh upload metadata

The canonical HF model card is version-controlled at model_cards/huggingface.md. Refresh its staged copy after edits:

cp model_cards/huggingface.md release_artifacts/README.md
cp LICENSE WEIGHTS_LICENSE NOTICE CITATION.cff CODE_CHANGE_LOG.txt release_artifacts/
cp checkpoints/manifest.yaml release_artifacts/manifest.yaml
cp -r configs model_cards docs release_artifacts/
(cd release_artifacts && sha256sum -c SHA256SUMS)

All six checksums must report OK. This does not rewrite weight files.

3. Upload to Hugging Face

Use a machine/node with internet access and an account with write access to the gdslab organization:

pip install -U huggingface_hub
hf auth login

If the model repository does not exist, create it as public:

hf repos create gdslab/alma --repo-type model

Skip creation if it already exists. Upload the prepared folder:

hf upload gdslab/alma ./release_artifacts .

Commands follow the official HF CLI guide. Keep the full 40-character commit hash from the upload result or the model repository's commit history.

4. Pin and verify download links

Replace COMMIT_HASH with that actual weight-upload commit hash:

python scripts/set_weight_urls.py --repo-id gdslab/alma --revision COMMIT_HASH
cp checkpoints/manifest.yaml release_artifacts/manifest.yaml
hf upload gdslab/alma release_artifacts/manifest.yaml manifest.yaml

The manifest-only upload creates another commit; weight URLs should remain pinned to the earlier commit containing the verified weights. Never insert a made-up commit or replace the revision with main.

Use a fresh directory so existing local files cannot bypass download testing:

ALMA_VERIFY_DIR=$(mktemp -d /tmp/alma-download-verify.XXXXXX)
for regime in QL1 QL2; do
  for kind in encoder mae seg; do
    python scripts/download_weights.py "ALMA-${regime}-Indiana-${kind}" --out-dir "$ALMA_VERIFY_DIR" || break 2
  done
done

Require six verified= messages. A download/checksum failure must be resolved before publishing the code release. The verification directory holds about 1.44 GB and can be removed after verification.

Record the weight commit, completed verification, and manifest change in CODE_CHANGE_LOG.txt and the workspace's global audit log before committing. The initial upload commit was 5307569f57e16b59eca53484c42304d8ddd8c38e. The terminology update is uploaded at aa51bb453ec2aac74e74e1d84e2d0903625575e7, and the manifest is repinned. Fresh public-download verification remains to be completed.

5. Publish the GitHub repository

Initial local state on 2026-09-30: no commits, no remote, branch master. Create an empty public GitHub repository named gdslab/alma if needed. Do not generate another README, license, or gitignore.

Inspect remote history before the first push:

git ls-remote https://github.com/gdslab/alma.git

For an empty remote and the initial local state:

git branch -M main
git remote add origin https://github.com/gdslab/alma.git
git add .
git diff --cached --stat
git diff --cached --name-only

Review the staged files: no weights, datasets, credentials, or outputs should be present. Then:

git commit -m "Release ALMA pretraining and segmentation"
git push -u origin main
git tag -a v0.1.0 -m "ALMA initial public release"
git push origin v0.1.0

If a remote or commits already exist, inspect them and preserve their history before proceeding; do not force-push or reset the remote. Authenticate with your GitHub account using your established Git credential setup.

Completion

Check both public repository pages, license display, cross-links, all six weight files, and the pinned manifest. From a fresh GitHub checkout, install the package and download the weights using the manifest; run the GPU checks with that checkout to confirm the public release path works.

CITATION.cff contains the title and authors. Add manuscript DOI/venue when available. For an optional archival DOI, follow Zenodo's GitHub release instructions. Git-ignored weights are not included in a code archive; archive them separately if a DOI for the model files is needed.