jantrom's picture
Update README.md
2884f0c verified
|
Raw History Blame Contribute Delete
3.96 kB
metadata
license: mit
datasets:
  - jantrom/tile-obstacle-maps
tags:
  - Robotics
  - motion-planning
  - path-planning
  - diffusion

Assets to host outside GitHub

This directory mirrors the exact paths scripts/download_assets.py expects, so an upload can be verified and reproduced without guessing where anything goes.

It is a local staging area. It is not uploaded: subir.py ignores it, and the Hugging Face repositories carry their own dataset and model cards.

Layout

upload_assets/
β”œβ”€β”€ README.md                  this file (not uploaded)
β”œβ”€β”€ checksums.txt              sha256 of every asset below
β”œβ”€β”€ subir.py                   the upload script (not uploaded)
β”œβ”€β”€ tile-obstacle-maps.npz     -> dataset repo root
β”œβ”€β”€ artifacts/
β”‚   └── ckpt_tiles_seed1{01..06}/   -> same path in the model repo
β”‚       β”œβ”€β”€ diffusion_model.pt      the weights used for evaluation
β”‚       β”œβ”€β”€ modelo_final_ema.pt     the same weights, EMA copy
β”‚       └── ultimo.pt               RESUME checkpoint: weights + EMA + AdamW state
└── optional_derived/          NOT uploaded, NOT fetched by download_assets.py
    └── artifacts/metrics_seed*/.../maps.npz   per-repetition generated maps

The assets are split across two Hugging Face repositories, because HF treats the two kinds differently:

  • datasets/jantrom/tile-obstacle-maps β€” the dataset and both held-out splits.
  • jantrom/tile-obstacle-diffusion β€” the artifacts/ckpt_tiles_seed*/ tree.

What went up

Sizes are decimal MB, the same unit Hugging Face displays.

Path Uploaded Notes
tile-obstacle-maps.npz yes 171 MB. sha256 95a20e222a46057d1bfc3cb1d55c359fa1bca9d9ac064b8302c4bd2ceabdfcfe, the value in Table I.
ckpt_*/diffusion_model.pt yes 39.2 MB each. What the evaluation loads. Seed 101 alone reproduces the whole battery.
ckpt_*/modelo_final_ema.pt yes 39.2 MB each, the EMA copy of the same weights.
ckpt_*/ultimo.pt yes 156.9 MB each. Only needed to resume training β€” it carries the AdamW optimizer state, which roughly triples the file. Uploaded anyway so the model repo matches checksums.txt and CHECKPOINT_FILES in download_assets.py, which both list all three files.
optional_derived/ no 98 small maps.npz (~9 MB) holding the generated maps per mosaic repetition. Raw evidence behind Table VI and the diversity metrics; nothing reads them and they are regenerable.

Totals: 171 MB in the dataset repo, 1.41 GB in the model repo (6 Γ— 39.2 + 6 Γ— 39.2 + 6 Γ— 156.9).

Dropping the six ultimo.pt would save 941 MB, but it would also mean editing checksums.txt and CHECKPOINT_FILES so that download_assets.py --check does not fail on six missing files. Keeping them costs nothing but disk.

Verify before and after uploading

# every asset against the manifest
python3 - <<'PY'
import hashlib, os
for line in open("checksums.txt", encoding="utf-8"):
    s = line.strip()
    if not s or s.startswith("#"):
        continue
    h, p = s.split(None, 1)
    p = p.strip()
    if not os.path.exists(p):
        print("MISSING", p)
        continue
    d = hashlib.sha256()
    with open(p, "rb") as f:
        for blk in iter(lambda: f.read(1 << 20), b""):
            d.update(blk)
    print("OK  " if d.hexdigest() == h else "BAD ", p)
PY

After uploading, the same check from the other end:

python3 scripts/download_assets.py --check

A note on disk

The .pt and .npz files here are hardlinks to the copies in the repository working tree: they share the same inode, so this directory costs no extra disk. Deleting the repository's copies is safe β€” the data stays here. Moving this directory to another filesystem turns the hardlinks into real copies and the space is spent for real.