| --- |
| license: mit |
| datasets: |
| - jantrom/tile-obstacle-maps |
| tags: |
| - Robotics |
| - motion-planning |
| - path-planning |
| - diffusion |
| --- |
| # Assets to host outside GitHub |
|
|
| This directory mirrors the exact paths `scripts/download_assets.py` expects, so |
| an upload can be verified and reproduced without guessing where anything goes. |
|
|
| It is a local staging area. It is **not** uploaded: `subir.py` ignores it, and |
| the Hugging Face repositories carry their own dataset and model cards. |
|
|
| ## Layout |
|
|
| upload_assets/ |
| ├── README.md this file (not uploaded) |
| ├── checksums.txt sha256 of every asset below |
| ├── subir.py the upload script (not uploaded) |
| ├── tile-obstacle-maps.npz -> dataset repo root |
| ├── artifacts/ |
| │ └── ckpt_tiles_seed1{01..06}/ -> same path in the model repo |
| │ ├── diffusion_model.pt the weights used for evaluation |
| │ ├── modelo_final_ema.pt the same weights, EMA copy |
| │ └── ultimo.pt RESUME checkpoint: weights + EMA + AdamW state |
| └── optional_derived/ NOT uploaded, NOT fetched by download_assets.py |
| └── artifacts/metrics_seed*/.../maps.npz per-repetition generated maps |
| |
| The assets are split across two Hugging Face repositories, because HF treats the |
| two kinds differently: |
|
|
| * `datasets/jantrom/tile-obstacle-maps` — the dataset and both held-out splits. |
| * `jantrom/tile-obstacle-diffusion` — the `artifacts/ckpt_tiles_seed*/` tree. |
|
|
| ## What went up |
|
|
| Sizes are decimal MB, the same unit Hugging Face displays. |
|
|
| | Path | Uploaded | Notes | |
| |---|---|---| |
| | `tile-obstacle-maps.npz` | yes | 171 MB. sha256 `95a20e222a46057d1bfc3cb1d55c359fa1bca9d9ac064b8302c4bd2ceabdfcfe`, the value in Table I. | |
| | `ckpt_*/diffusion_model.pt` | yes | 39.2 MB each. What the evaluation loads. Seed 101 alone reproduces the whole battery. | |
| | `ckpt_*/modelo_final_ema.pt` | yes | 39.2 MB each, the EMA copy of the same weights. | |
| | `ckpt_*/ultimo.pt` | yes | 156.9 MB each. Only needed to resume training — it carries the AdamW optimizer state, which roughly triples the file. Uploaded anyway so the model repo matches `checksums.txt` and `CHECKPOINT_FILES` in `download_assets.py`, which both list all three files. | |
| | `optional_derived/` | no | 98 small `maps.npz` (~9 MB) holding the generated maps per mosaic repetition. Raw evidence behind Table VI and the diversity metrics; nothing reads them and they are regenerable. | |
|
|
| Totals: **171 MB** in the dataset repo, **1.41 GB** in the model repo |
| (6 × 39.2 + 6 × 39.2 + 6 × 156.9). |
|
|
| Dropping the six `ultimo.pt` would save 941 MB, but it would also mean editing |
| `checksums.txt` and `CHECKPOINT_FILES` so that `download_assets.py --check` does |
| not fail on six missing files. Keeping them costs nothing but disk. |
|
|
| ## Verify before and after uploading |
|
|
| # every asset against the manifest |
| python3 - <<'PY' |
| import hashlib, os |
| for line in open("checksums.txt", encoding="utf-8"): |
| s = line.strip() |
| if not s or s.startswith("#"): |
| continue |
| h, p = s.split(None, 1) |
| p = p.strip() |
| if not os.path.exists(p): |
| print("MISSING", p) |
| continue |
| d = hashlib.sha256() |
| with open(p, "rb") as f: |
| for blk in iter(lambda: f.read(1 << 20), b""): |
| d.update(blk) |
| print("OK " if d.hexdigest() == h else "BAD ", p) |
| PY |
| |
| After uploading, the same check from the other end: |
|
|
| python3 scripts/download_assets.py --check |
| |
| ## A note on disk |
|
|
| The `.pt` and `.npz` files here are hardlinks to the copies in the repository |
| working tree: they share the same inode, so this directory costs no extra disk. |
| Deleting the repository's copies is safe — the data stays here. Moving this |
| directory to another filesystem turns the hardlinks into real copies and the |
| space is spent for real. |
