|
Download README.md from ANABUR/photon-memory: direct link, hf CLI and curl.
- Browser
- Download file 4.01 kB
-
https://huggingface.co/ANABUR/photon-memory/resolve/main/README.md
- Command line
-
hf download hf://ANABUR/photon-memory/README.md
-
curl -L -o README.md https://huggingface.co/ANABUR/photon-memory/resolve/main/README.md
4.01 kB
| license: mit | |
| tags: | |
| - photon-memory | |
| - rendering | |
| - density-estimation | |
| - test-time-training | |
| library_name: pytorch | |
| # Photon Memory checkpoints (Rough Flower / PureLight comparison) | |
| Companion weights for the Photon Memory reader that matches / beats PureLight on directional NLL | |
| for the released Rough Flower luminaire. Code: [ANABUR920/renderTTT](https://github.com/ANABUR920/renderTTT) (`lightmem/`). | |
| ## Files | |
| | Path | Role | Size | | |
| |------|------|------| | |
| | `checkpoints/v3_ksched_model.pt` | Meta-trained PhotonKernel reader (`k_feat=True`) | ~0.7 MB | | |
| | `checkpoints/big_1e8_k512_ttt.pt` | Reader-only TTT on 1e8 memory (`k=512`) | ~0.7 MB | | |
| | `checkpoints/big_match_res2b_best.pt` | Intermediate matched TTT + residual (~NLL β4.54) | ~42 MB | | |
| | `checkpoints/big_match_res2c_best.pt` | Final matched TTT + residual (**NLL β6.00**, render default) | ~42 MB | | |
| | `logs/*.jsonl` | Training / eval logs for the above TTT runs | small | | |
| ## Download | |
| ```bash | |
| hf download ANABUR/photon-memory --local-dir ./photon-memory-ckpts | |
| # Photons (separate dataset): | |
| hf download ANABUR/photon-memory-photons --repo-type dataset --local-dir ./photon-memory-photons | |
| ``` | |
| ## Reproduce the β6.00 NLL claim (same protocol as paper) | |
| Needs: CUDA GPU (~40 GB for 1e8 + residual), traced Rough Flower photons (`flower_surface_*` 11e6 held-out + `flowerbig_surface_*` 1e8 memory), and this repo's `scripts/ttt_big.py`. | |
| Winning warm-start chain (do not claim a cold start from the meta reader equals β6.00 in one shot): | |
| 1. Start from `v3_ksched_model.pt` β reader-only TTT β `big_1e8_k512_ttt.pt` (~β0.53). | |
| 2. Joint reader + residual matched TTT β `big_match_res2b_best.pt` (~β4.54). | |
| 3. Continue: | |
| ```bash | |
| python scripts/ttt_big.py \ | |
| --data flowerbig_surface --N 100000000 --k 512 --wmult 2 \ | |
| --ttt_matched --n_ttt_q 1000000 --steps 8000 --residual \ | |
| --res_K 32 --res_log2 19 --res_finest 2048 \ | |
| --init_ttt checkpoints/big_match_res2b_best.pt \ | |
| --lr 5e-6 --res_lr_grid 5e-3 --res_lr_head 5e-4 \ | |
| --name big_match_res2c | |
| ``` | |
| Zero-shot (no TTT) with `v3_ksched_model.pt` at `N=1e8, k=512` reaches RGB-mean NLL β **β0.460**, matching the PureLight teacher plateau (~β0.46). The β6.00 number is **matched test-time adaptation** on the non-eval remainder of the same 11e6 photon file as the held-out set (memory may be a separately traced 1e8 bank). Leave-one-out TTT on the 1e8 memory alone stalls near β0.53. It is not zero-shot and not a fresh-retrace claim. | |
| ## Honest notes for reviewers / friends | |
| - Held-out queries are always the last 20k of `default_rng(0).permutation(11_000_000)` on `flower_surface_*`. | |
| - `--ttt_matched` trains on the non-eval remainder of that same 11e6 file while retrieving from the separately traced 1e8 memory. | |
| - A frozen-reader hash residual alone peaks ~β0.38 and does **not** beat PureLight. | |
| - **Photons** for Rough Flower are published separately: [`ANABUR/photon-memory-photons`](https://huggingface.co/datasets/ANABUR/photon-memory-photons) (`flower_surface_*` 11e6 + `flowerbig_surface_*` 1e8, channels mean/r/g/b). PureLight teacher weights and photon maps remain third-party; otherwise re-trace with `scripts/gen_flower.py`. | |
| ## Photons (required for reproduction) | |
| Rough Flower OptiX traces live in [`ANABUR/photon-memory-photons`](https://huggingface.co/datasets/ANABUR/photon-memory-photons): | |
| ```bash | |
| hf download ANABUR/photon-memory-photons --repo-type dataset --local-dir ./photon-memory-photons | |
| ``` | |
| Layout: `flower_surface_{mean,r,g,b}/val/rough_flower.npz` (11e6) and `flowerbig_surface_{mean,r,g,b}/val/rough_flower.npz` (1e8). | |
| ## Load in code | |
| ```python | |
| import torch | |
| from lightmem.models import PhotonKernel, ResidualLobes | |
| sd = torch.load("checkpoints/big_match_res2c_best.pt", map_location="cpu") | |
| reader = PhotonKernel(k=512, k_feat=True) | |
| reader.load_state_dict(sd["reader"]) | |
| residual = ResidualLobes(K=32, finest=2048, log2_table=19) | |
| residual.load_state_dict(sd["residual"]) | |
| # sd["nll"] β -6.00, sd["step"] == 8000 | |
| ``` | |