ald-tao-gpu / BUCKET_BUNDLE.md
TiGa-RCE's picture
Improve onboarding, useful defaults and continuation guidance
2a138ca verified
|
Raw History Blame Contribute Delete
5.84 kB
# Portable bucket bundles v1 and v2
The browser supports explicit local-file export/import and optional direct save/restore to an existing private bucket. Local export downloads one JSON bundle. Restore validates the complete bundle before staging the network and any optional track or recipe in the current page. A restored receipt is kept in page memory and offered as a separate download. Restore never starts a run or writes browser storage.
Bucket storage starts disconnected. After an explicit HF OAuth authorization for `openid profile write-repos`, the user enters an existing private bucket ID and connects. Connected-run autosave is on by default and can be turned off; disconnected or manual-only runs make no bucket storage calls. HF describes `write-repos` as read/write access to personal repositories; the grant is broader than one selected bucket, is not a bucket-only permission, and its mapping to bucket access and minimum required scope still needs live qualification. During storage operations, the page-memory token and selected bundle pass through this app server to the user-owned bucket. The app removes temporary files after an operation and exposes no bucket deletion, overwrite, creation or visibility controls. An observed expired/denied response clears the connection; the UI does not claim advance token-expiry knowledge. Provider-side logs and retention are outside the app's deletion guarantee. The token is not included in the portable bundle. With autosave, the browser verifies output and confirms server deletion before sending the network and sanitized run report to the bucket. A failure preserves the received local candidate; a batch stops before its next run if saving is not confirmed. A separate local CLI guide is distributed with the candidate tooling for users who prefer local bucket transfer.
Recipe libraries have separate explicit Save, Refresh, and Import actions. They contain settings and batch references only—not networks, tracks, run results, or credentials. Import asks before replacing the current browser library and never starts a run. Run autosave does not upload the recipe library.
## Envelope
The only top-level fields are `schema`, `version`, and `artifacts`. The schema is `ald.portable-bundle`. Legacy version `1` remains readable. Version `2` adds a validated analysis archive for new training and frozen-evaluation results. The required artifact slot is `selected_network`; optional fixed slots are `selected_track`, `selected_recipe`, and `run_receipt`. Version 2 also carries `analysis_archive`. Unknown slots or fields are rejected.
- A network stores its original `.evolution` bytes as base64 with byte length, SHA-256, format `evolution`, and kind `input` or `result`. The page validates decoded network content with its existing game-network validator and keeps the original bytes when staging it.
- A track stores original `.track` bytes with length and SHA-256. A built-in track also records its catalog ID and display label; an uploaded track has no extra identifying metadata.
- A recipe stores only version, A–J slot, name, and the existing validated training settings. Restore stages it into the form; saving it to the browser recipe library remains a separate user action.
- A run receipt is an allowlisted summary only: build ID, backend, output hash, whitelisted report fields for vehicle, counts, metrics, file hashes, and settings, plus up to 1,000 validated generation-history rows. It excludes paths, filenames, usernames, account data, job IDs/keys, and raw errors. Legacy receipts must match the bundled network and included track. With v2, the selected network may be any manifested winner; the run receipt retains the original fitness-result identity, and the validator checks both bindings independently. Frozen evaluation binds the selected network to its unchanged input. A connected auto-save bundles the exact verified run output, its captured input track, and matching run receipt.
Caps are 4 MiB for network bytes, 1 MiB for track bytes, 32 KiB for recipe JSON, 512 KiB for receipt JSON, and 48 MiB for the complete bundle. Payload bytes and hashes must match exactly.
## Analysis archive
The v2 `artifacts.analysis_archive` slot has `format: "ald.run-analysis-archive+gzip"`, `bytes`, `sha256`, and `data_base64`. Its bounded gzip payload expands to one JSON envelope with schema `ald.run-analysis-archive`, version 1, and a list of fixed-name files. Each entry carries its name, byte count, SHA-256 and base64 contents. The compressed cap is 32 MiB; expanded JSON is capped at 256 MiB, with an aggregate 192 MiB cap on decoded members to leave room for base64 overhead. Descriptors contain at most 128,000 complete car observations. Exceeding a limit is an error, never permission to truncate.
Required members are `analysis.json` and `descriptors.jsonl`. Train results include `winners/fitness.evolution`, plus fastest-lap and clean-lap files only when those categories have qualifying candidates. Frozen evaluation includes `evaluation.json` and no new winner files. Descriptors use the exact 18-channel order declared by the analysis manifest, storing compact value, population-relative rank, and unavailable-reason arrays.
Bucket saves preserve the canonical bundle, readable report/history, compressed archive, complete descriptor and metadata files, and individual winner networks. Every uploaded object is read back and checked before saving is reported as verified. Files live under a new run-specific prefix. The app never deletes existing bucket content or overwrites a previous run.
The local CLI can include a downloaded archive with `pack --analysis`; unpacking v2 restores its fixed analysis subdirectories into a new destination. Remote CLI operations remain explicit and use the local HF credential store.