| ---
|
| license: unknown
|
| tags:
|
| - pytorch
|
| - perceiver
|
| - generation
|
| - wip
|
| ---
|
|
|
| # Generation (Perceiver)
|
|
|
| ## Overview
|
|
|
| This is an experimental **Perceiver** codebase for **Generation**. It keeps the **giant** setup intentionally manageable so architecture changes can be inspected before a full training run.
|
|
|
| Feedback on the structure is welcome; the code is intentionally short.
|
|
|
| ---
|
|
|
| ## Default Experiment Recipe
|
|
|
| The generated recipe uses rmsprop with a cosine schedule.
|
|
|
| These values exist so the script runs end-to-end. A meaningful experiment needs its own tuning budget and at least three seeds.
|
|
|
| ---
|
|
|
| ## Limitations
|
|
|
| * weights are random-init, so outputs carry no signal
|
| * robustness, fairness and domain transfer are untested
|
| * the API may change without notice while the design settles
|
|
|
| ---
|
|
|
| ## Repository Status
|
|
|
| * Single-file implementation; the `__main__` block runs a smoke test.
|
| * `config.json` and `training_args.json` hold the generated settings.
|
| * `model.safetensors` is freshly initialized — plumbing, not results.
|
| * No benchmark numbers are reported on purpose.
|
|
|
| ---
|
|
|
| ## Training Details
|
|
|
| * Weights initialized with a kaiming normal scheme.
|
| * Runs on a single consumer GPU.
|
| * Full smoke test takes minutes, not hours.
|
| * No distributed setup required.
|
|
|
| ---
|
|
|
| ## Changelog
|
|
|
| * 2026-10-18 — initial upload
|
| * 2026-10-19 — config and docs sync
|
|
|
| ---
|
|
|
| ## Evaluation Guidance
|
|
|
| A first evaluation could use a task-specific held-out set. Report the metric across at least three seeds and include a matched-capacity baseline; keep logs and library versions with any published number.
|
|
|
| ---
|
|
|
| ## Files
|
|
|
| * `eval.py` — primary artifact
|
| * `README.md` — this documentation
|
| * `config.json` — architecture configuration
|
| * `training_args.json` — default experiment settings
|
| * `model.safetensors` — initialization checkpoint
|
| * `USE_POLICY.md` — use and redistribution notes
|
|
|
| ---
|
|
|
| ## License
|
|
|
| Released under **unknown**. Review the source-data terms separately when this repository is used with external datasets.
|
|
|
