Generation (Perceiver)

Overview

This is an experimental Perceiver codebase for Generation. It keeps the giant setup intentionally manageable so architecture changes can be inspected before a full training run.

Feedback on the structure is welcome; the code is intentionally short.

---

Default Experiment Recipe

The generated recipe uses rmsprop with a cosine schedule.

These values exist so the script runs end-to-end. A meaningful experiment needs its own tuning budget and at least three seeds.

---

Limitations

• weights are random-init, so outputs carry no signal
• robustness, fairness and domain transfer are untested
• the API may change without notice while the design settles

---

Repository Status

• Single-file implementation; the __main__ block runs a smoke test.
• config.json and training_args.json hold the generated settings.
• model.safetensors is freshly initialized — plumbing, not results.
• No benchmark numbers are reported on purpose.

---

Training Details

• Weights initialized with a kaiming normal scheme.
• Runs on a single consumer GPU.
• Full smoke test takes minutes, not hours.
• No distributed setup required.

---

Changelog

• 2026-10-18 — initial upload
• 2026-10-19 — config and docs sync

---

Evaluation Guidance

A first evaluation could use a task-specific held-out set. Report the metric across at least three seeds and include a matched-capacity baseline; keep logs and library versions with any published number.

---

Files

• eval.py — primary artifact
• README.md — this documentation
• config.json — architecture configuration
• training_args.json — default experiment settings
• model.safetensors — initialization checkpoint
• USE_POLICY.md — use and redistribution notes

---

License

Released under unknown. Review the source-data terms separately when this repository is used with external datasets.