Casantos73/paper-retrieval-2024

Retrieval (Albef)

Overview

A small Albef implementation for Retrieval, packaged with an explicit configuration and an initialization checkpoint. The xlarge variant is a reproducible starting point, not a trained model release.

Expect rough edges — this is a working sketch, not a polished release.

---

Training Details

1. Weights initialized with a trunc normal scheme.
2. Runs on a single consumer GPU.
3. Full smoke test takes minutes, not hours.
4. No distributed setup required.

---

Repository Status

1. Single-file implementation; the __main__ block runs a smoke test.
2. config.json and training_args.json hold the generated settings.
3. model.safetensors is freshly initialized — plumbing, not results.
4. No benchmark numbers are reported on purpose.

---

Changelog

1. 2026-10-16 — initial upload
2. 2026-10-21 — config and docs sync

---

Default Experiment Recipe

The generated recipe uses rmsprop with a step schedule.

These values exist so the script runs end-to-end. A meaningful experiment needs its own tuning budget and at least three seeds.

---

Environment

1. Python: >= 3.9
2. torch: 2.x
3. safetensors: any recent
4. transformers: optional, for loaders

---

Limitations

The initialization checkpoint has not been trained or audited for robustness, fairness, or domain transfer. The implementation should be treated as an experimental starting point. Results from a future trained checkpoint must be documented separately from the defaults shipped here.

---

Quick Check

Run python predict.py --help for the built-in smoke test. Because this is a custom implementation, generic automatic loading APIs require an explicit adapter before use.

---

Files

1. predict.py — primary artifact
2. README.md — this documentation
3. config.json — architecture configuration
4. training_args.json — default experiment settings
5. model.safetensors — initialization checkpoint
6. USE_POLICY.md — use and redistribution notes
7. examples/sample_usage.py — usage sketch

---

License

Released under bsd-2-clause. Review the source-data terms separately when this repository is used with external datasets.