coca-matching / README.md
jamesdavie's picture
Add finetune.py, README.md, config.json, training_args.json, model.safetensors, USE_POLICY.md
9f048a7 verified
|
Raw History Blame Contribute Delete
2.36 kB
---
license: unknown
tags:
- pytorch
- coca
- matching
- smoke-test
- prototype
---
# Coca experiments: Matching
## Overview
This is an experimental **Coca** codebase for **Matching**. It keeps the **large** setup intentionally manageable so architecture changes can be inspected before a full training run.
Expect rough edges — this is a working sketch, not a polished release.
## Contact
Questions or corrections: open a discussion in this repository.
## Limitations
The initialization checkpoint has not been trained or audited for robustness, fairness, or domain transfer. The implementation should be treated as an experimental starting point. Results from a future trained checkpoint must be documented separately from the defaults shipped here.
## Repository Status
One Python module contains the model and a runnable example.
Generated configuration lives in `config.json`; the recipe in `training_args.json`.
The checkpoint is an initialization artifact and should not be read as a trained model.
Scores are deliberately absent.
## Architecture
| Item | Value |
|---|---|
| Architecture | Coca |
| Scale | large |
| Attention | standard |
| Fusion | tucker |
| Activation | gelu tanh |
| Normalization | groupnorm |
## Default Experiment Recipe
The generated recipe uses adam with a exponential schedule.
These values exist so the script runs end-to-end. A meaningful experiment needs its own tuning budget and at least three seeds.
## Evaluation Guidance
A first evaluation could use a paired validation set. Report the metric across at least three seeds and include a matched-capacity baseline; keep logs and library versions with any published number.
## Quick Check
Run `python finetune.py --help` for the built-in smoke test. Because this is a custom implementation, generic automatic loading APIs require an explicit adapter before use.
## Files
* `finetune.py` — primary artifact
* `README.md` — this documentation
* `config.json` — architecture configuration
* `training_args.json` — default experiment settings
* `model.safetensors` — initialization checkpoint
* `USE_POLICY.md` — use and redistribution notes
## License
Distributed under the **unknown** license. External datasets keep their own terms — check them separately.