laya-banking77 / code /README.md
VibeHash's picture
Add laya-banking77: Laya fine-tuned on Banking77, its scores and the code that made it
4e09985 verified
|
Raw History Blame Contribute Delete
2.04 kB
# laya-banking77: code
The code that built and scored the `VibeHash/laya-banking77` model on Hugging Face. The model card there has the
results, how to use the model and its limits. This folder is how to make the model again and check its numbers.
Everything runs on one GPU. Data, weights and runs go under `~/.laya-banking77/` (set `LAYA_B77_HOME` to move it).
Set `LAYA_B77_BASE` to a local copy of the base Laya checkpoint to skip downloading it; its weights are still checked
against the pinned SHA-256.
```bash
pip install torch --index-url https://download.pytorch.org/whl/cu126 # or the build for your GPU
pip install -r requirements.txt
python -m banking77.data # Banking77 at a pinned revision; writes and locks the test
python -m banking77.evaluate --model base --name zero-shot
python -m banking77.train --seed 0 # 26 minutes on an RTX 4070 SUPER
python -m banking77.evaluate --model ~/.laya-banking77/runs/seed0/model --name seed0
python -m banking77.offtopic --model ~/.laya-banking77/runs/seed0/model --name seed0
python -m banking77.compare zero-shot seed0
python -m banking77.package seed0 --repo VibeHash/laya-banking77
python -m unittest discover -s tests -t .
```
| Module | What it does |
|---|---|
| `banking77/data.py` | Downloads Banking77 at a pinned revision, checks its SHA-256, writes Laya's rows, locks the test file |
| `banking77/train.py` | Laya's own `finetune`, with `head_max_len` 512 and `target_error` 0.01 |
| `banking77/evaluate.py` | Scores a checkpoint through `laya.load(...).predict_batch`: accuracy, interval, calibration, threshold, speed |
| `banking77/offtopic.py` | Counts how many of CLINC150's out-of-scope queries pass the threshold |
| `banking77/compare.py` | Compares two runs message by message, with the exact McNemar test |
| `banking77/package.py` | Assembles the folder uploaded as the model and checks it for local paths |
| `banking77/metrics.py` | The scores themselves, in plain Python |
Apache 2.0 (`LICENSE`); credits in `NOTICE`.