decy-preview / README.md
beyoru's picture
card: public results
287ebe7 verified
|
Raw History Blame Contribute Delete
3.56 kB
---
license: cc-by-nc-4.0
base_model: google/gemma-4-31B-it
base_model_relation: finetune
library_name: transformers
language:
- en
- vi
pipeline_tag: text-classification
tags:
- decision-making
- multiple-choice
- probabilities
- decision-index
- gemma4
---
# decy-preview
> Ask a question with fixed answers. Get one answer back, and how likely each answer is.
decy turns a model call into a decision. Instead of generating text you then have to parse, it reads your input once
and returns a probability for every allowed answer. Questions can be about plain text, JSON records, chat logs or agent
traces, in English or Vietnamese.
*Preview release: weights and interface may still change.*
## Try it
```python
import sys
from huggingface_hub import snapshot_download
sys.path.insert(0, snapshot_download("PNPM244/decy-preview"))
from decy import Decy
decy = Decy(model="PNPM244/decy-preview", device="cuda:0")
order = {"item": "running shoes", "status": "delivered", "days_since_delivery": 41,
"message": "They hurt my feet, I want my money back."}
result, _ = decy(order, {
"refund": {"type": "choice", "instructions": "How should this refund request be handled?",
"criteria": {"approve": "Within the return window and policy allows it",
"escalate": "Needs a human agent to decide",
"decline": "Outside the 30-day return window"}},
"upset": {"type": "noul", "instructions": "The customer sounds upset."},
})
print(result["answers"])
```
Several questions about the same input go in one call. `choice` questions pick one of the listed answers;
`noul` questions are yes/no and return the probability of yes.
## Why use it
- **One pass, fixed cost.** No sampling, no output parsing, no retries for malformed answers.
- **Probabilities you can threshold.** Route low-confidence cases to a person instead of guessing.
- **Judges the answers on their merits.** Catch-all answers like "None of the other options is correct" are picked only
when the specific answers are actually wrong.
## Hardware
BF16 weights take about 62 GB, so a single 80 GB GPU (A100/H100/H200) runs it. `decy.py` loads the model with
`transformers`; `bidir.py` switches the global attention layers to read the input in both directions, which is how the
model was trained and evaluated. Keep the defaults (`noncausal=True`, `layout="state"`).
## Benchmarks
**Decision Index, public 0.3 suite: 68.03** (complete run, 140,620 requests, all answered; scored with the leaderboard's
code). Results: [PNPM244/decy-preview-decision-index](https://huggingface.co/datasets/PNPM244/decy-preview-decision-index/tree/main/runs/decy-preview).
Official Full scores (private tests run by the maintainers) will be added when available.
| Public index | Knowledge | Language | Retrieval | Tools | Arts |
|---:|---:|---:|---:|---:|---:|
| **68.03** | 56.9 | 72.2 | 68.8 | 83.0 | 56.9 |
Latency on one H200, one request at a time: median 63 ms, mean 286 ms, p80 214 ms.
Reproduce the public suite with:
```
PYTHONPATH=<snapshot> python -m decision_index pipeline --engine decy:Decy \
--option model=<snapshot> --option device=cuda:0 --out runs/decy-preview
```
## License
Released under [CC BY-NC 4.0](LICENSE) (weights, code and card). Non-commercial use only, with attribution;
ask the author for commercial terms. Derived from Gemma 4, Apache 2.0: see [NOTICE](NOTICE) and
[LICENSE-APACHE-2.0](LICENSE-APACHE-2.0).
Author: Tran Minh Duc ([PNPM244](https://huggingface.co/PNPM244)), 2026.