--- license: cc-by-nc-4.0 base_model: google/gemma-4-31B-it base_model_relation: finetune library_name: transformers language: - en - vi pipeline_tag: text-classification tags: - decision-making - multiple-choice - probabilities - decision-index - gemma4 --- # decy-preview > Ask a question with fixed answers. Get one answer back, and how likely each answer is. decy turns a model call into a decision. Instead of generating text you then have to parse, it reads your input once and returns a probability for every allowed answer. Questions can be about plain text, JSON records, chat logs or agent traces, in English or Vietnamese. *Preview release: weights and interface may still change.* ## Try it ```python import sys from huggingface_hub import snapshot_download sys.path.insert(0, snapshot_download("PNPM244/decy-preview")) from decy import Decy decy = Decy(model="PNPM244/decy-preview", device="cuda:0") order = {"item": "running shoes", "status": "delivered", "days_since_delivery": 41, "message": "They hurt my feet, I want my money back."} result, _ = decy(order, { "refund": {"type": "choice", "instructions": "How should this refund request be handled?", "criteria": {"approve": "Within the return window and policy allows it", "escalate": "Needs a human agent to decide", "decline": "Outside the 30-day return window"}}, "upset": {"type": "noul", "instructions": "The customer sounds upset."}, }) print(result["answers"]) ``` Several questions about the same input go in one call. `choice` questions pick one of the listed answers; `noul` questions are yes/no and return the probability of yes. ## Why use it - **One pass, fixed cost.** No sampling, no output parsing, no retries for malformed answers. - **Probabilities you can threshold.** Route low-confidence cases to a person instead of guessing. - **Judges the answers on their merits.** Catch-all answers like "None of the other options is correct" are picked only when the specific answers are actually wrong. ## Hardware BF16 weights take about 62 GB, so a single 80 GB GPU (A100/H100/H200) runs it. `decy.py` loads the model with `transformers`; `bidir.py` switches the global attention layers to read the input in both directions, which is how the model was trained and evaluated. Keep the defaults (`noncausal=True`, `layout="state"`). ## Benchmarks **Decision Index, public 0.3 suite: 68.03** (complete run, 140,620 requests, all answered; scored with the leaderboard's code). Results: [PNPM244/decy-preview-decision-index](https://huggingface.co/datasets/PNPM244/decy-preview-decision-index/tree/main/runs/decy-preview). Official Full scores (private tests run by the maintainers) will be added when available. | Public index | Knowledge | Language | Retrieval | Tools | Arts | |---:|---:|---:|---:|---:|---:| | **68.03** | 56.9 | 72.2 | 68.8 | 83.0 | 56.9 | Latency on one H200, one request at a time: median 63 ms, mean 286 ms, p80 214 ms. Reproduce the public suite with: ``` PYTHONPATH= python -m decision_index pipeline --engine decy:Decy \ --option model= --option device=cuda:0 --out runs/decy-preview ``` ## License Released under [CC BY-NC 4.0](LICENSE) (weights, code and card). Non-commercial use only, with attribution; ask the author for commercial terms. Derived from Gemma 4, Apache 2.0: see [NOTICE](NOTICE) and [LICENSE-APACHE-2.0](LICENSE-APACHE-2.0). Author: Tran Minh Duc ([PNPM244](https://huggingface.co/PNPM244)), 2026.