decider-2b / source /README.md
Jeetu456's picture
Decider 2B
cd02163
|
Raw History Blame Contribute Delete
907 Bytes

decider

The Python package for Jeetu456/decider-2b.

pip install ./source            # from a downloaded copy of the model repo
pip install "./source[fast]"    # + fast linear-attention kernels (CUDA)
pip install "./source[quant]"   # + 8-bit / 4-bit loading (CUDA)
from decider import Decider

decider = Decider.from_pretrained("Jeetu456/decider-2b")      # repo id or local folder
decider.decide({"state": "...", "questions": {...}})            # one request -> one response
decider.decide([request_1, request_2, request_3])               # a list -> a list, batched on the GPU
decider.logits([request_1])                                     # raw option logits, before temperature

decider on the command line is the same tool as examples/decider_cli.py. decider/_core.py is a copy of the repository's decider_model.py.