Pivot / serving /README.md
Q1z's picture
Expand Pivot model card, benchmarks, CPU tools and charts
7c85c7e verified
|
Raw History Blame Contribute Delete
1.08 kB
# Pivot serving interfaces
The model exposes three synchronous methods after loading `AutoModel` and `AutoTokenizer` with `trust_remote_code=True`:
1. `choose(tokenizer, context, options)` returns a simple choice, index and probability vector.
2. `decide_native(tokenizer, context, candidates)` accepts stable machine IDs, semantic candidate text and at most one explicit abstain candidate.
3. `decide(tokenizer, state, questions)` returns a typed collection of choice, yes/no and score decisions.
The model operates on supplied options; it does not create new options or generate explanatory text. The default serving limits in this update are 512 context tokens and 128 tokens per option. Keep a stable option set if comparing scores between requests.
The pre-existing [typed request](example_request.json) and [response](example_response.json) and [native request](native_example_request.json) and [response](native_example_response.json) illustrate the wire shapes. Run [a typed Python example](typed_decisions.py) or read the [full inference guide](../docs/INFERENCE.md).