File size: 1,081 Bytes
7c85c7e | 1 2 3 4 5 6 7 8 9 10 11 12 | # Pivot serving interfaces
The model exposes three synchronous methods after loading `AutoModel` and `AutoTokenizer` with `trust_remote_code=True`:
1. `choose(tokenizer, context, options)` returns a simple choice, index and probability vector.
2. `decide_native(tokenizer, context, candidates)` accepts stable machine IDs, semantic candidate text and at most one explicit abstain candidate.
3. `decide(tokenizer, state, questions)` returns a typed collection of choice, yes/no and score decisions.
The model operates on supplied options; it does not create new options or generate explanatory text. The default serving limits in this update are 512 context tokens and 128 tokens per option. Keep a stable option set if comparing scores between requests.
The pre-existing [typed request](example_request.json) and [response](example_response.json) and [native request](native_example_request.json) and [response](native_example_response.json) illustrate the wire shapes. Run [a typed Python example](typed_decisions.py) or read the [full inference guide](../docs/INFERENCE.md).
|