Pivot

Fast closed-set decisions from context and candidate actions

Pivot is a 357.6M-parameter bidirectional decision encoder. Give it a context and two or more candidate answers; it scores the set in one forward pass and returns the chosen answer, its position, and a probability for every candidate. It does not generate free-form text. The full FP32 checkpoint, tokenizer and Transformers custom runtime are stored in this model repository.

Measured result	Pivot
JevBench v1.4.1 public accuracy	46.32% (107 / 231)
NVIDIA H200 warm single-decision p50 / p95	15.8 / 19.9 ms
NVIDIA H200 throughput, batch 32	545.3 decisions/s
4-thread Xeon CPU warm single-decision p50 / p95	797.6 / 1,089.0 ms
4-thread Xeon CPU throughput, batch 4	3.77 decisions/s

These are measurements on the pinned checkpoint, in FP32, including tokenization and scoring. GPU and CPU throughput used different batch sizes. The 46.32% figure is public-task accuracy; the official JevBench v1.4 composite score has not been measured because the sealed/judge portion and cost input are unavailable. Protocol and limitations · Full measurement details

Start with one decision

Install a suitable PyTorch build and the runtime packages:

python -m pip install "transformers==5.17.0" "safetensors==0.8.0"

import torch
from transformers import AutoModel, AutoTokenizer

tokenizer = AutoTokenizer.from_pretrained("Q1z/Pivot", trust_remote_code=True)
model = AutoModel.from_pretrained(
    "Q1z/Pivot", trust_remote_code=True, dtype=torch.float32
).eval()

context = "CONTEXT:\nA customer disputes an invoice and asks for a correction."
options = [
    "route to billing support",
    "route to technical support",
    "route to sales",
]
decision = model.choose(tokenizer, context, options)
print(decision)  # {"choice": ..., "index": ..., "probs": [...]}

The serving configuration now defaults to 512 context tokens and 128 option tokens, matching the published public evaluation. It does not change the checkpoint weights. Run on CUDA with model.to("cuda") if your PyTorch build supports it. Review the repository's custom model code before enabling trust_remote_code=True.

Three ways to use Pivot

Method	Input	Output
model.choose(tokenizer, context, options)	Context and ordered answer strings	Chosen answer, index, probability vector
model.decide_native(tokenizer, context, candidates)	Candidate IDs, semantic text, optional abstain action	Selected ID, relative confidence, per-candidate probabilities
model.decide(tokenizer, state, questions)	Several typed choice / yes-no / score questions	Typed decision response

For repeated decisions using the same options, encode_candidates, encode_context, and choose_cached reuse candidate representations. This can avoid repeated candidate encoding; the measured throughput above uses the uncached path. Examples: basic, structured decisions, cached candidates, and full inference guide.

Pivot's probabilities are relative to the supplied candidate set. Give each candidate a clear, distinct meaning. An abstain route must be an explicit candidate; a high relative probability alone does not establish real-world correctness or safety.

Evaluated performance

The official JevBench v1.4.1 public tasks were scored with the official per-task scorer at commit 24b9b5c1609a7a9e8fa14f49e5985a836c9dc842. The exact evaluated model commit was 14bf8c26bf344ebdf88e22a4b6152dc5f75f3578.

Public tier	Correct / tasks	Accuracy	ECE, 10 bins
Original	27 / 72	37.50%	0.525
Easy	39 / 48	81.25%	0.114
Hard	41 / 111	36.94%	0.379
All public tasks	107 / 231	46.32%	—

The frozen evaluation format is a CONTEXT: prefix, task rubric descriptions where provided (otherwise humanized labels) in official label order, right truncation, 512 context tokens and 128 option tokens. Public accuracy is not an official leaderboard score. Reproduce public results or run the CPU speed entry point. The CPU notebook provides an interactive alternative.

Repository guide

• Inference methods and examples
• Serving interfaces and request formats
• JevBench protocol, revision, and score scope
• Results, hardware, charts, and caveats
• Reproducible public benchmark script
• Standalone CPU speed script
• Pinned result files and chart
• What changed in this update

The files model.safetensors, modeling_pivot.py, modeling_lfm2_bidirectional.py, pivot_model.py and pivot_infer.py contain the checkpoint and model runtime. Package manifest records file hashes. The original checkpoint and evaluated metrics remain anchored to the exact revision above.