README.md

---
library_name: transformers
pipeline_tag: feature-extraction
base_model: LiquidAI/LFM2.5-Encoder-350M
tags:
- pivot
- decision-making
- classification
- scoring
- custom_code
---

# Pivot

### Fast closed-set decisions from context and candidate actions

Pivot is a **357.6M-parameter** bidirectional decision encoder. Give it a context and two or more candidate answers; it scores the set in one forward pass and returns the chosen answer, its position, and a probability for every candidate. It does not generate free-form text. The full FP32 checkpoint, tokenizer and Transformers custom runtime are stored in this model repository.

| Measured result | Pivot |
|---|---:|
| JevBench v1.4.1 public accuracy | **46.32%** (107 / 231) |
| NVIDIA H200 warm single-decision p50 / p95 | **15.8 / 19.9 ms** |
| NVIDIA H200 throughput, batch 32 | **545.3 decisions/s** |
| 4-thread Xeon CPU warm single-decision p50 / p95 | **797.6 / 1,089.0 ms** |
| 4-thread Xeon CPU throughput, batch 4 | **3.77 decisions/s** |

These are measurements on the pinned checkpoint, in FP32, including tokenization and scoring. GPU and CPU throughput used **different batch sizes**. The 46.32% figure is public-task accuracy; the **official JevBench v1.4 composite score has not been measured** because the sealed/judge portion and cost input are unavailable. [Protocol and limitations](docs/BENCHMARK.md) · [Full measurement details](docs/PERFORMANCE.md)

![Public accuracy and warm local speed](evaluation/2026-09-24/performance_overview.png)

## Start with one decision

Install a suitable PyTorch build and the runtime packages:

```bash
python -m pip install "transformers==5.17.0" "safetensors==0.8.0"
```

```python
import torch
from transformers import AutoModel, AutoTokenizer

tokenizer = AutoTokenizer.from_pretrained("Q1z/Pivot", trust_remote_code=True)
model = AutoModel.from_pretrained(
    "Q1z/Pivot", trust_remote_code=True, dtype=torch.float32
).eval()

context = "CONTEXT:\nA customer disputes an invoice and asks for a correction."
options = [
    "route to billing support",
    "route to technical support",
    "route to sales",
]
decision = model.choose(tokenizer, context, options)
print(decision)  # {"choice": ..., "index": ..., "probs": [...]}
```

The serving configuration now defaults to **512 context tokens and 128 option tokens**, matching the published public evaluation. It does not change the checkpoint weights. Run on CUDA with `model.to("cuda")` if your PyTorch build supports it. Review the repository's custom model code before enabling `trust_remote_code=True`.

## Three ways to use Pivot

| Method | Input | Output |
|---|---|---|
| `model.choose(tokenizer, context, options)` | Context and ordered answer strings | Chosen answer, index, probability vector |
| `model.decide_native(tokenizer, context, candidates)` | Candidate IDs, semantic text, optional abstain action | Selected ID, relative confidence, per-candidate probabilities |
| `model.decide(tokenizer, state, questions)` | Several typed choice / yes-no / score questions | Typed decision response |

For repeated decisions using the same options, `encode_candidates`, `encode_context`, and `choose_cached` reuse candidate representations. This can avoid repeated candidate encoding; the measured throughput above uses the **uncached** path. Examples: [basic](examples/quickstart.py), [structured decisions](examples/native_decision.py), [cached candidates](examples/cached_candidates.py), and [full inference guide](docs/INFERENCE.md).

Pivot's probabilities are **relative to the supplied candidate set**. Give each candidate a clear, distinct meaning. An abstain route must be an explicit candidate; a high relative probability alone does not establish real-world correctness or safety.

## Evaluated performance

The official JevBench v1.4.1 **public tasks** were scored with the official per-task scorer at commit `24b9b5c1609a7a9e8fa14f49e5985a836c9dc842`. The exact evaluated model commit was `14bf8c26bf344ebdf88e22a4b6152dc5f75f3578`.

| Public tier | Correct / tasks | Accuracy | ECE, 10 bins |
|---|---:|---:|---:|
| Original | 27 / 72 | 37.50% | 0.525 |
| Easy | 39 / 48 | 81.25% | 0.114 |
| Hard | 41 / 111 | 36.94% | 0.379 |
| **All public tasks** | **107 / 231** | **46.32%** | — |

The frozen evaluation format is a `CONTEXT:` prefix, task rubric descriptions where provided (otherwise humanized labels) in official label order, right truncation, 512 context tokens and 128 option tokens. Public accuracy is not an official leaderboard score. [Reproduce public results](benchmarks/README.md) or [run the CPU speed entry point](cpu-speed/README.md). The [CPU notebook](notebooks/Pivot_CPU_Speed.ipynb) provides an interactive alternative.

## Repository guide

- [Inference methods and examples](docs/INFERENCE.md)
- [Serving interfaces and request formats](serving/README.md)
- [JevBench protocol, revision, and score scope](docs/BENCHMARK.md)
- [Results, hardware, charts, and caveats](docs/PERFORMANCE.md)
- [Reproducible public benchmark script](benchmarks/jevbench_public.py)
- [Standalone CPU speed script](cpu-speed/benchmark.py)
- [Pinned result files and chart](evaluation/2026-09-24/README.md)
- [What changed in this update](docs/UPDATE_NOTES.md)

The files `model.safetensors`, `modeling_pivot.py`, `modeling_lfm2_bidirectional.py`, `pivot_model.py` and `pivot_infer.py` contain the checkpoint and model runtime. [Package manifest](manifest.json) records file hashes. The original checkpoint and evaluated metrics remain anchored to the exact revision above.