---
license: apache-2.0
language:
- en
tags:
- factuality-verification
- routing
- cascading
- gradient-boosting
- numpy
- factcascade
---

# FactCascade: Utility-Guided Routing for Selective Factuality Verification

This repository provides the **trained frozen utility-router checkpoint** and the code/configuration needed to load it. It is a small NumPy histogram gradient booster (31 features, 100 trees, maximum depth 2), not a verifier language model. All learned tree splits/leaf values, histogram thresholds and train-only imputation/scaling parameters are in `router.json`.

The router estimates the expected signed change in correctness when a Bespoke decision replaces a cheap-verifier fallback. The frozen FactCascade allocation combines equal-weight empirical midranks of this estimate and the raw FT5/DeBERTa score gap, then selects a half-up quota of `920/3337` per component, with stable row-key ties. `config.json` gives the exact feature order, thresholds, quota and pinned verifier revisions.

## Repository contents and quick start

| Files | Purpose |
|---|---|
| `router.json` | Existing trained utility-router weights and DEV-only preprocessing |
| `config.json` | Frozen31feature order,policy,thresholds and upstream verifier revisions |
| `factcascade/` | NumPy feature,checkpoint loading,rank fusion,quota,replacement,metrics and adapter code |
| [MODEL_REFERENCE.md](MODEL_REFERENCE.md) | Model names,exact checkpoint revisions,pinned Hugging Face links and license scope |
| [docs/USAGE.md](docs/USAGE.md) | Download/setup,inputs,batch routing and selective replacement integration |
| [docs/FEATURES.md](docs/FEATURES.md) | All31slot definitions and DEV/external adapter differences |
| [examples/selective_replacement.py](examples/selective_replacement.py) | Runnable synthetic feature-to-routing-to-replacement example |
| `VALIDATION.json`, `USAGE_VALIDATION.json` | Original serialization parity and synthetic usage checks,respectively |

```bash
hf download nhnk/FactCascade --local-dir FactCascade
cd FactCascade
python -m pip install -r requirements.txt
python -B example.py
python -B examples/selective_replacement.py
```

The synthetic example uses invented scores/text and executes no verifier. Real deployment supplies the pinned native FT5/B192 observations and then calls Bespoke on the selected rows. Changing packet segmentation or batch boundaries changes the scientific operation;see the usage guide. The original checkpoint revision remains `08bd580bb9f81908295354a678b4c048ac72e40a`;this extension changes documentation/examples,not trained parameters.

## Load the existing trained checkpoint

Download the repository files, install `numpy==1.26.4`, and run:

```python
import numpy as np
from factcascade.utility_model import UtilityModel

model = UtilityModel.load("router.json")
# X must contain the exact ordered 31 frozen features; zeros only illustrate shape.
X = np.zeros((1, 31), dtype=np.float64)
utility = model.predict(X)
```

`example.py` runs this loading example. No fitting is needed. The JSON loader validates the schema, finite preprocessing parameters, feature order and tree structure. There is no executable pickle checkpoint in this public repository. This custom NumPy router does not use Transformers `AutoModel`.

For full selective verification, callers must produce the canonical features from the frozen verifier scores and evidence geometry, combine utility/disagreement ranks within the intended component, and apply the saved replacement policy. A feature vector from a different adapter, chunking rule or score convention is not interchangeable. Included API/adapters explain the expected record fields; this lightweight release does not execute the three verifier models.

## Training and evaluation scope

The saved router was trained on 3,337 DEV rows in 1,397 natural groups. Its target is per-row signed correctness change under unweighted squared loss; this surrogate does not directly optimize component-macro balanced accuracy. The final fusion policy was selected after observing DEV and external results. Results on exposed cohorts must be treated as retrospective; this release makes no claim that fusion is optimal or generally superior to disagreement or predicted-correctness routing.

This checkpoint is the original frozen full-DEV utility model. It is distinct from later diagnostic OOF/source-heldout fits and from the separate saved FCE predicted-correctness heads. Those baseline heads and verifier weights are not included.

## Serialization validation

`VALIDATION.json` records exact parameter-structure and CPU prediction parity with the original frozen checkpoint. Validation includes 1,024 synthetic rows, NaN/infinity imputation and every learned split boundary, plus equal fusion scores and selected synthetic routes. No training, real labels, new quality benchmark or GPU inference was performed. This verifies faithful export; it is not an additional generalization experiment.

## Verifier dependencies and license

FactCascade requires scores from [MiniCheck-Flan-T5-Large](https://huggingface.co/lytang/MiniCheck-Flan-T5-Large), [MiniCheck-DeBERTa-v3-Large](https://huggingface.co/lytang/MiniCheck-DeBERTa-v3-Large) and [Bespoke-MiniCheck-7B](https://huggingface.co/bespokelabs/Bespoke-MiniCheck-7B); exact revisions are in `config.json`. Obtain those models separately and follow their licenses. In particular, the pinned Bespoke model card specifies CC BY-NC 4.0. The Apache-2.0 license here covers the FactCascade router checkpoint and own code, and does not relicense upstream verifier models or datasets.

Only model parameters, own code, frozen configuration and aggregate validation metadata are released. No real dataset claim/evidence text, gold labels, dataset rows, saved real example scores, private paths or credentials are included. The usage example contains explicitly invented toy text and observations.