FactCascade / docs /USAGE.md
nhnk's picture
Add model references, feature guide and executable selective-routing example
d91fdcc verified
|
Raw History Blame Contribute Delete
4.07 kB
# Using the frozen FactCascade router
## Download the release
Download this repository with the Hugging Face browser or CLI, then work from its root:
```bash
hf download nhnk/FactCascade --local-dir FactCascade
cd FactCascade
python -m pip install -r requirements.txt
python -B example.py
python -B examples/selective_replacement.py
```
The first example loads the existing trained JSON checkpoint. The second constructs synthetic31-slot features, routes two components under the frozen quota and replaces only selected rows. All verifier observations/text in that example are invented; no native verifier, real benchmark, gold label, fit or quality measurement is executed.
The original router/config/code can also be retrieved at weight-release revision `08bd580bb9f81908295354a678b4c048ac72e40a`. Documentation/examples were extended in later commits while keeping those bytes unchanged. Use a pinned release revision when reproducing a workflow.
## Supply the required observations
For each row, obtain canonical selected-packet FT5, full-packet FT5 and B192 support scores using the native frozen adapters. Provide claim text,selected/full evidence,chunk counts,selected-source count,coverage ratio and modality. See [FEATURES.md](FEATURES.md) and `factcascade/features.py`.
Scores must be finite support observations in[0,1]; they are not arbitrary logits. FT5 segmentation/full-packet aggregation, geometry/source-coverage definitions and component membership must match the intended source protocol. `build_features(adapter="external")` and `adapter="dev"` retain different historical conventions for one extra-feature disagreement signal; use the exact appropriate adapter. This repo does not reconstruct evidence packets or run upstream verifier inference.
```python
import numpy as np
from factcascade.features import build_features
from factcascade.utility_model import UtilityModel
from factcascade.routing import route_batch, apply_replacement
# observations is a caller-supplied list of canonical score/geometry/text mappings.
X = np.vstack([build_features(**row) for row in observations])
model = UtilityModel.load("router.json")
routing = route_batch(
selected_ft5_scores=selected_scores,
deberta_scores=deberta_scores,
batch_ids=component_ids,
features=X, utility_model=model, row_keys=stable_row_keys,
)
# Caller executes the pinned native Bespoke verifier ONLY on selected indices.
indices = np.flatnonzero(routing.route_mask)
bespoke_scores = np.full(len(X), np.nan)
bespoke_scores[indices] = run_native_bespoke_for_indices(indices)
predictions = apply_replacement(selected_scores, bespoke_scores, routing.route_mask)
```
The second snippet is an integration sketch;caller-supplied observations and `run_native_bespoke_for_indices` are placeholders,not an included inference service. The executable self-contained synthetic version is `examples/selective_replacement.py`.
## Allocation and replacement semantics
Ranking is batch/component-relative. Equal utility/disagreement midranks are averaged,then top-k selected with integer half-up quota920/3337 and stable ascending row-key tie breaks. Changing component boundaries,online window size or tie keys changes routes. This is batch allocation,not independent per-row online threshold routing.
Unrouted decisions remain `selected_FT5_score>0.763`. Routed decisions become `Bespoke_score>0.5`. Unrouted expensive scores may beNaN; the API does not require all-row Bespoke scoring. Selected rows may still receive harmful reversals; selection does not certify correctness.
## Reproduction scope
The released weights come from the historical unweighted signed-correctness target and full-DEV fit. They are not newly fitted diagnostics. Synthetically verified API/serialization behavior is not additional generalization evidence. No real examples/labels/verifierweights or complete inference environment are published;dataset licenses and upstreammodel terms remain separate. This lightweight router release is distinct from the cached reviewer reproduction capsule.