AutoCyberAI/crp-prm-deberta-v1

CRP PRM DeBERTa — process reward model (step verifier)

Part of the Context Relay Protocol (CRP) ML-first governance layer. Scores whether a reasoning step is entailed by / consistent with its premises — VALID or INVALID — inside the CRP Verification Relay (crp/vr/prm.py, SPEC-049). Input format:

premises: <problem and prior steps> [SEP] step: <current step>

Verified results (independent harness, 2026-08-01)

Metric	Value
ROC AUC (389 held-out prm800k steps)	0.793
Curated reasoning cases (mixed domains)	8/10
VALID recall (threshold ≥ 0.15)	1.000 — never false-flags good steps
Best operating point (this slice)	t=0.1 → INVALID recall 0.577 / VALID recall 0.890

Trainer-side eval (on a training-matched mix): eval_loss 0.133, INVALID F1 0.977. See Calibration below before using thresholds.

Intended use — advisory scorer

Wire as an advisory step-quality score: it never false-halts good steps (VALID recall 1.0) and catches blatant bad ones, but subtle math-domain errors score low (score compression on out-of-training distribution). Hard INVALID gating should stay with symbolic verifiers / human checkpoints — exactly how crp/vr/prm.py consumes it.

Calibration (important)

The config's exported prm_threshold (0.675) was calibrated on a mix that includes the same synthetic patterns as training, and does not transfer to real distributions — on pure prm800k test steps, P(INVALID) compresses below 0.5 (mean 0.098 on true INVALID vs 0.053 on true VALID; ranking is real, absolute calibration is distribution-dependent). Recalibrate on your own traffic before thresholding, or use argmax (0.5).

Training data

~400k step-level examples (stratified ~50/50): trl-lib/prm800k (human labels), peiyi9979/Math-Shepherd (GPT-4 step labels), UW-Madison-Lee-Lab/MMLU-Pro-CoT-Train-Labeled (14 non-math domains), RLHFlow/Mistral-PRM-Data, plus 20k synthetic agentic (devops / compliance / code) reasoning examples. DeBERTa-v3-large, bf16, 2 epochs, lr 1e-5, RunPod RTX 4090.

Limitations

Not a substitute for formal/symbolic verification. Weakest on long, subtle multi-step math derivations (premises truncation at 384 tokens). English only. Scores are distribution-sensitive — see Calibration.

License

Elastic License 2.0 — see the CRP repository for details.