--- license: apache-2.0 library_name: onnx pipeline_tag: text-classification base_model: typestrict/TS-Ettin-150M tags: - typestrict - decision-model - typed-decisions - onnx - cpu ---

# TS-PRGuard A decision model trained on [typestrict.ai](https://typestrict.ai) that answers 5 questions in one pass, each with the probability of every option and a confidence, on a CPU. > [!TIP] > **Beats Jev on 4 of 5 questions**, the market reference model, on held-out examples never used in training, by the [refund calculation](https://docs.typestrict.ai/refund) (a model of several questions has to win on two thirds of them). Anyone can recompute the verdict from the [audit file](https://typestrict.ai/models/01a19e01-da22-729e-8d6c-4869efc4691e). - **Typed decisions**: each question is a `choice` (one option out of many), a `noul` (is this statement true) or a `score` (a value on an ordered rubric), with the probability of every option and a confidence. - **Small and fast**: 150M parameters, an 8-bit ONNX file that runs on a CPU, in a browser or on a phone. - **Measured, not claimed**: every number below comes from examples held out before training, and the audit file recomputes the verdict.

TS-PRGuard against Jev, question by question

## 📊 Results Macro-F1 of this model and of Jev on the same held-out examples, and the gain the [refund calculation](https://docs.typestrict.ai/refund) measures (a weighted score of four metrics, with its 95% bootstrap interval). A question is won when the gain holds up under resampling (the interval stays above zero) and no metric collapses.

Question	Held-out examples	This model	Jev	Gain over Jev (95% CI)	Result
Does the code change do what the pull request description says?	436	0.928	0.807	+0.109 [+0.078; +0.141]	Beats Jev
How hard is this change to review?	221	0.660	0.383	+0.196 [+0.131; +0.248]	Beats Jev
Does this change break compatibility for the users of this code?	2,000	0.870	0.910	−0.037 [−0.048; −0.026]	Jev ahead
Do the tests in this change cover the changed code?	2,000	0.921	0.879	+0.036 [+0.025; +0.047]	Beats Jev
Does this code change introduce a security vulnerability?	597	0.904	0.755	+0.139 [+0.108; +0.168]	Beats Jev

Question	Held-out examples	This model	Jev	Gain over Jev (95% CI)	Result
Does the code change do what the pull request description says?	436	0.928	0.807	+0.109 [+0.078; +0.141]	Beats Jev
How hard is this change to review?	221	0.660	0.383	+0.196 [+0.131; +0.248]	Beats Jev
Does this change break compatibility for the users of this code?	2,000	0.870	0.910	−0.037 [−0.048; −0.026]	Jev ahead
Do the tests in this change cover the changed code?	2,000	0.921	0.879	+0.036 [+0.025; +0.047]	Beats Jev
Does this code change introduce a security vulnerability?	597	0.904	0.755	+0.139 [+0.108; +0.168]	Beats Jev

## ❓ The questions

Question	Type	Options
Does the code change do what the pull request description says?	`noul`	No, the diff does something different from the description. Yes, the diff implements what the description says.
How hard is this change to review?	`choice`	Trivial: a few lines in one place Small: a short, focused change Medium: several files or some new branching logic Large: many lines or files, or a lot of new branching Very large: hundreds of lines across many files with much new logic
Does this change break compatibility for the users of this code?	`noul`	No, existing users keep working without changes. Yes, existing users must change their code or configuration.
Do the tests in this change cover the changed code?	`noul`	No, the change needs tests and the ones included do not cover it. Yes, the tests in the change exercise the changed code.
Does this code change introduce a security vulnerability?	`noul`	No, the change does not introduce a security vulnerability. Yes, the change introduces a security vulnerability.

Question	Type	Options
Does the code change do what the pull request description says?	`noul`	No, the diff does something different from the description. Yes, the diff implements what the description says.
How hard is this change to review?	`choice`	Trivial: a few lines in one place Small: a short, focused change Medium: several files or some new branching logic Large: many lines or files, or a lot of new branching Very large: hundreds of lines across many files with much new logic
Does this change break compatibility for the users of this code?	`noul`	No, existing users keep working without changes. Yes, existing users must change their code or configuration.
Do the tests in this change cover the changed code?	`noul`	No, the change needs tests and the ones included do not cover it. Yes, the tests in the change exercise the changed code.
Does this code change introduce a security vulnerability?	`noul`	No, the change does not introduce a security vulnerability. Yes, the change introduces a security vulnerability.

## 🏃 Use it `decision_ai.json` describes the package: the files, the inputs and outputs of the ONNX model, and each question with its options (`tasks`). ```bash pip install huggingface_hub huggingface-cli download typestrict/TS-PRGuard --local-dir ts-prguard ``` - **Flutter (phone, browser):** the [decision_ai](https://pub.dev/packages/decision_ai) package reads this package as it is. - **Python, Node, any runtime:** ONNX Runtime with the inputs listed in `decision_ai.json`. - **Try it first:** the [playground](https://typestrict.ai/models/01a19e01-da22-729e-8d6c-4869efc4691e/playground) runs it in your browser. ## 📚 Training data 124,194 labeled examples; 10,389 of them held out from training to compare with Jev. The questions, the examples and the training run are on the [model page](https://typestrict.ai/models/01a19e01-da22-729e-8d6c-4869efc4691e). Sources and their licenses, as given by the owner: Built from the [TS-PRGuard dataset](https://huggingface.co/datasets/typestrict/TS-PRGuard) (labels under CC BY 4.0; every diff under its repository's license, kept with its `repo`, `ref` and `license`), plus more test-coverage examples from SWE-rebench-V2 and more pull requests labeled as breaking. Commercial use is allowed by every source. | Source | License | Questions | |---|---|---| | Git histories of permissively licensed repositories (MIT, Apache-2.0, BSD and similar) | Each repository's plexity, tests, vulnerability| | [bigcode/commitpack](https:bigcode/commitpack) | MIT(code under each repository's license) | description, complexity | | Pull requests labeled as brter pull requests labeled `c:API break` | Each repository's license (Flutter: BSD-3-Clause) | breaking change | | [nebius/SWE-rebench-V2](httts/nebius/SWE-rebench-V2) | CC BY 4.0 | tests | | [GitHub Advisory Database](advisory-database) | CC BY 4.0 | vulnerability | | [MoreFixes](https://zenodo. BY 4.0 | vulnerability | ## 🚀 Train your own on typestrict.ai This model started from [TS-Ettin-150M](https://huggingface.co/typestrict/TS-Ettin-150M) and learned its questions from its owner's data on [typestrict.ai](https://typestrict.ai). Yours can too: send your data, review what the agent proposes, and get a model measured against Jev on your held-out examples. If it does not beat Jev, the credit comes back. **From your AI assistant** (ChatGPT, Claude, Cursor, Codex, Copilot and others), with the remote MCP server: ```bash claude mcp add --transport http typestrict https://api.typestrict.ai/mcp ``` **From a terminal or CI**, with the [typestrict CLI](https://github.com/typestrict-lab/cli): ```bash curl -fsSL https://typestrict.ai/install.sh | sh && typestrict login ``` ## 🗒️ Model details

Property	TS-PRGuard
Base model	TS-Ettin-150M (Apache-2.0)
Parameters	150M
Languages	EN
Questions	5 (`noul`, `choice`)
Files	`decision_ai.json`, `tokenizer.json`, `onnx/model_8bit.onnx` (8-bit ONNX)
Runs on	CPU, in a browser or on a phone (ONNX Runtime)
Version	v1 on typestrict.ai
License	Apache-2.0

Property	TS-PRGuard
Base model	TS-Ettin-150M (Apache-2.0)
Parameters	150M
Languages	EN
Questions	5 (`noul`, `choice`)
Files	`decision_ai.json`, `tokenizer.json`, `onnx/model_8bit.onnx` (8-bit ONNX)
Runs on	CPU, in a browser or on a phone (ONNX Runtime)
Version	v1 on typestrict.ai
License	Apache-2.0

## Links - **Model page**: [typestrict.ai/models/01a19e01-da22-729e-8d6c-4869efc4691e](https://typestrict.ai/models/01a19e01-da22-729e-8d6c-4869efc4691e), with the data, the training run, the audit file and the playground - **typestrict.ai**: [typestrict.ai](https://typestrict.ai), the docs at [docs.typestrict.ai](https://docs.typestrict.ai) - **Open source**: [CLI and MCP server](https://github.com/typestrict-lab/cli), [decision_ai for Flutter](https://pub.dev/packages/decision_ai)