TS-PRGuard v1, trained with typestrict.ai

.gitattributes

@@ -33,3 +33,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+figures/results.png filter=lfs diff=lfs merge=lfs -text

README.md

@@ -1,3 +1,183 @@
 ---
 license: apache-2.0
+library_name: onnx
+pipeline_tag: text-classification
+base_model: typestrict/TS-Ettin-150M
+tags:
+- typestrict
+- decision-model
+- typed-decisions
+- onnx
+- cpu
 ---
+
+<p align="center">
+  <a href="https://typestrict.ai/models/01a19e01-da22-729e-8d6c-4869efc4691e"><img src="https://huggingface.co/typestrict/TS-PRGuard/resolve/main/figures/banner.png" width="100%" alt="TS-PRGuard by typestrict.ai"></a>
+</p>
+
+<p align="center">
+  <a href="https://typestrict.ai"><b>Train your own on typestrict.ai</b></a> |
+  <a href="https://typestrict.ai/models/01a19e01-da22-729e-8d6c-4869efc4691e">Model page</a> |
+  <a href="https://docs.typestrict.ai">Docs</a> |
+  <a href="https://docs.typestrict.ai/agents">MCP for AI assistants</a> |
+  <a href="https://huggingface.co/typestrict">All TS models</a>
+  <br>
+  <b>License</b>: Apache 2.0 |
+  <b>Authors</b>: <a href="https://huggingface.co/typestrict">typestrict</a>, trained with <a href="https://typestrict.ai">typestrict.ai</a>
+</p>
+
+# TS-PRGuard
+
+A decision model trained on [typestrict.ai](https://typestrict.ai) that answers 5 questions in one pass, each with the probability of every option and a confidence, on a CPU.
+
+> [!TIP]
+> **Beats Jev on 4 of 5 questions**, the market reference model, on held-out examples never used in training, by the [refund calculation](https://docs.typestrict.ai/refund) (a model of several questions has to win on two thirds of them). Anyone can recompute the verdict from the [audit file](https://typestrict.ai/models/01a19e01-da22-729e-8d6c-4869efc4691e).
+
+- **Typed decisions**: each question is a `choice` (one option out of many), a `noul` (is this statement true) or a `score` (a value on an ordered rubric), with the probability of every option and a confidence.
+- **Small and fast**: 150M parameters, an 8-bit ONNX file that runs on a CPU, in a browser or on a phone.
+- **Measured, not claimed**: every number below comes from examples held out before training, and the audit file recomputes the verdict.
+
+<p align="center">
+  <img src="https://huggingface.co/typestrict/TS-PRGuard/resolve/main/figures/results.png" width="100%" alt="TS-PRGuard against Jev, question by question">
+</p>
+
+## 📊 Results
+
+Macro-F1 of this model and of Jev on the same held-out examples, and the gain the [refund calculation](https://docs.typestrict.ai/refund) measures (a weighted score of four metrics, with its 95% bootstrap interval). A question is won when the gain holds up under resampling (the interval stays above zero) and no metric collapses.
+
+<div class="dark:hidden" style="overflow-x:auto;">
+<table align="center" style="display:table;width:auto;max-width:100%;margin-left:auto;margin-right:auto;border-collapse:collapse;">
+<thead><tr><th scope="col" style="padding:12px;text-align:left;background:#1A1806;color:#FFD400;">Question</th><th scope="col" style="padding:12px;text-align:center;background:#1A1806;color:#FFD400;">Held-out examples</th><th scope="col" style="padding:12px;text-align:center;background:#1A1806;color:#FFD400;">This model</th><th scope="col" style="padding:12px;text-align:center;background:#1A1806;color:#FFD400;">Jev</th><th scope="col" style="padding:12px;text-align:center;background:#1A1806;color:#FFD400;">Gain over Jev (95% CI)</th><th scope="col" style="padding:12px;text-align:center;background:#1A1806;color:#FFD400;">Result</th></tr></thead>
+<tbody>
+<tr><th scope="row" style="padding:10px 12px;text-align:left;border-bottom:1px solid #E8E2C0;color:#1A1806;font-weight:700;background:#FFE766;">Does the code change do what the pull request description says?</th><td style="padding:10px 12px;text-align:center;border-bottom:1px solid #E8E2C0;color:#1A1806;font-weight:700;background:#FFE766;">436</td><td style="padding:10px 12px;text-align:center;border-bottom:1px solid #E8E2C0;color:#1A1806;font-weight:700;background:#FFE766;">0.928</td><td style="padding:10px 12px;text-align:center;border-bottom:1px solid #E8E2C0;color:#1A1806;font-weight:700;background:#FFE766;">0.807</td><td style="padding:10px 12px;text-align:center;border-bottom:1px solid #E8E2C0;color:#1A1806;font-weight:700;background:#FFE766;">+0.109 [+0.078; +0.141]</td><td style="padding:10px 12px;text-align:center;border-bottom:1px solid #E8E2C0;color:#1A1806;font-weight:700;background:#FFE766;">Beats Jev</td></tr>
+<tr><th scope="row" style="padding:10px 12px;text-align:left;border-bottom:1px solid #E8E2C0;color:#1A1806;font-weight:700;background:#FFE766;">How hard is this change to review?</th><td style="padding:10px 12px;text-align:center;border-bottom:1px solid #E8E2C0;color:#1A1806;font-weight:700;background:#FFE766;">221</td><td style="padding:10px 12px;text-align:center;border-bottom:1px solid #E8E2C0;color:#1A1806;font-weight:700;background:#FFE766;">0.660</td><td style="padding:10px 12px;text-align:center;border-bottom:1px solid #E8E2C0;color:#1A1806;font-weight:700;background:#FFE766;">0.383</td><td style="padding:10px 12px;text-align:center;border-bottom:1px solid #E8E2C0;color:#1A1806;font-weight:700;background:#FFE766;">+0.196 [+0.131; +0.248]</td><td style="padding:10px 12px;text-align:center;border-bottom:1px solid #E8E2C0;color:#1A1806;font-weight:700;background:#FFE766;">Beats Jev</td></tr>
+<tr><th scope="row" style="padding:10px 12px;text-align:left;border-bottom:1px solid #E8E2C0;color:#1A1806;font-weight:400;">Does this change break compatibility for the users of this code?</th><td style="padding:10px 12px;text-align:center;border-bottom:1px solid #E8E2C0;color:#1A1806;font-weight:400;">2,000</td><td style="padding:10px 12px;text-align:center;border-bottom:1px solid #E8E2C0;color:#1A1806;font-weight:400;">0.870</td><td style="padding:10px 12px;text-align:center;border-bottom:1px solid #E8E2C0;color:#1A1806;font-weight:400;">0.910</td><td style="padding:10px 12px;text-align:center;border-bottom:1px solid #E8E2C0;color:#1A1806;font-weight:400;">−0.037 [−0.048; −0.026]</td><td style="padding:10px 12px;text-align:center;border-bottom:1px solid #E8E2C0;color:#1A1806;font-weight:400;">Jev ahead</td></tr>
+<tr><th scope="row" style="padding:10px 12px;text-align:left;border-bottom:1px solid #E8E2C0;color:#1A1806;font-weight:700;background:#FFE766;">Do the tests in this change cover the changed code?</th><td style="padding:10px 12px;text-align:center;border-bottom:1px solid #E8E2C0;color:#1A1806;font-weight:700;background:#FFE766;">2,000</td><td style="padding:10px 12px;text-align:center;border-bottom:1px solid #E8E2C0;color:#1A1806;font-weight:700;background:#FFE766;">0.921</td><td style="padding:10px 12px;text-align:center;border-bottom:1px solid #E8E2C0;color:#1A1806;font-weight:700;background:#FFE766;">0.879</td><td style="padding:10px 12px;text-align:center;border-bottom:1px solid #E8E2C0;color:#1A1806;font-weight:700;background:#FFE766;">+0.036 [+0.025; +0.047]</td><td style="padding:10px 12px;text-align:center;border-bottom:1px solid #E8E2C0;color:#1A1806;font-weight:700;background:#FFE766;">Beats Jev</td></tr>
+<tr><th scope="row" style="padding:10px 12px;text-align:left;border-bottom:1px solid #E8E2C0;color:#1A1806;font-weight:700;background:#FFE766;">Does this code change introduce a security vulnerability?</th><td style="padding:10px 12px;text-align:center;border-bottom:1px solid #E8E2C0;color:#1A1806;font-weight:700;background:#FFE766;">597</td><td style="padding:10px 12px;text-align:center;border-bottom:1px solid #E8E2C0;color:#1A1806;font-weight:700;background:#FFE766;">0.904</td><td style="padding:10px 12px;text-align:center;border-bottom:1px solid #E8E2C0;color:#1A1806;font-weight:700;background:#FFE766;">0.755</td><td style="padding:10px 12px;text-align:center;border-bottom:1px solid #E8E2C0;color:#1A1806;font-weight:700;background:#FFE766;">+0.139 [+0.108; +0.168]</td><td style="padding:10px 12px;text-align:center;border-bottom:1px solid #E8E2C0;color:#1A1806;font-weight:700;background:#FFE766;">Beats Jev</td></tr>
+</tbody>
+</table>
+</div>
+<div class="hidden dark:block" style="overflow-x:auto;">
+<table align="center" style="display:table;width:auto;max-width:100%;margin-left:auto;margin-right:auto;border-collapse:collapse;">
+<thead><tr><th scope="col" style="padding:12px;text-align:left;background:#FFD400;color:#1A1806;">Question</th><th scope="col" style="padding:12px;text-align:center;background:#FFD400;color:#1A1806;">Held-out examples</th><th scope="col" style="padding:12px;text-align:center;background:#FFD400;color:#1A1806;">This model</th><th scope="col" style="padding:12px;text-align:center;background:#FFD400;color:#1A1806;">Jev</th><th scope="col" style="padding:12px;text-align:center;background:#FFD400;color:#1A1806;">Gain over Jev (95% CI)</th><th scope="col" style="padding:12px;text-align:center;background:#FFD400;color:#1A1806;">Result</th></tr></thead>
+<tbody>
+<tr><th scope="row" style="padding:10px 12px;text-align:left;border-bottom:1px solid #2E2B14;color:#FFD400;font-weight:700;background:#3D3300;">Does the code change do what the pull request description says?</th><td style="padding:10px 12px;text-align:center;border-bottom:1px solid #2E2B14;color:#FFD400;font-weight:700;background:#3D3300;">436</td><td style="padding:10px 12px;text-align:center;border-bottom:1px solid #2E2B14;color:#FFD400;font-weight:700;background:#3D3300;">0.928</td><td style="padding:10px 12px;text-align:center;border-bottom:1px solid #2E2B14;color:#FFD400;font-weight:700;background:#3D3300;">0.807</td><td style="padding:10px 12px;text-align:center;border-bottom:1px solid #2E2B14;color:#FFD400;font-weight:700;background:#3D3300;">+0.109 [+0.078; +0.141]</td><td style="padding:10px 12px;text-align:center;border-bottom:1px solid #2E2B14;color:#FFD400;font-weight:700;background:#3D3300;">Beats Jev</td></tr>
+<tr><th scope="row" style="padding:10px 12px;text-align:left;border-bottom:1px solid #2E2B14;color:#FFD400;font-weight:700;background:#3D3300;">How hard is this change to review?</th><td style="padding:10px 12px;text-align:center;border-bottom:1px solid #2E2B14;color:#FFD400;font-weight:700;background:#3D3300;">221</td><td style="padding:10px 12px;text-align:center;border-bottom:1px solid #2E2B14;color:#FFD400;font-weight:700;background:#3D3300;">0.660</td><td style="padding:10px 12px;text-align:center;border-bottom:1px solid #2E2B14;color:#FFD400;font-weight:700;background:#3D3300;">0.383</td><td style="padding:10px 12px;text-align:center;border-bottom:1px solid #2E2B14;color:#FFD400;font-weight:700;background:#3D3300;">+0.196 [+0.131; +0.248]</td><td style="padding:10px 12px;text-align:center;border-bottom:1px solid #2E2B14;color:#FFD400;font-weight:700;background:#3D3300;">Beats Jev</td></tr>
+<tr><th scope="row" style="padding:10px 12px;text-align:left;border-bottom:1px solid #2E2B14;color:#E8E4CF;font-weight:400;">Does this change break compatibility for the users of this code?</th><td style="padding:10px 12px;text-align:center;border-bottom:1px solid #2E2B14;color:#E8E4CF;font-weight:400;">2,000</td><td style="padding:10px 12px;text-align:center;border-bottom:1px solid #2E2B14;color:#E8E4CF;font-weight:400;">0.870</td><td style="padding:10px 12px;text-align:center;border-bottom:1px solid #2E2B14;color:#E8E4CF;font-weight:400;">0.910</td><td style="padding:10px 12px;text-align:center;border-bottom:1px solid #2E2B14;color:#E8E4CF;font-weight:400;">−0.037 [−0.048; −0.026]</td><td style="padding:10px 12px;text-align:center;border-bottom:1px solid #2E2B14;color:#E8E4CF;font-weight:400;">Jev ahead</td></tr>
+<tr><th scope="row" style="padding:10px 12px;text-align:left;border-bottom:1px solid #2E2B14;color:#FFD400;font-weight:700;background:#3D3300;">Do the tests in this change cover the changed code?</th><td style="padding:10px 12px;text-align:center;border-bottom:1px solid #2E2B14;color:#FFD400;font-weight:700;background:#3D3300;">2,000</td><td style="padding:10px 12px;text-align:center;border-bottom:1px solid #2E2B14;color:#FFD400;font-weight:700;background:#3D3300;">0.921</td><td style="padding:10px 12px;text-align:center;border-bottom:1px solid #2E2B14;color:#FFD400;font-weight:700;background:#3D3300;">0.879</td><td style="padding:10px 12px;text-align:center;border-bottom:1px solid #2E2B14;color:#FFD400;font-weight:700;background:#3D3300;">+0.036 [+0.025; +0.047]</td><td style="padding:10px 12px;text-align:center;border-bottom:1px solid #2E2B14;color:#FFD400;font-weight:700;background:#3D3300;">Beats Jev</td></tr>
+<tr><th scope="row" style="padding:10px 12px;text-align:left;border-bottom:1px solid #2E2B14;color:#FFD400;font-weight:700;background:#3D3300;">Does this code change introduce a security vulnerability?</th><td style="padding:10px 12px;text-align:center;border-bottom:1px solid #2E2B14;color:#FFD400;font-weight:700;background:#3D3300;">597</td><td style="padding:10px 12px;text-align:center;border-bottom:1px solid #2E2B14;color:#FFD400;font-weight:700;background:#3D3300;">0.904</td><td style="padding:10px 12px;text-align:center;border-bottom:1px solid #2E2B14;color:#FFD400;font-weight:700;background:#3D3300;">0.755</td><td style="padding:10px 12px;text-align:center;border-bottom:1px solid #2E2B14;color:#FFD400;font-weight:700;background:#3D3300;">+0.139 [+0.108; +0.168]</td><td style="padding:10px 12px;text-align:center;border-bottom:1px solid #2E2B14;color:#FFD400;font-weight:700;background:#3D3300;">Beats Jev</td></tr>
+</tbody>
+</table>
+</div>
+
+## ❓ The questions
+
+<div class="dark:hidden" style="overflow-x:auto;">
+<table align="center" style="display:table;width:auto;max-width:100%;margin-left:auto;margin-right:auto;border-collapse:collapse;">
+<thead><tr><th scope="col" style="padding:12px;text-align:left;background:#1A1806;color:#FFD400;">Question</th><th scope="col" style="padding:12px;text-align:center;background:#1A1806;color:#FFD400;">Type</th><th scope="col" style="padding:12px;text-align:left;background:#1A1806;color:#FFD400;">Options</th></tr></thead>
+<tbody>
+<tr><th scope="row" style="padding:10px 12px;text-align:left;border-bottom:1px solid #E8E2C0;color:#1A1806;font-weight:400;">Does the code change do what the pull request description says?</th><td style="padding:10px 12px;text-align:center;border-bottom:1px solid #E8E2C0;color:#1A1806;font-weight:400;"><code>noul</code></td><td style="padding:10px 12px;text-align:left;border-bottom:1px solid #E8E2C0;color:#1A1806;font-weight:400;">No, the diff does something different from the description.<br>Yes, the diff implements what the description says.</td></tr>
+<tr><th scope="row" style="padding:10px 12px;text-align:left;border-bottom:1px solid #E8E2C0;color:#1A1806;font-weight:400;">How hard is this change to review?</th><td style="padding:10px 12px;text-align:center;border-bottom:1px solid #E8E2C0;color:#1A1806;font-weight:400;"><code>choice</code></td><td style="padding:10px 12px;text-align:left;border-bottom:1px solid #E8E2C0;color:#1A1806;font-weight:400;">Trivial: a few lines in one place<br>Small: a short, focused change<br>Medium: several files or some new branching logic<br>Large: many lines or files, or a lot of new branching<br>Very large: hundreds of lines across many files with much new logic</td></tr>
+<tr><th scope="row" style="padding:10px 12px;text-align:left;border-bottom:1px solid #E8E2C0;color:#1A1806;font-weight:400;">Does this change break compatibility for the users of this code?</th><td style="padding:10px 12px;text-align:center;border-bottom:1px solid #E8E2C0;color:#1A1806;font-weight:400;"><code>noul</code></td><td style="padding:10px 12px;text-align:left;border-bottom:1px solid #E8E2C0;color:#1A1806;font-weight:400;">No, existing users keep working without changes.<br>Yes, existing users must change their code or configuration.</td></tr>
+<tr><th scope="row" style="padding:10px 12px;text-align:left;border-bottom:1px solid #E8E2C0;color:#1A1806;font-weight:400;">Do the tests in this change cover the changed code?</th><td style="padding:10px 12px;text-align:center;border-bottom:1px solid #E8E2C0;color:#1A1806;font-weight:400;"><code>noul</code></td><td style="padding:10px 12px;text-align:left;border-bottom:1px solid #E8E2C0;color:#1A1806;font-weight:400;">No, the change needs tests and the ones included do not cover it.<br>Yes, the tests in the change exercise the changed code.</td></tr>
+<tr><th scope="row" style="padding:10px 12px;text-align:left;border-bottom:1px solid #E8E2C0;color:#1A1806;font-weight:400;">Does this code change introduce a security vulnerability?</th><td style="padding:10px 12px;text-align:center;border-bottom:1px solid #E8E2C0;color:#1A1806;font-weight:400;"><code>noul</code></td><td style="padding:10px 12px;text-align:left;border-bottom:1px solid #E8E2C0;color:#1A1806;font-weight:400;">No, the change does not introduce a security vulnerability.<br>Yes, the change introduces a security vulnerability.</td></tr>
+</tbody>
+</table>
+</div>
+<div class="hidden dark:block" style="overflow-x:auto;">
+<table align="center" style="display:table;width:auto;max-width:100%;margin-left:auto;margin-right:auto;border-collapse:collapse;">
+<thead><tr><th scope="col" style="padding:12px;text-align:left;background:#FFD400;color:#1A1806;">Question</th><th scope="col" style="padding:12px;text-align:center;background:#FFD400;color:#1A1806;">Type</th><th scope="col" style="padding:12px;text-align:left;background:#FFD400;color:#1A1806;">Options</th></tr></thead>
+<tbody>
+<tr><th scope="row" style="padding:10px 12px;text-align:left;border-bottom:1px solid #2E2B14;color:#E8E4CF;font-weight:400;">Does the code change do what the pull request description says?</th><td style="padding:10px 12px;text-align:center;border-bottom:1px solid #2E2B14;color:#E8E4CF;font-weight:400;"><code>noul</code></td><td style="padding:10px 12px;text-align:left;border-bottom:1px solid #2E2B14;color:#E8E4CF;font-weight:400;">No, the diff does something different from the description.<br>Yes, the diff implements what the description says.</td></tr>
+<tr><th scope="row" style="padding:10px 12px;text-align:left;border-bottom:1px solid #2E2B14;color:#E8E4CF;font-weight:400;">How hard is this change to review?</th><td style="padding:10px 12px;text-align:center;border-bottom:1px solid #2E2B14;color:#E8E4CF;font-weight:400;"><code>choice</code></td><td style="padding:10px 12px;text-align:left;border-bottom:1px solid #2E2B14;color:#E8E4CF;font-weight:400;">Trivial: a few lines in one place<br>Small: a short, focused change<br>Medium: several files or some new branching logic<br>Large: many lines or files, or a lot of new branching<br>Very large: hundreds of lines across many files with much new logic</td></tr>
+<tr><th scope="row" style="padding:10px 12px;text-align:left;border-bottom:1px solid #2E2B14;color:#E8E4CF;font-weight:400;">Does this change break compatibility for the users of this code?</th><td style="padding:10px 12px;text-align:center;border-bottom:1px solid #2E2B14;color:#E8E4CF;font-weight:400;"><code>noul</code></td><td style="padding:10px 12px;text-align:left;border-bottom:1px solid #2E2B14;color:#E8E4CF;font-weight:400;">No, existing users keep working without changes.<br>Yes, existing users must change their code or configuration.</td></tr>
+<tr><th scope="row" style="padding:10px 12px;text-align:left;border-bottom:1px solid #2E2B14;color:#E8E4CF;font-weight:400;">Do the tests in this change cover the changed code?</th><td style="padding:10px 12px;text-align:center;border-bottom:1px solid #2E2B14;color:#E8E4CF;font-weight:400;"><code>noul</code></td><td style="padding:10px 12px;text-align:left;border-bottom:1px solid #2E2B14;color:#E8E4CF;font-weight:400;">No, the change needs tests and the ones included do not cover it.<br>Yes, the tests in the change exercise the changed code.</td></tr>
+<tr><th scope="row" style="padding:10px 12px;text-align:left;border-bottom:1px solid #2E2B14;color:#E8E4CF;font-weight:400;">Does this code change introduce a security vulnerability?</th><td style="padding:10px 12px;text-align:center;border-bottom:1px solid #2E2B14;color:#E8E4CF;font-weight:400;"><code>noul</code></td><td style="padding:10px 12px;text-align:left;border-bottom:1px solid #2E2B14;color:#E8E4CF;font-weight:400;">No, the change does not introduce a security vulnerability.<br>Yes, the change introduces a security vulnerability.</td></tr>
+</tbody>
+</table>
+</div>
+
+## 🏃 Use it
+
+`decision_ai.json` describes the package: the files, the inputs and outputs of the ONNX model, and each question with its options (`tasks`).
+
+```bash
+pip install huggingface_hub
+huggingface-cli download typestrict/TS-PRGuard --local-dir ts-prguard
+```
+
+- **Flutter (phone, browser):** the [decision_ai](https://pub.dev/packages/decision_ai) package reads this package as it is.
+- **Python, Node, any runtime:** ONNX Runtime with the inputs listed in `decision_ai.json`.
+- **Try it first:** the [playground](https://typestrict.ai/models/01a19e01-da22-729e-8d6c-4869efc4691e/playground) runs it in your browser.
+
+## 📚 Training data
+
+124,194 labeled examples; 10,389 of them held out from training to compare with Jev. The questions, the examples and the training run are on the [model page](https://typestrict.ai/models/01a19e01-da22-729e-8d6c-4869efc4691e).
+
+Sources and their licenses, as given by the owner:
+
+Built from the [TS-PRGuard dataset](https://huggingface.co/datasets/typestrict/TS-PRGuard) (labels under CC BY 4.0; every diff under its repository's license, kept with its `repo`, `ref` and `license`), plus more test-coverage examples from SWE-rebench-V2 and more pull requests labeled as breaking. Commercial use is allowed by every source.
+
+| Source | License | Questions |
+|---|---|---|
+| Git histories of permissively licensed repositories (MIT, Apache-2.0, BSD and similar) | Each repository's plexity, tests, vulnerability|
+| [bigcode/commitpack](https:bigcode/commitpack) | MIT(code under each repository's license) | description, complexity |
+| Pull requests labeled as brter pull requests labeled `c:API break` | Each repository's license (Flutter: BSD-3-Clause) | breaking change |
+| [nebius/SWE-rebench-V2](httts/nebius/SWE-rebench-V2) | CC BY 4.0 | tests |
+| [GitHub Advisory Database](advisory-database) | CC BY 4.0 | vulnerability |
+| [MoreFixes](https://zenodo. BY 4.0 | vulnerability |
+
+## 🚀 Train your own on typestrict.ai
+
+This model started from [TS-Ettin-150M](https://huggingface.co/typestrict/TS-Ettin-150M) and learned its questions from its owner's data on [typestrict.ai](https://typestrict.ai). Yours can too: send your data, review what the agent proposes, and get a model measured against Jev on your held-out examples. If it does not beat Jev, the credit comes back.
+
+**From your AI assistant** (ChatGPT, Claude, Cursor, Codex, Copilot and others), with the remote MCP server:
+
+```bash
+claude mcp add --transport http typestrict https://api.typestrict.ai/mcp
+```
+
+**From a terminal or CI**, with the [typestrict CLI](https://github.com/typestrict-lab/cli):
+
+```bash
+curl -fsSL https://typestrict.ai/install.sh | sh && typestrict login
+```
+
+## 🗒️ Model details
+
+<div class="dark:hidden" style="overflow-x:auto;">
+<table align="center" style="display:table;width:auto;max-width:100%;margin-left:auto;margin-right:auto;border-collapse:collapse;">
+<thead><tr><th scope="col" style="padding:12px;text-align:left;background:#1A1806;color:#FFD400;">Property</th><th scope="col" style="padding:12px;text-align:left;background:#1A1806;color:#FFD400;">TS-PRGuard</th></tr></thead>
+<tbody>
+<tr><th scope="row" style="padding:10px 12px;text-align:left;border-bottom:1px solid #E8E2C0;color:#1A1806;font-weight:400;">Base model</th><td style="padding:10px 12px;text-align:left;border-bottom:1px solid #E8E2C0;color:#1A1806;font-weight:400;"><a href="https://huggingface.co/typestrict/TS-Ettin-150M">TS-Ettin-150M</a> (Apache-2.0)</td></tr>
+<tr><th scope="row" style="padding:10px 12px;text-align:left;border-bottom:1px solid #E8E2C0;color:#1A1806;font-weight:400;">Parameters</th><td style="padding:10px 12px;text-align:left;border-bottom:1px solid #E8E2C0;color:#1A1806;font-weight:400;">150M</td></tr>
+<tr><th scope="row" style="padding:10px 12px;text-align:left;border-bottom:1px solid #E8E2C0;color:#1A1806;font-weight:400;">Languages</th><td style="padding:10px 12px;text-align:left;border-bottom:1px solid #E8E2C0;color:#1A1806;font-weight:400;">EN</td></tr>
+<tr><th scope="row" style="padding:10px 12px;text-align:left;border-bottom:1px solid #E8E2C0;color:#1A1806;font-weight:400;">Questions</th><td style="padding:10px 12px;text-align:left;border-bottom:1px solid #E8E2C0;color:#1A1806;font-weight:400;">5 (<code>noul</code>, <code>choice</code>)</td></tr>
+<tr><th scope="row" style="padding:10px 12px;text-align:left;border-bottom:1px solid #E8E2C0;color:#1A1806;font-weight:400;">Files</th><td style="padding:10px 12px;text-align:left;border-bottom:1px solid #E8E2C0;color:#1A1806;font-weight:400;"><code>decision_ai.json</code>, <code>tokenizer.json</code>, <code>onnx/model_8bit.onnx</code> (8-bit ONNX)</td></tr>
+<tr><th scope="row" style="padding:10px 12px;text-align:left;border-bottom:1px solid #E8E2C0;color:#1A1806;font-weight:400;">Runs on</th><td style="padding:10px 12px;text-align:left;border-bottom:1px solid #E8E2C0;color:#1A1806;font-weight:400;">CPU, in a browser or on a phone (ONNX Runtime)</td></tr>
+<tr><th scope="row" style="padding:10px 12px;text-align:left;border-bottom:1px solid #E8E2C0;color:#1A1806;font-weight:400;">Version</th><td style="padding:10px 12px;text-align:left;border-bottom:1px solid #E8E2C0;color:#1A1806;font-weight:400;">v1 on typestrict.ai</td></tr>
+<tr><th scope="row" style="padding:10px 12px;text-align:left;border-bottom:1px solid #E8E2C0;color:#1A1806;font-weight:400;">License</th><td style="padding:10px 12px;text-align:left;border-bottom:1px solid #E8E2C0;color:#1A1806;font-weight:400;">Apache-2.0</td></tr>
+</tbody>
+</table>
+</div>
+<div class="hidden dark:block" style="overflow-x:auto;">
+<table align="center" style="display:table;width:auto;max-width:100%;margin-left:auto;margin-right:auto;border-collapse:collapse;">
+<thead><tr><th scope="col" style="padding:12px;text-align:left;background:#FFD400;color:#1A1806;">Property</th><th scope="col" style="padding:12px;text-align:left;background:#FFD400;color:#1A1806;">TS-PRGuard</th></tr></thead>
+<tbody>
+<tr><th scope="row" style="padding:10px 12px;text-align:left;border-bottom:1px solid #2E2B14;color:#E8E4CF;font-weight:400;">Base model</th><td style="padding:10px 12px;text-align:left;border-bottom:1px solid #2E2B14;color:#E8E4CF;font-weight:400;"><a href="https://huggingface.co/typestrict/TS-Ettin-150M">TS-Ettin-150M</a> (Apache-2.0)</td></tr>
+<tr><th scope="row" style="padding:10px 12px;text-align:left;border-bottom:1px solid #2E2B14;color:#E8E4CF;font-weight:400;">Parameters</th><td style="padding:10px 12px;text-align:left;border-bottom:1px solid #2E2B14;color:#E8E4CF;font-weight:400;">150M</td></tr>
+<tr><th scope="row" style="padding:10px 12px;text-align:left;border-bottom:1px solid #2E2B14;color:#E8E4CF;font-weight:400;">Languages</th><td style="padding:10px 12px;text-align:left;border-bottom:1px solid #2E2B14;color:#E8E4CF;font-weight:400;">EN</td></tr>
+<tr><th scope="row" style="padding:10px 12px;text-align:left;border-bottom:1px solid #2E2B14;color:#E8E4CF;font-weight:400;">Questions</th><td style="padding:10px 12px;text-align:left;border-bottom:1px solid #2E2B14;color:#E8E4CF;font-weight:400;">5 (<code>noul</code>, <code>choice</code>)</td></tr>
+<tr><th scope="row" style="padding:10px 12px;text-align:left;border-bottom:1px solid #2E2B14;color:#E8E4CF;font-weight:400;">Files</th><td style="padding:10px 12px;text-align:left;border-bottom:1px solid #2E2B14;color:#E8E4CF;font-weight:400;"><code>decision_ai.json</code>, <code>tokenizer.json</code>, <code>onnx/model_8bit.onnx</code> (8-bit ONNX)</td></tr>
+<tr><th scope="row" style="padding:10px 12px;text-align:left;border-bottom:1px solid #2E2B14;color:#E8E4CF;font-weight:400;">Runs on</th><td style="padding:10px 12px;text-align:left;border-bottom:1px solid #2E2B14;color:#E8E4CF;font-weight:400;">CPU, in a browser or on a phone (ONNX Runtime)</td></tr>
+<tr><th scope="row" style="padding:10px 12px;text-align:left;border-bottom:1px solid #2E2B14;color:#E8E4CF;font-weight:400;">Version</th><td style="padding:10px 12px;text-align:left;border-bottom:1px solid #2E2B14;color:#E8E4CF;font-weight:400;">v1 on typestrict.ai</td></tr>
+<tr><th scope="row" style="padding:10px 12px;text-align:left;border-bottom:1px solid #2E2B14;color:#E8E4CF;font-weight:400;">License</th><td style="padding:10px 12px;text-align:left;border-bottom:1px solid #2E2B14;color:#E8E4CF;font-weight:400;">Apache-2.0</td></tr>
+</tbody>
+</table>
+</div>
+
+## Links
+
+- **Model page**: [typestrict.ai/models/01a19e01-da22-729e-8d6c-4869efc4691e](https://typestrict.ai/models/01a19e01-da22-729e-8d6c-4869efc4691e), with the data, the training run, the audit file and the playground
+- **typestrict.ai**: [typestrict.ai](https://typestrict.ai), the docs at [docs.typestrict.ai](https://docs.typestrict.ai)
+- **Open source**: [CLI and MCP server](https://github.com/typestrict-lab/cli), [decision_ai for Flutter](https://pub.dev/packages/decision_ai)

decision_ai.json

@@ -0,0 +1,96 @@
+{
+  "format": "decision-ai/option-reader@1",
+  "name": "typestrict multi-task model",
+  "license": "apache-2.0",
+  "files": {
+    "model": "onnx/model_8bit.onnx",
+    "tokenizer": "tokenizer.json"
+  },
+  "sha256": {
+    "onnx/model_8bit.onnx": "5f61cf8beeb8cdc2f2d112549f0a66ff42f75e05fc216f4c8e2d0766d9815891",
+    "tokenizer.json": "858fe1cd418807d6c71c01e24b6d3239a224a1e3c995472fef20841e09eab056"
+  },
+  "max_length": 7999,
+  "pad_multiple": 64,
+  "tokens": {
+    "cls": "[CLS]",
+    "sep": "[SEP]",
+    "pad": "[PAD]",
+    "option": "[OPT]"
+  },
+  "question_types": {
+    "noul": 0,
+    "choice": 1,
+    "score": 2
+  },
+  "noul_false_default": "A afirmação não se aplica.",
+  "inputs": {
+    "ids": "ids",
+    "pad_mask": "pad_mask",
+    "option_positions": "opt_positions",
+    "option_mask": "opt_mask",
+    "question_type": "qtype"
+  },
+  "outputs": {
+    "logits": "logits",
+    "confidence_logit": "conf_logit"
+  },
+  "calibration": {
+    "temperature": null,
+    "platt": null
+  },
+  "truncation": {
+    "max_length": 4096,
+    "max_criterion_tokens": 160
+  },
+  "tasks": [
+    {
+      "name": "pr-descricao-noul",
+      "kind": "noul",
+      "question": "Does the code change do what the pull request description says?",
+      "options": [
+        "No, the diff does something different from the description.",
+        "Yes, the diff implements what the description says."
+      ]
+    },
+    {
+      "name": "prg-complexidade",
+      "kind": "choice",
+      "question": "How hard is this change to review?",
+      "options": [
+        "Trivial: a few lines in one place",
+        "Small: a short, focused change",
+        "Medium: several files or some new branching logic",
+        "Large: many lines or files, or a lot of new branching",
+        "Very large: hundreds of lines across many files with much new logic"
+      ]
+    },
+    {
+      "name": "prg-quebra",
+      "kind": "noul",
+      "question": "Does this change break compatibility for the users of this code?",
+      "options": [
+        "No, existing users keep working without changes.",
+        "Yes, existing users must change their code or configuration."
+      ]
+    },
+    {
+      "name": "prg-testes",
+      "kind": "noul",
+      "question": "Do the tests in this change cover the changed code?",
+      "options": [
+        "No, the change needs tests and the ones included do not cover it.",
+        "Yes, the tests in the change exercise the changed code."
+      ]
+    },
+    {
+      "name": "prg-vul",
+      "kind": "noul",
+      "question": "Does this code change introduce a security vulnerability?",
+      "options": [
+        "No, the change does not introduce a security vulnerability.",
+        "Yes, the change introduces a security vulnerability."
+      ]
+    }
+  ]
+}

onnx/model_8bit.onnx

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5f61cf8beeb8cdc2f2d112549f0a66ff42f75e05fc216f4c8e2d0766d9815891
+size 164505784