readme: the 49/57 in the core sample table is OpenSPP's
Browse filesREADME's core sample table gave stored-compute-needs-complete-depends a
previous figure of "49/57 OCA". 49 true of 57 decided is OpenSPP's
reading (classification-openspp-2026-09.json: 49 true, 8 false,
3 undecidable), the same numbers README's second-ecosystem table prints.
OCA's was 41 of 41 decided when the row was written in 210b09e, and is
42 of 42 since 03128a2. The row now says OpenSPP. ARCHITECTURE 3.52 is
dated, so it gets a corrected note under its table rather than a rewrite.
Four comments still gave the corpus as 27 rules, which it has not been
since e9d9bcb: bin/context.py, bin/subject.py and two in bin/eval.py.
They lose the number rather than gain a new one, since nothing keeps it
current. eval.py's retrievability docstring also said nine queries at
top 6 could not have reached most of the corpus; nine times six is 54
places for 27 rules, so it now says what is true, that the sample is
too small to say anything about a rule that did not appear.
794 passed, 2 skipped; every marked figure agrees.
- ARCHITECTURE.md +5 -0
- README.md +1 -1
- bin/context.py +1 -1
- bin/eval.py +4 -4
- bin/subject.py +1 -1
|
@@ -2548,6 +2548,11 @@ making 70% of them, in `classification-odoo-core-2026-09.json`.
|
|
| 2548 |
| `assert-is-not-validation` | **3/15** | 6/10 OCA | 7–45% |
|
| 2549 |
| `sudo-bypasses-record-rules` | **0/15** | 6 true on OpenSPP | 0–20% |
|
| 2550 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 2551 |
**The finding is that precision depends on what kind of code is under review, and this
|
| 2552 |
is the first tree here that made that legible.** Two rules hold up on core. Two collapse,
|
| 2553 |
and both collapse for the same reason rather than because the checkers are broken.
|
|
|
|
| 2548 |
| `assert-is-not-validation` | **3/15** | 6/10 OCA | 7–45% |
|
| 2549 |
| `sudo-bypasses-record-rules` | **0/15** | 6 true on OpenSPP | 0–20% |
|
| 2550 |
|
| 2551 |
+
Corrected 2026-09-27: the 49/57 in the `stored-compute-needs-complete-depends` row is
|
| 2552 |
+
OpenSPP's reading, not OCA's (`classification-openspp-2026-09.json`: 49 true, 8 false,
|
| 2553 |
+
3 undecidable). OCA's was 41 true of 41 decided when this was written, and 42 of 42 since
|
| 2554 |
+
§3.71.
|
| 2555 |
+
|
| 2556 |
**The finding is that precision depends on what kind of code is under review, and this
|
| 2557 |
is the first tree here that made that legible.** Two rules hold up on core. Two collapse,
|
| 2558 |
and both collapse for the same reason rather than because the checkers are broken.
|
|
@@ -1055,7 +1055,7 @@ are now classified across the four rules that make 70% of them:
|
|
| 1055 |
| rule | core | previously measured |
|
| 1056 |
|---|---|---|
|
| 1057 |
| `ondelete-is-a-decision` | **13/15** | 12/12 OCA, 14/14 OpenSPP |
|
| 1058 |
-
| `stored-compute-needs-complete-depends` | **11/15** | 49/57
|
| 1059 |
| `assert-is-not-validation` | **3/15** | 6/10 OCA |
|
| 1060 |
| `sudo-bypasses-record-rules` | **0/15** | 6 true on OpenSPP |
|
| 1061 |
| `constrains-must-list-every-field-read` | **<!--fig:cls.odoo-core.constrains-must-list-every-field-read.true-->10<!--/fig-->/15** | <!--fig:cls.openspp.constrains-must-list-every-field-read.true-->38<!--/fig-->/<!--fig:cls.openspp.constrains-must-list-every-field-read.rows-->45<!--/fig--> OpenSPP, <!--fig:cls.oca.constrains-must-list-every-field-read.true-->8<!--/fig-->/<!--fig:cls.oca.constrains-must-list-every-field-read.rows-->8<!--/fig--> OCA |
|
|
|
|
| 1055 |
| rule | core | previously measured |
|
| 1056 |
|---|---|---|
|
| 1057 |
| `ondelete-is-a-decision` | **13/15** | 12/12 OCA, 14/14 OpenSPP |
|
| 1058 |
+
| `stored-compute-needs-complete-depends` | **11/15** | 49/57 OpenSPP |
|
| 1059 |
| `assert-is-not-validation` | **3/15** | 6/10 OCA |
|
| 1060 |
| `sudo-bypasses-record-rules` | **0/15** | 6 true on OpenSPP |
|
| 1061 |
| `constrains-must-list-every-field-read` | **<!--fig:cls.odoo-core.constrains-must-list-every-field-read.true-->10<!--/fig-->/15** | <!--fig:cls.openspp.constrains-must-list-every-field-read.true-->38<!--/fig-->/<!--fig:cls.openspp.constrains-must-list-every-field-read.rows-->45<!--/fig--> OpenSPP, <!--fig:cls.oca.constrains-must-list-every-field-read.true-->8<!--/fig-->/<!--fig:cls.oca.constrains-must-list-every-field-read.rows-->8<!--/fig--> OCA |
|
|
@@ -68,7 +68,7 @@ _ODOO_IMPORT_RE = re.compile(r"^\s*from\s+odoo[\w.]*\s+import\s+(.+)$", re.M)
|
|
| 68 |
#
|
| 69 |
# .py is where every rule but one lives; .xml carries the noupdate rule, whose
|
| 70 |
# violation is a <record> in a data file. Anything else is refused by name
|
| 71 |
-
# rather than reviewed:
|
| 72 |
# nothing reads as a clean bill of health.
|
| 73 |
REVIEWABLE = (".py", ".xml")
|
| 74 |
|
|
|
|
| 68 |
#
|
| 69 |
# .py is where every rule but one lives; .xml carries the noupdate rule, whose
|
| 70 |
# violation is a <record> in a data file. Anything else is refused by name
|
| 71 |
+
# rather than reviewed: the Odoo rules run over a .ts file find nothing, and
|
| 72 |
# nothing reads as a clean bill of health.
|
| 73 |
REVIEWABLE = (".py", ".xml")
|
| 74 |
|
|
@@ -70,7 +70,7 @@ DEFAULT_QUESTIONS = [
|
|
| 70 |
("odoo", "compare two prices for equality"),
|
| 71 |
# The plain-Python half of the corpus, reached through Odoo-shaped
|
| 72 |
# questions because that is where these rules are now selected from: an
|
| 73 |
-
# eval that never asks one measures
|
| 74 |
("odoo", "check an amount is positive before posting a payment"),
|
| 75 |
("odoo", "store a total in a monetary field without losing a penny"),
|
| 76 |
("odoo", "compare a date typed by a user against the current time"),
|
|
@@ -179,9 +179,9 @@ def retrievability(stack=None, k=None):
|
|
| 179 |
This is the measurement `nexa report rules` cannot make. Telemetry can only
|
| 180 |
say what WAS retrieved by the questions somebody happened to ask, and a rule
|
| 181 |
absent from that list may simply never have had a relevant question put to
|
| 182 |
-
it. Nine recorded queries against a TOP_K of 6
|
| 183 |
-
|
| 184 |
-
|
| 185 |
|
| 186 |
A probe is a question aimed at one rule. If the rule does not come back for
|
| 187 |
it, retrieval has genuinely missed it, and the rule is unreachable no matter
|
|
|
|
| 70 |
("odoo", "compare two prices for equality"),
|
| 71 |
# The plain-Python half of the corpus, reached through Odoo-shaped
|
| 72 |
# questions because that is where these rules are now selected from: an
|
| 73 |
+
# eval that never asks one measures the ORM rules alone.
|
| 74 |
("odoo", "check an amount is positive before posting a payment"),
|
| 75 |
("odoo", "store a total in a monetary field without losing a penny"),
|
| 76 |
("odoo", "compare a date typed by a user against the current time"),
|
|
|
|
| 179 |
This is the measurement `nexa report rules` cannot make. Telemetry can only
|
| 180 |
say what WAS retrieved by the questions somebody happened to ask, and a rule
|
| 181 |
absent from that list may simply never have had a relevant question put to
|
| 182 |
+
it. Nine recorded queries against a TOP_K of 6 is a sample too small to say
|
| 183 |
+
anything about a rule that did not appear, however well it was written --
|
| 184 |
+
arithmetic, not evidence about the rules.
|
| 185 |
|
| 186 |
A probe is a question aimed at one rule. If the rule does not come back for
|
| 187 |
it, retrieval has genuinely missed it, and the rule is unreachable no matter
|
|
@@ -45,7 +45,7 @@ STACKS = ("odoo",)
|
|
| 45 |
class Unreviewable(Exception):
|
| 46 |
"""The file is in a language this corpus has no rules for.
|
| 47 |
|
| 48 |
-
Refusing is the point. Running
|
| 49 |
empty finding list, and an empty finding list is read as "nothing wrong
|
| 50 |
here" -- so silence about a file nobody checked would be the one lie the
|
| 51 |
tool is built not to tell.
|
|
|
|
| 45 |
class Unreviewable(Exception):
|
| 46 |
"""The file is in a language this corpus has no rules for.
|
| 47 |
|
| 48 |
+
Refusing is the point. Running the Odoo rules over a .ts file produces an
|
| 49 |
empty finding list, and an empty finding list is read as "nothing wrong
|
| 50 |
here" -- so silence about a file nobody checked would be the one lie the
|
| 51 |
tool is built not to tell.
|