TWO UNRELATED THINGS THE LAST RED PIPELINE TAUGHT, one of which was not a
bug in this repository at all.
THE PIPELINE DID NOT FAIL ON A TEST. Job 16530892473 never ran a line of
script: the runner could not reach GitLab's own CI gateway, spent 130 seconds
on a connect timeout, and git exited 128. The API reports that as
`script_failure`, which is how it read as a code regression at the exact commit
that reworked the banner -- it was a coincidence, and `ascii` was simply the
job that drew the short straw. GET_SOURCES_ATTEMPTS makes the runner retry the
fetch, which is the narrow fix; `retry:when: script_failure` would also re-run
genuinely failing tests until they passed once.
THE FALLBACK WAS NEVER EXERCISED. Chasing the above turned up the real defect.
_enable_utf8 calls stdout.reconfigure(encoding="utf-8"), which SUCCEEDS on a
pipe whatever PYTHONIOENCODING asked for -- so UNICODE was always True, the
`ascii` job spent its whole life on the UTF-8 branch it was written to avoid,
and…
Four stacks become one. The TypeScript and JavaScript rules are removed, the
eight plain-Python rules are composed into the Odoo stack, and a file in any
other language is refused by name rather than reviewed against rules that
cannot match it.
WHY REMOVE RATHER THAN LABEL. Firing the other twenty-five rules at real code
decided it: JavaScript classified out at 21 true against 22 false, TypeScript
was measured at 3.8/kloc and never classified. A rule that costs nothing at
query time is not free -- it is a claim the README carries and a file `check`
accepts and then says nothing useful about. An empty finding list reads as a
clean bill of health, so `subject.Unreviewable` refuses the file instead.
WHY COMPOSE RATHER THAN DELETE. Selection was strict equality on `stack`, so an
Odoo module got the 19 ORM rules and none of the 8 Python ones -- and Odoo
modules are Python. `fieldtest --stack python` over OpenSPP found 32 hits
`--stack odoo` never reported. "ruff covers it" does not hold a…
evidence: prune by the time in the bundle, not the time on the inode
CI went red on a test that had passed on this machine every time: eight
bundles written with NEXA_EVIDENCE_KEEP=3, and the three left were t-03, t-04
and t-06. Not the newest three, and not a stable wrong answer either.
_prune ranked bundles with os.path.getmtime. Linux stamps inodes from a clock
that only moves once a tick, so all eight writes carried one identical mtime.
sort() is stable, so the tie fell through to os.listdir order -- hash order on
ext4 -- and the tail it unlinked was an arbitrary subset. That run deleted
t-07, the bundle write() had that moment written, and write() returned its
path regardless: a run printing an evidence id for a file already gone, which
is the promise _keep_count already had to stop NEXA_EVIDENCE_KEEP=0 from
breaking. Windows stamps at sub-millisecond resolution, which is why no
version of this has ever failed here.
The bundle carries its own time at microsecond resolution, and it is the time
the listing already shows people. _written_at reads it…
review: a pass over the tool itself, and the silence it was keeping
The corpus has been measured from every angle; the code running it had not
been read once. The worst thing here was not a wrong answer but an absent
one.
A SYNTAX ERROR ANYWHERE ERASED EVERY AST FINDING. Each semantic checker
parses for itself and returns [] when that fails, so nine of them agreed
there was nothing to say and `check` printed "no checker findings" and
exited 0. A file with a real stale-compute bug reported it; the same file
with a trailing `def oops(:` reported nothing. scan_source now parses once
and returns a gap with the line number in it.
@api.depends DID NOT COVER ITS OWN PREFIX. Declaring 'line_ids.price'
registers the whole chain, so a body reading rec.line_ids needs nothing
more. Exact string matching called that stale: 42 of the rule's 104 hits
on OpenSPP, 40% of its output. Its own `correct` sample hid it by
declaring the prefix redundantly, which is why validate never saw it.
THE OTHER 25 RULES HAD NEVER BEEN FIRED AT REAL CODE. Every committed
artifact wa…
empirical: check what a snippet reaches for, not what it binds
NEXA_EMPIRICAL_EXEC was on the list of documented-but-never-set variables and
I missed it on the first pass. It is the one that makes Nexa run model-written
code, so it was the one most worth setting. Two defects, in opposite
directions, from the same root:
from decimal import Decimal binds Decimal, reaches decimal
import os as math binds math, reaches os
_try_execute asked semantics.imported_names(), which answers "what does this
module BIND" -- exactly right for resolving an undefined name, and the wrong
question for an allowlist, which needs "what does this REACH FOR".
So every `from X import Y` was refused however safe X was, making the feature
far narrower than the comment above _EXEC_SAFE_MODULES describes. And `import
os as math` bound a name that IS on the allowlist, so it would have been
executed.
layers.py is explicit that the subprocess is not a security boundary and this
does not make it one. But "snippets are only ever executed when they reach…
tests: the environment matrix, and the two coverage gaps
Environment variables. Seven were documented and none had ever been set:
NEXA_EVIDENCE, NEXA_EVIDENCE_DIR, NEXA_EVIDENCE_KEEP, LOCALAI_TELEMETRY,
LOCALAI_TELEMETRY_CONTENT, LOCALAI_TELEMETRY_FILE, NO_COLOR/TERM, plus
OLLAMA_HOST. Documenting a switch nobody has flipped is a promise, not a
feature. NEXA_EVIDENCE_KEEP defaults to 200, so its pruning branch had never
executed at all; it runs here at 3.
The colour opt-outs are exercised through a real subprocess, because COLOR is
decided at import against sys.stdout and an in-process call would be
measuring pytest's capture rather than a terminal. Same for the ASCII banner:
a cp437 stdout does not reproduce it, since stdout.reconfigure re-encodes a
pipe happily, so the test replaces the stream with one that refuses -- which
is the situation the fallback is actually for.
Coverage, the two gaps named in the open items:
bin/corpus.py 24% -> 62% the ask path, Modelfiles, prompt,
coverage, stats, …
