CONTRIBUTING.md

Contributing to frontier-check

What this project is

A small tool that routes claims by their training-data status. The core contribution is the 2×2 matrix over (mechanism, checkable sub-claim) that separates FRONTIER from FABRICATED. Everything else is plumbing.

What this project is not

• Not a general-purpose fact-checker.
• Not a replacement for the LLM that produces the claim.
• Not a proof that a FRONTIER claim is correct.
• Not calibrated against real reader outcomes yet.

How to propose a change

1. Open an issue describing the change and why the current behavior is wrong. Include a specific claim that routes incorrectly and the correct routing for it.
2. Add the claim to REGRESSION_CLAIMS with the expected status and verdict.
3. Confirm the new claim fails on the current code (pytest -q).
4. Fix the code.
5. Confirm all regression claims pass, including the new one.
6. Open the PR.

The regression suite is the spec. If your change is not covered by a regression claim, add one before touching the code.

What "the current behavior is wrong" means

The classifier is heuristic. It will route some claims incorrectly. The question is not "does it make mistakes" but "does it make the same mistakes reliably." A change is correct if:

• The eight existing regression claims still pass.
• The new claim now passes.
• The change does not introduce a mechanism that could route other claims incorrectly. (Predict what classes of claims the change affects, and add regression claims for those.)

Adding a theorem to the library

A theorem is a claim that no possible world makes the input correct. To add one:

1. It must be a real theorem. Rank bounds, cardinality bounds, entropy bounds, Euclidean angle sums. Not empirical regularities.
2. It must have a detector function that returns (violated, note).
3. The detector must be conservative — false positives (refusing a correct claim) are worse than false negatives.
4. Add regression claims for both the violating case and the near-miss case.

If you cannot state the theorem in one sentence and prove it in three, it does not belong in the library.

Adding a paper to the corpus

1. The paper must be real. No fabricated citations.
2. Set on_target=True only if the paper makes the same claim the corpus is testing. Sharing topic words is not enough.
3. The keywords list should be content tokens, not generic words. effective, rank, matrix are generic. dragonfly, tctn, matched are content.
4. Add a regression claim that exercises the new paper.

What not to contribute

• Changes that increase FRONTIER rate without adding structure. FRONTIER is a candidate, not a goal. If a change makes more claims qualify without a corresponding improvement in the structural checks, it is a regression.
• Changes that route unfamiliar claims to FABRICATED on the basis of a prior. The FABRICATED verdict requires structural evidence (no mechanism, no checkable sub-claim), not a prior toward dismissal.
• New dependencies for the core module. The core is stdlib only. Optional backends (arXiv retrieval, LLM classifier) live behind extras.

The version history

Read CHANGELOG.md before proposing a change. Three of four iterations had the same bug class: a gate that uses topical similarity as a proxy for claim identity. If your change touches retrieval or literature matching, check whether it introduces the same pattern.

Running the tests

pip install -e ".[dev]"
pytest -q
python -m frontier_check --strict "any claim"