Offlin33er/qwen25-coder-7b-solidity-audit-v3

Qwen2.5-Coder-7B — Solidity Audit Adapter V3 (Full Fallback Resolution)

V3 of the Solidity audit adapters, trained on Offlin33er/solidity-audit-instruct-v3 — the V2 dataset with a tier-3 verified mapping pass that reclassified residual fallback rows (406 → 52 of 2,472). Same base model, same QLoRA recipe (r=32, NF4, 1 epoch = 139 steps, final loss consistent with V2), same split boundaries.

Part of the Solidity Audit Scanner project.

Evaluation — V1 vs V2 vs V3 (identical held-out code, n=100 stratified, greedy, seed 42)

Metric	V1	V2	V3
Detection precision	1.00	0.986	0.986
Detection recall	1.00	0.971	0.986
Detection F1	1.00	0.978	0.986
False-positive rate on clean	0.00	0.033	0.033
Finding-type match	0.30 (lenient)	0.485 (exact canonical)	0.449 (exact canonical)

Full per-row records: eval_results.json.

The honest headline: remapping the fallback rows did NOT improve type labeling

• V3 type match 0.449 vs V2's 0.485 — a 3-row difference (31/69 vs 33/68), within sampling noise for n≈70. This is a null result, not a regression: label-noise reduction in the fallback tier did not measurably lift type matching.
• What did improve: detection — recall rose 0.971 → 0.986 and the single FN shifted (the clean-row false positive remained the same token-fee constructor, idx 29 in both runs).
• Reading: the binding constraint was never just label noise. The residual errors are semantic ambiguity in the reference labels themselves — e.g. the model calls an allowance-manipulation snippet "Arithmetic Error" where the source firm labeled it "reentrancy" (a defensible disagreement, not a clear miss). Fixing that needs better data (more snippets, multi-label annotations) rather than more relabeling.
• V3 remains the best detection config (F1 0.986) and has the cleanest labels; V2 remains the best type-match config by a whisker. Neither difference is statistically decisive at n=100.

Lineage

• V1: raw multi-firm labels → dataset v1, 30% type match
• V2: canonical 8-class taxonomy (406 fallback rows) → dataset v2, 48.5% type match
• V3: verified tier-3 fallback resolution (52 residual) → dataset v3, 44.9% type match, F1 0.986

Training tracked on the trackio dashboard.

Usage

Same snippet as V1/V2 with this adapter id; the system prompt must be the V3 taxonomy prompt from the dataset card (trained verbatim).

Limitations

Same as V2. Not a substitute for a professional audit. Defensive security tooling: analyzes code you paste in. Only audit contracts you are authorized to review.