anesdevtoday/rosetta-flow-auditor-lora

rosetta-flow auditor adapters (Apertus 8B)

LoRA adapters for rosetta-flow, an open, on-premise auditor for AI-agent runs built with Apertus for Hack Apertus 2026 (Track 2B). The auditor reads an agent run like a code repository (each step is a file), investigates it with tools (read_spans, search, list_spans) and ends with a typed report whose evidence quotes are checked word for word against the cited steps.

These adapters are an honest experiment: fine-tuning made the 8B more reliable and faster, not more accurate. The shipped system uses the untrained 8B with the harness as screener and Apertus 70B as confirmer. Details, data and code are in the rosetta-flow repository and technical report.

Adapters

Folder	What	Training
full-2ep/	v1: SFT on 70B investigations	2 epochs, 4-bit QLoRA, bf16 compute, L4, 92 min
full-3ep-bf16/	v2: same data, unquantised	3 epochs, bf16, batch 4 x accum 2, A100, 38 min
full-3ep-bf16-analysis/	v3: v2 plus the 70B's rule-by-rule analysis before each report	3 epochs, bf16, A100, 47 min
rl-screener-v1/	GRPO single-pass screener (ok / issues + probability)	80 steps, 8 generations, lr 2e-5, 85 min

All: base andreasmartin/apertus-v1.5-8b-text (text-only Apertus v1.5 8B), LoRA rank 16, alpha 32, dropout 0.05, all linear projections, lr 1e-4 (SFT), max length 8,192, seed 0, one run per configuration. Each folder has train_log.json with the arguments and loss history.

Data: 414 balanced examples (207 with an injected error, 207 clean; refunds 240, manufacturing 174), kept by rejection sampling from Apertus 70B investigations of simulated agent runs (557 of 829 accepted). Loss only on the auditor's turns (~9 % of tokens; 23 % for v3).

Results (held-out test, 262 runs, balanced accuracy)

System	Refunds	Manufacturing	Failed audits
B1 untrained 8B + harness	0.62	0.54	20
B2 v2 adapter + harness	0.64	0.52	1
B3 Apertus 70B + harness	0.69	0.71	0

v2: failed audits 20 -> 1, verified evidence 0.60 -> 0.78, 2-3x faster, but more false alarms (0.39 -> 0.51). v3 on dev was too cautious (0.56 / 0.48). The GRPO screener did not beat the untrained 8B on dev (0.53 / 0.48 vs 0.54 / 0.48) and its probabilities were not calibrated. All under a small budget (one seed, no search); not proof that training cannot help.

Use

vllm serve andreasmartin/apertus-v1.5-8b-text --enable-lora --max-lora-rank 16 \
  --lora-modules auditor=<local path>/full-3ep-bf16/adapter \
  --chat-template-content-format string --enable-auto-tool-choice --tool-call-parser apertus

Then point rosetta-flow at it with LLM_BASE_URL=http://localhost:8000/v1 and LLM_NAME=auditor.

Limits

Trained only on simulated runs in two domains (refunds, manufacturing quality) and four languages. Not validated on real production traces. Use as a research artefact, with a human reviewing findings.

Licence

Apache-2.0, like Apertus. Base model: swiss-ai Apertus v1.5 (Apache-2.0), text-only conversion by FHNW.