| --- |
| license: apache-2.0 |
| base_model: andreasmartin/apertus-v1.5-8b-text |
| library_name: peft |
| language: [de, fr, it, en] |
| tags: [apertus, lora, agent-audit, tool-calling, hack-apertus] |
| --- |
| |
| # rosetta-flow auditor adapters (Apertus 8B) |
|
|
| LoRA adapters for **rosetta-flow**, an open, on-premise auditor for AI-agent runs built with Apertus for |
| Hack Apertus 2026 (Track 2B). The auditor reads an agent run like a code repository (each step is a file), investigates |
| it with tools (`read_spans`, `search`, `list_spans`) and ends with a typed `report` whose evidence quotes are checked |
| word for word against the cited steps. |
|
|
| These adapters are an **honest experiment**: fine-tuning made the 8B more reliable and faster, not more accurate. The |
| shipped system uses the untrained 8B with the harness as screener and Apertus 70B as confirmer. Details, data and code |
| are in the rosetta-flow repository and technical report. |
|
|
| ## Adapters |
|
|
| | Folder | What | Training | |
| |---|---|---| |
| | `full-2ep/` | v1: SFT on 70B investigations | 2 epochs, 4-bit QLoRA, bf16 compute, L4, 92 min | |
| | `full-3ep-bf16/` | **v2**: same data, unquantised | 3 epochs, bf16, batch 4 x accum 2, A100, 38 min | |
| | `full-3ep-bf16-analysis/` | v3: v2 plus the 70B's rule-by-rule analysis before each report | 3 epochs, bf16, A100, 47 min | |
| | `rl-screener-v1/` | GRPO single-pass screener (ok / issues + probability) | 80 steps, 8 generations, lr 2e-5, 85 min | |
|
|
| All: base `andreasmartin/apertus-v1.5-8b-text` (text-only Apertus v1.5 8B), LoRA rank 16, alpha 32, dropout 0.05, all |
| linear projections, lr 1e-4 (SFT), max length 8,192, seed 0, one run per configuration. Each folder has `train_log.json` |
| with the arguments and loss history. |
|
|
| **Data:** 414 balanced examples (207 with an injected error, 207 clean; refunds 240, manufacturing 174), kept by |
| rejection sampling from Apertus 70B investigations of simulated agent runs (557 of 829 accepted). Loss only on the |
| auditor's turns (~9 % of tokens; 23 % for v3). |
|
|
| ## Results (held-out test, 262 runs, balanced accuracy) |
|
|
| | System | Refunds | Manufacturing | Failed audits | |
| |---|---|---|---| |
| | B1 untrained 8B + harness | 0.62 | 0.54 | 20 | |
| | B2 v2 adapter + harness | 0.64 | 0.52 | 1 | |
| | B3 Apertus 70B + harness | 0.69 | 0.71 | 0 | |
|
|
| v2: failed audits 20 -> 1, verified evidence 0.60 -> 0.78, 2-3x faster, but more false alarms (0.39 -> 0.51). v3 on dev |
| was too cautious (0.56 / 0.48). The GRPO screener did not beat the untrained 8B on dev (0.53 / 0.48 vs 0.54 / 0.48) and |
| its probabilities were not calibrated. All under a small budget (one seed, no search); not proof that training cannot |
| help. |
|
|
| ## Use |
|
|
| ```bash |
| vllm serve andreasmartin/apertus-v1.5-8b-text --enable-lora --max-lora-rank 16 \ |
| --lora-modules auditor=<local path>/full-3ep-bf16/adapter \ |
| --chat-template-content-format string --enable-auto-tool-choice --tool-call-parser apertus |
| ``` |
|
|
| Then point rosetta-flow at it with `LLM_BASE_URL=http://localhost:8000/v1` and `LLM_NAME=auditor`. |
|
|
| ## Limits |
|
|
| Trained only on simulated runs in two domains (refunds, manufacturing quality) and four languages. Not validated on |
| real production traces. Use as a research artefact, with a human reviewing findings. |
|
|
| ## Licence |
|
|
| Apache-2.0, like Apertus. Base model: swiss-ai Apertus v1.5 (Apache-2.0), text-only conversion by FHNW. |
|
|
