README.md

# SecureCoder — training runbook

A QLoRA fine-tune that targets three skills at once: **coding**, **tool calling**
and **cybersecurity** (offence *and* defence), on a base chosen to train cheaply
and run fast. Everything here is measured or verified on real jobs, not assumed —
each claim carries the job id you can inspect.

## 1. Base model choice

Screened on the Hub on 2026-09-24 (params / licence / tool-calling support):

| Candidate | Params | Licence | Verdict |
| --- | --- | --- | --- |
| **`Qwen/Qwen3-Coder-30B-A3B-Instruct`** | 30.5B MoE, ~3B active | Apache-2.0 | **chosen** — code-specialised, native tool-calling template, MoE means it *trains* like a 3B model and fits 48 GB in 4-bit |
| `Qwen/Qwen3.8-27B` | 27.8B dense | Apache-2.0 | best raw quality, ~4x slower per step; good upgrade path once the pipeline is proven |
| `Qwen/Qwen3-Coder-Next` | 79.7B | Apache-2.0 | needs an 80 GB card (A100 80 GB `$2.50/h`); too big for the first run |
| `Qwen/Qwen3.8-Flash-Next` | 180B | **license: other** | rejected — non-commercial-style licence on an 180B model |
| `openbmb/MiniCPM5-2B` | 2.5B | Apache-2.0 | `tool-calling` tag, tiny; good for a T4 smoke test, weak ceiling |
| `TokenRhythm/NeoHorse-1-9B` | 9.0B | Apache-2.0 | `tool-use`, `reasoning`; the fallback if T4-only |

Why not start from an existing *abliterated* checkpoint? Because refusal removal is
a separate weight-edit (section 6) that can be applied to **any** checkpoint after
fine-tuning, and pre-abliterated bases are community re-uploads with unclear
provenance. Fine-tuning first keeps the lineage clean and the base licensed.

## 2. Data mix

The cap per source *is* the recipe — sources differ hugely in size, so the mix is
balanced by taking a fixed slice of each. Verified on a real CPU job
(`Taimwe/6ab5ae256b030d633f68faef`) at 25 rows/source:

| Source | Rows taken | Kind | What it teaches |
| --- | --- | --- | --- |
| `NousResearch/hermes-function-calling-v1` `[func_calling]` | 9,000 | tool calling | full tool-call conversations + JSON schemas |
| `NousResearch/hermes-function-calling-v1` `[func_calling_singleturn]` | 3,000 | tool calling | picking the right function, no chatter |
| `lockon/xlam-function-calling-60k` | 10,000 | tool calling | 60k API-call pairs (query → call) |
| `ise-uiuc/Magicoder-OSS-Instruct-75K` | 10,000 | coding | self-instruct problems + solutions |
| `Trendyol/Trendyol-Cybersecurity-Instruction-Tuning-Dataset` | 8,000 | security | security instruction tuning |
| `AlicanKiraz0/Cybersecurity-Dataset-Fenrir-v2.1` | 5,000 | security | broad security Q&A |
| `Humanlearning/CyberSecurity_OWASP-sft-dataset` | 3,000 | secure coding | OWASP / defensive coding |
| `MrClipperz134/CTF-Instruct` | 3,000 | CTF | challenge instruction → solve |
| `TrueNix/ctf-solver-dataset` | 3,000 | CTF | solving trajectories |
| `mlabonne/FineTome-100k` | 3,000 | replay | general chat, so instruction-following does not collapse |

**Total ≈ 57,000 rows, ~30% of them tool-calling.** Every source validated at
25/25 kept with zero render failures.

Excluded deliberately:

* `dpevzner/Cybersecurity_Reasoning_Dataset` — its builder configs were renamed
  (`default` → `mistral|deepseek|chatml|gemma`) and the fields the card documents
  (`unified_interpretation`) are empty in the published revision, so every row was
  dropped. Re-addable once its schema settles.
* Anything whose purpose is building malware or weaponised exploits. Recon and
  enumeration knowledge, exploit *concepts*, CTF solving and defensive
  engineering are in; end-to-end attack tooling is not. This is a deliberate
  scope line, not an oversight.

## 3. What training does with the data

Each row is converted to chat messages and rendered through the model's own chat
template, so the model learns the exact format it will be asked for at inference
time (`<|im_start|>…<tool_call><function=NAME><parameter=…>` for Qwen3-Coder).

Two format traps found the hard way and handled in code:

* Qwen3-Coder iterates `tool.parameters.properties` — tool schemas must be **flat**
  (`{"name", "description", "parameters"}`), not wrapped in `"function"`.
* It also iterates `tool_call.arguments | items`, so `arguments` must be a
  **mapping**. A JSON string there fails with *"Can only get item pairs from a
  mapping"* — which is exactly how xLAM was silently contributing 0 rows until fixed.

`render_record()` therefore detects the template's convention once (flat/nested ×
dict/string) and reuses it, instead of hard-coding one model's dialect.

## 4. Compute and cost

Verified `hf jobs hardware` prices (per hour), 2026-09-24:

| Flavor | VRAM | $/hour | Use |
| --- | --- | --- | --- |
| `cpu-basic` | — | $0.01 | data-mix validation, class checks |
| `t4-small` | 16 GB (T4) | $0.40 | plumbing checks; too small for a 30B in 4-bit |
| `l4x1` | 24 GB | $0.80 | 9B runs on a budget |
| `a10g-large` | 24 GB | $1.50 | 9B–14B comfortably |
| **`l40sx1`** | **48 GB** | **$1.80** | **the 30B-A3B QLoRA** |
| `a100-large` | 80 GB | $2.50 | faster per step, larger batch |
| `rtx-pro-6000` | 96 GB | $2.75 | headroom for longer context |
| `h200` | 141 GB | $5.00 | full fine-tune territory, overkill here |

Rough budget for the full 1-epoch run. **These are now measured, not guessed** — the
smoke test (`Taimwe/6ab5b3ed52d0dbd7f1d8d454`, a100-large) trained at
**0.8 rows/s** (20 s per step at batch 2 × grad-accum 8, max seq 2048, packing on),
which includes a few minutes of `torch.compile` warm-up that does not recur per step:

| Mix | Rows | Time at 0.8 rows/s | Cost on `l40sx1` ($1.80/h) |
| --- | --- | --- | --- |
| full mix as specified | ~57,000 | ~20 h | **~$36** |
| half caps (`--max-steps` or edit the table) | ~28,000 | ~10 h | ~$18 |
| lean mix (cut caps to ~1/3) | ~19,000 | ~7 h | ~$12 |

Because warm-up inflates that rate, a **300-step measured run** (~$4 on a100-large)
gives a trustworthy steady-state number before committing to a long job. Levers that
move the cost most, in order: rows in the mix, `--max-seq-length` (2048 vs 4096),
`--batch-size` (raise it on 80–96 GB cards), `--lora-r`.

The `--smoke` run itself costs under $1 and proves model load, data rendering,
training, evaluation and the Hub push end to end.

No local GPU needed. Colab works the same way with `uv run train_securecoder.py …`
(Colab Free's T4 cannot hold a 30B in 4-bit — use Colab Pro L4/A100, or point
`--base-model` at a 9B).

## 5. Runbook (copy-paste)

```bash
# 0. auth (token needs repo.write, and job.write for Jobs)
hf auth login --token hf_xxx

# 1. validate the mix: 25 rows/source, no GPU, ~50 s, ~$0.0002
hf jobs run -d --flavor cpu-basic --timeout 30m --name validate-mix \
  ghcr.io/astral-sh/uv:python3.12-bookworm \
  uv run --no-project https://huggingface.co/Taimwe/securecoder-scripts/resolve/main/train_securecoder.py \
  --validate-only --validate-per-source 25 --show-samples 3

# 2. smoke test on a real GPU: 200 rows/source, 20 steps, pushes an adapter (~$0.50)
hf jobs run -d --flavor l40sx1 --timeout 40m --secrets HF_TOKEN --name smoke-train \
  ghcr.io/astral-sh/uv:python3.12-bookworm \
  uv run --no-project https://huggingface.co/Taimwe/securecoder-scripts/resolve/main/train_securecoder.py \
  --smoke --output-repo Taimwe/securecoder-smoke --private --report-to none

# 3. the real run (~$9–14)
hf jobs run -d --flavor l40sx1 --timeout 12h --secrets HF_TOKEN --name securecoder-run1 \
  ghcr.io/astral-sh/uv:python3.12-bookworm \
  uv run --no-project https://huggingface.co/Taimwe/securecoder-scripts/resolve/main/train_securecoder.py \
  --num-epochs 1 --output-repo Taimwe/securecoder-30b-pro \
  --trackio-space Taimwe/securecoder-trackio

# monitor
hf jobs inspect Taimwe/<job_id> --json
hf jobs logs    Taimwe/<job_id>
hf jobs stats   Taimwe/<job_id>
hf jobs cancel  Taimwe/<job_id>
```

### Platform traps already worked around (verified 2026-09-24)

| Symptom | Cause | Workaround |
| --- | --- | --- |
| `failed to set MOUNT_ATTR_IDMAP on /usr/bin/nvidia-cuda-mps-control` on a GPU flavor | `hf jobs uv run` mounts an artifacts bucket; the idmap mount fails on some GPU nodes (reproduced twice, 4 s after start) | use `hf jobs run` + the script's **public Hub URL** (works: `Taimwe/6ab5ad3c6b030d633f68face`) or a read-only repo mount `-v hf://models/Taimwe/securecoder-scripts:/scripts:ro` (works: `Taimwe/6ab5ad3d52d0dbd7f1d8d286`) |
| inner quotes stripped from `python -c "…"` | PowerShell native-argument quoting | use a UV script file, never `-c` one-liners |
| `charmap codec can't encode` while reading logs | Windows console encoding | set `$env:PYTHONIOENCODING='utf-8'` before `hf jobs logs` |
| HF's trainer skill says Jobs need a paid plan | docs caveat | not enforced on this account — CPU and GPU jobs both ran |

## 6. Turning down refusals (optional, after SFT)

Abliteration is a weight edit, not a training run: find the "refusal direction" in
the residual stream and project it out of the writing matrices.

1. **`Heretic`** (p-e-w/heretic) — the tool behind the recent wave of
   `*-heretic-abliterated-*` checkpoints (e.g.
   `culturerevolt/gemma-4-12b-heretic-abliterated-GGUF`, 174k downloads). Runs on
   the merged 16-bit model and optimises the ablation to trade refusal rate against
   KL divergence, so it preserves capability better than a hand-rolled ablation.
2. **Manual directional ablation** (Arditi et al., *Refusal in LLMs is mediated by a
   single direction*): collect residual activations for harmful vs. harmless
   prompts, take the mean difference per layer, and zero that component in
   `o_proj` / `down_proj`.

Honest expectations: **it is not free.** Refusals drop, and some performance on the
same layers' other duties drops with them; scope-setting behaviour ("this needs
authorisation") can weaken too. Measure both sides (section 7) and document the
result on the model card, or you are shipping an unmeasured change.

## 7. Evaluation before publishing

1. **Tool-call validity** — 200 prompts with real tool schemas; parse the emitted
   `<function=NAME><parameter=…>` block back to JSON and score syntax validity,
   correct function choice, and schema-conformant arguments. This is the number
   that matters most for "top-notch tool calling".
2. **Code sanity** — generate 100 solutions, check compilation, run the bundled
   tests on a small HumanEval-style subset.
3. **Security knowledge** — fixed MCQ/triage set (e.g. `CyberNative/CyberSecurityEval`),
   scored before *and* after fine-tuning so the mix's effect is visible.
4. **Regression** — the same three on the untouched base, same prompts. A fine-tune
   that feels better but scores worse is a regression with extra steps.
5. **Refusal rate** — if abliterated, publish before/after refusal rate next to the
   eval deltas.

## 8. Known risks

* MoE LoRA defaults to attention + router (`q/k/v/o/gate`). `--target-modules
  all-linear` on a 128-expert model means ~800M trainable parameters — much slower
  and hungrier. Start conservative; revisit if security knowledge is the weak score.
* 4-bit QLoRA of a 30B (~17 GB) is tight on a 24 GB card; 48 GB allows batch 2 with
  4096-token samples.
* One 1-epoch pass over three domains is a *start*, not a finished model. Iterate on
  the mix ratios — they live in one table at the top of the script.
* Nothing here adds safety training, and the security slice includes
  recon/enumeration knowledge framed for authorised use. Publish explicit
  intended-use and limitations sections; do not present it as a hardened model.