SecureCoder runbook

README.md

@@ -0,0 +1,201 @@
+# SecureCoder — training runbook
+
+A QLoRA fine-tune that targets three skills at once: **coding**, **tool calling**
+and **cybersecurity** (offence *and* defence), on a base chosen to train cheaply
+and run fast. Everything here is measured or verified on real jobs, not assumed —
+each claim carries the job id you can inspect.
+
+## 1. Base model choice
+
+Screened on the Hub on 2026-09-24 (params / licence / tool-calling support):
+
+| Candidate | Params | Licence | Verdict |
+| --- | --- | --- | --- |
+| **`Qwen/Qwen3-Coder-30B-A3B-Instruct`** | 30.5B MoE, ~3B active | Apache-2.0 | **chosen** — code-specialised, native tool-calling template, MoE means it *trains* like a 3B model and fits 48 GB in 4-bit |
+| `Qwen/Qwen3.8-27B` | 27.8B dense | Apache-2.0 | best raw quality, ~4x slower per step; good upgrade path once the pipeline is proven |
+| `Qwen/Qwen3-Coder-Next` | 79.7B | Apache-2.0 | needs an 80 GB card (A100 80 GB `$2.50/h`); too big for the first run |
+| `Qwen/Qwen3.8-Flash-Next` | 180B | **license: other** | rejected — non-commercial-style licence on an 180B model |
+| `openbmb/MiniCPM5-2B` | 2.5B | Apache-2.0 | `tool-calling` tag, tiny; good for a T4 smoke test, weak ceiling |
+| `TokenRhythm/NeoHorse-1-9B` | 9.0B | Apache-2.0 | `tool-use`, `reasoning`; the fallback if T4-only |
+
+Why not start from an existing *abliterated* checkpoint? Because refusal removal is
+a separate weight-edit (section 6) that can be applied to **any** checkpoint after
+fine-tuning, and pre-abliterated bases are community re-uploads with unclear
+provenance. Fine-tuning first keeps the lineage clean and the base licensed.
+
+## 2. Data mix
+
+The cap per source *is* the recipe — sources differ hugely in size, so the mix is
+balanced by taking a fixed slice of each. Verified on a real CPU job
+(`Taimwe/6ab5ae256b030d633f68faef`) at 25 rows/source:
+
+| Source | Rows taken | Kind | What it teaches |
+| --- | --- | --- | --- |
+| `NousResearch/hermes-function-calling-v1` `[func_calling]` | 9,000 | tool calling | full tool-call conversations + JSON schemas |
+| `NousResearch/hermes-function-calling-v1` `[func_calling_singleturn]` | 3,000 | tool calling | picking the right function, no chatter |
+| `lockon/xlam-function-calling-60k` | 10,000 | tool calling | 60k API-call pairs (query → call) |
+| `ise-uiuc/Magicoder-OSS-Instruct-75K` | 10,000 | coding | self-instruct problems + solutions |
+| `Trendyol/Trendyol-Cybersecurity-Instruction-Tuning-Dataset` | 8,000 | security | security instruction tuning |
+| `AlicanKiraz0/Cybersecurity-Dataset-Fenrir-v2.1` | 5,000 | security | broad security Q&A |
+| `Humanlearning/CyberSecurity_OWASP-sft-dataset` | 3,000 | secure coding | OWASP / defensive coding |
+| `MrClipperz134/CTF-Instruct` | 3,000 | CTF | challenge instruction → solve |
+| `TrueNix/ctf-solver-dataset` | 3,000 | CTF | solving trajectories |
+| `mlabonne/FineTome-100k` | 3,000 | replay | general chat, so instruction-following does not collapse |
+
+**Total ≈ 57,000 rows, ~30% of them tool-calling.** Every source validated at
+25/25 kept with zero render failures.
+
+Excluded deliberately:
+
+* `dpevzner/Cybersecurity_Reasoning_Dataset` — its builder configs were renamed
+  (`default` → `mistral|deepseek|chatml|gemma`) and the fields the card documents
+  (`unified_interpretation`) are empty in the published revision, so every row was
+  dropped. Re-addable once its schema settles.
+* Anything whose purpose is building malware or weaponised exploits. Recon and
+  enumeration knowledge, exploit *concepts*, CTF solving and defensive
+  engineering are in; end-to-end attack tooling is not. This is a deliberate
+  scope line, not an oversight.
+
+## 3. What training does with the data
+
+Each row is converted to chat messages and rendered through the model's own chat
+template, so the model learns the exact format it will be asked for at inference
+time (`<|im_start|>…<tool_call><function=NAME><parameter=…>` for Qwen3-Coder).
+
+Two format traps found the hard way and handled in code:
+
+* Qwen3-Coder iterates `tool.parameters.properties` — tool schemas must be **flat**
+  (`{"name", "description", "parameters"}`), not wrapped in `"function"`.
+* It also iterates `tool_call.arguments | items`, so `arguments` must be a
+  **mapping**. A JSON string there fails with *"Can only get item pairs from a
+  mapping"* — which is exactly how xLAM was silently contributing 0 rows until fixed.
+
+`render_record()` therefore detects the template's convention once (flat/nested ×
+dict/string) and reuses it, instead of hard-coding one model's dialect.
+
+## 4. Compute and cost
+
+Verified `hf jobs hardware` prices (per hour), 2026-09-24:
+
+| Flavor | VRAM | $/hour | Use |
+| --- | --- | --- | --- |
+| `cpu-basic` | — | $0.01 | data-mix validation, class checks |
+| `t4-small` | 16 GB (T4) | $0.40 | plumbing checks; too small for a 30B in 4-bit |
+| `l4x1` | 24 GB | $0.80 | 9B runs on a budget |
+| `a10g-large` | 24 GB | $1.50 | 9B–14B comfortably |
+| **`l40sx1`** | **48 GB** | **$1.80** | **the 30B-A3B QLoRA** |
+| `a100-large` | 80 GB | $2.50 | faster per step, larger batch |
+| `rtx-pro-6000` | 96 GB | $2.75 | headroom for longer context |
+| `h200` | 141 GB | $5.00 | full fine-tune territory, overkill here |
+
+Rough budget for the full 1-epoch run. **These are now measured, not guessed** — the
+smoke test (`Taimwe/6ab5b3ed52d0dbd7f1d8d454`, a100-large) trained at
+**0.8 rows/s** (20 s per step at batch 2 × grad-accum 8, max seq 2048, packing on),
+which includes a few minutes of `torch.compile` warm-up that does not recur per step:
+
+| Mix | Rows | Time at 0.8 rows/s | Cost on `l40sx1` ($1.80/h) |
+| --- | --- | --- | --- |
+| full mix as specified | ~57,000 | ~20 h | **~$36** |
+| half caps (`--max-steps` or edit the table) | ~28,000 | ~10 h | ~$18 |
+| lean mix (cut caps to ~1/3) | ~19,000 | ~7 h | ~$12 |
+
+Because warm-up inflates that rate, a **300-step measured run** (~$4 on a100-large)
+gives a trustworthy steady-state number before committing to a long job. Levers that
+move the cost most, in order: rows in the mix, `--max-seq-length` (2048 vs 4096),
+`--batch-size` (raise it on 80–96 GB cards), `--lora-r`.
+
+The `--smoke` run itself costs under $1 and proves model load, data rendering,
+training, evaluation and the Hub push end to end.
+
+No local GPU needed. Colab works the same way with `uv run train_securecoder.py …`
+(Colab Free's T4 cannot hold a 30B in 4-bit — use Colab Pro L4/A100, or point
+`--base-model` at a 9B).
+
+## 5. Runbook (copy-paste)
+
+```bash
+# 0. auth (token needs repo.write, and job.write for Jobs)
+hf auth login --token hf_xxx
+
+# 1. validate the mix: 25 rows/source, no GPU, ~50 s, ~$0.0002
+hf jobs run -d --flavor cpu-basic --timeout 30m --name validate-mix \
+  ghcr.io/astral-sh/uv:python3.12-bookworm \
+  uv run --no-project https://huggingface.co/Taimwe/securecoder-scripts/resolve/main/train_securecoder.py \
+  --validate-only --validate-per-source 25 --show-samples 3
+
+# 2. smoke test on a real GPU: 200 rows/source, 20 steps, pushes an adapter (~$0.50)
+hf jobs run -d --flavor l40sx1 --timeout 40m --secrets HF_TOKEN --name smoke-train \
+  ghcr.io/astral-sh/uv:python3.12-bookworm \
+  uv run --no-project https://huggingface.co/Taimwe/securecoder-scripts/resolve/main/train_securecoder.py \
+  --smoke --output-repo Taimwe/securecoder-smoke --private --report-to none
+
+# 3. the real run (~$9–14)
+hf jobs run -d --flavor l40sx1 --timeout 12h --secrets HF_TOKEN --name securecoder-run1 \
+  ghcr.io/astral-sh/uv:python3.12-bookworm \
+  uv run --no-project https://huggingface.co/Taimwe/securecoder-scripts/resolve/main/train_securecoder.py \
+  --num-epochs 1 --output-repo Taimwe/securecoder-30b-pro \
+  --trackio-space Taimwe/securecoder-trackio
+
+# monitor
+hf jobs inspect Taimwe/<job_id> --json
+hf jobs logs    Taimwe/<job_id>
+hf jobs stats   Taimwe/<job_id>
+hf jobs cancel  Taimwe/<job_id>
+```
+
+### Platform traps already worked around (verified 2026-09-24)
+
+| Symptom | Cause | Workaround |
+| --- | --- | --- |
+| `failed to set MOUNT_ATTR_IDMAP on /usr/bin/nvidia-cuda-mps-control` on a GPU flavor | `hf jobs uv run` mounts an artifacts bucket; the idmap mount fails on some GPU nodes (reproduced twice, 4 s after start) | use `hf jobs run` + the script's **public Hub URL** (works: `Taimwe/6ab5ad3c6b030d633f68face`) or a read-only repo mount `-v hf://models/Taimwe/securecoder-scripts:/scripts:ro` (works: `Taimwe/6ab5ad3d52d0dbd7f1d8d286`) |
+| inner quotes stripped from `python -c "…"` | PowerShell native-argument quoting | use a UV script file, never `-c` one-liners |
+| `charmap codec can't encode` while reading logs | Windows console encoding | set `$env:PYTHONIOENCODING='utf-8'` before `hf jobs logs` |
+| HF's trainer skill says Jobs need a paid plan | docs caveat | not enforced on this account — CPU and GPU jobs both ran |
+
+## 6. Turning down refusals (optional, after SFT)
+
+Abliteration is a weight edit, not a training run: find the "refusal direction" in
+the residual stream and project it out of the writing matrices.
+
+1. **`Heretic`** (p-e-w/heretic) — the tool behind the recent wave of
+   `*-heretic-abliterated-*` checkpoints (e.g.
+   `culturerevolt/gemma-4-12b-heretic-abliterated-GGUF`, 174k downloads). Runs on
+   the merged 16-bit model and optimises the ablation to trade refusal rate against
+   KL divergence, so it preserves capability better than a hand-rolled ablation.
+2. **Manual directional ablation** (Arditi et al., *Refusal in LLMs is mediated by a
+   single direction*): collect residual activations for harmful vs. harmless
+   prompts, take the mean difference per layer, and zero that component in
+   `o_proj` / `down_proj`.
+
+Honest expectations: **it is not free.** Refusals drop, and some performance on the
+same layers' other duties drops with them; scope-setting behaviour ("this needs
+authorisation") can weaken too. Measure both sides (section 7) and document the
+result on the model card, or you are shipping an unmeasured change.
+
+## 7. Evaluation before publishing
+
+1. **Tool-call validity** — 200 prompts with real tool schemas; parse the emitted
+   `<function=NAME><parameter=…>` block back to JSON and score syntax validity,
+   correct function choice, and schema-conformant arguments. This is the number
+   that matters most for "top-notch tool calling".
+2. **Code sanity** — generate 100 solutions, check compilation, run the bundled
+   tests on a small HumanEval-style subset.
+3. **Security knowledge** — fixed MCQ/triage set (e.g. `CyberNative/CyberSecurityEval`),
+   scored before *and* after fine-tuning so the mix's effect is visible.
+4. **Regression** — the same three on the untouched base, same prompts. A fine-tune
+   that feels better but scores worse is a regression with extra steps.
+5. **Refusal rate** — if abliterated, publish before/after refusal rate next to the
+   eval deltas.
+
+## 8. Known risks
+
+* MoE LoRA defaults to attention + router (`q/k/v/o/gate`). `--target-modules
+  all-linear` on a 128-expert model means ~800M trainable parameters — much slower
+  and hungrier. Start conservative; revisit if security knowledge is the weak score.
+* 4-bit QLoRA of a 30B (~17 GB) is tight on a 24 GB card; 48 GB allows batch 2 with
+  4096-token samples.
+* One 1-epoch pass over three domains is a *start*, not a finished model. Iterate on
+  the mix ratios — they live in one table at the top of the script.
+* Nothing here adds safety training, and the security slice includes
+  recon/enumeration knowledge framed for authorised use. Publish explicit
+  intended-use and limitations sections; do not present it as a hardened model.