FLvdW

/

Zeroth42-4B

Like 1

No CUDA. No cloud. No prompt scaffolding. Just a model, three tools, and a task where being wrong means a disk gets wiped.

By Zeroth42 (FLvdW) — independent developer, South Africa. We train and ship on ~$1,500 of second-hand AMD hardware. Zeroth42-4B is our installer co-pilot; this post is the 27B that helps build it.

The question nobody asks about "practical experience"

Benchmarks tell you a model can answer. They don't tell you whether you can trust it to act — which is a different question
entirely, and the one that actually matters when the task is dangerous-adjacent.

We're building an AI-assisted Linux installer. Its "verify" half reads a disk's partition table and judges it against golden safety
rules — is there an ESP? is it big enough? is swap present when hibernation is wanted? — and prints PASS or BLOCK before anything touches the disk. In that job, a wrong answer isn't a bad benchmark score. It's a wiped drive.

So we asked: can a quantized 27B on a used 2018 datacenter GPU do genuine, multi-step tool use through a plain OpenAI-compatible endpoint — with no few-shot prompts, no CoT templates, no scaffolding — and produce receipts we'd stake a disk on?

The rig

The harness is ~280 lines: a loop that gives the model three tools (bash, read_file, write_file), executes returned `tool_calls' in a read-only-hardened sandbox, and appends every exchange to a JSONL transcript.
The transcript — not a summary — is the artifact.

The task: write partition_check.py — inspect real block devices via lsblk -J, check against golden rules, print PASS/BLOCK with reasons. UEFI profile must block a disk with no ESP or no swap. BIOS profile must pass a dos-layout ext4 disk. Strictly read-only.

What actually happened

Four runs, four different quants, each served on its own port — same harness, same settings (temperature 0.2), same task. All four read the real devices, read the rules, wrote the tool, ran it on real hardware, and reported. Three delivered a working tool on the first write. The fourth found a real bug in its own code — that's the story worth reading.

An earlier run whose quant couldn't be proven from logs is deliberately excluded. Receipts before adjectives — that's the house rule.

The moment that made the whole experiment worth it

Run A wrote its tool, ran it — and noticed a partition being classified as an ESP candidate despite carrying the real ESP GUID
(c12a7328-f81f-11d2-ba4b-00a0c93ec93b).

Then it did the thing you hope for but never script: it diagnosed the cause (a metadata-merge bug in its own lsblk tree-walking), rewrote the ESP classification, and re-ran the full test matrix to prove the fix.

A quantized model, on a 2018 card, catching a subtle GUID-classification bug in code it wrote ninety seconds earlier — unscripted, mid-task — and then verifying its own fix. That's not a benchmark number. That's the closest thing to "it can actually do the job" that a transcript can show.

A genuinely different failure mode (base-Q5)

Run D tried to emit the entire ~400-line deliverable in one write_file payload — ~14K chars of escaped source. Twice the
arguments JSON arrived truncated (reasoning had consumed the generation budget) → malformed JSON → server 500. Raising the budget to 24K tokens let it complete in one shot. We kept the failure transcripts. Different quants aren't just faster or slower — they behave differently.

What we learned about tool-calling (the part you can reuse)

1. Native function calling just works on llama-server. Standard OpenAI tools array. No plugins, no hacks.
2. Thinking mode eats the budget. Observed single-turn reasoning blocks of 12K–35K chars across all four quants. Give the agent a big max_tokens and long timeouts (900s), or "think then act" stalls.
3. Handle empty turns. The model sometimes returns reasoning only. The harness must nudge ("continue / deliver") instead of hanging.
4. Verify which model you actually served. Our first run's quant was unprovable from its logs, so that run is excluded from every claim here. Later runs embed the endpoint and server-log proof in the transcript itself; the earliest verified run's proof lives in the session log that launched it, not in the transcript.
5. Read-only sandbox is non-negotiable for dangerous-adjacent tasks. Across all runs, the model never once attempted a destructive command.
6. The same 27B now runs as a real agent in our pi harness (qwen38-worker subagent) — same model, no retraining, the tool surface scales from 3 hand-rolled functions to a full toolkit.

Why this matters

"Practical experience" with a model isn't only a tok/s table. It's evidence that a quantized 27B on AMD silicon — used 2018 hardware, zero CUDA — can run a real safety-critical verify loop, catch its own bugs, and do it reproducibly across four quants, with transcripts as the receipt.

That's the exact trust boundary our product lives on. The 4B co-pilot Zeroth42-4B talks; the 27B helps build the machinery that checks. Both run on the same open stack, on hardware anyone can afford.

Zeroth42 — honest process on display. We publish the audit trail, not just the answer.

Related · FLvdW/Zeroth42-4B — the 4B installer co-pilot · companion write-up (dual-AMD llama.cpp, quant table, thinking-budget tuning) lands Sunday.

Full JSONL transcripts from all seven runs available on request. Discussion welcome — especially if you're running any of this on your own AMD cards.