nihalpj/docslm-sft-v1

license: apache-2.0 base_model: nihalpj/docslm-sft-v1 tags:

document-processing agent tool-calling docx pdf xlsx pptx quantized gguf fine-tuned model_creator: Nihal model_name: DocSLM-SFT-v1 pipeline_tag: text-generation language: en DocSLM-SFT-v1 (DocSLM-0.8B) A domain-specialized agentic small language model — a full fine-tune of Qwen/Qwen3.5-0.8B (752M params) — built to run a document-skills agent stack (DOCX / PDF / XLSX / PPTX) locally: cheap, fast, private, and offline-capable, with graceful escalation to a frontier model for the tasks it cannot close.

The model is trained to operate as a tool-calling agent: it reads a document-task request, reasons in a block, emits a containing executable code (run_python / run_node), reacts to the sandbox result (including failures — it was explicitly trained to repair), and verifies its own output via a postcheck tool. Training data was generated by a data engine that keeps only trajectories that actually executed and passed verification in the real sandbox — no vibes-based data.

Model details Base model Qwen/Qwen3.5-0.8B Parameters 752M (text; vision tower frozen at train time, not used) Architecture Qwen3.5-text: hybrid linear attention — full attention every 4th layer, gated linear (SSM-style) attention elsewhere Layers / hidden 24 (+1 MTP draft layer declared in config, absent from weights — see GGUF export notes) Attention 8 query heads / 2 KV heads (GQA), head_dim 256, partial RoPE (0.25), interleaved MRoPE sections 11/11/10 Intermediate size 3584 Context (train) 262,144 Vocab 248,320 (Qwen3.5 tokenizer) Precision bf16 checkpoint License Apache-2.0 (inherited from base) Direct-use outputs This model is an SFT checkpoint (stage B of CPT → SFT → DPO/GRPO). It emits:

Agentic trajectories — reasoning + tool calls against run_python / run_node / postcheck, serialized in Qwen3 native chat format. Skill routing — single-shot classification of a document request into one of 30 capabilities ({"skill": ..., "route": ...}). Training data Produced by the repo's data engine (data_engine/): teacher rollouts from a frontier model over synthetic + CC/open-data corpora, executed in the real document sandbox, filtered by execution + postcheck + judge, then deduplicated and decontaminated against the held-out bench.

SFT trajectories 930 (7,594 messages; 573 include repair turns mined from real failure traces) Routing cases 1,240 classification examples Capabilities 30 routes: docx ×7 (create / edit / read / format / comment / footer_fix / toc_fix), pdf ×14 (merge / split / rotate / crop / report / qa / extract_text / extract_table / extract_image / form_fill / font_check / meta_edit / palette / toc_check), xlsx ×7 (create / edit / inspect / analyze / audit / pivot / scan), pptx ×2 (create / inspect_edit) Serialization Qwen3 native <|im_start|> format with assistant-only loss masking; -style reasoning recorded per turn Licensing of data Synthetic or CC/open-data inputs; the released model contains no skill text (skill files are retrieved at runtime, not memorized) Training procedure Full fine-tune (not LoRA) on a single rented RTX 4090 (24 GB):

Hyperparameter Value Precision bf16 weights Optimizer paged AdamW 8-bit (bitsandbytes) Schedule cosine, LR 1e-5, warmup ratio 0.03 Epochs 3 Effective batch 32 (per-device 2 × grad-accum 16) @ 6144-token sequences Gradient checkpointing on Vision tower frozen (text-only training) Train tokens ~2.5M / epoch Wall clock ~30–60 min on a 4090 (flash-attn) A LoRA fallback (r=64, α=128, all linear layers, ~12 GB) exists in the training script but was not used for this checkpoint.

Prompt format Qwen3 native chat format. The model expects a document-skill system prompt (capability contract, artifact naming, sandbox paths), the user request, and — in later turns — tool results. It replies with reasoning and tool calls:

text <|im_start|>assistant Route: pdf/merge, scene 'report', difficulty L3, output 'merged.pdf'. Steps: open both PDFs with pypdf, append pages, write output. {"name": "run_python", "arguments": {"code": "from pypdf import PdfWriter\n..."}} <|im_end|> Tool result turns look like:

json {"role": "tool", "tool_call_id": "call_0", "name": "run_python", "content": "{"exit": 0, "artifact": "merged.pdf"}"} GGUF exports Quantized with an importance matrix calibrated on the project's own document-domain text (200 calibration trajectories) rather than generic wiki text:

File Size Notes docslm-f16.gguf 1.5 GB reference / ceiling docslm-q4_k_m.gguf 505 MB ship candidate docslm-iq4_xs.gguf 481 MB imatrix quant, ~4.25 bpw Converter note (important): the checkpoint config declares mtp_num_hidden_layers: 1 (Qwen3.5's multi-token-prediction draft layer) but the saved weights contain no MTP tensors. Convert with convert_hf_to_gguf.py ... --no-mtp (or strip the mtp_* keys from config.json first). Without it the GGUF metadata claims 25 blocks while carrying 24, and llama.cpp servers exit with tensor 'blk.24.attn_norm.weight' not found on load. The stock Unsloth Qwen3.5 GGUFs (24 blocks) are the correct reference layout.

Evaluation Machine-verifiable harness (evaluation/benchmark.py): every SFT case is replayed as a live agentic episode (max 4 model turns, repair budget 2) in a real sandbox; artifacts are scored by each capability's authoritative checker (postcheck / structural / plugin-CLI). Gates: exec ≥ 85%, verify ≥ 80%, repair ≥ 60%, format ≥ 95%, routing ≥ 95%; ship gate = Q4_K_M within −2 pts of the unquantized run.

Run exec verify repair format routing Gold self-test (harness ceiling, 930 + 1,240 cases) 100% 100% 100% 100% 100% Q4_K_M agentic (in progress) — — — — — The gold row is a harness self-test (replays recorded ground-truth actions; it validates the scoring pipeline, not the model). The first live model scores are being computed; see datasets/eval.md in the project repo for continuously updated interim numbers. Early signal: format is at gate (the model reliably emits parseable reasoning + tool calls), while agentic exec/verify for the 4-bit export are trending far below gate — treat Q4_K_M as unproven until the final table lands; Q5_K_M / IQ4_XS are the fallback path.

Serving notes llama.cpp / Unsloth Studio: loads like any Qwen3.5 GGUF. With --reasoning on, servers re-box … into a separate reasoning_content field — re-attach it if your harness scores on raw content. The model can occasionally emit tool-call JSON with unescaped control characters (e.g. raw tabs inside code strings); strict server-side tool-call parsers may reject those requests with HTTP 500. Serving with --jinja and parsing from content is the more forgiving path. Context: benchmark episodes stay under ~10k tokens; production serving at 32–64k context is ample. Intended use & limitations Intended: local, private, offline document automation behind an agent runtime that provides the sandbox (docx-js, openpyxl, pptxgenjs, reportlab, tectonic), verification, and escalation to a frontier model after repeated verification failure.

Limitations:

752M parameters tuned to a narrow recipe domain — it is not a general assistant and will hallucinate outside document tasks. Code quality degrades under aggressive quantization (4-bit exports are being gated separately; see Evaluation). English-centric training data. Requires the surrounding agent stack (sandbox + verifiers) to be useful; raw completion quality is not the product surface. Files in this repository config.json · model.safetensors (bf16) · generation_config.json · tokenizer.json · tokenizer_config.json · chat_template.jinja

Acknowledgements Built on Qwen3.5-0.8B (Qwen team), quantized with llama.cpp, trained with Unsloth / HF Transformers tooling.