--- license: apache-2.0 base_model: Qwen/Qwen3.5-0.8B tags: - gguf - llama.cpp - distillation - structured-output - grammar-constrained language: - en --- # Tree Compiler 0.8B A Qwen3.5-0.8B fine-tune that turns a job seeker's preferences, written in plain words, into a JSON scoring model (**mm-tree/1, factors form**): lists, reads, hard rules, weighted factors and verdict bands. A separate local program runs that object on each job or recruiter email and returns **Through / Maybe / Held** with reasons. The model writes the decision procedure once; it never sees the emails. This repository holds two versions. **v1 (the files in the repository root) is the recommended one**. Use it with the arity grammar and the validate-and-retry step described below; this is what the public demo runs. **v2** (folder `v2/`) is an **experimental** retrain: it agrees with the gold verdicts more often, but it hides more good jobs (see Results). ## v2 (experimental) Qwen3.5-0.8B, LoRA SFT, merged into the weights and exported as q8_0. It was trained only on synthetic data: invented job seekers and invented jobs, generated with the second version of the synthetic set ("synth v2"), whose preference texts state hard rules in firm wording. The training trees come from three teachers (Claude Haiku, Claude Sonnet and Gemini Flash); for each user, the candidate tree that did best when **executed** against the gold verdicts was kept. ### Files (`v2/`) | file | what | |---|---| | `tree-compiler-0.8b-v2-q8_0.gguf` | the model, q8_0, 812 MB (llama.cpp / wllama) | | `mm_factors.gbnf` | the decoding grammar **with fixed arities**: each operator accepts exactly the number of arguments the executor expects | | `example-messages.json` | the chat messages for one invented user (system line + user message) | | `example-output.json` | the tree v2 wrote for that user (greedy, with the grammar). It passes the tree validator but was not scored against gold verdicts, and it is not a faithful reading of the message: it adds a pay ramp the user never asked for, and it has no accept rule for the dream companies | | `SHA256SUMS` | checksums of the files above | ### How to use it (v1 and v2 alike) For **v1** (recommended), use the root GGUF with the same arity grammar (`v2/mm_factors.gbnf`) and the same retry; for v2, use the `v2/` GGUF. 1. Decode with the arity grammar `v2/mm_factors.gbnf`, greedy, `enable_thinking` off in the chat template, up to 4096 new tokens. 2. **Validate** the tree with a semantic validator (arities, names that are declared, lists that exist). 3. If it is not valid, **retry up to 2 times** at temperature 0.3 with seeds 1234+k (k = 1, 2), no top-k/top-p/min-p, with the same grammar, and keep the first valid tree. If none is valid, treat the user as failed; do not run an invalid tree. ```bash llama-cli -m v2/tree-compiler-0.8b-v2-q8_0.gguf --grammar-file v2/mm_factors.gbnf --temp 0 -n 4096 \ -sys "$(jq -r '.[0].content' v2/example-messages.json)" -p "$(jq -r '.[1].content' v2/example-messages.json)" ``` In the browser, [wllama](https://github.com/ngxson/wllama) loads the file directly (WebGPU, or WASM with 4 threads behind COOP/COEP headers). The validator and the executor that runs the trees are not part of this release. ### Results Synthetic v2 **dev** set: 100 users x 120 jobs, Mac, q8_0, the arity grammar plus the retry above, with the "unknown policy cap" (a job that would be Through becomes Maybe when a deciding read is unknown). Agreement counts an invalid tree as wrong on every job. A *wrong hold* hides a job that fits the user, the costliest error. | model | valid trees | agreement (all) | wrong holds per 100 jobs | |---|---|---|---| | **v1, same setup (recommended)** | 0.98 | 0.694 | 0.52 | | v2 (experimental) | 0.96 | 0.753 | 0.84 | Read this table carefully: v2 agrees more often with the gold verdicts, but it has **more** wrong holds per 100 jobs than v1 under this setup (0.84 against 0.52), and slightly fewer valid trees (0.96 against 0.98). - The release bar for wrong holds was 0.30 per 100 jobs, set from an unconstrained GPU run. Under the arity grammar and retry on the Mac, neither version meets it, and v2 is further from it than v1. Because a wrong hold is the costlier error, v1 stays the recommended model; v2 is published as experimental, not as a release that passed the bar. - All numbers are on a dev set. **v2 has not been evaluated on a held-out test set yet.** - Real job postings were used only for qualitative checks, never for the numbers above. - The v1 numbers in its own card (valid 0.89, agreement 0.773, wrong holds 0.27) were measured with an earlier setup (an unconstrained run, no arity grammar, no retry) and are not comparable with this table. ### Known limitations - **Relocation conditions** ("only for a dream company or $X") are often written too strictly, or with the wrong polarity. - Some trees still fail semantic validation even after two retries (about 4 users in 100 on dev). - Trained only on synthetic, English, US job preferences. - A tree is a program written by a small model: validate it, and look at the "explain" lines before trusting the verdicts. ### Notes The GGUF header was edited to drop Qwen3.5's unused multi-token-prediction layer (`block_count` 24, `nextn_predict_layers` 0), which wllama otherwise refuses. The tensor data are byte for byte the ones of the trained export. All data used to train and evaluate is invented. License: Apache-2.0, as for v1. ## v1 (repository root, unchanged; recommended) `tree-compiler-0.8b-q8_0.gguf`, `mm_factors.gbnf` (the earlier grammar without arities), `example-messages.json`, `example-output.json`, and the 4-bit phone shards under `phone/`. Results on its dev set (valid 0.89, agreement 0.773, wrong holds 0.27 per 100, unconstrained run) and its limits are as stated in the v1 card below. There is **no 4-bit build of v2**; the phone shards are v1 only. ### v1 card (as published) Distilled by SFT from the best of several teacher samples (Claude Sonnet and Haiku, Gemini Flash) per user, chosen by **executing** each candidate tree against gold verdicts. About 11% of v1 trees fail semantic validation (an operator with the wrong number of arguments, a reference to a name never declared, an undefined list). The v1 grammar only ensures well-formed JSON and the top-level shape. Made by Csaba Garay, October 2026.