Instructions to use csabag76/tree-compiler-0.8b with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use csabag76/tree-compiler-0.8b with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible server with a web UI: llama serve -hf csabag76/tree-compiler-0.8b:Q4_K_M # Run inference directly in the terminal: llama cli -hf csabag76/tree-compiler-0.8b:Q4_K_M
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama serve -hf csabag76/tree-compiler-0.8b:Q4_K_M # Run inference directly in the terminal: llama cli -hf csabag76/tree-compiler-0.8b:Q4_K_M
Build from source code
git clone https://github.com/ggml-org/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-app # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama serve -hf csabag76/tree-compiler-0.8b:Q4_K_M # Run inference directly in the terminal: ./build/bin/llama cli -hf csabag76/tree-compiler-0.8b:Q4_K_M
Use Docker
docker model run hf.co/csabag76/tree-compiler-0.8b:Q4_K_M
- Llama new
- LM Studio
- Jan
- Ollama
How to use csabag76/tree-compiler-0.8b with Ollama:
ollama run hf.co/csabag76/tree-compiler-0.8b:Q4_K_M
- Unsloth Desktop
- Pi
How to use csabag76/tree-compiler-0.8b with Pi:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf csabag76/tree-compiler-0.8b:Q4_K_M
Configure the model in Pi
# Install Pi: npm install -g @earendil-works/pi-coding-agent # Add to ~/.pi/agent/models.json: { "providers": { "llama-cpp": { "baseUrl": "http://localhost:9931/v1", "api": "openai-completions", "apiKey": "none", "models": [ { "id": "csabag76/tree-compiler-0.8b:Q4_K_M" } ] } } }Run Pi
# Start Pi in your project directory: pi
- Docker Model Runner
How to use csabag76/tree-compiler-0.8b with Docker Model Runner:
docker model run hf.co/csabag76/tree-compiler-0.8b:Q4_K_M
- Lemonade
How to use csabag76/tree-compiler-0.8b with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull csabag76/tree-compiler-0.8b:Q4_K_M
Run and chat with the model
lemonade run user.tree-compiler-0.8b-Q4_K_M
List all available models
lemonade list
- Hermes Agent
How to use csabag76/tree-compiler-0.8b with Hermes Agent:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf csabag76/tree-compiler-0.8b:Q4_K_M
Configure Hermes
# Install Hermes: curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash hermes setup # Point Hermes at the local server: hermes config set model.provider custom hermes config set model.base_url http://127.0.0.1:9931/v1 hermes config set model.default csabag76/tree-compiler-0.8b:Q4_K_M
Run Hermes
hermes
- Atomic Chat
- OpenClaw
How to use csabag76/tree-compiler-0.8b with OpenClaw:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf csabag76/tree-compiler-0.8b:Q4_K_M
Configure OpenClaw
# Install OpenClaw: npm install -g openclaw@latest # Register the local server and set it as the default model: openclaw onboard --non-interactive --mode local \ --auth-choice custom-api-key \ --custom-base-url http://127.0.0.1:9931/v1 \ --custom-model-id "csabag76/tree-compiler-0.8b:Q4_K_M" \ --custom-provider-id llama-cpp \ --custom-compatibility openai \ --custom-text-input \ --accept-risk \ --skip-health
Run OpenClaw
openclaw agent --local --agent main --message "Hello from Hugging Face"
Card: v1 recommended (demo), v2 experimental
Browse files
README.md
CHANGED
|
@@ -18,10 +18,11 @@ A Qwen3.5-0.8B fine-tune that turns a job seeker's preferences, written in plain
|
|
| 18 |
runs that object on each job or recruiter email and returns **Through / Maybe / Held** with reasons. The model writes
|
| 19 |
the decision procedure once; it never sees the emails.
|
| 20 |
|
| 21 |
-
This repository holds two versions. **
|
| 22 |
-
|
|
|
|
| 23 |
|
| 24 |
-
## v2 (
|
| 25 |
|
| 26 |
Qwen3.5-0.8B, LoRA SFT, merged into the weights and exported as q8_0. It was trained only on synthetic data: invented
|
| 27 |
job seekers and invented jobs, generated with the second version of the synthetic set ("synth v2"), whose preference
|
|
@@ -38,9 +39,12 @@ Gemini Flash); for each user, the candidate tree that did best when **executed**
|
|
| 38 |
| `example-output.json` | the tree v2 wrote for that user (greedy, with the grammar). It passes the tree validator but was not scored against gold verdicts, and it is not a faithful reading of the message: it adds a pay ramp the user never asked for, and it has no accept rule for the dream companies |
|
| 39 |
| `SHA256SUMS` | checksums of the files above |
|
| 40 |
|
| 41 |
-
### How to use it
|
| 42 |
|
| 43 |
-
|
|
|
|
|
|
|
|
|
|
| 44 |
2. **Validate** the tree with a semantic validator (arities, names that are declared, lists that exist).
|
| 45 |
3. If it is not valid, **retry up to 2 times** at temperature 0.3 with seeds 1234+k (k = 1, 2), no top-k/top-p/min-p, with the same grammar, and keep the first valid tree. If none is valid, treat the user as failed; do not run an invalid tree.
|
| 46 |
|
|
@@ -60,15 +64,15 @@ invalid tree as wrong on every job. A *wrong hold* hides a job that fits the use
|
|
| 60 |
|
| 61 |
| model | valid trees | agreement (all) | wrong holds per 100 jobs |
|
| 62 |
|---|---|---|---|
|
| 63 |
-
| v1, same setup | 0.98 | 0.694 | 0.52 |
|
| 64 |
-
|
|
| 65 |
|
| 66 |
Read this table carefully: v2 agrees more often with the gold verdicts, but it has **more** wrong holds per 100 jobs
|
| 67 |
than v1 under this setup (0.84 against 0.52), and slightly fewer valid trees (0.96 against 0.98).
|
| 68 |
|
| 69 |
-
- The release bar for wrong holds was 0.30 per 100 jobs
|
| 70 |
-
|
| 71 |
-
|
| 72 |
- All numbers are on a dev set. **v2 has not been evaluated on a held-out test set yet.**
|
| 73 |
- Real job postings were used only for qualitative checks, never for the numbers above.
|
| 74 |
- The v1 numbers in its own card (valid 0.89, agreement 0.773, wrong holds 0.27) were measured with an earlier setup (an unconstrained run, no arity grammar, no retry) and are not comparable with this table.
|
|
@@ -86,7 +90,7 @@ The GGUF header was edited to drop Qwen3.5's unused multi-token-prediction layer
|
|
| 86 |
`nextn_predict_layers` 0), which wllama otherwise refuses. The tensor data are byte for byte the ones of the trained
|
| 87 |
export. All data used to train and evaluate is invented. License: Apache-2.0, as for v1.
|
| 88 |
|
| 89 |
-
## v1 (repository root, unchanged)
|
| 90 |
|
| 91 |
`tree-compiler-0.8b-q8_0.gguf`, `mm_factors.gbnf` (the earlier grammar without arities), `example-messages.json`,
|
| 92 |
`example-output.json`, and the 4-bit phone shards under `phone/`. Results on its dev set (valid 0.89, agreement 0.773,
|
|
|
|
| 18 |
runs that object on each job or recruiter email and returns **Through / Maybe / Held** with reasons. The model writes
|
| 19 |
the decision procedure once; it never sees the emails.
|
| 20 |
|
| 21 |
+
This repository holds two versions. **v1 (the files in the repository root) is the recommended one**. Use it with the
|
| 22 |
+
arity grammar and the validate-and-retry step described below; this is what the public demo runs. **v2** (folder `v2/`)
|
| 23 |
+
is an **experimental** retrain: it agrees with the gold verdicts more often, but it hides more good jobs (see Results).
|
| 24 |
|
| 25 |
+
## v2 (experimental)
|
| 26 |
|
| 27 |
Qwen3.5-0.8B, LoRA SFT, merged into the weights and exported as q8_0. It was trained only on synthetic data: invented
|
| 28 |
job seekers and invented jobs, generated with the second version of the synthetic set ("synth v2"), whose preference
|
|
|
|
| 39 |
| `example-output.json` | the tree v2 wrote for that user (greedy, with the grammar). It passes the tree validator but was not scored against gold verdicts, and it is not a faithful reading of the message: it adds a pay ramp the user never asked for, and it has no accept rule for the dream companies |
|
| 40 |
| `SHA256SUMS` | checksums of the files above |
|
| 41 |
|
| 42 |
+
### How to use it (v1 and v2 alike)
|
| 43 |
|
| 44 |
+
For **v1** (recommended), use the root GGUF with the same arity grammar (`v2/mm_factors.gbnf`) and the same retry; for v2, use the `v2/` GGUF.
|
| 45 |
+
|
| 46 |
+
|
| 47 |
+
1. Decode with the arity grammar `v2/mm_factors.gbnf`, greedy, `enable_thinking` off in the chat template, up to 4096 new tokens.
|
| 48 |
2. **Validate** the tree with a semantic validator (arities, names that are declared, lists that exist).
|
| 49 |
3. If it is not valid, **retry up to 2 times** at temperature 0.3 with seeds 1234+k (k = 1, 2), no top-k/top-p/min-p, with the same grammar, and keep the first valid tree. If none is valid, treat the user as failed; do not run an invalid tree.
|
| 50 |
|
|
|
|
| 64 |
|
| 65 |
| model | valid trees | agreement (all) | wrong holds per 100 jobs |
|
| 66 |
|---|---|---|---|
|
| 67 |
+
| **v1, same setup (recommended)** | 0.98 | 0.694 | 0.52 |
|
| 68 |
+
| v2 (experimental) | 0.96 | 0.753 | 0.84 |
|
| 69 |
|
| 70 |
Read this table carefully: v2 agrees more often with the gold verdicts, but it has **more** wrong holds per 100 jobs
|
| 71 |
than v1 under this setup (0.84 against 0.52), and slightly fewer valid trees (0.96 against 0.98).
|
| 72 |
|
| 73 |
+
- The release bar for wrong holds was 0.30 per 100 jobs, set from an unconstrained GPU run. Under the arity grammar and
|
| 74 |
+
retry on the Mac, neither version meets it, and v2 is further from it than v1. Because a wrong hold is the costlier
|
| 75 |
+
error, v1 stays the recommended model; v2 is published as experimental, not as a release that passed the bar.
|
| 76 |
- All numbers are on a dev set. **v2 has not been evaluated on a held-out test set yet.**
|
| 77 |
- Real job postings were used only for qualitative checks, never for the numbers above.
|
| 78 |
- The v1 numbers in its own card (valid 0.89, agreement 0.773, wrong holds 0.27) were measured with an earlier setup (an unconstrained run, no arity grammar, no retry) and are not comparable with this table.
|
|
|
|
| 90 |
`nextn_predict_layers` 0), which wllama otherwise refuses. The tensor data are byte for byte the ones of the trained
|
| 91 |
export. All data used to train and evaluate is invented. License: Apache-2.0, as for v1.
|
| 92 |
|
| 93 |
+
## v1 (repository root, unchanged; recommended)
|
| 94 |
|
| 95 |
`tree-compiler-0.8b-q8_0.gguf`, `mm_factors.gbnf` (the earlier grammar without arities), `example-messages.json`,
|
| 96 |
`example-output.json`, and the 4-bit phone shards under `phone/`. Results on its dev set (valid 0.89, agreement 0.773,
|