README.md

---
language: en
library_name: mlx
pipeline_tag: text-generation
base_model: mlx-community/Qwen3.5-0.8B
tags:
- mlx
- query-understanding
license: apache-2.0
---

# Keyword-0.8B-4bit

Extracts boolean search terms from a natural-language question. One half of a two-model query-planning pair used by SyncNotes to turn a
user's question into a deterministic search, so that retrieval is driven by a model
rather than by a stop-word split.

## Output

Greedy decoding, temperature 0. Emits a single small JSON object:

```json
{"terms":["invoice","acme"],"alignment":"both"}
```

Parse defensively. Under an off-distribution prompt these specialists can emit degenerate
repeated text with no closing brace. A caller must treat unparseable output as a planning
failure and fall back — never present it, and never report a full model-driven pipeline
when planning actually degraded.

## Training data

**Synthetic corpus only.** Fine-tuned on a synthetic Search Quality Lab corpus of 192
generated personas (6,384 synthetic notes, 2,575 synthetic questions). No real user notes,
note bodies, OCR text, or question text were used at any point. That was an explicit
constraint of the training campaign, not an afterthought.

## Build

Full fine-tune (`fine_tune_type: full`, not LoRA) over a `Qwen3.5-0.8B` base, fused and
then quantized locally with `mlx_lm` 0.31.1 / `mlx` 0.31.1. Affine 4-bit, group size 64;
the converter reported 4.508 bits per weight.

## Intended use

Query planning for on-device personal search. These models do not write prose and are not
answer composers — they plan a search that a deterministic engine then runs.