manuelgscom/askmdb-0.8b-GGUF

askmdb-0.8b (GGUF)

A full fine-tune of Qwen/Qwen3.5-0.8B that turns a question about a MongoDB database, in Spanish or English, into one read-only aggregate pipeline, answered as a small JSON object. It reads a compact description of the database's schema in the prompt, so it works on databases it has never seen. It is the default model of askmdb, an offline command-line tool that samples the schema, runs the model locally through llama.cpp in a single native binary, checks the pipeline with a deterministic read-only verifier and only then runs it. On machines with a suitable dedicated GPU, askmdb can use the larger askmdb-4b or askmdb-9b instead; askmdb-0.8b stays the CPU model and the fallback (family).

• Author: Manuel García Sanz
• Website: https://manuelgs.com
• Source code of askmdb: https://github.com/manuelgscom/askmdb
• Format: GGUF, Q4_K_M (imatrix) · 529.3 MB
• Test accuracy: 83.6 % (418/500), scored by execution, versus 47.8 % for Qwen3.5-9B and 45.4 % for Gemma 4 E4B-it without fine-tuning
• In the askmdb CLI: the CPU model, default and fallback. 1.18 s median per question with the schema cached, on one reference desktop (details)
• The askmdb family: four sizes of the same model (0.8B, 2B, 4B, 9B), all in one collection

askmdb live: ask in plain English; it shows the pipeline, explains it, checks that it only reads and runs it (in auto mode, a join asks first). Real, unedited recording with askmdb-0.8b on a desktop CPU.

Resumen en español

askmdb-0.8b es un ajuste completo de Qwen3.5-0.8B que convierte una pregunta en castellano o en inglés sobre una base de datos MongoDB en un único pipeline de aggregate de solo lectura, con una explicación breve. Lee en el prompt una descripción compacta del esquema, así que sirve para bases que nunca ha visto. Es el modelo por defecto de la CLI askmdb, que funciona sin conexión: muestrea el esquema, ejecuta el modelo en local con llama.cpp, revisa el pipeline con un verificador determinista de solo lectura y solo entonces lo ejecuta. Se usa en la CPU y como vuelta segura; con una GPU dedicada adecuada, askmdb puede usar askmdb-4b o askmdb-9b. En el test público de 500 casos, puntuado ejecutando cada consulta, acierta 418 de 500 (83,6 %), frente al 47,8 % de Qwen3.5-9B sin ajustar. El modelo no decide la seguridad: dentro de askmdb lo hace un verificador determinista y solo se ejecutan pipelines de lectura; fuera de askmdb no hay verificador, así que revisa siempre la consulta y conéctate con un usuario de solo lectura.

The askmdb family

The same task, data, contract and prompt in four sizes. Scores on the public test set manuelgscom/askmdb-test (500 cases, scored by execution), all in Q4_K_M.

Model	Base	GGUF (Q4_K_M)	Test	In the askmdb CLI	Repository
askmdb-0.8b	Qwen3.5-0.8B	529.3 MB	83.6 %	CPU (default and fallback)	manuelgscom/askmdb-0.8b-GGUF (this one)
askmdb-2b	Qwen3.5-2B	1,274.4 MB	86.8 %	not used (llama-server only)	manuelgscom/askmdb-2b-GGUF
askmdb-4b	Qwen3.5-4B	2,708.8 MB	89.6 %	GPU, ≥ 4 GiB free VRAM	manuelgscom/askmdb-4b-GGUF
askmdb-9b	Qwen3.5-9B	5,629.1 MB	91.6 %	GPU, ≥ 8 GiB free VRAM, if you choose it	manuelgscom/askmdb-9b-GGUF

Which one?

 the askmdb CLI, first run:
   dedicated GPU, ≥ 8 GiB free VRAM ──► you choose: 1) askmdb-4b (Enter, faster)  2) askmdb-9b (more accurate)
   dedicated GPU, ≥ 4 GiB free VRAM ──► askmdb-4b
   anything else (CPU, integrated GPU) ──► askmdb-0.8b on the CPU

 your own application (llama-server, CPU or GPU) ──► any of the four, askmdb-2b included:
                                                     pick by memory and accuracy

What it is and intended use

The model is meant to be used through the askmdb CLI:

 question ─► schema block ─► model ─► strict parser ─► read-only verifier ─► mode ─► aggregate
             (sampled from      (one      (exact contract;   (deterministic C++:     (suggest,
              200 documents      JSON      anything else      read, warn or           confirm,
              per collection,    object)   runs nothing)      blocked)                auto)
              cached)

• The model proposes; code decides. The model writes one aggregate pipeline (or refuse / unsure). It never decides whether that pipeline is safe to run.
• Honest when it cannot answer. It answers refuse for writes, administration or off-topic requests, and unsure when the data is not in the schema, instead of inventing a query.
• Schema-aware. The prompt carries a compact schema block (collections, field paths, types and frequent values), so the field names and spellings come from your database.
• Local. After the first model download, askmdb makes no network requests other than to your MongoDB server: no accounts, no API keys, no telemetry.

The model was trained only for this contract. It is not a general chat model.

In the askmdb CLI

askmdb is one binary for CPU and GPU. On the first run it looks for a dedicated GPU through Vulkan. Without one (CPU only, integrated GPU, no Vulkan loader), or if the GPU check fails, it uses this model on the CPU: 1.18 s median per question with the schema cached, and 3.02 s for the first question against a database (the schema is sampled and the prompt prefix computed once, then cached).

With a dedicated GPU and at least 4 GiB of free VRAM it offers askmdb-4b, and with 8 GiB or more it lets you choose between askmdb-4b and askmdb-9b. It asks before every download, checks it with SHA-256 and saves the choice (askmdb --detect asks again).

GPU support has been tested on NVIDIA only. Detection and inference use standard Vulkan, but AMD and Intel GPUs have not been tested yet.

Times are for the whole askmdb process per question (start, model load, prompt, generation), as the user perceives it, on one reference desktop with a 16 GB GPU (CPU runs with 8 threads): 30 questions, five runs each with the schema already cached and one first question against the database. They depend on the machine. askmdb starts one process per question and loads the weights each time, which is most of the time on the GPU.

Model · device	Per question, schema cached (median / p90)	First question against a database (median / p90)	VRAM in use
askmdb-0.8b · CPU	1.18 s / 1.79 s	3.02 s / 3.60 s	none
askmdb-0.8b · GPU (with --device gpu)	0.72 s / 0.82 s	0.79 s / 0.92 s	1.2 GiB
askmdb-2b · GPU (reference, not used by the CLI)	0.94 s / 1.12 s	0.98 s / 1.13 s	1.9 GiB
askmdb-4b · GPU	1.34 s / 1.68 s	1.54 s / 1.85 s	3.3 GiB
askmdb-9b · GPU	1.85 s / 2.40 s	2.14 s / 2.63 s	5.5 GiB

Safety in askmdb: the read-only verifier

Safety in askmdb is decided by code, not by the model: a write request gets an honest "no", and a hand-written $out hidden in a $facet is blocked by the verifier (shown with its standalone tool, askmdb_verify, built from the same source). Real, unedited recording with askmdb-0.8b; the verifier is the same whatever model runs.

• A deterministic verifier in C++ walks the exact BSON that would be sent to the server and gives every pipeline a level: read (known read-only stages and operators), warn (reads, but may be expensive or surprising, such as $lookup or $sample: never runs without an explicit yes) or blocked (writes, administration, JavaScript, other databases, or anything it does not understand: never runs, in any mode).
• The most restrictive part wins, sub-pipelines included ($facet, $lookup, $unionWith). Duplicated keys and multi-key stages are blocked.
• askmdb only runs aggregate, with a time limit and a row limit. It is not a replacement for access control: connect with a user that has the read role.

Files

File	Quantization	Size (bytes)	SHA-256
askmdb-0.8b-1.0.0-Q4_K_M.gguf	Q4_K_M, with imatrix	529,298,016	856c3502c6fe4061a77d9a4940a5e36670f35dded1d32cecc298af978cb3509c
LICENSE	Apache-2.0

sha256sum askmdb-0.8b-1.0.0-Q4_K_M.gguf

The GGUF carries general.* metadata (name, author, URL, version, license, base model). The tensors are identical to the file evaluated below; only the metadata was added.

Using it with llama-server (CPU or GPU)

askmdb builds the prompt itself, so none of this is needed to use the CLI. To call the model from your own code, follow the contract (version 1) exactly; the model was trained on this format and nothing else. Reasoning is always disabled.

Prompt

The prompt is Qwen3.5's chat template with thinking disabled (apply_chat_template(..., add_generation_prompt=True, enable_thinking=False)), with two turns only: a system turn (fixed instructions, a blank line, then the schema block) and the question. No few-shot examples, no reasoning.

Instructions (exactly two lines; $$NOW is literal):

Answer the question with one MongoDB aggregate pipeline over the database described below.
Reply with only JSON {"type":"query|refuse|unsure","collection":"...","pipeline":[...],"explanation":"..."}: type is refuse for writes or off-topic requests and unsure if the schema lacks the data, and then collection is "" and pipeline is []; use $$NOW for relative dates; explanation is one short sentence in the language of the question.

Schema block (an invented example database):

db=bakery
## orders
_id oid
placed_at date
customer_id str
channel str ["shop","web","phone"]
status str ["delivered","pending","cancelled"]
total double
items[] obj
items[].product_id str
items[].qty int
## products
_id str
name str
category str ["bread","pastry","cake"]
price double
gluten_free bool

• First line db=<database>, then one ## <collection> header per collection, in alphabetical order.
• One line per field path: path type [frequent values]. Dotted paths for subdocuments; items[] is an array and items[].qty a field of its elements. A ? after the path marks a field missing in some sampled documents.
• Types: oid str int long double decimal bool date obj arr null bin other; mixed types as unions (int|str).
• Frequent values (a JSON array) only for strings with at most 12 distinct values of at most 30 characters; never values that look like an email or a phone number.
• Limits: depth 4, at most 40 paths per collection, about 4,500 bytes in total (trimmed deterministically: values first, then the deepest paths). In askmdb the block comes from sampling 200 documents per collection; askmdb --print-schema prints it, so you can reuse exactly the block the CLI would send.

User turn: the question, with whitespace collapsed to single spaces, at most 1000 bytes.

Full prompt string (line breaks shown for readability; \n are real newlines):

<|im_start|>system\n{instructions}\n\n{schema block}<|im_end|>\n<|im_start|>user\n{question}<|im_end|>\n<|im_start|>assistant\n<think>\n\n</think>\n\n

The prompt ends with an empty <think> block, so the model starts writing the JSON straight away. The schema block has no trailing newline (the template trims every message). Tokenize with special tokens parsed and without BOS. A context of 4096 tokens is enough for the largest schema block plus the answer.

Answer

Exactly one JSON object with exactly these four keys, in this order:

{"type":"query|refuse|unsure","collection":"...","pipeline":[...],"explanation":"..."}

• type is query, refuse (writes, administration or off-topic) or unsure (the data is not in the schema).
• With query, collection is one collection of the schema and pipeline is an array of 1 to 20 stages, each an object with a single key. Otherwise collection is "" and pipeline is [].
• Special values in relaxed Extended JSON: {"$date":"2026-01-01T00:00:00Z"} (UTC), {"$oid":"…"}, and {"$numberDecimal":"…"} only for decimal fields.
• Relative dates use $$NOW with $dateTrunc / $dateSubtract / $dateAdd inside $expr; MongoDB resolves $$NOW when the pipeline runs.
• The model adds $limit only when the question asks for a top N; askmdb always appends its own row limit.
• explanation is one short sentence in the language of the question.
• Nothing before or after the object. askmdb cuts the output at the closing top-level } and rejects anything else as invalid (it runs nothing).

Decode greedily and cap at 384 tokens (as askmdb does).

Example

llama-server -m askmdb-0.8b-1.0.0-Q4_K_M.gguf -c 4096 --reasoning off --port 8080

A GPU build of llama.cpp offloads layers to the GPU on its own (-ngl auto; -ngl 99 puts every layer there); add -ngl 0 (or --device none) to keep it on the CPU. Any llama.cpp backend works (CPU, CUDA, Vulkan…); the askmdb CLI uses Vulkan. --reasoning off sets enable_thinking=false for the chat API (checked with llama.cpp v0.7.0; the flag exists since build b8287); older builds use --jinja --chat-template-kwargs '{"enable_thinking":false}', which v0.7.0 still accepts with a deprecation warning.

Raw prompt with /completion (recommended: you control the exact string):

import json, urllib.request

INSTRUCTIONS = (
    "Answer the question with one MongoDB aggregate pipeline over the database described below.\n"
    'Reply with only JSON {"type":"query|refuse|unsure","collection":"...","pipeline":[...],"explanation":"..."}: '
    'type is refuse for writes or off-topic requests and unsure if the schema lacks the data, '
    'and then collection is "" and pipeline is []; use $$NOW for relative dates; '
    "explanation is one short sentence in the language of the question."
)
SCHEMA = """db=bakery
## orders
_id oid
placed_at date
customer_id str
channel str ["shop","web","phone"]
status str ["delivered","pending","cancelled"]
total double
items[] obj
items[].product_id str
items[].qty int
## products
_id str
name str
category str ["bread","pastry","cake"]
price double
gluten_free bool"""
question = "Revenue per sales channel in the last 30 days"

system = INSTRUCTIONS + "\n\n" + SCHEMA
prompt = (
    f"<|im_start|>system\n{system}<|im_end|>\n"
    f"<|im_start|>user\n{question}<|im_end|>\n"
    "<|im_start|>assistant\n<think>\n\n</think>\n\n"
)
body = {"prompt": prompt, "n_predict": 384, "temperature": 0, "top_k": 1}
req = urllib.request.Request("http://127.0.0.1:8080/completion",
                             data=json.dumps(body).encode(),
                             headers={"Content-Type": "application/json"})
answer = json.loads(json.load(urllib.request.urlopen(req))["content"])
print(answer["collection"], json.dumps(answer["pipeline"]))

Chat API with /v1/chat/completions: the server applies the chat template stored in the GGUF; turn reasoning off per request with chat_template_kwargs (or with --reasoning off on the server, as above):

body = {
    "messages": [
        {"role": "system", "content": system},
        {"role": "user", "content": question},
    ],
    "chat_template_kwargs": {"enable_thinking": False},
    "max_tokens": 384, "temperature": 0, "top_k": 1,
}
req = urllib.request.Request("http://127.0.0.1:8080/v1/chat/completions",
                             data=json.dumps(body).encode(),
                             headers={"Content-Type": "application/json"})
print(json.load(urllib.request.urlopen(req))["choices"][0]["message"]["content"])

The chat template stored in this GGUF is the one the model was trained with: Qwen3.5's template, which keeps reasoning off unless enable_thinking is true. llama-server, however, passes enable_thinking: true by default (--reasoning auto): without --reasoning off on the server or enable_thinking: false in the request, the prompt ends with an open <think> block, outside the contract: the JSON may come back in reasoning_content with an empty content, or after a reasoning the model was not trained to write. A request's chat_template_kwargs overrides the server flag.

Answers of this model (Q4_K_M, CPU, greedy) on that schema:

Question
Revenue per sales channel in the last 30 days
¿Cuáles son los 5 productos con más unidades vendidas este año?
Delete the cancelled orders
Which courier delivered each order?

{"type":"query","collection":"orders","pipeline":[{"$match":{"$expr":{"$gte":["$placed_at",{"$dateSubtract":{"startDate":"$$NOW","unit":"day","amount":30}}]}}},{"$group":{"_id":"$channel","total":{"$sum":"$total"}}}],"explanation":"Sums the order totals per sales channel over the last 30 days."}
{"type":"query","collection":"orders","pipeline":[{"$match":{"$expr":{"$gte":["$placed_at",{"$dateTrunc":{"date":"$$NOW","unit":"year"}}]}}},{"$unwind":"$items"},{"$group":{"_id":"$items.product_id","unidades":{"$sum":"$items.qty"}}},{"$sort":{"unidades":-1}},{"$limit":5}],"explanation":"Muestra los 5 productos con más unidades vendidas este año."}
{"type":"refuse","collection":"","pipeline":[],"explanation":"askmdb only reads data: it cannot delete documents."}
{"type":"unsure","collection":"","pipeline":[],"explanation":"The schema has no courier data for orders."}

Never run a pipeline blindly. Outside askmdb there is no verifier: reject anything that is not a plain read ($out, $merge, unknown stages, JavaScript operators such as $function or $where), and connect with a user that only has the read role.

Evaluation

Test set: manuelgscom/askmdb-test, 500 questions on 10 synthetic MongoDB databases (254 in Spanish, 246 in English; 449 with a query, 31 that must be refused, 20 whose data is not in the schema). Every case is scored by execution: the predicted pipeline runs on the same database as the reference and the results are compared; no model grades the answers.

All models in Q4_K_M, reasoning disabled, greedy decoding, same llama.cpp version, 384-token cap. The askmdb models get the plain contract above, through the askmdb binary. The models without fine-tuning get the same instructions plus six worked examples on a separate example database (few-shot), through llama-server, which helps them.

Model	Params	Fine-tuned	GGUF	Correct	Accuracy	95 % CI (Wilson)	Invalid JSON
Qwen3.5-0.8B	0.8 B	no (few-shot)	532.5 MB	33/500	6.6 %	4.7–9.1 %	258
Qwen3.5-2B	2 B	no (few-shot)	1,280.8 MB	102/500	20.4 %	17.1–24.2 %	72
Gemma 4 E2B-it	2 B effective	no (few-shot)	3,106.7 MB	221/500	44.2 %	39.9–48.6 %	43
Qwen3.5-4B	4 B	no (few-shot)	2,740.9 MB	187/500	37.4 %	33.3–41.7 %	65
Gemma 4 E4B-it	4 B effective	no (few-shot)	4,977.2 MB	227/500	45.4 %	41.1–49.8 %	112
Qwen3.5-9B	9 B	no (few-shot)	5,680.5 MB	239/500	47.8 %	43.5–52.2 %	61
askmdb-0.8b	0.8 B	yes	529.3 MB	418/500	83.6 %	80.1–86.6 %	3
askmdb-2b	2 B	yes	1,274.4 MB	434/500	86.8 %	83.6–89.5 %	3
askmdb-4b	4 B	yes	2,708.8 MB	448/500	89.6 %	86.6–92.0 %	1
askmdb-9b	9 B	yes	5,629.1 MB	458/500	91.6 %	88.8–93.7 %	2

MB = 10^6 bytes. The Qwen3.5 rows are the published Qwen/Qwen3.5-* models; the Gemma 4 models are the instruction-tuned releases, with effective parameters (their files are larger than a Qwen model with the same count).

Where the errors are

Metric	askmdb-0.8b	askmdb-2b	askmdb-4b	askmdb-9b	Qwen3.5-9B	Gemma 4 E4B-it
Invalid JSON	3	3	1	2	61	112
Wrong type	3	2	1	1	34	18
Wrong collection	5	6	6	2	3	3
Blocked by the verifier	1	0	1	3	40	57
Execution error	1	3	1	0	5	6
Different result	69	52	42	34	118	77
Total wrong	82	66	52	42	261	273
Correct refuse (of 31)	31	31	31	31	30	31
Correct unsure (of 20)	18	18	20	20	0	3
False refusals (of 449 query cases)	1	0	1	1	14	2

Fine-tuning almost removes format errors (invalid JSON, wrong type, blocked pipelines) and the "when not to answer" errors. Most of the remaining errors are a valid, read-only pipeline that runs but computes something else: 69 of this model's 82 wrong answers.

By language: 81.5 % in Spanish (207/254) and 85.8 % in English (211/246).

By category: the hardest categories for this model are joins ($lookup) (15/31) and arrays (33/48). Categories are small (a few dozen cases each), so differences of a few cases are noise.

Per-category table (correct answers)

Category	Cases	askmdb-0.8b	askmdb-2b	askmdb-4b	askmdb-9b	Qwen3.5-9B	Gemma 4 E4B-it
grouping	56	45	47	48	51	29	28
arrays	48	33	40	41	41	17	25
counting	45	41	39	39	44	29	32
distinct values	24	19	21	19	22	12	6
absolute dates	31	26	29	30	30	0	11
relative dates	79	67	65	69	70	8	0
filters	62	57	58	60	58	54	54
joins ($lookup)	31	15	17	20	22	13	6
refuse (writes, off-topic)	31	31	31	31	31	30	31
subdocuments	38	37	35	37	38	31	18
top N	35	29	34	34	31	16	13
unsure (data not in the schema)	20	18	18	20	20	0	3

Methodology

• Independent, synthetic test. 10 databases filled by a seeded generator from declarative specs (published with the test set); half of them come from domains that no training database covers. Each question and its reference pipeline were written by one LLM; an LLM of a different family wrote a second pipeline blind, and a case was kept only when both returned the same result and the reference passed mutation checks (small changes to the pipeline must change the result). The test was frozen with a SHA-256 before any model was evaluated, and the training data was decontaminated against it.
• Scored by execution, lenient on form, strict on content. Column names, key order, extra columns, numeric types, a date or ObjectId written as a string or rounding to 2 decimals do not matter; a different value, a missing or extra row, or a wrong order (when the question asks for one) do. $$NOW is replaced by each database's fixed reference date, so results do not depend on the day of scoring.
• A pipeline the askmdb verifier blocks counts as wrong, even if it would return the right data.
• What it does not measure. The test only keeps questions with one clear answer; real questions can be ambiguous. Verdicts have no manual review. With 500 cases the 95 % interval is about ±3 points, and much wider per category. Each size was trained once (one seed), so neighbouring sizes a few cases apart are not clearly ordered.

Training

Base	Qwen/Qwen3.5-0.8B (revision 2fc06364715b967f1860aea9cf38778875588b17)
Method	Full fine-tune of all weights (no LoRA, no adapters)
Data	14,248 training examples, 752 validation
Hyperparameters	lr 1e-5 (cosine, 3 % warmup), effective batch 32, 2 epochs with early stopping on the validation loss, max sequence length 896, fp32 master weights with bf16 autocast (F32 tensors kept), 8-bit AdamW, loss on the answer only, seed 3407
Hardware	1 GPU with 96 GB, about 20 minutes
Quantization	Q4_K_M with an importance matrix (imatrix) computed on a sample of the training data (never the test)

Data. 15,000 synthetic examples (14,248 for training, 752 for validation), questions in Spanish and English (about half each) on about 110 generated training databases, none of them a test database. They were written with LLM assistance in three ways: a question and two pipelines from two independent LLM authors, kept only when both return the same result; pipelines built by code from intent templates with real values, phrased as questions by an LLM; and templated refuse / unsure cases. Every example passed the output contract, the askmdb verifier, execution on its database with a non-empty result, mutation checks, near-duplicate removal and decontamination against the test. The schema block in each prompt was produced by the askmdb binary itself. The training data and the training scripts are not published.

Limitations

• aggregate only, read only. No find, no writes: write requests are refused by the model and blocked by askmdb. MongoDB 5.0 or newer.
• No Atlas Search or vector search ($search, $vectorSearch).
• It can be wrong: it gets 418 of the 500 test cases right. A wrong pipeline can look plausible and run fine. Always check the result.
• It is not a security tool. The model can propose a pipeline that writes or is expensive; it does not classify risk. In askmdb the verifier does; outside askmdb, your application must.
• It only knows what the schema block shows. askmdb samples 200 documents per collection: rare fields can be missing, and very large schemas are trimmed (at most 40 paths per collection, depth 4).
• Joins ($lookup) are the weakest category for every size.
• Trained for one task with one fixed prompt. Other prompts, chat use or reasoning mode are unsupported.
• The askmdb CLI runs on Linux x86_64 (on Windows, through WSL 2). Its GPU support has been tested on NVIDIA only; it uses standard Vulkan, but AMD and Intel GPUs have not been tested yet. The GGUF itself runs anywhere llama.cpp does.

Changes from the base model

• All weights fine-tuned for the contract above (question + schema block → one aggregate pipeline as JSON, with refuse and unsure).
• Tokenizer unchanged. Chat template: the Qwen3.5 template the model was trained with, which keeps reasoning off unless enable_thinking is true.
• Language model only: no vision encoder and no multi-token-prediction head.
• Converted to GGUF and quantized to Q4_K_M with imatrix; general.* metadata added.

License

Apache-2.0, the same license as the base model Qwen3.5-0.8B. See LICENSE. The test set manuelgscom/askmdb-test is CC BY 4.0.

MongoDB is a trademark of MongoDB, Inc. askmdb is an independent project, not affiliated with or endorsed by MongoDB, Inc.