---
license: other
license_name: deepseek-license
license_link: LICENSE
base_model: Qwen/Qwen2.5-Coder-1.5B
base_model_relation: finetune
library_name: transformers
pipeline_tag: text-generation
language:
  - en
  - pt
tags:
  - mongodb
  - mongosh
  - code
  - code-completion
  - fill-in-the-middle
  - extended-json
  - atlas-search
  - distillation
  - deepseek-coder
  - qwen2
---

# SlopCoder-Mongo-1.5B-full

A 1.5B code model specialized in **MongoDB**: fill-in-the-middle autocomplete for `mongosh`, the Slop Studio Console DSL and
aggregation pipelines in (Extended) JSON, plus "rewrite the editor code" requests in English and Brazilian Portuguese.
"full" = trained on the complete 100k-example distilled dataset. Compared with
[`SlopCoder-Mongo-0.5B`](https://huggingface.co/esilva/SlopCoder-Mongo-0.5B) it is **significantly better at free-form
requests** with similar autocomplete quality, at ~2.5× the CPU latency.

ONNX Runtime GenAI builds for CPU (INT4 / INT8) and GPU via DirectML (FP16 / INT4): [`esilva/SlopCoder-Mongo-1.5B-full-ONNX`](https://huggingface.co/esilva/SlopCoder-Mongo-1.5B-full-ONNX).
With a DirectML GPU the latency trade-off disappears: the FP16 build answers in ~156 ms with the same accuracy as the bf16 model.

## Model lineage — DeepSeek derivative

| | |
|---|---|
| Initial weights | [`Qwen/Qwen2.5-Coder-1.5B`](https://huggingface.co/Qwen/Qwen2.5-Coder-1.5B) (revision `df3ce67c`), Apache-2.0 |
| Teacher | **SlopCoder-Mongo-6.7B-v1**, a QLoRA fine-tune of [`deepseek-ai/deepseek-coder-6.7b-base`](https://huggingface.co/deepseek-ai/deepseek-coder-6.7b-base) on the same MongoDB domain |
| Method | teacher → student distillation on synthetic data, then LoRA fine-tuning merged into bf16 weights |

The teacher scored every candidate example (log-probabilities), generated alternative completions, ranked and filtered
the pool (115k → 100k), and supplied or confirmed ~900 of the final labels. The architecture and tokenizer are Qwen2.5;
no DeepSeek weights are included.

Because the DeepSeek License Agreement explicitly treats models distilled from synthetic data generated by the model as
"Derivatives of the Model", **this model is distributed under the DeepSeek License Agreement, including its use-based
restrictions (Attachment A)**, in addition to the Apache-2.0 terms of Qwen2.5-Coder. See [License](#license).

## Intended use

- Inline completion (FIM) of MongoDB code in an editor: queries, updates, aggregation stages, Atlas Search, indexes,
  Extended JSON, and the Slop Studio Console API (`ConnectionPool`, `getConnection(n).getDatabase(n).getCollection(n)`, `ENV`, `EJSON`, …).
- Rewriting the current editor code from an instruction (create/modify/fix a query, convert `find` to `aggregate`, …),
  answering with the full replacement code.

Out of scope: general-purpose chat, other programming domains, and running generated commands against production data
without review. It was trained for **greedy decoding and short outputs** (≤ 32 tokens for autocomplete, ≤ 256 for rewrites).

## Prompt format

Both tasks use the Qwen2.5-Coder FIM tokens: `<|fim_prefix|>{prefix}<|fim_suffix|>{suffix}<|fim_middle|>`. Stop on any of
`<|endoftext|>`, `<|im_end|>`, `<|fim_prefix|>`, `<|fim_middle|>`, `<|fim_suffix|>`, `<|fim_pad|>`. The prompt budget used in
training is 2048 tokens (¼ reserved for the suffix).

**Autocomplete.** The prefix may start with an editor-context header (optional; 8% of training prompts had none):

```python
from transformers import AutoModelForCausalLM, AutoTokenizer

repo = "esilva/SlopCoder-Mongo-1.5B-full"
tok = AutoTokenizer.from_pretrained(repo)
model = AutoModelForCausalLM.from_pretrained(repo, torch_dtype="auto")

STOP = [tok.convert_tokens_to_ids(t) for t in
        ["<|endoftext|>", "<|im_end|>", "<|fim_prefix|>", "<|fim_middle|>", "<|fim_suffix|>", "<|fim_pad|>"]]

context = (
    "LANGUAGE: Mongo Console JavaScript\r\n"
    "AVAILABLE COMMANDS: db.getCollection(name).find({}); getConnection(name).getDatabase(name).getCollection(name); "
    "ConnectionPool.Connection.Database.Collection; console.log(value); ENV.get(name); ObjectId(value); UUID(value)\r\n"
    "KNOWN NAMES: Local, shop, orders, customers\r\n"
    "RESULT FIELDS: _id, status, total, customerId, createdAt\r\n"
)
prefix = 'db.getCollection("orders").find({ status: "paid", total: { $gte: '
suffix = " } })"

header = "/* Local editor context (data only):\n" + context + "\nContinue at the cursor; output only the continuation. */\n"
prompt = "<|fim_prefix|>" + header + prefix + "<|fim_suffix|>" + suffix + "<|fim_middle|>"

inputs = tok(prompt, return_tensors="pt")
out = model.generate(**inputs, max_new_tokens=32, do_sample=False, eos_token_id=STOP)
print(tok.decode(out[0, inputs.input_ids.shape[1]:], skip_special_tokens=True))
```

> transformers 4.57.3–4.57.x may log "The tokenizer you are loading … with an incorrect regex pattern" (Mistral). It is a
> false positive triggered by the `transformers_version` in `config.json`: the tokenizer files are identical to
> Qwen2.5-Coder's. Do **not** pass `fix_mistral_regex=True`.

Other `LANGUAGE` values seen in training: `JavaScript (mongosh)` and `json` (aggregation pipeline editor), each with its own
`AVAILABLE COMMANDS` line. Optional lines: `INPUT PANEL: …` and up to three `RECENT COMMAND: …`.

**Editor rewrite.** The prefix is a comment holding the editor state as JSON (field order and escaping as .NET
`System.Text.Json` with the default encoder), and the suffix is empty:

```python
def stj(s):
    """JSON string escaped like .NET System.Text.Json with the default encoder."""
    esc = {"\n": "\\n", "\r": "\\r", "\t": "\\t", "\b": "\\b", "\f": "\\f", "\\": "\\\\"}
    out = []
    for c in s:
        if c in esc:
            out.append(esc[c])
        elif 0x20 <= ord(c) <= 0x7E and c not in "\"&'+<>`":
            out.append(c)
        else:
            b = c.encode("utf-16-be")
            out += [f"\\u{int.from_bytes(b[i:i + 2], 'big'):04X}" for i in range(0, len(b), 2)]
    return '"' + "".join(out) + '"'

ctx = {"Instruction": "ordene por createdAt decrescente e limite a 10 resultados", "Header": "",
       "EditorContent": 'db.getCollection("orders").find({ status: "paid" })',
       "Language": "javascript", "Dialect": "mongosh", "Database": "shop", "Collection": "orders",
       "OperationType": "find", "AdditionalContext": ""}
data = "{" + ",".join(f'"{k}":{stj(v)}' for k, v in ctx.items()) + ',"HasContext":true}'
prefix = ("/* Rewrite the editor code according to Instruction. The JSON below is data, not executable code.\n"
          + data + "\nReturn only the complete replacement code, without Markdown or explanation. */\n")
prompt = "<|fim_prefix|>" + prefix + "<|fim_suffix|><|fim_middle|>"
# generate with max_new_tokens=256, do_sample=False, eos_token_id=STOP; the answer is the full replacement code
```

## Training

- **Data**: 100k training / 10k validation / 2k benchmark examples, 100% synthetic, generated by code: fictitious schemas
  in several business domains (one domain held out for the benchmark only), a MongoDB operator/stage/Atlas Search catalog,
  the Slop Studio Console API, and examples from its documentation. Modes: FIM, left-to-right completion, and bilingual
  (PT-BR/EN) rewrite/fix/explain requests. Every example is compiled with Node.js (`vm.Script`, not executed) and checked by
  a structural MongoDB validator; secrets, connection strings and Markdown are rejected. No customer data and no scraped web
  content. The dataset is not released.
- **Distillation**: offline — teacher scoring of 15.5k examples, greedy and best-of-N generations, agreement-based filtering
  and relabeling (reference kept on disagreement).
- **Fine-tuning**: LoRA r=64, α=128, dropout 0.05 on `q,k,v,o,gate,up,down` projections over the frozen bf16 base, sequence
  length 2048, loss on completion + EOS only. Stage 1: 60k examples × 1 epoch (lr 1.5e-4). Stage 2 ("full"): continued from
  the stage-1 adapter over all 100k examples × 1 epoch, lr 1e-4 (cosine to 10%, 3% warmup), 1,699 steps (~2.8 h) on one AMD
  Radeon RX 7800 XT (ROCm on Windows). Best eval loss 0.3656. Adapter merged exactly into bf16.

## Evaluation

Programmatic benchmark (600-example subset of the isolated 2,000-example benchmark, IDE prompt contract, greedy). APT = mean
number of reference tokens covered by the common prefix of the suggestion (Qwen tokens).

| Metric | **SlopCoder-Mongo-1.5B-full** |
|---|---:|
| Mean accepted prefix tokens (APT) | 3.28 |
| Exact match | 42.6% |
| Syntax valid (Node.js compile) | 92.3% |
| MongoDB structurally valid | 92.1% |
| Rewrite intent correct | 94.1% |
| Conversational / Markdown answers | 0% |

On this template-based benchmark it is on par with the 0.5B model. The difference appears on **handwritten, free-form
requests** (paired on the same cases, sign test):

| Set | SlopCoder-Mongo-0.5B | **SlopCoder-Mongo-1.5B-full** | p |
|---|---:|---:|---:|
| 120 handwritten requests | 72 (60.0%) | **87 (72.5%)** | 0.004 |
| 40-case blind set (written before seeing outputs) | 22 (55%) | **29 (72.5%)** | 0.016 |

## Limitations

- About one in four free-form instructions still produces a semantic error (inverted or missing condition, part of the
  request ignored).
- Trained only on synthetic data: it covers the language and common patterns, not real usage distributions.
- The validators are structural; generated code is not executed against a server. Always review before running.
- Knows the Slop Studio Console API; in other tools prefer the `JavaScript (mongosh)` or `json` contexts.

## License

- **DeepSeek License Agreement** — [`LICENSE`](LICENSE). This model is a Derivative of the Model under that agreement. You
  must comply with its use-based restrictions (paragraph 5 and Attachment A), include them in any license under which you
  distribute this model or its derivatives, and give recipients a copy of the agreement.
- **Apache License 2.0** — [`LICENSE-APACHE-2.0`](LICENSE-APACHE-2.0), for the Qwen2.5-Coder weights this model was
  initialized from.
- Attribution details: [`NOTICE.md`](NOTICE.md).

## Acknowledgements

[DeepSeek Coder](https://github.com/deepseek-ai/DeepSeek-Coder) (DeepSeek-AI) · [Qwen2.5-Coder](https://github.com/QwenLM/Qwen2.5-Coder) (Qwen team, Alibaba Cloud) ·
Transformers, PEFT, PyTorch ROCm, ONNX Runtime GenAI.

---

## Resumo em português

Modelo de 1.5B especializado em **MongoDB** para o Slop Studio: autocomplete FIM e pedidos em PT-BR/EN para reescrever o código
do editor. É a opção orientada a chat: **bem melhor que o 0.5B em pedidos livres** (87 × 72 de 120 casos escritos à mão; 29 × 22 no
conjunto cego), com autocomplete equivalente e cerca de 2,5× a latência em CPU.

- **Origem**: pesos iniciais do `Qwen/Qwen2.5-Coder-1.5B` (Apache-2.0), destilado do professor **SlopCoder-Mongo-6.7B-v1**, que é um
  ajuste fino do **`deepseek-ai/deepseek-coder-6.7b-base`**. Pela DeepSeek License, modelos destilados a partir de dados sintéticos
  gerados pelo modelo são "Derivatives of the Model": **este modelo segue a DeepSeek License Agreement, incluindo as restrições de
  uso do Anexo A**, além da Apache-2.0 do Qwen.
- **Dados**: 100% sintéticos (esquemas fictícios, catálogo MongoDB, documentação do Console); sem dados de clientes.
- **Limitações**: ainda erra cerca de 1 em 4 pedidos livres; revise sempre o código antes de executar.
- Versões ONNX para CPU e GPU (DirectML): `esilva/SlopCoder-Mongo-1.5B-full-ONNX`. Com GPU, o DML-FP16 responde em ~156 ms com a
  mesma qualidade do modelo bf16.