---
license: apache-2.0
base_model: HuggingFaceTB/SmolLM2-135M
library_name: transformers
pipeline_tag: text-generation
tags: [shell, terminal, autocomplete, command-line, gguf]
language: [en]
---

# terllm

Small local models that complete the shell command you are typing (fish / zsh-autosuggestions style ghost text), fine-tuned from
[SmolLM2-135M](https://huggingface.co/HuggingFaceTB/SmolLM2-135M). Runs locally through llama.cpp (~25 ms per keystroke on CPU, Q4_K_M).
Code, data generator, zsh plugin and daemon: <https://github.com/partum55/terllm>.

## Prompt format (v3; v2 is the same without `<|fs|>`)

Plain text with added special tokens; the model continues the typed prefix and ends with `<|end|>`:

```
<|cwd|>~/proj<|ls|>README.md src<|git|>main -> origin/main | ?? x.py<|env|>VIRTUAL_ENV=.venv
<|hist|>$ git clone ...
$ cd proj<|cand|>git checkout main<|fs|>src/ scripts/<|cmd|>cd s
```

- `<|hist|>` last commands (with exit codes), `<|err|>`/`<|out|>` previous output (optional), `<|nl|>` natural-language request (optional)
- `<|cand|>` older history commands that extend the typed prefix (what zsh-autosuggestions would offer)
- `<|fs|>` (new in v3) up to 8 tab-completion candidates for the word being typed: files, directories, PATH commands. The model decides whether to
  copy one, write a new name (`mkdir`, branch names) or stop (command already complete)

The reference implementation of the prompt is `datagen/render.py` in the repository; use the daemon (`runtime/terllmd.py`) rather than building prompts by hand.
Show only the confident prefix of the completion (all tokens with probability >= 0.8); the policy is in `runtime/policy.py`.

## Models in this repo

| folder | what | files |
|---|---|---|
| `v3/` | current: adds the `<|fs|>` file/dir/command candidates | `terllm-v3-q4_k_m.gguf` (llama.cpp, 105 MB), `model.safetensors` + tokenizer (bf16) |
| `v2/` | previous: history candidates (`<|cand|>`), no `<|fs|>` — run the daemon with `--no-fs` | `terllm-v2-q4_k_m.gguf`, `model.safetensors` + tokenizer |

```python
from transformers import AutoModelForCausalLM, AutoTokenizer
tok = AutoTokenizer.from_pretrained("partum/terllm", subfolder="v3")
model = AutoModelForCausalLM.from_pretrained("partum/terllm", subfolder="v3")
```

## Evaluation (honest numbers)

| | v2 | v3 |
|---|---|---|
| chars saved on a real zsh history (150 commands, cwd unknown, threshold 0.8) | 15.8% | 18.4% (with `<|fs|>`) |
| exact match, synthetic path probe (fresh names) | 33% | 73.5% |
| exact match, synthetic val (no `<|fs|>` blocks) | 60.8% | 60-61% |

For reference, history-prefix matching (zsh-autosuggestions) saves 0.4% on the same commands. The real-history sample is small (about +-1-2 points) and context-poor;
it has not been validated on live usage yet. On command/tree shapes outside the training scenarios, path suggestions are noticeably weaker, and the model sometimes
ignores a single obvious candidate. Treat it as a suggestion engine: it never executes anything.

## Training data

Synthetic terminal sessions executed in a shell/git simulator (so exit codes, stderr and state are consistent), seed scenarios plus LLM-proposed sessions
(Qwen2.5-Coder, every command re-executed by the simulator), and text-to-command pairs derived from
[tldr-pages](https://github.com/tldr-pages/tldr) (CC BY 4.0, "tldr-pages team and contributors") and
[NL2Bash](https://github.com/TellinaTool/nl2bash) (MIT). Details and licenses: `data/SOURCES.md` in the repository. No personal shell history was used for training.