Swap the model behind moth

moth's pet, hooks, chat page and moth tail all talk to one local server (moth.serve). The model behind that server can be changed with one setting: a later moth, another creator's small persona model, or any small local chat model. Everything else stays the same: the hooks, the reaction batching across agent sessions, the home folder, the chat page and the pet.

moth config model ~/models/moth-2                          # a weights folder (its moth.json says how to run it)
moth config model someone/persona-100m                     # an HF repo id
moth config model "openai:http://127.0.0.1:11434/v1#qwen3:0.6b"   # a local llama.cpp / Ollama / vLLM server
moth config model default                                  # back to moth's own model

The setting applies from moth's next start. --model <same forms> overrides it for one run (python -m moth, python -m moth.serve, python -m moth.cli, python -m moth.pet --model ...), and moth setup --model ... saves it as part of setup. moth doctor shows which backend, model and character are active.

Backends

backend	runs	needs	tools	phrase ban	context kept across turns
`tiny-agent` (default)	moth-112m and anything trained with this repo's pipeline (`tiny_agent/`)	nothing extra	yes	yes	KV cache: each turn feeds only the new text; int8 on the CPU, device graphs on CUDA/XPU
`transformers`	any causal LM `transformers` loads (a folder or an HF repo id) with a chat template, CPU or GPU	`pip install "moth-companion[transformers]"`	if the manifest says so	yes	no: each turn re-reads the chat
`openai`	any OpenAI-compatible `/v1/chat/completions` server: llama.cpp `llama-server`, Ollama, vLLM, LM Studio	a running server	if the manifest says so	no	no (the server may cache prefixes)

How moth picks the backend for a model spec:

A folder. If it has a moth.json, that file decides. Without one, it's a tiny-agent folder, which is how every existing moth folder keeps working unchanged. The exception is a plain transformers save, a config.json with a model_type, which runs on transformers with the general-model defaults below.
An HF repo id (owner/name). moth fetches only the repo's moth.json, a few hundred bytes, and uses it.
- tiny-agent repos download their weights into ~/.cache/moth/<owner>/<name>/ on first start, the same way moth-112m does.
- Repos without a moth.json load through transformers, which uses its own Hugging Face cache.
- A moth.json from the Hub can't name an openai endpoint. Where moth's text goes is your call, never a downloaded file's: use an openai: spec or a moth.json of your own for that.
openai:<base_url>#<model>. An OpenAI-compatible server. base_url can be http://host:port or http://host:port/v1.

moth.json

Put it next to the weights. Every key is optional, and a missing key takes its default. Unknown keys are ignored (a newer manifest still loads on an older moth). A key with the wrong type stops moth with a message that names it.

key	default	what it does
`backend`	`"tiny-agent"`	`tiny-agent`, `transformers` or `openai`
`character`	`"moth"`	the name shown where it is display text (the pet's tray tooltip and notification title, `moth.cli`'s prompt, doctor). Commands and the package stay `moth`
`chat_format`	`"chatml"` (tiny-agent), `"template"` (others)	`chatml`: moth's own `<\|im_start\|>` format. `template`: the tokenizer's chat template (transformers) or the server's own (openai)
`system`	moth-112m's trained prompt (tiny-agent), a short moth character sheet (others)	the system prompt
`agent_card`	`"[the agent said]\n{message}"`	how a coding agent's final message is shown to the model. `{message}` is the cleaned text (markdown stripped, long messages cut to head + tail). Must contain `{message}`
`reaction`	none (tiny-agent), a "react in one short silly line…" nudge (others)	added after the agent card for that one reaction only. History keeps the plain card
`sampling`	`{"temperature": 0.8, "top_p": 0.95, "rep_penalty": 1.3, "max_tokens": 200}`	defaults for `--temperature`, `--top-p`, `--rep-penalty` and `--max-turn-tokens`. Flags still win
`tools`	`true` (tiny-agent), `false` (others)	moth's home tools: `bash` / `read` / `write` in its own little folder, as `<tool_call>` JSON. Off means every turn only talks
`lowercase`	`true`	moth only speaks lowercase
`context`	3000	tokens of chat kept in mind. Older turns drop out first. `--ctx-budget` wins
`stop`	`[]`	extra stop strings
`model`	the folder itself	transformers: a folder or an HF repo id. openai: the server's model name
`base_url`	none	openai: the server, e.g. `http://127.0.0.1:8080/v1`
`api_key_env`	`MOTH_OPENAI_API_KEY`	openai: the environment variable holding a key, if the server wants one. It must start with `MOTH_`, so a manifest can't send another key (`GITHUB_TOKEN`...) anywhere. Keys never go in moth.json or moth's config
`tokenizer`	`tokenizer.json` in the folder	tiny-agent: the tokenizer file, relative to the folder
`repo`	none (moth-112m: its own repo)	the HF repo `moth update` refreshes this model from
`version`	none	the model's own version, shown by doctor and the serve log
`needs_moth`	none	the oldest moth that can run this model, e.g. `"0.4.0"`: an older moth says how to upgrade instead of loading it, and a pip install's `moth update` won't fetch it

moth-112m (today's model, no file needed)

A folder without moth.json is exactly this. Spelled out:

{
  "backend": "tiny-agent",
  "character": "moth",
  "chat_format": "chatml",
  "agent_card": "[the agent said]\n{message}",
  "sampling": {"temperature": 0.8, "top_p": 0.95, "rep_penalty": 1.3, "max_tokens": 200},
  "tools": true,
  "repo": "darioooooo0o/moth-112m"
}

A hypothetical moth 2 (same pipeline, new weights)

{
  "backend": "tiny-agent",
  "character": "moth",
  "system": "you are moth. you live on a graphics card. ...the prompt moth 2 was trained with...",
  "sampling": {"temperature": 0.7},
  "repo": "darioooooo0o/moth-2",
  "version": "2.0"
}

Upload it with model.safetensors, config.json and tokenizer.json. If it needs code an older moth doesn't have (a new architecture), set "needs_moth" to the first moth version that can run it. A config.json with settings an older moth doesn't know is caught too: that moth says how to upgrade rather than failing to load. Users then run moth config model darioooooo0o/moth-2. The weights download on first start, and moth update keeps them current from that repo.

A general small chat model through transformers

Any instruct model with a chat template works. It has never been trained as moth, so the manifest gives it the character: either the default moth sheet (leave system out) or your own.

{
  "backend": "transformers",
  "model": "HuggingFaceTB/SmolLM2-360M-Instruct",
  "character": "moth",
  "sampling": {"temperature": 0.7, "max_tokens": 60},
  "context": 2048
}

Without a moth.json, moth config model HuggingFaceTB/SmolLM2-360M-Instruct does the same with these defaults: chat template, no tools, the moth character sheet, and the reaction nudge. A 0.3-0.5B model answers in a second or two on a CPU. To give it the home tools, set "tools": true and describe the <tool_call> format in system. Only models that can follow that format should get them.

llama.cpp / Ollama (an OpenAI-compatible server)

llama-server -m persona-100m-q8_0.gguf --port 8080           # or: ollama serve  (and ollama pull qwen3:0.6b)
moth config model "openai:http://127.0.0.1:8080/v1#persona"

Or as a folder that holds only this moth.json, to set the character and prompt too:

{
  "backend": "openai",
  "base_url": "http://127.0.0.1:11434/v1",
  "model": "qwen3:0.6b",
  "character": "moth",
  "sampling": {"temperature": 0.8, "max_tokens": 80}
}

The repetition penalty goes to the server as repeat_penalty (llama.cpp) and repetition_penalty (vLLM). If a server rejects unknown fields, moth sends the request again without them. Ollama's OpenAI endpoint ignores them. <think> blocks in a reply are dropped. A server that's down gives moth's "my head went all fuzzy" line, and the chat goes on.

Another creator's persona model

If it ships as transformers weights with a chat template, write a moth.json with "backend": "transformers", its name as character and the system prompt it was trained with as system.
If it ships as a GGUF, serve it with llama-server and use the openai backend.
If it trained on its own prompt format, set agent_card to whatever it expects for "someone else said this", and reaction to null if it already reacts in character.
A custom architecture that neither can load needs a small new backend in moth/backends/: load, count, generate, as in moth/backends/base.py.

Privacy

moth's promise is that nothing leaves your machine, and the tiny-agent and transformers backends keep it: the model runs inside the moth server. The openai backend sends every agent message moth reacts to, and everything you type to moth, to the server you name.

On localhost, 127.x or ::1 that server is on your machine, and moth never goes through a proxy to reach it.
On any other host, including a computer on your LAN, the text leaves your machine. moth warns at startup, moth config model says so when you set it, and moth doctor shows a privacy line on every run. Doctor never contacts a non-local endpoint itself.
API keys come only from an environment variable (MOTH_OPENAI_API_KEY, or the one api_key_env names). They are never written to moth's files.

Updating

moth update refreshes only models it downloaded from the Hub. That means moth-112m, and tiny-agent models from another repo, where it uses the manifest's repo (model files only, with the same backup and --rollback). For anything else it explains what to do instead:

transformers models: hf download <repo>
an openai server: update the model where it runs, e.g. ollama pull

For those it says so and then updates moth's own install as usual (its code, and moth's own model), so code updates and the pet's daily "new moth out" check keep working.

What changes for the pet and the page

With a non-default model the server's token-free GET /api/health also says "character" and "backend". The pet shows the character name in its tray tooltip and notifications, and older servers keep showing "moth". The phrase memory (no same joke twice), the reaction throttle, the batching across agent sessions and the one-turn-per-session history are moth's own engine and work the same on every backend. Only the phrase ban needs token-level sampling, so it is off on the openai backend.