lilchat

lilbase (297m params, trained from scratch) finetuned into a chat model on smol-smoltalk.

it follows the chat format, answers in full sentences, stops when its done, and does lists and code blocks. it does not know things. expect confident wrong answers on facts and maths, and weak memory across turns.

run it

ollama run navthings/lilchat

or try it in your browser: https://navthings.github.io/playground/

files

file	size	notes
model.safetensors + config + tokenizer	1.19gb	transformers LlamaForCausalLM, chat template included
lilchat-q8_0.gguf	379mb
lilchat-q4_k_m.gguf	274mb	smallest

prompt format

llama-2 style, with </s> (id 2) as the start token and after every assistant reply:

</s>[INST] hi [/INST] Hello! How can I help you today?</s>[INST] next message [/INST]

system prompts go in <<SYS>>\n...\n<</SYS>>\n\n before the first [INST].

finetune

full sft in mlx on a macbook air, 1,136 steps of 32k tokens (~37m tokens, about 12% of smol-smoltalk), loss only on assistant turns. held-out loss 1.337 vs 2.230 for lilbase on the same data.

temperature 0.4 works best. greedy decoding loops, keep some temperature.

code

https://github.com/navthings/sprout (sft/)