ThoxMicro-1bit-16M

From-Scratch THOX Ternary Model — a THOX model trained from scratch: a BitNet b1.58 ternary decoder (16M parameters, 86.6% of weights ternary) trained on TinyStories

GGUF: Q8_0, TQ2_0 · Ollama Modelfile

Your AI. Your Data. Your Rules.™

Status: PUBLISHED — RESEARCH MODEL. A THOX model trained from scratch: a BitNet b1.58 ternary decoder (16M parameters) trained on TinyStories. See Files for exactly what this repo contains.

Summary

ThoxMicro-1bit-16M is a THOX model trained from scratch: a BitNet b1.58 ternary decoder (16M parameters, 86.6% of weights ternary) trained on TinyStories. It is the deeper sibling of ThoxMicro-1bit-9M, an edge research model, and the base for the THOX 16M device-role models. Neither the 9M nor the 16M supersedes the other.

Highlights

ThoxMicro-1bit-16M is a THOX model trained from scratch: a BitNet b1.58 ternary decoder (16M parameters, 86.6% of weights ternary) trained on TinyStories.
bash hf download Thox-ai/ThoxMicro-1bit-16M --include '*.gguf' --local-dir ./ThoxMicro-1bit-16M llama-cli -m ./ThoxMicro-1bit-16M/thoxmicro-1bit-16m.TQ2_0.gguf -p "Hello" Ternary files need a recent llama.cpp build; Ollama and LM Studio load only the Q8_0 file.

Specifications


Base model	See card body
Parameters	See card body
Formats	GGUF: Q8_0, TQ2_0 · Ollama Modelfile
License	Apache-2.0
This repository contains	GGUF build(s) in this repo: thoxmicro-1bit-16m.Q8_0.gguf, thoxmicro-1bit-16m.TQ2_0.gguf

Files

File	Size	SHA-256
`thoxmicro-1bit-16m.Q8_0.gguf`	17,033,504 B	`e0c8d621ea3c9caa405add44494823ccf033aadb47c8df09762696accfabd192`
`thoxmicro-1bit-16m.TQ2_0.gguf`	8,030,496 B	`13642c72e84db9c337ac22bc72b7a60e8e97df5738d58a3ca56104385d8def98`
`.gitattributes` / `Modelfile`	—	Supporting files (see repo)

Architecture

Field	Value
Architecture	BitNet b1.58 ternary decoder
Layers	16
Hidden size	256
Attention heads	8
KV heads	8
FFN / intermediate	768
Vocab	8,192 (own byte-level BPE)
Max context	512
Tied embeddings	yes

Run it

hf download Thox-ai/ThoxMicro-1bit-16M --include '*.gguf' --local-dir ./ThoxMicro-1bit-16M
llama-cli -m ./ThoxMicro-1bit-16M/thoxmicro-1bit-16m.TQ2_0.gguf -p "Hello"

Ternary files need a recent llama.cpp build; Ollama and LM Studio load only the Q8_0 file.

Platforms

Platform / device	Run path
Hugging Face	Download the builds listed under Files from this repo (see Run it)

Intended use

On-device and edge text generation within the THOX stack, and research on ternary-weight models.
Not a safety-aligned public assistant unless deployed behind THOX guardrails.
Out of scope: medical, legal, financial, safety-critical, or other high-stakes decisions without human review.

Training data and procedure

Trained by THOX from scratch as part of the THOX small-model research line. The corpus and procedure are as described in this card at release; no external base model was used.

Evaluation

No THOX GATE49 result is published for this model.

Limitations

A 16M research model: story completion only, no instruction following or factual reliability.
Behaviour and bias reflect the training data; quality outside short English story text is low.
Quantization trades accuracy for size.

License

Apache License 2.0. This model was trained by THOX from scratch and is released under the Apache License 2.0. Training data note: the training corpus is TinyStories, distributed under CDLA-Sharing-1.0; that license governs the dataset itself.

THOX.ai — Your AI. Your Data. Your Rules.™

Provenance

Published by THOX.ai as Thox-ai/ThoxMicro-1bit-16M. THOX from-scratch research line; no upstream base model. This card was conformed to THOX Model Card Standard v1.0 on 2026-10-31 (frontmatter, tags, header, files, and section structure); the technical content is as published at release.

Citation

@misc{thoxai2026thoxmicro1bit16m,
      title={ThoxMicro-1bit-16M},
      author={THOX.ai},
      year={2026},
      url={https://huggingface.co/Thox-ai/ThoxMicro-1bit-16M}
}