pico-type 🔍

A tiny byte-level multi-head content classifier — ~1.5M params, ~200KB ONNX, <12ms inference.

Classifies any content into 7 categories from raw bytes: coarse type, modality, subtype, code language, text language, file MIME, and risk flags.

---

✨ Features

• No tokenizer — operates directly on raw UTF-8 bytes (supports all languages)
• 7 heads, one forward pass — coarse type, modality, subtype, code lang, text lang, file MIME, risk
• 4 Matryoshka tiers — tiny (16d) → small (64d) → base (192d) → pro (576d)
• ~200KB ONNX — deploy on edge devices, serverless, browser (WebAssembly)
• <12ms inference on CPU via ONNX Runtime
• CLI, Gradio Space, MCP server — ready to use

📊 Performance

Head	Classes	Accuracy
coarse	12	100%
modality	8	100%
subtype	24	93.8%
code_lang	62	41.7%
text_lang	30	94.3%
file_mime	90	100%
risk (mAP)	6	100%

500 evaluation samples, 1700 training steps, base tier, ~13ms inference.

v0.2.0 is now available! eulogik/pico-type-v02 — trained on real code and text data with improved accuracy (code_lang 60.3%, text_lang 98.3%).

🚀 Quick Start

CLI

\Defaulting to user installation because normal site-packages is not writeable

Python

\

MCP Server (Claude/Cursor)

\

🏗 Architecture

\

• ByteEmbed: lookup-free byte embedding (256 vocab, 96 dim)
• Conv1D: 3 parallel kernels (width 3, 5, 7) with residual + layer norm
• BiAttention: bidirectional self-attention with RoPE, 4 heads, 96 dim
• Pool: mean + max + std concatenation
• Matryoshka Heads: 4 slices of the pooled vector (16/64/192/576 dim) → 7 linear classifiers

Total parameters: 1.43M (tiny) / 1.45M (small) / 1.48M (base) / 1.56M (pro)

🔧 Model Tiers

Tier	Dim	Params	ONNX Size
tiny	16	1.43M	203 KB
small	64	1.45M	203 KB
base	192	1.48M	206 KB
pro	576	1.56M	202 KB

All tiers share the same trunk; only the final linear layers differ.

🧪 Classification Heads

Head	Classes	Examples
coarse	12	text, code, link, image, file, config, markup, data, error, secret, archive, binary
modality	8	textual, binary_image, binary_archive, binary_executable, etc.
subtype	24	json, yaml, toml, csv, html, markdown, sql, log, dockerfile, etc.
code_lang	62	python, javascript, typescript, java, c, cpp, go, rust, etc.
text_lang	30	en, es, fr, de, it, pt, ru, zh, ja, ko, ar, hi, etc.
file_mime	90	text/html, application/json, application/pdf, image/png, video/mp4, etc.
risk	6	api_key, jwt, password, email, phone, ssh_key

🌐 Deployment

Platform	Location
HuggingFace Space	eulogik/pico-type
HuggingFace Model (v0.1)	eulogik/pico-type
HuggingFace Model (v0.2)	eulogik/pico-type-v02
GitHub	eulogik/pico-type
PyPI	\Defaulting to user installation because normal site-packages is not writeable

📄 License

Apache 2.0