Local-first LLMs
Small, capable models I run or track for a single RTX 3090 (Ollama / llama.cpp / transformers) — the backbone of self-hosted, sovereign AI.
27B • Updated • 6.83M • 6.13kNote Daily driver on the 3090 (qwen3.8:27b, Q4_K_M + MTP, 17.7 GB, vision+tools+thinking). Beat Qwen3.6-27B in a local A/B on 2026-10-13: same accuracy, ~2x faster.
-
empero-ai/Qwen3.8-35B-A3B-Distill-GGUF
Text Generation • 36B • Updated • 1.34M • 284 -
Kwaipilot/KAT-Coder-V2.5-Dev
Text Generation • 35B • Updated • 54k • 688 -
Qwen/Qwen3-Coder-30B-A3B-Instruct
Text Generation • 31B • Updated • 1.92M • 1.34k -
unsloth/Qwen3.5-9B-MTP-GGUF
Image-Text-to-Text • 9B • Updated • 110k • 232
Qwen/Qwen3.6-27B
Image-Text-to-Text • 28B • Updated • 2.46M • • 2.36kNote Previous daily driver, replaced by Qwen3.8-27B on 2026-10-13. Kept as the base of the abliterated 27B I run for CTF work.
-
Qwen/Qwen3.5-9B
Image-Text-to-Text • 10B • Updated • 6.22M • • 2.39k -
google/gemma-4-12B-it
Any-to-Any • 12B • Updated • 1.62M • 1.75k
unsloth/gemma-4-26B-A4B-it-qat-GGUF
Image-Text-to-Text • 25B • Updated • 214k • 513Note Official QAT GGUF of Gemma 4 26B MoE (4B active) — a 3090-friendly middleweight agent
ornith-ai/Ornith-1.5-9B
Text Generation • 10B • Updated • 575k • 420Note Tracked, no longer installed. Ran on the 3090 as ornith-1.5:9b (Q4_K_M, 6.6 GB, ctx 262k, tools+thinking+vision) from 2026-08, removed 2026-11-14 for lack of use. Replaced Ornith-1.0-9B.
-
XiaomiMiMo/MiMo-V2.6-Distill-Qwen-9B
Image-Text-to-Text • 9B • Updated • 27.3k • 710
Qwen/Qwen3-VL-8B-Instruct
Image-Text-to-Text • 9B • Updated • 6.73M • • 1.26kNote Local vision on the RTX 3090 (runs as qwen3-vl:8b via ollama) — screenshots, docs, UI grounding
tencent/Hy-MT2-7B
Translation • 8B • Updated • 34.6k • 217Note Translation specialist (33 languages incl. FR/PT/EN). Runs on the 3090 as hf.co/tencent/Hy-MT2-7B-GGUF:Q8_0 since 2026-10-08.
-
openbmb/MiniCPM5-2B
Text Generation • 3B • Updated • 987k • 1.86k -
Qwen/Qwen3-Embedding-0.6B
Feature Extraction • 0.6B • Updated • 8.03M • • 1.35k
google/embeddinggemma-2
Feature Extraction • 0.7B • Updated • 591k • 3.53kNote Google's multimodal embedder (text/image/audio/video, 270M-740M). Watching: Ollama support lands with llama.cpp in 0.40.4.
-
zeroentropy/zerank-2-reranker
Text Ranking • 4B • Updated • 54.6k • 149 -
LiquidAI/LFM2.5-24B-A2B
Text Generation • 24B • Updated • 30.6k • • 620 -
Nanbeige/Nanbeige4.3-33B
Text Generation • 33B • Updated • 23.8k • 1.09k -
unsloth/Qwen3.6-35B-A3B-GGUF
Image-Text-to-Text • 35B • Updated • 1.75M • 1.81k -
empero-ai/Qwythos-9B-Claude-Mythos-5-1M
Text Generation • 9B • Updated • 3.54k • 1.02k -
ornith-ai/Ornith-1.0-9B
Text Generation • 1.47M • Updated • 795k • 571 -
empero-ai/Qwythos-9B-v2-GGUF
Image-Text-to-Text • 9B • Updated • 222k • 336