Tabby-Tavern Development Log
Engineering history for the containerized local AI lab.
Hardware baseline
- GPU: NVIDIA GeForce RTX 4070
- Environment: Linux + Docker Compose with NVIDIA GPU passthrough
- Primary inference path: TabbyAPI + EXL3 / ExLlamaV3
- Secondary path: Ollama (GGUF)
Week 1 — Core integration
- Consolidated TabbyAPI, SillyTavern, Open WebUI, Ollama, SearXNG (+ Redis) into one compose file
- Adopted EXL3 weights for faster VRAM load vs earlier experiments
- Fixed cross-container TabbyAPI auth/whitelist failures that blocked peer services
- Added persistent volume mounts for configs and data directories
Week 2 — Model bring-up & GPU tuning
- Loaded Llama-3.1-8B-Instruct EXL3 (6.0 bpw class) into
tabby_models/ - Tuned container env: flash attention / KV cache flags,
shm_size: 16g, CUDA device ordering - Documented start/stop operational loop (
docker compose down && up -d)
Week 3 — Public packaging
- Created sanitized publish tree (no weights, no user DBs, no tokens)
- Added SECURITY.md and example TabbyAPI config
- Mirrored narrative to GitHub portfolio rebuild (Aug 2026)
Open follow-ups
- One-command bootstrap that builds the TabbyAPI image + prints next model download
- Healthcheck targets in compose
- Optional Traefik/Caddy reverse-proxy profile for LAN HTTPS