- Hard constraint #3: quantized AWQ model fits 16GB T4 with headroom
- Reduced max-model-len to 4096 for smaller KV cache on T4
- Removed prometheus patch (unnecessary with vllm 0.6.3)
- PRIMARY_MODEL=Qwen/Qwen2.5-7B-Instruct-AWQ, FALLBACK=GPTQ-Int4
- GPTQ-Int8 also available if AWQ has issues
Apply hard constraints: T4-only GPU, single GPU, 4B fallback if 7B OOMs, min_containers=1
- modal_serve_qwen3b.py: added constraint comments, fallback model vars
- README.md: updated for NIM-only app, added Modal deployment section
- Updated status tracking with correct model names
Switch Modal to Qwen2.5-7B, fix keep_warm->min_containers
- Add modal_serve_qwen3b.py: vLLM serve for Qwen2.5-3B on Modal T4
- Add app_sidewalk_fm.py: Gradio ledger app with dual-model architecture
- Extraction: nvidia/nemotron-mini-4b-instruct (NIM)
- Advisory: Qwen/Qwen2.5-3B-Instruct (Modal)
- All models <=4B: Tiny Titan badge eligible
- Backyard AI track: real problem for small traders
Next: test Modal endpoint, add custom UI, deploy to HF Spaces