Spaces:
Running
Running
Commit History
-
feat: Kubernetes-ready multi-provider LLM gateway v0.2.0
- Add Google Gemini adapter (gemini-1.5-flash/pro) with real SDK + mock mode - Add local Ollama adapter (llama3/mistral) via httpx /api/chat - Add Redis-backed circuit breaker (CLOSED/OPEN/HALF_OPEN state machine) - Upgrade router: SLO-aware (p50/p95/p99), sliding window percentiles, 4 backends - Upgrade cache: prefix-aware LRU + request deduplication via Redis pub/sub - Expand DB schema: provider, slo_met, dedup_hit, prefix_cache_hit, circuit_state - Add new metrics: SLO violations, CB state/trips, dedup hits, prefix cache hits - Add /v1/providers and /v1/routing/status endpoints - Add Dockerfile (multi-stage, non-root) and updated docker-compose.yml - Add 9 Kubernetes manifests (namespace/configmap/secret/deployment/service/hpa/redis/postgres/ingress) - Add circuit breaker unit tests (8 scenarios) - Add cache unit tests (8 scenarios) - Add SLO router unit tests (9 scenarios) - Add Gemini/Ollama adapter mock tests - Add Locust load test (4 user classes: Normal/Burst/Cost/Chaos) - Add …8a841b2