|
Download docs/IMPLEMENTATION_TASKS.md from shimen/kir: direct link, hf CLI and curl.
- Browser
- Download file 10.6 kB
-
https://huggingface.co/spaces/shimen/kir/resolve/main/docs/IMPLEMENTATION_TASKS.md
- Command line
-
hf download hf://spaces/shimen/kir/docs/IMPLEMENTATION_TASKS.md
-
curl -L -o IMPLEMENTATION_TASKS.md https://huggingface.co/spaces/shimen/kir/resolve/main/docs/IMPLEMENTATION_TASKS.md
10.6 kB
| # Implementation Tasks | |
| # Kiro Gateway: Metrics, Dashboard & Load Balancing | |
| ## Master Task List | |
| --- | |
| ## Phase 1: Metrics Foundation | |
| ### 1.1 Backend Metrics Module | |
| - [ ] **TASK-1.1.1** Create `/home/kiro/metrics/` directory structure | |
| - [ ] **TASK-1.1.2** Create `__init__.py` with module exports | |
| - [ ] **TASK-1.1.3** Implement `prometheus.py`: | |
| - Define Counter for `kiro_requests_total` | |
| - Define Histogram for `kiro_request_duration_seconds` | |
| - Define Counter for `kiro_tokens_input_total` | |
| - Define Counter for `kiro_tokens_output_total` | |
| - Define Gauge for `kiro_active_requests` | |
| - Define Counter for `kiro_errors_total` | |
| - [ ] **TASK-1.1.4** Implement `opentelemetry.py`: | |
| - Configure TracerProvider | |
| - Configure MeterProvider | |
| - Set up OTLP exporter | |
| - Auto-instrument FastAPI | |
| - [ ] **TASK-1.1.5** Implement `middleware.py`: | |
| - Create `MetricsMiddleware` class | |
| - Track request start/end times | |
| - Record status codes | |
| - Handle streaming responses | |
| - [ ] **TASK-1.1.6** Implement `collectors.py`: | |
| - Custom collector for token usage | |
| - Per-model metrics aggregation | |
| ### 1.2 Integration | |
| - [ ] **TASK-1.2.1** Add metrics middleware to `main.py` | |
| - [ ] **TASK-1.2.2** Add `/metrics` endpoint for Prometheus scraping | |
| - [ ] **TASK-1.2.3** Integrate token counting in `streaming_openai.py` | |
| - [ ] **TASK-1.2.4** Integrate token counting in `streaming_anthropic.py` | |
| - [ ] **TASK-1.2.5** Add metrics config to `config.py`: | |
| - `METRICS_ENABLED` | |
| - `METRICS_PREFIX` | |
| - `OTEL_EXPORTER_OTLP_ENDPOINT` | |
| - [ ] **TASK-1.2.6** Update `requirements.txt` with metrics dependencies | |
| ### 1.3 Docker & Testing | |
| - [ ] **TASK-1.3.1** Create `docker/prometheus/prometheus.yml` | |
| - [ ] **TASK-1.3.2** Update `docker-compose.yml` with Prometheus service | |
| - [ ] **TASK-1.3.3** Write unit tests for metrics collection | |
| - [ ] **TASK-1.3.4** Test Prometheus scraping | |
| --- | |
| ## Phase 2: Dashboard MVP | |
| ### 2.1 Project Setup | |
| - [ ] **TASK-2.1.1** Initialize Next.js 16 project: | |
| ```bash | |
| npx create-next-app@latest dashboard --typescript --tailwind --app --src-dir | |
| ``` | |
| - [ ] **TASK-2.1.2** Configure `tailwind.config.js` | |
| - [ ] **TASK-2.1.3** Install and configure shadcn/ui: | |
| ```bash | |
| npx shadcn-ui@latest init | |
| ``` | |
| - [ ] **TASK-2.1.4** Install Recharts: | |
| ```bash | |
| npm install recharts | |
| ``` | |
| - [ ] **TASK-2.1.5** Install dependencies: | |
| ```bash | |
| npm install @tanstack/react-query zustand socket.io-client lucide-react | |
| ``` | |
| - [ ] **TASK-2.1.6** Set up project structure (folders) | |
| ### 2.2 Core Components | |
| - [ ] **TASK-2.2.1** Create `components/ui/` with shadcn components: | |
| - Button, Card, Input, Select, Table, Tabs, Badge, Avatar | |
| - [ ] **TASK-2.2.2** Create `components/dashboard/Sidebar.tsx` | |
| - [ ] **TASK-2.2.3** Create `components/dashboard/Header.tsx` | |
| - [ ] **TASK-2.2.4** Create `components/dashboard/StatsCards.tsx` | |
| - [ ] **TASK-2.2.5** Create `components/common/ThemeToggle.tsx` | |
| - [ ] **TASK-2.2.6** Create `app/(dashboard)/layout.tsx` | |
| ### 2.3 Charts (Recharts) | |
| - [ ] **TASK-2.3.1** Create `components/charts/RequestsChart.tsx` (AreaChart) | |
| - [ ] **TASK-2.3.2** Create `components/charts/LatencyChart.tsx` (BarChart) | |
| - [ ] **TASK-2.3.3** Create `components/charts/TokenUsageChart.tsx` (ComposedChart) | |
| - [ ] **TASK-2.3.4** Create `components/charts/ErrorRateChart.tsx` (PieChart) | |
| - [ ] **TASK-2.3.5** Create `components/charts/AccountHealthChart.tsx` (BarChart) | |
| ### 2.4 Pages | |
| - [ ] **TASK-2.4.1** Implement `app/(dashboard)/overview/page.tsx`: | |
| - Stats cards (requests, tokens, latency, uptime) | |
| - Mini charts | |
| - Recent activity feed | |
| - [ ] **TASK-2.4.2** Implement `app/(dashboard)/metrics/page.tsx`: | |
| - Time range selector | |
| - Model filter | |
| - Full-size interactive charts | |
| - [ ] **TASK-2.4.3** Implement `app/(dashboard)/accounts/page.tsx`: | |
| - Account list table | |
| - Health status indicators | |
| - Add/Edit modal | |
| - Delete confirmation | |
| - [ ] **TASK-2.4.4** Implement `app/(dashboard)/config/page.tsx`: | |
| - Configuration viewer | |
| - Edit form for editable settings | |
| - Model aliases manager | |
| - [ ] **TASK-2.4.5** Implement `app/(dashboard)/logs/page.tsx`: | |
| - Log level filter | |
| - Search input | |
| - Virtual scrolling log list | |
| - WebSocket connection | |
| ### 2.5 API & State | |
| - [ ] **TASK-2.5.1** Create `lib/api.ts`: | |
| - Axios/fetch client setup | |
| - API base URL configuration | |
| - Auth header injection | |
| - [ ] **TASK-2.5.2** Create `lib/websocket.ts`: | |
| - Socket.io connection | |
| - Event handlers | |
| - Reconnection logic | |
| - [ ] **TASK-2.5.3** Create `hooks/useMetrics.ts` with TanStack Query | |
| - [ ] **TASK-2.5.4** Create `hooks/useAccounts.ts` with mutations | |
| - [ ] **TASK-2.5.5** Create Zustand store for global state | |
| - [ ] **TASK-2.5.6** Create `types/` TypeScript definitions | |
| --- | |
| ## Phase 3: Load Balancer Core | |
| ### 3.1 Module Structure | |
| - [ ] **TASK-3.1.1** Create `/home/kiro/loadbalancer/` directory | |
| - [ ] **TASK-3.1.2** Create `__init__.py` with exports | |
| - [ ] **TASK-3.1.3** Create `models.py`: | |
| - `KiroAccount` Pydantic model | |
| - `LoadBalancerConfig` model | |
| - `AccountHealth` enum | |
| ### 3.2 Core Implementation | |
| - [ ] **TASK-3.2.1** Implement `pool.py`: | |
| - `AccountPool` class | |
| - Account registration/removal | |
| - Get next account method | |
| - Account state tracking | |
| - [ ] **TASK-3.2.2** Implement `strategies.py`: | |
| - `LoadBalancingStrategy` abstract base | |
| - `RoundRobinStrategy` | |
| - `WeightedRoundRobinStrategy` | |
| - `LeastConnectionsStrategy` | |
| - `RandomStrategy` | |
| - `FailoverStrategy` | |
| - [ ] **TASK-3.2.3** Implement `health_checker.py`: | |
| - Async health check loop | |
| - Token refresh verification | |
| - API ping test | |
| - Error rate tracking | |
| ### 3.3 Integration | |
| - [ ] **TASK-3.3.1** Modify `KiroAuthManager` to support multiple instances | |
| - [ ] **TASK-3.3.2** Create account factory function | |
| - [ ] **TASK-3.3.3** Update `main.py` lifespan to initialize pool | |
| - [ ] **TASK-3.3.4** Update `routes_openai.py` to use load balancer: | |
| - Get account from pool before request | |
| - Return account to pool after request | |
| - Handle account failures | |
| - [ ] **TASK-3.3.5** Update `routes_anthropic.py` similarly | |
| - [ ] **TASK-3.3.6** Add load balancer config to `config.py`: | |
| - `LOAD_BALANCER_ENABLED` | |
| - `LOAD_BALANCER_STRATEGY` | |
| - `LOAD_BALANCER_ACCOUNTS_FILE` | |
| - `LOAD_BALANCER_HEALTH_CHECK_INTERVAL` | |
| ### 3.4 Account Management | |
| - [ ] **TASK-3.4.1** Create `accounts.json` schema | |
| - [ ] **TASK-3.4.2** Implement account loading from JSON file | |
| - [ ] **TASK-3.4.3** Implement account loading from environment variables | |
| - [ ] **TASK-3.4.4** Implement runtime account CRUD API | |
| --- | |
| ## Phase 4: Dashboard Backend API | |
| ### 4.1 API Implementation | |
| - [ ] **TASK-4.1.1** Create `/home/kiro/dashboard_api/` directory | |
| - [ ] **TASK-4.1.2** Create `models.py`: | |
| - `DashboardStats` response model | |
| - `MetricsQuery` request model | |
| - `AccountCreate/Update` models | |
| - [ ] **TASK-4.1.3** Implement `routes.py`: | |
| - `GET /api/dashboard/stats` | |
| - `GET /api/dashboard/metrics` | |
| - `GET /api/dashboard/accounts` | |
| - `POST /api/dashboard/accounts` | |
| - `PUT /api/dashboard/accounts/{id}` | |
| - `DELETE /api/dashboard/accounts/{id}` | |
| - `GET /api/dashboard/config` | |
| - `PUT /api/dashboard/config` | |
| - [ ] **TASK-4.1.4** Implement `websocket.py`: | |
| - WebSocket endpoint for logs | |
| - Real-time metrics streaming | |
| - [ ] **TASK-4.1.5** Register routes in `main.py` | |
| ### 4.2 CORS & Security | |
| - [ ] **TASK-4.2.1** Configure CORS for dashboard origin | |
| - [ ] **TASK-4.2.2** Add dashboard API key verification | |
| - [ ] **TASK-4.2.3** Mask sensitive data in responses | |
| --- | |
| ## Phase 5: Polish & Deployment | |
| ### 5.1 Dashboard Polish | |
| - [ ] **TASK-5.1.1** Implement dark/light theme persistence | |
| - [ ] **TASK-5.1.2** Add loading states to all components | |
| - [ ] **TASK-5.1.3** Add error boundaries | |
| - [ ] **TASK-5.1.4** Implement responsive design for mobile | |
| - [ ] **TASK-5.1.5** Add keyboard shortcuts | |
| - [ ] **TASK-5.1.6** Create empty states for no data | |
| ### 5.2 Testing | |
| - [ ] **TASK-5.2.1** Write unit tests for load balancer strategies | |
| - [ ] **TASK-5.2.2** Write unit tests for health checker | |
| - [ ] **TASK-5.2.3** Write integration tests for dashboard API | |
| - [ ] **TASK-5.2.4** Write Playwright E2E tests for dashboard | |
| - [ ] **TASK-5.2.5** Performance testing for metrics overhead | |
| ### 5.3 Docker & Deployment | |
| - [ ] **TASK-5.3.1** Create `docker/Dockerfile.dashboard` | |
| - [ ] **TASK-5.3.2** Create `docker/docker-compose.full.yml`: | |
| - Kiro Gateway service | |
| - Dashboard service | |
| - Prometheus service | |
| - Grafana service (optional) | |
| - [ ] **TASK-5.3.3** Create Grafana dashboard templates | |
| - [ ] **TASK-5.3.4** Update main README with new features | |
| - [ ] **TASK-5.3.5** Create deployment guide | |
| --- | |
| ## Quick Reference: File Locations | |
| ### Backend (Python) | |
| ``` | |
| /home/kiro/metrics/__init__.py | |
| /home/kiro/metrics/prometheus.py | |
| /home/kiro/metrics/opentelemetry.py | |
| /home/kiro/metrics/middleware.py | |
| /home/kiro/metrics/collectors.py | |
| /home/kiro/loadbalancer/__init__.py | |
| /home/kiro/loadbalancer/pool.py | |
| /home/kiro/loadbalancer/strategies.py | |
| /home/kiro/loadbalancer/health_checker.py | |
| /home/kiro/loadbalancer/models.py | |
| /home/kiro/dashboard_api/__init__.py | |
| /home/kiro/dashboard_api/routes.py | |
| /home/kiro/dashboard_api/models.py | |
| /home/kiro/dashboard_api/websocket.py | |
| ``` | |
| ### Dashboard (Next.js) | |
| ``` | |
| /home/dashboard/app/layout.tsx | |
| /home/dashboard/app/(dashboard)/overview/page.tsx | |
| /home/dashboard/app/(dashboard)/metrics/page.tsx | |
| /home/dashboard/app/(dashboard)/accounts/page.tsx | |
| /home/dashboard/app/(dashboard)/config/page.tsx | |
| /home/dashboard/app/(dashboard)/logs/page.tsx | |
| /home/dashboard/components/ui/* | |
| /home/dashboard/components/charts/* | |
| /home/dashboard/components/dashboard/* | |
| /home/dashboard/lib/api.ts | |
| /home/dashboard/lib/websocket.ts | |
| /home/dashboard/hooks/* | |
| /home/dashboard/types/* | |
| ``` | |
| ### Docker | |
| ``` | |
| /home/docker/Dockerfile.dashboard | |
| /home/docker/docker-compose.full.yml | |
| /home/docker/prometheus/prometheus.yml | |
| ``` | |
| --- | |
| ## Estimated Timeline | |
| | Phase | Duration | Dependencies | | |
| |-------|----------|--------------| | |
| | Phase 1: Metrics | 1 week | None | | |
| | Phase 2: Dashboard MVP | 2 weeks | Phase 1 partial | | |
| | Phase 3: Load Balancer | 1.5 weeks | None | | |
| | Phase 4: Dashboard API | 1 week | Phase 1, 3 | | |
| | Phase 5: Polish | 1 week | All phases | | |
| **Total: ~6-7 weeks** | |
| --- | |
| ## Priority Order (MCP - Most Critical Path) | |
| 1. **HIGH** - TASK-1.1.3 (Prometheus metrics) - Foundation for everything | |
| 2. **HIGH** - TASK-3.2.1 (Account pool) - Core load balancer | |
| 3. **HIGH** - TASK-2.1.1 (Next.js setup) - Dashboard foundation | |
| 4. **MEDIUM** - TASK-1.2.2 (/metrics endpoint) - Enables monitoring | |
| 5. **MEDIUM** - TASK-3.2.2 (LB strategies) - Load balancer features | |
| 6. **MEDIUM** - TASK-2.4.1 (Overview page) - Dashboard MVP | |
| 7. **LOW** - TASK-5.3.3 (Grafana templates) - Nice to have | |