|
Download kiro-gateway/docs/en/ARCHITECTURE.md from shimen/API: direct link, hf CLI and curl.
- Browser
- Download file 38.8 kB
-
https://huggingface.co/spaces/shimen/API/resolve/main/kiro-gateway/docs/en/ARCHITECTURE.md
- Command line
-
hf download hf://spaces/shimen/API/kiro-gateway/docs/en/ARCHITECTURE.md
-
curl -L -o ARCHITECTURE.md https://huggingface.co/spaces/shimen/API/resolve/main/kiro-gateway/docs/en/ARCHITECTURE.md
38.8 kB
| # Architectural Overview: Kiro Gateway | |
| ## 1. System Purpose and Goals | |
| The project is a high-level proxy gateway implementing the **"Adapter"** structural design pattern. | |
| The main goal of the system is to provide transparent compatibility between multiple heterogeneous interfaces: | |
| ### Supported API Formats | |
| | API | Endpoints | Status | | |
| |-----|-----------|--------| | |
| | **OpenAI** | `/v1/models`, `/v1/chat/completions` | β Supported | | |
| | **Anthropic** | `/v1/messages` | β Supported | | |
| ### Architectural Model | |
| ``` | |
| βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ | |
| β Clients β | |
| β βββββββββββββββββββββββ βββββββββββββββββββββββ β | |
| β β OpenAI SDK/Tools β β Anthropic SDK/Tools β β | |
| β β (Cursor, Cline, β β (Claude Code, β β | |
| β β Continue, etc.) β β Anthropic SDK) β β | |
| β ββββββββββββ¬βββββββββββ ββββββββββββ¬βββββββββββ β | |
| βββββββββββββββΌβββββββββββββββββββββββββββββββΌββββββββββββββββββββ | |
| β β | |
| βΌ βΌ | |
| βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ | |
| β Kiro Gateway β | |
| β βββββββββββββββββββββββ βββββββββββββββββββββββ β | |
| β β OpenAI Adapter β β Anthropic Adapter β β | |
| β β /v1/chat/... β β /v1/messages β β | |
| β ββββββββββββ¬βββββββββββ ββββββββββββ¬βββββββββββ β | |
| β ββββββββββββββββ¬ββββββββββββββββ β | |
| β βΌ β | |
| β βββββββββββββββββββββββββββββββ β | |
| β β Core Layer β β | |
| β β (Shared conversion logic) β β | |
| β ββββββββββββββββ¬βββββββββββββββ β | |
| ββββββββββββββββββββββββββββββΌβββββββββββββββββββββββββββββββββββββ | |
| β | |
| βΌ | |
| βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ | |
| β Kiro API β | |
| β (AWS CodeWhisperer Backend) β | |
| βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ | |
| ``` | |
| The system acts as a "translator", allowing the use of any tools, libraries, and IDE plugins developed for OpenAI and Anthropic ecosystems with Claude models through the Kiro API. | |
| **Both APIs work simultaneously** on the same server without any configuration switching. | |
| ## 2. Project Structure | |
| The project is organized as a modular Python package `kiro/`: | |
| ``` | |
| kiro-gateway/ | |
| βββ main.py # Entry point, FastAPI application creation | |
| βββ requirements.txt # Python dependencies | |
| βββ .env.example # Environment configuration example | |
| β | |
| βββ kiro/ # Main package | |
| β βββ __init__.py # Package exports, version | |
| β β | |
| β β # βββββββββββββββββββββββββββββββββββββββββββββββββββββββ | |
| β β # SHARED LAYER - Reused by all APIs | |
| β β # βββββββββββββββββββββββββββββββββββββββββββββββββββββββ | |
| β βββ config.py # Configuration and constants | |
| β βββ auth.py # KiroAuthManager - token management | |
| β βββ cache.py # ModelInfoCache - model cache | |
| β βββ http_client.py # HTTP client with retry logic | |
| β βββ parsers.py # AWS SSE stream parsers | |
| β βββ utils.py # Helper utilities | |
| β βββ tokenizer.py # Token counting (tiktoken) | |
| β βββ debug_logger.py # Debug request logging | |
| β βββ exceptions.py # Exception handlers | |
| β βββ thinking_parser.py # Thinking blocks parser | |
| β β | |
| β β # βββββββββββββββββββββββββββββββββββββββββββββββββββββββ | |
| β β # CORE LAYER - Shared core for all APIs | |
| β β # βββββββββββββββββββββββββββββββββββββββββββββββββββββββ | |
| β βββ converters_core.py # Shared Kiro payload building logic | |
| β βββ streaming_core.py # Shared Kiro stream parsing logic | |
| β β | |
| β β # βββββββββββββββββββββββββββββββββββββββββββββββββββββββ | |
| β β # OPENAI API LAYER | |
| β β # βββββββββββββββββββββββββββββββββββββββββββββββββββββββ | |
| β βββ models_openai.py # Pydantic models for OpenAI API | |
| β βββ converters_openai.py # OpenAI β Kiro adapter | |
| β βββ routes_openai.py # FastAPI routes for OpenAI | |
| β βββ streaming_openai.py # Kiro β OpenAI SSE formatter | |
| β β | |
| β β # βββββββββββββββββββββββββββββββββββββββββββββββββββββββ | |
| β β # ANTHROPIC API LAYER | |
| β β # βββββββββββββββββββββββββββββββββββββββββββββββββββββββ | |
| β βββ models_anthropic.py # Pydantic models for Anthropic API | |
| β βββ converters_anthropic.py # Anthropic β Kiro adapter | |
| β βββ routes_anthropic.py # FastAPI routes for Anthropic | |
| β βββ streaming_anthropic.py # Kiro β Anthropic SSE formatter | |
| β | |
| βββ tests/ # Tests | |
| β βββ conftest.py # Pytest fixtures | |
| β βββ unit/ # Unit tests | |
| β βββ integration/ # Integration tests | |
| β | |
| βββ docs/ # Documentation | |
| β βββ ru/ # Russian version | |
| β βββ en/ # English version | |
| β | |
| βββ debug_logs/ # Debug logs (generated when DEBUG_LAST_REQUEST=true) | |
| ``` | |
| ### Organization Principle: Shared Core + Thin Adapters | |
| The architecture is built on the principle of **maximum code reuse**: | |
| | Layer | Purpose | Files | | |
| |-------|---------|-------| | |
| | **Shared Layer** | Infrastructure independent of API format | `auth.py`, `http_client.py`, `cache.py`, `parsers.py`, `tokenizer.py` | | |
| | **Core Layer** | Shared business logic for conversion | `converters_core.py`, `streaming_core.py` | | |
| | **API Layer** | Thin adapters for specific formats | `*_openai.py`, `*_anthropic.py` | | |
| ## 3. Architectural Topology and Components | |
| The system is built on the asynchronous `FastAPI` framework and uses an event-driven lifecycle management model (`Lifespan Events`). | |
| ### 3.1. Entry Point (`main.py`) | |
| The `main.py` file is responsible for: | |
| 1. **Logging configuration** β Loguru setup with colored output | |
| 2. **Configuration validation** β `validate_configuration()` function checks: | |
| - Presence of `.env` file | |
| - Presence of credentials (REFRESH_TOKEN or KIRO_CREDS_FILE) | |
| 3. **Lifespan Manager** β creation and initialization of: | |
| - `KiroAuthManager` for token management | |
| - `ModelInfoCache` for model caching | |
| 4. **Error handler registration** β `validation_exception_handler` for 422 errors | |
| 5. **Route connection** β `app.include_router(router)` | |
| ### 3.2. Configuration Module (`kiro/config.py`) | |
| Centralized storage of all settings: | |
| | Parameter | Description | Default Value | | |
| |-----------|-------------|---------------| | |
| | `PROXY_API_KEY` | API key for proxy access | `changeme_proxy_secret` | | |
| | `REFRESH_TOKEN` | Kiro refresh token | from `.env` | | |
| | `PROFILE_ARN` | AWS CodeWhisperer profile ARN | from `.env` | | |
| | `REGION` | AWS region | `us-east-1` | | |
| | `KIRO_CREDS_FILE` | Path to JSON credentials file | from `.env` | | |
| | `TOKEN_REFRESH_THRESHOLD` | Time before token refresh | 600 sec (10 min) | | |
| | `MAX_RETRIES` | Max retry attempts | 3 | | |
| | `BASE_RETRY_DELAY` | Base retry delay | 1.0 sec | | |
| | `MODEL_CACHE_TTL` | Model cache TTL | 3600 sec (1 hour) | | |
| | `DEFAULT_MAX_INPUT_TOKENS` | Default max input tokens | 200000 | | |
| | `TOOL_DESCRIPTION_MAX_LENGTH` | Max tool description length | 10000 characters | | |
| | `DEBUG_LAST_REQUEST` | Enable debug logging | `false` | | |
| | `DEBUG_DIR` | Debug logs directory | `debug_logs` | | |
| | `APP_VERSION` | Application version | `0.0.0` | | |
| **Helper functions:** | |
| - `get_kiro_refresh_url(region)` β URL for token refresh | |
| - `get_kiro_api_host(region)` β main API host | |
| - `get_kiro_q_host(region)` β Q API host | |
| - `get_internal_model_id(external_model)` β model name conversion | |
| ### 3.3. Pydantic Models (`kiro/models_openai.py`) | |
| #### Models for `/v1/models` | |
| | Model | Description | | |
| |-------|-------------| | |
| | `OpenAIModel` | AI model description (id, object, created, owned_by) | | |
| | `ModelList` | Model list for endpoint response | | |
| #### Models for `/v1/chat/completions` | |
| | Model | Description | | |
| |-------|-------------| | |
| | `ChatMessage` | Chat message (role, content, tool_calls, tool_call_id) | | |
| | `ToolFunction` | Tool function description (name, description, parameters) | | |
| | `Tool` | OpenAI format tool (type, function) | | |
| | `ChatCompletionRequest` | Generation request (model, messages, stream, tools, ...) | | |
| #### Response Models | |
| | Model | Description | | |
| |-------|-------------| | |
| | `ChatCompletionChoice` | Single response variant | | |
| | `ChatCompletionUsage` | Token information (prompt_tokens, completion_tokens, credits_used) | | |
| | `ChatCompletionResponse` | Full response (non-streaming) | | |
| | `ChatCompletionChunk` | Streaming chunk | | |
| | `ChatCompletionChunkDelta` | Delta changes in chunk | | |
| | `ChatCompletionChunkChoice` | Variant in streaming chunk | | |
| ### 3.4. State Management Layer | |
| #### KiroAuthManager (`kiro/auth.py`) | |
| **Role:** Stateful singleton encapsulating Kiro token management logic. | |
| **Capabilities:** | |
| - Loading credentials from `.env` or JSON file | |
| - Support for `expiresAt` to check token expiration time | |
| - Automatic token refresh 10 minutes before expiration | |
| - Saving updated tokens back to JSON file | |
| - Support for different AWS regions | |
| - Unique fingerprint generation for User-Agent | |
| **Concurrency Control:** Uses `asyncio.Lock` to protect against race conditions. | |
| **Main methods:** | |
| - `get_access_token()` β returns valid token, refreshing if necessary | |
| - `force_refresh()` β forced token refresh (on 403) | |
| - `is_token_expiring_soon()` β expiration time check | |
| **Properties:** | |
| - `profile_arn` β profile ARN | |
| - `region` β AWS region | |
| - `api_host` β API host for region | |
| - `q_host` β Q API host for region | |
| - `fingerprint` β unique machine fingerprint | |
| ```python | |
| # Usage example | |
| auth_manager = KiroAuthManager( | |
| refresh_token="your_token", | |
| region="us-east-1", | |
| creds_file="~/.aws/sso/cache/kiro-auth-token.json" | |
| ) | |
| token = await auth_manager.get_access_token() | |
| ``` | |
| #### ModelInfoCache (`kiro/cache.py`) | |
| **Role:** Thread-safe storage for model configurations. | |
| **Population Strategy:** | |
| - Lazy Loading via `/ListAvailableModels` | |
| - Cache TTL: 1 hour | |
| - Fallback to static model list | |
| **Main methods:** | |
| - `update(models_data)` β cache update | |
| - `get(model_id)` β get model information | |
| - `get_max_input_tokens(model_id)` β get token limit | |
| - `is_empty()` / `is_stale()` β cache state check | |
| - `get_all_model_ids()` β list of all model IDs | |
| ### 3.5. Helper Utilities (`kiro/utils.py`) | |
| | Function | Description | | |
| |----------|-------------| | |
| | `get_machine_fingerprint()` | SHA256 hash of `{hostname}-{username}-kiro-gateway` | | |
| | `get_kiro_headers(auth_manager, token)` | Form headers for Kiro API | | |
| | `generate_completion_id()` | ID in format `chatcmpl-{uuid_hex}` | | |
| | `generate_conversation_id()` | UUID for conversation | | |
| | `generate_tool_call_id()` | ID in format `call_{uuid_hex[:8]}` | | |
| ### 3.6. Conversion Layer (`kiro/converters_openai.py`) | |
| #### Message Conversion | |
| OpenAI messages are transformed into Kiro conversationState: | |
| 1. **System prompt** β added to the first user message | |
| 2. **Message history** β fully passed in `history` array | |
| 3. **Adjacent message merging** β messages with the same role are merged | |
| 4. **Tool calls** β OpenAI tools format support | |
| 5. **Tool results** β correct transmission of tool call results | |
| #### Long Tool Description Handling | |
| **Problem:** Kiro API returns error 400 for too long descriptions in `toolSpecification.description`. | |
| **Solution:** Tool Documentation Reference Pattern | |
| - If `description β€ TOOL_DESCRIPTION_MAX_LENGTH` β leave as is | |
| - If `description > TOOL_DESCRIPTION_MAX_LENGTH`: | |
| * In `toolSpecification.description` β reference: `"[Full documentation in system prompt under '## Tool: {name}']"` | |
| * In system prompt, section `"## Tool: {name}"` with full description is added | |
| **Function:** `process_tools_with_long_descriptions(tools)` β `(processed_tools, tool_documentation)` | |
| #### Main Functions | |
| | Function | Description | | |
| |----------|-------------| | |
| | `extract_text_content(content)` | Extract text from various formats | | |
| | `merge_adjacent_messages(messages)` | Merge adjacent messages with same role | | |
| | `build_kiro_history(messages, model_id)` | Build history array for Kiro | | |
| | `build_kiro_payload(request_data, conversation_id, profile_arn)` | Full payload for request | | |
| #### Model Mapping | |
| External model names are converted to internal Kiro IDs: | |
| | External Name | Internal Kiro ID | | |
| |---------------|------------------| | |
| | `claude-opus-4-5` | `claude-opus-4.5` | | |
| | `claude-opus-4-5-20251101` | `claude-opus-4.5` | | |
| | `claude-haiku-4-5` | `claude-haiku-4.5` | | |
| | `claude-haiku-4.5` | `claude-haiku-4.5` (direct passthrough) | | |
| | `claude-sonnet-4-5` | `CLAUDE_SONNET_4_5_20250929_V1_0` | | |
| | `claude-sonnet-4-5-20250929` | `CLAUDE_SONNET_4_5_20250929_V1_0` | | |
| | `claude-sonnet-4` | `CLAUDE_SONNET_4_20250514_V1_0` | | |
| | `claude-sonnet-4-20250514` | `CLAUDE_SONNET_4_20250514_V1_0` | | |
| | `claude-3-7-sonnet-20250219` | `CLAUDE_3_7_SONNET_20250219_V1_0` | | |
| | `auto` | `claude-sonnet-4.5` (alias) | | |
| ### 3.7. Parsing Layer (`kiro/parsers.py`) | |
| #### AwsEventStreamParser | |
| Advanced AWS SSE format parser with support for: | |
| - **Bracket counting** β correct parsing of nested JSON objects | |
| - **Content deduplication** β filtering of duplicate events | |
| - **Tool calls** β parsing of structured and bracket-style tool calls | |
| - **Escape sequences** β decoding of `\n` and others | |
| #### Event Types | |
| | Event | Description | | |
| |-------|-------------| | |
| | `content` | Text content of the response | | |
| | `tool_start` | Start of tool call (name, toolUseId) | | |
| | `tool_input` | Continuation of input for tool call | | |
| | `tool_stop` | End of tool call | | |
| | `usage` | Credit consumption information | | |
| | `context_usage` | Context usage percentage | | |
| #### Helper Functions | |
| | Function | Description | | |
| |----------|-------------| | |
| | `find_matching_brace(text, start_pos)` | Find closing brace with nesting support | | |
| | `parse_bracket_tool_calls(response_text)` | Parse `[Called func with args: {...}]` | | |
| | `deduplicate_tool_calls(tool_calls)` | Remove duplicate tool calls | | |
| ### 3.8. Streaming (`kiro/streaming_openai.py`) | |
| #### stream_kiro_to_openai | |
| Async generator for transforming Kiro stream to OpenAI format. | |
| **Functionality:** | |
| - Parse AWS SSE stream via `AwsEventStreamParser` | |
| - Form OpenAI `chat.completion.chunk` | |
| - Handle tool calls (structured and bracket-style) | |
| - Calculate usage based on `contextUsagePercentage` | |
| - Debug logging via `debug_logger` | |
| #### collect_stream_response | |
| Collects full response from streaming for non-streaming mode. | |
| ### 3.9. HTTP Client (`kiro/http_client.py`) | |
| #### KiroHttpClient | |
| Automatic error handling with exponential backoff: | |
| | Error Code | Action | | |
| |------------|--------| | |
| | `403` | Token refresh via `force_refresh()` + retry | | |
| | `429` | Exponential backoff: `BASE_RETRY_DELAY * (2 ** attempt)` | | |
| | `5xx` | Exponential backoff (up to MAX_RETRIES attempts) | | |
| | Timeout | Exponential backoff | | |
| **Delay formula:** `1s, 2s, 4s` (with `BASE_RETRY_DELAY=1.0`) | |
| **Methods:** | |
| - `request_with_retry(method, url, json_data, stream)` β request with retry | |
| - `close()` β close client | |
| Supports async context manager (`async with`). | |
| ### 3.10. Routes (`kiro/routes_openai.py`) | |
| | Endpoint | Method | Description | | |
| |----------|--------|-------------| | |
| | `/` | GET | Health check (status, message, version) | | |
| | `/health` | GET | Detailed health check (status, timestamp, version) | | |
| | `/v1/models` | GET | List of available models (requires API key) | | |
| | `/v1/chat/completions` | POST | Chat completions (requires API key) | | |
| **Authentication:** Bearer token in `Authorization` header | |
| ### 3.11. Exception Handling (`kiro/exceptions.py`) | |
| | Function | Description | | |
| |----------|-------------| | |
| | `sanitize_validation_errors(errors)` | Convert bytes to strings for JSON serialization | | |
| | `validation_exception_handler(request, exc)` | Pydantic validation error handler (422) | | |
| ### 3.12. Debug Logging (`kiro/debug_logger.py`) | |
| **Class:** `DebugLogger` (singleton) | |
| **Activation:** `DEBUG_LAST_REQUEST=true` in `.env` | |
| **Methods:** | |
| | Method | Description | | |
| |--------|-------------| | |
| | `prepare_new_request()` | Clear directory for new request | | |
| | `log_request_body(body)` | Save incoming request | | |
| | `log_kiro_request_body(body)` | Save request to Kiro API | | |
| | `log_raw_chunk(chunk)` | Append raw chunk from Kiro | | |
| | `log_modified_chunk(chunk)` | Append transformed chunk | | |
| **Files in `debug_logs/`:** | |
| - `request_body.json` β incoming request (OpenAI format) | |
| - `kiro_request_body.json` β request to Kiro API | |
| - `response_stream_raw.txt` β raw stream from Kiro | |
| - `response_stream_modified.txt` β transformed stream (OpenAI format) | |
| ### 3.13. Tokenizer (`kiro/tokenizer.py`) | |
| **Problem:** Kiro API does not return token counts directly. Instead, the API only provides `context_usage_percentage` β the percentage of model context usage. | |
| **Solution:** Tokenizer module based on `tiktoken` (OpenAI's Rust library) for fast token counting. | |
| **Features:** | |
| - Uses `cl100k_base` encoding (GPT-4), close to Claude tokenization | |
| - Correction factor `CLAUDE_CORRECTION_FACTOR = 1.15` for improved accuracy | |
| - Lazy initialization for faster imports | |
| - Fallback to rough estimation if tiktoken is unavailable | |
| **Token calculation formula in response:** | |
| ``` | |
| total_tokens = context_usage_percentage Γ max_input_tokens (from Kiro API) | |
| completion_tokens = tiktoken(response) (our calculation) | |
| prompt_tokens = total_tokens - completion_tokens (subtraction) | |
| ``` | |
| **Main functions:** | |
| | Function | Description | | |
| |----------|-------------| | |
| | `count_tokens(text)` | Count tokens in text | | |
| | `count_message_tokens(messages)` | Count tokens in message list | | |
| | `count_tools_tokens(tools)` | Count tokens in tool definitions | | |
| | `estimate_request_tokens(messages, tools)` | Full request token estimation | | |
| **Debug log:** | |
| ``` | |
| [Usage] claude-opus-4-5: prompt_tokens=142211 (subtraction), completion_tokens=769 (tiktoken), total_tokens=142980 (API Kiro) | |
| ``` | |
| **Accuracy:** ~97-99.7% compared to API data. | |
| ### 3.14. Kiro API Endpoints | |
| All URLs are dynamically formed based on the region: | |
| * **Token Refresh:** `POST https://prod.{region}.auth.desktop.kiro.dev/refreshToken` | |
| * **List Models:** `GET https://q.{region}.amazonaws.com/ListAvailableModels` | |
| * **Generate Response:** `POST https://codewhisperer.{region}.amazonaws.com/generateAssistantResponse` | |
| ## 4. Detailed Data Flow | |
| ### 4.1 Multi-API Overview | |
| ``` | |
| βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ | |
| β CLIENTS β | |
| β βββββββββββββββββββββββ βββββββββββββββββββββββ β | |
| β β OpenAI Client β β Anthropic Client β β | |
| β ββββββββββββ¬βββββββββββ ββββββββββββ¬βββββββββββ β | |
| βββββββββββββββΌβββββββββββββββββββββββββββββββΌββββββββββββββββββββ | |
| β β | |
| β POST /v1/chat/completions β POST /v1/messages | |
| βΌ βΌ | |
| βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ | |
| β API LAYER β | |
| β βββββββββββββββββββββββ βββββββββββββββββββββββ β | |
| β β routes_openai.py β β routes_anthropic.py β β | |
| β β Security Gate β β Security Gate β β | |
| β ββββββββββββ¬βββββββββββ ββββββββββββ¬βββββββββββ β | |
| β β β β | |
| β βΌ βΌ β | |
| β βββββββββββββββββββββββ βββββββββββββββββββββββ β | |
| β βconverters_openai.py β βconverters_anthropic β β | |
| β β Extract system β β System already β β | |
| β β from messages β β separate in request β β | |
| β ββββββββββββ¬βββββββββββ ββββββββββββ¬βββββββββββ β | |
| βββββββββββββββΌβββββββββββββββββββββββββββββββΌββββββββββββββββββββ | |
| β β | |
| ββββββββββββββββ¬ββββββββββββββββ | |
| β | |
| βΌ | |
| βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ | |
| β CORE LAYER β | |
| β βββββββββββββββββββββββββββββββ β | |
| β β converters_core.py β β | |
| β β build_kiro_payload() β β | |
| β β build_kiro_history() β β | |
| β β process_tools() β β | |
| β ββββββββββββββββ¬βββββββββββββββ β | |
| ββββββββββββββββββββββββββββββΌβββββββββββββββββββββββββββββββββββββ | |
| β | |
| βΌ | |
| βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ | |
| β SHARED LAYER β | |
| β βββββββββββββββββββ βββββββββββββββββββ βββββββββββββββββββ β | |
| β β KiroAuthManager β β KiroHttpClient β β ModelInfoCache β β | |
| β β (auth.py) β β(http_client.py) β β (cache.py) β β | |
| β ββββββββββ¬βββββββββ ββββββββββ¬βββββββββ βββββββββββββββββββ β | |
| βββββββββββββΌβββββββββββββββββββββΌβββββββββββββββββββββββββββββββββ | |
| β β | |
| β β POST /generateAssistantResponse | |
| β βΌ | |
| β βββββββββββββββββββββββββββββββββββββββ | |
| β β Kiro API β | |
| β ββββββββββββββββββββ¬βββββββββββββββββββββββ | |
| β β | |
| β β AWS SSE Stream | |
| β βΌ | |
| βββββββββββββΌβββββββββββββββββββββββββββββββββββββββββββββββββββββ | |
| β β CORE LAYER β | |
| β β βββββββββββββββββββββββββββββββ β | |
| β β β streaming_core.py β β | |
| β β β parse_kiro_stream() β β | |
| β β β β KiroEvent objects β β | |
| β β ββββββββββββββββ¬βββββββββββββββ β | |
| ββββββββββββββββββββββββββββββΌββββββββββββββββββββββββββββββββββββ | |
| β | |
| ββββββββββββββββ΄ββββββββββββββββ | |
| β β | |
| βΌ βΌ | |
| βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ | |
| β OUTPUT LAYER β | |
| β βββββββββββββββββββββββ βββββββββββββββββββββββ β | |
| β βstreaming_openai.py β βstreaming_anthropic β β | |
| β β format_openai_sse() β βformat_anthropic_sse β β | |
| β β β β β β | |
| β β data: {...} β β event: type β β | |
| β β data: [DONE] β β data: {...} β β | |
| β ββββββββββββ¬βββββββββββ ββββββββββββ¬βββββββββββ β | |
| βββββββββββββββΌβββββββββββββββββββββββββββββββΌββββββββββββββββββββ | |
| β β | |
| βΌ βΌ | |
| βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ | |
| β CLIENTS β | |
| β βββββββββββββββββββββββ βββββββββββββββββββββββ β | |
| β β OpenAI Client β β Anthropic Client β β | |
| β βββββββββββββββββββββββ βββββββββββββββββββββββ β | |
| βββββββββββββββββββββββββββββββββββ | |
| ``` | |
| ### 4.2 OpenAI API Flow | |
| ``` | |
| OpenAI Client | |
| β POST /v1/chat/completions | |
| βΌ | |
| routes_openai.py βββΊ converters_openai.py βββΊ converters_core.py | |
| β β | |
| β βΌ | |
| β Kiro Payload | |
| β β | |
| βΌ βΌ | |
| KiroAuthManager βββββββββββββββββββββββββββΊ KiroHttpClient | |
| β | |
| βΌ | |
| Kiro API | |
| β | |
| βΌ | |
| streaming_core.py ββββββββββββββββββββββββ AWS SSE Stream | |
| β | |
| βΌ | |
| streaming_openai.py | |
| β | |
| βΌ | |
| OpenAI SSE Format βββββββββββββββββββββββΊ OpenAI Client | |
| ``` | |
| ### 4.3 Anthropic API Flow | |
| ``` | |
| Anthropic Client | |
| β POST /v1/messages | |
| βΌ | |
| routes_anthropic.py βββΊ converters_anthropic.py βββΊ converters_core.py | |
| β β | |
| β βΌ | |
| β Kiro Payload | |
| β β | |
| βΌ βΌ | |
| KiroAuthManager βββββββββββββββββββββββββββββββββββΊ KiroHttpClient | |
| β | |
| βΌ | |
| Kiro API | |
| β | |
| βΌ | |
| streaming_core.py ββββββββββββββββββββββββββββββββ AWS SSE Stream | |
| β | |
| βΌ | |
| streaming_anthropic.py | |
| β | |
| βΌ | |
| Anthropic SSE Format βββββββββββββββββββββββββββΊ Anthropic Client | |
| ``` | |
| ## 5. Available Models | |
| | Model | Description | Credits | | |
| |-------|-------------|---------| | |
| | `claude-opus-4-5` | Top-tier model | ~2.2 | | |
| | `claude-opus-4-5-20251101` | Top-tier model (version) | ~2.2 | | |
| | `claude-sonnet-4-5` | Enhanced model | ~1.3 | | |
| | `claude-sonnet-4-5-20250929` | Enhanced model (version) | ~1.3 | | |
| | `claude-sonnet-4` | Balanced model | ~1.3 | | |
| | `claude-sonnet-4-20250514` | Balanced (version) | ~1.3 | | |
| | `claude-haiku-4-5` | Fast model | ~0.4 | | |
| | `claude-3-7-sonnet-20250219` | Legacy model | ~1.0 | | |
| ## 6. Configuration | |
| ### Environment Variables (.env) | |
| ```env | |
| # Required | |
| REFRESH_TOKEN="your_kiro_refresh_token" | |
| PROXY_API_KEY="your_proxy_secret" | |
| # Optional | |
| PROFILE_ARN="arn:aws:codewhisperer:..." | |
| KIRO_REGION="us-east-1" | |
| KIRO_CREDS_FILE="~/.aws/sso/cache/kiro-auth-token.json" | |
| # Debug | |
| DEBUG_LAST_REQUEST="false" | |
| DEBUG_DIR="debug_logs" | |
| # Limits | |
| TOOL_DESCRIPTION_MAX_LENGTH="10000" | |
| ``` | |
| ### JSON Credentials File (optional) | |
| ```json | |
| { | |
| "accessToken": "eyJ...", | |
| "refreshToken": "eyJ...", | |
| "expiresAt": "2025-01-12T23:00:00.000Z", | |
| "profileArn": "arn:aws:codewhisperer:us-east-1:...", | |
| "region": "us-east-1" | |
| } | |
| ``` | |
| ## 7. API Endpoints | |
| ### 7.1 Common Endpoints | |
| | Endpoint | Method | Description | | |
| |----------|--------|-------------| | |
| | `/` | GET | Health check | | |
| | `/health` | GET | Detailed health check | | |
| ### 7.2 OpenAI-compatible Endpoints | |
| | Endpoint | Method | Description | | |
| |----------|--------|-------------| | |
| | `/v1/models` | GET | List of available models | | |
| | `/v1/chat/completions` | POST | Chat completions (streaming/non-streaming) | | |
| **Authentication:** `Authorization: Bearer {PROXY_API_KEY}` | |
| ### 7.3 Anthropic-compatible Endpoints | |
| | Endpoint | Method | Description | | |
| |----------|--------|-------------| | |
| | `/v1/messages` | POST | Messages API (streaming/non-streaming) | | |
| **Authentication:** `x-api-key: {PROXY_API_KEY}` + `anthropic-version: 2023-06-01` | |
| ### 7.4 Format Comparison | |
| | Aspect | OpenAI | Anthropic | | |
| |--------|--------|-----------| | |
| | System prompt | In `messages` with `role: "system"` | Separate `system` field | | |
| | Content | String or array | Always array of content blocks | | |
| | Stop reason | `finish_reason: "stop"` | `stop_reason: "end_turn"` | | |
| | Usage | `prompt_tokens`, `completion_tokens` | `input_tokens`, `output_tokens` | | |
| | Streaming | `data: {...}\n\n` + `data: [DONE]` | `event: type\ndata: {...}\n\n` | | |
| | Tool format | `{type: "function", function: {...}}` | `{name: "...", input_schema: {...}}` | | |
| ## 8. Implementation Features | |
| ### Tool Calling | |
| Support for OpenAI-compatible tools format: | |
| ```json | |
| { | |
| "tools": [{ | |
| "type": "function", | |
| "function": { | |
| "name": "get_weather", | |
| "description": "Get weather for a location", | |
| "parameters": { | |
| "type": "object", | |
| "properties": { | |
| "location": {"type": "string"} | |
| } | |
| } | |
| } | |
| }] | |
| } | |
| ``` | |
| ### Streaming | |
| Full SSE streaming support with correct OpenAI format: | |
| ``` | |
| data: {"id":"chatcmpl-...","object":"chat.completion.chunk",...} | |
| data: [DONE] | |
| ``` | |
| ### Debugging | |
| When `DEBUG_LAST_REQUEST=true`, all requests and responses are logged in `debug_logs/`: | |
| - `request_body.json` β incoming request | |
| - `kiro_request_body.json` β request to Kiro API | |
| - `response_stream_raw.txt` β raw stream from Kiro | |
| - `response_stream_modified.txt` β transformed stream | |
| ## 9. Extensibility | |
| ### Adding a New API Format | |
| The modular architecture allows easy addition of support for other API formats. Thanks to the Core Layer, most of the logic is already implemented. | |
| #### Steps to Add a New Format (e.g., Gemini) | |
| 1. **Create models** β `models_gemini.py` | |
| ```python | |
| class GeminiRequest(BaseModel): | |
| """Pydantic model for Gemini request.""" | |
| contents: List[GeminiContent] | |
| ... | |
| ``` | |
| 2. **Create conversion adapter** β `converters_gemini.py` | |
| ```python | |
| from kiro.converters_core import build_kiro_payload | |
| def gemini_to_kiro(request: GeminiRequest, ...) -> dict: | |
| """Converts Gemini request to Kiro payload.""" | |
| # Extract data from Gemini format | |
| system_prompt = extract_system_instruction(request) | |
| messages = convert_gemini_contents(request.contents) | |
| tools = convert_gemini_tools(request.tools) | |
| # Use shared core | |
| return build_kiro_payload( | |
| messages=messages, | |
| system_prompt=system_prompt, | |
| tools=tools, | |
| ... | |
| ) | |
| ``` | |
| 3. **Create streaming formatter** β `streaming_gemini.py` | |
| ```python | |
| from kiro.streaming_core import parse_kiro_stream | |
| async def stream_to_gemini(response, ...) -> AsyncGenerator[str, None]: | |
| """Formats Kiro events to Gemini SSE.""" | |
| async for event in parse_kiro_stream(response): | |
| yield format_gemini_chunk(event) | |
| ``` | |
| 4. **Create routes** β `routes_gemini.py` | |
| ```python | |
| router = APIRouter() | |
| @router.post("/v1beta/models/{model}:generateContent") | |
| async def generate_content(request: GeminiRequest): | |
| ... | |
| ``` | |
| 5. **Connect in main.py** | |
| ```python | |
| from kiro.routes_gemini import router as gemini_router | |
| app.include_router(gemini_router) | |
| ``` | |
| ### What Gets Reused Automatically | |
| When adding a new format, the following components work out of the box: | |
| | Component | Functionality | | |
| |-----------|---------------| | |
| | `auth.py` | Kiro token management | | |
| | `http_client.py` | HTTP with retry logic | | |
| | `cache.py` | Model cache | | |
| | `parsers.py` | AWS SSE parsing | | |
| | `tokenizer.py` | Token counting | | |
| | `converters_core.py` | Kiro payload building | | |
| | `streaming_core.py` | Kiro stream parsing | | |
| ## 10. Dependencies | |
| Main project dependencies (from `requirements.txt`): | |
| | Package | Purpose | | |
| |---------|---------| | |
| | `fastapi` | Asynchronous web framework | | |
| | `uvicorn` | ASGI server | | |
| | `httpx` | Asynchronous HTTP client | | |
| | `pydantic` | Data validation and models | | |
| | `python-dotenv` | Environment variable loading | | |
| | `loguru` | Advanced logging | | |
| | `tiktoken` | Fast token counting | | |