ai-api-testing / Project_diagrams.md
Amitkasar's picture
Remove binary files from tracking
8085285
|
Raw History Blame Contribute Delete
10.9 kB

System Architecture β€” AI API Testing Assistant


1. High-Level System Overview

graph TB
    subgraph CLIENT["πŸ–₯️ Client Layer"]
        UI["React + Vite SPA<br/>(frontend-react)"]
    end

    subgraph BACKEND["βš™οΈ Backend Layer (FastAPI)"]
        MW["Middleware Stack<br/>(CORS Β· JWT Auth Β· Rate Limiter Β· Logger)"]
        API["REST API Router<br/>/api/v1"]
        SVC["Service Layer"]
        INT["Integrations"]
    end

    subgraph DATA["πŸ—„οΈ Data Layer"]
        PG[("PostgreSQL<br/>(via SQLAlchemy)")]
        FS["File Storage<br/>(Uploads Dir / Supabase)"]
        ALB["Alembic<br/>Migrations"]
    end

    subgraph LLM["πŸ€– LLM Providers"]
        MISTRAL["Mistral AI"]
        OPENAI["OpenAI"]
        ANTHROPIC["Anthropic Claude"]
        GEMINI["Google Gemini"]
        GROQ["Groq"]
    end

    subgraph EXT["🌐 External APIs Under Test"]
        TARGET["Target REST APIs<br/>(user-specified base URL)"]
    end

    UI -->|"HTTPS / REST (JWT)"| MW
    MW --> API
    API --> SVC
    SVC --> INT
    SVC --> PG
    SVC --> FS
    INT -->|"LangChain"| MISTRAL
    INT -->|"LangChain"| OPENAI
    INT -->|"LangChain"| ANTHROPIC
    INT -->|"LangChain"| GEMINI
    INT -->|"LangChain"| GROQ
    SVC -->|"HTTP Requests"| TARGET
    ALB -.->|"schema migrations"| PG

2. Frontend Architecture (React SPA)

graph TD
    subgraph ENTRY["Entry Point"]
        MAIN["main.jsx"]
        APP["App.jsx<br/>(routing + layout)"]
    end

    subgraph PAGES["Pages"]
        SPECS["SpecsPage<br/>Upload & manage API specs"]
        EXEC["Execution<br/>(workflow/WorkflowPage)"]
        HIST["HistoryPage<br/>Test run history"]
        CHAT["ChatAssistant<br/>AI assistant chat"]
        MEDIA["MediaHubPage<br/>Assets & media"]
        COLL["CollectionsPage<br/>Asset collections"]
        USAGE["UsagePage<br/>LLM token usage"]
        SETTINGS["SettingsPage<br/>LLM provider config"]
        PROFILE["ProfilePage<br/>User profile"]
        LOGIN["LoginPage<br/>JWT auth"]
    end

    subgraph SHARED["Shared Components"]
        CTX["AppContext.jsx<br/>(React Context β€” global state)"]
        TOAST["Toast.jsx"]
        ERR["ErrorBoundary.jsx"]
    end

    subgraph SVCL["API Service Layer"]
        AXIO["services/<br/>Axios calls to /api/v1"]
    end

    MAIN --> APP
    APP --> CTX
    APP --> PAGES
    PAGES --> SVCL
    SVCL -->|"REST + JWT Bearer"| BACKEND
    PAGES --> SHARED

3. Backend API Routes

graph LR
    subgraph ROUTER["FastAPI Router  (/api/v1)"]
        AUTH["/auth<br/>πŸ”“ Public<br/>Login Β· Register"]
        SPEC["/specs<br/>πŸ” Protected<br/>Upload Postman / Swagger"]
        SESS["/sessions<br/>πŸ” Protected<br/>CRUD test sessions"]
        EXEC["/execution<br/>πŸ” Protected<br/>Plan Β· Run Β· Analyze steps"]
        HIST["/history<br/>πŸ” Protected<br/>Execution history"]
        CHAT["/chat<br/>πŸ” Protected<br/>AI assistant messages"]
        ASSET["/assets<br/>πŸ” Protected<br/>File uploads (media/docs)"]
        USE["/usage<br/>πŸ” Protected<br/>LLM token usage stats"]
        SET["/settings<br/>πŸ” Protected<br/>LLM provider & model config"]
    end

    AUTH -->|"issues JWT"| CLIENT
    SPEC --> postman_parser & swagger_parser
    EXEC --> llm_planner & api_executor & response_analyzer
    CHAT --> chat_assistant
    ASSET --> asset_service
    USE --> usage_service
    SET --> llm_factory["LLM Factory<br/>(provider registry)"]

4. Service Layer Internals

graph TD
    subgraph SERVICES["backend/services/"]
        PP["postman_parser.py<br/>Parse Postman collections"]
        SP["swagger_parser.py<br/>Parse OpenAPI / Swagger specs"]
        LP["llm_planner.py<br/>Generate AI execution plans"]
        AE["api_executor.py<br/>Execute HTTP steps"]
        RA["response_analyzer.py<br/>Analyze API responses with LLM"]
        CA["chat_assistant.py<br/>AI chat with context"]
        AS["asset_service.py<br/>Manage file assets"]
        CM["context_manager.py<br/>Track session variables"]
        ST["storage.py<br/>File I/O & Supabase storage"]
        US["usage_service.py<br/>Track LLM token consumption"]
    end

    PP -->|parsed endpoints| LP
    SP -->|parsed endpoints| LP
    LP -->|execution plan JSON| AE
    AE -->|request + response| RA
    RA -->|analysis JSON| DB[(PostgreSQL)]
    CA -->|chat history| DB
    CM -->|context vars| AE
    AS --> ST
    US --> DB
    LP --> LLM_INT["LLM Integration Layer"]
    RA --> LLM_INT
    CA --> LLM_INT

5. LLM Integration Layer

graph TD
    subgraph INTEGRATIONS["backend/integrations/llm/"]
        FAC["factory.py<br/>LLMClientFactory"]
        REG["registry.py<br/>Provider Registry"]
        RES["resolver.py<br/>Task→Provider resolver"]
        BASE["base.py<br/>BaseLLMClient (ABC)"]
        PROV["providers.py<br/>Concrete Clients"]
    end

    subgraph PROVIDERS["LLM Provider Clients"]
        MC["MistralClient"]
        OC["OpenAIClient"]
        AC["AnthropicClient"]
        GC["GeminiClient"]
        GRC["GroqClient"]
    end

    subgraph TASK_CONFIG["DB: llm_task_config"]
        T1["plan_generation β†’ provider + model"]
        T2["step_execution β†’ provider + model"]
        T3["response_analysis β†’ provider + model"]
        T4["chat β†’ provider + model"]
    end

    FAC --> REG
    REG --> RES
    RES -->|reads| TASK_CONFIG
    FAC --> BASE
    BASE --> PROV
    PROV --> MC & OC & AC & GC & GRC
    MC -->|"LangChain"| MistralAI_API["Mistral AI API"]
    OC -->|"LangChain"| OpenAI_API["OpenAI API"]
    AC -->|"LangChain"| Anthropic_API["Anthropic API"]
    GC -->|"LangChain"| Gemini_API["Google Gemini API"]
    GRC -->|"LangChain"| Groq_API["Groq API"]

6. Database Schema (ERD)

erDiagram
    spec_documents {
        string id PK
        string name
        text raw_content
        json parsed_json
        string base_url
        json login_credentials
        json execution_plan
        text scenario
        string file_path
        datetime uploaded_at
    }

    sessions {
        string id PK
        string user_id
        string spec_id FK
        string spec_name
        string base_url
        text scenario
        json context_variables
        datetime created_at
    }

    execution_history {
        int id PK
        string session_id
        int step
        json request_json
        json response_json
        json analysis_json
        json step_definition_json
        datetime timestamp
    }

    collections {
        string id PK
        string name
        text description
        string spec_id FK
        datetime created_at
    }

    assets {
        string id PK
        string name
        string file_type
        string file_path
        int file_size
        string category
        text description
        json tags
        json metadata_json
        string spec_id FK
        datetime uploaded_at
    }

    asset_collections {
        string asset_id FK
        string collection_id FK
    }

    chat_sessions {
        string id PK
        string user_id
        string name
        string spec_id FK
        string session_id
        datetime created_at
        datetime updated_at
    }

    chat_messages {
        int id PK
        string chat_session_id FK
        string role
        text content
        json action_json
        datetime timestamp
    }

    llm_usage {
        int id PK
        string session_id
        string user_id
        string call_type
        int prompt_tokens
        int completion_tokens
        int total_tokens
        datetime timestamp
    }

    llm_providers {
        string provider_id PK
        json api_key_enc
        boolean is_active
    }

    llm_task_config {
        string task_name PK
        string provider_id FK
        string model_name
        float temperature
    }

    spec_documents ||--o{ sessions : "has"
    spec_documents ||--o{ assets : "has"
    spec_documents ||--o{ collections : "has"
    spec_documents ||--o{ chat_sessions : "linked to"
    assets }o--o{ collections : "asset_collections"
    chat_sessions ||--o{ chat_messages : "contains"
    llm_providers ||--o{ llm_task_config : "configures"

7. Authentication & Security Flow

sequenceDiagram
    participant Browser as πŸ–₯️ Browser (React)
    participant API as βš™οΈ FastAPI
    participant JWT as πŸ”‘ JWT Middleware
    participant DB as πŸ—„οΈ PostgreSQL
    participant Enc as πŸ”’ Fernet Encryption

    Browser->>API: POST /api/v1/auth/login
    API->>DB: validate credentials
    DB-->>API: user record
    API-->>Browser: JWT token (HS256)

    Browser->>API: GET /api/v1/specs (Bearer token)
    API->>JWT: extract_user_state middleware
    JWT->>JWT: decode JWT (HS256 + JWT_SECRET)
    JWT-->>API: request.state.user = user_id
    API->>API: verify_token dependency
    API->>DB: fetch protected data
    DB-->>API: result
    API-->>Browser: JSON response

    Note over API,Enc: LLM API keys stored<br/>encrypted with Fernet<br/>(ENCRYPTION_KEY from .env)
    API->>Enc: decrypt api_key_enc on use
    Enc-->>API: plaintext API key

8. API Test Execution Flow

sequenceDiagram
    participant User as πŸ‘€ User
    participant FE as πŸ–₯️ Frontend
    participant BE as βš™οΈ Backend
    participant LLM as πŸ€– LLM Provider
    participant Target as 🌐 Target API

    User->>FE: Upload Postman/Swagger spec
    FE->>BE: POST /api/v1/specs
    BE->>BE: postman_parser / swagger_parser
    BE-->>FE: spec_id + parsed endpoints

    User->>FE: Set base URL + scenario
    FE->>BE: POST /api/v1/sessions
    BE-->>FE: session_id

    User->>FE: Generate execution plan
    FE->>BE: POST /api/v1/execution/plan
    BE->>LLM: llm_planner (prompt with endpoints + scenario)
    LLM-->>BE: structured execution plan (JSON)
    BE-->>FE: plan steps

    loop For each step
        User->>FE: Run step
        FE->>BE: POST /api/v1/execution/step
        BE->>BE: context_manager (resolve variables)
        BE->>Target: HTTP request (with auth headers)
        Target-->>BE: HTTP response
        BE->>LLM: response_analyzer (analyze result)
        LLM-->>BE: analysis + pass/fail verdict
        BE->>DB: save execution_history
        BE-->>FE: step result + analysis
    end

9. Technology Stack Summary

Layer Technology
Frontend React 18, Vite, Vanilla CSS
Backend FastAPI (Python), Uvicorn
Database PostgreSQL (SQLAlchemy ORM, Alembic migrations)
File Storage Local filesystem / Supabase Storage
Authentication JWT (HS256), Fernet encryption for secrets
LLM Integration LangChain (Mistral, OpenAI, Anthropic, Gemini, Groq)
Rate Limiting SlowAPI
Deployment Procfile (Heroku-style), runtime.txt