Spaces:
Runtime error
Refactor backend: split routes, add services & tests
Browse filesRestructure the backend into modular API route files and core services. Added new FastAPI route modules (assets, chat, execution, history, sessions, specs) and supporting core modules (config, logging, errors, url_guard) plus LLM integration client and many service updates (api_executor, chat_assistant, llm_planner, response_analyzer). Implemented SSRF protection, rate limiting, spec upload/parse, execution planning and analysis, media hub asset handling, session/context endpoints, and exportable history. Added tests, pytest.ini, updated requirements and .gitignore, removed .env.example, and added documentation/plans/screenshots and upload placeholders. Overall prepares the project for clearer separation of concerns, improved security, and test coverage.
- .claude/settings.local.json +13 -0
- .env.example +0 -27
- .gitignore +40 -13
- frontend-react/src/assets/hero.png β Current UI/Screenshot 2026-05-01 135942.png +2 -2
- Current UI/Screenshot 2026-05-01 140023.png +3 -0
- Current UI/Screenshot 2026-05-01 140040.png +3 -0
- Current UI/Screenshot 2026-05-01 140058.png +3 -0
- Current UI/Screenshot 2026-05-01 140111.png +3 -0
- Current UI/Screenshot 2026-05-01 140124.png +3 -0
- Current UI/Screenshot 2026-05-01 140140.png +3 -0
- Old Chats claude/1.md +2 -0
- Plans/plan 1.md +231 -0
- Plans/plan 2.md +161 -0
- backend/app/__init__.py +0 -0
- backend/app/api/__init__.py +0 -0
- backend/app/api/routes/__init__.py +0 -0
- backend/app/api/routes/assets.py +109 -0
- backend/app/api/routes/chat.py +183 -0
- backend/app/api/routes/execution.py +343 -0
- backend/app/api/routes/history.py +93 -0
- backend/app/api/routes/sessions.py +45 -0
- backend/app/api/routes/specs.py +163 -0
- backend/core/__init__.py +0 -0
- backend/core/config.py +19 -0
- backend/core/errors.py +19 -0
- backend/core/logging.py +15 -0
- backend/core/url_guard.py +50 -0
- backend/integrations/__init__.py +0 -0
- backend/integrations/llm/__init__.py +0 -0
- backend/integrations/llm/client.py +34 -0
- backend/main.py +43 -913
- backend/services/api_executor.py +4 -17
- backend/services/chat_assistant.py +5 -7
- backend/services/llm_planner.py +13 -17
- backend/services/response_analyzer.py +11 -16
- backend/tests/__init__.py +0 -0
- backend/tests/conftest.py +44 -0
- backend/tests/test_api_executor.py +78 -0
- backend/tests/test_llm_planner.py +71 -0
- backend/tests/test_routes.py +140 -0
- backend/tests/test_url_guard.py +75 -0
- backend/uploads/audio/.gitkeep +0 -0
- backend/uploads/documents/.gitkeep +0 -0
- backend/uploads/images/.gitkeep +0 -0
- backend/uploads/specs/.gitkeep +0 -0
- backend/uploads/video/.gitkeep +0 -0
- PROJECT_DOCUMENTATION.md β documents/PROJECT_DOCUMENTATION.md +101 -51
- Project_Diagrams.md β documents/Project_Diagrams.md +79 -96
- README.md β documents/README.md +15 -5
- Research_Paper.md β documents/Research_Paper.md +7 -7
|
@@ -0,0 +1,13 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"permissions": {
|
| 3 |
+
"allow": [
|
| 4 |
+
"Bash(pip install *)",
|
| 5 |
+
"Bash(python -c \"from backend.main import app; print\\('OK β routes:', len\\(app.routes\\)\\)\")",
|
| 6 |
+
"Bash(python -c ' *)",
|
| 7 |
+
"Bash(python -c \"from backend.main import app; from backend.core.url_guard import assert_safe_url; print\\('OK routes:', len\\(app.routes\\)\\)\")",
|
| 8 |
+
"Bash(python -c \"from backend.main import app; from backend.core.errors import AppError; print\\('OK routes:', len\\(app.routes\\)\\)\")",
|
| 9 |
+
"Bash(Get-ChildItem -Path \"C:\\\\Users\\\\amitk\\\\OneDrive\\\\Desktop\\\\Testing\" -Recurse -Directory)",
|
| 10 |
+
"Bash(Select-Object -ExpandProperty FullName)"
|
| 11 |
+
]
|
| 12 |
+
}
|
| 13 |
+
}
|
|
@@ -1,27 +0,0 @@
|
|
| 1 |
-
# AI API Testing Tool - Environment Variables
|
| 2 |
-
|
| 3 |
-
# Mistral AI configuration
|
| 4 |
-
MISTRAL_API_KEY=your_mistral_api_key_here
|
| 5 |
-
MISTRAL_MODEL=mistral-large-latest
|
| 6 |
-
|
| 7 |
-
# Backend configuration (Local default: 8001)
|
| 8 |
-
# On Render, the PORT will be assigned automatically.
|
| 9 |
-
PORT=8001
|
| 10 |
-
|
| 11 |
-
# Database Configuration (Supabase PostgreSQL)
|
| 12 |
-
# Get your Connection String from Supabase -> Project Settings -> Database
|
| 13 |
-
DATABASE_URL=postgresql://postgres.your-project:your-password@aws-0-region.pooler.supabase.com:5432/postgres
|
| 14 |
-
|
| 15 |
-
# Supabase Storage & Client Configuration
|
| 16 |
-
# Get these from Supabase -> Project Settings -> API
|
| 17 |
-
SUPABASE_URL=https://your-project.supabase.co
|
| 18 |
-
# Use the Service Role Key for backend storage access
|
| 19 |
-
SUPABASE_SERVICE_ROLE_KEY=your-service-role-key-here
|
| 20 |
-
SUPABASE_STORAGE_BUCKET=assets
|
| 21 |
-
|
| 22 |
-
# Frontend Configuration (Vite)
|
| 23 |
-
# Points to the Backend URL.
|
| 24 |
-
# Local: http://localhost:8001
|
| 25 |
-
# Render: https://your-backend.onrender.com
|
| 26 |
-
VITE_BACKEND_URL=http://localhost:8001
|
| 27 |
-
VITE_API_URL=http://localhost:8001
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
@@ -1,36 +1,63 @@
|
|
| 1 |
-
#
|
| 2 |
node_modules/
|
| 3 |
dist/
|
| 4 |
build/
|
|
|
|
| 5 |
|
| 6 |
-
# Python
|
| 7 |
__pycache__/
|
| 8 |
-
*.
|
| 9 |
-
*.pyo
|
| 10 |
*.pyd
|
| 11 |
-
.
|
|
|
|
|
|
|
| 12 |
env/
|
| 13 |
venv/
|
|
|
|
| 14 |
pip-log.txt
|
| 15 |
-
pip-delete-this-directory.txt
|
| 16 |
|
| 17 |
-
#
|
| 18 |
-
|
| 19 |
-
|
|
|
|
|
|
|
| 20 |
|
| 21 |
-
# Environment
|
| 22 |
.env
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 23 |
|
| 24 |
-
#
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 25 |
*.log
|
|
|
|
| 26 |
|
| 27 |
-
# OS
|
| 28 |
.DS_Store
|
| 29 |
Thumbs.db
|
| 30 |
desktop.ini
|
|
|
|
| 31 |
|
| 32 |
-
# IDEs
|
| 33 |
.vscode/
|
| 34 |
.idea/
|
| 35 |
*.swp
|
| 36 |
*.swo
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# ββ Node βββββββββββββββββββββββββββββββββββββββββ
|
| 2 |
node_modules/
|
| 3 |
dist/
|
| 4 |
build/
|
| 5 |
+
.npm/
|
| 6 |
|
| 7 |
+
# ββ Python βββββββββββββββββββββββββββββββββββββββ
|
| 8 |
__pycache__/
|
| 9 |
+
*.py[cod]
|
|
|
|
| 10 |
*.pyd
|
| 11 |
+
*.pyo
|
| 12 |
+
*.egg-info/
|
| 13 |
+
.eggs/
|
| 14 |
env/
|
| 15 |
venv/
|
| 16 |
+
.venv/
|
| 17 |
pip-log.txt
|
|
|
|
| 18 |
|
| 19 |
+
# ββ Testing ββββββββββββββββββββββββββββββββββββββ
|
| 20 |
+
.pytest_cache/
|
| 21 |
+
htmlcov/
|
| 22 |
+
.coverage
|
| 23 |
+
coverage.xml
|
| 24 |
|
| 25 |
+
# ββ Environment & secrets βββββββββββββββββββββββββ
|
| 26 |
.env
|
| 27 |
+
.env.*
|
| 28 |
+
!.env.example
|
| 29 |
+
|
| 30 |
+
# ββ Uploaded user files βββββββββββββββββββββββββββ
|
| 31 |
+
backend/uploads/audio/*
|
| 32 |
+
backend/uploads/video/*
|
| 33 |
+
backend/uploads/documents/*
|
| 34 |
+
backend/uploads/images/*
|
| 35 |
+
backend/uploads/specs/*
|
| 36 |
+
!backend/uploads/**/.gitkeep
|
| 37 |
|
| 38 |
+
# ββ Archives & temp artifacts βββββββββββββββββββββ
|
| 39 |
+
*.zip
|
| 40 |
+
*.tar.gz
|
| 41 |
+
*.rar
|
| 42 |
+
temp_review/
|
| 43 |
+
tmp_*/
|
| 44 |
+
|
| 45 |
+
# ββ Logs βββββββββββββββββββββββββββββββββββββββββ
|
| 46 |
*.log
|
| 47 |
+
logs/
|
| 48 |
|
| 49 |
+
# ββ OS βββββββββββββββββββββββββββββββββββββββββββ
|
| 50 |
.DS_Store
|
| 51 |
Thumbs.db
|
| 52 |
desktop.ini
|
| 53 |
+
ehthumbs.db
|
| 54 |
|
| 55 |
+
# ββ IDEs βββββββββββββββββββββββββββββββββββββββββ
|
| 56 |
.vscode/
|
| 57 |
.idea/
|
| 58 |
*.swp
|
| 59 |
*.swo
|
| 60 |
+
|
| 61 |
+
# ββ Databases ββββββββββββββββββββββββββββββββββββ
|
| 62 |
+
*.db
|
| 63 |
+
*.sqlite3
|
|
File without changes
|
|
Git LFS Details
|
|
Git LFS Details
|
|
Git LFS Details
|
|
Git LFS Details
|
|
Git LFS Details
|
|
Git LFS Details
|
|
@@ -0,0 +1,2 @@
|
|
|
|
|
|
|
|
|
|
| 1 |
+
claude --resume b12b89ce-94d3-48fc-b8f8-b73ba5ddee29
|
| 2 |
+
claude --resume abc2b770-ce67-4c01-8f04-0e1e84cb120d , remaining
|
|
@@ -0,0 +1,231 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# Backend Refactor Execution Plan (Actionable)
|
| 2 |
+
|
| 3 |
+
## Phase 0 β Preparation
|
| 4 |
+
|
| 5 |
+
* Create new branch: `refactor/architecture-v1`
|
| 6 |
+
* Freeze new feature development
|
| 7 |
+
* Add `.env` and `core/config.py`
|
| 8 |
+
|
| 9 |
+
---
|
| 10 |
+
|
| 11 |
+
## Phase 1 β Restructure (No Logic Changes)
|
| 12 |
+
|
| 13 |
+
### Create Directories
|
| 14 |
+
|
| 15 |
+
```
|
| 16 |
+
backend/app/api/routes/
|
| 17 |
+
backend/core/
|
| 18 |
+
backend/domain/models/
|
| 19 |
+
backend/services/
|
| 20 |
+
backend/integrations/llm/
|
| 21 |
+
backend/integrations/http/
|
| 22 |
+
backend/integrations/parsers/
|
| 23 |
+
backend/repositories/
|
| 24 |
+
backend/workers/
|
| 25 |
+
backend/schemas/
|
| 26 |
+
backend/utils/
|
| 27 |
+
```
|
| 28 |
+
|
| 29 |
+
### Move Files
|
| 30 |
+
|
| 31 |
+
* `main.py` β `app/main.py`
|
| 32 |
+
* routes β `app/api/routes/`
|
| 33 |
+
* shared utils β `utils/`
|
| 34 |
+
* existing services β `services/`
|
| 35 |
+
|
| 36 |
+
### Update Imports
|
| 37 |
+
|
| 38 |
+
* Replace relative imports with absolute:
|
| 39 |
+
|
| 40 |
+
```
|
| 41 |
+
from backend.services...
|
| 42 |
+
```
|
| 43 |
+
|
| 44 |
+
---
|
| 45 |
+
|
| 46 |
+
## Phase 2 β Config + Core Systems
|
| 47 |
+
|
| 48 |
+
### Create `core/config.py`
|
| 49 |
+
|
| 50 |
+
* Use `BaseSettings`
|
| 51 |
+
* Add:
|
| 52 |
+
|
| 53 |
+
* `ALLOWED_ORIGINS`
|
| 54 |
+
* `API_KEYS`
|
| 55 |
+
* `UPLOAD_DIR`
|
| 56 |
+
* `TIMEOUT`
|
| 57 |
+
|
| 58 |
+
### Replace Hardcoded Values
|
| 59 |
+
|
| 60 |
+
* Replace all constants with config usage
|
| 61 |
+
|
| 62 |
+
### Add Logging
|
| 63 |
+
|
| 64 |
+
* Create `core/logging.py`
|
| 65 |
+
* Replace all `print()` with logger
|
| 66 |
+
|
| 67 |
+
---
|
| 68 |
+
|
| 69 |
+
## Phase 3 β LLM Isolation
|
| 70 |
+
|
| 71 |
+
### Move LLM Logic
|
| 72 |
+
|
| 73 |
+
FROM:
|
| 74 |
+
|
| 75 |
+
* `services/llm_planner.py`
|
| 76 |
+
* `services/chat_assistant.py`
|
| 77 |
+
|
| 78 |
+
TO:
|
| 79 |
+
|
| 80 |
+
```
|
| 81 |
+
integrations/llm/
|
| 82 |
+
client.py
|
| 83 |
+
planner.py
|
| 84 |
+
analyzer.py
|
| 85 |
+
```
|
| 86 |
+
|
| 87 |
+
### Enforce Rule
|
| 88 |
+
|
| 89 |
+
* No direct OpenAI calls outside `client.py`
|
| 90 |
+
|
| 91 |
+
---
|
| 92 |
+
|
| 93 |
+
## Phase 4 β Execution Isolation
|
| 94 |
+
|
| 95 |
+
### Move Executor
|
| 96 |
+
|
| 97 |
+
* `api_executor.py` β `integrations/http/sandbox.py`
|
| 98 |
+
|
| 99 |
+
### Add Controls
|
| 100 |
+
|
| 101 |
+
* allowlist domains
|
| 102 |
+
* enforce timeout
|
| 103 |
+
* restrict HTTP methods
|
| 104 |
+
|
| 105 |
+
---
|
| 106 |
+
|
| 107 |
+
## Phase 5 β Validation Layer
|
| 108 |
+
|
| 109 |
+
### Create
|
| 110 |
+
|
| 111 |
+
`services/validation_service.py`
|
| 112 |
+
|
| 113 |
+
### Add Checks
|
| 114 |
+
|
| 115 |
+
* validate LLM JSON schema
|
| 116 |
+
* reject unsafe URLs
|
| 117 |
+
* sanitize inputs
|
| 118 |
+
|
| 119 |
+
### Integrate
|
| 120 |
+
|
| 121 |
+
* execution_service MUST call validation before execution
|
| 122 |
+
|
| 123 |
+
---
|
| 124 |
+
|
| 125 |
+
## Phase 6 β Split Responsibilities
|
| 126 |
+
|
| 127 |
+
### Create
|
| 128 |
+
|
| 129 |
+
* `planning_service.py`
|
| 130 |
+
* `execution_service.py`
|
| 131 |
+
|
| 132 |
+
### Flow
|
| 133 |
+
|
| 134 |
+
```
|
| 135 |
+
route β planning_service β validation_service β execution_service
|
| 136 |
+
```
|
| 137 |
+
|
| 138 |
+
---
|
| 139 |
+
|
| 140 |
+
## Phase 7 β Remove Global State
|
| 141 |
+
|
| 142 |
+
### Delete
|
| 143 |
+
|
| 144 |
+
* `get_context`
|
| 145 |
+
* `set_context_variables`
|
| 146 |
+
|
| 147 |
+
### Replace With
|
| 148 |
+
|
| 149 |
+
* request-scoped data OR
|
| 150 |
+
* `domain/models/user_session.py`
|
| 151 |
+
* `repositories/session_repo.py`
|
| 152 |
+
|
| 153 |
+
---
|
| 154 |
+
|
| 155 |
+
## Phase 8 β Background Jobs
|
| 156 |
+
|
| 157 |
+
### Create
|
| 158 |
+
|
| 159 |
+
`workers/tasks.py`
|
| 160 |
+
|
| 161 |
+
### Move Heavy Tasks
|
| 162 |
+
|
| 163 |
+
* LLM calls
|
| 164 |
+
* API chains
|
| 165 |
+
|
| 166 |
+
### Use
|
| 167 |
+
|
| 168 |
+
* FastAPI BackgroundTasks (initial)
|
| 169 |
+
|
| 170 |
+
---
|
| 171 |
+
|
| 172 |
+
## Phase 9 β Security Layer
|
| 173 |
+
|
| 174 |
+
### Create
|
| 175 |
+
|
| 176 |
+
`core/security.py`
|
| 177 |
+
|
| 178 |
+
### Add
|
| 179 |
+
|
| 180 |
+
* JWT auth
|
| 181 |
+
* rate limiting middleware
|
| 182 |
+
|
| 183 |
+
### Apply
|
| 184 |
+
|
| 185 |
+
* protect all routes except health
|
| 186 |
+
|
| 187 |
+
---
|
| 188 |
+
|
| 189 |
+
## Phase 10 β Frontend Alignment
|
| 190 |
+
|
| 191 |
+
### Create
|
| 192 |
+
|
| 193 |
+
```
|
| 194 |
+
src/api/
|
| 195 |
+
src/hooks/
|
| 196 |
+
src/store/
|
| 197 |
+
```
|
| 198 |
+
|
| 199 |
+
### Add
|
| 200 |
+
|
| 201 |
+
* API client wrapper
|
| 202 |
+
* React Query
|
| 203 |
+
|
| 204 |
+
---
|
| 205 |
+
|
| 206 |
+
## Phase 11 β Cleanup
|
| 207 |
+
|
| 208 |
+
* Remove unused files
|
| 209 |
+
* Remove duplicate logic
|
| 210 |
+
* Remove debug prints
|
| 211 |
+
|
| 212 |
+
---
|
| 213 |
+
|
| 214 |
+
## Phase 12 β Testing
|
| 215 |
+
|
| 216 |
+
* Add unit tests for:
|
| 217 |
+
|
| 218 |
+
* validation_service
|
| 219 |
+
* execution_service
|
| 220 |
+
* Add integration tests for routes
|
| 221 |
+
|
| 222 |
+
---
|
| 223 |
+
|
| 224 |
+
## Final Architecture Rule
|
| 225 |
+
|
| 226 |
+
* routes = HTTP only
|
| 227 |
+
* services = orchestration only
|
| 228 |
+
* integrations = external calls only
|
| 229 |
+
* domain = pure logic only
|
| 230 |
+
|
| 231 |
+
NO cross-layer violations allowed.
|
|
@@ -0,0 +1,161 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# Project Completion Plan β AI API Testing Assistant
|
| 2 |
+
|
| 3 |
+
**Goal:** Make the project fully functional, secure, and production-ready.
|
| 4 |
+
**Based on:** Actual code audit of the current codebase.
|
| 5 |
+
|
| 6 |
+
---
|
| 7 |
+
|
| 8 |
+
## β
Phase 1 β Fix Broken Foundations β DONE
|
| 9 |
+
|
| 10 |
+
- [x] Fix `requirements.txt`: removed `streamlit`, `websockets`; added `prance`, `pyyaml`, `pydantic-settings`, `slowapi`, `cryptography`; upgraded LangChain + Mistral packages; fixed `psycopg2-binary>=2.9.10` (Python 3.13 wheel), `sqlalchemy>=2.0.36`, gunicorn platform guard
|
| 11 |
+
- [x] Fix CORS: replaced `allow_origins=["*"]` with `settings.allowed_origins_list` from env var
|
| 12 |
+
- [x] Fix silent failures: removed bare `except: pass` blocks; all errors logged or returned as HTTP
|
| 13 |
+
- [x] Clean `App.jsx`: deleted ~800 lines of commented-out dead code at top; active code only
|
| 14 |
+
- [x] Fix `chat_assistant.py`: replaced `HumanMessage` with `AIMessage` for assistant replies
|
| 15 |
+
- [x] Add `.gitignore`: covers `*.zip`, `*.tar.gz`, `.env.*`, `__pycache__`, `.venv`, uploads, etc.
|
| 16 |
+
|
| 17 |
+
---
|
| 18 |
+
|
| 19 |
+
## β
Phase 2 β Backend Restructure β DONE
|
| 20 |
+
|
| 21 |
+
- [x] Created `backend/core/config.py` β `BaseSettings` with all env vars
|
| 22 |
+
- [x] Created `backend/core/logging.py` β `get_logger()` structured logger
|
| 23 |
+
- [x] Created `backend/core/errors.py` β `AppError` + `app_error_handler`
|
| 24 |
+
- [x] Split `main.py` into 6 focused route files (no logic changes):
|
| 25 |
+
- `backend/app/api/routes/specs.py` β `/specs`, `/upload-spec`, `/credentials`
|
| 26 |
+
- `backend/app/api/routes/sessions.py` β `/session`, `/context`
|
| 27 |
+
- `backend/app/api/routes/execution.py` β `/plan`, `/execute-step`, `/analyze-response`
|
| 28 |
+
- `backend/app/api/routes/assets.py` β `/assets`, `/collections`
|
| 29 |
+
- `backend/app/api/routes/chat.py` β `/chat`, `/chat-sessions`
|
| 30 |
+
- `backend/app/api/routes/history.py` β `/history`
|
| 31 |
+
- [x] `main.py` reduced to ~60 lines (app setup only)
|
| 32 |
+
- [x] All 36 routes preserved throughout
|
| 33 |
+
|
| 34 |
+
---
|
| 35 |
+
|
| 36 |
+
## β
Phase 3 β Security & Validation β DONE
|
| 37 |
+
|
| 38 |
+
- [x] URL sandboxing (`backend/core/url_guard.py`): blocks localhost, private IP ranges (10.x, 172.16-31.x, 192.168.x, 169.254.x, ::1, fc00::/7), non-HTTP schemes; resolves hostname to IP before check
|
| 39 |
+
- [x] `assert_safe_url()` called in `/execute-step` before every outbound HTTP request
|
| 40 |
+
- [x] Spec file size validation: 5MB limit in `/upload-spec`
|
| 41 |
+
- [x] Rate limiting via `slowapi`: `/plan` 10/min, `/execute-step` 30/min, `/chat` 20/min
|
| 42 |
+
|
| 43 |
+
---
|
| 44 |
+
|
| 45 |
+
## β
Phase 4 β Error Handling β DONE
|
| 46 |
+
|
| 47 |
+
- [x] Standardized `AppError` exception with `{ "error", "code", "detail" }` shape
|
| 48 |
+
- [x] LLM retry: one automatic retry on JSON parse failure in both `llm_planner.py` and `response_analyzer.py`
|
| 49 |
+
- [x] Execution error context: `TimeoutException β 504`, `ConnectError β 502`, 4xx/5xx responses logged with URL + body snippet
|
| 50 |
+
|
| 51 |
+
---
|
| 52 |
+
|
| 53 |
+
## β
Phase 5 β Complete Half-Built Features β DONE
|
| 54 |
+
|
| 55 |
+
- [x] OTP modal UI: inline modal shown when `is_otp_required` detected; saves to session context
|
| 56 |
+
- [x] Selection modal UI: inline modal shown when `is_selection_required` detected; saves chosen value
|
| 57 |
+
- [x] Token usage display: shown per-step and as running total in the top bar
|
| 58 |
+
- [x] **Run All** button: runs all execution plan steps sequentially; stops if OTP/selection prompt appears
|
| 59 |
+
- [x] **Retry** button: amber button appears next to failed steps (status = 'failure')
|
| 60 |
+
- [x] **Export session** (`GET /history/{session_id}/export`): returns full JSON report with pass/fail counts; forces browser download
|
| 61 |
+
- [x] `HistoryService.exportSession()` in `api.js`
|
| 62 |
+
- [x] Export button in `HistoryPage.jsx` (Download icon, between Delete and Import)
|
| 63 |
+
|
| 64 |
+
---
|
| 65 |
+
|
| 66 |
+
## π² Phase 6 β LLM Isolation
|
| 67 |
+
|
| 68 |
+
Consolidate all LLM calls into one place.
|
| 69 |
+
|
| 70 |
+
### Create `backend/integrations/llm/client.py`
|
| 71 |
+
```python
|
| 72 |
+
class MistralClient:
|
| 73 |
+
def complete(self, prompt, temperature=0, timeout=60) -> str: ...
|
| 74 |
+
def structured(self, prompt, schema) -> dict: ... # validates JSON output
|
| 75 |
+
```
|
| 76 |
+
|
| 77 |
+
### Migrate all callers:
|
| 78 |
+
- `llm_planner.py` β uses `MistralClient`
|
| 79 |
+
- `api_executor.py` β uses `MistralClient`
|
| 80 |
+
- `response_analyzer.py` β uses `MistralClient`
|
| 81 |
+
- `chat_assistant.py` β uses `MistralClient`
|
| 82 |
+
|
| 83 |
+
**Rule:** No direct `ChatMistralAI` instantiation outside `client.py`.
|
| 84 |
+
|
| 85 |
+
---
|
| 86 |
+
|
| 87 |
+
## π² Phase 7 β Frontend Polish
|
| 88 |
+
|
| 89 |
+
### 7.1 Error UI
|
| 90 |
+
- Replace `alert()` calls with toast notifications or inline error messages
|
| 91 |
+
- Add error boundary component to catch React crashes
|
| 92 |
+
- Show which step failed and why
|
| 93 |
+
|
| 94 |
+
### 7.2 API Client Improvements (`services/api.js`)
|
| 95 |
+
- Add global axios error interceptor
|
| 96 |
+
- Centralize timeout and base URL
|
| 97 |
+
|
| 98 |
+
### 7.3 Frontend Structure (optional)
|
| 99 |
+
```
|
| 100 |
+
src/
|
| 101 |
+
βββ api/ # Move api.js here, split by domain
|
| 102 |
+
βββ hooks/ # Custom hooks (useSpec, usePlan, useExecution)
|
| 103 |
+
βββ store/ # AppContext split into smaller contexts
|
| 104 |
+
```
|
| 105 |
+
|
| 106 |
+
---
|
| 107 |
+
|
| 108 |
+
## π² Phase 8 β Testing
|
| 109 |
+
|
| 110 |
+
### Unit Tests
|
| 111 |
+
- `services/url_validator.py` β URL blocking, private IP detection
|
| 112 |
+
- `services/llm_planner.py` β mock LLM, test output parsing
|
| 113 |
+
- `services/api_executor.py` β test request building, context resolution
|
| 114 |
+
|
| 115 |
+
### Integration Tests
|
| 116 |
+
- `POST /upload-spec` β valid OpenAPI file β correct parse
|
| 117 |
+
- `POST /plan` β returns valid step list
|
| 118 |
+
- `POST /execute-step` β mock HTTP target β correct history entry
|
| 119 |
+
- `POST /chat` β returns valid action JSON
|
| 120 |
+
|
| 121 |
+
### Tools
|
| 122 |
+
- `pytest` + `pytest-asyncio`
|
| 123 |
+
- `httpx.AsyncClient` for route testing
|
| 124 |
+
- `respx` for mocking outbound HTTP
|
| 125 |
+
|
| 126 |
+
---
|
| 127 |
+
|
| 128 |
+
## π² Phase 9 β Deployment
|
| 129 |
+
|
| 130 |
+
### 9.1 Docker
|
| 131 |
+
```
|
| 132 |
+
Dockerfile # backend
|
| 133 |
+
docker-compose.yml # backend + postgres (local dev)
|
| 134 |
+
```
|
| 135 |
+
|
| 136 |
+
### 9.2 Environment Separation
|
| 137 |
+
- `.env.development` β local DB, debug logging
|
| 138 |
+
- `.env.production` β Supabase, structured JSON logs, strict CORS
|
| 139 |
+
|
| 140 |
+
### 9.3 Health Check
|
| 141 |
+
- `GET /health` returns `{ "status": "ok", "db": "ok", "llm": "ok" }`
|
| 142 |
+
|
| 143 |
+
---
|
| 144 |
+
|
| 145 |
+
## Progress Summary
|
| 146 |
+
|
| 147 |
+
| Phase | Status | Notes |
|
| 148 |
+
|-------|--------|-------|
|
| 149 |
+
| 1 β Fix Foundations | β
Done | |
|
| 150 |
+
| 2 β Backend Restructure | β
Done | 36 routes across 6 files |
|
| 151 |
+
| 3 β Security | β
Done | SSRF protection, rate limiting, file size validation |
|
| 152 |
+
| 4 β Error Handling | β
Done | LLM retry, typed HTTP errors |
|
| 153 |
+
| 5 β Complete Features | β
Done | Run All, Retry, Export, OTP/Selection modals, token display |
|
| 154 |
+
| 6 β LLM Isolation | π² Pending | |
|
| 155 |
+
| 7 β Frontend Polish | π² Pending | Replace alerts with toasts, error boundary |
|
| 156 |
+
| 8 β Testing | π² Pending | |
|
| 157 |
+
| 9 β Deployment | π² Pending | Docker + health check |
|
| 158 |
+
|
| 159 |
+
|
| 160 |
+
|
| 161 |
+
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
@@ -0,0 +1,109 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
import json
|
| 2 |
+
from typing import Optional
|
| 3 |
+
|
| 4 |
+
from fastapi import APIRouter, Depends, File, Form, HTTPException, UploadFile
|
| 5 |
+
from sqlalchemy import or_
|
| 6 |
+
from sqlalchemy.orm import Session
|
| 7 |
+
|
| 8 |
+
from backend import database
|
| 9 |
+
from backend.services.asset_service import AssetService
|
| 10 |
+
|
| 11 |
+
router = APIRouter()
|
| 12 |
+
|
| 13 |
+
|
| 14 |
+
@router.post("/assets")
|
| 15 |
+
async def upload_asset(
|
| 16 |
+
spec_id: str = Form(None),
|
| 17 |
+
collection_id: str = Form(None),
|
| 18 |
+
description: str = Form(None),
|
| 19 |
+
tags: str = Form("[]"),
|
| 20 |
+
file: UploadFile = File(...),
|
| 21 |
+
db: Session = Depends(database.get_db),
|
| 22 |
+
):
|
| 23 |
+
try:
|
| 24 |
+
tag_list = json.loads(tags)
|
| 25 |
+
asset = AssetService.upload_asset(db, file, spec_id, collection_id, description, tag_list)
|
| 26 |
+
return {
|
| 27 |
+
"id": asset.id,
|
| 28 |
+
"name": asset.name,
|
| 29 |
+
"type": asset.file_type,
|
| 30 |
+
"size": asset.file_size,
|
| 31 |
+
"category": asset.category,
|
| 32 |
+
"description": asset.description,
|
| 33 |
+
"tags": asset.tags,
|
| 34 |
+
"spec_id": asset.spec_id,
|
| 35 |
+
"uploaded_at": asset.uploaded_at.isoformat(),
|
| 36 |
+
}
|
| 37 |
+
except Exception as e:
|
| 38 |
+
raise HTTPException(status_code=500, detail=f"Upload failed: {str(e)}")
|
| 39 |
+
|
| 40 |
+
|
| 41 |
+
@router.get("/assets")
|
| 42 |
+
def list_assets(
|
| 43 |
+
spec_id: Optional[str] = None,
|
| 44 |
+
category: Optional[str] = None,
|
| 45 |
+
db: Session = Depends(database.get_db),
|
| 46 |
+
):
|
| 47 |
+
assets = AssetService.get_assets(db, spec_id, category)
|
| 48 |
+
return [
|
| 49 |
+
{
|
| 50 |
+
"id": a.id,
|
| 51 |
+
"name": a.name,
|
| 52 |
+
"type": a.file_type,
|
| 53 |
+
"size": a.file_size,
|
| 54 |
+
"category": a.category,
|
| 55 |
+
"description": a.description,
|
| 56 |
+
"tags": a.tags,
|
| 57 |
+
"spec_id": a.spec_id,
|
| 58 |
+
"path": a.file_path,
|
| 59 |
+
"uploaded_at": a.uploaded_at.isoformat(),
|
| 60 |
+
"collection_ids": [c.id for c in a.collections],
|
| 61 |
+
}
|
| 62 |
+
for a in assets
|
| 63 |
+
]
|
| 64 |
+
|
| 65 |
+
|
| 66 |
+
@router.delete("/assets/{asset_id}")
|
| 67 |
+
def delete_asset(asset_id: str, db: Session = Depends(database.get_db)):
|
| 68 |
+
success = AssetService.delete_asset(db, asset_id)
|
| 69 |
+
if not success:
|
| 70 |
+
raise HTTPException(status_code=404, detail="Asset not found")
|
| 71 |
+
return {"status": "deleted", "id": asset_id}
|
| 72 |
+
|
| 73 |
+
|
| 74 |
+
@router.get("/collections")
|
| 75 |
+
def list_collections(spec_id: Optional[str] = None, db: Session = Depends(database.get_db)):
|
| 76 |
+
query = db.query(database.Collection)
|
| 77 |
+
if spec_id:
|
| 78 |
+
query = query.filter(
|
| 79 |
+
or_(database.Collection.spec_id == spec_id, database.Collection.spec_id == None)
|
| 80 |
+
)
|
| 81 |
+
cols = query.all()
|
| 82 |
+
return [{"id": c.id, "name": c.name, "description": c.description, "spec_id": c.spec_id} for c in cols]
|
| 83 |
+
|
| 84 |
+
|
| 85 |
+
@router.post("/collections")
|
| 86 |
+
def create_collection(
|
| 87 |
+
name: str = Form(...),
|
| 88 |
+
spec_id: str = Form(None),
|
| 89 |
+
description: str = Form(None),
|
| 90 |
+
db: Session = Depends(database.get_db),
|
| 91 |
+
):
|
| 92 |
+
col = AssetService.create_collection(db, name, spec_id, description)
|
| 93 |
+
return {"id": col.id, "name": col.name, "spec_id": col.spec_id}
|
| 94 |
+
|
| 95 |
+
|
| 96 |
+
@router.post("/collections/{collection_id}/assets/{asset_id}")
|
| 97 |
+
def add_asset_to_collection(collection_id: str, asset_id: str, db: Session = Depends(database.get_db)):
|
| 98 |
+
success = AssetService.add_to_collection(db, asset_id, collection_id)
|
| 99 |
+
if not success:
|
| 100 |
+
raise HTTPException(status_code=404, detail="Asset or Collection not found")
|
| 101 |
+
return {"status": "success"}
|
| 102 |
+
|
| 103 |
+
|
| 104 |
+
@router.delete("/collections/{collection_id}")
|
| 105 |
+
def delete_collection(collection_id: str, db: Session = Depends(database.get_db)):
|
| 106 |
+
success = AssetService.delete_collection(db, collection_id)
|
| 107 |
+
if not success:
|
| 108 |
+
raise HTTPException(status_code=404, detail="Collection not found")
|
| 109 |
+
return {"status": "success"}
|
|
@@ -0,0 +1,183 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
import datetime
|
| 2 |
+
import uuid
|
| 3 |
+
from typing import Optional
|
| 4 |
+
|
| 5 |
+
from fastapi import APIRouter, Depends, HTTPException, Request
|
| 6 |
+
from slowapi import Limiter
|
| 7 |
+
from slowapi.util import get_remote_address
|
| 8 |
+
from sqlalchemy.orm import Session
|
| 9 |
+
|
| 10 |
+
from backend import database, schemas
|
| 11 |
+
from backend.services.chat_assistant import process_message as chat_process_message
|
| 12 |
+
|
| 13 |
+
router = APIRouter()
|
| 14 |
+
limiter = Limiter(key_func=get_remote_address)
|
| 15 |
+
|
| 16 |
+
|
| 17 |
+
@router.post("/chat", response_model=schemas.ChatResponse)
|
| 18 |
+
@limiter.limit("20/minute")
|
| 19 |
+
def chat_endpoint(request: Request, body: schemas.ChatRequest, db: Session = Depends(database.get_db)):
|
| 20 |
+
try:
|
| 21 |
+
chat_session = None
|
| 22 |
+
if body.chat_session_id:
|
| 23 |
+
chat_session = db.query(database.ChatSession).filter(
|
| 24 |
+
database.ChatSession.id == body.chat_session_id
|
| 25 |
+
).first()
|
| 26 |
+
|
| 27 |
+
if not chat_session:
|
| 28 |
+
session_name = body.message[:60].strip()
|
| 29 |
+
if len(body.message) > 60:
|
| 30 |
+
session_name += "..."
|
| 31 |
+
chat_session = database.ChatSession(
|
| 32 |
+
id=str(uuid.uuid4()),
|
| 33 |
+
name=session_name,
|
| 34 |
+
spec_id=body.spec_id,
|
| 35 |
+
session_id=body.session_id,
|
| 36 |
+
)
|
| 37 |
+
db.add(chat_session)
|
| 38 |
+
db.flush()
|
| 39 |
+
|
| 40 |
+
user_msg = database.ChatMessage(
|
| 41 |
+
chat_session_id=chat_session.id,
|
| 42 |
+
role="user",
|
| 43 |
+
content=body.message,
|
| 44 |
+
)
|
| 45 |
+
db.add(user_msg)
|
| 46 |
+
db.flush()
|
| 47 |
+
|
| 48 |
+
history_rows = (
|
| 49 |
+
db.query(database.ChatMessage)
|
| 50 |
+
.filter(database.ChatMessage.chat_session_id == chat_session.id)
|
| 51 |
+
.order_by(database.ChatMessage.id.asc())
|
| 52 |
+
.all()
|
| 53 |
+
)
|
| 54 |
+
chat_history = [
|
| 55 |
+
{"role": m.role, "content": m.content}
|
| 56 |
+
for m in history_rows[:-1]
|
| 57 |
+
]
|
| 58 |
+
|
| 59 |
+
ai_result = chat_process_message(
|
| 60 |
+
message=body.message,
|
| 61 |
+
spec_name=body.spec_name,
|
| 62 |
+
execution_plan=body.execution_plan or [],
|
| 63 |
+
context_variables=body.context_variables or {},
|
| 64 |
+
last_step_result=body.last_step_result,
|
| 65 |
+
all_step_results=body.all_step_results,
|
| 66 |
+
chat_history=chat_history,
|
| 67 |
+
)
|
| 68 |
+
|
| 69 |
+
assistant_content = ai_result.get("message", "")
|
| 70 |
+
db.add(database.ChatMessage(
|
| 71 |
+
chat_session_id=chat_session.id,
|
| 72 |
+
role="assistant",
|
| 73 |
+
content=assistant_content,
|
| 74 |
+
action_json=ai_result,
|
| 75 |
+
))
|
| 76 |
+
|
| 77 |
+
chat_session.updated_at = datetime.datetime.utcnow()
|
| 78 |
+
db.commit()
|
| 79 |
+
|
| 80 |
+
return schemas.ChatResponse(
|
| 81 |
+
chat_session_id=chat_session.id,
|
| 82 |
+
chat_session_name=chat_session.name,
|
| 83 |
+
action=ai_result.get("action", "reply"),
|
| 84 |
+
message=assistant_content,
|
| 85 |
+
step=ai_result.get("step"),
|
| 86 |
+
changes=ai_result.get("changes"),
|
| 87 |
+
assertions=ai_result.get("assertions"),
|
| 88 |
+
explanation=ai_result.get("explanation"),
|
| 89 |
+
)
|
| 90 |
+
except Exception as e:
|
| 91 |
+
import traceback
|
| 92 |
+
traceback.print_exc()
|
| 93 |
+
raise HTTPException(status_code=500, detail=f"Chat failed: {str(e)}")
|
| 94 |
+
|
| 95 |
+
|
| 96 |
+
@router.get("/chat-sessions")
|
| 97 |
+
def list_chat_sessions(spec_id: Optional[str] = None, db: Session = Depends(database.get_db)):
|
| 98 |
+
query = db.query(database.ChatSession)
|
| 99 |
+
if spec_id:
|
| 100 |
+
query = query.filter(database.ChatSession.spec_id == spec_id)
|
| 101 |
+
sessions = query.order_by(database.ChatSession.updated_at.desc()).all()
|
| 102 |
+
|
| 103 |
+
return [
|
| 104 |
+
{
|
| 105 |
+
"id": s.id,
|
| 106 |
+
"name": s.name,
|
| 107 |
+
"spec_id": s.spec_id,
|
| 108 |
+
"session_id": s.session_id,
|
| 109 |
+
"message_count": db.query(database.ChatMessage)
|
| 110 |
+
.filter(database.ChatMessage.chat_session_id == s.id)
|
| 111 |
+
.count(),
|
| 112 |
+
"created_at": s.created_at.isoformat() if s.created_at else None,
|
| 113 |
+
"updated_at": s.updated_at.isoformat() if s.updated_at else None,
|
| 114 |
+
}
|
| 115 |
+
for s in sessions
|
| 116 |
+
]
|
| 117 |
+
|
| 118 |
+
|
| 119 |
+
@router.get("/chat-sessions/{chat_session_id}")
|
| 120 |
+
def get_chat_session(chat_session_id: str, db: Session = Depends(database.get_db)):
|
| 121 |
+
session = db.query(database.ChatSession).filter(
|
| 122 |
+
database.ChatSession.id == chat_session_id
|
| 123 |
+
).first()
|
| 124 |
+
if not session:
|
| 125 |
+
raise HTTPException(status_code=404, detail="Chat session not found")
|
| 126 |
+
|
| 127 |
+
messages = (
|
| 128 |
+
db.query(database.ChatMessage)
|
| 129 |
+
.filter(database.ChatMessage.chat_session_id == chat_session_id)
|
| 130 |
+
.order_by(database.ChatMessage.id.asc())
|
| 131 |
+
.all()
|
| 132 |
+
)
|
| 133 |
+
return {
|
| 134 |
+
"id": session.id,
|
| 135 |
+
"name": session.name,
|
| 136 |
+
"spec_id": session.spec_id,
|
| 137 |
+
"session_id": session.session_id,
|
| 138 |
+
"created_at": session.created_at.isoformat() if session.created_at else None,
|
| 139 |
+
"updated_at": session.updated_at.isoformat() if session.updated_at else None,
|
| 140 |
+
"messages": [
|
| 141 |
+
{
|
| 142 |
+
"id": m.id,
|
| 143 |
+
"role": m.role,
|
| 144 |
+
"content": m.content,
|
| 145 |
+
"action_json": m.action_json,
|
| 146 |
+
"timestamp": m.timestamp.isoformat() if m.timestamp else None,
|
| 147 |
+
}
|
| 148 |
+
for m in messages
|
| 149 |
+
],
|
| 150 |
+
}
|
| 151 |
+
|
| 152 |
+
|
| 153 |
+
@router.patch("/chat-sessions/{chat_session_id}")
|
| 154 |
+
def rename_chat_session(chat_session_id: str, payload: dict, db: Session = Depends(database.get_db)):
|
| 155 |
+
session = db.query(database.ChatSession).filter(
|
| 156 |
+
database.ChatSession.id == chat_session_id
|
| 157 |
+
).first()
|
| 158 |
+
if not session:
|
| 159 |
+
raise HTTPException(status_code=404, detail="Chat session not found")
|
| 160 |
+
if "name" in payload:
|
| 161 |
+
session.name = payload["name"]
|
| 162 |
+
db.commit()
|
| 163 |
+
return {"id": session.id, "name": session.name}
|
| 164 |
+
|
| 165 |
+
|
| 166 |
+
@router.delete("/chat-sessions/{chat_session_id}")
|
| 167 |
+
def delete_chat_session(chat_session_id: str, db: Session = Depends(database.get_db)):
|
| 168 |
+
session = db.query(database.ChatSession).filter(
|
| 169 |
+
database.ChatSession.id == chat_session_id
|
| 170 |
+
).first()
|
| 171 |
+
if not session:
|
| 172 |
+
raise HTTPException(status_code=404, detail="Chat session not found")
|
| 173 |
+
db.delete(session)
|
| 174 |
+
db.commit()
|
| 175 |
+
return {"status": "deleted", "id": chat_session_id}
|
| 176 |
+
|
| 177 |
+
|
| 178 |
+
@router.delete("/chat-sessions")
|
| 179 |
+
def clear_all_chat_sessions(db: Session = Depends(database.get_db)):
|
| 180 |
+
db.query(database.ChatMessage).delete()
|
| 181 |
+
db.query(database.ChatSession).delete()
|
| 182 |
+
db.commit()
|
| 183 |
+
return {"status": "cleared"}
|
|
@@ -0,0 +1,343 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
import json
|
| 2 |
+
import os
|
| 3 |
+
import re
|
| 4 |
+
import traceback
|
| 5 |
+
from urllib.parse import parse_qsl, urlparse
|
| 6 |
+
|
| 7 |
+
import httpx
|
| 8 |
+
from fastapi import APIRouter, Depends, HTTPException, Request
|
| 9 |
+
from slowapi import Limiter
|
| 10 |
+
from slowapi.util import get_remote_address
|
| 11 |
+
from sqlalchemy.orm import Session
|
| 12 |
+
from sqlalchemy.orm.attributes import flag_modified
|
| 13 |
+
|
| 14 |
+
from backend import database, schemas
|
| 15 |
+
from backend.core.logging import get_logger
|
| 16 |
+
from backend.core.url_guard import assert_safe_url
|
| 17 |
+
from backend.services.api_executor import prepare_step_request
|
| 18 |
+
from backend.services.asset_service import AssetService, storage
|
| 19 |
+
from backend.services.context_manager import set_context_variables
|
| 20 |
+
from backend.services.llm_planner import generate_execution_plan
|
| 21 |
+
from backend.services.response_analyzer import analyze_api_response
|
| 22 |
+
|
| 23 |
+
router = APIRouter()
|
| 24 |
+
logger = get_logger(__name__)
|
| 25 |
+
limiter = Limiter(key_func=get_remote_address)
|
| 26 |
+
|
| 27 |
+
PLACEHOLDER_HOSTS = {"api.example.com", "example.com", "<base_url>", "your-api.com"}
|
| 28 |
+
|
| 29 |
+
|
| 30 |
+
def _resolve_refs(obj, full_spec, _resolving=None):
|
| 31 |
+
if _resolving is None:
|
| 32 |
+
_resolving = set()
|
| 33 |
+
if isinstance(obj, dict):
|
| 34 |
+
if "$ref" in obj:
|
| 35 |
+
ref = obj["$ref"]
|
| 36 |
+
if ref in _resolving:
|
| 37 |
+
return {"$ref_loop": ref}
|
| 38 |
+
_resolving = _resolving | {ref}
|
| 39 |
+
parts = ref.lstrip("#/").split("/")
|
| 40 |
+
target = full_spec
|
| 41 |
+
for part in parts:
|
| 42 |
+
target = target.get(part, {}) if isinstance(target, dict) else {}
|
| 43 |
+
return _resolve_refs(target, full_spec, _resolving)
|
| 44 |
+
return {k: _resolve_refs(v, full_spec, _resolving) for k, v in obj.items()}
|
| 45 |
+
elif isinstance(obj, list):
|
| 46 |
+
return [_resolve_refs(item, full_spec, _resolving) for item in obj]
|
| 47 |
+
return obj
|
| 48 |
+
|
| 49 |
+
|
| 50 |
+
def _shorthand_spec(obj):
|
| 51 |
+
if isinstance(obj, dict):
|
| 52 |
+
if "type" in obj:
|
| 53 |
+
t = obj["type"]
|
| 54 |
+
fmt = f":{obj['format']}" if "format" in obj else ""
|
| 55 |
+
enum_vals = f"[{'|'.join(map(str, obj['enum']))}]" if "enum" in obj else ""
|
| 56 |
+
if t == "object" and "properties" in obj:
|
| 57 |
+
req_list = obj.get("required", [])
|
| 58 |
+
props = [
|
| 59 |
+
f"{k}{'*' if k in req_list or v.get('nullable') == False else ''}({_shorthand_spec(v)})"
|
| 60 |
+
for k, v in obj["properties"].items()
|
| 61 |
+
]
|
| 62 |
+
return "{" + ",".join(props) + "}"
|
| 63 |
+
if t == "array" and "items" in obj:
|
| 64 |
+
return f"[{_shorthand_spec(obj['items'])}]"
|
| 65 |
+
return f"{t}{fmt}{enum_vals}"
|
| 66 |
+
return {k: _shorthand_spec(v) for k, v in obj.items()}
|
| 67 |
+
return str(obj)
|
| 68 |
+
|
| 69 |
+
|
| 70 |
+
def _prune_spec(obj):
|
| 71 |
+
DROP_KEYS = {"description", "summary", "example", "examples", "format", "title", "default", "enum_descriptions"}
|
| 72 |
+
if isinstance(obj, dict):
|
| 73 |
+
return {k: _prune_spec(v) for k, v in obj.items() if k not in DROP_KEYS}
|
| 74 |
+
elif isinstance(obj, list):
|
| 75 |
+
return [_prune_spec(item) for item in obj]
|
| 76 |
+
return obj
|
| 77 |
+
|
| 78 |
+
|
| 79 |
+
def _match_path(path_template: str, actual_path: str) -> bool:
|
| 80 |
+
pattern = re.sub(r'\{[^}]+\}', r'[^/]+', path_template)
|
| 81 |
+
pattern = "^" + pattern.rstrip("/") + "/?$"
|
| 82 |
+
return bool(re.match(pattern, actual_path.split("?")[0]))
|
| 83 |
+
|
| 84 |
+
|
| 85 |
+
def _is_placeholder(url: str) -> bool:
|
| 86 |
+
try:
|
| 87 |
+
host = urlparse(url).hostname or ""
|
| 88 |
+
return host.lower() in PLACEHOLDER_HOSTS or url.startswith("<BASE_URL>")
|
| 89 |
+
except Exception:
|
| 90 |
+
return False
|
| 91 |
+
|
| 92 |
+
|
| 93 |
+
@router.post("/plan", response_model=schemas.ExecutionPlan)
|
| 94 |
+
@limiter.limit("10/minute")
|
| 95 |
+
def generate_plan(request: Request, body: schemas.PlanRequest, db: Session = Depends(database.get_db)):
|
| 96 |
+
try:
|
| 97 |
+
parsed_spec = json.loads(body.spec_content) if isinstance(body.spec_content, str) else body.spec_content
|
| 98 |
+
|
| 99 |
+
credentials = body.user_credentials or {}
|
| 100 |
+
if body.spec_id:
|
| 101 |
+
spec = db.query(database.SpecDocument).filter(database.SpecDocument.id == body.spec_id).first()
|
| 102 |
+
if spec and spec.login_credentials:
|
| 103 |
+
credentials = spec.login_credentials
|
| 104 |
+
|
| 105 |
+
plan = generate_execution_plan(parsed_spec, body.base_url, body.scenario, credentials)
|
| 106 |
+
|
| 107 |
+
if body.spec_id:
|
| 108 |
+
spec = db.query(database.SpecDocument).filter(database.SpecDocument.id == body.spec_id).first()
|
| 109 |
+
if spec:
|
| 110 |
+
spec.execution_plan = plan.model_dump()["execution_plan"]
|
| 111 |
+
spec.scenario = body.scenario
|
| 112 |
+
flag_modified(spec, "execution_plan")
|
| 113 |
+
flag_modified(spec, "scenario")
|
| 114 |
+
|
| 115 |
+
if body.session_id:
|
| 116 |
+
session = db.query(database.SessionModel).filter(
|
| 117 |
+
database.SessionModel.id == body.session_id
|
| 118 |
+
).first()
|
| 119 |
+
if session:
|
| 120 |
+
session.spec_id = spec.id
|
| 121 |
+
session.spec_name = spec.name
|
| 122 |
+
session.scenario = body.scenario
|
| 123 |
+
session.base_url = body.base_url
|
| 124 |
+
|
| 125 |
+
if plan.suggested_credentials and isinstance(plan.suggested_credentials, dict):
|
| 126 |
+
current_creds = dict(spec.login_credentials or {})
|
| 127 |
+
if any(k not in current_creds or current_creds[k] != v for k, v in plan.suggested_credentials.items()):
|
| 128 |
+
current_creds.update(plan.suggested_credentials)
|
| 129 |
+
spec.login_credentials = current_creds
|
| 130 |
+
flag_modified(spec, "login_credentials")
|
| 131 |
+
logger.info("AI suggested credentials merged for spec '%s'", spec.name)
|
| 132 |
+
|
| 133 |
+
db.commit()
|
| 134 |
+
|
| 135 |
+
return plan
|
| 136 |
+
except Exception as e:
|
| 137 |
+
raise HTTPException(status_code=500, detail=f"Planning failed: {str(e)}")
|
| 138 |
+
|
| 139 |
+
|
| 140 |
+
@router.post("/execute-step")
|
| 141 |
+
@limiter.limit("30/minute")
|
| 142 |
+
async def execute_step(request: Request, body: schemas.ExecuteStepRequest, db: Session = Depends(database.get_db)):
|
| 143 |
+
try:
|
| 144 |
+
endpoint_str = body.step_definition.get("endpoint", "")
|
| 145 |
+
actual_endpoint_spec = {"endpoint": endpoint_str}
|
| 146 |
+
|
| 147 |
+
if body.spec_id:
|
| 148 |
+
spec_doc = db.query(database.SpecDocument).filter(
|
| 149 |
+
database.SpecDocument.id == body.spec_id
|
| 150 |
+
).first()
|
| 151 |
+
if spec_doc and spec_doc.parsed_json:
|
| 152 |
+
full_spec = spec_doc.parsed_json
|
| 153 |
+
paths = full_spec.get("paths", {})
|
| 154 |
+
parts = endpoint_str.split(" ", 1)
|
| 155 |
+
path_only = parts[1] if len(parts) == 2 else parts[0]
|
| 156 |
+
|
| 157 |
+
matched_path = path_only if path_only in paths else next(
|
| 158 |
+
(p for p in paths if _match_path(p, path_only)), None
|
| 159 |
+
)
|
| 160 |
+
|
| 161 |
+
if matched_path:
|
| 162 |
+
raw_spec = {matched_path: paths[matched_path]}
|
| 163 |
+
resolved = _resolve_refs(raw_spec, full_spec)
|
| 164 |
+
path_data = resolved.get(matched_path, {})
|
| 165 |
+
for method in path_data:
|
| 166 |
+
if isinstance(path_data[method], dict):
|
| 167 |
+
path_data[method].pop("responses", None)
|
| 168 |
+
actual_endpoint_spec = _shorthand_spec(_prune_spec(resolved))
|
| 169 |
+
else:
|
| 170 |
+
logger.warning("Path match failed for %s, falling back", path_only)
|
| 171 |
+
actual_endpoint_spec = {"endpoint": endpoint_str, "warning": "Full schema match failed"}
|
| 172 |
+
|
| 173 |
+
rich_assets = AssetService.get_ai_context(db, body.spec_id)
|
| 174 |
+
|
| 175 |
+
effective_credentials = body.user_credentials
|
| 176 |
+
if not effective_credentials and body.spec_id:
|
| 177 |
+
spec = db.query(database.SpecDocument).filter(
|
| 178 |
+
database.SpecDocument.id == body.spec_id
|
| 179 |
+
).first()
|
| 180 |
+
if spec and spec.login_credentials:
|
| 181 |
+
effective_credentials = spec.login_credentials
|
| 182 |
+
logger.info("Using stored credentials for spec '%s'", spec.name)
|
| 183 |
+
|
| 184 |
+
req_plan, executor_token_usage = prepare_step_request(
|
| 185 |
+
endpoint_spec=actual_endpoint_spec,
|
| 186 |
+
base_url=body.base_url,
|
| 187 |
+
scenario="",
|
| 188 |
+
step_definition=body.step_definition,
|
| 189 |
+
context_variables=body.context_variables,
|
| 190 |
+
user_credentials=effective_credentials,
|
| 191 |
+
available_assets=rich_assets,
|
| 192 |
+
)
|
| 193 |
+
|
| 194 |
+
url = req_plan.request.url.strip()
|
| 195 |
+
if url.startswith("/"):
|
| 196 |
+
url = body.base_url.rstrip("/") + url
|
| 197 |
+
elif _is_placeholder(url):
|
| 198 |
+
parsed = urlparse(url)
|
| 199 |
+
path = parsed.path or "/"
|
| 200 |
+
url = body.base_url.rstrip("/") + path
|
| 201 |
+
if parsed.query:
|
| 202 |
+
url += "?" + parsed.query
|
| 203 |
+
logger.info("Placeholder URL replaced: %s", url)
|
| 204 |
+
elif not url.startswith("http"):
|
| 205 |
+
url = f"https://{url}"
|
| 206 |
+
|
| 207 |
+
parsed = urlparse(url)
|
| 208 |
+
if parsed.query:
|
| 209 |
+
url_keys = {k for k, _ in parse_qsl(parsed.query)}
|
| 210 |
+
for k in [k for k in req_plan.request.query_params if k in url_keys]:
|
| 211 |
+
del req_plan.request.query_params[k]
|
| 212 |
+
|
| 213 |
+
# Block SSRF β reject private/internal IPs before executing
|
| 214 |
+
assert_safe_url(url)
|
| 215 |
+
|
| 216 |
+
async with httpx.AsyncClient(timeout=120.0) as client:
|
| 217 |
+
http_req = client.build_request(
|
| 218 |
+
method=req_plan.request.method.strip().upper(),
|
| 219 |
+
url=url,
|
| 220 |
+
headers=req_plan.request.headers,
|
| 221 |
+
params=req_plan.request.query_params,
|
| 222 |
+
data=req_plan.request.form_data if req_plan.request.form_data else None,
|
| 223 |
+
json=req_plan.request.body if req_plan.request.body and not req_plan.request.form_data else None,
|
| 224 |
+
)
|
| 225 |
+
|
| 226 |
+
files_payload = {}
|
| 227 |
+
opened_files = []
|
| 228 |
+
try:
|
| 229 |
+
if req_plan.request.files:
|
| 230 |
+
req_plan.request.headers = {
|
| 231 |
+
k: v for k, v in req_plan.request.headers.items() if k.lower() != "content-type"
|
| 232 |
+
}
|
| 233 |
+
for field_name, file_name in req_plan.request.files.items():
|
| 234 |
+
asset = db.query(database.Asset).filter(database.Asset.name == file_name).first()
|
| 235 |
+
if asset:
|
| 236 |
+
full_path = storage.get_full_path(asset.file_path)
|
| 237 |
+
if os.path.exists(full_path):
|
| 238 |
+
f = open(full_path, "rb")
|
| 239 |
+
opened_files.append(f)
|
| 240 |
+
files_payload[field_name] = (asset.name, f, asset.file_type)
|
| 241 |
+
else:
|
| 242 |
+
logger.warning("Asset file not found on disk: %s", full_path)
|
| 243 |
+
else:
|
| 244 |
+
logger.warning("Asset not found in Media Hub: %s", file_name)
|
| 245 |
+
|
| 246 |
+
logger.info("Executing %s %s", http_req.method, http_req.url)
|
| 247 |
+
|
| 248 |
+
try:
|
| 249 |
+
if files_payload:
|
| 250 |
+
response = await client.request(
|
| 251 |
+
method=req_plan.request.method.strip().upper(),
|
| 252 |
+
url=url,
|
| 253 |
+
headers=req_plan.request.headers,
|
| 254 |
+
params=req_plan.request.query_params,
|
| 255 |
+
data=req_plan.request.form_data if req_plan.request.form_data else None,
|
| 256 |
+
files=files_payload,
|
| 257 |
+
)
|
| 258 |
+
else:
|
| 259 |
+
response = await client.send(http_req)
|
| 260 |
+
except httpx.TimeoutException:
|
| 261 |
+
raise HTTPException(
|
| 262 |
+
status_code=504,
|
| 263 |
+
detail=f"Request to {url} timed out after {120}s. Check the API server is reachable.",
|
| 264 |
+
)
|
| 265 |
+
except httpx.ConnectError as e:
|
| 266 |
+
raise HTTPException(
|
| 267 |
+
status_code=502,
|
| 268 |
+
detail=f"Could not connect to {url}: {str(e)}",
|
| 269 |
+
)
|
| 270 |
+
except Exception as e:
|
| 271 |
+
raise HTTPException(status_code=500, detail=f"Request to {url} failed: {str(e)}")
|
| 272 |
+
finally:
|
| 273 |
+
for f in opened_files:
|
| 274 |
+
f.close()
|
| 275 |
+
|
| 276 |
+
body_text = response.text
|
| 277 |
+
resp_data = {
|
| 278 |
+
"status_code": response.status_code,
|
| 279 |
+
"body": body_text,
|
| 280 |
+
"headers": dict(response.headers),
|
| 281 |
+
}
|
| 282 |
+
|
| 283 |
+
# Log a clear error line for 4xx/5xx so it's visible in the backend console
|
| 284 |
+
if response.status_code >= 400:
|
| 285 |
+
snippet = body_text[:300].replace("\n", " ")
|
| 286 |
+
logger.warning(
|
| 287 |
+
"Step %s returned %s from %s: %s",
|
| 288 |
+
req_plan.step, response.status_code, url, snippet,
|
| 289 |
+
)
|
| 290 |
+
|
| 291 |
+
req_plan.request.url = url
|
| 292 |
+
|
| 293 |
+
db.add(database.ExecutionHistory(
|
| 294 |
+
session_id=body.session_id,
|
| 295 |
+
step=req_plan.step,
|
| 296 |
+
request_json=req_plan.request.model_dump(),
|
| 297 |
+
response_json=resp_data,
|
| 298 |
+
step_definition_json=body.step_definition,
|
| 299 |
+
))
|
| 300 |
+
db.commit()
|
| 301 |
+
|
| 302 |
+
return {
|
| 303 |
+
"executed_request": req_plan.request.model_dump(),
|
| 304 |
+
"response": resp_data,
|
| 305 |
+
"analysis_needed": req_plan.step,
|
| 306 |
+
"token_usage": executor_token_usage,
|
| 307 |
+
}
|
| 308 |
+
except Exception as e:
|
| 309 |
+
logger.error("Execution error: %s", traceback.format_exc())
|
| 310 |
+
raise HTTPException(status_code=500, detail=f"Execution failed: {str(e)}")
|
| 311 |
+
|
| 312 |
+
|
| 313 |
+
@router.post("/analyze-response", response_model=schemas.AnalyzeResponse)
|
| 314 |
+
def analyze_response_endpoint(request: schemas.AnalyzeRequest, db: Session = Depends(database.get_db)):
|
| 315 |
+
try:
|
| 316 |
+
step_def = request.step_definition or {}
|
| 317 |
+
analysis = analyze_api_response(
|
| 318 |
+
step=request.step,
|
| 319 |
+
endpoint=step_def.get("endpoint", "Unknown"),
|
| 320 |
+
purpose=step_def.get("purpose", "Unknown"),
|
| 321 |
+
status_code=request.response_data.get("status_code", 0),
|
| 322 |
+
response_body=request.response_data.get("body", ""),
|
| 323 |
+
)
|
| 324 |
+
|
| 325 |
+
if analysis.extracted_variables:
|
| 326 |
+
set_context_variables(db, request.session_id, analysis.extracted_variables)
|
| 327 |
+
|
| 328 |
+
history = (
|
| 329 |
+
db.query(database.ExecutionHistory)
|
| 330 |
+
.filter(
|
| 331 |
+
database.ExecutionHistory.session_id == request.session_id,
|
| 332 |
+
database.ExecutionHistory.step == request.step,
|
| 333 |
+
)
|
| 334 |
+
.order_by(database.ExecutionHistory.id.desc())
|
| 335 |
+
.first()
|
| 336 |
+
)
|
| 337 |
+
if history:
|
| 338 |
+
history.analysis_json = analysis.model_dump()
|
| 339 |
+
db.commit()
|
| 340 |
+
|
| 341 |
+
return analysis
|
| 342 |
+
except Exception as e:
|
| 343 |
+
raise HTTPException(status_code=500, detail=f"Analysis failed: {str(e)}")
|
|
@@ -0,0 +1,93 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
from fastapi import APIRouter, Depends, HTTPException
|
| 2 |
+
from fastapi.responses import JSONResponse
|
| 3 |
+
from sqlalchemy.orm import Session
|
| 4 |
+
|
| 5 |
+
from backend import database
|
| 6 |
+
|
| 7 |
+
router = APIRouter()
|
| 8 |
+
|
| 9 |
+
|
| 10 |
+
@router.get("/history/{session_id}/export")
|
| 11 |
+
def export_history_session(session_id: str, db: Session = Depends(database.get_db)):
|
| 12 |
+
sess = db.query(database.SessionModel).filter(database.SessionModel.id == session_id).first()
|
| 13 |
+
if not sess:
|
| 14 |
+
raise HTTPException(status_code=404, detail="Session not found")
|
| 15 |
+
|
| 16 |
+
steps = (
|
| 17 |
+
db.query(database.ExecutionHistory)
|
| 18 |
+
.filter(database.ExecutionHistory.session_id == session_id)
|
| 19 |
+
.order_by(database.ExecutionHistory.step)
|
| 20 |
+
.all()
|
| 21 |
+
)
|
| 22 |
+
|
| 23 |
+
report = {
|
| 24 |
+
"session_id": session_id,
|
| 25 |
+
"spec_name": sess.spec_name or "Unknown",
|
| 26 |
+
"scenario": sess.scenario or "",
|
| 27 |
+
"base_url": sess.base_url or "",
|
| 28 |
+
"context_variables": sess.context_variables or {},
|
| 29 |
+
"steps": [
|
| 30 |
+
{
|
| 31 |
+
"step": h.step,
|
| 32 |
+
"step_definition": h.step_definition_json,
|
| 33 |
+
"request": h.request_json,
|
| 34 |
+
"response": h.response_json,
|
| 35 |
+
"analysis": h.analysis_json,
|
| 36 |
+
"timestamp": h.timestamp.isoformat(),
|
| 37 |
+
}
|
| 38 |
+
for h in steps
|
| 39 |
+
],
|
| 40 |
+
"total_steps": len(steps),
|
| 41 |
+
"passed": sum(1 for h in steps if (h.analysis_json or {}).get("status") == "success"),
|
| 42 |
+
"failed": sum(1 for h in steps if (h.analysis_json or {}).get("status") != "success"),
|
| 43 |
+
}
|
| 44 |
+
|
| 45 |
+
return JSONResponse(
|
| 46 |
+
content=report,
|
| 47 |
+
headers={"Content-Disposition": f'attachment; filename="session_{session_id[:8]}.json"'},
|
| 48 |
+
)
|
| 49 |
+
|
| 50 |
+
|
| 51 |
+
@router.get("/history")
|
| 52 |
+
def get_all_history(db: Session = Depends(database.get_db)):
|
| 53 |
+
results = (
|
| 54 |
+
db.query(database.ExecutionHistory, database.SessionModel)
|
| 55 |
+
.join(database.SessionModel, database.ExecutionHistory.session_id == database.SessionModel.id)
|
| 56 |
+
.order_by(database.ExecutionHistory.timestamp.desc())
|
| 57 |
+
.all()
|
| 58 |
+
)
|
| 59 |
+
return [
|
| 60 |
+
{
|
| 61 |
+
"id": hist.id,
|
| 62 |
+
"session_id": hist.session_id,
|
| 63 |
+
"spec_id": sess.spec_id,
|
| 64 |
+
"spec_name": sess.spec_name or "Unknown",
|
| 65 |
+
"scenario": sess.scenario or "No scenario",
|
| 66 |
+
"context_variables": sess.context_variables or {},
|
| 67 |
+
"step": hist.step,
|
| 68 |
+
"request": hist.request_json,
|
| 69 |
+
"response": hist.response_json,
|
| 70 |
+
"analysis": hist.analysis_json,
|
| 71 |
+
"step_definition": hist.step_definition_json,
|
| 72 |
+
"timestamp": hist.timestamp.isoformat(),
|
| 73 |
+
}
|
| 74 |
+
for hist, sess in results
|
| 75 |
+
]
|
| 76 |
+
|
| 77 |
+
|
| 78 |
+
@router.delete("/history/{session_id}")
|
| 79 |
+
def delete_history_session(session_id: str, db: Session = Depends(database.get_db)):
|
| 80 |
+
db.query(database.ExecutionHistory).filter(
|
| 81 |
+
database.ExecutionHistory.session_id == session_id
|
| 82 |
+
).delete()
|
| 83 |
+
db.query(database.SessionModel).filter(database.SessionModel.id == session_id).delete()
|
| 84 |
+
db.commit()
|
| 85 |
+
return {"status": "deleted", "session_id": session_id}
|
| 86 |
+
|
| 87 |
+
|
| 88 |
+
@router.delete("/history")
|
| 89 |
+
def clear_all_history(db: Session = Depends(database.get_db)):
|
| 90 |
+
db.query(database.ExecutionHistory).delete()
|
| 91 |
+
db.query(database.SessionModel).delete()
|
| 92 |
+
db.commit()
|
| 93 |
+
return {"status": "cleared"}
|
|
@@ -0,0 +1,45 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
import uuid
|
| 2 |
+
|
| 3 |
+
from fastapi import APIRouter, Depends
|
| 4 |
+
from sqlalchemy.orm import Session
|
| 5 |
+
|
| 6 |
+
from backend import database, schemas
|
| 7 |
+
from backend.services.context_manager import get_context, set_context_variables
|
| 8 |
+
|
| 9 |
+
router = APIRouter()
|
| 10 |
+
|
| 11 |
+
|
| 12 |
+
@router.post("/session")
|
| 13 |
+
def create_session(db: Session = Depends(database.get_db)):
|
| 14 |
+
session_id = str(uuid.uuid4())
|
| 15 |
+
new_session = database.SessionModel(id=session_id)
|
| 16 |
+
db.add(new_session)
|
| 17 |
+
db.commit()
|
| 18 |
+
return {"session_id": session_id}
|
| 19 |
+
|
| 20 |
+
|
| 21 |
+
@router.delete("/session/{session_id}")
|
| 22 |
+
def clear_session(session_id: str, db: Session = Depends(database.get_db)):
|
| 23 |
+
session = db.query(database.SessionModel).filter(database.SessionModel.id == session_id).first()
|
| 24 |
+
if session:
|
| 25 |
+
db.delete(session)
|
| 26 |
+
db.query(database.ExecutionHistory).filter(
|
| 27 |
+
database.ExecutionHistory.session_id == session_id
|
| 28 |
+
).delete()
|
| 29 |
+
db.commit()
|
| 30 |
+
return {"status": "ok"}
|
| 31 |
+
|
| 32 |
+
|
| 33 |
+
@router.get("/context/{session_id}")
|
| 34 |
+
def get_context_endpoint(session_id: str, db: Session = Depends(database.get_db)):
|
| 35 |
+
return get_context(db, session_id)
|
| 36 |
+
|
| 37 |
+
|
| 38 |
+
@router.post("/context/{session_id}")
|
| 39 |
+
def update_context_endpoint(
|
| 40 |
+
session_id: str,
|
| 41 |
+
payload: schemas.ContextUpdate,
|
| 42 |
+
db: Session = Depends(database.get_db),
|
| 43 |
+
):
|
| 44 |
+
variables = {payload.key: payload.value}
|
| 45 |
+
return set_context_variables(db, session_id, variables)
|
|
@@ -0,0 +1,163 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
import json
|
| 2 |
+
import uuid
|
| 3 |
+
from typing import Optional
|
| 4 |
+
|
| 5 |
+
from fastapi import APIRouter, Depends, File, HTTPException, UploadFile
|
| 6 |
+
from sqlalchemy.orm import Session
|
| 7 |
+
from sqlalchemy.orm.attributes import flag_modified
|
| 8 |
+
|
| 9 |
+
from backend import database, schemas
|
| 10 |
+
from backend.core.logging import get_logger
|
| 11 |
+
from backend.services.swagger_parser import parse_spec
|
| 12 |
+
from backend.services.postman_parser import parse_postman_collection as parse_postman
|
| 13 |
+
from backend.services.asset_service import AssetService, storage
|
| 14 |
+
|
| 15 |
+
router = APIRouter()
|
| 16 |
+
logger = get_logger(__name__)
|
| 17 |
+
|
| 18 |
+
|
| 19 |
+
@router.post("/upload-spec")
|
| 20 |
+
async def upload_spec(file: UploadFile = File(...), db: Session = Depends(database.get_db)):
|
| 21 |
+
try:
|
| 22 |
+
content = await file.read()
|
| 23 |
+
if len(content) > 5 * 1024 * 1024:
|
| 24 |
+
raise HTTPException(status_code=400, detail="Spec file exceeds 5 MB limit.")
|
| 25 |
+
decoded_content = content.decode("utf-8", errors="replace")
|
| 26 |
+
|
| 27 |
+
is_postman = False
|
| 28 |
+
try:
|
| 29 |
+
temp_json = json.loads(decoded_content)
|
| 30 |
+
if isinstance(temp_json, dict):
|
| 31 |
+
info = temp_json.get("info", {})
|
| 32 |
+
if isinstance(info, dict) and "postman.com/json/collection" in info.get("schema", ""):
|
| 33 |
+
is_postman = True
|
| 34 |
+
elif "info" in temp_json and "item" in temp_json:
|
| 35 |
+
if "openapi" not in temp_json and "swagger" not in temp_json:
|
| 36 |
+
is_postman = True
|
| 37 |
+
except Exception:
|
| 38 |
+
pass
|
| 39 |
+
|
| 40 |
+
parsed_spec = parse_postman(decoded_content) if is_postman else parse_spec(decoded_content)
|
| 41 |
+
safe_spec = json.loads(json.dumps(parsed_spec, ensure_ascii=True))
|
| 42 |
+
|
| 43 |
+
spec_id = str(uuid.uuid4())
|
| 44 |
+
file_path = f"specs/{spec_id}_{file.filename}"
|
| 45 |
+
|
| 46 |
+
logger.info("Uploading %s to %s", file.filename, file_path)
|
| 47 |
+
await file.seek(0)
|
| 48 |
+
storage.save(file.file, file_path)
|
| 49 |
+
|
| 50 |
+
spec_doc = database.SpecDocument(
|
| 51 |
+
id=spec_id,
|
| 52 |
+
name=file.filename,
|
| 53 |
+
raw_content=decoded_content,
|
| 54 |
+
parsed_json=safe_spec,
|
| 55 |
+
file_path=file_path,
|
| 56 |
+
)
|
| 57 |
+
db.add(spec_doc)
|
| 58 |
+
db.commit()
|
| 59 |
+
|
| 60 |
+
safe_spec["id"] = spec_id
|
| 61 |
+
return {"status": "success", "spec_id": spec_id, "parsed_spec": safe_spec}
|
| 62 |
+
except Exception as e:
|
| 63 |
+
import traceback
|
| 64 |
+
traceback.print_exc()
|
| 65 |
+
raise HTTPException(status_code=400, detail=f"Failed to parse spec: {str(e)}")
|
| 66 |
+
|
| 67 |
+
|
| 68 |
+
@router.get("/specs")
|
| 69 |
+
def list_specs(db: Session = Depends(database.get_db)):
|
| 70 |
+
specs = db.query(database.SpecDocument).order_by(database.SpecDocument.uploaded_at.desc()).all()
|
| 71 |
+
return [
|
| 72 |
+
{"id": s.id, "name": s.name, "base_url": s.base_url or "", "uploaded_at": s.uploaded_at.isoformat()}
|
| 73 |
+
for s in specs
|
| 74 |
+
]
|
| 75 |
+
|
| 76 |
+
|
| 77 |
+
@router.get("/specs/{spec_id}")
|
| 78 |
+
def get_spec(spec_id: str, db: Session = Depends(database.get_db)):
|
| 79 |
+
spec = db.query(database.SpecDocument).filter(database.SpecDocument.id == spec_id).first()
|
| 80 |
+
if not spec:
|
| 81 |
+
raise HTTPException(status_code=404, detail="Spec not found")
|
| 82 |
+
return {
|
| 83 |
+
"id": spec.id,
|
| 84 |
+
"name": spec.name,
|
| 85 |
+
"base_url": spec.base_url or "",
|
| 86 |
+
"login_credentials": spec.login_credentials or {},
|
| 87 |
+
"parsed_spec": spec.parsed_json,
|
| 88 |
+
"execution_plan": spec.execution_plan,
|
| 89 |
+
"scenario": spec.scenario,
|
| 90 |
+
"uploaded_at": spec.uploaded_at.isoformat(),
|
| 91 |
+
}
|
| 92 |
+
|
| 93 |
+
|
| 94 |
+
@router.patch("/specs/{spec_id}")
|
| 95 |
+
def update_spec(spec_id: str, payload: dict, db: Session = Depends(database.get_db)):
|
| 96 |
+
spec = db.query(database.SpecDocument).filter(database.SpecDocument.id == spec_id).first()
|
| 97 |
+
if not spec:
|
| 98 |
+
raise HTTPException(status_code=404, detail="Spec not found")
|
| 99 |
+
if "base_url" in payload:
|
| 100 |
+
spec.base_url = payload["base_url"]
|
| 101 |
+
db.commit()
|
| 102 |
+
return {"status": "updated", "id": spec.id, "base_url": spec.base_url}
|
| 103 |
+
|
| 104 |
+
|
| 105 |
+
@router.delete("/specs/{spec_id}")
|
| 106 |
+
def delete_spec(spec_id: str, db: Session = Depends(database.get_db)):
|
| 107 |
+
spec = db.query(database.SpecDocument).filter(database.SpecDocument.id == spec_id).first()
|
| 108 |
+
if not spec:
|
| 109 |
+
raise HTTPException(status_code=404, detail="Spec not found")
|
| 110 |
+
|
| 111 |
+
assets = db.query(database.Asset).filter(database.Asset.spec_id == spec_id).all()
|
| 112 |
+
for asset in assets:
|
| 113 |
+
AssetService.delete_asset(db, asset.id)
|
| 114 |
+
|
| 115 |
+
if spec.file_path:
|
| 116 |
+
try:
|
| 117 |
+
storage.delete(spec.file_path)
|
| 118 |
+
except Exception as e:
|
| 119 |
+
logger.warning("Failed to delete spec file %s: %s", spec.file_path, e)
|
| 120 |
+
|
| 121 |
+
db.delete(spec)
|
| 122 |
+
db.commit()
|
| 123 |
+
return {"status": "deleted", "id": spec_id, "assets_removed": len(assets)}
|
| 124 |
+
|
| 125 |
+
|
| 126 |
+
@router.get("/specs/{spec_id}/credentials")
|
| 127 |
+
def get_credentials(spec_id: str, db: Session = Depends(database.get_db)):
|
| 128 |
+
spec = db.query(database.SpecDocument).filter(database.SpecDocument.id == spec_id).first()
|
| 129 |
+
if not spec:
|
| 130 |
+
raise HTTPException(status_code=404, detail="Spec not found")
|
| 131 |
+
return spec.login_credentials or {}
|
| 132 |
+
|
| 133 |
+
|
| 134 |
+
@router.post("/specs/{spec_id}/credentials")
|
| 135 |
+
def save_credentials(spec_id: str, payload: schemas.CredentialUpdate, db: Session = Depends(database.get_db)):
|
| 136 |
+
spec = db.query(database.SpecDocument).filter(database.SpecDocument.id == spec_id).first()
|
| 137 |
+
if not spec:
|
| 138 |
+
raise HTTPException(status_code=404, detail="Spec not found")
|
| 139 |
+
|
| 140 |
+
if payload.extra and payload.extra.get("_clear"):
|
| 141 |
+
spec.login_credentials = {}
|
| 142 |
+
flag_modified(spec, "login_credentials")
|
| 143 |
+
db.commit()
|
| 144 |
+
db.refresh(spec)
|
| 145 |
+
return {"status": "cleared", "credentials": {}}
|
| 146 |
+
|
| 147 |
+
current: dict = dict(spec.login_credentials or {})
|
| 148 |
+
if payload.name is not None:
|
| 149 |
+
current["name"] = payload.name
|
| 150 |
+
if payload.username is not None:
|
| 151 |
+
current["username"] = payload.username
|
| 152 |
+
current["email"] = payload.username
|
| 153 |
+
if payload.password is not None:
|
| 154 |
+
current["password"] = payload.password
|
| 155 |
+
if payload.extra:
|
| 156 |
+
extra_clean = {k: v for k, v in payload.extra.items() if not k.startswith("_")}
|
| 157 |
+
current.update(extra_clean)
|
| 158 |
+
|
| 159 |
+
spec.login_credentials = current
|
| 160 |
+
flag_modified(spec, "login_credentials")
|
| 161 |
+
db.commit()
|
| 162 |
+
db.refresh(spec)
|
| 163 |
+
return {"status": "saved", "credentials": spec.login_credentials}
|
|
File without changes
|
|
@@ -0,0 +1,19 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
from pydantic_settings import BaseSettings
|
| 2 |
+
|
| 3 |
+
|
| 4 |
+
class Settings(BaseSettings):
|
| 5 |
+
MISTRAL_API_KEY: str = ""
|
| 6 |
+
DATABASE_URL: str = ""
|
| 7 |
+
ALLOWED_ORIGINS: str = "http://localhost:5173,http://localhost:3000"
|
| 8 |
+
UPLOAD_DIR: str = "backend/uploads"
|
| 9 |
+
LLM_TIMEOUT: int = 60
|
| 10 |
+
HTTP_TIMEOUT: int = 120
|
| 11 |
+
|
| 12 |
+
model_config = {"env_file": ".env", "extra": "ignore"}
|
| 13 |
+
|
| 14 |
+
@property
|
| 15 |
+
def allowed_origins_list(self) -> list[str]:
|
| 16 |
+
return [o.strip() for o in self.ALLOWED_ORIGINS.split(",") if o.strip()]
|
| 17 |
+
|
| 18 |
+
|
| 19 |
+
settings = Settings()
|
|
@@ -0,0 +1,19 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
from fastapi import Request
|
| 2 |
+
from fastapi.responses import JSONResponse
|
| 3 |
+
|
| 4 |
+
|
| 5 |
+
class AppError(Exception):
|
| 6 |
+
"""Raise this anywhere in route/service code for a clean, consistent error response."""
|
| 7 |
+
def __init__(self, message: str, code: str = "ERROR", status: int = 400, detail: dict = None):
|
| 8 |
+
self.message = message
|
| 9 |
+
self.code = code
|
| 10 |
+
self.status = status
|
| 11 |
+
self.detail = detail or {}
|
| 12 |
+
super().__init__(message)
|
| 13 |
+
|
| 14 |
+
|
| 15 |
+
async def app_error_handler(request: Request, exc: AppError) -> JSONResponse:
|
| 16 |
+
return JSONResponse(
|
| 17 |
+
status_code=exc.status,
|
| 18 |
+
content={"error": exc.message, "code": exc.code, "detail": exc.detail},
|
| 19 |
+
)
|
|
@@ -0,0 +1,15 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
import logging
|
| 2 |
+
import sys
|
| 3 |
+
|
| 4 |
+
|
| 5 |
+
def get_logger(name: str) -> logging.Logger:
|
| 6 |
+
logger = logging.getLogger(name)
|
| 7 |
+
if not logger.handlers:
|
| 8 |
+
handler = logging.StreamHandler(sys.stdout)
|
| 9 |
+
handler.setFormatter(logging.Formatter(
|
| 10 |
+
"%(asctime)s [%(levelname)s] %(name)s: %(message)s",
|
| 11 |
+
datefmt="%Y-%m-%d %H:%M:%S",
|
| 12 |
+
))
|
| 13 |
+
logger.addHandler(handler)
|
| 14 |
+
logger.setLevel(logging.INFO)
|
| 15 |
+
return logger
|
|
@@ -0,0 +1,50 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
import ipaddress
|
| 2 |
+
import socket
|
| 3 |
+
from urllib.parse import urlparse
|
| 4 |
+
|
| 5 |
+
from fastapi import HTTPException
|
| 6 |
+
|
| 7 |
+
_BLOCKED_NETS = [
|
| 8 |
+
ipaddress.ip_network("127.0.0.0/8"),
|
| 9 |
+
ipaddress.ip_network("10.0.0.0/8"),
|
| 10 |
+
ipaddress.ip_network("172.16.0.0/12"),
|
| 11 |
+
ipaddress.ip_network("192.168.0.0/16"),
|
| 12 |
+
ipaddress.ip_network("0.0.0.0/8"),
|
| 13 |
+
ipaddress.ip_network("169.254.0.0/16"), # AWS/GCP metadata
|
| 14 |
+
ipaddress.ip_network("::1/128"),
|
| 15 |
+
ipaddress.ip_network("fc00::/7"),
|
| 16 |
+
]
|
| 17 |
+
|
| 18 |
+
_ALLOWED_SCHEMES = {"http", "https"}
|
| 19 |
+
|
| 20 |
+
|
| 21 |
+
def assert_safe_url(url: str) -> None:
|
| 22 |
+
"""Raise HTTP 400 if the URL targets a private/internal address or disallowed scheme."""
|
| 23 |
+
try:
|
| 24 |
+
parsed = urlparse(url)
|
| 25 |
+
except Exception:
|
| 26 |
+
raise HTTPException(status_code=400, detail=f"Malformed URL: {url}")
|
| 27 |
+
|
| 28 |
+
if parsed.scheme not in _ALLOWED_SCHEMES:
|
| 29 |
+
raise HTTPException(
|
| 30 |
+
status_code=400,
|
| 31 |
+
detail=f"URL scheme '{parsed.scheme}' is not allowed. Only http/https are permitted.",
|
| 32 |
+
)
|
| 33 |
+
|
| 34 |
+
hostname = parsed.hostname
|
| 35 |
+
if not hostname:
|
| 36 |
+
raise HTTPException(status_code=400, detail="URL has no hostname.")
|
| 37 |
+
|
| 38 |
+
# Resolve to IP and check against blocked ranges
|
| 39 |
+
try:
|
| 40 |
+
ip_str = socket.getaddrinfo(hostname, None)[0][4][0]
|
| 41 |
+
ip = ipaddress.ip_address(ip_str)
|
| 42 |
+
except Exception:
|
| 43 |
+
raise HTTPException(status_code=400, detail=f"Could not resolve hostname: {hostname}")
|
| 44 |
+
|
| 45 |
+
for net in _BLOCKED_NETS:
|
| 46 |
+
if ip in net:
|
| 47 |
+
raise HTTPException(
|
| 48 |
+
status_code=400,
|
| 49 |
+
detail=f"Requests to private/internal addresses are not allowed (resolved to {ip}).",
|
| 50 |
+
)
|
|
File without changes
|
|
File without changes
|
|
@@ -0,0 +1,34 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
from langchain_mistralai.chat_models import ChatMistralAI
|
| 2 |
+
from backend.core.logging import get_logger
|
| 3 |
+
|
| 4 |
+
logger = get_logger(__name__)
|
| 5 |
+
|
| 6 |
+
_MODEL = "mistral-large-latest"
|
| 7 |
+
|
| 8 |
+
|
| 9 |
+
def _extract_usage(ai_msg) -> dict | None:
|
| 10 |
+
usage = ai_msg.response_metadata.get("token_usage", {})
|
| 11 |
+
if not usage:
|
| 12 |
+
return None
|
| 13 |
+
return {
|
| 14 |
+
"prompt_tokens": usage.get("prompt_tokens", 0),
|
| 15 |
+
"completion_tokens": usage.get("completion_tokens", 0),
|
| 16 |
+
"total_tokens": usage.get("total_tokens", 0),
|
| 17 |
+
}
|
| 18 |
+
|
| 19 |
+
|
| 20 |
+
class MistralClient:
|
| 21 |
+
"""Centralized Mistral LLM client. All services use the module-level instances below."""
|
| 22 |
+
|
| 23 |
+
def __init__(self, temperature: float = 0, timeout: int = 120):
|
| 24 |
+
self._llm = ChatMistralAI(model_name=_MODEL, temperature=temperature, timeout=timeout)
|
| 25 |
+
|
| 26 |
+
def invoke(self, messages) -> tuple[str, dict | None]:
|
| 27 |
+
"""Invoke the LLM. Returns (content_str, token_usage_or_None)."""
|
| 28 |
+
ai_msg = self._llm.invoke(messages)
|
| 29 |
+
return ai_msg.content, _extract_usage(ai_msg)
|
| 30 |
+
|
| 31 |
+
|
| 32 |
+
# One instance per temperature setting β instantiated once per process, not per request.
|
| 33 |
+
deterministic = MistralClient(temperature=0)
|
| 34 |
+
chat = MistralClient(temperature=0.2)
|
|
@@ -1,935 +1,65 @@
|
|
| 1 |
-
from fastapi import FastAPI, Depends, HTTPException, File, UploadFile, Form
|
| 2 |
-
from fastapi.staticfiles import StaticFiles
|
| 3 |
-
from sqlalchemy.orm import Session
|
| 4 |
-
from sqlalchemy.orm.attributes import flag_modified
|
| 5 |
-
from . import database, schemas
|
| 6 |
-
from typing import Dict, Any, Optional
|
| 7 |
-
import httpx
|
| 8 |
-
import json
|
| 9 |
-
import uuid
|
| 10 |
import os
|
| 11 |
-
|
| 12 |
-
from fastapi
|
| 13 |
from fastapi.exceptions import RequestValidationError
|
|
|
|
| 14 |
from fastapi.responses import JSONResponse
|
|
|
|
|
|
|
|
|
|
|
|
|
| 15 |
|
| 16 |
-
from backend
|
| 17 |
-
from backend.
|
| 18 |
-
from backend.
|
| 19 |
-
from backend.
|
| 20 |
-
from backend.
|
| 21 |
-
from backend.services.asset_service import storage
|
| 22 |
-
from backend.services.context_manager import get_context, set_context_variables
|
| 23 |
-
from backend.services.asset_service import AssetService
|
| 24 |
-
from backend.services.chat_assistant import process_message as chat_process_message
|
| 25 |
|
|
|
|
| 26 |
|
| 27 |
-
#
|
| 28 |
-
|
| 29 |
-
CATEGORIES = ["audio", "video", "documents", "images"]
|
| 30 |
-
for cat in CATEGORIES:
|
| 31 |
-
os.makedirs(os.path.join(UPLOAD_DIR, cat), exist_ok=True)
|
| 32 |
|
| 33 |
-
|
|
|
|
|
|
|
| 34 |
|
| 35 |
-
#
|
| 36 |
-
|
| 37 |
|
| 38 |
-
|
| 39 |
-
|
| 40 |
-
|
| 41 |
-
|
| 42 |
-
return JSONResponse(
|
| 43 |
-
status_code=422,
|
| 44 |
-
content={"detail": exc.errors(), "body": str(await request.body())},
|
| 45 |
-
)
|
| 46 |
|
| 47 |
-
#
|
|
|
|
|
|
|
|
|
|
| 48 |
app.add_middleware(
|
| 49 |
CORSMiddleware,
|
| 50 |
-
allow_origins=
|
| 51 |
allow_credentials=True,
|
| 52 |
allow_methods=["*"],
|
| 53 |
allow_headers=["*"],
|
| 54 |
)
|
| 55 |
|
| 56 |
-
@app.get("/")
|
| 57 |
-
def read_root():
|
| 58 |
-
return {"status": "ok", "message": "AI API Testing Tool Backend running."}
|
| 59 |
-
|
| 60 |
-
@app.post("/session")
|
| 61 |
-
def create_session(db: Session = Depends(database.get_db)):
|
| 62 |
-
session_id = str(uuid.uuid4())
|
| 63 |
-
new_session = database.SessionModel(id=session_id)
|
| 64 |
-
db.add(new_session)
|
| 65 |
-
db.commit()
|
| 66 |
-
return {"session_id": session_id}
|
| 67 |
-
|
| 68 |
-
@app.post("/upload-spec")
|
| 69 |
-
async def upload_spec(file: UploadFile = File(...), db: Session = Depends(database.get_db)):
|
| 70 |
-
try:
|
| 71 |
-
content = await file.read()
|
| 72 |
-
decoded_content = content.decode("utf-8", errors="replace")
|
| 73 |
-
|
| 74 |
-
# Detect if it's a Postman collection
|
| 75 |
-
is_postman = False
|
| 76 |
-
try:
|
| 77 |
-
temp_json = json.loads(decoded_content)
|
| 78 |
-
if isinstance(temp_json, dict):
|
| 79 |
-
# Primary check: schema URL
|
| 80 |
-
info = temp_json.get("info", {})
|
| 81 |
-
if isinstance(info, dict) and "postman.com/json/collection" in info.get("schema", ""):
|
| 82 |
-
is_postman = True
|
| 83 |
-
# Secondary check: unique structure (info + item)
|
| 84 |
-
elif "info" in temp_json and "item" in temp_json:
|
| 85 |
-
# Swagger/OpenAPI usually has 'openapi' or 'swagger' key at top level
|
| 86 |
-
if "openapi" not in temp_json and "swagger" not in temp_json:
|
| 87 |
-
is_postman = True
|
| 88 |
-
except:
|
| 89 |
-
pass
|
| 90 |
-
|
| 91 |
-
if is_postman:
|
| 92 |
-
parsed_spec = parse_postman(decoded_content)
|
| 93 |
-
else:
|
| 94 |
-
parsed_spec = parse_spec(decoded_content)
|
| 95 |
-
|
| 96 |
-
safe_spec = json.loads(json.dumps(parsed_spec, ensure_ascii=True))
|
| 97 |
-
|
| 98 |
-
spec_id = str(uuid.uuid4())
|
| 99 |
-
# PERSIST to Storage (Supabase or Local)
|
| 100 |
-
file_path = f"specs/{spec_id}_{file.filename}"
|
| 101 |
-
|
| 102 |
-
# Log for debugging
|
| 103 |
-
print(f"[DEBUG] Uploading {file.filename} to {file_path}")
|
| 104 |
-
|
| 105 |
-
# Reset cursor since we read it for parsing
|
| 106 |
-
await file.seek(0)
|
| 107 |
-
storage.save(file.file, file_path)
|
| 108 |
-
|
| 109 |
-
# Persist the spec to the database
|
| 110 |
-
spec_doc = database.SpecDocument(
|
| 111 |
-
id=spec_id,
|
| 112 |
-
name=file.filename,
|
| 113 |
-
raw_content=decoded_content,
|
| 114 |
-
parsed_json=safe_spec,
|
| 115 |
-
file_path=file_path
|
| 116 |
-
)
|
| 117 |
-
db.add(spec_doc)
|
| 118 |
-
db.commit()
|
| 119 |
-
|
| 120 |
-
# Prepare return object including the ID for the frontend
|
| 121 |
-
safe_spec["id"] = spec_id
|
| 122 |
-
|
| 123 |
-
return {"status": "success", "spec_id": spec_id, "parsed_spec": safe_spec}
|
| 124 |
-
except Exception as e:
|
| 125 |
-
print(f"[ERROR] upload_spec failed: {str(e)}")
|
| 126 |
-
import traceback
|
| 127 |
-
traceback.print_exc()
|
| 128 |
-
raise HTTPException(status_code=400, detail=str(e))
|
| 129 |
-
except Exception as e:
|
| 130 |
-
raise HTTPException(status_code=400, detail=f"Failed to parse spec: {str(e)}")
|
| 131 |
-
|
| 132 |
-
@app.get("/specs")
|
| 133 |
-
def list_specs(db: Session = Depends(database.get_db)):
|
| 134 |
-
specs = db.query(database.SpecDocument).order_by(database.SpecDocument.uploaded_at.desc()).all()
|
| 135 |
-
return [
|
| 136 |
-
{"id": s.id, "name": s.name, "base_url": s.base_url or "", "uploaded_at": s.uploaded_at.isoformat()}
|
| 137 |
-
for s in specs
|
| 138 |
-
]
|
| 139 |
-
|
| 140 |
-
@app.get("/specs/{spec_id}")
|
| 141 |
-
def get_spec(spec_id: str, db: Session = Depends(database.get_db)):
|
| 142 |
-
spec = db.query(database.SpecDocument).filter(database.SpecDocument.id == spec_id).first()
|
| 143 |
-
if not spec:
|
| 144 |
-
raise HTTPException(status_code=404, detail="Spec not found")
|
| 145 |
-
return {
|
| 146 |
-
"id": spec.id,
|
| 147 |
-
"name": spec.name,
|
| 148 |
-
"base_url": spec.base_url or "",
|
| 149 |
-
"login_credentials": spec.login_credentials or {},
|
| 150 |
-
"parsed_spec": spec.parsed_json,
|
| 151 |
-
"execution_plan": spec.execution_plan,
|
| 152 |
-
"scenario": spec.scenario,
|
| 153 |
-
"uploaded_at": spec.uploaded_at.isoformat()
|
| 154 |
-
}
|
| 155 |
-
|
| 156 |
-
@app.patch("/specs/{spec_id}")
|
| 157 |
-
def update_spec(spec_id: str, payload: dict, db: Session = Depends(database.get_db)):
|
| 158 |
-
spec = db.query(database.SpecDocument).filter(database.SpecDocument.id == spec_id).first()
|
| 159 |
-
if not spec:
|
| 160 |
-
raise HTTPException(status_code=404, detail="Spec not found")
|
| 161 |
-
if "base_url" in payload:
|
| 162 |
-
spec.base_url = payload["base_url"]
|
| 163 |
-
db.commit()
|
| 164 |
-
return {"status": "updated", "id": spec.id, "base_url": spec.base_url}
|
| 165 |
-
|
| 166 |
-
|
| 167 |
-
def get_category(filename: str, content_type: str) -> str:
|
| 168 |
-
ext = os.path.splitext(filename)[1].lower()
|
| 169 |
-
if ext in ['.mp3', '.wav', '.ogg', '.m4a'] or (content_type and content_type.startswith('audio/')):
|
| 170 |
-
return 'audio'
|
| 171 |
-
if ext in ['.mp4', '.mov', '.avi', '.mkv'] or (content_type and content_type.startswith('video/')):
|
| 172 |
-
return 'video'
|
| 173 |
-
return 'documents'
|
| 174 |
-
|
| 175 |
-
@app.post("/assets")
|
| 176 |
-
async def upload_asset(
|
| 177 |
-
spec_id: str = Form(None),
|
| 178 |
-
collection_id: str = Form(None),
|
| 179 |
-
description: str = Form(None),
|
| 180 |
-
tags: str = Form("[]"),
|
| 181 |
-
file: UploadFile = File(...),
|
| 182 |
-
db: Session = Depends(database.get_db)
|
| 183 |
-
):
|
| 184 |
-
try:
|
| 185 |
-
tag_list = json.loads(tags)
|
| 186 |
-
asset = AssetService.upload_asset(db, file, spec_id, collection_id, description, tag_list)
|
| 187 |
-
return {
|
| 188 |
-
"id": asset.id,
|
| 189 |
-
"name": asset.name,
|
| 190 |
-
"type": asset.file_type,
|
| 191 |
-
"size": asset.file_size,
|
| 192 |
-
"category": asset.category,
|
| 193 |
-
"description": asset.description,
|
| 194 |
-
"tags": asset.tags,
|
| 195 |
-
"spec_id": asset.spec_id,
|
| 196 |
-
"uploaded_at": asset.uploaded_at.isoformat()
|
| 197 |
-
}
|
| 198 |
-
except Exception as e:
|
| 199 |
-
raise HTTPException(status_code=500, detail=f"Upload failed: {str(e)}")
|
| 200 |
-
|
| 201 |
-
@app.get("/assets")
|
| 202 |
-
def list_assets(spec_id: Optional[str] = None, category: Optional[str] = None, db: Session = Depends(database.get_db)):
|
| 203 |
-
assets = AssetService.get_assets(db, spec_id, category)
|
| 204 |
-
return [
|
| 205 |
-
{
|
| 206 |
-
"id": a.id,
|
| 207 |
-
"name": a.name,
|
| 208 |
-
"type": a.file_type,
|
| 209 |
-
"size": a.file_size,
|
| 210 |
-
"category": a.category,
|
| 211 |
-
"description": a.description,
|
| 212 |
-
"tags": a.tags,
|
| 213 |
-
"spec_id": a.spec_id,
|
| 214 |
-
"path": a.file_path,
|
| 215 |
-
"uploaded_at": a.uploaded_at.isoformat(),
|
| 216 |
-
"collection_ids": [c.id for c in a.collections]
|
| 217 |
-
}
|
| 218 |
-
for a in assets
|
| 219 |
-
]
|
| 220 |
-
|
| 221 |
-
@app.delete("/assets/{asset_id}")
|
| 222 |
-
def delete_asset(asset_id: str, db: Session = Depends(database.get_db)):
|
| 223 |
-
success = AssetService.delete_asset(db, asset_id)
|
| 224 |
-
if not success:
|
| 225 |
-
raise HTTPException(status_code=404, detail="Asset not found")
|
| 226 |
-
return {"status": "deleted", "id": asset_id}
|
| 227 |
-
|
| 228 |
-
@app.get("/collections")
|
| 229 |
-
def list_collections(spec_id: Optional[str] = None, db: Session = Depends(database.get_db)):
|
| 230 |
-
query = db.query(database.Collection)
|
| 231 |
-
if spec_id:
|
| 232 |
-
from sqlalchemy import or_
|
| 233 |
-
query = query.filter(or_(database.Collection.spec_id == spec_id, database.Collection.spec_id == None))
|
| 234 |
-
cols = query.all()
|
| 235 |
-
return [{"id": c.id, "name": c.name, "description": c.description, "spec_id": c.spec_id} for c in cols]
|
| 236 |
-
|
| 237 |
-
@app.post("/collections")
|
| 238 |
-
def create_collection(name: str = Form(...), spec_id: str = Form(None), description: str = Form(None), db: Session = Depends(database.get_db)):
|
| 239 |
-
col = AssetService.create_collection(db, name, spec_id, description)
|
| 240 |
-
return {"id": col.id, "name": col.name, "spec_id": col.spec_id}
|
| 241 |
-
|
| 242 |
-
@app.post("/collections/{collection_id}/assets/{asset_id}")
|
| 243 |
-
def add_asset_to_collection(collection_id: str, asset_id: str, db: Session = Depends(database.get_db)):
|
| 244 |
-
success = AssetService.add_to_collection(db, asset_id, collection_id)
|
| 245 |
-
if not success:
|
| 246 |
-
raise HTTPException(status_code=404, detail="Asset or Collection not found")
|
| 247 |
-
return {"status": "success"}
|
| 248 |
-
|
| 249 |
-
@app.delete("/collections/{collection_id}")
|
| 250 |
-
def delete_collection(collection_id: str, db: Session = Depends(database.get_db)):
|
| 251 |
-
success = AssetService.delete_collection(db, collection_id)
|
| 252 |
-
if not success:
|
| 253 |
-
raise HTTPException(status_code=404, detail="Collection not found")
|
| 254 |
-
return {"status": "success"}
|
| 255 |
-
|
| 256 |
-
@app.delete("/specs/{spec_id}")
|
| 257 |
-
def delete_spec(spec_id: str, db: Session = Depends(database.get_db)):
|
| 258 |
-
spec = db.query(database.SpecDocument).filter(database.SpecDocument.id == spec_id).first()
|
| 259 |
-
if not spec:
|
| 260 |
-
raise HTTPException(status_code=404, detail="Spec not found")
|
| 261 |
-
|
| 262 |
-
# Delete all associated assets (storage + DB)
|
| 263 |
-
assets = db.query(database.Asset).filter(database.Asset.spec_id == spec_id).all()
|
| 264 |
-
for asset in assets:
|
| 265 |
-
AssetService.delete_asset(db, asset.id)
|
| 266 |
-
|
| 267 |
-
# Delete spec file from storage
|
| 268 |
-
if spec.file_path:
|
| 269 |
-
try:
|
| 270 |
-
storage.delete(spec.file_path)
|
| 271 |
-
except Exception as e:
|
| 272 |
-
print(f"[CLEANUP ERROR] Failed to delete spec file {spec.file_path}: {e}")
|
| 273 |
-
|
| 274 |
-
db.delete(spec)
|
| 275 |
-
db.commit()
|
| 276 |
-
return {"status": "deleted", "id": spec_id, "assets_removed": len(assets)}
|
| 277 |
-
|
| 278 |
-
@app.post("/plan", response_model=schemas.ExecutionPlan)
|
| 279 |
-
def generate_plan(request: schemas.PlanRequest, db: Session = Depends(database.get_db)):
|
| 280 |
-
try:
|
| 281 |
-
# Expected `request.spec_content` to be the dict from `/upload-spec`
|
| 282 |
-
parsed_spec = json.loads(request.spec_content) if isinstance(request.spec_content, str) else request.spec_content
|
| 283 |
-
|
| 284 |
-
# Override with DB-stored credentials if a spec_id is provided
|
| 285 |
-
credentials = request.user_credentials or {}
|
| 286 |
-
if request.spec_id:
|
| 287 |
-
spec = db.query(database.SpecDocument).filter(database.SpecDocument.id == request.spec_id).first()
|
| 288 |
-
if spec and spec.login_credentials:
|
| 289 |
-
credentials = spec.login_credentials
|
| 290 |
-
|
| 291 |
-
plan = generate_execution_plan(
|
| 292 |
-
parsed_spec,
|
| 293 |
-
request.base_url,
|
| 294 |
-
request.scenario,
|
| 295 |
-
credentials
|
| 296 |
-
)
|
| 297 |
-
|
| 298 |
-
# PERSIST: Save the generated plan and update session metadata
|
| 299 |
-
if request.spec_id:
|
| 300 |
-
spec = db.query(database.SpecDocument).filter(database.SpecDocument.id == request.spec_id).first()
|
| 301 |
-
if spec:
|
| 302 |
-
spec.execution_plan = plan.model_dump()["execution_plan"]
|
| 303 |
-
spec.scenario = request.scenario
|
| 304 |
-
flag_modified(spec, "execution_plan")
|
| 305 |
-
flag_modified(spec, "scenario")
|
| 306 |
-
|
| 307 |
-
# Update session metadata if session_id provided
|
| 308 |
-
if request.session_id:
|
| 309 |
-
session = db.query(database.SessionModel).filter(database.SessionModel.id == request.session_id).first()
|
| 310 |
-
if session:
|
| 311 |
-
session.spec_id = spec.id
|
| 312 |
-
session.spec_name = spec.name
|
| 313 |
-
session.scenario = request.scenario
|
| 314 |
-
session.base_url = request.base_url
|
| 315 |
-
|
| 316 |
-
# NEW: Merge AI-generated credentials if present
|
| 317 |
-
if plan.suggested_credentials and isinstance(plan.suggested_credentials, dict):
|
| 318 |
-
current_creds = dict(spec.login_credentials or {})
|
| 319 |
-
# Only merge if there's actually something new to add
|
| 320 |
-
if any(k not in current_creds or current_creds[k] != v for k, v in plan.suggested_credentials.items()):
|
| 321 |
-
current_creds.update(plan.suggested_credentials)
|
| 322 |
-
spec.login_credentials = current_creds
|
| 323 |
-
flag_modified(spec, "login_credentials")
|
| 324 |
-
print(f"[AUTH MERGE] AI suggested credentials for '{spec.name}' merged into DB.")
|
| 325 |
-
|
| 326 |
-
db.commit()
|
| 327 |
-
|
| 328 |
-
return plan
|
| 329 |
-
except Exception as e:
|
| 330 |
-
raise HTTPException(status_code=500, detail=f"Planning failed: {str(e)}")
|
| 331 |
-
|
| 332 |
-
@app.post("/execute-step")
|
| 333 |
-
async def execute_step(request: schemas.ExecuteStepRequest, db: Session = Depends(database.get_db)):
|
| 334 |
-
try:
|
| 335 |
-
# Extract endpoint string (e.g., "GET /buses/{bus_id}" or "/buses/{bus_id}")
|
| 336 |
-
endpoint_str = request.step_definition.get("endpoint", "")
|
| 337 |
-
|
| 338 |
-
def resolve_refs(obj, full_spec, _resolving=None):
|
| 339 |
-
"""Recursively inline all $ref pointers so the LLM gets a flat, self-contained spec."""
|
| 340 |
-
if _resolving is None:
|
| 341 |
-
_resolving = set()
|
| 342 |
-
if isinstance(obj, dict):
|
| 343 |
-
if "$ref" in obj:
|
| 344 |
-
ref = obj["$ref"]
|
| 345 |
-
if ref in _resolving:
|
| 346 |
-
return {"$ref_loop": ref}
|
| 347 |
-
_resolving = _resolving | {ref}
|
| 348 |
-
# Only support local refs: "#/components/schemas/Foo"
|
| 349 |
-
parts = ref.lstrip("#/").split("/")
|
| 350 |
-
target = full_spec
|
| 351 |
-
for part in parts:
|
| 352 |
-
target = target.get(part, {}) if isinstance(target, dict) else {}
|
| 353 |
-
return resolve_refs(target, full_spec, _resolving)
|
| 354 |
-
return {k: resolve_refs(v, full_spec, _resolving) for k, v in obj.items()}
|
| 355 |
-
elif isinstance(obj, list):
|
| 356 |
-
return [resolve_refs(item, full_spec, _resolving) for item in obj]
|
| 357 |
-
return obj
|
| 358 |
-
|
| 359 |
-
def shorthand_spec(obj):
|
| 360 |
-
"""Convert JSON schema to ultra-compact shorthand string including enums and formats."""
|
| 361 |
-
if isinstance(obj, dict):
|
| 362 |
-
if "type" in obj:
|
| 363 |
-
t = obj["type"]
|
| 364 |
-
fmt = f":{obj['format']}" if "format" in obj else ""
|
| 365 |
-
enum_vals = f"[{'|'.join(map(str, obj['enum']))}]" if "enum" in obj else ""
|
| 366 |
-
|
| 367 |
-
if t == "object" and "properties" in obj:
|
| 368 |
-
req_list = obj.get("required", [])
|
| 369 |
-
props = []
|
| 370 |
-
for k, v in obj["properties"].items():
|
| 371 |
-
req = "*" if k in req_list or v.get("nullable") == False else ""
|
| 372 |
-
props.append(f"{k}{req}({shorthand_spec(v)})")
|
| 373 |
-
return "{" + ",".join(props) + "}"
|
| 374 |
-
if t == "array" and "items" in obj:
|
| 375 |
-
return f"[{shorthand_spec(obj['items'])}]"
|
| 376 |
-
|
| 377 |
-
return f"{t}{fmt}{enum_vals}"
|
| 378 |
-
return {k: shorthand_spec(v) for k, v in obj.items()}
|
| 379 |
-
return str(obj)
|
| 380 |
-
|
| 381 |
-
def prune_spec_for_execution(obj):
|
| 382 |
-
"""Recursively remove non-functional metadata (descriptions, examples, formats) to save tokens."""
|
| 383 |
-
if isinstance(obj, dict):
|
| 384 |
-
# Keys we definitely don't need for building an HTTP request
|
| 385 |
-
DROP_KEYS = {"description", "summary", "example", "examples", "format", "title", "default", "enum_descriptions"}
|
| 386 |
-
pruned = {}
|
| 387 |
-
for k, v in obj.items():
|
| 388 |
-
if k in DROP_KEYS:
|
| 389 |
-
continue
|
| 390 |
-
pruned[k] = prune_spec_for_execution(v)
|
| 391 |
-
return pruned
|
| 392 |
-
elif isinstance(obj, list):
|
| 393 |
-
return [prune_spec_for_execution(item) for item in obj]
|
| 394 |
-
return obj
|
| 395 |
-
|
| 396 |
-
actual_endpoint_spec = {"endpoint": endpoint_str}
|
| 397 |
-
|
| 398 |
-
if request.spec_id:
|
| 399 |
-
spec_doc = db.query(database.SpecDocument).filter(database.SpecDocument.id == request.spec_id).first()
|
| 400 |
-
if spec_doc and spec_doc.parsed_json:
|
| 401 |
-
full_spec = spec_doc.parsed_json
|
| 402 |
-
paths = full_spec.get("paths", {})
|
| 403 |
-
parts = endpoint_str.split(" ", 1)
|
| 404 |
-
def _match_path(path_template, actual_path):
|
| 405 |
-
"""Simple matcher for OpenAPI path templates like /buses/{bus_id}"""
|
| 406 |
-
import re
|
| 407 |
-
# Replace {param} with a regex group
|
| 408 |
-
pattern = re.sub(r'\{[^}]+\}', r'[^/]+', path_template)
|
| 409 |
-
# Handle trailing slashes
|
| 410 |
-
pattern = "^" + pattern.rstrip("/") + "/?$"
|
| 411 |
-
return bool(re.match(pattern, actual_path.split("?")[0]))
|
| 412 |
-
|
| 413 |
-
parts = endpoint_str.split(" ", 1)
|
| 414 |
-
path_only = parts[1] if len(parts) == 2 else parts[0]
|
| 415 |
-
|
| 416 |
-
# Try to find the exact or template match
|
| 417 |
-
matched_path = None
|
| 418 |
-
if path_only in paths:
|
| 419 |
-
matched_path = path_only
|
| 420 |
-
else:
|
| 421 |
-
for p in paths:
|
| 422 |
-
if _match_path(p, path_only):
|
| 423 |
-
matched_path = p
|
| 424 |
-
break
|
| 425 |
-
|
| 426 |
-
if matched_path:
|
| 427 |
-
raw_spec = {matched_path: paths[matched_path]}
|
| 428 |
-
# Inline all $ref so LLM sees the exact required fields
|
| 429 |
-
resolved = resolve_refs(raw_spec, full_spec)
|
| 430 |
-
|
| 431 |
-
# Optimization: Prune 'responses' as they are not needed for building the request
|
| 432 |
-
path_data = resolved.get(matched_path, {})
|
| 433 |
-
for method in path_data:
|
| 434 |
-
if isinstance(path_data[method], dict):
|
| 435 |
-
path_data[method].pop("responses", None)
|
| 436 |
-
|
| 437 |
-
# Pass through aggressive pruning to remove descriptions, examples, etc.
|
| 438 |
-
pruned = prune_spec_for_execution(resolved)
|
| 439 |
-
# CONVERT TO SHORTHAND
|
| 440 |
-
actual_endpoint_spec = shorthand_spec(pruned)
|
| 441 |
-
else:
|
| 442 |
-
print(f"[PATH MATCH FAIL] Could not find match for {path_only}. Falling back to simplified path list.")
|
| 443 |
-
# Fallback: Just send the relevant path/method info if we can't find the full schema
|
| 444 |
-
actual_endpoint_spec = {"endpoint": endpoint_str, "warning": "Full schema match failed"}
|
| 445 |
-
|
| 446 |
-
# Fetch semantic assets for AI context
|
| 447 |
-
rich_assets = AssetService.get_ai_context(db, request.spec_id)
|
| 448 |
-
|
| 449 |
-
# NEW: Fallback to stored credentials if none provided in request
|
| 450 |
-
effective_credentials = request.user_credentials
|
| 451 |
-
if not effective_credentials and request.spec_id:
|
| 452 |
-
spec = db.query(database.SpecDocument).filter(database.SpecDocument.id == request.spec_id).first()
|
| 453 |
-
if spec and spec.login_credentials:
|
| 454 |
-
effective_credentials = spec.login_credentials
|
| 455 |
-
print(f"[AUTH FALLBACK] Using stored credentials for spec '{spec.name}'")
|
| 456 |
-
|
| 457 |
-
# 1. Ask LLM to format the actual httpx request
|
| 458 |
-
req_plan, executor_token_usage = prepare_step_request(
|
| 459 |
-
endpoint_spec=actual_endpoint_spec,
|
| 460 |
-
base_url=request.base_url,
|
| 461 |
-
scenario="",
|
| 462 |
-
step_definition=request.step_definition,
|
| 463 |
-
context_variables=request.context_variables,
|
| 464 |
-
user_credentials=effective_credentials,
|
| 465 |
-
available_assets=rich_assets
|
| 466 |
-
)
|
| 467 |
-
|
| 468 |
-
# Sanitize URL: fix relative, missing-scheme, or placeholder URLs from the LLM
|
| 469 |
-
url = req_plan.request.url.strip()
|
| 470 |
-
PLACEHOLDER_HOSTS = {"api.example.com", "example.com", "<base_url>", "your-api.com"}
|
| 471 |
|
| 472 |
-
|
| 473 |
-
|
| 474 |
-
|
| 475 |
-
|
| 476 |
-
return host.lower() in PLACEHOLDER_HOSTS or u.startswith("<BASE_URL>")
|
| 477 |
-
except Exception:
|
| 478 |
-
return False
|
| 479 |
-
|
| 480 |
-
if url.startswith("/"):
|
| 481 |
-
# Relative path β prepend base URL
|
| 482 |
-
url = request.base_url.rstrip("/") + url
|
| 483 |
-
elif _is_placeholder(url):
|
| 484 |
-
# LLM returned a placeholder domain β extract just the path and rebuild
|
| 485 |
-
from urllib.parse import urlparse
|
| 486 |
-
parsed = urlparse(url)
|
| 487 |
-
path = parsed.path or "/"
|
| 488 |
-
url = request.base_url.rstrip("/") + path
|
| 489 |
-
if parsed.query:
|
| 490 |
-
url += "?" + parsed.query
|
| 491 |
-
print(f"[URL FIX] Placeholder URL detected, rebuilt to: {url}")
|
| 492 |
-
elif not url.startswith("http"):
|
| 493 |
-
url = f"https://{url}"
|
| 494 |
-
|
| 495 |
-
# Deduplicate query parameters gently: remove from dict if already in URL string
|
| 496 |
-
from urllib.parse import urlparse, parse_qsl
|
| 497 |
-
parsed = urlparse(url)
|
| 498 |
-
if parsed.query:
|
| 499 |
-
url_keys = set(k for k, v in parse_qsl(parsed.query))
|
| 500 |
-
# Remove keys from query_params if they are already hardcoded in the url
|
| 501 |
-
keys_to_delete = [k for k in req_plan.request.query_params if k in url_keys]
|
| 502 |
-
for k in keys_to_delete:
|
| 503 |
-
del req_plan.request.query_params[k]
|
| 504 |
-
|
| 505 |
-
# 2. Execute via httpx
|
| 506 |
-
async with httpx.AsyncClient(timeout=120.0) as client:
|
| 507 |
-
http_req = client.build_request(
|
| 508 |
-
method=req_plan.request.method.strip().upper(),
|
| 509 |
-
url=url,
|
| 510 |
-
headers=req_plan.request.headers,
|
| 511 |
-
params=req_plan.request.query_params,
|
| 512 |
-
data=req_plan.request.form_data if req_plan.request.form_data else None,
|
| 513 |
-
json=req_plan.request.body if req_plan.request.body and not req_plan.request.form_data else None
|
| 514 |
-
)
|
| 515 |
-
|
| 516 |
-
# Handle File Attachments if provided by LLM
|
| 517 |
-
files_payload = {}
|
| 518 |
-
opened_files = []
|
| 519 |
-
|
| 520 |
-
try:
|
| 521 |
-
if req_plan.request.files:
|
| 522 |
-
# Strip manually set Content-Type if files are present to let httpx handle the boundary
|
| 523 |
-
original_headers = {k: v for k, v in req_plan.request.headers.items() if k.lower() != 'content-type'}
|
| 524 |
-
req_plan.request.headers = original_headers
|
| 525 |
-
|
| 526 |
-
for field_name, file_name in req_plan.request.files.items():
|
| 527 |
-
# Find asset record by name
|
| 528 |
-
asset = db.query(database.Asset).filter(database.Asset.name == file_name).first()
|
| 529 |
-
if asset:
|
| 530 |
-
# Use StorageProvider to get full path
|
| 531 |
-
from backend.services.asset_service import storage
|
| 532 |
-
full_path = storage.get_full_path(asset.file_path)
|
| 533 |
-
if os.path.exists(full_path):
|
| 534 |
-
# Open file for reading
|
| 535 |
-
f = open(full_path, "rb")
|
| 536 |
-
opened_files.append(f)
|
| 537 |
-
files_payload[field_name] = (asset.name, f, asset.file_type)
|
| 538 |
-
else:
|
| 539 |
-
print(f"[ASSET ERROR] File '{file_name}' not found on disk at {full_path}")
|
| 540 |
-
else:
|
| 541 |
-
print(f"[ASSET ERROR] File '{file_name}' not found in Media Hub.")
|
| 542 |
-
|
| 543 |
-
# Print to backend console for debugging
|
| 544 |
-
print(f"Executing: {http_req.method} {http_req.url}")
|
| 545 |
-
|
| 546 |
-
try:
|
| 547 |
-
if files_payload:
|
| 548 |
-
response = await client.request(
|
| 549 |
-
method=req_plan.request.method.strip().upper(),
|
| 550 |
-
url=url,
|
| 551 |
-
headers=req_plan.request.headers,
|
| 552 |
-
params=req_plan.request.query_params,
|
| 553 |
-
data=req_plan.request.form_data if req_plan.request.form_data else None,
|
| 554 |
-
files=files_payload
|
| 555 |
-
)
|
| 556 |
-
else:
|
| 557 |
-
response = await client.send(http_req)
|
| 558 |
-
except Exception as e:
|
| 559 |
-
raise HTTPException(status_code=500, detail=f"Request to {url} failed: {str(e)}")
|
| 560 |
-
finally:
|
| 561 |
-
# CLOSE ALL FILE HANDLES
|
| 562 |
-
for f in opened_files:
|
| 563 |
-
f.close()
|
| 564 |
-
|
| 565 |
-
# Format the raw response
|
| 566 |
-
resp_data = {
|
| 567 |
-
"status_code": response.status_code,
|
| 568 |
-
"body": response.text,
|
| 569 |
-
"headers": dict(response.headers)
|
| 570 |
-
}
|
| 571 |
-
|
| 572 |
-
# Update the plan to reflect any URL cleanup
|
| 573 |
-
req_plan.request.url = url
|
| 574 |
-
|
| 575 |
-
# 3. Save to History
|
| 576 |
-
history = database.ExecutionHistory(
|
| 577 |
-
session_id=request.session_id,
|
| 578 |
-
step=req_plan.step,
|
| 579 |
-
request_json=req_plan.request.model_dump(),
|
| 580 |
-
response_json=resp_data,
|
| 581 |
-
step_definition_json=request.step_definition
|
| 582 |
-
)
|
| 583 |
-
db.add(history)
|
| 584 |
-
db.commit()
|
| 585 |
-
|
| 586 |
-
return {
|
| 587 |
-
"executed_request": req_plan.request.model_dump(),
|
| 588 |
-
"response": resp_data,
|
| 589 |
-
"analysis_needed": req_plan.step,
|
| 590 |
-
"token_usage": executor_token_usage
|
| 591 |
-
}
|
| 592 |
-
except Exception as e:
|
| 593 |
-
import traceback
|
| 594 |
-
trace = traceback.format_exc()
|
| 595 |
-
print(f"EXECUTION ERROR: {trace}")
|
| 596 |
-
raise HTTPException(status_code=500, detail=f"Execution failed: {str(e)}\n{trace}")
|
| 597 |
-
|
| 598 |
-
@app.post("/analyze-response", response_model=schemas.AnalyzeResponse)
|
| 599 |
-
def analyze_response_endpoint(request: schemas.AnalyzeRequest, db: Session = Depends(database.get_db)):
|
| 600 |
-
try:
|
| 601 |
-
step_def = request.step_definition or {}
|
| 602 |
-
endpoint_str = step_def.get("endpoint", "Unknown")
|
| 603 |
-
purpose_str = step_def.get("purpose", "Unknown")
|
| 604 |
-
|
| 605 |
-
# 1. Analyze it
|
| 606 |
-
analysis = analyze_api_response(
|
| 607 |
-
step=request.step,
|
| 608 |
-
endpoint=endpoint_str,
|
| 609 |
-
purpose=purpose_str,
|
| 610 |
-
status_code=request.response_data.get("status_code", 0),
|
| 611 |
-
response_body=request.response_data.get("body", "")
|
| 612 |
-
)
|
| 613 |
-
|
| 614 |
-
# 2. Save extracted variables ONLY to session context (session-scoped, not persisted to spec)
|
| 615 |
-
if analysis.extracted_variables:
|
| 616 |
-
set_context_variables(db, request.session_id, analysis.extracted_variables)
|
| 617 |
-
|
| 618 |
-
# 3. Update History
|
| 619 |
-
history = db.query(database.ExecutionHistory).filter(
|
| 620 |
-
database.ExecutionHistory.session_id == request.session_id,
|
| 621 |
-
database.ExecutionHistory.step == request.step
|
| 622 |
-
).order_by(database.ExecutionHistory.id.desc()).first()
|
| 623 |
-
|
| 624 |
-
if history:
|
| 625 |
-
history.analysis_json = analysis.model_dump()
|
| 626 |
-
db.commit()
|
| 627 |
-
|
| 628 |
-
return analysis
|
| 629 |
-
except Exception as e:
|
| 630 |
-
raise HTTPException(status_code=500, detail=f"Analysis failed: {str(e)}")
|
| 631 |
-
|
| 632 |
-
# βββββββββββββββββββββββββββββββββββββββββββββ
|
| 633 |
-
# CREDENTIAL ENDPOINTS (per spec)
|
| 634 |
-
# βββββββββββββββββββββββββββββββββββββββββββββ
|
| 635 |
-
|
| 636 |
-
@app.get("/specs/{spec_id}/credentials")
|
| 637 |
-
def get_credentials(spec_id: str, db: Session = Depends(database.get_db)):
|
| 638 |
-
"""Return stored login credentials for a spec."""
|
| 639 |
-
spec = db.query(database.SpecDocument).filter(database.SpecDocument.id == spec_id).first()
|
| 640 |
-
if not spec:
|
| 641 |
-
raise HTTPException(status_code=404, detail="Spec not found")
|
| 642 |
-
return spec.login_credentials or {}
|
| 643 |
-
|
| 644 |
-
@app.post("/specs/{spec_id}/credentials")
|
| 645 |
-
def save_credentials(spec_id: str, payload: schemas.CredentialUpdate, db: Session = Depends(database.get_db)):
|
| 646 |
-
"""Save/merge login credentials to a spec. Called manually from the UI."""
|
| 647 |
-
spec = db.query(database.SpecDocument).filter(database.SpecDocument.id == spec_id).first()
|
| 648 |
-
if not spec:
|
| 649 |
-
raise HTTPException(status_code=404, detail="Spec not found")
|
| 650 |
-
|
| 651 |
-
# If _clear flag is present, wipe all credentials
|
| 652 |
-
if payload.extra and payload.extra.get("_clear"):
|
| 653 |
-
spec.login_credentials = {}
|
| 654 |
-
flag_modified(spec, "login_credentials")
|
| 655 |
-
db.commit()
|
| 656 |
-
db.refresh(spec)
|
| 657 |
-
return {"status": "cleared", "credentials": {}}
|
| 658 |
-
|
| 659 |
-
current: dict = dict(spec.login_credentials or {})
|
| 660 |
-
if payload.name is not None:
|
| 661 |
-
current["name"] = payload.name
|
| 662 |
-
if payload.username is not None:
|
| 663 |
-
current["username"] = payload.username
|
| 664 |
-
current["email"] = payload.username # store under both keys
|
| 665 |
-
if payload.password is not None:
|
| 666 |
-
current["password"] = payload.password
|
| 667 |
-
if payload.extra:
|
| 668 |
-
# Filter out internal-only keys like _clear before saving
|
| 669 |
-
extra_clean = {k: v for k, v in payload.extra.items() if not k.startswith("_")}
|
| 670 |
-
current.update(extra_clean)
|
| 671 |
-
spec.login_credentials = current
|
| 672 |
-
flag_modified(spec, "login_credentials")
|
| 673 |
-
db.commit()
|
| 674 |
-
db.refresh(spec)
|
| 675 |
-
return {"status": "saved", "credentials": spec.login_credentials}
|
| 676 |
-
|
| 677 |
-
@app.get("/context/{session_id}")
|
| 678 |
-
def get_context_endpoint(session_id: str, db: Session = Depends(database.get_db)):
|
| 679 |
-
"""Return only session context variables (dynamic values from previous steps)."""
|
| 680 |
-
context = get_context(db, session_id)
|
| 681 |
-
return context
|
| 682 |
-
|
| 683 |
-
@app.post("/context/{session_id}")
|
| 684 |
-
def update_context_endpoint(session_id: str, payload: schemas.ContextUpdate, db: Session = Depends(database.get_db)):
|
| 685 |
-
"""Update session context variables manually (e.g., from selection)."""
|
| 686 |
-
# Assuming set_context_variables exists and merges the dict
|
| 687 |
-
variables = {payload.key: payload.value}
|
| 688 |
-
updated_context = set_context_variables(db, session_id, variables)
|
| 689 |
-
return updated_context
|
| 690 |
-
|
| 691 |
-
@app.delete("/session/{session_id}")
|
| 692 |
-
def clear_session(session_id: str, db: Session = Depends(database.get_db)):
|
| 693 |
-
session = db.query(database.SessionModel).filter(database.SessionModel.id == session_id).first()
|
| 694 |
-
if session:
|
| 695 |
-
db.delete(session)
|
| 696 |
-
db.query(database.ExecutionHistory).filter(database.ExecutionHistory.session_id == session_id).delete()
|
| 697 |
-
db.commit()
|
| 698 |
-
return {"status": "ok"}
|
| 699 |
-
|
| 700 |
-
@app.get("/history")
|
| 701 |
-
def get_all_history(db: Session = Depends(database.get_db)):
|
| 702 |
-
"""Fetch all execution history joined with session metadata."""
|
| 703 |
-
results = db.query(
|
| 704 |
-
database.ExecutionHistory,
|
| 705 |
-
database.SessionModel
|
| 706 |
-
).join(
|
| 707 |
-
database.SessionModel,
|
| 708 |
-
database.ExecutionHistory.session_id == database.SessionModel.id
|
| 709 |
-
).order_by(database.ExecutionHistory.timestamp.desc()).all()
|
| 710 |
-
|
| 711 |
-
history_list = []
|
| 712 |
-
for hist, sess in results:
|
| 713 |
-
history_list.append({
|
| 714 |
-
"id": hist.id,
|
| 715 |
-
"session_id": hist.session_id,
|
| 716 |
-
"spec_id": sess.spec_id,
|
| 717 |
-
"spec_name": sess.spec_name or "Unknown",
|
| 718 |
-
"scenario": sess.scenario or "No scenario",
|
| 719 |
-
"context_variables": sess.context_variables or {},
|
| 720 |
-
"step": hist.step,
|
| 721 |
-
"request": hist.request_json,
|
| 722 |
-
"response": hist.response_json,
|
| 723 |
-
"analysis": hist.analysis_json,
|
| 724 |
-
"step_definition": hist.step_definition_json,
|
| 725 |
-
"timestamp": hist.timestamp.isoformat()
|
| 726 |
-
})
|
| 727 |
-
return history_list
|
| 728 |
-
|
| 729 |
-
@app.delete("/history/{session_id}")
|
| 730 |
-
def delete_history_session(session_id: str, db: Session = Depends(database.get_db)):
|
| 731 |
-
"""Delete a specific session and all its execution history."""
|
| 732 |
-
db.query(database.ExecutionHistory).filter(database.ExecutionHistory.session_id == session_id).delete()
|
| 733 |
-
db.query(database.SessionModel).filter(database.SessionModel.id == session_id).delete()
|
| 734 |
-
db.commit()
|
| 735 |
-
return {"status": "deleted", "session_id": session_id}
|
| 736 |
-
|
| 737 |
-
@app.delete("/history")
|
| 738 |
-
def clear_all_history_endpoint(db: Session = Depends(database.get_db)):
|
| 739 |
-
"""Wipe all execution history and session metadata."""
|
| 740 |
-
db.query(database.ExecutionHistory).delete()
|
| 741 |
-
db.query(database.SessionModel).delete()
|
| 742 |
-
db.commit()
|
| 743 |
-
return {"status": "cleared"}
|
| 744 |
-
|
| 745 |
-
|
| 746 |
-
# βββββββββββββββββββββββββββββββββββββββββββββ
|
| 747 |
-
# CHAT ASSISTANT ENDPOINTS
|
| 748 |
-
# βββββββββββββββββββββββββββββββββββββββββββββ
|
| 749 |
-
|
| 750 |
-
@app.post("/chat", response_model=schemas.ChatResponse)
|
| 751 |
-
def chat_endpoint(request: schemas.ChatRequest, db: Session = Depends(database.get_db)):
|
| 752 |
-
"""
|
| 753 |
-
Send a message to the AI chat assistant.
|
| 754 |
-
- If chat_session_id is None β auto-creates a new named chat session.
|
| 755 |
-
- Saves user message + AI response to DB (persistent across refreshes).
|
| 756 |
-
- Returns a structured action so the frontend can trigger executions, etc.
|
| 757 |
-
"""
|
| 758 |
-
try:
|
| 759 |
-
# 1. Resolve or create chat session
|
| 760 |
-
chat_session = None
|
| 761 |
-
if request.chat_session_id:
|
| 762 |
-
chat_session = db.query(database.ChatSession).filter(
|
| 763 |
-
database.ChatSession.id == request.chat_session_id
|
| 764 |
-
).first()
|
| 765 |
-
|
| 766 |
-
if not chat_session:
|
| 767 |
-
# Auto-name from first message (truncate to 60 chars)
|
| 768 |
-
session_name = request.message[:60].strip()
|
| 769 |
-
if len(request.message) > 60:
|
| 770 |
-
session_name += "..."
|
| 771 |
-
chat_session = database.ChatSession(
|
| 772 |
-
id=str(uuid.uuid4()),
|
| 773 |
-
name=session_name,
|
| 774 |
-
spec_id=request.spec_id,
|
| 775 |
-
session_id=request.session_id,
|
| 776 |
-
)
|
| 777 |
-
db.add(chat_session)
|
| 778 |
-
db.flush() # get the ID before committing
|
| 779 |
-
|
| 780 |
-
# 2. Save user message to DB
|
| 781 |
-
user_msg = database.ChatMessage(
|
| 782 |
-
chat_session_id=chat_session.id,
|
| 783 |
-
role="user",
|
| 784 |
-
content=request.message,
|
| 785 |
-
)
|
| 786 |
-
db.add(user_msg)
|
| 787 |
-
db.flush()
|
| 788 |
-
|
| 789 |
-
# 3. Load chat history for context window (last 20 messages)
|
| 790 |
-
history_rows = db.query(database.ChatMessage).filter(
|
| 791 |
-
database.ChatMessage.chat_session_id == chat_session.id
|
| 792 |
-
).order_by(database.ChatMessage.id.asc()).all()
|
| 793 |
-
|
| 794 |
-
chat_history = [
|
| 795 |
-
{"role": m.role, "content": m.content}
|
| 796 |
-
for m in history_rows[:-1] # exclude the message we just inserted
|
| 797 |
-
]
|
| 798 |
-
|
| 799 |
-
# 4. Call the AI
|
| 800 |
-
ai_result = chat_process_message(
|
| 801 |
-
message=request.message,
|
| 802 |
-
spec_name=request.spec_name,
|
| 803 |
-
execution_plan=request.execution_plan or [],
|
| 804 |
-
context_variables=request.context_variables or {},
|
| 805 |
-
last_step_result=request.last_step_result,
|
| 806 |
-
all_step_results=request.all_step_results,
|
| 807 |
-
chat_history=chat_history,
|
| 808 |
-
)
|
| 809 |
-
|
| 810 |
-
# 5. Save assistant response to DB
|
| 811 |
-
assistant_content = ai_result.get("message", "")
|
| 812 |
-
assistant_msg = database.ChatMessage(
|
| 813 |
-
chat_session_id=chat_session.id,
|
| 814 |
-
role="assistant",
|
| 815 |
-
content=assistant_content,
|
| 816 |
-
action_json=ai_result,
|
| 817 |
-
)
|
| 818 |
-
db.add(assistant_msg)
|
| 819 |
-
|
| 820 |
-
# 6. Update session timestamp
|
| 821 |
-
chat_session.updated_at = __import__("datetime").datetime.utcnow()
|
| 822 |
-
db.commit()
|
| 823 |
-
|
| 824 |
-
return schemas.ChatResponse(
|
| 825 |
-
chat_session_id=chat_session.id,
|
| 826 |
-
chat_session_name=chat_session.name,
|
| 827 |
-
action=ai_result.get("action", "reply"),
|
| 828 |
-
message=assistant_content,
|
| 829 |
-
step=ai_result.get("step"),
|
| 830 |
-
changes=ai_result.get("changes"),
|
| 831 |
-
assertions=ai_result.get("assertions"),
|
| 832 |
-
explanation=ai_result.get("explanation"),
|
| 833 |
-
)
|
| 834 |
-
|
| 835 |
-
except Exception as e:
|
| 836 |
-
import traceback
|
| 837 |
-
traceback.print_exc()
|
| 838 |
-
raise HTTPException(status_code=500, detail=f"Chat failed: {str(e)}")
|
| 839 |
-
|
| 840 |
-
|
| 841 |
-
@app.get("/chat-sessions")
|
| 842 |
-
def list_chat_sessions(spec_id: Optional[str] = None, db: Session = Depends(database.get_db)):
|
| 843 |
-
"""List all chat sessions, optionally filtered by spec_id. Sorted newest first."""
|
| 844 |
-
query = db.query(database.ChatSession)
|
| 845 |
-
if spec_id:
|
| 846 |
-
query = query.filter(database.ChatSession.spec_id == spec_id)
|
| 847 |
-
sessions = query.order_by(database.ChatSession.updated_at.desc()).all()
|
| 848 |
-
|
| 849 |
-
result = []
|
| 850 |
-
for s in sessions:
|
| 851 |
-
msg_count = db.query(database.ChatMessage).filter(
|
| 852 |
-
database.ChatMessage.chat_session_id == s.id
|
| 853 |
-
).count()
|
| 854 |
-
result.append({
|
| 855 |
-
"id": s.id,
|
| 856 |
-
"name": s.name,
|
| 857 |
-
"spec_id": s.spec_id,
|
| 858 |
-
"session_id": s.session_id,
|
| 859 |
-
"message_count": msg_count,
|
| 860 |
-
"created_at": s.created_at.isoformat() if s.created_at else None,
|
| 861 |
-
"updated_at": s.updated_at.isoformat() if s.updated_at else None,
|
| 862 |
-
})
|
| 863 |
-
return result
|
| 864 |
-
|
| 865 |
-
|
| 866 |
-
@app.get("/chat-sessions/{chat_session_id}")
|
| 867 |
-
def get_chat_session(chat_session_id: str, db: Session = Depends(database.get_db)):
|
| 868 |
-
"""Get a single chat session with its full message history."""
|
| 869 |
-
session = db.query(database.ChatSession).filter(
|
| 870 |
-
database.ChatSession.id == chat_session_id
|
| 871 |
-
).first()
|
| 872 |
-
if not session:
|
| 873 |
-
raise HTTPException(status_code=404, detail="Chat session not found")
|
| 874 |
-
|
| 875 |
-
messages = db.query(database.ChatMessage).filter(
|
| 876 |
-
database.ChatMessage.chat_session_id == chat_session_id
|
| 877 |
-
).order_by(database.ChatMessage.id.asc()).all()
|
| 878 |
-
|
| 879 |
-
return {
|
| 880 |
-
"id": session.id,
|
| 881 |
-
"name": session.name,
|
| 882 |
-
"spec_id": session.spec_id,
|
| 883 |
-
"session_id": session.session_id,
|
| 884 |
-
"created_at": session.created_at.isoformat() if session.created_at else None,
|
| 885 |
-
"updated_at": session.updated_at.isoformat() if session.updated_at else None,
|
| 886 |
-
"messages": [
|
| 887 |
-
{
|
| 888 |
-
"id": m.id,
|
| 889 |
-
"role": m.role,
|
| 890 |
-
"content": m.content,
|
| 891 |
-
"action_json": m.action_json,
|
| 892 |
-
"timestamp": m.timestamp.isoformat() if m.timestamp else None,
|
| 893 |
-
}
|
| 894 |
-
for m in messages
|
| 895 |
-
],
|
| 896 |
-
}
|
| 897 |
-
|
| 898 |
-
|
| 899 |
-
@app.patch("/chat-sessions/{chat_session_id}")
|
| 900 |
-
def rename_chat_session(chat_session_id: str, payload: dict, db: Session = Depends(database.get_db)):
|
| 901 |
-
"""Rename a chat session."""
|
| 902 |
-
session = db.query(database.ChatSession).filter(
|
| 903 |
-
database.ChatSession.id == chat_session_id
|
| 904 |
-
).first()
|
| 905 |
-
if not session:
|
| 906 |
-
raise HTTPException(status_code=404, detail="Chat session not found")
|
| 907 |
-
if "name" in payload:
|
| 908 |
-
session.name = payload["name"]
|
| 909 |
-
db.commit()
|
| 910 |
-
return {"id": session.id, "name": session.name}
|
| 911 |
-
|
| 912 |
-
|
| 913 |
-
@app.delete("/chat-sessions/{chat_session_id}")
|
| 914 |
-
def delete_chat_session(chat_session_id: str, db: Session = Depends(database.get_db)):
|
| 915 |
-
"""Delete a chat session and all its messages (cascade)."""
|
| 916 |
-
session = db.query(database.ChatSession).filter(
|
| 917 |
-
database.ChatSession.id == chat_session_id
|
| 918 |
-
).first()
|
| 919 |
-
if not session:
|
| 920 |
-
raise HTTPException(status_code=404, detail="Chat session not found")
|
| 921 |
-
db.delete(session)
|
| 922 |
-
db.commit()
|
| 923 |
-
return {"status": "deleted", "id": chat_session_id}
|
| 924 |
|
| 925 |
|
| 926 |
-
|
| 927 |
-
|
| 928 |
-
|
| 929 |
-
|
| 930 |
-
|
| 931 |
-
|
| 932 |
-
|
| 933 |
-
return {"status": "cleared"}
|
| 934 |
|
| 935 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
import os
|
| 2 |
+
|
| 3 |
+
from fastapi import FastAPI, Request
|
| 4 |
from fastapi.exceptions import RequestValidationError
|
| 5 |
+
from fastapi.middleware.cors import CORSMiddleware
|
| 6 |
from fastapi.responses import JSONResponse
|
| 7 |
+
from fastapi.staticfiles import StaticFiles
|
| 8 |
+
from slowapi import Limiter, _rate_limit_exceeded_handler
|
| 9 |
+
from slowapi.errors import RateLimitExceeded
|
| 10 |
+
from slowapi.util import get_remote_address
|
| 11 |
|
| 12 |
+
from backend import database
|
| 13 |
+
from backend.core.config import settings
|
| 14 |
+
from backend.core.errors import AppError, app_error_handler
|
| 15 |
+
from backend.core.logging import get_logger
|
| 16 |
+
from backend.app.api.routes import specs, assets, sessions, history, chat, execution
|
|
|
|
|
|
|
|
|
|
|
|
|
| 17 |
|
| 18 |
+
logger = get_logger(__name__)
|
| 19 |
|
| 20 |
+
# Rate limiter β keyed by client IP
|
| 21 |
+
limiter = Limiter(key_func=get_remote_address, default_limits=["200/minute"])
|
|
|
|
|
|
|
|
|
|
| 22 |
|
| 23 |
+
# Ensure upload directories exist
|
| 24 |
+
for category in ["audio", "video", "documents", "images", "specs"]:
|
| 25 |
+
os.makedirs(os.path.join(settings.UPLOAD_DIR, category), exist_ok=True)
|
| 26 |
|
| 27 |
+
# Create DB tables
|
| 28 |
+
database.Base.metadata.create_all(bind=database.engine)
|
| 29 |
|
| 30 |
+
app = FastAPI(title="AI API Testing Assistant")
|
| 31 |
+
app.state.limiter = limiter
|
| 32 |
+
app.add_exception_handler(RateLimitExceeded, _rate_limit_exceeded_handler)
|
| 33 |
+
app.add_exception_handler(AppError, app_error_handler)
|
|
|
|
|
|
|
|
|
|
|
|
|
| 34 |
|
| 35 |
+
# Static file serving for uploads
|
| 36 |
+
app.mount("/uploads", StaticFiles(directory=settings.UPLOAD_DIR), name="uploads")
|
| 37 |
+
|
| 38 |
+
# CORS
|
| 39 |
app.add_middleware(
|
| 40 |
CORSMiddleware,
|
| 41 |
+
allow_origins=settings.allowed_origins_list,
|
| 42 |
allow_credentials=True,
|
| 43 |
allow_methods=["*"],
|
| 44 |
allow_headers=["*"],
|
| 45 |
)
|
| 46 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 47 |
|
| 48 |
+
@app.exception_handler(RequestValidationError)
|
| 49 |
+
async def validation_exception_handler(request, exc):
|
| 50 |
+
logger.warning("Validation error: %s", exc.errors())
|
| 51 |
+
return JSONResponse(status_code=422, content={"detail": exc.errors()})
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 52 |
|
| 53 |
|
| 54 |
+
# Routers
|
| 55 |
+
app.include_router(specs.router)
|
| 56 |
+
app.include_router(assets.router)
|
| 57 |
+
app.include_router(sessions.router)
|
| 58 |
+
app.include_router(history.router)
|
| 59 |
+
app.include_router(chat.router)
|
| 60 |
+
app.include_router(execution.router)
|
|
|
|
| 61 |
|
| 62 |
|
| 63 |
+
@app.get("/")
|
| 64 |
+
def health_check():
|
| 65 |
+
return {"status": "ok", "message": "AI API Testing Tool Backend running."}
|
|
@@ -1,9 +1,9 @@
|
|
| 1 |
import json
|
| 2 |
from typing import Dict, Any, List
|
| 3 |
from langchain_core.prompts import PromptTemplate
|
| 4 |
-
from langchain_mistralai.chat_models import ChatMistralAI
|
| 5 |
from langchain_core.output_parsers import JsonOutputParser
|
| 6 |
from pydantic import BaseModel, Field
|
|
|
|
| 7 |
|
| 8 |
# Using Pydantic to structure LangChain output for step execution
|
| 9 |
class APIRequest(BaseModel):
|
|
@@ -104,9 +104,6 @@ def prepare_step_request(
|
|
| 104 |
input_variables=["endpoint_spec", "base_url", "scenario", "step_definition", "user_credentials", "context_variables", "available_assets"],
|
| 105 |
)
|
| 106 |
|
| 107 |
-
llm = ChatMistralAI(model_name="mistral-large-latest", temperature=0)
|
| 108 |
-
chain = prompt | llm | parser
|
| 109 |
-
|
| 110 |
msg = prompt.format_prompt(
|
| 111 |
endpoint_spec=json.dumps(endpoint_spec, separators=(',', ':')),
|
| 112 |
base_url=base_url,
|
|
@@ -117,17 +114,7 @@ def prepare_step_request(
|
|
| 117 |
available_assets=json.dumps(available_assets, separators=(',', ':'))
|
| 118 |
).to_messages()
|
| 119 |
|
| 120 |
-
|
| 121 |
-
parsed_response = parser.parse(
|
| 122 |
-
|
| 123 |
-
# Extract token usage
|
| 124 |
-
usage = ai_msg.response_metadata.get("token_usage", {})
|
| 125 |
-
token_usage = None
|
| 126 |
-
if usage:
|
| 127 |
-
token_usage = {
|
| 128 |
-
"prompt_tokens": usage.get("prompt_tokens", 0),
|
| 129 |
-
"completion_tokens": usage.get("completion_tokens", 0),
|
| 130 |
-
"total_tokens": usage.get("total_tokens", 0)
|
| 131 |
-
}
|
| 132 |
-
|
| 133 |
return StepExecutionPlan(**parsed_response), token_usage
|
|
|
|
| 1 |
import json
|
| 2 |
from typing import Dict, Any, List
|
| 3 |
from langchain_core.prompts import PromptTemplate
|
|
|
|
| 4 |
from langchain_core.output_parsers import JsonOutputParser
|
| 5 |
from pydantic import BaseModel, Field
|
| 6 |
+
from backend.integrations.llm.client import deterministic as _llm
|
| 7 |
|
| 8 |
# Using Pydantic to structure LangChain output for step execution
|
| 9 |
class APIRequest(BaseModel):
|
|
|
|
| 104 |
input_variables=["endpoint_spec", "base_url", "scenario", "step_definition", "user_credentials", "context_variables", "available_assets"],
|
| 105 |
)
|
| 106 |
|
|
|
|
|
|
|
|
|
|
| 107 |
msg = prompt.format_prompt(
|
| 108 |
endpoint_spec=json.dumps(endpoint_spec, separators=(',', ':')),
|
| 109 |
base_url=base_url,
|
|
|
|
| 114 |
available_assets=json.dumps(available_assets, separators=(',', ':'))
|
| 115 |
).to_messages()
|
| 116 |
|
| 117 |
+
content, token_usage = _llm.invoke(msg)
|
| 118 |
+
parsed_response = parser.parse(content)
|
| 119 |
+
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 120 |
return StepExecutionPlan(**parsed_response), token_usage
|
|
@@ -9,8 +9,8 @@ what to do (execute a step, explain an error, modify a request, generate tests,
|
|
| 9 |
import json
|
| 10 |
import re
|
| 11 |
from typing import Any, Dict, List, Optional
|
| 12 |
-
from
|
| 13 |
-
from
|
| 14 |
|
| 15 |
SYSTEM_PROMPT = """You are an AI assistant integrated into an API testing tool (similar to Postman AI).
|
| 16 |
Your job is to help the user control their API testing workflow through natural language.
|
|
@@ -112,7 +112,7 @@ def _build_history_messages(chat_history: List[Dict]) -> List:
|
|
| 112 |
if role == "user":
|
| 113 |
messages.append(HumanMessage(content=content))
|
| 114 |
else:
|
| 115 |
-
messages.append(
|
| 116 |
return messages
|
| 117 |
|
| 118 |
|
|
@@ -133,15 +133,13 @@ def process_message(
|
|
| 133 |
|
| 134 |
full_system = SYSTEM_PROMPT + f"\n\n--- CURRENT CONTEXT ---\n{context_block}\n--- END CONTEXT ---"
|
| 135 |
|
| 136 |
-
llm = ChatMistralAI(model_name="mistral-large-latest", temperature=0.2, timeout=120)
|
| 137 |
-
|
| 138 |
messages = [SystemMessage(content=full_system)]
|
| 139 |
messages.extend(_build_history_messages(chat_history))
|
| 140 |
messages.append(HumanMessage(content=message))
|
| 141 |
|
| 142 |
try:
|
| 143 |
-
|
| 144 |
-
raw =
|
| 145 |
|
| 146 |
# Strip markdown code fences if present
|
| 147 |
raw = re.sub(r"^```(?:json)?\s*", "", raw, flags=re.MULTILINE)
|
|
|
|
| 9 |
import json
|
| 10 |
import re
|
| 11 |
from typing import Any, Dict, List, Optional
|
| 12 |
+
from langchain_core.messages import SystemMessage, HumanMessage, AIMessage
|
| 13 |
+
from backend.integrations.llm.client import chat as _llm
|
| 14 |
|
| 15 |
SYSTEM_PROMPT = """You are an AI assistant integrated into an API testing tool (similar to Postman AI).
|
| 16 |
Your job is to help the user control their API testing workflow through natural language.
|
|
|
|
| 112 |
if role == "user":
|
| 113 |
messages.append(HumanMessage(content=content))
|
| 114 |
else:
|
| 115 |
+
messages.append(AIMessage(content=content))
|
| 116 |
return messages
|
| 117 |
|
| 118 |
|
|
|
|
| 133 |
|
| 134 |
full_system = SYSTEM_PROMPT + f"\n\n--- CURRENT CONTEXT ---\n{context_block}\n--- END CONTEXT ---"
|
| 135 |
|
|
|
|
|
|
|
| 136 |
messages = [SystemMessage(content=full_system)]
|
| 137 |
messages.extend(_build_history_messages(chat_history))
|
| 138 |
messages.append(HumanMessage(content=message))
|
| 139 |
|
| 140 |
try:
|
| 141 |
+
content, _ = _llm.invoke(messages)
|
| 142 |
+
raw = content.strip()
|
| 143 |
|
| 144 |
# Strip markdown code fences if present
|
| 145 |
raw = re.sub(r"^```(?:json)?\s*", "", raw, flags=re.MULTILINE)
|
|
@@ -1,9 +1,12 @@
|
|
| 1 |
import json
|
| 2 |
from typing import Dict, Any, List, Optional
|
| 3 |
from langchain_core.prompts import PromptTemplate
|
| 4 |
-
from langchain_mistralai.chat_models import ChatMistralAI
|
| 5 |
from langchain_core.output_parsers import JsonOutputParser
|
|
|
|
| 6 |
from backend.schemas import ExecutionPlan
|
|
|
|
|
|
|
|
|
|
| 7 |
|
| 8 |
# Prompt template from the user prompt
|
| 9 |
PLANNER_PROMPT = """
|
|
@@ -73,28 +76,21 @@ def generate_execution_plan(parsed_spec: Dict[str, Any], base_url: str, scenario
|
|
| 73 |
input_variables=["parsed_spec", "base_url", "scenario", "user_credentials_str"],
|
| 74 |
)
|
| 75 |
|
| 76 |
-
# Initialize Mistral Model (requires MISTRAL_API_KEY in env)
|
| 77 |
-
llm = ChatMistralAI(model_name="mistral-large-latest", temperature=0, timeout=120)
|
| 78 |
-
|
| 79 |
msg = prompt.format_prompt(
|
| 80 |
parsed_spec=json.dumps(parsed_spec.get("endpoints", parsed_spec), separators=(',', ':')),
|
| 81 |
base_url=base_url,
|
| 82 |
scenario=scenario,
|
| 83 |
user_credentials_str=json.dumps(user_credentials or {}, separators=(',', ':'))
|
| 84 |
).to_messages()
|
| 85 |
-
|
| 86 |
-
|
| 87 |
-
|
| 88 |
-
|
| 89 |
-
|
| 90 |
-
|
| 91 |
-
|
| 92 |
-
|
| 93 |
-
|
| 94 |
-
"prompt_tokens": usage.get("prompt_tokens", 0),
|
| 95 |
-
"completion_tokens": usage.get("completion_tokens", 0),
|
| 96 |
-
"total_tokens": usage.get("total_tokens", 0)
|
| 97 |
-
}
|
| 98 |
|
| 99 |
# Return as Pydantic model
|
| 100 |
return ExecutionPlan(
|
|
|
|
| 1 |
import json
|
| 2 |
from typing import Dict, Any, List, Optional
|
| 3 |
from langchain_core.prompts import PromptTemplate
|
|
|
|
| 4 |
from langchain_core.output_parsers import JsonOutputParser
|
| 5 |
+
from backend.core.logging import get_logger
|
| 6 |
from backend.schemas import ExecutionPlan
|
| 7 |
+
from backend.integrations.llm.client import deterministic as _llm
|
| 8 |
+
|
| 9 |
+
logger = get_logger(__name__)
|
| 10 |
|
| 11 |
# Prompt template from the user prompt
|
| 12 |
PLANNER_PROMPT = """
|
|
|
|
| 76 |
input_variables=["parsed_spec", "base_url", "scenario", "user_credentials_str"],
|
| 77 |
)
|
| 78 |
|
|
|
|
|
|
|
|
|
|
| 79 |
msg = prompt.format_prompt(
|
| 80 |
parsed_spec=json.dumps(parsed_spec.get("endpoints", parsed_spec), separators=(',', ':')),
|
| 81 |
base_url=base_url,
|
| 82 |
scenario=scenario,
|
| 83 |
user_credentials_str=json.dumps(user_credentials or {}, separators=(',', ':'))
|
| 84 |
).to_messages()
|
| 85 |
+
|
| 86 |
+
# Invoke with one retry on parse failure
|
| 87 |
+
content, token_usage = _llm.invoke(msg)
|
| 88 |
+
try:
|
| 89 |
+
parsed_response = parser.parse(content)
|
| 90 |
+
except Exception as parse_err:
|
| 91 |
+
logger.warning("Plan parse failed, retrying: %s", parse_err)
|
| 92 |
+
content, token_usage = _llm.invoke(msg)
|
| 93 |
+
parsed_response = parser.parse(content)
|
|
|
|
|
|
|
|
|
|
|
|
|
| 94 |
|
| 95 |
# Return as Pydantic model
|
| 96 |
return ExecutionPlan(
|
|
@@ -1,9 +1,12 @@
|
|
| 1 |
import json
|
| 2 |
from typing import Dict, Any
|
| 3 |
from langchain_core.prompts import PromptTemplate
|
| 4 |
-
from langchain_mistralai.chat_models import ChatMistralAI
|
| 5 |
from langchain_core.output_parsers import JsonOutputParser
|
|
|
|
| 6 |
from backend.schemas import AnalyzeResponse
|
|
|
|
|
|
|
|
|
|
| 7 |
|
| 8 |
ANALYSIS_PROMPT = """
|
| 9 |
You are a response analyzer.
|
|
@@ -44,9 +47,6 @@ def analyze_api_response(
|
|
| 44 |
input_variables=["step_info", "status_code", "response_body"],
|
| 45 |
)
|
| 46 |
|
| 47 |
-
llm = ChatMistralAI(model_name="mistral-large-latest", temperature=0)
|
| 48 |
-
chain = prompt | llm | parser
|
| 49 |
-
|
| 50 |
# Aggressive truncation: keep only 500 chars from start and end (1KB total)
|
| 51 |
if len(response_body) > 1000:
|
| 52 |
response_body = response_body[:500] + "\n... [TRUNCATED] ...\n" + response_body[-500:]
|
|
@@ -57,18 +57,13 @@ def analyze_api_response(
|
|
| 57 |
response_body=response_body
|
| 58 |
).to_messages()
|
| 59 |
|
| 60 |
-
|
| 61 |
-
|
| 62 |
-
|
| 63 |
-
|
| 64 |
-
|
| 65 |
-
|
| 66 |
-
|
| 67 |
-
token_usage = {
|
| 68 |
-
"prompt_tokens": usage.get("prompt_tokens", 0),
|
| 69 |
-
"completion_tokens": usage.get("completion_tokens", 0),
|
| 70 |
-
"total_tokens": usage.get("total_tokens", 0)
|
| 71 |
-
}
|
| 72 |
|
| 73 |
# Ensure step matches the requested step and add token usage
|
| 74 |
parsed_response["step"] = step
|
|
|
|
| 1 |
import json
|
| 2 |
from typing import Dict, Any
|
| 3 |
from langchain_core.prompts import PromptTemplate
|
|
|
|
| 4 |
from langchain_core.output_parsers import JsonOutputParser
|
| 5 |
+
from backend.core.logging import get_logger
|
| 6 |
from backend.schemas import AnalyzeResponse
|
| 7 |
+
from backend.integrations.llm.client import deterministic as _llm
|
| 8 |
+
|
| 9 |
+
logger = get_logger(__name__)
|
| 10 |
|
| 11 |
ANALYSIS_PROMPT = """
|
| 12 |
You are a response analyzer.
|
|
|
|
| 47 |
input_variables=["step_info", "status_code", "response_body"],
|
| 48 |
)
|
| 49 |
|
|
|
|
|
|
|
|
|
|
| 50 |
# Aggressive truncation: keep only 500 chars from start and end (1KB total)
|
| 51 |
if len(response_body) > 1000:
|
| 52 |
response_body = response_body[:500] + "\n... [TRUNCATED] ...\n" + response_body[-500:]
|
|
|
|
| 57 |
response_body=response_body
|
| 58 |
).to_messages()
|
| 59 |
|
| 60 |
+
content, token_usage = _llm.invoke(msg)
|
| 61 |
+
try:
|
| 62 |
+
parsed_response = parser.parse(content)
|
| 63 |
+
except Exception as parse_err:
|
| 64 |
+
logger.warning("Analysis parse failed, retrying: %s", parse_err)
|
| 65 |
+
content, token_usage = _llm.invoke(msg)
|
| 66 |
+
parsed_response = parser.parse(content)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 67 |
|
| 68 |
# Ensure step matches the requested step and add token usage
|
| 69 |
parsed_response["step"] = step
|
|
File without changes
|
|
@@ -0,0 +1,44 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
import os
|
| 2 |
+
|
| 3 |
+
# Must be set before any backend module is imported β database.py reads this at import time.
|
| 4 |
+
os.environ["DATABASE_URL"] = "sqlite:///:memory:"
|
| 5 |
+
os.environ.setdefault("MISTRAL_API_KEY", "test-key")
|
| 6 |
+
os.environ.setdefault("ALLOWED_ORIGINS", "http://localhost:3000")
|
| 7 |
+
|
| 8 |
+
import pytest
|
| 9 |
+
from sqlalchemy import create_engine
|
| 10 |
+
from sqlalchemy.orm import sessionmaker
|
| 11 |
+
from fastapi.testclient import TestClient
|
| 12 |
+
|
| 13 |
+
from backend.database import Base, get_db
|
| 14 |
+
from backend.main import app
|
| 15 |
+
|
| 16 |
+
_engine = create_engine(
|
| 17 |
+
"sqlite:///:memory:",
|
| 18 |
+
connect_args={"check_same_thread": False},
|
| 19 |
+
)
|
| 20 |
+
_TestingSession = sessionmaker(autocommit=False, autoflush=False, bind=_engine)
|
| 21 |
+
|
| 22 |
+
|
| 23 |
+
def _override_get_db():
|
| 24 |
+
db = _TestingSession()
|
| 25 |
+
try:
|
| 26 |
+
yield db
|
| 27 |
+
finally:
|
| 28 |
+
db.close()
|
| 29 |
+
|
| 30 |
+
|
| 31 |
+
app.dependency_overrides[get_db] = _override_get_db
|
| 32 |
+
|
| 33 |
+
|
| 34 |
+
@pytest.fixture(scope="session", autouse=True)
|
| 35 |
+
def setup_db():
|
| 36 |
+
Base.metadata.create_all(bind=_engine)
|
| 37 |
+
yield
|
| 38 |
+
Base.metadata.drop_all(bind=_engine)
|
| 39 |
+
|
| 40 |
+
|
| 41 |
+
@pytest.fixture
|
| 42 |
+
def client():
|
| 43 |
+
with TestClient(app) as c:
|
| 44 |
+
yield c
|
|
@@ -0,0 +1,78 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
import json
|
| 2 |
+
from unittest.mock import patch
|
| 3 |
+
|
| 4 |
+
from backend.services.api_executor import prepare_step_request
|
| 5 |
+
|
| 6 |
+
_VALID_RESPONSE = {
|
| 7 |
+
"step": 1,
|
| 8 |
+
"request": {
|
| 9 |
+
"method": "POST",
|
| 10 |
+
"url": "https://api.example.com/auth/login",
|
| 11 |
+
"headers": {"Content-Type": "application/json"},
|
| 12 |
+
"query_params": {},
|
| 13 |
+
"form_data": {},
|
| 14 |
+
"body": {"username": "test@example.com", "password": "Pass123!"},
|
| 15 |
+
"files": {},
|
| 16 |
+
},
|
| 17 |
+
"analysis": {
|
| 18 |
+
"required_inputs": ["username", "password"],
|
| 19 |
+
"values_from_previous_steps": [],
|
| 20 |
+
"generated_test_data": [],
|
| 21 |
+
},
|
| 22 |
+
}
|
| 23 |
+
|
| 24 |
+
_TOKEN_USAGE = {"prompt_tokens": 200, "completion_tokens": 80, "total_tokens": 280}
|
| 25 |
+
|
| 26 |
+
|
| 27 |
+
def _run(**overrides):
|
| 28 |
+
kwargs = dict(
|
| 29 |
+
endpoint_spec={"method": "POST", "path": "/auth/login"},
|
| 30 |
+
base_url="https://api.example.com",
|
| 31 |
+
scenario="Login user",
|
| 32 |
+
step_definition={"step": 1, "endpoint": "POST /auth/login", "purpose": "Login"},
|
| 33 |
+
context_variables={},
|
| 34 |
+
user_credentials={"username": "test@example.com", "password": "Pass123!"},
|
| 35 |
+
)
|
| 36 |
+
kwargs.update(overrides)
|
| 37 |
+
return prepare_step_request(**kwargs)
|
| 38 |
+
|
| 39 |
+
|
| 40 |
+
def test_returns_step_execution_plan():
|
| 41 |
+
with patch("backend.services.api_executor._llm") as mock_llm:
|
| 42 |
+
mock_llm.invoke.return_value = (json.dumps(_VALID_RESPONSE), _TOKEN_USAGE)
|
| 43 |
+
|
| 44 |
+
result, token_usage = _run()
|
| 45 |
+
|
| 46 |
+
assert result.request.method == "POST"
|
| 47 |
+
assert result.request.url == "https://api.example.com/auth/login"
|
| 48 |
+
assert result.request.body == {"username": "test@example.com", "password": "Pass123!"}
|
| 49 |
+
|
| 50 |
+
|
| 51 |
+
def test_token_usage_returned():
|
| 52 |
+
with patch("backend.services.api_executor._llm") as mock_llm:
|
| 53 |
+
mock_llm.invoke.return_value = (json.dumps(_VALID_RESPONSE), _TOKEN_USAGE)
|
| 54 |
+
|
| 55 |
+
_, token_usage = _run()
|
| 56 |
+
|
| 57 |
+
assert token_usage["total_tokens"] == 280
|
| 58 |
+
|
| 59 |
+
|
| 60 |
+
def test_context_variables_included_in_prompt():
|
| 61 |
+
with patch("backend.services.api_executor._llm") as mock_llm:
|
| 62 |
+
mock_llm.invoke.return_value = (json.dumps(_VALID_RESPONSE), None)
|
| 63 |
+
|
| 64 |
+
_run(context_variables={"access_token": "tok_abc"})
|
| 65 |
+
|
| 66 |
+
call_args = mock_llm.invoke.call_args
|
| 67 |
+
# The rendered prompt messages should contain the context variable
|
| 68 |
+
prompt_text = str(call_args)
|
| 69 |
+
assert "access_token" in prompt_text
|
| 70 |
+
|
| 71 |
+
|
| 72 |
+
def test_token_usage_none_when_llm_returns_none():
|
| 73 |
+
with patch("backend.services.api_executor._llm") as mock_llm:
|
| 74 |
+
mock_llm.invoke.return_value = (json.dumps(_VALID_RESPONSE), None)
|
| 75 |
+
|
| 76 |
+
_, token_usage = _run()
|
| 77 |
+
|
| 78 |
+
assert token_usage is None
|
|
@@ -0,0 +1,71 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
import json
|
| 2 |
+
from unittest.mock import patch
|
| 3 |
+
|
| 4 |
+
import pytest
|
| 5 |
+
|
| 6 |
+
from backend.services.llm_planner import generate_execution_plan
|
| 7 |
+
|
| 8 |
+
_VALID_RESPONSE = {
|
| 9 |
+
"execution_plan": [
|
| 10 |
+
{"step": 1, "endpoint": "POST /auth/login", "purpose": "Authenticate the user"},
|
| 11 |
+
{"step": 2, "endpoint": "GET /profile", "purpose": "Fetch user profile"},
|
| 12 |
+
],
|
| 13 |
+
"suggested_credentials": {"username": "tester@example.com", "password": "Pass123!"},
|
| 14 |
+
}
|
| 15 |
+
|
| 16 |
+
_TOKEN_USAGE = {"prompt_tokens": 120, "completion_tokens": 60, "total_tokens": 180}
|
| 17 |
+
|
| 18 |
+
|
| 19 |
+
def test_returns_execution_plan():
|
| 20 |
+
with patch("backend.services.llm_planner._llm") as mock_llm:
|
| 21 |
+
mock_llm.invoke.return_value = (json.dumps(_VALID_RESPONSE), _TOKEN_USAGE)
|
| 22 |
+
|
| 23 |
+
result = generate_execution_plan(
|
| 24 |
+
parsed_spec={"endpoints": []},
|
| 25 |
+
base_url="https://api.example.com",
|
| 26 |
+
scenario="Login and view profile",
|
| 27 |
+
)
|
| 28 |
+
|
| 29 |
+
assert len(result.execution_plan) == 2
|
| 30 |
+
assert result.execution_plan[0]["step"] == 1
|
| 31 |
+
assert result.suggested_credentials["username"] == "tester@example.com"
|
| 32 |
+
|
| 33 |
+
|
| 34 |
+
def test_token_usage_is_passed_through():
|
| 35 |
+
with patch("backend.services.llm_planner._llm") as mock_llm:
|
| 36 |
+
mock_llm.invoke.return_value = (json.dumps(_VALID_RESPONSE), _TOKEN_USAGE)
|
| 37 |
+
|
| 38 |
+
result = generate_execution_plan(
|
| 39 |
+
parsed_spec={}, base_url="https://api.example.com", scenario="Login"
|
| 40 |
+
)
|
| 41 |
+
|
| 42 |
+
assert result.token_usage == _TOKEN_USAGE
|
| 43 |
+
|
| 44 |
+
|
| 45 |
+
def test_retries_once_on_parse_failure():
|
| 46 |
+
with patch("backend.services.llm_planner._llm") as mock_llm:
|
| 47 |
+
# First call returns invalid JSON; second returns valid
|
| 48 |
+
mock_llm.invoke.side_effect = [
|
| 49 |
+
("not valid json {{{{", None),
|
| 50 |
+
(json.dumps(_VALID_RESPONSE), _TOKEN_USAGE),
|
| 51 |
+
]
|
| 52 |
+
|
| 53 |
+
result = generate_execution_plan(
|
| 54 |
+
parsed_spec={}, base_url="https://api.example.com", scenario="Login"
|
| 55 |
+
)
|
| 56 |
+
|
| 57 |
+
assert mock_llm.invoke.call_count == 2
|
| 58 |
+
assert len(result.execution_plan) == 2
|
| 59 |
+
|
| 60 |
+
|
| 61 |
+
def test_raises_after_two_parse_failures():
|
| 62 |
+
with patch("backend.services.llm_planner._llm") as mock_llm:
|
| 63 |
+
mock_llm.invoke.side_effect = [
|
| 64 |
+
("bad json", None),
|
| 65 |
+
("still bad json", None),
|
| 66 |
+
]
|
| 67 |
+
|
| 68 |
+
with pytest.raises(Exception):
|
| 69 |
+
generate_execution_plan(
|
| 70 |
+
parsed_spec={}, base_url="https://api.example.com", scenario="Login"
|
| 71 |
+
)
|
|
@@ -0,0 +1,140 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""
|
| 2 |
+
Integration tests for HTTP routes.
|
| 3 |
+
All LLM calls are mocked so no real Mistral API key is needed.
|
| 4 |
+
"""
|
| 5 |
+
import io
|
| 6 |
+
import json
|
| 7 |
+
from unittest.mock import patch
|
| 8 |
+
|
| 9 |
+
import pytest
|
| 10 |
+
|
| 11 |
+
# client fixture is provided by conftest.py
|
| 12 |
+
|
| 13 |
+
|
| 14 |
+
# ββ Health / root βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
|
| 15 |
+
|
| 16 |
+
def test_root_health(client):
|
| 17 |
+
response = client.get("/")
|
| 18 |
+
assert response.status_code == 200
|
| 19 |
+
assert response.json()["status"] == "ok"
|
| 20 |
+
|
| 21 |
+
|
| 22 |
+
# ββ Session βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
|
| 23 |
+
|
| 24 |
+
def test_create_session(client):
|
| 25 |
+
response = client.post("/session")
|
| 26 |
+
assert response.status_code == 200
|
| 27 |
+
data = response.json()
|
| 28 |
+
assert "session_id" in data
|
| 29 |
+
|
| 30 |
+
|
| 31 |
+
# ββ Spec upload βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
|
| 32 |
+
|
| 33 |
+
_MINIMAL_SPEC = json.dumps({
|
| 34 |
+
"openapi": "3.0.0",
|
| 35 |
+
"info": {"title": "Test API", "version": "1.0.0"},
|
| 36 |
+
"paths": {
|
| 37 |
+
"/ping": {
|
| 38 |
+
"get": {
|
| 39 |
+
"summary": "Ping",
|
| 40 |
+
"responses": {"200": {"description": "OK"}},
|
| 41 |
+
}
|
| 42 |
+
}
|
| 43 |
+
},
|
| 44 |
+
})
|
| 45 |
+
|
| 46 |
+
|
| 47 |
+
def test_upload_spec_valid(client):
|
| 48 |
+
response = client.post(
|
| 49 |
+
"/upload-spec",
|
| 50 |
+
files={"file": ("api.json", _MINIMAL_SPEC.encode(), "application/json")},
|
| 51 |
+
)
|
| 52 |
+
assert response.status_code == 200
|
| 53 |
+
data = response.json()
|
| 54 |
+
assert "id" in data
|
| 55 |
+
return data["id"]
|
| 56 |
+
|
| 57 |
+
|
| 58 |
+
def test_upload_spec_rejects_large_file(client):
|
| 59 |
+
big_content = b"x" * (6 * 1024 * 1024) # 6 MB β over the 5 MB limit
|
| 60 |
+
response = client.post(
|
| 61 |
+
"/upload-spec",
|
| 62 |
+
files={"file": ("big.json", big_content, "application/json")},
|
| 63 |
+
)
|
| 64 |
+
assert response.status_code == 400
|
| 65 |
+
|
| 66 |
+
|
| 67 |
+
def test_list_specs(client):
|
| 68 |
+
response = client.get("/specs")
|
| 69 |
+
assert response.status_code == 200
|
| 70 |
+
assert isinstance(response.json(), list)
|
| 71 |
+
|
| 72 |
+
|
| 73 |
+
# ββ Plan generation (mocked LLM) ββββββββββββββββββββββββββββββββββββββββββββββ
|
| 74 |
+
|
| 75 |
+
_MOCK_PLAN = {
|
| 76 |
+
"execution_plan": [
|
| 77 |
+
{"step": 1, "endpoint": "GET /ping", "purpose": "Health check"},
|
| 78 |
+
],
|
| 79 |
+
"suggested_credentials": {},
|
| 80 |
+
}
|
| 81 |
+
|
| 82 |
+
_MOCK_TOKEN_USAGE = {"prompt_tokens": 50, "completion_tokens": 20, "total_tokens": 70}
|
| 83 |
+
|
| 84 |
+
|
| 85 |
+
def test_plan_endpoint(client):
|
| 86 |
+
# Upload a spec first to get a valid spec_id
|
| 87 |
+
upload = client.post(
|
| 88 |
+
"/upload-spec",
|
| 89 |
+
files={"file": ("api.json", _MINIMAL_SPEC.encode(), "application/json")},
|
| 90 |
+
)
|
| 91 |
+
spec_id = upload.json()["id"]
|
| 92 |
+
|
| 93 |
+
session = client.post("/session").json()["session_id"]
|
| 94 |
+
|
| 95 |
+
with patch("backend.services.llm_planner._llm") as mock_llm:
|
| 96 |
+
mock_llm.invoke.return_value = (json.dumps(_MOCK_PLAN), _MOCK_TOKEN_USAGE)
|
| 97 |
+
|
| 98 |
+
response = client.post("/plan", json={
|
| 99 |
+
"spec_content": _MINIMAL_SPEC,
|
| 100 |
+
"base_url": "https://api.example.com",
|
| 101 |
+
"scenario": "Ping the API",
|
| 102 |
+
"user_credentials": {},
|
| 103 |
+
"spec_id": spec_id,
|
| 104 |
+
"session_id": session,
|
| 105 |
+
})
|
| 106 |
+
|
| 107 |
+
assert response.status_code == 200
|
| 108 |
+
data = response.json()
|
| 109 |
+
assert len(data["execution_plan"]) == 1
|
| 110 |
+
assert data["token_usage"]["total_tokens"] == 70
|
| 111 |
+
|
| 112 |
+
|
| 113 |
+
# ββ Chat endpoint (mocked LLM) ββββββββββββββββββββββββββββββββββββββββββββββββ
|
| 114 |
+
|
| 115 |
+
_MOCK_CHAT_RESPONSE = json.dumps({
|
| 116 |
+
"action": "reply",
|
| 117 |
+
"message": "I can help you run step 1.",
|
| 118 |
+
})
|
| 119 |
+
|
| 120 |
+
|
| 121 |
+
def test_chat_endpoint(client):
|
| 122 |
+
session = client.post("/session").json()["session_id"]
|
| 123 |
+
|
| 124 |
+
with patch("backend.services.chat_assistant._llm") as mock_llm:
|
| 125 |
+
mock_llm.invoke.return_value = (_MOCK_CHAT_RESPONSE, None)
|
| 126 |
+
|
| 127 |
+
response = client.post("/chat", json={
|
| 128 |
+
"message": "Run step 1 for me",
|
| 129 |
+
"session_id": session,
|
| 130 |
+
"spec_id": None,
|
| 131 |
+
"execution_plan": [],
|
| 132 |
+
"context_variables": {},
|
| 133 |
+
"last_step_result": None,
|
| 134 |
+
"all_step_results": {},
|
| 135 |
+
"chat_session_id": None,
|
| 136 |
+
})
|
| 137 |
+
|
| 138 |
+
assert response.status_code == 200
|
| 139 |
+
data = response.json()
|
| 140 |
+
assert data["action"] == "reply"
|
|
@@ -0,0 +1,75 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
import socket
|
| 2 |
+
from unittest.mock import patch
|
| 3 |
+
import pytest
|
| 4 |
+
from fastapi import HTTPException
|
| 5 |
+
|
| 6 |
+
from backend.core.url_guard import assert_safe_url
|
| 7 |
+
|
| 8 |
+
|
| 9 |
+
def _mock_resolve(ip: str):
|
| 10 |
+
"""Return a socket.getaddrinfo patch that resolves any hostname to `ip`."""
|
| 11 |
+
return patch("backend.core.url_guard.socket.getaddrinfo", return_value=[(None, None, None, None, (ip, 0))])
|
| 12 |
+
|
| 13 |
+
|
| 14 |
+
# ββ Scheme checks βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
|
| 15 |
+
|
| 16 |
+
def test_blocks_ftp_scheme():
|
| 17 |
+
with pytest.raises(HTTPException) as exc:
|
| 18 |
+
assert_safe_url("ftp://example.com/file")
|
| 19 |
+
assert exc.value.status_code == 400
|
| 20 |
+
|
| 21 |
+
|
| 22 |
+
def test_blocks_file_scheme():
|
| 23 |
+
with pytest.raises(HTTPException):
|
| 24 |
+
assert_safe_url("file:///etc/passwd")
|
| 25 |
+
|
| 26 |
+
|
| 27 |
+
def test_blocks_no_hostname():
|
| 28 |
+
with pytest.raises(HTTPException):
|
| 29 |
+
assert_safe_url("http:///path")
|
| 30 |
+
|
| 31 |
+
|
| 32 |
+
# ββ Private / loopback IP checks ββββββββββββββββββββββββββββββββββββββββββββββ
|
| 33 |
+
|
| 34 |
+
def test_blocks_localhost_ip():
|
| 35 |
+
with _mock_resolve("127.0.0.1"):
|
| 36 |
+
with pytest.raises(HTTPException) as exc:
|
| 37 |
+
assert_safe_url("http://localhost/api")
|
| 38 |
+
assert exc.value.status_code == 400
|
| 39 |
+
|
| 40 |
+
|
| 41 |
+
def test_blocks_10x_range():
|
| 42 |
+
with _mock_resolve("10.0.0.1"):
|
| 43 |
+
with pytest.raises(HTTPException):
|
| 44 |
+
assert_safe_url("http://internal.corp/api")
|
| 45 |
+
|
| 46 |
+
|
| 47 |
+
def test_blocks_192168_range():
|
| 48 |
+
with _mock_resolve("192.168.1.100"):
|
| 49 |
+
with pytest.raises(HTTPException):
|
| 50 |
+
assert_safe_url("http://router.local/api")
|
| 51 |
+
|
| 52 |
+
|
| 53 |
+
def test_blocks_172_16_range():
|
| 54 |
+
with _mock_resolve("172.16.0.5"):
|
| 55 |
+
with pytest.raises(HTTPException):
|
| 56 |
+
assert_safe_url("http://private.example.com/api")
|
| 57 |
+
|
| 58 |
+
|
| 59 |
+
def test_blocks_link_local():
|
| 60 |
+
with _mock_resolve("169.254.169.254"):
|
| 61 |
+
with pytest.raises(HTTPException):
|
| 62 |
+
assert_safe_url("http://metadata.example.com/api")
|
| 63 |
+
|
| 64 |
+
|
| 65 |
+
# ββ Allow public addresses βββββββββββββββββββββββββββββββββββββββββββββββββββββ
|
| 66 |
+
|
| 67 |
+
def test_allows_public_ip():
|
| 68 |
+
with _mock_resolve("93.184.216.34"): # example.com
|
| 69 |
+
assert_safe_url("https://example.com/api") # should not raise
|
| 70 |
+
|
| 71 |
+
|
| 72 |
+
def test_allows_http_and_https():
|
| 73 |
+
with _mock_resolve("93.184.216.34"):
|
| 74 |
+
assert_safe_url("http://example.com/api")
|
| 75 |
+
assert_safe_url("https://example.com/api")
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
@@ -59,27 +59,29 @@
|
|
| 59 |
| Web Framework | **FastAPI** | β₯0.109 | REST API server |
|
| 60 |
| ASGI Server | **Uvicorn** | β₯0.27 | Runs FastAPI in development |
|
| 61 |
| Production Server | **Gunicorn** | 21.2 | Runs Uvicorn workers in production |
|
| 62 |
-
| LLM Orchestration | **LangChain** | 0.
|
| 63 |
| LLM Provider | **Mistral AI** | `mistral-large-latest` | Plan generation, step execution, analysis |
|
| 64 |
| HTTP Client | **httpx** | β₯0.27 | Sends actual API requests during execution |
|
| 65 |
-
| ORM | **SQLAlchemy** | 2.0.
|
| 66 |
| Database | **PostgreSQL** (via Supabase) | β | Persistent storage |
|
| 67 |
-
| DB Driver | **psycopg2-binary** | 2.9.
|
| 68 |
| Cloud Storage | **Supabase Storage** | 2.28.2 | Stores uploaded spec files and assets |
|
| 69 |
| Spec Parsing | **prance** + **PyYAML** | β | Parse OpenAPI YAML/JSON |
|
| 70 |
| Validation | **Pydantic** | β₯2.6 | Request/response data models |
|
|
|
|
| 71 |
| Auth/Config | **python-dotenv** | 1.0.0 | Load env variables from `.env` |
|
|
|
|
| 72 |
|
| 73 |
### Frontend
|
| 74 |
|
| 75 |
| Layer | Technology | Version | Purpose |
|
| 76 |
|---|---|---|---|
|
| 77 |
-
| UI Framework | **React** | 19.
|
| 78 |
| Build Tool | **Vite** | 8.x | Fast dev server + bundler |
|
| 79 |
-
| Styling | **Tailwind CSS** | 4.
|
| 80 |
| Icons | **lucide-react** | 0.577 | Icon library |
|
| 81 |
| HTTP Client | **Axios** | 1.x | API communication with backend |
|
| 82 |
-
| Routing | **react-router-dom** | 7.
|
| 83 |
|
| 84 |
---
|
| 85 |
|
|
@@ -95,22 +97,35 @@ Testing/ β Project root
|
|
| 95 |
βββ runtime.txt β Python version pin
|
| 96 |
βββ start.ps1 β PowerShell startup script
|
| 97 |
βββ openapi.yaml β Sample OpenAPI spec for testing
|
|
|
|
| 98 |
β
|
| 99 |
βββ backend/ β FastAPI application
|
| 100 |
-
β βββ
|
| 101 |
-
β βββ main.py β All FastAPI routes (API gateway)
|
| 102 |
β βββ database.py β SQLAlchemy models + DB connection
|
| 103 |
β βββ schemas.py β Pydantic request/response models
|
| 104 |
-
β
|
| 105 |
-
β
|
| 106 |
-
β
|
| 107 |
-
β
|
| 108 |
-
β
|
| 109 |
-
β
|
| 110 |
-
β
|
| 111 |
-
β
|
| 112 |
-
β
|
| 113 |
-
β
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 114 |
β
|
| 115 |
βββ frontend-react/ β React + Vite frontend
|
| 116 |
βββ index.html
|
|
@@ -118,18 +133,17 @@ Testing/ β Project root
|
|
| 118 |
βββ package.json
|
| 119 |
βββ src/
|
| 120 |
βββ main.jsx β React entry point
|
| 121 |
-
βββ App.jsx β Main layout +
|
| 122 |
-
βββ App.css / index.css β
|
| 123 |
-
βββ components/
|
| 124 |
-
β βββ AppContext.jsx β Global
|
| 125 |
-
β βββ CollectionsPage.jsx β
|
| 126 |
-
β βββ ProfilePage.jsx β
|
| 127 |
-
β βββ HistoryPage.jsx β
|
| 128 |
-
β βββ MediaHubPage.jsx β
|
| 129 |
-
β
|
| 130 |
-
β βββ SettingsPage.jsx β Application settings
|
| 131 |
βββ services/
|
| 132 |
-
βββ api.js β
|
| 133 |
```
|
| 134 |
|
| 135 |
---
|
|
@@ -182,21 +196,22 @@ Testing/ β Project root
|
|
| 182 |
|
| 183 |
### 5.1 Entry Point β `main.py`
|
| 184 |
|
| 185 |
-
`main.py` is the
|
| 186 |
|
| 187 |
**Key responsibilities:**
|
| 188 |
-
-
|
| 189 |
-
-
|
| 190 |
-
-
|
| 191 |
-
-
|
| 192 |
-
|
| 193 |
-
|
| 194 |
-
|
| 195 |
-
|
| 196 |
-
|
| 197 |
-
|
| 198 |
-
|
| 199 |
-
|
|
|
|
| 200 |
|
| 201 |
---
|
| 202 |
|
|
@@ -213,6 +228,8 @@ Uses **SQLAlchemy 2.0** with a declarative base. Database URL is loaded from the
|
|
| 213 |
| `ExecutionHistory` | `execution_history` | One row per step executed. Stores request JSON, response JSON, and AI analysis |
|
| 214 |
| `Asset` | `assets` | Uploaded files (audio/video/docs). Tagged, categorized, linked to a spec |
|
| 215 |
| `Collection` | `collections` | Named groups of assets. Many-to-many with Asset |
|
|
|
|
|
|
|
| 216 |
|
| 217 |
#### Relationships
|
| 218 |
|
|
@@ -221,6 +238,8 @@ SpecDocument ββ< Asset (one-to-many)
|
|
| 221 |
Asset >ββ< Collection (many-to-many via asset_collections junction table)
|
| 222 |
SessionModel ββ> SpecDocument (foreign key, nullable)
|
| 223 |
ExecutionHistory ββ> SessionModel (foreign key via session_id)
|
|
|
|
|
|
|
| 224 |
```
|
| 225 |
|
| 226 |
#### Connection Details
|
|
@@ -245,9 +264,15 @@ All request/response validation is done via Pydantic v2 models:
|
|
| 245 |
| `ExecutionPlan` | β Backend | Response from `/plan` β list of steps + suggested credentials |
|
| 246 |
| `StepDefinition` | β | Single step: step number, endpoint string, purpose text |
|
| 247 |
| `ExecuteStepRequest` | β Backend | Payload for `/execute-step` β session ID, step def, context vars |
|
| 248 |
-
| `AnalyzeRequest` | β Backend | Payload for `/analyze-response` β step number, response data |
|
| 249 |
-
| `AnalyzeResponse` | β Backend | AI analysis result β status,
|
| 250 |
-
| `
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 251 |
| `TokenUsage` | β Backend | LLM token counts (prompt, completion, total) |
|
| 252 |
|
| 253 |
---
|
|
@@ -277,6 +302,7 @@ All request/response validation is done via Pydantic v2 models:
|
|
| 277 |
- Returns `StepExecutionPlan` containing the complete HTTP request object (`method`, `url`, `headers`, `query_params`, `form_data`, `body`, `files`)
|
| 278 |
- **Token-efficient schema notation** (shorthand): Instead of sending verbose JSON schema, converts it to compact strings like `{name*(str),age(int),status(str[active|inactive])}`
|
| 279 |
- **Data source priority**: Context Variables β User Credentials β Step Definition
|
|
|
|
| 280 |
|
| 281 |
#### `response_analyzer.py`
|
| 282 |
- Uses `ChatMistralAI` to analyze HTTP response (status code + body)
|
|
@@ -499,9 +525,11 @@ User clicks "Execute" on a step
|
|
| 499 |
| Method | Path | Description |
|
| 500 |
|---|---|---|
|
| 501 |
| `POST` | `/chat` | Send a message; returns structured AI action + saves the conversation to DB |
|
| 502 |
-
| `GET` | `/chat
|
| 503 |
-
| `GET` | `/chat
|
| 504 |
-
| `
|
|
|
|
|
|
|
| 505 |
|
| 506 |
### History
|
| 507 |
|
|
@@ -595,6 +623,26 @@ CREATE TABLE asset_collections (
|
|
| 595 |
collection_id VARCHAR REFERENCES collections(id),
|
| 596 |
PRIMARY KEY (asset_id, collection_id)
|
| 597 |
);
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 598 |
```
|
| 599 |
|
| 600 |
---
|
|
@@ -642,8 +690,10 @@ The `ChatAssistant` maintains conversation history in the database (`ChatSession
|
|
| 642 |
| `DATABASE_URL` | β
| PostgreSQL connection string (e.g., `postgresql://user:pass@host/db`) |
|
| 643 |
| `MISTRAL_API_KEY` | β
| API key for Mistral AI (used by LangChain) |
|
| 644 |
| `SUPABASE_URL` | β¬ | Supabase project URL (enables cloud storage; falls back to local if missing) |
|
| 645 |
-
| `
|
| 646 |
-
| `
|
|
|
|
|
|
|
| 647 |
|
| 648 |
> **Note:** The `DATABASE_URL` string starting with `postgres://` is auto-converted to `postgresql://` for SQLAlchemy compatibility.
|
| 649 |
|
|
@@ -715,5 +765,5 @@ Use this table when explaining or comparing this project to similar tools:
|
|
| 715 |
|
| 716 |
---
|
| 717 |
|
| 718 |
-
*Documentation updated: April 2026*
|
| 719 |
*Project: AI API Testing Assistant | Stack: FastAPI + React + Mistral AI + PostgreSQL*
|
|
|
|
| 59 |
| Web Framework | **FastAPI** | β₯0.109 | REST API server |
|
| 60 |
| ASGI Server | **Uvicorn** | β₯0.27 | Runs FastAPI in development |
|
| 61 |
| Production Server | **Gunicorn** | 21.2 | Runs Uvicorn workers in production |
|
| 62 |
+
| LLM Orchestration | **LangChain** | β₯0.3.0 | Prompt chaining with Mistral |
|
| 63 |
| LLM Provider | **Mistral AI** | `mistral-large-latest` | Plan generation, step execution, analysis |
|
| 64 |
| HTTP Client | **httpx** | β₯0.27 | Sends actual API requests during execution |
|
| 65 |
+
| ORM | **SQLAlchemy** | β₯2.0.36 | Database queries and model definitions |
|
| 66 |
| Database | **PostgreSQL** (via Supabase) | β | Persistent storage |
|
| 67 |
+
| DB Driver | **psycopg2-binary** | 2.9.10 | PostgreSQL adapter |
|
| 68 |
| Cloud Storage | **Supabase Storage** | 2.28.2 | Stores uploaded spec files and assets |
|
| 69 |
| Spec Parsing | **prance** + **PyYAML** | β | Parse OpenAPI YAML/JSON |
|
| 70 |
| Validation | **Pydantic** | β₯2.6 | Request/response data models |
|
| 71 |
+
| Rate Limiting | **slowapi** | 0.1.9 | API request throttling |
|
| 72 |
| Auth/Config | **python-dotenv** | 1.0.0 | Load env variables from `.env` |
|
| 73 |
+
| Security | **cryptography** | 42.0.0 | Sensitive data handling |
|
| 74 |
|
| 75 |
### Frontend
|
| 76 |
|
| 77 |
| Layer | Technology | Version | Purpose |
|
| 78 |
|---|---|---|---|
|
| 79 |
+
| UI Framework | **React** | 19.2.4 | Component-based UI |
|
| 80 |
| Build Tool | **Vite** | 8.x | Fast dev server + bundler |
|
| 81 |
+
| Styling | **Tailwind CSS** | 4.2.1 | Utility-first CSS |
|
| 82 |
| Icons | **lucide-react** | 0.577 | Icon library |
|
| 83 |
| HTTP Client | **Axios** | 1.x | API communication with backend |
|
| 84 |
+
| Routing | **react-router-dom** | 7.13 | Client-side navigation |
|
| 85 |
|
| 86 |
---
|
| 87 |
|
|
|
|
| 97 |
βββ runtime.txt β Python version pin
|
| 98 |
βββ start.ps1 β PowerShell startup script
|
| 99 |
βββ openapi.yaml β Sample OpenAPI spec for testing
|
| 100 |
+
βββ documents/ β Project documentation and diagrams
|
| 101 |
β
|
| 102 |
βββ backend/ β FastAPI application
|
| 103 |
+
β βββ main.py β Entry point & app configuration
|
|
|
|
| 104 |
β βββ database.py β SQLAlchemy models + DB connection
|
| 105 |
β βββ schemas.py β Pydantic request/response models
|
| 106 |
+
β βββ app/
|
| 107 |
+
β β βββ api/
|
| 108 |
+
β β βββ routes/ β Route definitions (APIRouters)
|
| 109 |
+
β β βββ specs.py β Spec management routes
|
| 110 |
+
β β βββ assets.py β Media hub routes
|
| 111 |
+
β β βββ sessions.py β Test session routes
|
| 112 |
+
β β βββ history.py β Audit log routes
|
| 113 |
+
β β βββ chat.py β AI chat assistant routes
|
| 114 |
+
β β βββ execution.py β Core LLM execution routes
|
| 115 |
+
β βββ core/ β Shared utilities & config
|
| 116 |
+
β β βββ config.py β Settings management
|
| 117 |
+
β β βββ errors.py β Error handling logic
|
| 118 |
+
β β βββ logging.py β Centralized logger
|
| 119 |
+
β βββ integrations/ β External service clients
|
| 120 |
+
β β βββ llm/ β Mistral/LangChain integration
|
| 121 |
+
β βββ services/ β Business logic layer
|
| 122 |
+
β β βββ swagger_parser.py β OpenAPI/Postman parsing
|
| 123 |
+
β β βββ llm_planner.py β Execution plan generation
|
| 124 |
+
β β βββ api_executor.py β HTTP request construction
|
| 125 |
+
β β βββ response_analyzer.py β Semantic analysis
|
| 126 |
+
β β βββ chat_assistant.py β AI chat logic
|
| 127 |
+
β β βββ storage.py β Local/Supabase file storage
|
| 128 |
+
β βββ tests/ β Automated test suite
|
| 129 |
β
|
| 130 |
βββ frontend-react/ β React + Vite frontend
|
| 131 |
βββ index.html
|
|
|
|
| 133 |
βββ package.json
|
| 134 |
βββ src/
|
| 135 |
βββ main.jsx β React entry point
|
| 136 |
+
βββ App.jsx β Main layout + routing
|
| 137 |
+
βββ App.css / index.css β Styles and theming
|
| 138 |
+
βββ components/ β UI Components
|
| 139 |
+
β βββ AppContext.jsx β Global state management
|
| 140 |
+
β βββ CollectionsPage.jsx β Spec management
|
| 141 |
+
β βββ ProfilePage.jsx β Test user profile management
|
| 142 |
+
β βββ HistoryPage.jsx β Session history & audit log
|
| 143 |
+
β βββ MediaHubPage.jsx β Asset management
|
| 144 |
+
β βββ ChatAssistant.jsx β Persistent AI chat
|
|
|
|
| 145 |
βββ services/
|
| 146 |
+
βββ api.js β Axios service layer
|
| 147 |
```
|
| 148 |
|
| 149 |
---
|
|
|
|
| 196 |
|
| 197 |
### 5.1 Entry Point β `main.py`
|
| 198 |
|
| 199 |
+
`main.py` is the entry point for the FastAPI application. It initializes the app, ensures directory structures, and mounts the various module routers.
|
| 200 |
|
| 201 |
**Key responsibilities:**
|
| 202 |
+
- **Rate Limiting**: Uses `slowapi` to protect endpoints (default 200/min).
|
| 203 |
+
- **Global Error Handling**: Custom handlers for `AppError` and `RequestValidationError`.
|
| 204 |
+
- **CORS Configuration**: Loaded from `settings`.
|
| 205 |
+
- **Static Files**: Serves the `/uploads` directory.
|
| 206 |
+
- **Router Orchestration**: Includes routers from `backend.app.api.routes`.
|
| 207 |
+
|
| 208 |
+
**Modules (Routers):**
|
| 209 |
+
- `specs`: Spec upload, retrieval, and credential management.
|
| 210 |
+
- `assets`: Media Hub file operations.
|
| 211 |
+
- `sessions`: Test session lifecycle.
|
| 212 |
+
- `history`: Audit log access.
|
| 213 |
+
- `chat`: Persistent AI chat interactions.
|
| 214 |
+
- `execution`: Core LLM pipeline (`/plan`, `/execute-step`, `/analyze-response`).
|
| 215 |
|
| 216 |
---
|
| 217 |
|
|
|
|
| 228 |
| `ExecutionHistory` | `execution_history` | One row per step executed. Stores request JSON, response JSON, and AI analysis |
|
| 229 |
| `Asset` | `assets` | Uploaded files (audio/video/docs). Tagged, categorized, linked to a spec |
|
| 230 |
| `Collection` | `collections` | Named groups of assets. Many-to-many with Asset |
|
| 231 |
+
| `ChatSession` | `chat_sessions` | A persistent AI chat thread. Linked to a spec and optionally to a test session. Auto-named from first message. |
|
| 232 |
+
| `ChatMessage` | `chat_messages` | A single message within a chat session β stores role, content, and the structured `action_json` returned by the AI |
|
| 233 |
|
| 234 |
#### Relationships
|
| 235 |
|
|
|
|
| 238 |
Asset >ββ< Collection (many-to-many via asset_collections junction table)
|
| 239 |
SessionModel ββ> SpecDocument (foreign key, nullable)
|
| 240 |
ExecutionHistory ββ> SessionModel (foreign key via session_id)
|
| 241 |
+
ChatSession ββ> SpecDocument (foreign key, ON DELETE SET NULL, nullable)
|
| 242 |
+
ChatMessage ββ> ChatSession (foreign key, ON DELETE CASCADE)
|
| 243 |
```
|
| 244 |
|
| 245 |
#### Connection Details
|
|
|
|
| 264 |
| `ExecutionPlan` | β Backend | Response from `/plan` β list of steps + suggested credentials |
|
| 265 |
| `StepDefinition` | β | Single step: step number, endpoint string, purpose text |
|
| 266 |
| `ExecuteStepRequest` | β Backend | Payload for `/execute-step` β session ID, step def, context vars |
|
| 267 |
+
| `AnalyzeRequest` | β Backend | Payload for `/analyze-response` β step number, response data, optional `spec_id` and `step_definition` |
|
| 268 |
+
| `AnalyzeResponse` | β Backend | AI analysis result β `status`, `is_otp_required`, `otp_field_name`, `is_selection_required`, `selection_field_name`, `selection_options`, `response_summary`, `extracted_variables`, `token_usage` |
|
| 269 |
+
| `SelectionOption` | β | A single choosable item when `is_selection_required=true` β `label` (display text) and `value` (ID to save) |
|
| 270 |
+
| `ContextUpdate` | β Backend | Payload for `POST /context/{session_id}` β `key` and `value` to merge into session context |
|
| 271 |
+
| `CredentialUpdate` | β Backend | Payload for credential save β `name`, `username`, `password`, `extra` dict. Passing `extra._clear=true` **wipes all stored credentials** for the spec. |
|
| 272 |
+
| `ChatRequest` | β Backend | Payload for `/chat` β message text, optional `chat_session_id`, spec context, execution plan, step results |
|
| 273 |
+
| `ChatResponse` | β Backend | Structured AI response β `chat_session_id`, `chat_session_name`, `action`, `message`, optional `step`, `changes`, `assertions`, `explanation` |
|
| 274 |
+
| `ChatSessionSummary` | β Backend | Summary of a chat session β id, name, spec_id, session_id, message count, timestamps |
|
| 275 |
+
| `ChatMessageSchema` | β Backend | A single chat message β role, content, action_json, timestamp |
|
| 276 |
| `TokenUsage` | β Backend | LLM token counts (prompt, completion, total) |
|
| 277 |
|
| 278 |
---
|
|
|
|
| 302 |
- Returns `StepExecutionPlan` containing the complete HTTP request object (`method`, `url`, `headers`, `query_params`, `form_data`, `body`, `files`)
|
| 303 |
- **Token-efficient schema notation** (shorthand): Instead of sending verbose JSON schema, converts it to compact strings like `{name*(str),age(int),status(str[active|inactive])}`
|
| 304 |
- **Data source priority**: Context Variables β User Credentials β Step Definition
|
| 305 |
+
- **Asset Integration**: Consults `asset_service` to provide the LLM with a list of available files from the Media Hub.
|
| 306 |
|
| 307 |
#### `response_analyzer.py`
|
| 308 |
- Uses `ChatMistralAI` to analyze HTTP response (status code + body)
|
|
|
|
| 525 |
| Method | Path | Description |
|
| 526 |
|---|---|---|
|
| 527 |
| `POST` | `/chat` | Send a message; returns structured AI action + saves the conversation to DB |
|
| 528 |
+
| `GET` | `/chat-sessions` | List all chat sessions, optionally filtered by `spec_id` |
|
| 529 |
+
| `GET` | `/chat-sessions/{chat_session_id}` | Get a single chat session with its full message history |
|
| 530 |
+
| `PATCH` | `/chat-sessions/{chat_session_id}` | Rename a chat session |
|
| 531 |
+
| `DELETE` | `/chat-sessions/{chat_session_id}` | Delete a chat session and all its messages (cascade) |
|
| 532 |
+
| `DELETE` | `/chat-sessions` | Wipe **all** chat sessions and messages |
|
| 533 |
|
| 534 |
### History
|
| 535 |
|
|
|
|
| 623 |
collection_id VARCHAR REFERENCES collections(id),
|
| 624 |
PRIMARY KEY (asset_id, collection_id)
|
| 625 |
);
|
| 626 |
+
|
| 627 |
+
-- AI Chat sessions (each represents one conversation thread)
|
| 628 |
+
CREATE TABLE chat_sessions (
|
| 629 |
+
id VARCHAR PRIMARY KEY, -- UUID
|
| 630 |
+
name VARCHAR NOT NULL, -- auto-named from first message; user-renameable
|
| 631 |
+
spec_id VARCHAR REFERENCES spec_documents(id) ON DELETE SET NULL,
|
| 632 |
+
session_id VARCHAR, -- linked API test session (nullable)
|
| 633 |
+
created_at TIMESTAMP DEFAULT NOW(),
|
| 634 |
+
updated_at TIMESTAMP DEFAULT NOW()
|
| 635 |
+
);
|
| 636 |
+
|
| 637 |
+
-- Individual messages within a chat session
|
| 638 |
+
CREATE TABLE chat_messages (
|
| 639 |
+
id SERIAL PRIMARY KEY,
|
| 640 |
+
chat_session_id VARCHAR REFERENCES chat_sessions(id) ON DELETE CASCADE NOT NULL,
|
| 641 |
+
role VARCHAR NOT NULL, -- 'user' | 'assistant'
|
| 642 |
+
content TEXT NOT NULL, -- raw message text
|
| 643 |
+
action_json JSON, -- structured action returned by AI (execute_step, explain, etc.)
|
| 644 |
+
timestamp TIMESTAMP DEFAULT NOW()
|
| 645 |
+
);
|
| 646 |
```
|
| 647 |
|
| 648 |
---
|
|
|
|
| 690 |
| `DATABASE_URL` | β
| PostgreSQL connection string (e.g., `postgresql://user:pass@host/db`) |
|
| 691 |
| `MISTRAL_API_KEY` | β
| API key for Mistral AI (used by LangChain) |
|
| 692 |
| `SUPABASE_URL` | β¬ | Supabase project URL (enables cloud storage; falls back to local if missing) |
|
| 693 |
+
| `SUPABASE_SERVICE_ROLE_KEY` | β¬ | **Preferred** Supabase key β bypasses Row-Level Security for storage operations |
|
| 694 |
+
| `SUPABASE_SECRET_KEY` | β¬ | Alternative Supabase key name (checked if `SUPABASE_SERVICE_ROLE_KEY` is absent) |
|
| 695 |
+
| `SUPABASE_KEY` | β¬ | Fallback Supabase key (used only if the two above are not set) |
|
| 696 |
+
| `SUPABASE_STORAGE_BUCKET` | β¬ | Name of the Supabase Storage bucket (defaults to `"assets"` if not set) |
|
| 697 |
|
| 698 |
> **Note:** The `DATABASE_URL` string starting with `postgres://` is auto-converted to `postgresql://` for SQLAlchemy compatibility.
|
| 699 |
|
|
|
|
| 765 |
|
| 766 |
---
|
| 767 |
|
| 768 |
+
*Documentation updated: April 2026 β reflects actual codebase state as of this date*
|
| 769 |
*Project: AI API Testing Assistant | Stack: FastAPI + React + Mistral AI + PostgreSQL*
|
|
@@ -8,49 +8,45 @@ This document contains ready-to-use **Mermaid diagram code** for all major diagr
|
|
| 8 |
*Shows the high-level structure of the entire system and how the four tiers interact.*
|
| 9 |
|
| 10 |
```mermaid
|
| 11 |
-
graph
|
| 12 |
-
subgraph "
|
| 13 |
-
|
| 14 |
-
|
| 15 |
-
|
| 16 |
-
|
| 17 |
-
|
| 18 |
-
M5["Media Hub"]
|
| 19 |
-
M6["AI Chat Assistant"]
|
| 20 |
-
end
|
| 21 |
-
|
| 22 |
-
subgraph "Orchestration Tier"
|
| 23 |
-
API["FastAPI Backend<br/>(Uvicorn on port 8001)"]
|
| 24 |
-
CORS["CORS Middleware"]
|
| 25 |
-
VALID["Request Validation<br/>(Pydantic)"]
|
| 26 |
end
|
| 27 |
|
| 28 |
-
subgraph "
|
| 29 |
-
|
| 30 |
-
|
| 31 |
-
|
| 32 |
-
|
| 33 |
-
|
| 34 |
end
|
| 35 |
|
| 36 |
-
subgraph "
|
| 37 |
-
|
| 38 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 39 |
end
|
| 40 |
|
| 41 |
-
subgraph "
|
| 42 |
-
|
|
|
|
|
|
|
|
|
|
| 43 |
end
|
| 44 |
|
| 45 |
-
|
| 46 |
-
|
| 47 |
-
API --> LLM2
|
| 48 |
-
API --> LLM3
|
| 49 |
-
API --> LLM4
|
| 50 |
-
LLM1 & LLM2 & LLM3 & LLM4 --> MISTRAL
|
| 51 |
-
API --> PG
|
| 52 |
-
API --> SUP
|
| 53 |
-
LLM2 --> TARGET
|
| 54 |
```
|
| 55 |
|
| 56 |
---
|
|
@@ -60,50 +56,43 @@ graph TB
|
|
| 60 |
|
| 61 |
```mermaid
|
| 62 |
sequenceDiagram
|
| 63 |
-
|
| 64 |
-
participant
|
| 65 |
-
participant
|
| 66 |
-
participant DB as
|
| 67 |
-
participant LLM as
|
| 68 |
-
participant
|
| 69 |
-
|
| 70 |
-
User->>
|
| 71 |
-
|
| 72 |
-
|
| 73 |
-
|
| 74 |
-
|
| 75 |
-
|
| 76 |
-
|
| 77 |
-
|
| 78 |
-
|
| 79 |
-
|
| 80 |
-
|
| 81 |
-
LLM-->>BE: {execution_plan[], suggested_credentials}
|
| 82 |
-
BE->>DB: UPDATE SpecDocument (plan, merged creds)
|
| 83 |
-
BE-->>FE: {execution_plan[]}
|
| 84 |
-
|
| 85 |
-
loop For each step in plan
|
| 86 |
-
User->>FE: Click "Execute Step N"
|
| 87 |
-
FE->>BE: POST /execute-step {step_def, session_id}
|
| 88 |
-
BE->>DB: GET full spec schema
|
| 89 |
-
BE->>BE: resolve_refs() + prune + shorthand
|
| 90 |
-
BE->>DB: GET context_variables for session
|
| 91 |
-
BE->>LLM: prepare_step_request(schema, context, creds)
|
| 92 |
-
LLM-->>BE: {StepExecutionPlan}
|
| 93 |
-
BE->>BE: sanitize_url()
|
| 94 |
-
BE->>API: HTTP request (method, url, headers, body)
|
| 95 |
-
API-->>BE: {status_code, body}
|
| 96 |
-
BE->>DB: INSERT ExecutionHistory (request + response)
|
| 97 |
-
BE-->>FE: {request, response}
|
| 98 |
-
|
| 99 |
-
FE->>BE: POST /analyze-response {step, response}
|
| 100 |
-
BE->>LLM: analyze_api_response(status, body, purpose)
|
| 101 |
-
LLM-->>BE: {status, extracted_variables, is_otp_required}
|
| 102 |
-
BE->>DB: UPDATE context_variables (merge new vars)
|
| 103 |
-
BE->>DB: UPDATE ExecutionHistory (analysis)
|
| 104 |
-
BE-->>FE: {analysis_result}
|
| 105 |
-
FE-->>User: Show result + extracted variables
|
| 106 |
end
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 107 |
```
|
| 108 |
|
| 109 |
---
|
|
@@ -113,8 +102,14 @@ sequenceDiagram
|
|
| 113 |
|
| 114 |
```mermaid
|
| 115 |
erDiagram
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 116 |
SPEC_DOCUMENT {
|
| 117 |
-
|
| 118 |
string name
|
| 119 |
text raw_content
|
| 120 |
json parsed_json
|
|
@@ -122,44 +117,32 @@ erDiagram
|
|
| 122 |
json login_credentials
|
| 123 |
json execution_plan
|
| 124 |
text scenario
|
| 125 |
-
string file_path
|
| 126 |
-
datetime uploaded_at
|
| 127 |
}
|
| 128 |
|
| 129 |
SESSION_MODEL {
|
| 130 |
-
|
| 131 |
-
|
| 132 |
-
string spec_name
|
| 133 |
-
string base_url
|
| 134 |
-
text scenario
|
| 135 |
json context_variables
|
| 136 |
-
|
| 137 |
}
|
| 138 |
|
| 139 |
EXECUTION_HISTORY {
|
| 140 |
int id PK
|
| 141 |
-
|
| 142 |
int step
|
| 143 |
json request_json
|
| 144 |
json response_json
|
| 145 |
json analysis_json
|
| 146 |
json step_definition_json
|
| 147 |
-
datetime timestamp
|
| 148 |
}
|
| 149 |
|
| 150 |
ASSET {
|
| 151 |
-
|
| 152 |
string name
|
| 153 |
string file_type
|
| 154 |
string file_path
|
| 155 |
-
int file_size
|
| 156 |
string category
|
| 157 |
-
text description
|
| 158 |
json tags
|
| 159 |
-
json metadata_json
|
| 160 |
-
string spec_id FK
|
| 161 |
-
datetime uploaded_at
|
| 162 |
-
}
|
| 163 |
|
| 164 |
COLLECTION {
|
| 165 |
string id PK
|
|
|
|
| 8 |
*Shows the high-level structure of the entire system and how the four tiers interact.*
|
| 9 |
|
| 10 |
```mermaid
|
| 11 |
+
graph TD
|
| 12 |
+
subgraph "Frontend (React + Vite)"
|
| 13 |
+
UI[User Interface]
|
| 14 |
+
State[AppContext / State]
|
| 15 |
+
Axios[Axios API Client]
|
| 16 |
+
UI --> State
|
| 17 |
+
State --> Axios
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 18 |
end
|
| 19 |
|
| 20 |
+
subgraph "Orchestration (FastAPI)"
|
| 21 |
+
Router[main.py / APIRouters]
|
| 22 |
+
Core[Core Utilities / Config]
|
| 23 |
+
Services[Business Logic / Services]
|
| 24 |
+
Router --> Core
|
| 25 |
+
Router --> Services
|
| 26 |
end
|
| 27 |
|
| 28 |
+
subgraph "Intelligence (LLM Services)"
|
| 29 |
+
Planner[LLM Planner]
|
| 30 |
+
Executor[API Executor]
|
| 31 |
+
Analyzer[Response Analyzer]
|
| 32 |
+
Chat[Chat Assistant]
|
| 33 |
+
Integrations[Integrations / LLM]
|
| 34 |
+
Services --> Planner
|
| 35 |
+
Services --> Executor
|
| 36 |
+
Services --> Analyzer
|
| 37 |
+
Services --> Chat
|
| 38 |
+
Planner & Executor & Analyzer & Chat --> Integrations
|
| 39 |
end
|
| 40 |
|
| 41 |
+
subgraph "Data & Storage"
|
| 42 |
+
DB[(PostgreSQL / SQLAlchemy)]
|
| 43 |
+
Storage[Supabase / Local Storage]
|
| 44 |
+
Services --> DB
|
| 45 |
+
Services --> Storage
|
| 46 |
end
|
| 47 |
|
| 48 |
+
Axios <--> Router
|
| 49 |
+
Integrations <--> LLM_Provider[Mistral-Large API]
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 50 |
```
|
| 51 |
|
| 52 |
---
|
|
|
|
| 56 |
|
| 57 |
```mermaid
|
| 58 |
sequenceDiagram
|
| 59 |
+
participant User
|
| 60 |
+
participant Frontend
|
| 61 |
+
participant FastAPI as router/execution.py
|
| 62 |
+
participant DB as database.py
|
| 63 |
+
participant LLM as integrations/llm
|
| 64 |
+
participant TargetAPI
|
| 65 |
+
|
| 66 |
+
User->>Frontend: Clicks "Execute Step"
|
| 67 |
+
Frontend->>FastAPI: POST /execute-step (StepDef)
|
| 68 |
+
|
| 69 |
+
rect rgb(240, 240, 240)
|
| 70 |
+
Note over FastAPI: PRE-EXECUTION PIPELINE
|
| 71 |
+
FastAPI->>DB: Fetch Spec (parsed_json)
|
| 72 |
+
FastAPI->>FastAPI: resolve_refs() + prune + shorthand
|
| 73 |
+
FastAPI->>DB: Fetch session context + profile
|
| 74 |
+
FastAPI->>LLM: prepare_step_request(schema, context)
|
| 75 |
+
LLM-->>FastAPI: {APIRequest}
|
| 76 |
+
FastAPI->>FastAPI: sanitize_url(url)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 77 |
end
|
| 78 |
+
|
| 79 |
+
rect rgb(220, 240, 220)
|
| 80 |
+
Note over FastAPI: HTTP EXECUTION
|
| 81 |
+
FastAPI->>TargetAPI: httpx.send(request)
|
| 82 |
+
TargetAPI-->>FastAPI: {status, body, headers}
|
| 83 |
+
end
|
| 84 |
+
|
| 85 |
+
FastAPI->>DB: Save to ExecutionHistory
|
| 86 |
+
FastAPI-->>Frontend: {request, response}
|
| 87 |
+
Frontend->>User: Display Result (Cards)
|
| 88 |
+
|
| 89 |
+
Frontend->>FastAPI: POST /analyze-response {step, response}
|
| 90 |
+
FastAPI->>LLM: analyze_api_response(status, body, purpose)
|
| 91 |
+
LLM-->>FastAPI: {status, extracted_variables, is_otp_required}
|
| 92 |
+
FastAPI->>DB: UPDATE context_variables (merge new vars)
|
| 93 |
+
FastAPI->>DB: UPDATE ExecutionHistory (analysis)
|
| 94 |
+
FastAPI-->>Frontend: {analysis_result}
|
| 95 |
+
Frontend-->>User: Show result + extracted variables
|
| 96 |
```
|
| 97 |
|
| 98 |
---
|
|
|
|
| 102 |
|
| 103 |
```mermaid
|
| 104 |
erDiagram
|
| 105 |
+
SPEC_DOCUMENT ||--o{ SESSION_MODEL : "has"
|
| 106 |
+
SPEC_DOCUMENT ||--o{ ASSET : "linked to"
|
| 107 |
+
SESSION_MODEL ||--o{ EXECUTION_HISTORY : "logs"
|
| 108 |
+
CHAT_SESSION ||--o{ CHAT_MESSAGE : "contains"
|
| 109 |
+
SPEC_DOCUMENT ||--o{ CHAT_SESSION : "associated with"
|
| 110 |
+
|
| 111 |
SPEC_DOCUMENT {
|
| 112 |
+
uuid id PK
|
| 113 |
string name
|
| 114 |
text raw_content
|
| 115 |
json parsed_json
|
|
|
|
| 117 |
json login_credentials
|
| 118 |
json execution_plan
|
| 119 |
text scenario
|
|
|
|
|
|
|
| 120 |
}
|
| 121 |
|
| 122 |
SESSION_MODEL {
|
| 123 |
+
uuid id PK
|
| 124 |
+
uuid spec_id FK
|
|
|
|
|
|
|
|
|
|
| 125 |
json context_variables
|
| 126 |
+
string base_url
|
| 127 |
}
|
| 128 |
|
| 129 |
EXECUTION_HISTORY {
|
| 130 |
int id PK
|
| 131 |
+
uuid session_id FK
|
| 132 |
int step
|
| 133 |
json request_json
|
| 134 |
json response_json
|
| 135 |
json analysis_json
|
| 136 |
json step_definition_json
|
|
|
|
| 137 |
}
|
| 138 |
|
| 139 |
ASSET {
|
| 140 |
+
uuid id PK
|
| 141 |
string name
|
| 142 |
string file_type
|
| 143 |
string file_path
|
|
|
|
| 144 |
string category
|
|
|
|
| 145 |
json tags
|
|
|
|
|
|
|
|
|
|
|
|
|
| 146 |
|
| 147 |
COLLECTION {
|
| 148 |
string id PK
|
|
@@ -4,11 +4,13 @@ An automated API testing system powered by Mistral AI, FastAPI, and React. This
|
|
| 4 |
|
| 5 |
## Features
|
| 6 |
|
| 7 |
-
- **
|
| 8 |
-
- **
|
| 9 |
-
- **
|
| 10 |
-
- **
|
| 11 |
-
- **
|
|
|
|
|
|
|
| 12 |
|
| 13 |
## Requirements
|
| 14 |
|
|
@@ -57,6 +59,14 @@ cd frontend-react && npm run dev
|
|
| 57 |
| Backend | http://localhost:8001 |
|
| 58 |
| API Docs | http://localhost:8001/docs |
|
| 59 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 60 |
## Usage
|
| 61 |
|
| 62 |
1. Open the frontend at `http://localhost:5173`.
|
|
|
|
| 4 |
|
| 5 |
## Features
|
| 6 |
|
| 7 |
+
- **Intelligent Planning**: Generates multi-step test scenarios from natural language.
|
| 8 |
+
- **Context-Aware Execution**: Automatically propagates tokens and variables (IDs, slugs, etc.) between steps.
|
| 9 |
+
- **Persistent AI Chat**: An interactive assistant that understands your full testing context and can control the workflow.
|
| 10 |
+
- **Media Hub**: Manage assets (images, PDFs, audio) for API requests requiring file uploads.
|
| 11 |
+
- **Zero-Code Testing**: Describe your intent (e.g., "Register, login, and update my profile picture") and let the AI handle the rest.
|
| 12 |
+
- **Format Agnostic**: Supports OpenAPI (Swagger) and Postman Collections.
|
| 13 |
+
- **Clean Organization**: Documentation is centrally managed in this `documents/` folder.
|
| 14 |
|
| 15 |
## Requirements
|
| 16 |
|
|
|
|
| 59 |
| Backend | http://localhost:8001 |
|
| 60 |
| API Docs | http://localhost:8001/docs |
|
| 61 |
|
| 62 |
+
## π Project Structure
|
| 63 |
+
|
| 64 |
+
- `backend/`: FastAPI server with LLM pipeline logic.
|
| 65 |
+
- `frontend-react/`: Modern React dashboard built with Vite and Tailwind CSS.
|
| 66 |
+
- `documents/`: Comprehensive documentation, diagrams, and research notes (you are here).
|
| 67 |
+
- `start.ps1`: One-click startup script for Windows.
|
| 68 |
+
- `project_for_review.zip`: Optimized archive for sharing the project with reviewers.
|
| 69 |
+
|
| 70 |
## Usage
|
| 71 |
|
| 72 |
1. Open the frontend at `http://localhost:5173`.
|
|
@@ -2,7 +2,7 @@
|
|
| 2 |
|
| 3 |
**Author:** [Your Name]
|
| 4 |
**Affiliation:** [Your Institution / Department of Computer Science]
|
| 5 |
-
**Date:**
|
| 6 |
|
| 7 |
---
|
| 8 |
|
|
@@ -80,7 +80,7 @@ Despite progress in individual components, a gap exists in the integration of LL
|
|
| 80 |
|
| 81 |
## 3. Methodology
|
| 82 |
|
| 83 |
-
This section provides a comprehensive and technically detailed description of the system's design philosophy, data architecture, algorithmic components, prompt engineering strategies, and end-to-end execution workflow. The system is founded on a novel **
|
| 84 |
|
| 85 |
---
|
| 86 |
|
|
@@ -643,15 +643,15 @@ Both entry points share the same backend and conversation history, providing a s
|
|
| 643 |
| Layer | Technology | Version / Notes |
|
| 644 |
|---|---|---|
|
| 645 |
| Backend Language | Python | 3.10+ |
|
| 646 |
-
| API Framework | FastAPI | β₯ 0.109.0 (async,
|
| 647 |
-
| LLM Orchestration | LangChain
|
| 648 |
-
| LLM Model | Mistral Large
|
| 649 |
| HTTP Client | httpx | β₯ 0.27.2 (async) |
|
| 650 |
-
| ORM | SQLAlchemy | 2.0.
|
| 651 |
| Database | PostgreSQL (via Supabase) | + `psycopg2-binary` driver |
|
| 652 |
| File Storage | Supabase Storage | SDK v2.28.2 |
|
| 653 |
| Frontend Framework | React + Vite | ES6+, SWC compiler |
|
| 654 |
-
| Frontend Styling | Tailwind CSS |
|
| 655 |
| ASGI Server | Uvicorn (dev) / Gunicorn (prod) | With standard extras |
|
| 656 |
|
| 657 |
### 4.2 FastAPI Endpoint Design
|
|
|
|
| 2 |
|
| 3 |
**Author:** [Your Name]
|
| 4 |
**Affiliation:** [Your Institution / Department of Computer Science]
|
| 5 |
+
**Date:** May 2026
|
| 6 |
|
| 7 |
---
|
| 8 |
|
|
|
|
| 80 |
|
| 81 |
## 3. Methodology
|
| 82 |
|
| 83 |
+
This section provides a comprehensive and technically detailed description of the system's design philosophy, data architecture, algorithmic components, prompt engineering strategies, and end-to-end execution workflow. The system is founded on a novel **Four-Stage AI Pipeline** β Strategic Planning, Context-Aware Execution, Semantic Response Analysis, and Conversational AI Assistance β underpinned by a persistent, session-scoped state management system and a modular service architecture.
|
| 84 |
|
| 85 |
---
|
| 86 |
|
|
|
|
| 643 |
| Layer | Technology | Version / Notes |
|
| 644 |
|---|---|---|
|
| 645 |
| Backend Language | Python | 3.10+ |
|
| 646 |
+
| API Framework | FastAPI | β₯ 0.109.0 (async, routers) |
|
| 647 |
+
| LLM Orchestration | LangChain | β₯ 0.3.0 |
|
| 648 |
+
| LLM Model | Mistral Large | `mistral-large-latest`, temp=0 |
|
| 649 |
| HTTP Client | httpx | β₯ 0.27.2 (async) |
|
| 650 |
+
| ORM | SQLAlchemy | β₯ 2.0.36 |
|
| 651 |
| Database | PostgreSQL (via Supabase) | + `psycopg2-binary` driver |
|
| 652 |
| File Storage | Supabase Storage | SDK v2.28.2 |
|
| 653 |
| Frontend Framework | React + Vite | ES6+, SWC compiler |
|
| 654 |
+
| Frontend Styling | Tailwind CSS | 4.x Modern Design |
|
| 655 |
| ASGI Server | Uvicorn (dev) / Gunicorn (prod) | With standard extras |
|
| 656 |
|
| 657 |
### 4.2 FastAPI Endpoint Design
|