Amitkasar commited on
Commit
ef865f2
Β·
1 Parent(s): 6270070

Refactor backend: split routes, add services & tests

Browse files

Restructure the backend into modular API route files and core services. Added new FastAPI route modules (assets, chat, execution, history, sessions, specs) and supporting core modules (config, logging, errors, url_guard) plus LLM integration client and many service updates (api_executor, chat_assistant, llm_planner, response_analyzer). Implemented SSRF protection, rate limiting, spec upload/parse, execution planning and analysis, media hub asset handling, session/context endpoints, and exportable history. Added tests, pytest.ini, updated requirements and .gitignore, removed .env.example, and added documentation/plans/screenshots and upload placeholders. Overall prepares the project for clearer separation of concerns, improved security, and test coverage.

This view is limited to 50 files because it contains too many changes. Β  See raw diff
Files changed (50) hide show
  1. .claude/settings.local.json +13 -0
  2. .env.example +0 -27
  3. .gitignore +40 -13
  4. frontend-react/src/assets/hero.png β†’ Current UI/Screenshot 2026-05-01 135942.png +2 -2
  5. Current UI/Screenshot 2026-05-01 140023.png +3 -0
  6. Current UI/Screenshot 2026-05-01 140040.png +3 -0
  7. Current UI/Screenshot 2026-05-01 140058.png +3 -0
  8. Current UI/Screenshot 2026-05-01 140111.png +3 -0
  9. Current UI/Screenshot 2026-05-01 140124.png +3 -0
  10. Current UI/Screenshot 2026-05-01 140140.png +3 -0
  11. Old Chats claude/1.md +2 -0
  12. Plans/plan 1.md +231 -0
  13. Plans/plan 2.md +161 -0
  14. backend/app/__init__.py +0 -0
  15. backend/app/api/__init__.py +0 -0
  16. backend/app/api/routes/__init__.py +0 -0
  17. backend/app/api/routes/assets.py +109 -0
  18. backend/app/api/routes/chat.py +183 -0
  19. backend/app/api/routes/execution.py +343 -0
  20. backend/app/api/routes/history.py +93 -0
  21. backend/app/api/routes/sessions.py +45 -0
  22. backend/app/api/routes/specs.py +163 -0
  23. backend/core/__init__.py +0 -0
  24. backend/core/config.py +19 -0
  25. backend/core/errors.py +19 -0
  26. backend/core/logging.py +15 -0
  27. backend/core/url_guard.py +50 -0
  28. backend/integrations/__init__.py +0 -0
  29. backend/integrations/llm/__init__.py +0 -0
  30. backend/integrations/llm/client.py +34 -0
  31. backend/main.py +43 -913
  32. backend/services/api_executor.py +4 -17
  33. backend/services/chat_assistant.py +5 -7
  34. backend/services/llm_planner.py +13 -17
  35. backend/services/response_analyzer.py +11 -16
  36. backend/tests/__init__.py +0 -0
  37. backend/tests/conftest.py +44 -0
  38. backend/tests/test_api_executor.py +78 -0
  39. backend/tests/test_llm_planner.py +71 -0
  40. backend/tests/test_routes.py +140 -0
  41. backend/tests/test_url_guard.py +75 -0
  42. backend/uploads/audio/.gitkeep +0 -0
  43. backend/uploads/documents/.gitkeep +0 -0
  44. backend/uploads/images/.gitkeep +0 -0
  45. backend/uploads/specs/.gitkeep +0 -0
  46. backend/uploads/video/.gitkeep +0 -0
  47. PROJECT_DOCUMENTATION.md β†’ documents/PROJECT_DOCUMENTATION.md +101 -51
  48. Project_Diagrams.md β†’ documents/Project_Diagrams.md +79 -96
  49. README.md β†’ documents/README.md +15 -5
  50. Research_Paper.md β†’ documents/Research_Paper.md +7 -7
.claude/settings.local.json ADDED
@@ -0,0 +1,13 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "permissions": {
3
+ "allow": [
4
+ "Bash(pip install *)",
5
+ "Bash(python -c \"from backend.main import app; print\\('OK β€” routes:', len\\(app.routes\\)\\)\")",
6
+ "Bash(python -c ' *)",
7
+ "Bash(python -c \"from backend.main import app; from backend.core.url_guard import assert_safe_url; print\\('OK routes:', len\\(app.routes\\)\\)\")",
8
+ "Bash(python -c \"from backend.main import app; from backend.core.errors import AppError; print\\('OK routes:', len\\(app.routes\\)\\)\")",
9
+ "Bash(Get-ChildItem -Path \"C:\\\\Users\\\\amitk\\\\OneDrive\\\\Desktop\\\\Testing\" -Recurse -Directory)",
10
+ "Bash(Select-Object -ExpandProperty FullName)"
11
+ ]
12
+ }
13
+ }
.env.example DELETED
@@ -1,27 +0,0 @@
1
- # AI API Testing Tool - Environment Variables
2
-
3
- # Mistral AI configuration
4
- MISTRAL_API_KEY=your_mistral_api_key_here
5
- MISTRAL_MODEL=mistral-large-latest
6
-
7
- # Backend configuration (Local default: 8001)
8
- # On Render, the PORT will be assigned automatically.
9
- PORT=8001
10
-
11
- # Database Configuration (Supabase PostgreSQL)
12
- # Get your Connection String from Supabase -> Project Settings -> Database
13
- DATABASE_URL=postgresql://postgres.your-project:your-password@aws-0-region.pooler.supabase.com:5432/postgres
14
-
15
- # Supabase Storage & Client Configuration
16
- # Get these from Supabase -> Project Settings -> API
17
- SUPABASE_URL=https://your-project.supabase.co
18
- # Use the Service Role Key for backend storage access
19
- SUPABASE_SERVICE_ROLE_KEY=your-service-role-key-here
20
- SUPABASE_STORAGE_BUCKET=assets
21
-
22
- # Frontend Configuration (Vite)
23
- # Points to the Backend URL.
24
- # Local: http://localhost:8001
25
- # Render: https://your-backend.onrender.com
26
- VITE_BACKEND_URL=http://localhost:8001
27
- VITE_API_URL=http://localhost:8001
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
.gitignore CHANGED
@@ -1,36 +1,63 @@
1
- # Dependencies
2
  node_modules/
3
  dist/
4
  build/
 
5
 
6
- # Python
7
  __pycache__/
8
- *.pyc
9
- *.pyo
10
  *.pyd
11
- .Python
 
 
12
  env/
13
  venv/
 
14
  pip-log.txt
15
- pip-delete-this-directory.txt
16
 
17
- # Databases
18
- *.db
19
- *.sqlite3
 
 
20
 
21
- # Environment variables
22
  .env
 
 
 
 
 
 
 
 
 
 
23
 
24
- # Logs
 
 
 
 
 
 
 
25
  *.log
 
26
 
27
- # OS files
28
  .DS_Store
29
  Thumbs.db
30
  desktop.ini
 
31
 
32
- # IDEs
33
  .vscode/
34
  .idea/
35
  *.swp
36
  *.swo
 
 
 
 
 
1
+ # ── Node ─────────────────────────────────────────
2
  node_modules/
3
  dist/
4
  build/
5
+ .npm/
6
 
7
+ # ── Python ───────────────────────────────────────
8
  __pycache__/
9
+ *.py[cod]
 
10
  *.pyd
11
+ *.pyo
12
+ *.egg-info/
13
+ .eggs/
14
  env/
15
  venv/
16
+ .venv/
17
  pip-log.txt
 
18
 
19
+ # ── Testing ──────────────────────────────────────
20
+ .pytest_cache/
21
+ htmlcov/
22
+ .coverage
23
+ coverage.xml
24
 
25
+ # ── Environment & secrets ─────────────────────────
26
  .env
27
+ .env.*
28
+ !.env.example
29
+
30
+ # ── Uploaded user files ───────────────────────────
31
+ backend/uploads/audio/*
32
+ backend/uploads/video/*
33
+ backend/uploads/documents/*
34
+ backend/uploads/images/*
35
+ backend/uploads/specs/*
36
+ !backend/uploads/**/.gitkeep
37
 
38
+ # ── Archives & temp artifacts ─────────────────────
39
+ *.zip
40
+ *.tar.gz
41
+ *.rar
42
+ temp_review/
43
+ tmp_*/
44
+
45
+ # ── Logs ─────────────────────────────────────────
46
  *.log
47
+ logs/
48
 
49
+ # ── OS ───────────────────────────────────────────
50
  .DS_Store
51
  Thumbs.db
52
  desktop.ini
53
+ ehthumbs.db
54
 
55
+ # ── IDEs ─────────────────────────────────────────
56
  .vscode/
57
  .idea/
58
  *.swp
59
  *.swo
60
+
61
+ # ── Databases ────────────────────────────────────
62
+ *.db
63
+ *.sqlite3
frontend-react/src/assets/hero.png β†’ Current UI/Screenshot 2026-05-01 135942.png RENAMED
File without changes
Current UI/Screenshot 2026-05-01 140023.png ADDED

Git LFS Details

  • SHA256: c47a749977cf7f6e459492d77ee276198f36afea293428ab98220668ba098744
  • Pointer size: 130 Bytes
  • Size of remote file: 96 kB
Current UI/Screenshot 2026-05-01 140040.png ADDED

Git LFS Details

  • SHA256: 88b8441a748c59f39d1dc7c63d771b43ed23f6fee2c50b2ee826c471f8dac07c
  • Pointer size: 130 Bytes
  • Size of remote file: 89.1 kB
Current UI/Screenshot 2026-05-01 140058.png ADDED

Git LFS Details

  • SHA256: a45ee7a048f5ac25c833b155bff8bf46cb7361c58943deca881c5c80098d47b1
  • Pointer size: 131 Bytes
  • Size of remote file: 136 kB
Current UI/Screenshot 2026-05-01 140111.png ADDED

Git LFS Details

  • SHA256: ee152153f3d9700ab939f56c1c7136ac7ceac714d4b9356ce6cd712089d36b4a
  • Pointer size: 130 Bytes
  • Size of remote file: 61.1 kB
Current UI/Screenshot 2026-05-01 140124.png ADDED

Git LFS Details

  • SHA256: c1d2a7f0208ca0e1f40cc84cfe5122785d4c8f280c75e9a5a953b5565f9d8ae3
  • Pointer size: 130 Bytes
  • Size of remote file: 98.5 kB
Current UI/Screenshot 2026-05-01 140140.png ADDED

Git LFS Details

  • SHA256: e127abbc76002a1caa2c056e01b35e75265cf768079cb0f4aee0adfc5a8a9289
  • Pointer size: 130 Bytes
  • Size of remote file: 79.6 kB
Old Chats claude/1.md ADDED
@@ -0,0 +1,2 @@
 
 
 
1
+ claude --resume b12b89ce-94d3-48fc-b8f8-b73ba5ddee29
2
+ claude --resume abc2b770-ce67-4c01-8f04-0e1e84cb120d , remaining
Plans/plan 1.md ADDED
@@ -0,0 +1,231 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # Backend Refactor Execution Plan (Actionable)
2
+
3
+ ## Phase 0 β€” Preparation
4
+
5
+ * Create new branch: `refactor/architecture-v1`
6
+ * Freeze new feature development
7
+ * Add `.env` and `core/config.py`
8
+
9
+ ---
10
+
11
+ ## Phase 1 β€” Restructure (No Logic Changes)
12
+
13
+ ### Create Directories
14
+
15
+ ```
16
+ backend/app/api/routes/
17
+ backend/core/
18
+ backend/domain/models/
19
+ backend/services/
20
+ backend/integrations/llm/
21
+ backend/integrations/http/
22
+ backend/integrations/parsers/
23
+ backend/repositories/
24
+ backend/workers/
25
+ backend/schemas/
26
+ backend/utils/
27
+ ```
28
+
29
+ ### Move Files
30
+
31
+ * `main.py` β†’ `app/main.py`
32
+ * routes β†’ `app/api/routes/`
33
+ * shared utils β†’ `utils/`
34
+ * existing services β†’ `services/`
35
+
36
+ ### Update Imports
37
+
38
+ * Replace relative imports with absolute:
39
+
40
+ ```
41
+ from backend.services...
42
+ ```
43
+
44
+ ---
45
+
46
+ ## Phase 2 β€” Config + Core Systems
47
+
48
+ ### Create `core/config.py`
49
+
50
+ * Use `BaseSettings`
51
+ * Add:
52
+
53
+ * `ALLOWED_ORIGINS`
54
+ * `API_KEYS`
55
+ * `UPLOAD_DIR`
56
+ * `TIMEOUT`
57
+
58
+ ### Replace Hardcoded Values
59
+
60
+ * Replace all constants with config usage
61
+
62
+ ### Add Logging
63
+
64
+ * Create `core/logging.py`
65
+ * Replace all `print()` with logger
66
+
67
+ ---
68
+
69
+ ## Phase 3 β€” LLM Isolation
70
+
71
+ ### Move LLM Logic
72
+
73
+ FROM:
74
+
75
+ * `services/llm_planner.py`
76
+ * `services/chat_assistant.py`
77
+
78
+ TO:
79
+
80
+ ```
81
+ integrations/llm/
82
+ client.py
83
+ planner.py
84
+ analyzer.py
85
+ ```
86
+
87
+ ### Enforce Rule
88
+
89
+ * No direct OpenAI calls outside `client.py`
90
+
91
+ ---
92
+
93
+ ## Phase 4 β€” Execution Isolation
94
+
95
+ ### Move Executor
96
+
97
+ * `api_executor.py` β†’ `integrations/http/sandbox.py`
98
+
99
+ ### Add Controls
100
+
101
+ * allowlist domains
102
+ * enforce timeout
103
+ * restrict HTTP methods
104
+
105
+ ---
106
+
107
+ ## Phase 5 β€” Validation Layer
108
+
109
+ ### Create
110
+
111
+ `services/validation_service.py`
112
+
113
+ ### Add Checks
114
+
115
+ * validate LLM JSON schema
116
+ * reject unsafe URLs
117
+ * sanitize inputs
118
+
119
+ ### Integrate
120
+
121
+ * execution_service MUST call validation before execution
122
+
123
+ ---
124
+
125
+ ## Phase 6 β€” Split Responsibilities
126
+
127
+ ### Create
128
+
129
+ * `planning_service.py`
130
+ * `execution_service.py`
131
+
132
+ ### Flow
133
+
134
+ ```
135
+ route β†’ planning_service β†’ validation_service β†’ execution_service
136
+ ```
137
+
138
+ ---
139
+
140
+ ## Phase 7 β€” Remove Global State
141
+
142
+ ### Delete
143
+
144
+ * `get_context`
145
+ * `set_context_variables`
146
+
147
+ ### Replace With
148
+
149
+ * request-scoped data OR
150
+ * `domain/models/user_session.py`
151
+ * `repositories/session_repo.py`
152
+
153
+ ---
154
+
155
+ ## Phase 8 β€” Background Jobs
156
+
157
+ ### Create
158
+
159
+ `workers/tasks.py`
160
+
161
+ ### Move Heavy Tasks
162
+
163
+ * LLM calls
164
+ * API chains
165
+
166
+ ### Use
167
+
168
+ * FastAPI BackgroundTasks (initial)
169
+
170
+ ---
171
+
172
+ ## Phase 9 β€” Security Layer
173
+
174
+ ### Create
175
+
176
+ `core/security.py`
177
+
178
+ ### Add
179
+
180
+ * JWT auth
181
+ * rate limiting middleware
182
+
183
+ ### Apply
184
+
185
+ * protect all routes except health
186
+
187
+ ---
188
+
189
+ ## Phase 10 β€” Frontend Alignment
190
+
191
+ ### Create
192
+
193
+ ```
194
+ src/api/
195
+ src/hooks/
196
+ src/store/
197
+ ```
198
+
199
+ ### Add
200
+
201
+ * API client wrapper
202
+ * React Query
203
+
204
+ ---
205
+
206
+ ## Phase 11 β€” Cleanup
207
+
208
+ * Remove unused files
209
+ * Remove duplicate logic
210
+ * Remove debug prints
211
+
212
+ ---
213
+
214
+ ## Phase 12 β€” Testing
215
+
216
+ * Add unit tests for:
217
+
218
+ * validation_service
219
+ * execution_service
220
+ * Add integration tests for routes
221
+
222
+ ---
223
+
224
+ ## Final Architecture Rule
225
+
226
+ * routes = HTTP only
227
+ * services = orchestration only
228
+ * integrations = external calls only
229
+ * domain = pure logic only
230
+
231
+ NO cross-layer violations allowed.
Plans/plan 2.md ADDED
@@ -0,0 +1,161 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # Project Completion Plan β€” AI API Testing Assistant
2
+
3
+ **Goal:** Make the project fully functional, secure, and production-ready.
4
+ **Based on:** Actual code audit of the current codebase.
5
+
6
+ ---
7
+
8
+ ## βœ… Phase 1 β€” Fix Broken Foundations β€” DONE
9
+
10
+ - [x] Fix `requirements.txt`: removed `streamlit`, `websockets`; added `prance`, `pyyaml`, `pydantic-settings`, `slowapi`, `cryptography`; upgraded LangChain + Mistral packages; fixed `psycopg2-binary>=2.9.10` (Python 3.13 wheel), `sqlalchemy>=2.0.36`, gunicorn platform guard
11
+ - [x] Fix CORS: replaced `allow_origins=["*"]` with `settings.allowed_origins_list` from env var
12
+ - [x] Fix silent failures: removed bare `except: pass` blocks; all errors logged or returned as HTTP
13
+ - [x] Clean `App.jsx`: deleted ~800 lines of commented-out dead code at top; active code only
14
+ - [x] Fix `chat_assistant.py`: replaced `HumanMessage` with `AIMessage` for assistant replies
15
+ - [x] Add `.gitignore`: covers `*.zip`, `*.tar.gz`, `.env.*`, `__pycache__`, `.venv`, uploads, etc.
16
+
17
+ ---
18
+
19
+ ## βœ… Phase 2 β€” Backend Restructure β€” DONE
20
+
21
+ - [x] Created `backend/core/config.py` β€” `BaseSettings` with all env vars
22
+ - [x] Created `backend/core/logging.py` β€” `get_logger()` structured logger
23
+ - [x] Created `backend/core/errors.py` β€” `AppError` + `app_error_handler`
24
+ - [x] Split `main.py` into 6 focused route files (no logic changes):
25
+ - `backend/app/api/routes/specs.py` β€” `/specs`, `/upload-spec`, `/credentials`
26
+ - `backend/app/api/routes/sessions.py` β€” `/session`, `/context`
27
+ - `backend/app/api/routes/execution.py` β€” `/plan`, `/execute-step`, `/analyze-response`
28
+ - `backend/app/api/routes/assets.py` β€” `/assets`, `/collections`
29
+ - `backend/app/api/routes/chat.py` β€” `/chat`, `/chat-sessions`
30
+ - `backend/app/api/routes/history.py` β€” `/history`
31
+ - [x] `main.py` reduced to ~60 lines (app setup only)
32
+ - [x] All 36 routes preserved throughout
33
+
34
+ ---
35
+
36
+ ## βœ… Phase 3 β€” Security & Validation β€” DONE
37
+
38
+ - [x] URL sandboxing (`backend/core/url_guard.py`): blocks localhost, private IP ranges (10.x, 172.16-31.x, 192.168.x, 169.254.x, ::1, fc00::/7), non-HTTP schemes; resolves hostname to IP before check
39
+ - [x] `assert_safe_url()` called in `/execute-step` before every outbound HTTP request
40
+ - [x] Spec file size validation: 5MB limit in `/upload-spec`
41
+ - [x] Rate limiting via `slowapi`: `/plan` 10/min, `/execute-step` 30/min, `/chat` 20/min
42
+
43
+ ---
44
+
45
+ ## βœ… Phase 4 β€” Error Handling β€” DONE
46
+
47
+ - [x] Standardized `AppError` exception with `{ "error", "code", "detail" }` shape
48
+ - [x] LLM retry: one automatic retry on JSON parse failure in both `llm_planner.py` and `response_analyzer.py`
49
+ - [x] Execution error context: `TimeoutException β†’ 504`, `ConnectError β†’ 502`, 4xx/5xx responses logged with URL + body snippet
50
+
51
+ ---
52
+
53
+ ## βœ… Phase 5 β€” Complete Half-Built Features β€” DONE
54
+
55
+ - [x] OTP modal UI: inline modal shown when `is_otp_required` detected; saves to session context
56
+ - [x] Selection modal UI: inline modal shown when `is_selection_required` detected; saves chosen value
57
+ - [x] Token usage display: shown per-step and as running total in the top bar
58
+ - [x] **Run All** button: runs all execution plan steps sequentially; stops if OTP/selection prompt appears
59
+ - [x] **Retry** button: amber button appears next to failed steps (status = 'failure')
60
+ - [x] **Export session** (`GET /history/{session_id}/export`): returns full JSON report with pass/fail counts; forces browser download
61
+ - [x] `HistoryService.exportSession()` in `api.js`
62
+ - [x] Export button in `HistoryPage.jsx` (Download icon, between Delete and Import)
63
+
64
+ ---
65
+
66
+ ## πŸ”² Phase 6 β€” LLM Isolation
67
+
68
+ Consolidate all LLM calls into one place.
69
+
70
+ ### Create `backend/integrations/llm/client.py`
71
+ ```python
72
+ class MistralClient:
73
+ def complete(self, prompt, temperature=0, timeout=60) -> str: ...
74
+ def structured(self, prompt, schema) -> dict: ... # validates JSON output
75
+ ```
76
+
77
+ ### Migrate all callers:
78
+ - `llm_planner.py` β†’ uses `MistralClient`
79
+ - `api_executor.py` β†’ uses `MistralClient`
80
+ - `response_analyzer.py` β†’ uses `MistralClient`
81
+ - `chat_assistant.py` β†’ uses `MistralClient`
82
+
83
+ **Rule:** No direct `ChatMistralAI` instantiation outside `client.py`.
84
+
85
+ ---
86
+
87
+ ## πŸ”² Phase 7 β€” Frontend Polish
88
+
89
+ ### 7.1 Error UI
90
+ - Replace `alert()` calls with toast notifications or inline error messages
91
+ - Add error boundary component to catch React crashes
92
+ - Show which step failed and why
93
+
94
+ ### 7.2 API Client Improvements (`services/api.js`)
95
+ - Add global axios error interceptor
96
+ - Centralize timeout and base URL
97
+
98
+ ### 7.3 Frontend Structure (optional)
99
+ ```
100
+ src/
101
+ β”œβ”€β”€ api/ # Move api.js here, split by domain
102
+ β”œβ”€β”€ hooks/ # Custom hooks (useSpec, usePlan, useExecution)
103
+ └── store/ # AppContext split into smaller contexts
104
+ ```
105
+
106
+ ---
107
+
108
+ ## πŸ”² Phase 8 β€” Testing
109
+
110
+ ### Unit Tests
111
+ - `services/url_validator.py` β€” URL blocking, private IP detection
112
+ - `services/llm_planner.py` β€” mock LLM, test output parsing
113
+ - `services/api_executor.py` β€” test request building, context resolution
114
+
115
+ ### Integration Tests
116
+ - `POST /upload-spec` β€” valid OpenAPI file β†’ correct parse
117
+ - `POST /plan` β€” returns valid step list
118
+ - `POST /execute-step` β€” mock HTTP target β†’ correct history entry
119
+ - `POST /chat` β€” returns valid action JSON
120
+
121
+ ### Tools
122
+ - `pytest` + `pytest-asyncio`
123
+ - `httpx.AsyncClient` for route testing
124
+ - `respx` for mocking outbound HTTP
125
+
126
+ ---
127
+
128
+ ## πŸ”² Phase 9 β€” Deployment
129
+
130
+ ### 9.1 Docker
131
+ ```
132
+ Dockerfile # backend
133
+ docker-compose.yml # backend + postgres (local dev)
134
+ ```
135
+
136
+ ### 9.2 Environment Separation
137
+ - `.env.development` β€” local DB, debug logging
138
+ - `.env.production` β€” Supabase, structured JSON logs, strict CORS
139
+
140
+ ### 9.3 Health Check
141
+ - `GET /health` returns `{ "status": "ok", "db": "ok", "llm": "ok" }`
142
+
143
+ ---
144
+
145
+ ## Progress Summary
146
+
147
+ | Phase | Status | Notes |
148
+ |-------|--------|-------|
149
+ | 1 β€” Fix Foundations | βœ… Done | |
150
+ | 2 β€” Backend Restructure | βœ… Done | 36 routes across 6 files |
151
+ | 3 β€” Security | βœ… Done | SSRF protection, rate limiting, file size validation |
152
+ | 4 β€” Error Handling | βœ… Done | LLM retry, typed HTTP errors |
153
+ | 5 β€” Complete Features | βœ… Done | Run All, Retry, Export, OTP/Selection modals, token display |
154
+ | 6 β€” LLM Isolation | πŸ”² Pending | |
155
+ | 7 β€” Frontend Polish | πŸ”² Pending | Replace alerts with toasts, error boundary |
156
+ | 8 β€” Testing | πŸ”² Pending | |
157
+ | 9 β€” Deployment | πŸ”² Pending | Docker + health check |
158
+
159
+
160
+
161
+
backend/app/__init__.py ADDED
File without changes
backend/app/api/__init__.py ADDED
File without changes
backend/app/api/routes/__init__.py ADDED
File without changes
backend/app/api/routes/assets.py ADDED
@@ -0,0 +1,109 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ import json
2
+ from typing import Optional
3
+
4
+ from fastapi import APIRouter, Depends, File, Form, HTTPException, UploadFile
5
+ from sqlalchemy import or_
6
+ from sqlalchemy.orm import Session
7
+
8
+ from backend import database
9
+ from backend.services.asset_service import AssetService
10
+
11
+ router = APIRouter()
12
+
13
+
14
+ @router.post("/assets")
15
+ async def upload_asset(
16
+ spec_id: str = Form(None),
17
+ collection_id: str = Form(None),
18
+ description: str = Form(None),
19
+ tags: str = Form("[]"),
20
+ file: UploadFile = File(...),
21
+ db: Session = Depends(database.get_db),
22
+ ):
23
+ try:
24
+ tag_list = json.loads(tags)
25
+ asset = AssetService.upload_asset(db, file, spec_id, collection_id, description, tag_list)
26
+ return {
27
+ "id": asset.id,
28
+ "name": asset.name,
29
+ "type": asset.file_type,
30
+ "size": asset.file_size,
31
+ "category": asset.category,
32
+ "description": asset.description,
33
+ "tags": asset.tags,
34
+ "spec_id": asset.spec_id,
35
+ "uploaded_at": asset.uploaded_at.isoformat(),
36
+ }
37
+ except Exception as e:
38
+ raise HTTPException(status_code=500, detail=f"Upload failed: {str(e)}")
39
+
40
+
41
+ @router.get("/assets")
42
+ def list_assets(
43
+ spec_id: Optional[str] = None,
44
+ category: Optional[str] = None,
45
+ db: Session = Depends(database.get_db),
46
+ ):
47
+ assets = AssetService.get_assets(db, spec_id, category)
48
+ return [
49
+ {
50
+ "id": a.id,
51
+ "name": a.name,
52
+ "type": a.file_type,
53
+ "size": a.file_size,
54
+ "category": a.category,
55
+ "description": a.description,
56
+ "tags": a.tags,
57
+ "spec_id": a.spec_id,
58
+ "path": a.file_path,
59
+ "uploaded_at": a.uploaded_at.isoformat(),
60
+ "collection_ids": [c.id for c in a.collections],
61
+ }
62
+ for a in assets
63
+ ]
64
+
65
+
66
+ @router.delete("/assets/{asset_id}")
67
+ def delete_asset(asset_id: str, db: Session = Depends(database.get_db)):
68
+ success = AssetService.delete_asset(db, asset_id)
69
+ if not success:
70
+ raise HTTPException(status_code=404, detail="Asset not found")
71
+ return {"status": "deleted", "id": asset_id}
72
+
73
+
74
+ @router.get("/collections")
75
+ def list_collections(spec_id: Optional[str] = None, db: Session = Depends(database.get_db)):
76
+ query = db.query(database.Collection)
77
+ if spec_id:
78
+ query = query.filter(
79
+ or_(database.Collection.spec_id == spec_id, database.Collection.spec_id == None)
80
+ )
81
+ cols = query.all()
82
+ return [{"id": c.id, "name": c.name, "description": c.description, "spec_id": c.spec_id} for c in cols]
83
+
84
+
85
+ @router.post("/collections")
86
+ def create_collection(
87
+ name: str = Form(...),
88
+ spec_id: str = Form(None),
89
+ description: str = Form(None),
90
+ db: Session = Depends(database.get_db),
91
+ ):
92
+ col = AssetService.create_collection(db, name, spec_id, description)
93
+ return {"id": col.id, "name": col.name, "spec_id": col.spec_id}
94
+
95
+
96
+ @router.post("/collections/{collection_id}/assets/{asset_id}")
97
+ def add_asset_to_collection(collection_id: str, asset_id: str, db: Session = Depends(database.get_db)):
98
+ success = AssetService.add_to_collection(db, asset_id, collection_id)
99
+ if not success:
100
+ raise HTTPException(status_code=404, detail="Asset or Collection not found")
101
+ return {"status": "success"}
102
+
103
+
104
+ @router.delete("/collections/{collection_id}")
105
+ def delete_collection(collection_id: str, db: Session = Depends(database.get_db)):
106
+ success = AssetService.delete_collection(db, collection_id)
107
+ if not success:
108
+ raise HTTPException(status_code=404, detail="Collection not found")
109
+ return {"status": "success"}
backend/app/api/routes/chat.py ADDED
@@ -0,0 +1,183 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ import datetime
2
+ import uuid
3
+ from typing import Optional
4
+
5
+ from fastapi import APIRouter, Depends, HTTPException, Request
6
+ from slowapi import Limiter
7
+ from slowapi.util import get_remote_address
8
+ from sqlalchemy.orm import Session
9
+
10
+ from backend import database, schemas
11
+ from backend.services.chat_assistant import process_message as chat_process_message
12
+
13
+ router = APIRouter()
14
+ limiter = Limiter(key_func=get_remote_address)
15
+
16
+
17
+ @router.post("/chat", response_model=schemas.ChatResponse)
18
+ @limiter.limit("20/minute")
19
+ def chat_endpoint(request: Request, body: schemas.ChatRequest, db: Session = Depends(database.get_db)):
20
+ try:
21
+ chat_session = None
22
+ if body.chat_session_id:
23
+ chat_session = db.query(database.ChatSession).filter(
24
+ database.ChatSession.id == body.chat_session_id
25
+ ).first()
26
+
27
+ if not chat_session:
28
+ session_name = body.message[:60].strip()
29
+ if len(body.message) > 60:
30
+ session_name += "..."
31
+ chat_session = database.ChatSession(
32
+ id=str(uuid.uuid4()),
33
+ name=session_name,
34
+ spec_id=body.spec_id,
35
+ session_id=body.session_id,
36
+ )
37
+ db.add(chat_session)
38
+ db.flush()
39
+
40
+ user_msg = database.ChatMessage(
41
+ chat_session_id=chat_session.id,
42
+ role="user",
43
+ content=body.message,
44
+ )
45
+ db.add(user_msg)
46
+ db.flush()
47
+
48
+ history_rows = (
49
+ db.query(database.ChatMessage)
50
+ .filter(database.ChatMessage.chat_session_id == chat_session.id)
51
+ .order_by(database.ChatMessage.id.asc())
52
+ .all()
53
+ )
54
+ chat_history = [
55
+ {"role": m.role, "content": m.content}
56
+ for m in history_rows[:-1]
57
+ ]
58
+
59
+ ai_result = chat_process_message(
60
+ message=body.message,
61
+ spec_name=body.spec_name,
62
+ execution_plan=body.execution_plan or [],
63
+ context_variables=body.context_variables or {},
64
+ last_step_result=body.last_step_result,
65
+ all_step_results=body.all_step_results,
66
+ chat_history=chat_history,
67
+ )
68
+
69
+ assistant_content = ai_result.get("message", "")
70
+ db.add(database.ChatMessage(
71
+ chat_session_id=chat_session.id,
72
+ role="assistant",
73
+ content=assistant_content,
74
+ action_json=ai_result,
75
+ ))
76
+
77
+ chat_session.updated_at = datetime.datetime.utcnow()
78
+ db.commit()
79
+
80
+ return schemas.ChatResponse(
81
+ chat_session_id=chat_session.id,
82
+ chat_session_name=chat_session.name,
83
+ action=ai_result.get("action", "reply"),
84
+ message=assistant_content,
85
+ step=ai_result.get("step"),
86
+ changes=ai_result.get("changes"),
87
+ assertions=ai_result.get("assertions"),
88
+ explanation=ai_result.get("explanation"),
89
+ )
90
+ except Exception as e:
91
+ import traceback
92
+ traceback.print_exc()
93
+ raise HTTPException(status_code=500, detail=f"Chat failed: {str(e)}")
94
+
95
+
96
+ @router.get("/chat-sessions")
97
+ def list_chat_sessions(spec_id: Optional[str] = None, db: Session = Depends(database.get_db)):
98
+ query = db.query(database.ChatSession)
99
+ if spec_id:
100
+ query = query.filter(database.ChatSession.spec_id == spec_id)
101
+ sessions = query.order_by(database.ChatSession.updated_at.desc()).all()
102
+
103
+ return [
104
+ {
105
+ "id": s.id,
106
+ "name": s.name,
107
+ "spec_id": s.spec_id,
108
+ "session_id": s.session_id,
109
+ "message_count": db.query(database.ChatMessage)
110
+ .filter(database.ChatMessage.chat_session_id == s.id)
111
+ .count(),
112
+ "created_at": s.created_at.isoformat() if s.created_at else None,
113
+ "updated_at": s.updated_at.isoformat() if s.updated_at else None,
114
+ }
115
+ for s in sessions
116
+ ]
117
+
118
+
119
+ @router.get("/chat-sessions/{chat_session_id}")
120
+ def get_chat_session(chat_session_id: str, db: Session = Depends(database.get_db)):
121
+ session = db.query(database.ChatSession).filter(
122
+ database.ChatSession.id == chat_session_id
123
+ ).first()
124
+ if not session:
125
+ raise HTTPException(status_code=404, detail="Chat session not found")
126
+
127
+ messages = (
128
+ db.query(database.ChatMessage)
129
+ .filter(database.ChatMessage.chat_session_id == chat_session_id)
130
+ .order_by(database.ChatMessage.id.asc())
131
+ .all()
132
+ )
133
+ return {
134
+ "id": session.id,
135
+ "name": session.name,
136
+ "spec_id": session.spec_id,
137
+ "session_id": session.session_id,
138
+ "created_at": session.created_at.isoformat() if session.created_at else None,
139
+ "updated_at": session.updated_at.isoformat() if session.updated_at else None,
140
+ "messages": [
141
+ {
142
+ "id": m.id,
143
+ "role": m.role,
144
+ "content": m.content,
145
+ "action_json": m.action_json,
146
+ "timestamp": m.timestamp.isoformat() if m.timestamp else None,
147
+ }
148
+ for m in messages
149
+ ],
150
+ }
151
+
152
+
153
+ @router.patch("/chat-sessions/{chat_session_id}")
154
+ def rename_chat_session(chat_session_id: str, payload: dict, db: Session = Depends(database.get_db)):
155
+ session = db.query(database.ChatSession).filter(
156
+ database.ChatSession.id == chat_session_id
157
+ ).first()
158
+ if not session:
159
+ raise HTTPException(status_code=404, detail="Chat session not found")
160
+ if "name" in payload:
161
+ session.name = payload["name"]
162
+ db.commit()
163
+ return {"id": session.id, "name": session.name}
164
+
165
+
166
+ @router.delete("/chat-sessions/{chat_session_id}")
167
+ def delete_chat_session(chat_session_id: str, db: Session = Depends(database.get_db)):
168
+ session = db.query(database.ChatSession).filter(
169
+ database.ChatSession.id == chat_session_id
170
+ ).first()
171
+ if not session:
172
+ raise HTTPException(status_code=404, detail="Chat session not found")
173
+ db.delete(session)
174
+ db.commit()
175
+ return {"status": "deleted", "id": chat_session_id}
176
+
177
+
178
+ @router.delete("/chat-sessions")
179
+ def clear_all_chat_sessions(db: Session = Depends(database.get_db)):
180
+ db.query(database.ChatMessage).delete()
181
+ db.query(database.ChatSession).delete()
182
+ db.commit()
183
+ return {"status": "cleared"}
backend/app/api/routes/execution.py ADDED
@@ -0,0 +1,343 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ import json
2
+ import os
3
+ import re
4
+ import traceback
5
+ from urllib.parse import parse_qsl, urlparse
6
+
7
+ import httpx
8
+ from fastapi import APIRouter, Depends, HTTPException, Request
9
+ from slowapi import Limiter
10
+ from slowapi.util import get_remote_address
11
+ from sqlalchemy.orm import Session
12
+ from sqlalchemy.orm.attributes import flag_modified
13
+
14
+ from backend import database, schemas
15
+ from backend.core.logging import get_logger
16
+ from backend.core.url_guard import assert_safe_url
17
+ from backend.services.api_executor import prepare_step_request
18
+ from backend.services.asset_service import AssetService, storage
19
+ from backend.services.context_manager import set_context_variables
20
+ from backend.services.llm_planner import generate_execution_plan
21
+ from backend.services.response_analyzer import analyze_api_response
22
+
23
+ router = APIRouter()
24
+ logger = get_logger(__name__)
25
+ limiter = Limiter(key_func=get_remote_address)
26
+
27
+ PLACEHOLDER_HOSTS = {"api.example.com", "example.com", "<base_url>", "your-api.com"}
28
+
29
+
30
+ def _resolve_refs(obj, full_spec, _resolving=None):
31
+ if _resolving is None:
32
+ _resolving = set()
33
+ if isinstance(obj, dict):
34
+ if "$ref" in obj:
35
+ ref = obj["$ref"]
36
+ if ref in _resolving:
37
+ return {"$ref_loop": ref}
38
+ _resolving = _resolving | {ref}
39
+ parts = ref.lstrip("#/").split("/")
40
+ target = full_spec
41
+ for part in parts:
42
+ target = target.get(part, {}) if isinstance(target, dict) else {}
43
+ return _resolve_refs(target, full_spec, _resolving)
44
+ return {k: _resolve_refs(v, full_spec, _resolving) for k, v in obj.items()}
45
+ elif isinstance(obj, list):
46
+ return [_resolve_refs(item, full_spec, _resolving) for item in obj]
47
+ return obj
48
+
49
+
50
+ def _shorthand_spec(obj):
51
+ if isinstance(obj, dict):
52
+ if "type" in obj:
53
+ t = obj["type"]
54
+ fmt = f":{obj['format']}" if "format" in obj else ""
55
+ enum_vals = f"[{'|'.join(map(str, obj['enum']))}]" if "enum" in obj else ""
56
+ if t == "object" and "properties" in obj:
57
+ req_list = obj.get("required", [])
58
+ props = [
59
+ f"{k}{'*' if k in req_list or v.get('nullable') == False else ''}({_shorthand_spec(v)})"
60
+ for k, v in obj["properties"].items()
61
+ ]
62
+ return "{" + ",".join(props) + "}"
63
+ if t == "array" and "items" in obj:
64
+ return f"[{_shorthand_spec(obj['items'])}]"
65
+ return f"{t}{fmt}{enum_vals}"
66
+ return {k: _shorthand_spec(v) for k, v in obj.items()}
67
+ return str(obj)
68
+
69
+
70
+ def _prune_spec(obj):
71
+ DROP_KEYS = {"description", "summary", "example", "examples", "format", "title", "default", "enum_descriptions"}
72
+ if isinstance(obj, dict):
73
+ return {k: _prune_spec(v) for k, v in obj.items() if k not in DROP_KEYS}
74
+ elif isinstance(obj, list):
75
+ return [_prune_spec(item) for item in obj]
76
+ return obj
77
+
78
+
79
+ def _match_path(path_template: str, actual_path: str) -> bool:
80
+ pattern = re.sub(r'\{[^}]+\}', r'[^/]+', path_template)
81
+ pattern = "^" + pattern.rstrip("/") + "/?$"
82
+ return bool(re.match(pattern, actual_path.split("?")[0]))
83
+
84
+
85
+ def _is_placeholder(url: str) -> bool:
86
+ try:
87
+ host = urlparse(url).hostname or ""
88
+ return host.lower() in PLACEHOLDER_HOSTS or url.startswith("<BASE_URL>")
89
+ except Exception:
90
+ return False
91
+
92
+
93
+ @router.post("/plan", response_model=schemas.ExecutionPlan)
94
+ @limiter.limit("10/minute")
95
+ def generate_plan(request: Request, body: schemas.PlanRequest, db: Session = Depends(database.get_db)):
96
+ try:
97
+ parsed_spec = json.loads(body.spec_content) if isinstance(body.spec_content, str) else body.spec_content
98
+
99
+ credentials = body.user_credentials or {}
100
+ if body.spec_id:
101
+ spec = db.query(database.SpecDocument).filter(database.SpecDocument.id == body.spec_id).first()
102
+ if spec and spec.login_credentials:
103
+ credentials = spec.login_credentials
104
+
105
+ plan = generate_execution_plan(parsed_spec, body.base_url, body.scenario, credentials)
106
+
107
+ if body.spec_id:
108
+ spec = db.query(database.SpecDocument).filter(database.SpecDocument.id == body.spec_id).first()
109
+ if spec:
110
+ spec.execution_plan = plan.model_dump()["execution_plan"]
111
+ spec.scenario = body.scenario
112
+ flag_modified(spec, "execution_plan")
113
+ flag_modified(spec, "scenario")
114
+
115
+ if body.session_id:
116
+ session = db.query(database.SessionModel).filter(
117
+ database.SessionModel.id == body.session_id
118
+ ).first()
119
+ if session:
120
+ session.spec_id = spec.id
121
+ session.spec_name = spec.name
122
+ session.scenario = body.scenario
123
+ session.base_url = body.base_url
124
+
125
+ if plan.suggested_credentials and isinstance(plan.suggested_credentials, dict):
126
+ current_creds = dict(spec.login_credentials or {})
127
+ if any(k not in current_creds or current_creds[k] != v for k, v in plan.suggested_credentials.items()):
128
+ current_creds.update(plan.suggested_credentials)
129
+ spec.login_credentials = current_creds
130
+ flag_modified(spec, "login_credentials")
131
+ logger.info("AI suggested credentials merged for spec '%s'", spec.name)
132
+
133
+ db.commit()
134
+
135
+ return plan
136
+ except Exception as e:
137
+ raise HTTPException(status_code=500, detail=f"Planning failed: {str(e)}")
138
+
139
+
140
+ @router.post("/execute-step")
141
+ @limiter.limit("30/minute")
142
+ async def execute_step(request: Request, body: schemas.ExecuteStepRequest, db: Session = Depends(database.get_db)):
143
+ try:
144
+ endpoint_str = body.step_definition.get("endpoint", "")
145
+ actual_endpoint_spec = {"endpoint": endpoint_str}
146
+
147
+ if body.spec_id:
148
+ spec_doc = db.query(database.SpecDocument).filter(
149
+ database.SpecDocument.id == body.spec_id
150
+ ).first()
151
+ if spec_doc and spec_doc.parsed_json:
152
+ full_spec = spec_doc.parsed_json
153
+ paths = full_spec.get("paths", {})
154
+ parts = endpoint_str.split(" ", 1)
155
+ path_only = parts[1] if len(parts) == 2 else parts[0]
156
+
157
+ matched_path = path_only if path_only in paths else next(
158
+ (p for p in paths if _match_path(p, path_only)), None
159
+ )
160
+
161
+ if matched_path:
162
+ raw_spec = {matched_path: paths[matched_path]}
163
+ resolved = _resolve_refs(raw_spec, full_spec)
164
+ path_data = resolved.get(matched_path, {})
165
+ for method in path_data:
166
+ if isinstance(path_data[method], dict):
167
+ path_data[method].pop("responses", None)
168
+ actual_endpoint_spec = _shorthand_spec(_prune_spec(resolved))
169
+ else:
170
+ logger.warning("Path match failed for %s, falling back", path_only)
171
+ actual_endpoint_spec = {"endpoint": endpoint_str, "warning": "Full schema match failed"}
172
+
173
+ rich_assets = AssetService.get_ai_context(db, body.spec_id)
174
+
175
+ effective_credentials = body.user_credentials
176
+ if not effective_credentials and body.spec_id:
177
+ spec = db.query(database.SpecDocument).filter(
178
+ database.SpecDocument.id == body.spec_id
179
+ ).first()
180
+ if spec and spec.login_credentials:
181
+ effective_credentials = spec.login_credentials
182
+ logger.info("Using stored credentials for spec '%s'", spec.name)
183
+
184
+ req_plan, executor_token_usage = prepare_step_request(
185
+ endpoint_spec=actual_endpoint_spec,
186
+ base_url=body.base_url,
187
+ scenario="",
188
+ step_definition=body.step_definition,
189
+ context_variables=body.context_variables,
190
+ user_credentials=effective_credentials,
191
+ available_assets=rich_assets,
192
+ )
193
+
194
+ url = req_plan.request.url.strip()
195
+ if url.startswith("/"):
196
+ url = body.base_url.rstrip("/") + url
197
+ elif _is_placeholder(url):
198
+ parsed = urlparse(url)
199
+ path = parsed.path or "/"
200
+ url = body.base_url.rstrip("/") + path
201
+ if parsed.query:
202
+ url += "?" + parsed.query
203
+ logger.info("Placeholder URL replaced: %s", url)
204
+ elif not url.startswith("http"):
205
+ url = f"https://{url}"
206
+
207
+ parsed = urlparse(url)
208
+ if parsed.query:
209
+ url_keys = {k for k, _ in parse_qsl(parsed.query)}
210
+ for k in [k for k in req_plan.request.query_params if k in url_keys]:
211
+ del req_plan.request.query_params[k]
212
+
213
+ # Block SSRF β€” reject private/internal IPs before executing
214
+ assert_safe_url(url)
215
+
216
+ async with httpx.AsyncClient(timeout=120.0) as client:
217
+ http_req = client.build_request(
218
+ method=req_plan.request.method.strip().upper(),
219
+ url=url,
220
+ headers=req_plan.request.headers,
221
+ params=req_plan.request.query_params,
222
+ data=req_plan.request.form_data if req_plan.request.form_data else None,
223
+ json=req_plan.request.body if req_plan.request.body and not req_plan.request.form_data else None,
224
+ )
225
+
226
+ files_payload = {}
227
+ opened_files = []
228
+ try:
229
+ if req_plan.request.files:
230
+ req_plan.request.headers = {
231
+ k: v for k, v in req_plan.request.headers.items() if k.lower() != "content-type"
232
+ }
233
+ for field_name, file_name in req_plan.request.files.items():
234
+ asset = db.query(database.Asset).filter(database.Asset.name == file_name).first()
235
+ if asset:
236
+ full_path = storage.get_full_path(asset.file_path)
237
+ if os.path.exists(full_path):
238
+ f = open(full_path, "rb")
239
+ opened_files.append(f)
240
+ files_payload[field_name] = (asset.name, f, asset.file_type)
241
+ else:
242
+ logger.warning("Asset file not found on disk: %s", full_path)
243
+ else:
244
+ logger.warning("Asset not found in Media Hub: %s", file_name)
245
+
246
+ logger.info("Executing %s %s", http_req.method, http_req.url)
247
+
248
+ try:
249
+ if files_payload:
250
+ response = await client.request(
251
+ method=req_plan.request.method.strip().upper(),
252
+ url=url,
253
+ headers=req_plan.request.headers,
254
+ params=req_plan.request.query_params,
255
+ data=req_plan.request.form_data if req_plan.request.form_data else None,
256
+ files=files_payload,
257
+ )
258
+ else:
259
+ response = await client.send(http_req)
260
+ except httpx.TimeoutException:
261
+ raise HTTPException(
262
+ status_code=504,
263
+ detail=f"Request to {url} timed out after {120}s. Check the API server is reachable.",
264
+ )
265
+ except httpx.ConnectError as e:
266
+ raise HTTPException(
267
+ status_code=502,
268
+ detail=f"Could not connect to {url}: {str(e)}",
269
+ )
270
+ except Exception as e:
271
+ raise HTTPException(status_code=500, detail=f"Request to {url} failed: {str(e)}")
272
+ finally:
273
+ for f in opened_files:
274
+ f.close()
275
+
276
+ body_text = response.text
277
+ resp_data = {
278
+ "status_code": response.status_code,
279
+ "body": body_text,
280
+ "headers": dict(response.headers),
281
+ }
282
+
283
+ # Log a clear error line for 4xx/5xx so it's visible in the backend console
284
+ if response.status_code >= 400:
285
+ snippet = body_text[:300].replace("\n", " ")
286
+ logger.warning(
287
+ "Step %s returned %s from %s: %s",
288
+ req_plan.step, response.status_code, url, snippet,
289
+ )
290
+
291
+ req_plan.request.url = url
292
+
293
+ db.add(database.ExecutionHistory(
294
+ session_id=body.session_id,
295
+ step=req_plan.step,
296
+ request_json=req_plan.request.model_dump(),
297
+ response_json=resp_data,
298
+ step_definition_json=body.step_definition,
299
+ ))
300
+ db.commit()
301
+
302
+ return {
303
+ "executed_request": req_plan.request.model_dump(),
304
+ "response": resp_data,
305
+ "analysis_needed": req_plan.step,
306
+ "token_usage": executor_token_usage,
307
+ }
308
+ except Exception as e:
309
+ logger.error("Execution error: %s", traceback.format_exc())
310
+ raise HTTPException(status_code=500, detail=f"Execution failed: {str(e)}")
311
+
312
+
313
+ @router.post("/analyze-response", response_model=schemas.AnalyzeResponse)
314
+ def analyze_response_endpoint(request: schemas.AnalyzeRequest, db: Session = Depends(database.get_db)):
315
+ try:
316
+ step_def = request.step_definition or {}
317
+ analysis = analyze_api_response(
318
+ step=request.step,
319
+ endpoint=step_def.get("endpoint", "Unknown"),
320
+ purpose=step_def.get("purpose", "Unknown"),
321
+ status_code=request.response_data.get("status_code", 0),
322
+ response_body=request.response_data.get("body", ""),
323
+ )
324
+
325
+ if analysis.extracted_variables:
326
+ set_context_variables(db, request.session_id, analysis.extracted_variables)
327
+
328
+ history = (
329
+ db.query(database.ExecutionHistory)
330
+ .filter(
331
+ database.ExecutionHistory.session_id == request.session_id,
332
+ database.ExecutionHistory.step == request.step,
333
+ )
334
+ .order_by(database.ExecutionHistory.id.desc())
335
+ .first()
336
+ )
337
+ if history:
338
+ history.analysis_json = analysis.model_dump()
339
+ db.commit()
340
+
341
+ return analysis
342
+ except Exception as e:
343
+ raise HTTPException(status_code=500, detail=f"Analysis failed: {str(e)}")
backend/app/api/routes/history.py ADDED
@@ -0,0 +1,93 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ from fastapi import APIRouter, Depends, HTTPException
2
+ from fastapi.responses import JSONResponse
3
+ from sqlalchemy.orm import Session
4
+
5
+ from backend import database
6
+
7
+ router = APIRouter()
8
+
9
+
10
+ @router.get("/history/{session_id}/export")
11
+ def export_history_session(session_id: str, db: Session = Depends(database.get_db)):
12
+ sess = db.query(database.SessionModel).filter(database.SessionModel.id == session_id).first()
13
+ if not sess:
14
+ raise HTTPException(status_code=404, detail="Session not found")
15
+
16
+ steps = (
17
+ db.query(database.ExecutionHistory)
18
+ .filter(database.ExecutionHistory.session_id == session_id)
19
+ .order_by(database.ExecutionHistory.step)
20
+ .all()
21
+ )
22
+
23
+ report = {
24
+ "session_id": session_id,
25
+ "spec_name": sess.spec_name or "Unknown",
26
+ "scenario": sess.scenario or "",
27
+ "base_url": sess.base_url or "",
28
+ "context_variables": sess.context_variables or {},
29
+ "steps": [
30
+ {
31
+ "step": h.step,
32
+ "step_definition": h.step_definition_json,
33
+ "request": h.request_json,
34
+ "response": h.response_json,
35
+ "analysis": h.analysis_json,
36
+ "timestamp": h.timestamp.isoformat(),
37
+ }
38
+ for h in steps
39
+ ],
40
+ "total_steps": len(steps),
41
+ "passed": sum(1 for h in steps if (h.analysis_json or {}).get("status") == "success"),
42
+ "failed": sum(1 for h in steps if (h.analysis_json or {}).get("status") != "success"),
43
+ }
44
+
45
+ return JSONResponse(
46
+ content=report,
47
+ headers={"Content-Disposition": f'attachment; filename="session_{session_id[:8]}.json"'},
48
+ )
49
+
50
+
51
+ @router.get("/history")
52
+ def get_all_history(db: Session = Depends(database.get_db)):
53
+ results = (
54
+ db.query(database.ExecutionHistory, database.SessionModel)
55
+ .join(database.SessionModel, database.ExecutionHistory.session_id == database.SessionModel.id)
56
+ .order_by(database.ExecutionHistory.timestamp.desc())
57
+ .all()
58
+ )
59
+ return [
60
+ {
61
+ "id": hist.id,
62
+ "session_id": hist.session_id,
63
+ "spec_id": sess.spec_id,
64
+ "spec_name": sess.spec_name or "Unknown",
65
+ "scenario": sess.scenario or "No scenario",
66
+ "context_variables": sess.context_variables or {},
67
+ "step": hist.step,
68
+ "request": hist.request_json,
69
+ "response": hist.response_json,
70
+ "analysis": hist.analysis_json,
71
+ "step_definition": hist.step_definition_json,
72
+ "timestamp": hist.timestamp.isoformat(),
73
+ }
74
+ for hist, sess in results
75
+ ]
76
+
77
+
78
+ @router.delete("/history/{session_id}")
79
+ def delete_history_session(session_id: str, db: Session = Depends(database.get_db)):
80
+ db.query(database.ExecutionHistory).filter(
81
+ database.ExecutionHistory.session_id == session_id
82
+ ).delete()
83
+ db.query(database.SessionModel).filter(database.SessionModel.id == session_id).delete()
84
+ db.commit()
85
+ return {"status": "deleted", "session_id": session_id}
86
+
87
+
88
+ @router.delete("/history")
89
+ def clear_all_history(db: Session = Depends(database.get_db)):
90
+ db.query(database.ExecutionHistory).delete()
91
+ db.query(database.SessionModel).delete()
92
+ db.commit()
93
+ return {"status": "cleared"}
backend/app/api/routes/sessions.py ADDED
@@ -0,0 +1,45 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ import uuid
2
+
3
+ from fastapi import APIRouter, Depends
4
+ from sqlalchemy.orm import Session
5
+
6
+ from backend import database, schemas
7
+ from backend.services.context_manager import get_context, set_context_variables
8
+
9
+ router = APIRouter()
10
+
11
+
12
+ @router.post("/session")
13
+ def create_session(db: Session = Depends(database.get_db)):
14
+ session_id = str(uuid.uuid4())
15
+ new_session = database.SessionModel(id=session_id)
16
+ db.add(new_session)
17
+ db.commit()
18
+ return {"session_id": session_id}
19
+
20
+
21
+ @router.delete("/session/{session_id}")
22
+ def clear_session(session_id: str, db: Session = Depends(database.get_db)):
23
+ session = db.query(database.SessionModel).filter(database.SessionModel.id == session_id).first()
24
+ if session:
25
+ db.delete(session)
26
+ db.query(database.ExecutionHistory).filter(
27
+ database.ExecutionHistory.session_id == session_id
28
+ ).delete()
29
+ db.commit()
30
+ return {"status": "ok"}
31
+
32
+
33
+ @router.get("/context/{session_id}")
34
+ def get_context_endpoint(session_id: str, db: Session = Depends(database.get_db)):
35
+ return get_context(db, session_id)
36
+
37
+
38
+ @router.post("/context/{session_id}")
39
+ def update_context_endpoint(
40
+ session_id: str,
41
+ payload: schemas.ContextUpdate,
42
+ db: Session = Depends(database.get_db),
43
+ ):
44
+ variables = {payload.key: payload.value}
45
+ return set_context_variables(db, session_id, variables)
backend/app/api/routes/specs.py ADDED
@@ -0,0 +1,163 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ import json
2
+ import uuid
3
+ from typing import Optional
4
+
5
+ from fastapi import APIRouter, Depends, File, HTTPException, UploadFile
6
+ from sqlalchemy.orm import Session
7
+ from sqlalchemy.orm.attributes import flag_modified
8
+
9
+ from backend import database, schemas
10
+ from backend.core.logging import get_logger
11
+ from backend.services.swagger_parser import parse_spec
12
+ from backend.services.postman_parser import parse_postman_collection as parse_postman
13
+ from backend.services.asset_service import AssetService, storage
14
+
15
+ router = APIRouter()
16
+ logger = get_logger(__name__)
17
+
18
+
19
+ @router.post("/upload-spec")
20
+ async def upload_spec(file: UploadFile = File(...), db: Session = Depends(database.get_db)):
21
+ try:
22
+ content = await file.read()
23
+ if len(content) > 5 * 1024 * 1024:
24
+ raise HTTPException(status_code=400, detail="Spec file exceeds 5 MB limit.")
25
+ decoded_content = content.decode("utf-8", errors="replace")
26
+
27
+ is_postman = False
28
+ try:
29
+ temp_json = json.loads(decoded_content)
30
+ if isinstance(temp_json, dict):
31
+ info = temp_json.get("info", {})
32
+ if isinstance(info, dict) and "postman.com/json/collection" in info.get("schema", ""):
33
+ is_postman = True
34
+ elif "info" in temp_json and "item" in temp_json:
35
+ if "openapi" not in temp_json and "swagger" not in temp_json:
36
+ is_postman = True
37
+ except Exception:
38
+ pass
39
+
40
+ parsed_spec = parse_postman(decoded_content) if is_postman else parse_spec(decoded_content)
41
+ safe_spec = json.loads(json.dumps(parsed_spec, ensure_ascii=True))
42
+
43
+ spec_id = str(uuid.uuid4())
44
+ file_path = f"specs/{spec_id}_{file.filename}"
45
+
46
+ logger.info("Uploading %s to %s", file.filename, file_path)
47
+ await file.seek(0)
48
+ storage.save(file.file, file_path)
49
+
50
+ spec_doc = database.SpecDocument(
51
+ id=spec_id,
52
+ name=file.filename,
53
+ raw_content=decoded_content,
54
+ parsed_json=safe_spec,
55
+ file_path=file_path,
56
+ )
57
+ db.add(spec_doc)
58
+ db.commit()
59
+
60
+ safe_spec["id"] = spec_id
61
+ return {"status": "success", "spec_id": spec_id, "parsed_spec": safe_spec}
62
+ except Exception as e:
63
+ import traceback
64
+ traceback.print_exc()
65
+ raise HTTPException(status_code=400, detail=f"Failed to parse spec: {str(e)}")
66
+
67
+
68
+ @router.get("/specs")
69
+ def list_specs(db: Session = Depends(database.get_db)):
70
+ specs = db.query(database.SpecDocument).order_by(database.SpecDocument.uploaded_at.desc()).all()
71
+ return [
72
+ {"id": s.id, "name": s.name, "base_url": s.base_url or "", "uploaded_at": s.uploaded_at.isoformat()}
73
+ for s in specs
74
+ ]
75
+
76
+
77
+ @router.get("/specs/{spec_id}")
78
+ def get_spec(spec_id: str, db: Session = Depends(database.get_db)):
79
+ spec = db.query(database.SpecDocument).filter(database.SpecDocument.id == spec_id).first()
80
+ if not spec:
81
+ raise HTTPException(status_code=404, detail="Spec not found")
82
+ return {
83
+ "id": spec.id,
84
+ "name": spec.name,
85
+ "base_url": spec.base_url or "",
86
+ "login_credentials": spec.login_credentials or {},
87
+ "parsed_spec": spec.parsed_json,
88
+ "execution_plan": spec.execution_plan,
89
+ "scenario": spec.scenario,
90
+ "uploaded_at": spec.uploaded_at.isoformat(),
91
+ }
92
+
93
+
94
+ @router.patch("/specs/{spec_id}")
95
+ def update_spec(spec_id: str, payload: dict, db: Session = Depends(database.get_db)):
96
+ spec = db.query(database.SpecDocument).filter(database.SpecDocument.id == spec_id).first()
97
+ if not spec:
98
+ raise HTTPException(status_code=404, detail="Spec not found")
99
+ if "base_url" in payload:
100
+ spec.base_url = payload["base_url"]
101
+ db.commit()
102
+ return {"status": "updated", "id": spec.id, "base_url": spec.base_url}
103
+
104
+
105
+ @router.delete("/specs/{spec_id}")
106
+ def delete_spec(spec_id: str, db: Session = Depends(database.get_db)):
107
+ spec = db.query(database.SpecDocument).filter(database.SpecDocument.id == spec_id).first()
108
+ if not spec:
109
+ raise HTTPException(status_code=404, detail="Spec not found")
110
+
111
+ assets = db.query(database.Asset).filter(database.Asset.spec_id == spec_id).all()
112
+ for asset in assets:
113
+ AssetService.delete_asset(db, asset.id)
114
+
115
+ if spec.file_path:
116
+ try:
117
+ storage.delete(spec.file_path)
118
+ except Exception as e:
119
+ logger.warning("Failed to delete spec file %s: %s", spec.file_path, e)
120
+
121
+ db.delete(spec)
122
+ db.commit()
123
+ return {"status": "deleted", "id": spec_id, "assets_removed": len(assets)}
124
+
125
+
126
+ @router.get("/specs/{spec_id}/credentials")
127
+ def get_credentials(spec_id: str, db: Session = Depends(database.get_db)):
128
+ spec = db.query(database.SpecDocument).filter(database.SpecDocument.id == spec_id).first()
129
+ if not spec:
130
+ raise HTTPException(status_code=404, detail="Spec not found")
131
+ return spec.login_credentials or {}
132
+
133
+
134
+ @router.post("/specs/{spec_id}/credentials")
135
+ def save_credentials(spec_id: str, payload: schemas.CredentialUpdate, db: Session = Depends(database.get_db)):
136
+ spec = db.query(database.SpecDocument).filter(database.SpecDocument.id == spec_id).first()
137
+ if not spec:
138
+ raise HTTPException(status_code=404, detail="Spec not found")
139
+
140
+ if payload.extra and payload.extra.get("_clear"):
141
+ spec.login_credentials = {}
142
+ flag_modified(spec, "login_credentials")
143
+ db.commit()
144
+ db.refresh(spec)
145
+ return {"status": "cleared", "credentials": {}}
146
+
147
+ current: dict = dict(spec.login_credentials or {})
148
+ if payload.name is not None:
149
+ current["name"] = payload.name
150
+ if payload.username is not None:
151
+ current["username"] = payload.username
152
+ current["email"] = payload.username
153
+ if payload.password is not None:
154
+ current["password"] = payload.password
155
+ if payload.extra:
156
+ extra_clean = {k: v for k, v in payload.extra.items() if not k.startswith("_")}
157
+ current.update(extra_clean)
158
+
159
+ spec.login_credentials = current
160
+ flag_modified(spec, "login_credentials")
161
+ db.commit()
162
+ db.refresh(spec)
163
+ return {"status": "saved", "credentials": spec.login_credentials}
backend/core/__init__.py ADDED
File without changes
backend/core/config.py ADDED
@@ -0,0 +1,19 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ from pydantic_settings import BaseSettings
2
+
3
+
4
+ class Settings(BaseSettings):
5
+ MISTRAL_API_KEY: str = ""
6
+ DATABASE_URL: str = ""
7
+ ALLOWED_ORIGINS: str = "http://localhost:5173,http://localhost:3000"
8
+ UPLOAD_DIR: str = "backend/uploads"
9
+ LLM_TIMEOUT: int = 60
10
+ HTTP_TIMEOUT: int = 120
11
+
12
+ model_config = {"env_file": ".env", "extra": "ignore"}
13
+
14
+ @property
15
+ def allowed_origins_list(self) -> list[str]:
16
+ return [o.strip() for o in self.ALLOWED_ORIGINS.split(",") if o.strip()]
17
+
18
+
19
+ settings = Settings()
backend/core/errors.py ADDED
@@ -0,0 +1,19 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ from fastapi import Request
2
+ from fastapi.responses import JSONResponse
3
+
4
+
5
+ class AppError(Exception):
6
+ """Raise this anywhere in route/service code for a clean, consistent error response."""
7
+ def __init__(self, message: str, code: str = "ERROR", status: int = 400, detail: dict = None):
8
+ self.message = message
9
+ self.code = code
10
+ self.status = status
11
+ self.detail = detail or {}
12
+ super().__init__(message)
13
+
14
+
15
+ async def app_error_handler(request: Request, exc: AppError) -> JSONResponse:
16
+ return JSONResponse(
17
+ status_code=exc.status,
18
+ content={"error": exc.message, "code": exc.code, "detail": exc.detail},
19
+ )
backend/core/logging.py ADDED
@@ -0,0 +1,15 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ import logging
2
+ import sys
3
+
4
+
5
+ def get_logger(name: str) -> logging.Logger:
6
+ logger = logging.getLogger(name)
7
+ if not logger.handlers:
8
+ handler = logging.StreamHandler(sys.stdout)
9
+ handler.setFormatter(logging.Formatter(
10
+ "%(asctime)s [%(levelname)s] %(name)s: %(message)s",
11
+ datefmt="%Y-%m-%d %H:%M:%S",
12
+ ))
13
+ logger.addHandler(handler)
14
+ logger.setLevel(logging.INFO)
15
+ return logger
backend/core/url_guard.py ADDED
@@ -0,0 +1,50 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ import ipaddress
2
+ import socket
3
+ from urllib.parse import urlparse
4
+
5
+ from fastapi import HTTPException
6
+
7
+ _BLOCKED_NETS = [
8
+ ipaddress.ip_network("127.0.0.0/8"),
9
+ ipaddress.ip_network("10.0.0.0/8"),
10
+ ipaddress.ip_network("172.16.0.0/12"),
11
+ ipaddress.ip_network("192.168.0.0/16"),
12
+ ipaddress.ip_network("0.0.0.0/8"),
13
+ ipaddress.ip_network("169.254.0.0/16"), # AWS/GCP metadata
14
+ ipaddress.ip_network("::1/128"),
15
+ ipaddress.ip_network("fc00::/7"),
16
+ ]
17
+
18
+ _ALLOWED_SCHEMES = {"http", "https"}
19
+
20
+
21
+ def assert_safe_url(url: str) -> None:
22
+ """Raise HTTP 400 if the URL targets a private/internal address or disallowed scheme."""
23
+ try:
24
+ parsed = urlparse(url)
25
+ except Exception:
26
+ raise HTTPException(status_code=400, detail=f"Malformed URL: {url}")
27
+
28
+ if parsed.scheme not in _ALLOWED_SCHEMES:
29
+ raise HTTPException(
30
+ status_code=400,
31
+ detail=f"URL scheme '{parsed.scheme}' is not allowed. Only http/https are permitted.",
32
+ )
33
+
34
+ hostname = parsed.hostname
35
+ if not hostname:
36
+ raise HTTPException(status_code=400, detail="URL has no hostname.")
37
+
38
+ # Resolve to IP and check against blocked ranges
39
+ try:
40
+ ip_str = socket.getaddrinfo(hostname, None)[0][4][0]
41
+ ip = ipaddress.ip_address(ip_str)
42
+ except Exception:
43
+ raise HTTPException(status_code=400, detail=f"Could not resolve hostname: {hostname}")
44
+
45
+ for net in _BLOCKED_NETS:
46
+ if ip in net:
47
+ raise HTTPException(
48
+ status_code=400,
49
+ detail=f"Requests to private/internal addresses are not allowed (resolved to {ip}).",
50
+ )
backend/integrations/__init__.py ADDED
File without changes
backend/integrations/llm/__init__.py ADDED
File without changes
backend/integrations/llm/client.py ADDED
@@ -0,0 +1,34 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ from langchain_mistralai.chat_models import ChatMistralAI
2
+ from backend.core.logging import get_logger
3
+
4
+ logger = get_logger(__name__)
5
+
6
+ _MODEL = "mistral-large-latest"
7
+
8
+
9
+ def _extract_usage(ai_msg) -> dict | None:
10
+ usage = ai_msg.response_metadata.get("token_usage", {})
11
+ if not usage:
12
+ return None
13
+ return {
14
+ "prompt_tokens": usage.get("prompt_tokens", 0),
15
+ "completion_tokens": usage.get("completion_tokens", 0),
16
+ "total_tokens": usage.get("total_tokens", 0),
17
+ }
18
+
19
+
20
+ class MistralClient:
21
+ """Centralized Mistral LLM client. All services use the module-level instances below."""
22
+
23
+ def __init__(self, temperature: float = 0, timeout: int = 120):
24
+ self._llm = ChatMistralAI(model_name=_MODEL, temperature=temperature, timeout=timeout)
25
+
26
+ def invoke(self, messages) -> tuple[str, dict | None]:
27
+ """Invoke the LLM. Returns (content_str, token_usage_or_None)."""
28
+ ai_msg = self._llm.invoke(messages)
29
+ return ai_msg.content, _extract_usage(ai_msg)
30
+
31
+
32
+ # One instance per temperature setting β€” instantiated once per process, not per request.
33
+ deterministic = MistralClient(temperature=0)
34
+ chat = MistralClient(temperature=0.2)
backend/main.py CHANGED
@@ -1,935 +1,65 @@
1
- from fastapi import FastAPI, Depends, HTTPException, File, UploadFile, Form
2
- from fastapi.staticfiles import StaticFiles
3
- from sqlalchemy.orm import Session
4
- from sqlalchemy.orm.attributes import flag_modified
5
- from . import database, schemas
6
- from typing import Dict, Any, Optional
7
- import httpx
8
- import json
9
- import uuid
10
  import os
11
- import shutil
12
- from fastapi.middleware.cors import CORSMiddleware
13
  from fastapi.exceptions import RequestValidationError
 
14
  from fastapi.responses import JSONResponse
 
 
 
 
15
 
16
- from backend.services.swagger_parser import parse_spec
17
- from backend.services.postman_parser import parse_postman_collection as parse_postman
18
- from backend.services.llm_planner import generate_execution_plan
19
- from backend.services.api_executor import prepare_step_request
20
- from backend.services.response_analyzer import analyze_api_response
21
- from backend.services.asset_service import storage
22
- from backend.services.context_manager import get_context, set_context_variables
23
- from backend.services.asset_service import AssetService
24
- from backend.services.chat_assistant import process_message as chat_process_message
25
 
 
26
 
27
- # Ensure uploads subdirectories exist
28
- UPLOAD_DIR = "backend/uploads"
29
- CATEGORIES = ["audio", "video", "documents", "images"]
30
- for cat in CATEGORIES:
31
- os.makedirs(os.path.join(UPLOAD_DIR, cat), exist_ok=True)
32
 
33
- app = FastAPI(title="AI API Testing Assistant")
 
 
34
 
35
- # Mount static files for uploads
36
- app.mount("/uploads", StaticFiles(directory=UPLOAD_DIR), name="uploads")
37
 
38
- @app.exception_handler(RequestValidationError)
39
- async def validation_exception_handler(request, exc):
40
- print(f"VALIDATION ERROR: {exc.errors()}")
41
- print(f"BODY: {await request.body()}")
42
- return JSONResponse(
43
- status_code=422,
44
- content={"detail": exc.errors(), "body": str(await request.body())},
45
- )
46
 
47
- # Configure CORS
 
 
 
48
  app.add_middleware(
49
  CORSMiddleware,
50
- allow_origins=["*"], # Allow all origins for production ease
51
  allow_credentials=True,
52
  allow_methods=["*"],
53
  allow_headers=["*"],
54
  )
55
 
56
- @app.get("/")
57
- def read_root():
58
- return {"status": "ok", "message": "AI API Testing Tool Backend running."}
59
-
60
- @app.post("/session")
61
- def create_session(db: Session = Depends(database.get_db)):
62
- session_id = str(uuid.uuid4())
63
- new_session = database.SessionModel(id=session_id)
64
- db.add(new_session)
65
- db.commit()
66
- return {"session_id": session_id}
67
-
68
- @app.post("/upload-spec")
69
- async def upload_spec(file: UploadFile = File(...), db: Session = Depends(database.get_db)):
70
- try:
71
- content = await file.read()
72
- decoded_content = content.decode("utf-8", errors="replace")
73
-
74
- # Detect if it's a Postman collection
75
- is_postman = False
76
- try:
77
- temp_json = json.loads(decoded_content)
78
- if isinstance(temp_json, dict):
79
- # Primary check: schema URL
80
- info = temp_json.get("info", {})
81
- if isinstance(info, dict) and "postman.com/json/collection" in info.get("schema", ""):
82
- is_postman = True
83
- # Secondary check: unique structure (info + item)
84
- elif "info" in temp_json and "item" in temp_json:
85
- # Swagger/OpenAPI usually has 'openapi' or 'swagger' key at top level
86
- if "openapi" not in temp_json and "swagger" not in temp_json:
87
- is_postman = True
88
- except:
89
- pass
90
-
91
- if is_postman:
92
- parsed_spec = parse_postman(decoded_content)
93
- else:
94
- parsed_spec = parse_spec(decoded_content)
95
-
96
- safe_spec = json.loads(json.dumps(parsed_spec, ensure_ascii=True))
97
-
98
- spec_id = str(uuid.uuid4())
99
- # PERSIST to Storage (Supabase or Local)
100
- file_path = f"specs/{spec_id}_{file.filename}"
101
-
102
- # Log for debugging
103
- print(f"[DEBUG] Uploading {file.filename} to {file_path}")
104
-
105
- # Reset cursor since we read it for parsing
106
- await file.seek(0)
107
- storage.save(file.file, file_path)
108
-
109
- # Persist the spec to the database
110
- spec_doc = database.SpecDocument(
111
- id=spec_id,
112
- name=file.filename,
113
- raw_content=decoded_content,
114
- parsed_json=safe_spec,
115
- file_path=file_path
116
- )
117
- db.add(spec_doc)
118
- db.commit()
119
-
120
- # Prepare return object including the ID for the frontend
121
- safe_spec["id"] = spec_id
122
-
123
- return {"status": "success", "spec_id": spec_id, "parsed_spec": safe_spec}
124
- except Exception as e:
125
- print(f"[ERROR] upload_spec failed: {str(e)}")
126
- import traceback
127
- traceback.print_exc()
128
- raise HTTPException(status_code=400, detail=str(e))
129
- except Exception as e:
130
- raise HTTPException(status_code=400, detail=f"Failed to parse spec: {str(e)}")
131
-
132
- @app.get("/specs")
133
- def list_specs(db: Session = Depends(database.get_db)):
134
- specs = db.query(database.SpecDocument).order_by(database.SpecDocument.uploaded_at.desc()).all()
135
- return [
136
- {"id": s.id, "name": s.name, "base_url": s.base_url or "", "uploaded_at": s.uploaded_at.isoformat()}
137
- for s in specs
138
- ]
139
-
140
- @app.get("/specs/{spec_id}")
141
- def get_spec(spec_id: str, db: Session = Depends(database.get_db)):
142
- spec = db.query(database.SpecDocument).filter(database.SpecDocument.id == spec_id).first()
143
- if not spec:
144
- raise HTTPException(status_code=404, detail="Spec not found")
145
- return {
146
- "id": spec.id,
147
- "name": spec.name,
148
- "base_url": spec.base_url or "",
149
- "login_credentials": spec.login_credentials or {},
150
- "parsed_spec": spec.parsed_json,
151
- "execution_plan": spec.execution_plan,
152
- "scenario": spec.scenario,
153
- "uploaded_at": spec.uploaded_at.isoformat()
154
- }
155
-
156
- @app.patch("/specs/{spec_id}")
157
- def update_spec(spec_id: str, payload: dict, db: Session = Depends(database.get_db)):
158
- spec = db.query(database.SpecDocument).filter(database.SpecDocument.id == spec_id).first()
159
- if not spec:
160
- raise HTTPException(status_code=404, detail="Spec not found")
161
- if "base_url" in payload:
162
- spec.base_url = payload["base_url"]
163
- db.commit()
164
- return {"status": "updated", "id": spec.id, "base_url": spec.base_url}
165
-
166
-
167
- def get_category(filename: str, content_type: str) -> str:
168
- ext = os.path.splitext(filename)[1].lower()
169
- if ext in ['.mp3', '.wav', '.ogg', '.m4a'] or (content_type and content_type.startswith('audio/')):
170
- return 'audio'
171
- if ext in ['.mp4', '.mov', '.avi', '.mkv'] or (content_type and content_type.startswith('video/')):
172
- return 'video'
173
- return 'documents'
174
-
175
- @app.post("/assets")
176
- async def upload_asset(
177
- spec_id: str = Form(None),
178
- collection_id: str = Form(None),
179
- description: str = Form(None),
180
- tags: str = Form("[]"),
181
- file: UploadFile = File(...),
182
- db: Session = Depends(database.get_db)
183
- ):
184
- try:
185
- tag_list = json.loads(tags)
186
- asset = AssetService.upload_asset(db, file, spec_id, collection_id, description, tag_list)
187
- return {
188
- "id": asset.id,
189
- "name": asset.name,
190
- "type": asset.file_type,
191
- "size": asset.file_size,
192
- "category": asset.category,
193
- "description": asset.description,
194
- "tags": asset.tags,
195
- "spec_id": asset.spec_id,
196
- "uploaded_at": asset.uploaded_at.isoformat()
197
- }
198
- except Exception as e:
199
- raise HTTPException(status_code=500, detail=f"Upload failed: {str(e)}")
200
-
201
- @app.get("/assets")
202
- def list_assets(spec_id: Optional[str] = None, category: Optional[str] = None, db: Session = Depends(database.get_db)):
203
- assets = AssetService.get_assets(db, spec_id, category)
204
- return [
205
- {
206
- "id": a.id,
207
- "name": a.name,
208
- "type": a.file_type,
209
- "size": a.file_size,
210
- "category": a.category,
211
- "description": a.description,
212
- "tags": a.tags,
213
- "spec_id": a.spec_id,
214
- "path": a.file_path,
215
- "uploaded_at": a.uploaded_at.isoformat(),
216
- "collection_ids": [c.id for c in a.collections]
217
- }
218
- for a in assets
219
- ]
220
-
221
- @app.delete("/assets/{asset_id}")
222
- def delete_asset(asset_id: str, db: Session = Depends(database.get_db)):
223
- success = AssetService.delete_asset(db, asset_id)
224
- if not success:
225
- raise HTTPException(status_code=404, detail="Asset not found")
226
- return {"status": "deleted", "id": asset_id}
227
-
228
- @app.get("/collections")
229
- def list_collections(spec_id: Optional[str] = None, db: Session = Depends(database.get_db)):
230
- query = db.query(database.Collection)
231
- if spec_id:
232
- from sqlalchemy import or_
233
- query = query.filter(or_(database.Collection.spec_id == spec_id, database.Collection.spec_id == None))
234
- cols = query.all()
235
- return [{"id": c.id, "name": c.name, "description": c.description, "spec_id": c.spec_id} for c in cols]
236
-
237
- @app.post("/collections")
238
- def create_collection(name: str = Form(...), spec_id: str = Form(None), description: str = Form(None), db: Session = Depends(database.get_db)):
239
- col = AssetService.create_collection(db, name, spec_id, description)
240
- return {"id": col.id, "name": col.name, "spec_id": col.spec_id}
241
-
242
- @app.post("/collections/{collection_id}/assets/{asset_id}")
243
- def add_asset_to_collection(collection_id: str, asset_id: str, db: Session = Depends(database.get_db)):
244
- success = AssetService.add_to_collection(db, asset_id, collection_id)
245
- if not success:
246
- raise HTTPException(status_code=404, detail="Asset or Collection not found")
247
- return {"status": "success"}
248
-
249
- @app.delete("/collections/{collection_id}")
250
- def delete_collection(collection_id: str, db: Session = Depends(database.get_db)):
251
- success = AssetService.delete_collection(db, collection_id)
252
- if not success:
253
- raise HTTPException(status_code=404, detail="Collection not found")
254
- return {"status": "success"}
255
-
256
- @app.delete("/specs/{spec_id}")
257
- def delete_spec(spec_id: str, db: Session = Depends(database.get_db)):
258
- spec = db.query(database.SpecDocument).filter(database.SpecDocument.id == spec_id).first()
259
- if not spec:
260
- raise HTTPException(status_code=404, detail="Spec not found")
261
-
262
- # Delete all associated assets (storage + DB)
263
- assets = db.query(database.Asset).filter(database.Asset.spec_id == spec_id).all()
264
- for asset in assets:
265
- AssetService.delete_asset(db, asset.id)
266
-
267
- # Delete spec file from storage
268
- if spec.file_path:
269
- try:
270
- storage.delete(spec.file_path)
271
- except Exception as e:
272
- print(f"[CLEANUP ERROR] Failed to delete spec file {spec.file_path}: {e}")
273
-
274
- db.delete(spec)
275
- db.commit()
276
- return {"status": "deleted", "id": spec_id, "assets_removed": len(assets)}
277
-
278
- @app.post("/plan", response_model=schemas.ExecutionPlan)
279
- def generate_plan(request: schemas.PlanRequest, db: Session = Depends(database.get_db)):
280
- try:
281
- # Expected `request.spec_content` to be the dict from `/upload-spec`
282
- parsed_spec = json.loads(request.spec_content) if isinstance(request.spec_content, str) else request.spec_content
283
-
284
- # Override with DB-stored credentials if a spec_id is provided
285
- credentials = request.user_credentials or {}
286
- if request.spec_id:
287
- spec = db.query(database.SpecDocument).filter(database.SpecDocument.id == request.spec_id).first()
288
- if spec and spec.login_credentials:
289
- credentials = spec.login_credentials
290
-
291
- plan = generate_execution_plan(
292
- parsed_spec,
293
- request.base_url,
294
- request.scenario,
295
- credentials
296
- )
297
-
298
- # PERSIST: Save the generated plan and update session metadata
299
- if request.spec_id:
300
- spec = db.query(database.SpecDocument).filter(database.SpecDocument.id == request.spec_id).first()
301
- if spec:
302
- spec.execution_plan = plan.model_dump()["execution_plan"]
303
- spec.scenario = request.scenario
304
- flag_modified(spec, "execution_plan")
305
- flag_modified(spec, "scenario")
306
-
307
- # Update session metadata if session_id provided
308
- if request.session_id:
309
- session = db.query(database.SessionModel).filter(database.SessionModel.id == request.session_id).first()
310
- if session:
311
- session.spec_id = spec.id
312
- session.spec_name = spec.name
313
- session.scenario = request.scenario
314
- session.base_url = request.base_url
315
-
316
- # NEW: Merge AI-generated credentials if present
317
- if plan.suggested_credentials and isinstance(plan.suggested_credentials, dict):
318
- current_creds = dict(spec.login_credentials or {})
319
- # Only merge if there's actually something new to add
320
- if any(k not in current_creds or current_creds[k] != v for k, v in plan.suggested_credentials.items()):
321
- current_creds.update(plan.suggested_credentials)
322
- spec.login_credentials = current_creds
323
- flag_modified(spec, "login_credentials")
324
- print(f"[AUTH MERGE] AI suggested credentials for '{spec.name}' merged into DB.")
325
-
326
- db.commit()
327
-
328
- return plan
329
- except Exception as e:
330
- raise HTTPException(status_code=500, detail=f"Planning failed: {str(e)}")
331
-
332
- @app.post("/execute-step")
333
- async def execute_step(request: schemas.ExecuteStepRequest, db: Session = Depends(database.get_db)):
334
- try:
335
- # Extract endpoint string (e.g., "GET /buses/{bus_id}" or "/buses/{bus_id}")
336
- endpoint_str = request.step_definition.get("endpoint", "")
337
-
338
- def resolve_refs(obj, full_spec, _resolving=None):
339
- """Recursively inline all $ref pointers so the LLM gets a flat, self-contained spec."""
340
- if _resolving is None:
341
- _resolving = set()
342
- if isinstance(obj, dict):
343
- if "$ref" in obj:
344
- ref = obj["$ref"]
345
- if ref in _resolving:
346
- return {"$ref_loop": ref}
347
- _resolving = _resolving | {ref}
348
- # Only support local refs: "#/components/schemas/Foo"
349
- parts = ref.lstrip("#/").split("/")
350
- target = full_spec
351
- for part in parts:
352
- target = target.get(part, {}) if isinstance(target, dict) else {}
353
- return resolve_refs(target, full_spec, _resolving)
354
- return {k: resolve_refs(v, full_spec, _resolving) for k, v in obj.items()}
355
- elif isinstance(obj, list):
356
- return [resolve_refs(item, full_spec, _resolving) for item in obj]
357
- return obj
358
-
359
- def shorthand_spec(obj):
360
- """Convert JSON schema to ultra-compact shorthand string including enums and formats."""
361
- if isinstance(obj, dict):
362
- if "type" in obj:
363
- t = obj["type"]
364
- fmt = f":{obj['format']}" if "format" in obj else ""
365
- enum_vals = f"[{'|'.join(map(str, obj['enum']))}]" if "enum" in obj else ""
366
-
367
- if t == "object" and "properties" in obj:
368
- req_list = obj.get("required", [])
369
- props = []
370
- for k, v in obj["properties"].items():
371
- req = "*" if k in req_list or v.get("nullable") == False else ""
372
- props.append(f"{k}{req}({shorthand_spec(v)})")
373
- return "{" + ",".join(props) + "}"
374
- if t == "array" and "items" in obj:
375
- return f"[{shorthand_spec(obj['items'])}]"
376
-
377
- return f"{t}{fmt}{enum_vals}"
378
- return {k: shorthand_spec(v) for k, v in obj.items()}
379
- return str(obj)
380
-
381
- def prune_spec_for_execution(obj):
382
- """Recursively remove non-functional metadata (descriptions, examples, formats) to save tokens."""
383
- if isinstance(obj, dict):
384
- # Keys we definitely don't need for building an HTTP request
385
- DROP_KEYS = {"description", "summary", "example", "examples", "format", "title", "default", "enum_descriptions"}
386
- pruned = {}
387
- for k, v in obj.items():
388
- if k in DROP_KEYS:
389
- continue
390
- pruned[k] = prune_spec_for_execution(v)
391
- return pruned
392
- elif isinstance(obj, list):
393
- return [prune_spec_for_execution(item) for item in obj]
394
- return obj
395
-
396
- actual_endpoint_spec = {"endpoint": endpoint_str}
397
-
398
- if request.spec_id:
399
- spec_doc = db.query(database.SpecDocument).filter(database.SpecDocument.id == request.spec_id).first()
400
- if spec_doc and spec_doc.parsed_json:
401
- full_spec = spec_doc.parsed_json
402
- paths = full_spec.get("paths", {})
403
- parts = endpoint_str.split(" ", 1)
404
- def _match_path(path_template, actual_path):
405
- """Simple matcher for OpenAPI path templates like /buses/{bus_id}"""
406
- import re
407
- # Replace {param} with a regex group
408
- pattern = re.sub(r'\{[^}]+\}', r'[^/]+', path_template)
409
- # Handle trailing slashes
410
- pattern = "^" + pattern.rstrip("/") + "/?$"
411
- return bool(re.match(pattern, actual_path.split("?")[0]))
412
-
413
- parts = endpoint_str.split(" ", 1)
414
- path_only = parts[1] if len(parts) == 2 else parts[0]
415
-
416
- # Try to find the exact or template match
417
- matched_path = None
418
- if path_only in paths:
419
- matched_path = path_only
420
- else:
421
- for p in paths:
422
- if _match_path(p, path_only):
423
- matched_path = p
424
- break
425
-
426
- if matched_path:
427
- raw_spec = {matched_path: paths[matched_path]}
428
- # Inline all $ref so LLM sees the exact required fields
429
- resolved = resolve_refs(raw_spec, full_spec)
430
-
431
- # Optimization: Prune 'responses' as they are not needed for building the request
432
- path_data = resolved.get(matched_path, {})
433
- for method in path_data:
434
- if isinstance(path_data[method], dict):
435
- path_data[method].pop("responses", None)
436
-
437
- # Pass through aggressive pruning to remove descriptions, examples, etc.
438
- pruned = prune_spec_for_execution(resolved)
439
- # CONVERT TO SHORTHAND
440
- actual_endpoint_spec = shorthand_spec(pruned)
441
- else:
442
- print(f"[PATH MATCH FAIL] Could not find match for {path_only}. Falling back to simplified path list.")
443
- # Fallback: Just send the relevant path/method info if we can't find the full schema
444
- actual_endpoint_spec = {"endpoint": endpoint_str, "warning": "Full schema match failed"}
445
-
446
- # Fetch semantic assets for AI context
447
- rich_assets = AssetService.get_ai_context(db, request.spec_id)
448
-
449
- # NEW: Fallback to stored credentials if none provided in request
450
- effective_credentials = request.user_credentials
451
- if not effective_credentials and request.spec_id:
452
- spec = db.query(database.SpecDocument).filter(database.SpecDocument.id == request.spec_id).first()
453
- if spec and spec.login_credentials:
454
- effective_credentials = spec.login_credentials
455
- print(f"[AUTH FALLBACK] Using stored credentials for spec '{spec.name}'")
456
-
457
- # 1. Ask LLM to format the actual httpx request
458
- req_plan, executor_token_usage = prepare_step_request(
459
- endpoint_spec=actual_endpoint_spec,
460
- base_url=request.base_url,
461
- scenario="",
462
- step_definition=request.step_definition,
463
- context_variables=request.context_variables,
464
- user_credentials=effective_credentials,
465
- available_assets=rich_assets
466
- )
467
-
468
- # Sanitize URL: fix relative, missing-scheme, or placeholder URLs from the LLM
469
- url = req_plan.request.url.strip()
470
- PLACEHOLDER_HOSTS = {"api.example.com", "example.com", "<base_url>", "your-api.com"}
471
 
472
- def _is_placeholder(u: str) -> bool:
473
- try:
474
- from urllib.parse import urlparse
475
- host = urlparse(u).hostname or ""
476
- return host.lower() in PLACEHOLDER_HOSTS or u.startswith("<BASE_URL>")
477
- except Exception:
478
- return False
479
-
480
- if url.startswith("/"):
481
- # Relative path β€” prepend base URL
482
- url = request.base_url.rstrip("/") + url
483
- elif _is_placeholder(url):
484
- # LLM returned a placeholder domain β€” extract just the path and rebuild
485
- from urllib.parse import urlparse
486
- parsed = urlparse(url)
487
- path = parsed.path or "/"
488
- url = request.base_url.rstrip("/") + path
489
- if parsed.query:
490
- url += "?" + parsed.query
491
- print(f"[URL FIX] Placeholder URL detected, rebuilt to: {url}")
492
- elif not url.startswith("http"):
493
- url = f"https://{url}"
494
-
495
- # Deduplicate query parameters gently: remove from dict if already in URL string
496
- from urllib.parse import urlparse, parse_qsl
497
- parsed = urlparse(url)
498
- if parsed.query:
499
- url_keys = set(k for k, v in parse_qsl(parsed.query))
500
- # Remove keys from query_params if they are already hardcoded in the url
501
- keys_to_delete = [k for k in req_plan.request.query_params if k in url_keys]
502
- for k in keys_to_delete:
503
- del req_plan.request.query_params[k]
504
-
505
- # 2. Execute via httpx
506
- async with httpx.AsyncClient(timeout=120.0) as client:
507
- http_req = client.build_request(
508
- method=req_plan.request.method.strip().upper(),
509
- url=url,
510
- headers=req_plan.request.headers,
511
- params=req_plan.request.query_params,
512
- data=req_plan.request.form_data if req_plan.request.form_data else None,
513
- json=req_plan.request.body if req_plan.request.body and not req_plan.request.form_data else None
514
- )
515
-
516
- # Handle File Attachments if provided by LLM
517
- files_payload = {}
518
- opened_files = []
519
-
520
- try:
521
- if req_plan.request.files:
522
- # Strip manually set Content-Type if files are present to let httpx handle the boundary
523
- original_headers = {k: v for k, v in req_plan.request.headers.items() if k.lower() != 'content-type'}
524
- req_plan.request.headers = original_headers
525
-
526
- for field_name, file_name in req_plan.request.files.items():
527
- # Find asset record by name
528
- asset = db.query(database.Asset).filter(database.Asset.name == file_name).first()
529
- if asset:
530
- # Use StorageProvider to get full path
531
- from backend.services.asset_service import storage
532
- full_path = storage.get_full_path(asset.file_path)
533
- if os.path.exists(full_path):
534
- # Open file for reading
535
- f = open(full_path, "rb")
536
- opened_files.append(f)
537
- files_payload[field_name] = (asset.name, f, asset.file_type)
538
- else:
539
- print(f"[ASSET ERROR] File '{file_name}' not found on disk at {full_path}")
540
- else:
541
- print(f"[ASSET ERROR] File '{file_name}' not found in Media Hub.")
542
-
543
- # Print to backend console for debugging
544
- print(f"Executing: {http_req.method} {http_req.url}")
545
-
546
- try:
547
- if files_payload:
548
- response = await client.request(
549
- method=req_plan.request.method.strip().upper(),
550
- url=url,
551
- headers=req_plan.request.headers,
552
- params=req_plan.request.query_params,
553
- data=req_plan.request.form_data if req_plan.request.form_data else None,
554
- files=files_payload
555
- )
556
- else:
557
- response = await client.send(http_req)
558
- except Exception as e:
559
- raise HTTPException(status_code=500, detail=f"Request to {url} failed: {str(e)}")
560
- finally:
561
- # CLOSE ALL FILE HANDLES
562
- for f in opened_files:
563
- f.close()
564
-
565
- # Format the raw response
566
- resp_data = {
567
- "status_code": response.status_code,
568
- "body": response.text,
569
- "headers": dict(response.headers)
570
- }
571
-
572
- # Update the plan to reflect any URL cleanup
573
- req_plan.request.url = url
574
-
575
- # 3. Save to History
576
- history = database.ExecutionHistory(
577
- session_id=request.session_id,
578
- step=req_plan.step,
579
- request_json=req_plan.request.model_dump(),
580
- response_json=resp_data,
581
- step_definition_json=request.step_definition
582
- )
583
- db.add(history)
584
- db.commit()
585
-
586
- return {
587
- "executed_request": req_plan.request.model_dump(),
588
- "response": resp_data,
589
- "analysis_needed": req_plan.step,
590
- "token_usage": executor_token_usage
591
- }
592
- except Exception as e:
593
- import traceback
594
- trace = traceback.format_exc()
595
- print(f"EXECUTION ERROR: {trace}")
596
- raise HTTPException(status_code=500, detail=f"Execution failed: {str(e)}\n{trace}")
597
-
598
- @app.post("/analyze-response", response_model=schemas.AnalyzeResponse)
599
- def analyze_response_endpoint(request: schemas.AnalyzeRequest, db: Session = Depends(database.get_db)):
600
- try:
601
- step_def = request.step_definition or {}
602
- endpoint_str = step_def.get("endpoint", "Unknown")
603
- purpose_str = step_def.get("purpose", "Unknown")
604
-
605
- # 1. Analyze it
606
- analysis = analyze_api_response(
607
- step=request.step,
608
- endpoint=endpoint_str,
609
- purpose=purpose_str,
610
- status_code=request.response_data.get("status_code", 0),
611
- response_body=request.response_data.get("body", "")
612
- )
613
-
614
- # 2. Save extracted variables ONLY to session context (session-scoped, not persisted to spec)
615
- if analysis.extracted_variables:
616
- set_context_variables(db, request.session_id, analysis.extracted_variables)
617
-
618
- # 3. Update History
619
- history = db.query(database.ExecutionHistory).filter(
620
- database.ExecutionHistory.session_id == request.session_id,
621
- database.ExecutionHistory.step == request.step
622
- ).order_by(database.ExecutionHistory.id.desc()).first()
623
-
624
- if history:
625
- history.analysis_json = analysis.model_dump()
626
- db.commit()
627
-
628
- return analysis
629
- except Exception as e:
630
- raise HTTPException(status_code=500, detail=f"Analysis failed: {str(e)}")
631
-
632
- # ─────────────────────────────────────────────
633
- # CREDENTIAL ENDPOINTS (per spec)
634
- # ─────────────────────────────────────────────
635
-
636
- @app.get("/specs/{spec_id}/credentials")
637
- def get_credentials(spec_id: str, db: Session = Depends(database.get_db)):
638
- """Return stored login credentials for a spec."""
639
- spec = db.query(database.SpecDocument).filter(database.SpecDocument.id == spec_id).first()
640
- if not spec:
641
- raise HTTPException(status_code=404, detail="Spec not found")
642
- return spec.login_credentials or {}
643
-
644
- @app.post("/specs/{spec_id}/credentials")
645
- def save_credentials(spec_id: str, payload: schemas.CredentialUpdate, db: Session = Depends(database.get_db)):
646
- """Save/merge login credentials to a spec. Called manually from the UI."""
647
- spec = db.query(database.SpecDocument).filter(database.SpecDocument.id == spec_id).first()
648
- if not spec:
649
- raise HTTPException(status_code=404, detail="Spec not found")
650
-
651
- # If _clear flag is present, wipe all credentials
652
- if payload.extra and payload.extra.get("_clear"):
653
- spec.login_credentials = {}
654
- flag_modified(spec, "login_credentials")
655
- db.commit()
656
- db.refresh(spec)
657
- return {"status": "cleared", "credentials": {}}
658
-
659
- current: dict = dict(spec.login_credentials or {})
660
- if payload.name is not None:
661
- current["name"] = payload.name
662
- if payload.username is not None:
663
- current["username"] = payload.username
664
- current["email"] = payload.username # store under both keys
665
- if payload.password is not None:
666
- current["password"] = payload.password
667
- if payload.extra:
668
- # Filter out internal-only keys like _clear before saving
669
- extra_clean = {k: v for k, v in payload.extra.items() if not k.startswith("_")}
670
- current.update(extra_clean)
671
- spec.login_credentials = current
672
- flag_modified(spec, "login_credentials")
673
- db.commit()
674
- db.refresh(spec)
675
- return {"status": "saved", "credentials": spec.login_credentials}
676
-
677
- @app.get("/context/{session_id}")
678
- def get_context_endpoint(session_id: str, db: Session = Depends(database.get_db)):
679
- """Return only session context variables (dynamic values from previous steps)."""
680
- context = get_context(db, session_id)
681
- return context
682
-
683
- @app.post("/context/{session_id}")
684
- def update_context_endpoint(session_id: str, payload: schemas.ContextUpdate, db: Session = Depends(database.get_db)):
685
- """Update session context variables manually (e.g., from selection)."""
686
- # Assuming set_context_variables exists and merges the dict
687
- variables = {payload.key: payload.value}
688
- updated_context = set_context_variables(db, session_id, variables)
689
- return updated_context
690
-
691
- @app.delete("/session/{session_id}")
692
- def clear_session(session_id: str, db: Session = Depends(database.get_db)):
693
- session = db.query(database.SessionModel).filter(database.SessionModel.id == session_id).first()
694
- if session:
695
- db.delete(session)
696
- db.query(database.ExecutionHistory).filter(database.ExecutionHistory.session_id == session_id).delete()
697
- db.commit()
698
- return {"status": "ok"}
699
-
700
- @app.get("/history")
701
- def get_all_history(db: Session = Depends(database.get_db)):
702
- """Fetch all execution history joined with session metadata."""
703
- results = db.query(
704
- database.ExecutionHistory,
705
- database.SessionModel
706
- ).join(
707
- database.SessionModel,
708
- database.ExecutionHistory.session_id == database.SessionModel.id
709
- ).order_by(database.ExecutionHistory.timestamp.desc()).all()
710
-
711
- history_list = []
712
- for hist, sess in results:
713
- history_list.append({
714
- "id": hist.id,
715
- "session_id": hist.session_id,
716
- "spec_id": sess.spec_id,
717
- "spec_name": sess.spec_name or "Unknown",
718
- "scenario": sess.scenario or "No scenario",
719
- "context_variables": sess.context_variables or {},
720
- "step": hist.step,
721
- "request": hist.request_json,
722
- "response": hist.response_json,
723
- "analysis": hist.analysis_json,
724
- "step_definition": hist.step_definition_json,
725
- "timestamp": hist.timestamp.isoformat()
726
- })
727
- return history_list
728
-
729
- @app.delete("/history/{session_id}")
730
- def delete_history_session(session_id: str, db: Session = Depends(database.get_db)):
731
- """Delete a specific session and all its execution history."""
732
- db.query(database.ExecutionHistory).filter(database.ExecutionHistory.session_id == session_id).delete()
733
- db.query(database.SessionModel).filter(database.SessionModel.id == session_id).delete()
734
- db.commit()
735
- return {"status": "deleted", "session_id": session_id}
736
-
737
- @app.delete("/history")
738
- def clear_all_history_endpoint(db: Session = Depends(database.get_db)):
739
- """Wipe all execution history and session metadata."""
740
- db.query(database.ExecutionHistory).delete()
741
- db.query(database.SessionModel).delete()
742
- db.commit()
743
- return {"status": "cleared"}
744
-
745
-
746
- # ─────────────────────────────────────────────
747
- # CHAT ASSISTANT ENDPOINTS
748
- # ─────────────────────────────────────────────
749
-
750
- @app.post("/chat", response_model=schemas.ChatResponse)
751
- def chat_endpoint(request: schemas.ChatRequest, db: Session = Depends(database.get_db)):
752
- """
753
- Send a message to the AI chat assistant.
754
- - If chat_session_id is None β†’ auto-creates a new named chat session.
755
- - Saves user message + AI response to DB (persistent across refreshes).
756
- - Returns a structured action so the frontend can trigger executions, etc.
757
- """
758
- try:
759
- # 1. Resolve or create chat session
760
- chat_session = None
761
- if request.chat_session_id:
762
- chat_session = db.query(database.ChatSession).filter(
763
- database.ChatSession.id == request.chat_session_id
764
- ).first()
765
-
766
- if not chat_session:
767
- # Auto-name from first message (truncate to 60 chars)
768
- session_name = request.message[:60].strip()
769
- if len(request.message) > 60:
770
- session_name += "..."
771
- chat_session = database.ChatSession(
772
- id=str(uuid.uuid4()),
773
- name=session_name,
774
- spec_id=request.spec_id,
775
- session_id=request.session_id,
776
- )
777
- db.add(chat_session)
778
- db.flush() # get the ID before committing
779
-
780
- # 2. Save user message to DB
781
- user_msg = database.ChatMessage(
782
- chat_session_id=chat_session.id,
783
- role="user",
784
- content=request.message,
785
- )
786
- db.add(user_msg)
787
- db.flush()
788
-
789
- # 3. Load chat history for context window (last 20 messages)
790
- history_rows = db.query(database.ChatMessage).filter(
791
- database.ChatMessage.chat_session_id == chat_session.id
792
- ).order_by(database.ChatMessage.id.asc()).all()
793
-
794
- chat_history = [
795
- {"role": m.role, "content": m.content}
796
- for m in history_rows[:-1] # exclude the message we just inserted
797
- ]
798
-
799
- # 4. Call the AI
800
- ai_result = chat_process_message(
801
- message=request.message,
802
- spec_name=request.spec_name,
803
- execution_plan=request.execution_plan or [],
804
- context_variables=request.context_variables or {},
805
- last_step_result=request.last_step_result,
806
- all_step_results=request.all_step_results,
807
- chat_history=chat_history,
808
- )
809
-
810
- # 5. Save assistant response to DB
811
- assistant_content = ai_result.get("message", "")
812
- assistant_msg = database.ChatMessage(
813
- chat_session_id=chat_session.id,
814
- role="assistant",
815
- content=assistant_content,
816
- action_json=ai_result,
817
- )
818
- db.add(assistant_msg)
819
-
820
- # 6. Update session timestamp
821
- chat_session.updated_at = __import__("datetime").datetime.utcnow()
822
- db.commit()
823
-
824
- return schemas.ChatResponse(
825
- chat_session_id=chat_session.id,
826
- chat_session_name=chat_session.name,
827
- action=ai_result.get("action", "reply"),
828
- message=assistant_content,
829
- step=ai_result.get("step"),
830
- changes=ai_result.get("changes"),
831
- assertions=ai_result.get("assertions"),
832
- explanation=ai_result.get("explanation"),
833
- )
834
-
835
- except Exception as e:
836
- import traceback
837
- traceback.print_exc()
838
- raise HTTPException(status_code=500, detail=f"Chat failed: {str(e)}")
839
-
840
-
841
- @app.get("/chat-sessions")
842
- def list_chat_sessions(spec_id: Optional[str] = None, db: Session = Depends(database.get_db)):
843
- """List all chat sessions, optionally filtered by spec_id. Sorted newest first."""
844
- query = db.query(database.ChatSession)
845
- if spec_id:
846
- query = query.filter(database.ChatSession.spec_id == spec_id)
847
- sessions = query.order_by(database.ChatSession.updated_at.desc()).all()
848
-
849
- result = []
850
- for s in sessions:
851
- msg_count = db.query(database.ChatMessage).filter(
852
- database.ChatMessage.chat_session_id == s.id
853
- ).count()
854
- result.append({
855
- "id": s.id,
856
- "name": s.name,
857
- "spec_id": s.spec_id,
858
- "session_id": s.session_id,
859
- "message_count": msg_count,
860
- "created_at": s.created_at.isoformat() if s.created_at else None,
861
- "updated_at": s.updated_at.isoformat() if s.updated_at else None,
862
- })
863
- return result
864
-
865
-
866
- @app.get("/chat-sessions/{chat_session_id}")
867
- def get_chat_session(chat_session_id: str, db: Session = Depends(database.get_db)):
868
- """Get a single chat session with its full message history."""
869
- session = db.query(database.ChatSession).filter(
870
- database.ChatSession.id == chat_session_id
871
- ).first()
872
- if not session:
873
- raise HTTPException(status_code=404, detail="Chat session not found")
874
-
875
- messages = db.query(database.ChatMessage).filter(
876
- database.ChatMessage.chat_session_id == chat_session_id
877
- ).order_by(database.ChatMessage.id.asc()).all()
878
-
879
- return {
880
- "id": session.id,
881
- "name": session.name,
882
- "spec_id": session.spec_id,
883
- "session_id": session.session_id,
884
- "created_at": session.created_at.isoformat() if session.created_at else None,
885
- "updated_at": session.updated_at.isoformat() if session.updated_at else None,
886
- "messages": [
887
- {
888
- "id": m.id,
889
- "role": m.role,
890
- "content": m.content,
891
- "action_json": m.action_json,
892
- "timestamp": m.timestamp.isoformat() if m.timestamp else None,
893
- }
894
- for m in messages
895
- ],
896
- }
897
-
898
-
899
- @app.patch("/chat-sessions/{chat_session_id}")
900
- def rename_chat_session(chat_session_id: str, payload: dict, db: Session = Depends(database.get_db)):
901
- """Rename a chat session."""
902
- session = db.query(database.ChatSession).filter(
903
- database.ChatSession.id == chat_session_id
904
- ).first()
905
- if not session:
906
- raise HTTPException(status_code=404, detail="Chat session not found")
907
- if "name" in payload:
908
- session.name = payload["name"]
909
- db.commit()
910
- return {"id": session.id, "name": session.name}
911
-
912
-
913
- @app.delete("/chat-sessions/{chat_session_id}")
914
- def delete_chat_session(chat_session_id: str, db: Session = Depends(database.get_db)):
915
- """Delete a chat session and all its messages (cascade)."""
916
- session = db.query(database.ChatSession).filter(
917
- database.ChatSession.id == chat_session_id
918
- ).first()
919
- if not session:
920
- raise HTTPException(status_code=404, detail="Chat session not found")
921
- db.delete(session)
922
- db.commit()
923
- return {"status": "deleted", "id": chat_session_id}
924
 
925
 
926
- @app.delete("/chat-sessions")
927
- def clear_all_chat_sessions(db: Session = Depends(database.get_db)):
928
- """Wipe all chat sessions and messages."""
929
- # Delete messages first due to FK constraint
930
- db.query(database.ChatMessage).delete()
931
- db.query(database.ChatSession).delete()
932
- db.commit()
933
- return {"status": "cleared"}
934
 
935
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  import os
2
+
3
+ from fastapi import FastAPI, Request
4
  from fastapi.exceptions import RequestValidationError
5
+ from fastapi.middleware.cors import CORSMiddleware
6
  from fastapi.responses import JSONResponse
7
+ from fastapi.staticfiles import StaticFiles
8
+ from slowapi import Limiter, _rate_limit_exceeded_handler
9
+ from slowapi.errors import RateLimitExceeded
10
+ from slowapi.util import get_remote_address
11
 
12
+ from backend import database
13
+ from backend.core.config import settings
14
+ from backend.core.errors import AppError, app_error_handler
15
+ from backend.core.logging import get_logger
16
+ from backend.app.api.routes import specs, assets, sessions, history, chat, execution
 
 
 
 
17
 
18
+ logger = get_logger(__name__)
19
 
20
+ # Rate limiter β€” keyed by client IP
21
+ limiter = Limiter(key_func=get_remote_address, default_limits=["200/minute"])
 
 
 
22
 
23
+ # Ensure upload directories exist
24
+ for category in ["audio", "video", "documents", "images", "specs"]:
25
+ os.makedirs(os.path.join(settings.UPLOAD_DIR, category), exist_ok=True)
26
 
27
+ # Create DB tables
28
+ database.Base.metadata.create_all(bind=database.engine)
29
 
30
+ app = FastAPI(title="AI API Testing Assistant")
31
+ app.state.limiter = limiter
32
+ app.add_exception_handler(RateLimitExceeded, _rate_limit_exceeded_handler)
33
+ app.add_exception_handler(AppError, app_error_handler)
 
 
 
 
34
 
35
+ # Static file serving for uploads
36
+ app.mount("/uploads", StaticFiles(directory=settings.UPLOAD_DIR), name="uploads")
37
+
38
+ # CORS
39
  app.add_middleware(
40
  CORSMiddleware,
41
+ allow_origins=settings.allowed_origins_list,
42
  allow_credentials=True,
43
  allow_methods=["*"],
44
  allow_headers=["*"],
45
  )
46
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
47
 
48
+ @app.exception_handler(RequestValidationError)
49
+ async def validation_exception_handler(request, exc):
50
+ logger.warning("Validation error: %s", exc.errors())
51
+ return JSONResponse(status_code=422, content={"detail": exc.errors()})
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
52
 
53
 
54
+ # Routers
55
+ app.include_router(specs.router)
56
+ app.include_router(assets.router)
57
+ app.include_router(sessions.router)
58
+ app.include_router(history.router)
59
+ app.include_router(chat.router)
60
+ app.include_router(execution.router)
 
61
 
62
 
63
+ @app.get("/")
64
+ def health_check():
65
+ return {"status": "ok", "message": "AI API Testing Tool Backend running."}
backend/services/api_executor.py CHANGED
@@ -1,9 +1,9 @@
1
  import json
2
  from typing import Dict, Any, List
3
  from langchain_core.prompts import PromptTemplate
4
- from langchain_mistralai.chat_models import ChatMistralAI
5
  from langchain_core.output_parsers import JsonOutputParser
6
  from pydantic import BaseModel, Field
 
7
 
8
  # Using Pydantic to structure LangChain output for step execution
9
  class APIRequest(BaseModel):
@@ -104,9 +104,6 @@ def prepare_step_request(
104
  input_variables=["endpoint_spec", "base_url", "scenario", "step_definition", "user_credentials", "context_variables", "available_assets"],
105
  )
106
 
107
- llm = ChatMistralAI(model_name="mistral-large-latest", temperature=0)
108
- chain = prompt | llm | parser
109
-
110
  msg = prompt.format_prompt(
111
  endpoint_spec=json.dumps(endpoint_spec, separators=(',', ':')),
112
  base_url=base_url,
@@ -117,17 +114,7 @@ def prepare_step_request(
117
  available_assets=json.dumps(available_assets, separators=(',', ':'))
118
  ).to_messages()
119
 
120
- ai_msg = llm.invoke(msg)
121
- parsed_response = parser.parse(ai_msg.content)
122
-
123
- # Extract token usage
124
- usage = ai_msg.response_metadata.get("token_usage", {})
125
- token_usage = None
126
- if usage:
127
- token_usage = {
128
- "prompt_tokens": usage.get("prompt_tokens", 0),
129
- "completion_tokens": usage.get("completion_tokens", 0),
130
- "total_tokens": usage.get("total_tokens", 0)
131
- }
132
-
133
  return StepExecutionPlan(**parsed_response), token_usage
 
1
  import json
2
  from typing import Dict, Any, List
3
  from langchain_core.prompts import PromptTemplate
 
4
  from langchain_core.output_parsers import JsonOutputParser
5
  from pydantic import BaseModel, Field
6
+ from backend.integrations.llm.client import deterministic as _llm
7
 
8
  # Using Pydantic to structure LangChain output for step execution
9
  class APIRequest(BaseModel):
 
104
  input_variables=["endpoint_spec", "base_url", "scenario", "step_definition", "user_credentials", "context_variables", "available_assets"],
105
  )
106
 
 
 
 
107
  msg = prompt.format_prompt(
108
  endpoint_spec=json.dumps(endpoint_spec, separators=(',', ':')),
109
  base_url=base_url,
 
114
  available_assets=json.dumps(available_assets, separators=(',', ':'))
115
  ).to_messages()
116
 
117
+ content, token_usage = _llm.invoke(msg)
118
+ parsed_response = parser.parse(content)
119
+
 
 
 
 
 
 
 
 
 
 
120
  return StepExecutionPlan(**parsed_response), token_usage
backend/services/chat_assistant.py CHANGED
@@ -9,8 +9,8 @@ what to do (execute a step, explain an error, modify a request, generate tests,
9
  import json
10
  import re
11
  from typing import Any, Dict, List, Optional
12
- from langchain_mistralai.chat_models import ChatMistralAI
13
- from langchain_core.messages import SystemMessage, HumanMessage
14
 
15
  SYSTEM_PROMPT = """You are an AI assistant integrated into an API testing tool (similar to Postman AI).
16
  Your job is to help the user control their API testing workflow through natural language.
@@ -112,7 +112,7 @@ def _build_history_messages(chat_history: List[Dict]) -> List:
112
  if role == "user":
113
  messages.append(HumanMessage(content=content))
114
  else:
115
- messages.append(HumanMessage(content=f"[Your previous response]: {content}"))
116
  return messages
117
 
118
 
@@ -133,15 +133,13 @@ def process_message(
133
 
134
  full_system = SYSTEM_PROMPT + f"\n\n--- CURRENT CONTEXT ---\n{context_block}\n--- END CONTEXT ---"
135
 
136
- llm = ChatMistralAI(model_name="mistral-large-latest", temperature=0.2, timeout=120)
137
-
138
  messages = [SystemMessage(content=full_system)]
139
  messages.extend(_build_history_messages(chat_history))
140
  messages.append(HumanMessage(content=message))
141
 
142
  try:
143
- response = llm.invoke(messages)
144
- raw = response.content.strip()
145
 
146
  # Strip markdown code fences if present
147
  raw = re.sub(r"^```(?:json)?\s*", "", raw, flags=re.MULTILINE)
 
9
  import json
10
  import re
11
  from typing import Any, Dict, List, Optional
12
+ from langchain_core.messages import SystemMessage, HumanMessage, AIMessage
13
+ from backend.integrations.llm.client import chat as _llm
14
 
15
  SYSTEM_PROMPT = """You are an AI assistant integrated into an API testing tool (similar to Postman AI).
16
  Your job is to help the user control their API testing workflow through natural language.
 
112
  if role == "user":
113
  messages.append(HumanMessage(content=content))
114
  else:
115
+ messages.append(AIMessage(content=content))
116
  return messages
117
 
118
 
 
133
 
134
  full_system = SYSTEM_PROMPT + f"\n\n--- CURRENT CONTEXT ---\n{context_block}\n--- END CONTEXT ---"
135
 
 
 
136
  messages = [SystemMessage(content=full_system)]
137
  messages.extend(_build_history_messages(chat_history))
138
  messages.append(HumanMessage(content=message))
139
 
140
  try:
141
+ content, _ = _llm.invoke(messages)
142
+ raw = content.strip()
143
 
144
  # Strip markdown code fences if present
145
  raw = re.sub(r"^```(?:json)?\s*", "", raw, flags=re.MULTILINE)
backend/services/llm_planner.py CHANGED
@@ -1,9 +1,12 @@
1
  import json
2
  from typing import Dict, Any, List, Optional
3
  from langchain_core.prompts import PromptTemplate
4
- from langchain_mistralai.chat_models import ChatMistralAI
5
  from langchain_core.output_parsers import JsonOutputParser
 
6
  from backend.schemas import ExecutionPlan
 
 
 
7
 
8
  # Prompt template from the user prompt
9
  PLANNER_PROMPT = """
@@ -73,28 +76,21 @@ def generate_execution_plan(parsed_spec: Dict[str, Any], base_url: str, scenario
73
  input_variables=["parsed_spec", "base_url", "scenario", "user_credentials_str"],
74
  )
75
 
76
- # Initialize Mistral Model (requires MISTRAL_API_KEY in env)
77
- llm = ChatMistralAI(model_name="mistral-large-latest", temperature=0, timeout=120)
78
-
79
  msg = prompt.format_prompt(
80
  parsed_spec=json.dumps(parsed_spec.get("endpoints", parsed_spec), separators=(',', ':')),
81
  base_url=base_url,
82
  scenario=scenario,
83
  user_credentials_str=json.dumps(user_credentials or {}, separators=(',', ':'))
84
  ).to_messages()
85
-
86
- ai_msg = llm.invoke(msg)
87
- parsed_response = parser.parse(ai_msg.content)
88
-
89
- # Extract token usage
90
- usage = ai_msg.response_metadata.get("token_usage", {})
91
- token_usage = None
92
- if usage:
93
- token_usage = {
94
- "prompt_tokens": usage.get("prompt_tokens", 0),
95
- "completion_tokens": usage.get("completion_tokens", 0),
96
- "total_tokens": usage.get("total_tokens", 0)
97
- }
98
 
99
  # Return as Pydantic model
100
  return ExecutionPlan(
 
1
  import json
2
  from typing import Dict, Any, List, Optional
3
  from langchain_core.prompts import PromptTemplate
 
4
  from langchain_core.output_parsers import JsonOutputParser
5
+ from backend.core.logging import get_logger
6
  from backend.schemas import ExecutionPlan
7
+ from backend.integrations.llm.client import deterministic as _llm
8
+
9
+ logger = get_logger(__name__)
10
 
11
  # Prompt template from the user prompt
12
  PLANNER_PROMPT = """
 
76
  input_variables=["parsed_spec", "base_url", "scenario", "user_credentials_str"],
77
  )
78
 
 
 
 
79
  msg = prompt.format_prompt(
80
  parsed_spec=json.dumps(parsed_spec.get("endpoints", parsed_spec), separators=(',', ':')),
81
  base_url=base_url,
82
  scenario=scenario,
83
  user_credentials_str=json.dumps(user_credentials or {}, separators=(',', ':'))
84
  ).to_messages()
85
+
86
+ # Invoke with one retry on parse failure
87
+ content, token_usage = _llm.invoke(msg)
88
+ try:
89
+ parsed_response = parser.parse(content)
90
+ except Exception as parse_err:
91
+ logger.warning("Plan parse failed, retrying: %s", parse_err)
92
+ content, token_usage = _llm.invoke(msg)
93
+ parsed_response = parser.parse(content)
 
 
 
 
94
 
95
  # Return as Pydantic model
96
  return ExecutionPlan(
backend/services/response_analyzer.py CHANGED
@@ -1,9 +1,12 @@
1
  import json
2
  from typing import Dict, Any
3
  from langchain_core.prompts import PromptTemplate
4
- from langchain_mistralai.chat_models import ChatMistralAI
5
  from langchain_core.output_parsers import JsonOutputParser
 
6
  from backend.schemas import AnalyzeResponse
 
 
 
7
 
8
  ANALYSIS_PROMPT = """
9
  You are a response analyzer.
@@ -44,9 +47,6 @@ def analyze_api_response(
44
  input_variables=["step_info", "status_code", "response_body"],
45
  )
46
 
47
- llm = ChatMistralAI(model_name="mistral-large-latest", temperature=0)
48
- chain = prompt | llm | parser
49
-
50
  # Aggressive truncation: keep only 500 chars from start and end (1KB total)
51
  if len(response_body) > 1000:
52
  response_body = response_body[:500] + "\n... [TRUNCATED] ...\n" + response_body[-500:]
@@ -57,18 +57,13 @@ def analyze_api_response(
57
  response_body=response_body
58
  ).to_messages()
59
 
60
- ai_msg = llm.invoke(msg)
61
- parsed_response = parser.parse(ai_msg.content)
62
-
63
- # Extract token usage
64
- usage = ai_msg.response_metadata.get("token_usage", {})
65
- token_usage = None
66
- if usage:
67
- token_usage = {
68
- "prompt_tokens": usage.get("prompt_tokens", 0),
69
- "completion_tokens": usage.get("completion_tokens", 0),
70
- "total_tokens": usage.get("total_tokens", 0)
71
- }
72
 
73
  # Ensure step matches the requested step and add token usage
74
  parsed_response["step"] = step
 
1
  import json
2
  from typing import Dict, Any
3
  from langchain_core.prompts import PromptTemplate
 
4
  from langchain_core.output_parsers import JsonOutputParser
5
+ from backend.core.logging import get_logger
6
  from backend.schemas import AnalyzeResponse
7
+ from backend.integrations.llm.client import deterministic as _llm
8
+
9
+ logger = get_logger(__name__)
10
 
11
  ANALYSIS_PROMPT = """
12
  You are a response analyzer.
 
47
  input_variables=["step_info", "status_code", "response_body"],
48
  )
49
 
 
 
 
50
  # Aggressive truncation: keep only 500 chars from start and end (1KB total)
51
  if len(response_body) > 1000:
52
  response_body = response_body[:500] + "\n... [TRUNCATED] ...\n" + response_body[-500:]
 
57
  response_body=response_body
58
  ).to_messages()
59
 
60
+ content, token_usage = _llm.invoke(msg)
61
+ try:
62
+ parsed_response = parser.parse(content)
63
+ except Exception as parse_err:
64
+ logger.warning("Analysis parse failed, retrying: %s", parse_err)
65
+ content, token_usage = _llm.invoke(msg)
66
+ parsed_response = parser.parse(content)
 
 
 
 
 
67
 
68
  # Ensure step matches the requested step and add token usage
69
  parsed_response["step"] = step
backend/tests/__init__.py ADDED
File without changes
backend/tests/conftest.py ADDED
@@ -0,0 +1,44 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ import os
2
+
3
+ # Must be set before any backend module is imported β€” database.py reads this at import time.
4
+ os.environ["DATABASE_URL"] = "sqlite:///:memory:"
5
+ os.environ.setdefault("MISTRAL_API_KEY", "test-key")
6
+ os.environ.setdefault("ALLOWED_ORIGINS", "http://localhost:3000")
7
+
8
+ import pytest
9
+ from sqlalchemy import create_engine
10
+ from sqlalchemy.orm import sessionmaker
11
+ from fastapi.testclient import TestClient
12
+
13
+ from backend.database import Base, get_db
14
+ from backend.main import app
15
+
16
+ _engine = create_engine(
17
+ "sqlite:///:memory:",
18
+ connect_args={"check_same_thread": False},
19
+ )
20
+ _TestingSession = sessionmaker(autocommit=False, autoflush=False, bind=_engine)
21
+
22
+
23
+ def _override_get_db():
24
+ db = _TestingSession()
25
+ try:
26
+ yield db
27
+ finally:
28
+ db.close()
29
+
30
+
31
+ app.dependency_overrides[get_db] = _override_get_db
32
+
33
+
34
+ @pytest.fixture(scope="session", autouse=True)
35
+ def setup_db():
36
+ Base.metadata.create_all(bind=_engine)
37
+ yield
38
+ Base.metadata.drop_all(bind=_engine)
39
+
40
+
41
+ @pytest.fixture
42
+ def client():
43
+ with TestClient(app) as c:
44
+ yield c
backend/tests/test_api_executor.py ADDED
@@ -0,0 +1,78 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ import json
2
+ from unittest.mock import patch
3
+
4
+ from backend.services.api_executor import prepare_step_request
5
+
6
+ _VALID_RESPONSE = {
7
+ "step": 1,
8
+ "request": {
9
+ "method": "POST",
10
+ "url": "https://api.example.com/auth/login",
11
+ "headers": {"Content-Type": "application/json"},
12
+ "query_params": {},
13
+ "form_data": {},
14
+ "body": {"username": "test@example.com", "password": "Pass123!"},
15
+ "files": {},
16
+ },
17
+ "analysis": {
18
+ "required_inputs": ["username", "password"],
19
+ "values_from_previous_steps": [],
20
+ "generated_test_data": [],
21
+ },
22
+ }
23
+
24
+ _TOKEN_USAGE = {"prompt_tokens": 200, "completion_tokens": 80, "total_tokens": 280}
25
+
26
+
27
+ def _run(**overrides):
28
+ kwargs = dict(
29
+ endpoint_spec={"method": "POST", "path": "/auth/login"},
30
+ base_url="https://api.example.com",
31
+ scenario="Login user",
32
+ step_definition={"step": 1, "endpoint": "POST /auth/login", "purpose": "Login"},
33
+ context_variables={},
34
+ user_credentials={"username": "test@example.com", "password": "Pass123!"},
35
+ )
36
+ kwargs.update(overrides)
37
+ return prepare_step_request(**kwargs)
38
+
39
+
40
+ def test_returns_step_execution_plan():
41
+ with patch("backend.services.api_executor._llm") as mock_llm:
42
+ mock_llm.invoke.return_value = (json.dumps(_VALID_RESPONSE), _TOKEN_USAGE)
43
+
44
+ result, token_usage = _run()
45
+
46
+ assert result.request.method == "POST"
47
+ assert result.request.url == "https://api.example.com/auth/login"
48
+ assert result.request.body == {"username": "test@example.com", "password": "Pass123!"}
49
+
50
+
51
+ def test_token_usage_returned():
52
+ with patch("backend.services.api_executor._llm") as mock_llm:
53
+ mock_llm.invoke.return_value = (json.dumps(_VALID_RESPONSE), _TOKEN_USAGE)
54
+
55
+ _, token_usage = _run()
56
+
57
+ assert token_usage["total_tokens"] == 280
58
+
59
+
60
+ def test_context_variables_included_in_prompt():
61
+ with patch("backend.services.api_executor._llm") as mock_llm:
62
+ mock_llm.invoke.return_value = (json.dumps(_VALID_RESPONSE), None)
63
+
64
+ _run(context_variables={"access_token": "tok_abc"})
65
+
66
+ call_args = mock_llm.invoke.call_args
67
+ # The rendered prompt messages should contain the context variable
68
+ prompt_text = str(call_args)
69
+ assert "access_token" in prompt_text
70
+
71
+
72
+ def test_token_usage_none_when_llm_returns_none():
73
+ with patch("backend.services.api_executor._llm") as mock_llm:
74
+ mock_llm.invoke.return_value = (json.dumps(_VALID_RESPONSE), None)
75
+
76
+ _, token_usage = _run()
77
+
78
+ assert token_usage is None
backend/tests/test_llm_planner.py ADDED
@@ -0,0 +1,71 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ import json
2
+ from unittest.mock import patch
3
+
4
+ import pytest
5
+
6
+ from backend.services.llm_planner import generate_execution_plan
7
+
8
+ _VALID_RESPONSE = {
9
+ "execution_plan": [
10
+ {"step": 1, "endpoint": "POST /auth/login", "purpose": "Authenticate the user"},
11
+ {"step": 2, "endpoint": "GET /profile", "purpose": "Fetch user profile"},
12
+ ],
13
+ "suggested_credentials": {"username": "tester@example.com", "password": "Pass123!"},
14
+ }
15
+
16
+ _TOKEN_USAGE = {"prompt_tokens": 120, "completion_tokens": 60, "total_tokens": 180}
17
+
18
+
19
+ def test_returns_execution_plan():
20
+ with patch("backend.services.llm_planner._llm") as mock_llm:
21
+ mock_llm.invoke.return_value = (json.dumps(_VALID_RESPONSE), _TOKEN_USAGE)
22
+
23
+ result = generate_execution_plan(
24
+ parsed_spec={"endpoints": []},
25
+ base_url="https://api.example.com",
26
+ scenario="Login and view profile",
27
+ )
28
+
29
+ assert len(result.execution_plan) == 2
30
+ assert result.execution_plan[0]["step"] == 1
31
+ assert result.suggested_credentials["username"] == "tester@example.com"
32
+
33
+
34
+ def test_token_usage_is_passed_through():
35
+ with patch("backend.services.llm_planner._llm") as mock_llm:
36
+ mock_llm.invoke.return_value = (json.dumps(_VALID_RESPONSE), _TOKEN_USAGE)
37
+
38
+ result = generate_execution_plan(
39
+ parsed_spec={}, base_url="https://api.example.com", scenario="Login"
40
+ )
41
+
42
+ assert result.token_usage == _TOKEN_USAGE
43
+
44
+
45
+ def test_retries_once_on_parse_failure():
46
+ with patch("backend.services.llm_planner._llm") as mock_llm:
47
+ # First call returns invalid JSON; second returns valid
48
+ mock_llm.invoke.side_effect = [
49
+ ("not valid json {{{{", None),
50
+ (json.dumps(_VALID_RESPONSE), _TOKEN_USAGE),
51
+ ]
52
+
53
+ result = generate_execution_plan(
54
+ parsed_spec={}, base_url="https://api.example.com", scenario="Login"
55
+ )
56
+
57
+ assert mock_llm.invoke.call_count == 2
58
+ assert len(result.execution_plan) == 2
59
+
60
+
61
+ def test_raises_after_two_parse_failures():
62
+ with patch("backend.services.llm_planner._llm") as mock_llm:
63
+ mock_llm.invoke.side_effect = [
64
+ ("bad json", None),
65
+ ("still bad json", None),
66
+ ]
67
+
68
+ with pytest.raises(Exception):
69
+ generate_execution_plan(
70
+ parsed_spec={}, base_url="https://api.example.com", scenario="Login"
71
+ )
backend/tests/test_routes.py ADDED
@@ -0,0 +1,140 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """
2
+ Integration tests for HTTP routes.
3
+ All LLM calls are mocked so no real Mistral API key is needed.
4
+ """
5
+ import io
6
+ import json
7
+ from unittest.mock import patch
8
+
9
+ import pytest
10
+
11
+ # client fixture is provided by conftest.py
12
+
13
+
14
+ # ── Health / root ─────────────────────────────────────────────────────────────
15
+
16
+ def test_root_health(client):
17
+ response = client.get("/")
18
+ assert response.status_code == 200
19
+ assert response.json()["status"] == "ok"
20
+
21
+
22
+ # ── Session ───────────────────────────────────────────────────────────────────
23
+
24
+ def test_create_session(client):
25
+ response = client.post("/session")
26
+ assert response.status_code == 200
27
+ data = response.json()
28
+ assert "session_id" in data
29
+
30
+
31
+ # ── Spec upload ───────────────────────────────────────────────────────────────
32
+
33
+ _MINIMAL_SPEC = json.dumps({
34
+ "openapi": "3.0.0",
35
+ "info": {"title": "Test API", "version": "1.0.0"},
36
+ "paths": {
37
+ "/ping": {
38
+ "get": {
39
+ "summary": "Ping",
40
+ "responses": {"200": {"description": "OK"}},
41
+ }
42
+ }
43
+ },
44
+ })
45
+
46
+
47
+ def test_upload_spec_valid(client):
48
+ response = client.post(
49
+ "/upload-spec",
50
+ files={"file": ("api.json", _MINIMAL_SPEC.encode(), "application/json")},
51
+ )
52
+ assert response.status_code == 200
53
+ data = response.json()
54
+ assert "id" in data
55
+ return data["id"]
56
+
57
+
58
+ def test_upload_spec_rejects_large_file(client):
59
+ big_content = b"x" * (6 * 1024 * 1024) # 6 MB β€” over the 5 MB limit
60
+ response = client.post(
61
+ "/upload-spec",
62
+ files={"file": ("big.json", big_content, "application/json")},
63
+ )
64
+ assert response.status_code == 400
65
+
66
+
67
+ def test_list_specs(client):
68
+ response = client.get("/specs")
69
+ assert response.status_code == 200
70
+ assert isinstance(response.json(), list)
71
+
72
+
73
+ # ── Plan generation (mocked LLM) ──────────────────────────────────────────────
74
+
75
+ _MOCK_PLAN = {
76
+ "execution_plan": [
77
+ {"step": 1, "endpoint": "GET /ping", "purpose": "Health check"},
78
+ ],
79
+ "suggested_credentials": {},
80
+ }
81
+
82
+ _MOCK_TOKEN_USAGE = {"prompt_tokens": 50, "completion_tokens": 20, "total_tokens": 70}
83
+
84
+
85
+ def test_plan_endpoint(client):
86
+ # Upload a spec first to get a valid spec_id
87
+ upload = client.post(
88
+ "/upload-spec",
89
+ files={"file": ("api.json", _MINIMAL_SPEC.encode(), "application/json")},
90
+ )
91
+ spec_id = upload.json()["id"]
92
+
93
+ session = client.post("/session").json()["session_id"]
94
+
95
+ with patch("backend.services.llm_planner._llm") as mock_llm:
96
+ mock_llm.invoke.return_value = (json.dumps(_MOCK_PLAN), _MOCK_TOKEN_USAGE)
97
+
98
+ response = client.post("/plan", json={
99
+ "spec_content": _MINIMAL_SPEC,
100
+ "base_url": "https://api.example.com",
101
+ "scenario": "Ping the API",
102
+ "user_credentials": {},
103
+ "spec_id": spec_id,
104
+ "session_id": session,
105
+ })
106
+
107
+ assert response.status_code == 200
108
+ data = response.json()
109
+ assert len(data["execution_plan"]) == 1
110
+ assert data["token_usage"]["total_tokens"] == 70
111
+
112
+
113
+ # ── Chat endpoint (mocked LLM) ────────────────────────────────────────────────
114
+
115
+ _MOCK_CHAT_RESPONSE = json.dumps({
116
+ "action": "reply",
117
+ "message": "I can help you run step 1.",
118
+ })
119
+
120
+
121
+ def test_chat_endpoint(client):
122
+ session = client.post("/session").json()["session_id"]
123
+
124
+ with patch("backend.services.chat_assistant._llm") as mock_llm:
125
+ mock_llm.invoke.return_value = (_MOCK_CHAT_RESPONSE, None)
126
+
127
+ response = client.post("/chat", json={
128
+ "message": "Run step 1 for me",
129
+ "session_id": session,
130
+ "spec_id": None,
131
+ "execution_plan": [],
132
+ "context_variables": {},
133
+ "last_step_result": None,
134
+ "all_step_results": {},
135
+ "chat_session_id": None,
136
+ })
137
+
138
+ assert response.status_code == 200
139
+ data = response.json()
140
+ assert data["action"] == "reply"
backend/tests/test_url_guard.py ADDED
@@ -0,0 +1,75 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ import socket
2
+ from unittest.mock import patch
3
+ import pytest
4
+ from fastapi import HTTPException
5
+
6
+ from backend.core.url_guard import assert_safe_url
7
+
8
+
9
+ def _mock_resolve(ip: str):
10
+ """Return a socket.getaddrinfo patch that resolves any hostname to `ip`."""
11
+ return patch("backend.core.url_guard.socket.getaddrinfo", return_value=[(None, None, None, None, (ip, 0))])
12
+
13
+
14
+ # ── Scheme checks ─────────────────────────────────────────────────────────────
15
+
16
+ def test_blocks_ftp_scheme():
17
+ with pytest.raises(HTTPException) as exc:
18
+ assert_safe_url("ftp://example.com/file")
19
+ assert exc.value.status_code == 400
20
+
21
+
22
+ def test_blocks_file_scheme():
23
+ with pytest.raises(HTTPException):
24
+ assert_safe_url("file:///etc/passwd")
25
+
26
+
27
+ def test_blocks_no_hostname():
28
+ with pytest.raises(HTTPException):
29
+ assert_safe_url("http:///path")
30
+
31
+
32
+ # ── Private / loopback IP checks ──────────────────────────────────────────────
33
+
34
+ def test_blocks_localhost_ip():
35
+ with _mock_resolve("127.0.0.1"):
36
+ with pytest.raises(HTTPException) as exc:
37
+ assert_safe_url("http://localhost/api")
38
+ assert exc.value.status_code == 400
39
+
40
+
41
+ def test_blocks_10x_range():
42
+ with _mock_resolve("10.0.0.1"):
43
+ with pytest.raises(HTTPException):
44
+ assert_safe_url("http://internal.corp/api")
45
+
46
+
47
+ def test_blocks_192168_range():
48
+ with _mock_resolve("192.168.1.100"):
49
+ with pytest.raises(HTTPException):
50
+ assert_safe_url("http://router.local/api")
51
+
52
+
53
+ def test_blocks_172_16_range():
54
+ with _mock_resolve("172.16.0.5"):
55
+ with pytest.raises(HTTPException):
56
+ assert_safe_url("http://private.example.com/api")
57
+
58
+
59
+ def test_blocks_link_local():
60
+ with _mock_resolve("169.254.169.254"):
61
+ with pytest.raises(HTTPException):
62
+ assert_safe_url("http://metadata.example.com/api")
63
+
64
+
65
+ # ── Allow public addresses ─────────────────────────────────────────────────────
66
+
67
+ def test_allows_public_ip():
68
+ with _mock_resolve("93.184.216.34"): # example.com
69
+ assert_safe_url("https://example.com/api") # should not raise
70
+
71
+
72
+ def test_allows_http_and_https():
73
+ with _mock_resolve("93.184.216.34"):
74
+ assert_safe_url("http://example.com/api")
75
+ assert_safe_url("https://example.com/api")
backend/uploads/audio/.gitkeep ADDED
File without changes
backend/uploads/documents/.gitkeep ADDED
File without changes
backend/uploads/images/.gitkeep ADDED
File without changes
backend/uploads/specs/.gitkeep ADDED
File without changes
backend/uploads/video/.gitkeep ADDED
File without changes
PROJECT_DOCUMENTATION.md β†’ documents/PROJECT_DOCUMENTATION.md RENAMED
@@ -59,27 +59,29 @@
59
  | Web Framework | **FastAPI** | β‰₯0.109 | REST API server |
60
  | ASGI Server | **Uvicorn** | β‰₯0.27 | Runs FastAPI in development |
61
  | Production Server | **Gunicorn** | 21.2 | Runs Uvicorn workers in production |
62
- | LLM Orchestration | **LangChain** | 0.1.0 | Prompt chaining with Mistral |
63
  | LLM Provider | **Mistral AI** | `mistral-large-latest` | Plan generation, step execution, analysis |
64
  | HTTP Client | **httpx** | β‰₯0.27 | Sends actual API requests during execution |
65
- | ORM | **SQLAlchemy** | 2.0.23 | Database queries and model definitions |
66
  | Database | **PostgreSQL** (via Supabase) | β€” | Persistent storage |
67
- | DB Driver | **psycopg2-binary** | 2.9.9 | PostgreSQL adapter |
68
  | Cloud Storage | **Supabase Storage** | 2.28.2 | Stores uploaded spec files and assets |
69
  | Spec Parsing | **prance** + **PyYAML** | β€” | Parse OpenAPI YAML/JSON |
70
  | Validation | **Pydantic** | β‰₯2.6 | Request/response data models |
 
71
  | Auth/Config | **python-dotenv** | 1.0.0 | Load env variables from `.env` |
 
72
 
73
  ### Frontend
74
 
75
  | Layer | Technology | Version | Purpose |
76
  |---|---|---|---|
77
- | UI Framework | **React** | 19.x | Component-based UI |
78
  | Build Tool | **Vite** | 8.x | Fast dev server + bundler |
79
- | Styling | **Tailwind CSS** | 4.x | Utility-first CSS |
80
  | Icons | **lucide-react** | 0.577 | Icon library |
81
  | HTTP Client | **Axios** | 1.x | API communication with backend |
82
- | Routing | **react-router-dom** | 7.x | Client-side navigation |
83
 
84
  ---
85
 
@@ -95,22 +97,35 @@ Testing/ ← Project root
95
  β”œβ”€β”€ runtime.txt ← Python version pin
96
  β”œβ”€β”€ start.ps1 ← PowerShell startup script
97
  β”œβ”€β”€ openapi.yaml ← Sample OpenAPI spec for testing
 
98
  β”‚
99
  β”œβ”€β”€ backend/ ← FastAPI application
100
- β”‚ β”œβ”€β”€ __init__.py
101
- β”‚ β”œβ”€β”€ main.py ← All FastAPI routes (API gateway)
102
  β”‚ β”œβ”€β”€ database.py ← SQLAlchemy models + DB connection
103
  β”‚ β”œβ”€β”€ schemas.py ← Pydantic request/response models
104
- β”‚ └── services/ ← Business logic layer
105
- β”‚ β”œβ”€β”€ swagger_parser.py ← Parses OpenAPI/Swagger specs
106
- β”‚ β”œβ”€β”€ postman_parser.py ← Parses Postman Collections
107
- β”‚ β”œβ”€β”€ llm_planner.py ← Mistral AI: generates execution plan
108
- β”‚ β”œβ”€β”€ api_executor.py ← Mistral AI: formats HTTP request for each step
109
- β”‚ β”œβ”€β”€ response_analyzer.py ← Mistral AI: analyzes step response + selection detection
110
- β”‚ β”œβ”€β”€ chat_assistant.py ← Mistral AI: powers the AI Chat Assistant
111
- β”‚ β”œβ”€β”€ context_manager.py ← Session context variable storage
112
- β”‚ β”œβ”€β”€ asset_service.py ← Media asset CRUD + collection management
113
- β”‚ └── storage.py ← Abstraction: local or Supabase file storage
 
 
 
 
 
 
 
 
 
 
 
 
 
114
  β”‚
115
  └── frontend-react/ ← React + Vite frontend
116
  β”œβ”€β”€ index.html
@@ -118,18 +133,17 @@ Testing/ ← Project root
118
  β”œβ”€β”€ package.json
119
  └── src/
120
  β”œβ”€β”€ main.jsx ← React entry point
121
- β”œβ”€β”€ App.jsx ← Main layout + execution workflow page
122
- β”œβ”€β”€ App.css / index.css ← Global styles + CSS variables (theme)
123
- β”œβ”€β”€ components/
124
- β”‚ β”œβ”€β”€ AppContext.jsx ← Global React Context (state management)
125
- β”‚ β”œβ”€β”€ CollectionsPage.jsx ← View/manage uploaded API specs
126
- β”‚ β”œβ”€β”€ ProfilePage.jsx ← Manage test-user credentials
127
- β”‚ β”œβ”€β”€ HistoryPage.jsx ← View all past test sessions
128
- β”‚ β”œβ”€β”€ MediaHubPage.jsx ← Upload and manage test assets
129
- β”‚ β”œβ”€β”€ ChatAssistant.jsx ← AI Chat Assistant (full-page + compact panel)
130
- β”‚ └── SettingsPage.jsx ← Application settings
131
  └── services/
132
- └── api.js ← All Axios API calls to the backend
133
  ```
134
 
135
  ---
@@ -182,21 +196,22 @@ Testing/ ← Project root
182
 
183
  ### 5.1 Entry Point β€” `main.py`
184
 
185
- `main.py` is the **single FastAPI application file** containing all route definitions. It does not use `APIRouter` β€” all routes are mounted directly on the `app` instance.
186
 
187
  **Key responsibilities:**
188
- - CORS middleware (currently allows all origins β€” suitable for dev/demo)
189
- - Static file serving for the `/uploads` directory
190
- - Validation error handler (returns 422 with body for debugging)
191
- - Orchestrating all service calls
192
-
193
- **Key logic in `/execute-step`:**
194
- The most complex route. It performs:
195
- 1. **Schema Resolution** β€” Looks up the matching OpenAPI path definition, inlines all `$ref` references recursively, prunes metadata (descriptions, examples), and converts the schema to a compact shorthand notation (`{prop*(str)}`) to minimize LLM tokens.
196
- 2. **URL Sanitization** β€” Detects if the LLM returned a placeholder URL (like `api.example.com`) and rewrites it to use the real base URL.
197
- 3. **Asset Loading** β€” Opens local files from the Media Hub for multipart uploads.
198
- 4. **HTTP Execution** β€” Uses `httpx.AsyncClient` to send the real request.
199
- 5. **History Persistence** β€” Saves request + response to the database.
 
200
 
201
  ---
202
 
@@ -213,6 +228,8 @@ Uses **SQLAlchemy 2.0** with a declarative base. Database URL is loaded from the
213
  | `ExecutionHistory` | `execution_history` | One row per step executed. Stores request JSON, response JSON, and AI analysis |
214
  | `Asset` | `assets` | Uploaded files (audio/video/docs). Tagged, categorized, linked to a spec |
215
  | `Collection` | `collections` | Named groups of assets. Many-to-many with Asset |
 
 
216
 
217
  #### Relationships
218
 
@@ -221,6 +238,8 @@ SpecDocument ──< Asset (one-to-many)
221
  Asset >──< Collection (many-to-many via asset_collections junction table)
222
  SessionModel ──> SpecDocument (foreign key, nullable)
223
  ExecutionHistory ──> SessionModel (foreign key via session_id)
 
 
224
  ```
225
 
226
  #### Connection Details
@@ -245,9 +264,15 @@ All request/response validation is done via Pydantic v2 models:
245
  | `ExecutionPlan` | ← Backend | Response from `/plan` β€” list of steps + suggested credentials |
246
  | `StepDefinition` | β€” | Single step: step number, endpoint string, purpose text |
247
  | `ExecuteStepRequest` | β†’ Backend | Payload for `/execute-step` β€” session ID, step def, context vars |
248
- | `AnalyzeRequest` | β†’ Backend | Payload for `/analyze-response` β€” step number, response data |
249
- | `AnalyzeResponse` | ← Backend | AI analysis result β€” status, extracted variables, OTP flag |
250
- | `CredentialUpdate` | β†’ Backend | Payload for credential save β€” username, password, extra fields |
 
 
 
 
 
 
251
  | `TokenUsage` | ← Backend | LLM token counts (prompt, completion, total) |
252
 
253
  ---
@@ -277,6 +302,7 @@ All request/response validation is done via Pydantic v2 models:
277
  - Returns `StepExecutionPlan` containing the complete HTTP request object (`method`, `url`, `headers`, `query_params`, `form_data`, `body`, `files`)
278
  - **Token-efficient schema notation** (shorthand): Instead of sending verbose JSON schema, converts it to compact strings like `{name*(str),age(int),status(str[active|inactive])}`
279
  - **Data source priority**: Context Variables β†’ User Credentials β†’ Step Definition
 
280
 
281
  #### `response_analyzer.py`
282
  - Uses `ChatMistralAI` to analyze HTTP response (status code + body)
@@ -499,9 +525,11 @@ User clicks "Execute" on a step
499
  | Method | Path | Description |
500
  |---|---|---|
501
  | `POST` | `/chat` | Send a message; returns structured AI action + saves the conversation to DB |
502
- | `GET` | `/chat/sessions` | List all chat sessions for a given spec or test session |
503
- | `GET` | `/chat/sessions/{chat_session_id}/messages` | Get full message history for a chat session |
504
- | `DELETE` | `/chat/sessions/{chat_session_id}` | Delete a chat session and its messages |
 
 
505
 
506
  ### History
507
 
@@ -595,6 +623,26 @@ CREATE TABLE asset_collections (
595
  collection_id VARCHAR REFERENCES collections(id),
596
  PRIMARY KEY (asset_id, collection_id)
597
  );
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
598
  ```
599
 
600
  ---
@@ -642,8 +690,10 @@ The `ChatAssistant` maintains conversation history in the database (`ChatSession
642
  | `DATABASE_URL` | βœ… | PostgreSQL connection string (e.g., `postgresql://user:pass@host/db`) |
643
  | `MISTRAL_API_KEY` | βœ… | API key for Mistral AI (used by LangChain) |
644
  | `SUPABASE_URL` | ⬜ | Supabase project URL (enables cloud storage; falls back to local if missing) |
645
- | `SUPABASE_KEY` | ⬜ | Supabase service role key |
646
- | `SUPABASE_BUCKET` | ⬜ | Name of the Supabase Storage bucket |
 
 
647
 
648
  > **Note:** The `DATABASE_URL` string starting with `postgres://` is auto-converted to `postgresql://` for SQLAlchemy compatibility.
649
 
@@ -715,5 +765,5 @@ Use this table when explaining or comparing this project to similar tools:
715
 
716
  ---
717
 
718
- *Documentation updated: April 2026*
719
  *Project: AI API Testing Assistant | Stack: FastAPI + React + Mistral AI + PostgreSQL*
 
59
  | Web Framework | **FastAPI** | β‰₯0.109 | REST API server |
60
  | ASGI Server | **Uvicorn** | β‰₯0.27 | Runs FastAPI in development |
61
  | Production Server | **Gunicorn** | 21.2 | Runs Uvicorn workers in production |
62
+ | LLM Orchestration | **LangChain** | β‰₯0.3.0 | Prompt chaining with Mistral |
63
  | LLM Provider | **Mistral AI** | `mistral-large-latest` | Plan generation, step execution, analysis |
64
  | HTTP Client | **httpx** | β‰₯0.27 | Sends actual API requests during execution |
65
+ | ORM | **SQLAlchemy** | β‰₯2.0.36 | Database queries and model definitions |
66
  | Database | **PostgreSQL** (via Supabase) | β€” | Persistent storage |
67
+ | DB Driver | **psycopg2-binary** | 2.9.10 | PostgreSQL adapter |
68
  | Cloud Storage | **Supabase Storage** | 2.28.2 | Stores uploaded spec files and assets |
69
  | Spec Parsing | **prance** + **PyYAML** | β€” | Parse OpenAPI YAML/JSON |
70
  | Validation | **Pydantic** | β‰₯2.6 | Request/response data models |
71
+ | Rate Limiting | **slowapi** | 0.1.9 | API request throttling |
72
  | Auth/Config | **python-dotenv** | 1.0.0 | Load env variables from `.env` |
73
+ | Security | **cryptography** | 42.0.0 | Sensitive data handling |
74
 
75
  ### Frontend
76
 
77
  | Layer | Technology | Version | Purpose |
78
  |---|---|---|---|
79
+ | UI Framework | **React** | 19.2.4 | Component-based UI |
80
  | Build Tool | **Vite** | 8.x | Fast dev server + bundler |
81
+ | Styling | **Tailwind CSS** | 4.2.1 | Utility-first CSS |
82
  | Icons | **lucide-react** | 0.577 | Icon library |
83
  | HTTP Client | **Axios** | 1.x | API communication with backend |
84
+ | Routing | **react-router-dom** | 7.13 | Client-side navigation |
85
 
86
  ---
87
 
 
97
  β”œβ”€β”€ runtime.txt ← Python version pin
98
  β”œβ”€β”€ start.ps1 ← PowerShell startup script
99
  β”œβ”€β”€ openapi.yaml ← Sample OpenAPI spec for testing
100
+ β”œβ”€β”€ documents/ ← Project documentation and diagrams
101
  β”‚
102
  β”œβ”€β”€ backend/ ← FastAPI application
103
+ β”‚ β”œβ”€β”€ main.py ← Entry point & app configuration
 
104
  β”‚ β”œβ”€β”€ database.py ← SQLAlchemy models + DB connection
105
  β”‚ β”œβ”€β”€ schemas.py ← Pydantic request/response models
106
+ β”‚ β”œβ”€β”€ app/
107
+ β”‚ β”‚ └── api/
108
+ β”‚ β”‚ └── routes/ ← Route definitions (APIRouters)
109
+ β”‚ β”‚ β”œβ”€β”€ specs.py ← Spec management routes
110
+ β”‚ β”‚ β”œβ”€β”€ assets.py ← Media hub routes
111
+ β”‚ β”‚ β”œβ”€β”€ sessions.py ← Test session routes
112
+ β”‚ β”‚ β”œβ”€β”€ history.py ← Audit log routes
113
+ β”‚ β”‚ β”œβ”€β”€ chat.py ← AI chat assistant routes
114
+ β”‚ β”‚ └── execution.py ← Core LLM execution routes
115
+ β”‚ β”œβ”€β”€ core/ ← Shared utilities & config
116
+ β”‚ β”‚ β”œβ”€β”€ config.py ← Settings management
117
+ β”‚ β”‚ β”œβ”€β”€ errors.py β”‚ Error handling logic
118
+ β”‚ β”‚ └── logging.py ← Centralized logger
119
+ β”‚ β”œβ”€β”€ integrations/ ← External service clients
120
+ β”‚ β”‚ └── llm/ ← Mistral/LangChain integration
121
+ β”‚ β”œβ”€β”€ services/ ← Business logic layer
122
+ β”‚ β”‚ β”œβ”€β”€ swagger_parser.py ← OpenAPI/Postman parsing
123
+ β”‚ β”‚ β”œβ”€β”€ llm_planner.py ← Execution plan generation
124
+ β”‚ β”‚ β”œβ”€β”€ api_executor.py ← HTTP request construction
125
+ β”‚ β”‚ β”œβ”€β”€ response_analyzer.py ← Semantic analysis
126
+ β”‚ β”‚ β”œβ”€β”€ chat_assistant.py ← AI chat logic
127
+ β”‚ β”‚ └── storage.py ← Local/Supabase file storage
128
+ β”‚ └── tests/ ← Automated test suite
129
  β”‚
130
  └── frontend-react/ ← React + Vite frontend
131
  β”œβ”€β”€ index.html
 
133
  β”œβ”€β”€ package.json
134
  └── src/
135
  β”œβ”€β”€ main.jsx ← React entry point
136
+ β”œβ”€β”€ App.jsx ← Main layout + routing
137
+ β”œβ”€β”€ App.css / index.css ← Styles and theming
138
+ β”œβ”€β”€ components/ ← UI Components
139
+ β”‚ β”œβ”€β”€ AppContext.jsx ← Global state management
140
+ β”‚ β”œβ”€β”€ CollectionsPage.jsx ← Spec management
141
+ β”‚ β”œβ”€β”€ ProfilePage.jsx ← Test user profile management
142
+ β”‚ β”œβ”€β”€ HistoryPage.jsx ← Session history & audit log
143
+ β”‚ β”œβ”€β”€ MediaHubPage.jsx ← Asset management
144
+ β”‚ └── ChatAssistant.jsx ← Persistent AI chat
 
145
  └── services/
146
+ └── api.js ← Axios service layer
147
  ```
148
 
149
  ---
 
196
 
197
  ### 5.1 Entry Point β€” `main.py`
198
 
199
+ `main.py` is the entry point for the FastAPI application. It initializes the app, ensures directory structures, and mounts the various module routers.
200
 
201
  **Key responsibilities:**
202
+ - **Rate Limiting**: Uses `slowapi` to protect endpoints (default 200/min).
203
+ - **Global Error Handling**: Custom handlers for `AppError` and `RequestValidationError`.
204
+ - **CORS Configuration**: Loaded from `settings`.
205
+ - **Static Files**: Serves the `/uploads` directory.
206
+ - **Router Orchestration**: Includes routers from `backend.app.api.routes`.
207
+
208
+ **Modules (Routers):**
209
+ - `specs`: Spec upload, retrieval, and credential management.
210
+ - `assets`: Media Hub file operations.
211
+ - `sessions`: Test session lifecycle.
212
+ - `history`: Audit log access.
213
+ - `chat`: Persistent AI chat interactions.
214
+ - `execution`: Core LLM pipeline (`/plan`, `/execute-step`, `/analyze-response`).
215
 
216
  ---
217
 
 
228
  | `ExecutionHistory` | `execution_history` | One row per step executed. Stores request JSON, response JSON, and AI analysis |
229
  | `Asset` | `assets` | Uploaded files (audio/video/docs). Tagged, categorized, linked to a spec |
230
  | `Collection` | `collections` | Named groups of assets. Many-to-many with Asset |
231
+ | `ChatSession` | `chat_sessions` | A persistent AI chat thread. Linked to a spec and optionally to a test session. Auto-named from first message. |
232
+ | `ChatMessage` | `chat_messages` | A single message within a chat session β€” stores role, content, and the structured `action_json` returned by the AI |
233
 
234
  #### Relationships
235
 
 
238
  Asset >──< Collection (many-to-many via asset_collections junction table)
239
  SessionModel ──> SpecDocument (foreign key, nullable)
240
  ExecutionHistory ──> SessionModel (foreign key via session_id)
241
+ ChatSession ──> SpecDocument (foreign key, ON DELETE SET NULL, nullable)
242
+ ChatMessage ──> ChatSession (foreign key, ON DELETE CASCADE)
243
  ```
244
 
245
  #### Connection Details
 
264
  | `ExecutionPlan` | ← Backend | Response from `/plan` β€” list of steps + suggested credentials |
265
  | `StepDefinition` | β€” | Single step: step number, endpoint string, purpose text |
266
  | `ExecuteStepRequest` | β†’ Backend | Payload for `/execute-step` β€” session ID, step def, context vars |
267
+ | `AnalyzeRequest` | β†’ Backend | Payload for `/analyze-response` β€” step number, response data, optional `spec_id` and `step_definition` |
268
+ | `AnalyzeResponse` | ← Backend | AI analysis result β€” `status`, `is_otp_required`, `otp_field_name`, `is_selection_required`, `selection_field_name`, `selection_options`, `response_summary`, `extracted_variables`, `token_usage` |
269
+ | `SelectionOption` | β€” | A single choosable item when `is_selection_required=true` β€” `label` (display text) and `value` (ID to save) |
270
+ | `ContextUpdate` | β†’ Backend | Payload for `POST /context/{session_id}` β€” `key` and `value` to merge into session context |
271
+ | `CredentialUpdate` | β†’ Backend | Payload for credential save β€” `name`, `username`, `password`, `extra` dict. Passing `extra._clear=true` **wipes all stored credentials** for the spec. |
272
+ | `ChatRequest` | β†’ Backend | Payload for `/chat` β€” message text, optional `chat_session_id`, spec context, execution plan, step results |
273
+ | `ChatResponse` | ← Backend | Structured AI response β€” `chat_session_id`, `chat_session_name`, `action`, `message`, optional `step`, `changes`, `assertions`, `explanation` |
274
+ | `ChatSessionSummary` | ← Backend | Summary of a chat session β€” id, name, spec_id, session_id, message count, timestamps |
275
+ | `ChatMessageSchema` | ← Backend | A single chat message β€” role, content, action_json, timestamp |
276
  | `TokenUsage` | ← Backend | LLM token counts (prompt, completion, total) |
277
 
278
  ---
 
302
  - Returns `StepExecutionPlan` containing the complete HTTP request object (`method`, `url`, `headers`, `query_params`, `form_data`, `body`, `files`)
303
  - **Token-efficient schema notation** (shorthand): Instead of sending verbose JSON schema, converts it to compact strings like `{name*(str),age(int),status(str[active|inactive])}`
304
  - **Data source priority**: Context Variables β†’ User Credentials β†’ Step Definition
305
+ - **Asset Integration**: Consults `asset_service` to provide the LLM with a list of available files from the Media Hub.
306
 
307
  #### `response_analyzer.py`
308
  - Uses `ChatMistralAI` to analyze HTTP response (status code + body)
 
525
  | Method | Path | Description |
526
  |---|---|---|
527
  | `POST` | `/chat` | Send a message; returns structured AI action + saves the conversation to DB |
528
+ | `GET` | `/chat-sessions` | List all chat sessions, optionally filtered by `spec_id` |
529
+ | `GET` | `/chat-sessions/{chat_session_id}` | Get a single chat session with its full message history |
530
+ | `PATCH` | `/chat-sessions/{chat_session_id}` | Rename a chat session |
531
+ | `DELETE` | `/chat-sessions/{chat_session_id}` | Delete a chat session and all its messages (cascade) |
532
+ | `DELETE` | `/chat-sessions` | Wipe **all** chat sessions and messages |
533
 
534
  ### History
535
 
 
623
  collection_id VARCHAR REFERENCES collections(id),
624
  PRIMARY KEY (asset_id, collection_id)
625
  );
626
+
627
+ -- AI Chat sessions (each represents one conversation thread)
628
+ CREATE TABLE chat_sessions (
629
+ id VARCHAR PRIMARY KEY, -- UUID
630
+ name VARCHAR NOT NULL, -- auto-named from first message; user-renameable
631
+ spec_id VARCHAR REFERENCES spec_documents(id) ON DELETE SET NULL,
632
+ session_id VARCHAR, -- linked API test session (nullable)
633
+ created_at TIMESTAMP DEFAULT NOW(),
634
+ updated_at TIMESTAMP DEFAULT NOW()
635
+ );
636
+
637
+ -- Individual messages within a chat session
638
+ CREATE TABLE chat_messages (
639
+ id SERIAL PRIMARY KEY,
640
+ chat_session_id VARCHAR REFERENCES chat_sessions(id) ON DELETE CASCADE NOT NULL,
641
+ role VARCHAR NOT NULL, -- 'user' | 'assistant'
642
+ content TEXT NOT NULL, -- raw message text
643
+ action_json JSON, -- structured action returned by AI (execute_step, explain, etc.)
644
+ timestamp TIMESTAMP DEFAULT NOW()
645
+ );
646
  ```
647
 
648
  ---
 
690
  | `DATABASE_URL` | βœ… | PostgreSQL connection string (e.g., `postgresql://user:pass@host/db`) |
691
  | `MISTRAL_API_KEY` | βœ… | API key for Mistral AI (used by LangChain) |
692
  | `SUPABASE_URL` | ⬜ | Supabase project URL (enables cloud storage; falls back to local if missing) |
693
+ | `SUPABASE_SERVICE_ROLE_KEY` | ⬜ | **Preferred** Supabase key β€” bypasses Row-Level Security for storage operations |
694
+ | `SUPABASE_SECRET_KEY` | ⬜ | Alternative Supabase key name (checked if `SUPABASE_SERVICE_ROLE_KEY` is absent) |
695
+ | `SUPABASE_KEY` | ⬜ | Fallback Supabase key (used only if the two above are not set) |
696
+ | `SUPABASE_STORAGE_BUCKET` | ⬜ | Name of the Supabase Storage bucket (defaults to `"assets"` if not set) |
697
 
698
  > **Note:** The `DATABASE_URL` string starting with `postgres://` is auto-converted to `postgresql://` for SQLAlchemy compatibility.
699
 
 
765
 
766
  ---
767
 
768
+ *Documentation updated: April 2026 β€” reflects actual codebase state as of this date*
769
  *Project: AI API Testing Assistant | Stack: FastAPI + React + Mistral AI + PostgreSQL*
Project_Diagrams.md β†’ documents/Project_Diagrams.md RENAMED
@@ -8,49 +8,45 @@ This document contains ready-to-use **Mermaid diagram code** for all major diagr
8
  *Shows the high-level structure of the entire system and how the four tiers interact.*
9
 
10
  ```mermaid
11
- graph TB
12
- subgraph "Presentation Tier"
13
- FE["React + Vite Frontend<br/>(SPA on port 5173)"]
14
- M1["API Collections Module"]
15
- M2["Execution Hub"]
16
- M3["Test User Profile"]
17
- M4["Execution History"]
18
- M5["Media Hub"]
19
- M6["AI Chat Assistant"]
20
- end
21
-
22
- subgraph "Orchestration Tier"
23
- API["FastAPI Backend<br/>(Uvicorn on port 8001)"]
24
- CORS["CORS Middleware"]
25
- VALID["Request Validation<br/>(Pydantic)"]
26
  end
27
 
28
- subgraph "Intelligence Tier"
29
- LLM1["LLM Planner<br/>(llm_planner.py)"]
30
- LLM2["API Executor<br/>(api_executor.py)"]
31
- LLM3["Response Analyzer<br/>(response_analyzer.py)"]
32
- LLM4["Chat Assistant<br/>(chat_assistant.py)"]
33
- MISTRAL["Mistral-Large-Latest<br/>via LangChain"]
34
  end
35
 
36
- subgraph "Data Tier"
37
- PG["PostgreSQL<br/>(via Supabase)"]
38
- SUP["Supabase File Storage<br/>(Specs + Assets)"]
 
 
 
 
 
 
 
 
39
  end
40
 
41
- subgraph "External"
42
- TARGET["Target API Server<br/>(Under Test)"]
 
 
 
43
  end
44
 
45
- FE --> CORS --> VALID --> API
46
- API --> LLM1
47
- API --> LLM2
48
- API --> LLM3
49
- API --> LLM4
50
- LLM1 & LLM2 & LLM3 & LLM4 --> MISTRAL
51
- API --> PG
52
- API --> SUP
53
- LLM2 --> TARGET
54
  ```
55
 
56
  ---
@@ -60,50 +56,43 @@ graph TB
60
 
61
  ```mermaid
62
  sequenceDiagram
63
- actor User
64
- participant FE as React Frontend
65
- participant BE as FastAPI Backend
66
- participant DB as PostgreSQL
67
- participant LLM as Mistral AI
68
- participant API as Target API
69
-
70
- User->>FE: Upload API spec file
71
- FE->>BE: POST /upload-spec (file)
72
- BE->>BE: Detect format (OpenAPI/Postman)
73
- BE->>BE: parse_spec() β†’ simplified endpoints
74
- BE->>DB: INSERT SpecDocument (raw + parsed)
75
- BE-->>FE: {spec_id, endpoints}
76
-
77
- User->>FE: Type natural language scenario
78
- FE->>BE: POST /plan {scenario, spec_id}
79
- BE->>DB: GET login_credentials for spec
80
- BE->>LLM: generate_execution_plan(endpoints, scenario, creds)
81
- LLM-->>BE: {execution_plan[], suggested_credentials}
82
- BE->>DB: UPDATE SpecDocument (plan, merged creds)
83
- BE-->>FE: {execution_plan[]}
84
-
85
- loop For each step in plan
86
- User->>FE: Click "Execute Step N"
87
- FE->>BE: POST /execute-step {step_def, session_id}
88
- BE->>DB: GET full spec schema
89
- BE->>BE: resolve_refs() + prune + shorthand
90
- BE->>DB: GET context_variables for session
91
- BE->>LLM: prepare_step_request(schema, context, creds)
92
- LLM-->>BE: {StepExecutionPlan}
93
- BE->>BE: sanitize_url()
94
- BE->>API: HTTP request (method, url, headers, body)
95
- API-->>BE: {status_code, body}
96
- BE->>DB: INSERT ExecutionHistory (request + response)
97
- BE-->>FE: {request, response}
98
-
99
- FE->>BE: POST /analyze-response {step, response}
100
- BE->>LLM: analyze_api_response(status, body, purpose)
101
- LLM-->>BE: {status, extracted_variables, is_otp_required}
102
- BE->>DB: UPDATE context_variables (merge new vars)
103
- BE->>DB: UPDATE ExecutionHistory (analysis)
104
- BE-->>FE: {analysis_result}
105
- FE-->>User: Show result + extracted variables
106
  end
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
107
  ```
108
 
109
  ---
@@ -113,8 +102,14 @@ sequenceDiagram
113
 
114
  ```mermaid
115
  erDiagram
 
 
 
 
 
 
116
  SPEC_DOCUMENT {
117
- string id PK
118
  string name
119
  text raw_content
120
  json parsed_json
@@ -122,44 +117,32 @@ erDiagram
122
  json login_credentials
123
  json execution_plan
124
  text scenario
125
- string file_path
126
- datetime uploaded_at
127
  }
128
 
129
  SESSION_MODEL {
130
- string id PK
131
- string spec_id FK
132
- string spec_name
133
- string base_url
134
- text scenario
135
  json context_variables
136
- datetime created_at
137
  }
138
 
139
  EXECUTION_HISTORY {
140
  int id PK
141
- string session_id FK
142
  int step
143
  json request_json
144
  json response_json
145
  json analysis_json
146
  json step_definition_json
147
- datetime timestamp
148
  }
149
 
150
  ASSET {
151
- string id PK
152
  string name
153
  string file_type
154
  string file_path
155
- int file_size
156
  string category
157
- text description
158
  json tags
159
- json metadata_json
160
- string spec_id FK
161
- datetime uploaded_at
162
- }
163
 
164
  COLLECTION {
165
  string id PK
 
8
  *Shows the high-level structure of the entire system and how the four tiers interact.*
9
 
10
  ```mermaid
11
+ graph TD
12
+ subgraph "Frontend (React + Vite)"
13
+ UI[User Interface]
14
+ State[AppContext / State]
15
+ Axios[Axios API Client]
16
+ UI --> State
17
+ State --> Axios
 
 
 
 
 
 
 
 
18
  end
19
 
20
+ subgraph "Orchestration (FastAPI)"
21
+ Router[main.py / APIRouters]
22
+ Core[Core Utilities / Config]
23
+ Services[Business Logic / Services]
24
+ Router --> Core
25
+ Router --> Services
26
  end
27
 
28
+ subgraph "Intelligence (LLM Services)"
29
+ Planner[LLM Planner]
30
+ Executor[API Executor]
31
+ Analyzer[Response Analyzer]
32
+ Chat[Chat Assistant]
33
+ Integrations[Integrations / LLM]
34
+ Services --> Planner
35
+ Services --> Executor
36
+ Services --> Analyzer
37
+ Services --> Chat
38
+ Planner & Executor & Analyzer & Chat --> Integrations
39
  end
40
 
41
+ subgraph "Data & Storage"
42
+ DB[(PostgreSQL / SQLAlchemy)]
43
+ Storage[Supabase / Local Storage]
44
+ Services --> DB
45
+ Services --> Storage
46
  end
47
 
48
+ Axios <--> Router
49
+ Integrations <--> LLM_Provider[Mistral-Large API]
 
 
 
 
 
 
 
50
  ```
51
 
52
  ---
 
56
 
57
  ```mermaid
58
  sequenceDiagram
59
+ participant User
60
+ participant Frontend
61
+ participant FastAPI as router/execution.py
62
+ participant DB as database.py
63
+ participant LLM as integrations/llm
64
+ participant TargetAPI
65
+
66
+ User->>Frontend: Clicks "Execute Step"
67
+ Frontend->>FastAPI: POST /execute-step (StepDef)
68
+
69
+ rect rgb(240, 240, 240)
70
+ Note over FastAPI: PRE-EXECUTION PIPELINE
71
+ FastAPI->>DB: Fetch Spec (parsed_json)
72
+ FastAPI->>FastAPI: resolve_refs() + prune + shorthand
73
+ FastAPI->>DB: Fetch session context + profile
74
+ FastAPI->>LLM: prepare_step_request(schema, context)
75
+ LLM-->>FastAPI: {APIRequest}
76
+ FastAPI->>FastAPI: sanitize_url(url)
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
77
  end
78
+
79
+ rect rgb(220, 240, 220)
80
+ Note over FastAPI: HTTP EXECUTION
81
+ FastAPI->>TargetAPI: httpx.send(request)
82
+ TargetAPI-->>FastAPI: {status, body, headers}
83
+ end
84
+
85
+ FastAPI->>DB: Save to ExecutionHistory
86
+ FastAPI-->>Frontend: {request, response}
87
+ Frontend->>User: Display Result (Cards)
88
+
89
+ Frontend->>FastAPI: POST /analyze-response {step, response}
90
+ FastAPI->>LLM: analyze_api_response(status, body, purpose)
91
+ LLM-->>FastAPI: {status, extracted_variables, is_otp_required}
92
+ FastAPI->>DB: UPDATE context_variables (merge new vars)
93
+ FastAPI->>DB: UPDATE ExecutionHistory (analysis)
94
+ FastAPI-->>Frontend: {analysis_result}
95
+ Frontend-->>User: Show result + extracted variables
96
  ```
97
 
98
  ---
 
102
 
103
  ```mermaid
104
  erDiagram
105
+ SPEC_DOCUMENT ||--o{ SESSION_MODEL : "has"
106
+ SPEC_DOCUMENT ||--o{ ASSET : "linked to"
107
+ SESSION_MODEL ||--o{ EXECUTION_HISTORY : "logs"
108
+ CHAT_SESSION ||--o{ CHAT_MESSAGE : "contains"
109
+ SPEC_DOCUMENT ||--o{ CHAT_SESSION : "associated with"
110
+
111
  SPEC_DOCUMENT {
112
+ uuid id PK
113
  string name
114
  text raw_content
115
  json parsed_json
 
117
  json login_credentials
118
  json execution_plan
119
  text scenario
 
 
120
  }
121
 
122
  SESSION_MODEL {
123
+ uuid id PK
124
+ uuid spec_id FK
 
 
 
125
  json context_variables
126
+ string base_url
127
  }
128
 
129
  EXECUTION_HISTORY {
130
  int id PK
131
+ uuid session_id FK
132
  int step
133
  json request_json
134
  json response_json
135
  json analysis_json
136
  json step_definition_json
 
137
  }
138
 
139
  ASSET {
140
+ uuid id PK
141
  string name
142
  string file_type
143
  string file_path
 
144
  string category
 
145
  json tags
 
 
 
 
146
 
147
  COLLECTION {
148
  string id PK
README.md β†’ documents/README.md RENAMED
@@ -4,11 +4,13 @@ An automated API testing system powered by Mistral AI, FastAPI, and React. This
4
 
5
  ## Features
6
 
7
- - **OpenAPI Parsing**: Upload your API spec (JSON or YAML) β€” extracts all endpoints, schemas, methods, and auth details.
8
- - **LLM-Powered Execution Plan**: Mistral AI analyzes your natural-language intent and builds a step-by-step execution plan.
9
- - **Dynamic API Execution**: Executes HTTP requests based on the plan, chaining responses automatically.
10
- - **Context Management**: Extracts tokens/IDs from responses and passes them to subsequent steps.
11
- - **React Frontend**: Modern UI built with React + Vite + Tailwind CSS.
 
 
12
 
13
  ## Requirements
14
 
@@ -57,6 +59,14 @@ cd frontend-react && npm run dev
57
  | Backend | http://localhost:8001 |
58
  | API Docs | http://localhost:8001/docs |
59
 
 
 
 
 
 
 
 
 
60
  ## Usage
61
 
62
  1. Open the frontend at `http://localhost:5173`.
 
4
 
5
  ## Features
6
 
7
+ - **Intelligent Planning**: Generates multi-step test scenarios from natural language.
8
+ - **Context-Aware Execution**: Automatically propagates tokens and variables (IDs, slugs, etc.) between steps.
9
+ - **Persistent AI Chat**: An interactive assistant that understands your full testing context and can control the workflow.
10
+ - **Media Hub**: Manage assets (images, PDFs, audio) for API requests requiring file uploads.
11
+ - **Zero-Code Testing**: Describe your intent (e.g., "Register, login, and update my profile picture") and let the AI handle the rest.
12
+ - **Format Agnostic**: Supports OpenAPI (Swagger) and Postman Collections.
13
+ - **Clean Organization**: Documentation is centrally managed in this `documents/` folder.
14
 
15
  ## Requirements
16
 
 
59
  | Backend | http://localhost:8001 |
60
  | API Docs | http://localhost:8001/docs |
61
 
62
+ ## πŸ“‚ Project Structure
63
+
64
+ - `backend/`: FastAPI server with LLM pipeline logic.
65
+ - `frontend-react/`: Modern React dashboard built with Vite and Tailwind CSS.
66
+ - `documents/`: Comprehensive documentation, diagrams, and research notes (you are here).
67
+ - `start.ps1`: One-click startup script for Windows.
68
+ - `project_for_review.zip`: Optimized archive for sharing the project with reviewers.
69
+
70
  ## Usage
71
 
72
  1. Open the frontend at `http://localhost:5173`.
Research_Paper.md β†’ documents/Research_Paper.md RENAMED
@@ -2,7 +2,7 @@
2
 
3
  **Author:** [Your Name]
4
  **Affiliation:** [Your Institution / Department of Computer Science]
5
- **Date:** April 2026
6
 
7
  ---
8
 
@@ -80,7 +80,7 @@ Despite progress in individual components, a gap exists in the integration of LL
80
 
81
  ## 3. Methodology
82
 
83
- This section provides a comprehensive and technically detailed description of the system's design philosophy, data architecture, algorithmic components, prompt engineering strategies, and end-to-end execution workflow. The system is founded on a novel **Triple-Stage AI Pipeline** β€” Strategic Planning, Context-Aware Execution, and Semantic Response Analysis β€” underpinned by a persistent, session-scoped state management system and a modular service architecture.
84
 
85
  ---
86
 
@@ -643,15 +643,15 @@ Both entry points share the same backend and conversation history, providing a s
643
  | Layer | Technology | Version / Notes |
644
  |---|---|---|
645
  | Backend Language | Python | 3.10+ |
646
- | API Framework | FastAPI | β‰₯ 0.109.0 (async, auto-docs) |
647
- | LLM Orchestration | LangChain + langchain-mistralai | 0.1.0 / 0.0.1 |
648
- | LLM Model | Mistral Large Latest | `mistral-large-latest`, temp=0 |
649
  | HTTP Client | httpx | β‰₯ 0.27.2 (async) |
650
- | ORM | SQLAlchemy | 2.0.23 |
651
  | Database | PostgreSQL (via Supabase) | + `psycopg2-binary` driver |
652
  | File Storage | Supabase Storage | SDK v2.28.2 |
653
  | Frontend Framework | React + Vite | ES6+, SWC compiler |
654
- | Frontend Styling | Tailwind CSS | Glassmorphic Dark Design |
655
  | ASGI Server | Uvicorn (dev) / Gunicorn (prod) | With standard extras |
656
 
657
  ### 4.2 FastAPI Endpoint Design
 
2
 
3
  **Author:** [Your Name]
4
  **Affiliation:** [Your Institution / Department of Computer Science]
5
+ **Date:** May 2026
6
 
7
  ---
8
 
 
80
 
81
  ## 3. Methodology
82
 
83
+ This section provides a comprehensive and technically detailed description of the system's design philosophy, data architecture, algorithmic components, prompt engineering strategies, and end-to-end execution workflow. The system is founded on a novel **Four-Stage AI Pipeline** β€” Strategic Planning, Context-Aware Execution, Semantic Response Analysis, and Conversational AI Assistance β€” underpinned by a persistent, session-scoped state management system and a modular service architecture.
84
 
85
  ---
86
 
 
643
  | Layer | Technology | Version / Notes |
644
  |---|---|---|
645
  | Backend Language | Python | 3.10+ |
646
+ | API Framework | FastAPI | β‰₯ 0.109.0 (async, routers) |
647
+ | LLM Orchestration | LangChain | β‰₯ 0.3.0 |
648
+ | LLM Model | Mistral Large | `mistral-large-latest`, temp=0 |
649
  | HTTP Client | httpx | β‰₯ 0.27.2 (async) |
650
+ | ORM | SQLAlchemy | β‰₯ 2.0.36 |
651
  | Database | PostgreSQL (via Supabase) | + `psycopg2-binary` driver |
652
  | File Storage | Supabase Storage | SDK v2.28.2 |
653
  | Frontend Framework | React + Vite | ES6+, SWC compiler |
654
+ | Frontend Styling | Tailwind CSS | 4.x Modern Design |
655
  | ASGI Server | Uvicorn (dev) / Gunicorn (prod) | With standard extras |
656
 
657
  ### 4.2 FastAPI Endpoint Design