Tripagra commited on
Commit
964103b
·
1 Parent(s): b33a861

Deploy to Vercel: Add serverless functions, frontend, and deployment docs

Browse files
.env.example CHANGED
@@ -1,6 +1,6 @@
1
  # LLM API Configuration
2
  # Google Gemini API
3
- GOOGLE_API_KEY=AIzaSyBEcuYN5-xzT_kU44WL8IvQfIPIW6CaKxs
4
 
5
  # Embedding Model Configuration
6
  EMBEDDING_MODEL=all-MiniLM-L6-v2
 
1
  # LLM API Configuration
2
  # Google Gemini API
3
+ GOOGLE_API_KEY=AIzaSyD18774cyKeznZP409LIqDFMasNgqPxfLw
4
 
5
  # Embedding Model Configuration
6
  EMBEDDING_MODEL=all-MiniLM-L6-v2
.gitignore CHANGED
@@ -52,3 +52,6 @@ frontend/node_modules/
52
  frontend/dist/
53
  frontend/.env
54
  frontend/.env.local
 
 
 
 
52
  frontend/dist/
53
  frontend/.env
54
  frontend/.env.local
55
+
56
+ # Kiro
57
+ .kiro/
.kiro/specs/smart-escalation-api/.config.kiro DELETED
@@ -1 +0,0 @@
1
- {"specId": "90ea9d74-8680-4a15-8227-fea1dfad3026", "workflowType": "requirements-first", "specType": "feature"}
 
 
.kiro/specs/smart-escalation-api/design.md DELETED
@@ -1,750 +0,0 @@
1
- # Design Document: Smart Escalation API
2
-
3
- ## Overview
4
-
5
- The Smart Escalation API is a lightweight RAG-based customer support system designed for 2-3.5 hour implementation. The system answers customer questions about TaskFlow using retrieval-augmented generation over 5 help articles, with intelligent escalation to human agents when confidence is low.
6
-
7
- ### Core Design Principles
8
-
9
- 1. **Simplicity First**: In-memory vector store, single API endpoint, minimal dependencies
10
- 2. **Escalation Over Uncertainty**: Conservative approach prioritizes accuracy over coverage
11
- 3. **Clear Separation**: Distinct modules for retrieval, generation, and escalation logic
12
- 4. **Free Hosting Compatible**: No external databases, deployable to Render/Railway/Vercel
13
-
14
- ### Key Components
15
-
16
- - **FastAPI Backend**: Single POST endpoint handling question processing
17
- - **RAG System**: In-memory FAISS vector store with sentence-transformers embeddings
18
- - **Escalation Engine**: Rule-based decision logic using relevance scores and LLM confidence signals
19
- - **React Chat UI**: Minimal interface for customer interaction
20
-
21
- ## Architecture
22
-
23
- ### System Architecture
24
-
25
- ```mermaid
26
- graph TB
27
- Customer[Customer] --> ChatUI[React Chat UI]
28
- ChatUI --> API[FastAPI Endpoint]
29
- API --> Orchestrator[Request Orchestrator]
30
- Orchestrator --> RAG[RAG System]
31
- RAG --> Embedder[Embedding Model]
32
- RAG --> VectorStore[FAISS Vector Store]
33
- Orchestrator --> Escalation[Escalation Engine]
34
- Escalation --> LLM[LLM API]
35
- Escalation --> Response[Response Builder]
36
- Response --> API
37
-
38
- HelpArticles[(5 Help Articles)] -.->|Load on startup| VectorStore
39
- ```
40
-
41
- ### Request Flow
42
-
43
- 1. **Question Submission**: Customer submits question via Chat UI
44
- 2. **Embedding Generation**: Question is embedded using sentence-transformers
45
- 3. **Retrieval**: Top-k chunks retrieved from FAISS vector store
46
- 4. **Escalation Decision**: Engine evaluates relevance scores and context quality
47
- 5. **Response Generation**:
48
- - If confident: LLM generates grounded answer
49
- - If uncertain: Return escalation message
50
- 6. **Response Delivery**: JSON response with answer/escalation + confidence explanation
51
-
52
- ### Technology Stack
53
-
54
- | Component | Technology | Rationale |
55
- |-----------|-----------|-----------|
56
- | Backend Framework | FastAPI | Fast, async, automatic OpenAPI docs |
57
- | Embeddings | sentence-transformers (all-MiniLM-L6-v2) | Lightweight, no API costs, good quality |
58
- | Vector Store | FAISS (in-memory) | Fast similarity search, no external DB |
59
- | LLM | Google Gemini API | Free tier available, high-quality generation |
60
- | Frontend | React + Vite | Fast setup, minimal boilerplate |
61
- | Deployment | Render/Railway | Free tier, easy Python deployment |
62
-
63
- ## Components and Interfaces
64
-
65
- ### 1. FastAPI Application (`main.py`)
66
-
67
- **Responsibilities**:
68
- - Expose POST `/ask` endpoint
69
- - Handle CORS configuration
70
- - Coordinate request processing
71
- - Return structured JSON responses
72
-
73
- **Interface**:
74
- ```python
75
- @app.post("/ask")
76
- async def ask_question(request: QuestionRequest) -> QuestionResponse:
77
- """
78
- Process customer question and return answer or escalation.
79
-
80
- Args:
81
- request: QuestionRequest with 'question' field
82
-
83
- Returns:
84
- QuestionResponse with response_type, message, confidence_explanation
85
- """
86
- ```
87
-
88
- **Models**:
89
- ```python
90
- class QuestionRequest(BaseModel):
91
- question: str
92
-
93
- class QuestionResponse(BaseModel):
94
- response_type: Literal["answer", "escalation"]
95
- message: str
96
- confidence_explanation: str
97
- sources: Optional[List[str]] = None
98
- ```
99
-
100
- ### 2. RAG System (`rag.py`)
101
-
102
- **Responsibilities**:
103
- - Load and chunk help articles on startup
104
- - Generate embeddings for chunks and questions
105
- - Perform similarity search
106
- - Return top-k relevant chunks with scores
107
-
108
- **Interface**:
109
- ```python
110
- class RAGSystem:
111
- def __init__(self, articles_dir: str, chunk_size: int = 500, chunk_overlap: int = 50):
112
- """Initialize RAG system with help articles."""
113
-
114
- def retrieve(self, question: str, top_k: int = 3) -> List[RetrievedChunk]:
115
- """
116
- Retrieve most relevant chunks for question.
117
-
118
- Args:
119
- question: Customer question
120
- top_k: Number of chunks to retrieve
121
-
122
- Returns:
123
- List of RetrievedChunk with content, score, source
124
- """
125
-
126
- class RetrievedChunk:
127
- content: str
128
- score: float # Cosine similarity score
129
- source: str # Help article filename
130
- ```
131
-
132
- **Chunking Strategy**:
133
- - Fixed-size chunks with overlap (500 chars, 50 char overlap)
134
- - Preserves sentence boundaries where possible
135
- - Metadata tracks source article for each chunk
136
-
137
- ### 3. Escalation Engine (`escalation.py`)
138
-
139
- **Responsibilities**:
140
- - Evaluate retrieval quality (relevance scores)
141
- - Decide whether to answer or escalate
142
- - Generate confidence explanations
143
- - Coordinate LLM calls when answering
144
-
145
- **Interface**:
146
- ```python
147
- class EscalationEngine:
148
- def __init__(self, llm_client: LLMClient, relevance_threshold: float = 0.5):
149
- """Initialize escalation engine with LLM client and thresholds."""
150
-
151
- def process_question(
152
- self,
153
- question: str,
154
- retrieved_chunks: List[RetrievedChunk]
155
- ) -> EscalationDecision:
156
- """
157
- Decide whether to answer or escalate.
158
-
159
- Args:
160
- question: Customer question
161
- retrieved_chunks: Retrieved context from RAG
162
-
163
- Returns:
164
- EscalationDecision with action, message, explanation
165
- """
166
-
167
- class EscalationDecision:
168
- action: Literal["answer", "escalate"]
169
- message: str
170
- confidence_explanation: str
171
- sources: Optional[List[str]] = None
172
- ```
173
-
174
- **Decision Logic**:
175
- ```python
176
- def should_escalate(chunks: List[RetrievedChunk]) -> bool:
177
- """
178
- Escalate if:
179
- 1. Best chunk score < 0.5 (low relevance)
180
- 2. All chunks from same article (narrow coverage)
181
- 3. LLM returns uncertainty signal in response
182
- """
183
- ```
184
-
185
- ### 4. LLM Client (`llm_client.py`)
186
-
187
- **Responsibilities**:
188
- - Abstract LLM API calls (Google Gemini)
189
- - Format prompts with retrieved context
190
- - Parse LLM responses
191
- - Handle API errors gracefully
192
-
193
- **Interface**:
194
- ```python
195
- class LLMClient:
196
- def generate_answer(
197
- self,
198
- question: str,
199
- context_chunks: List[str]
200
- ) -> LLMResponse:
201
- """
202
- Generate answer using LLM with retrieved context.
203
-
204
- Args:
205
- question: Customer question
206
- context_chunks: Retrieved help article chunks
207
-
208
- Returns:
209
- LLMResponse with answer text and confidence signal
210
- """
211
-
212
- class LLMResponse:
213
- answer: str
214
- uncertain: bool # True if LLM signals uncertainty
215
- ```
216
-
217
- **Prompt Template**:
218
- ```
219
- You are a friendly customer support agent for TaskFlow, a project management SaaS.
220
-
221
- Answer the customer's question using ONLY the information provided in the help articles below.
222
-
223
- IMPORTANT RULES:
224
- - Use a friendly, helpful tone
225
- - Ground your answer in the provided context
226
- - If the context doesn't contain enough information to answer confidently, respond with: "I'm not certain about this. Let me connect you with a human agent."
227
- - Do not make up or infer information not present in the context
228
-
229
- HELP ARTICLE CONTEXT:
230
- {context_chunks}
231
-
232
- CUSTOMER QUESTION:
233
- {question}
234
-
235
- YOUR ANSWER:
236
- ```
237
-
238
- ### 5. React Chat UI (`frontend/`)
239
-
240
- **Responsibilities**:
241
- - Render chat interface
242
- - Submit questions to API
243
- - Display answers and escalations
244
- - Show confidence explanations
245
-
246
- **Key Components**:
247
- ```typescript
248
- interface Message {
249
- role: 'user' | 'assistant';
250
- content: string;
251
- confidenceExplanation?: string;
252
- responseType?: 'answer' | 'escalation';
253
- }
254
-
255
- function ChatInterface() {
256
- const [messages, setMessages] = useState<Message[]>([]);
257
- const [input, setInput] = useState('');
258
- const [loading, setLoading] = useState(false);
259
-
260
- const handleSubmit = async () => {
261
- // POST to /ask endpoint
262
- // Update messages with response
263
- };
264
- }
265
- ```
266
-
267
- ## Data Models
268
-
269
- ### Help Article Structure
270
-
271
- ```python
272
- class HelpArticle:
273
- """Represents a single help article."""
274
- filename: str
275
- title: str
276
- content: str
277
- chunks: List[str] # Generated during preprocessing
278
- ```
279
-
280
- **Article Files** (stored as markdown in `data/articles/`):
281
- 1. `getting-started.md` - TaskFlow basics, account setup, first project
282
- 2. `billing.md` - Subscription plans, payment methods, billing cycles
283
- 3. `integrations.md` - Slack, GitHub, Jira integrations
284
- 4. `permissions.md` - Team roles, access control, admin settings
285
- 5. `troubleshooting.md` - Common errors, login issues, performance tips
286
-
287
- ### Vector Store Schema
288
-
289
- ```python
290
- # FAISS index stores:
291
- # - embeddings: np.ndarray of shape (n_chunks, embedding_dim)
292
- # - metadata: List[ChunkMetadata]
293
-
294
- class ChunkMetadata:
295
- chunk_id: int
296
- source_article: str
297
- chunk_text: str
298
- start_char: int # Position in original article
299
- ```
300
-
301
- ### API Request/Response Models
302
-
303
- **Request**:
304
- ```json
305
- {
306
- "question": "How do I add a team member to my project?"
307
- }
308
- ```
309
-
310
- **Response (Answer)**:
311
- ```json
312
- {
313
- "response_type": "answer",
314
- "message": "To add a team member to your project in TaskFlow, go to Project Settings > Team Members and click 'Invite Member'. You can set their role as Viewer, Editor, or Admin depending on the permissions you want to grant.",
315
- "confidence_explanation": "High confidence - retrieved relevant content from 'Managing Team Permissions' article with similarity score 0.82",
316
- "sources": ["permissions.md"]
317
- }
318
- ```
319
-
320
- **Response (Escalation)**:
321
- ```json
322
- {
323
- "response_type": "escalation",
324
- "message": "I'm not certain I can answer this accurately. Let me connect you with a human support agent who can help.",
325
- "confidence_explanation": "Low confidence - best retrieval score was 0.42, indicating question may be outside help article scope",
326
- "sources": null
327
- }
328
- ```
329
-
330
- ## Correctness Properties
331
-
332
- *A property is a characteristic or behavior that should hold true across all valid executions of a system—essentially, a formal statement about what the system should do. Properties serve as the bridge between human-readable specifications and machine-verifiable correctness guarantees.*
333
-
334
-
335
- ### Property 1: Invalid Request Handling
336
-
337
- *For any* malformed or invalid request (missing required fields, wrong data types, malformed JSON), the API SHALL return a 400 status code with an error message.
338
-
339
- **Validates: Requirements 1.4**
340
-
341
- ### Property 2: Top-K Retrieval Behavior
342
-
343
- *For any* customer question and any positive integer k, the RAG system SHALL return exactly k chunks (or fewer if total chunks < k) ordered by descending relevance score.
344
-
345
- **Validates: Requirements 2.4**
346
-
347
- ### Property 3: Chunking Semantic Coherence
348
-
349
- *For any* help article text, the chunking strategy SHALL NOT split words mid-character, SHALL maintain the specified overlap between consecutive chunks, and SHALL preserve complete tokens.
350
-
351
- **Validates: Requirements 2.6**
352
-
353
- ### Property 4: Escalation on Low Confidence
354
-
355
- *For any* question where retrieved chunks have maximum relevance score below the threshold OR the LLM signals uncertainty, the escalation logic SHALL return an escalation response rather than an answer.
356
-
357
- **Validates: Requirements 3.5, 4.2, 4.4**
358
-
359
- ### Property 5: Response Schema Consistency
360
-
361
- *For any* valid customer question, the API response SHALL conform to the defined JSON schema with required fields (response_type, message, confidence_explanation) having correct types, and response_type SHALL be either "answer" or "escalation".
362
-
363
- **Validates: Requirements 5.1, 6.1, 6.2, 6.3, 6.4, 6.5**
364
-
365
- ### Property 6: Prompt Construction Completeness
366
-
367
- *For any* customer question and any set of retrieved chunks, the constructed LLM prompt SHALL include both the complete question text and all retrieved chunk contents.
368
-
369
- **Validates: Requirements 9.4, 9.5**
370
-
371
- ## Error Handling
372
-
373
- ### API-Level Errors
374
-
375
- **Invalid Request Handling**:
376
- - Missing `question` field → 400 with message "Missing required field: question"
377
- - Empty question string → 400 with message "Question cannot be empty"
378
- - Malformed JSON → 400 with message "Invalid JSON format"
379
- - Request timeout (>10s) → 504 with message "Request timeout"
380
-
381
- **LLM API Errors**:
382
- - API key invalid → 500 with message "Configuration error"
383
- - Rate limit exceeded → 429 with message "Service temporarily unavailable, please try again"
384
- - LLM API timeout → Fallback to escalation with explanation "Unable to generate response, connecting you with human agent"
385
- - Network errors → Retry once, then escalate
386
-
387
- **Embedding Errors**:
388
- - Embedding generation fails → Log error, return 500 with message "Unable to process question"
389
- - Model not loaded → Fail fast on startup with clear error message
390
-
391
- ### Graceful Degradation
392
-
393
- **Retrieval Failures**:
394
- - No chunks retrieved (empty vector store) → Automatic escalation
395
- - All relevance scores = 0 → Automatic escalation
396
- - Vector store corruption → Fail fast with error, require restart
397
-
398
- **Response Generation**:
399
- - LLM returns empty response → Treat as uncertainty signal, escalate
400
- - LLM response exceeds length limit → Truncate with "..." and include full response in logs
401
- - Confidence explanation generation fails → Use default: "Unable to generate explanation"
402
-
403
- ### Error Logging
404
-
405
- All errors SHALL be logged with:
406
- - Timestamp
407
- - Error type and message
408
- - Request context (question, retrieved chunks if available)
409
- - Stack trace for 500-level errors
410
-
411
- ## Testing Strategy
412
-
413
- ### Overview
414
-
415
- The Smart Escalation API requires a **hybrid testing approach** combining unit tests, integration tests, and property-based tests. Given the system's reliance on external services (LLM APIs, embedding models) and non-deterministic behavior, property-based testing is appropriate only for the pure logic components.
416
-
417
- ### Unit Tests
418
-
419
- **Focus Areas**:
420
- - Chunking logic (text splitting, overlap calculation)
421
- - Escalation decision rules (threshold comparisons)
422
- - Response formatting (JSON construction)
423
- - Prompt template construction
424
- - Error handling paths
425
-
426
- **Example Test Cases**:
427
- ```python
428
- def test_chunking_preserves_sentences():
429
- """Verify chunks don't split mid-sentence."""
430
- article = "First sentence. Second sentence. Third sentence."
431
- chunks = chunk_text(article, chunk_size=20, overlap=5)
432
- for chunk in chunks:
433
- assert not chunk.endswith(" ") # No trailing spaces
434
- assert chunk[0].isupper() or chunk[0] in "0123456789" # Starts properly
435
-
436
- def test_escalation_on_low_score():
437
- """Verify escalation when relevance below threshold."""
438
- chunks = [RetrievedChunk(content="...", score=0.3, source="test.md")]
439
- decision = escalation_engine.process_question("test", chunks)
440
- assert decision.action == "escalate"
441
-
442
- def test_response_schema():
443
- """Verify response has required fields."""
444
- response = api_client.post("/ask", json={"question": "test"})
445
- assert "response_type" in response.json()
446
- assert "message" in response.json()
447
- assert "confidence_explanation" in response.json()
448
- ```
449
-
450
- ### Property-Based Tests
451
-
452
- **Testing Library**: Hypothesis (Python)
453
-
454
- **Configuration**: Minimum 100 iterations per property test
455
-
456
- **Property Test Cases**:
457
-
458
- ```python
459
- from hypothesis import given, strategies as st
460
-
461
- @given(st.text(min_size=1, max_size=5000))
462
- def test_property_chunking_no_word_splits(article_text):
463
- """
464
- Property 3: Chunking Semantic Coherence
465
- For any article text, chunks should not split words.
466
-
467
- Feature: smart-escalation-api, Property 3: Chunking preserves semantic coherence
468
- """
469
- chunks = chunk_text(article_text, chunk_size=500, overlap=50)
470
- for chunk in chunks:
471
- # No chunks should end with partial words (non-whitespace followed by cut)
472
- if len(chunk) == 500: # Full-size chunk
473
- assert chunk[-1].isspace() or chunk[-1] in ".,!?;:"
474
-
475
- @given(st.integers(min_value=1, max_value=10))
476
- def test_property_topk_retrieval(k):
477
- """
478
- Property 2: Top-K Retrieval Behavior
479
- For any k value, retrieval returns exactly k chunks (or fewer if not enough chunks exist).
480
-
481
- Feature: smart-escalation-api, Property 2: Top-k retrieval returns exactly k chunks
482
- """
483
- question = "How do I reset my password?"
484
- chunks = rag_system.retrieve(question, top_k=k)
485
- assert len(chunks) <= k
486
- # Verify descending order
487
- scores = [c.score for c in chunks]
488
- assert scores == sorted(scores, reverse=True)
489
-
490
- @given(st.floats(min_value=0.0, max_value=0.49))
491
- def test_property_escalation_on_low_confidence(max_score):
492
- """
493
- Property 4: Escalation on Low Confidence
494
- For any retrieval with max score below threshold (0.5), escalation should occur.
495
-
496
- Feature: smart-escalation-api, Property 4: Escalation triggered by low confidence
497
- """
498
- chunks = [
499
- RetrievedChunk(content="test", score=max_score, source="test.md")
500
- ]
501
- decision = escalation_engine.process_question("test question", chunks)
502
- assert decision.action == "escalate"
503
-
504
- @given(st.text(min_size=1, max_size=500))
505
- def test_property_response_schema(question):
506
- """
507
- Property 5: Response Schema Consistency
508
- For any valid question, response conforms to schema.
509
-
510
- Feature: smart-escalation-api, Property 5: Response schema consistency
511
- """
512
- response = api_client.post("/ask", json={"question": question})
513
- data = response.json()
514
- assert "response_type" in data
515
- assert data["response_type"] in ["answer", "escalation"]
516
- assert "message" in data
517
- assert isinstance(data["message"], str)
518
- assert "confidence_explanation" in data
519
- assert isinstance(data["confidence_explanation"], str)
520
-
521
- @given(
522
- st.text(min_size=1, max_size=200),
523
- st.lists(st.text(min_size=10, max_size=500), min_size=1, max_size=5)
524
- )
525
- def test_property_prompt_construction(question, chunks):
526
- """
527
- Property 6: Prompt Construction Completeness
528
- For any question and chunks, prompt includes both.
529
-
530
- Feature: smart-escalation-api, Property 6: Prompt includes context and question
531
- """
532
- prompt = llm_client.build_prompt(question, chunks)
533
- assert question in prompt
534
- for chunk in chunks:
535
- assert chunk in prompt
536
- ```
537
-
538
- ### Integration Tests
539
-
540
- **Focus Areas**:
541
- - End-to-end API flow (question → response)
542
- - LLM API integration (with mocked responses)
543
- - Embedding model integration
544
- - CORS configuration
545
- - Error handling with external services
546
-
547
- **Example Test Cases**:
548
- ```python
549
- def test_integration_answer_flow():
550
- """Test full flow for answerable question."""
551
- response = api_client.post("/ask", json={
552
- "question": "How do I create a new project?"
553
- })
554
- assert response.status_code == 200
555
- data = response.json()
556
- assert data["response_type"] in ["answer", "escalation"]
557
-
558
- def test_integration_cors_headers():
559
- """Verify CORS headers present."""
560
- response = api_client.options("/ask")
561
- assert "Access-Control-Allow-Origin" in response.headers
562
-
563
- @mock.patch('llm_client.openai_client')
564
- def test_integration_llm_timeout(mock_llm):
565
- """Verify graceful handling of LLM timeout."""
566
- mock_llm.side_effect = TimeoutError()
567
- response = api_client.post("/ask", json={"question": "test"})
568
- data = response.json()
569
- assert data["response_type"] == "escalation"
570
- ```
571
-
572
- ### Frontend Tests
573
-
574
- **Testing Library**: React Testing Library + Vitest
575
-
576
- **Focus Areas**:
577
- - Component rendering
578
- - User interactions (input, submit)
579
- - API response display
580
- - Loading states
581
-
582
- **Example Test Cases**:
583
- ```typescript
584
- test('displays user question in chat history', async () => {
585
- render(<ChatInterface />);
586
- const input = screen.getByRole('textbox');
587
- const submitButton = screen.getByRole('button', { name: /send/i });
588
-
589
- await userEvent.type(input, 'How do I reset my password?');
590
- await userEvent.click(submitButton);
591
-
592
- expect(screen.getByText('How do I reset my password?')).toBeInTheDocument();
593
- });
594
-
595
- test('shows loading state during API call', async () => {
596
- render(<ChatInterface />);
597
- const input = screen.getByRole('textbox');
598
- const submitButton = screen.getByRole('button', { name: /send/i });
599
-
600
- await userEvent.type(input, 'test question');
601
- await userEvent.click(submitButton);
602
-
603
- expect(screen.getByText(/loading/i)).toBeInTheDocument();
604
- });
605
- ```
606
-
607
- ### Test Coverage Goals
608
-
609
- - **Unit Tests**: 80%+ coverage of pure logic functions
610
- - **Property Tests**: All 6 correctness properties implemented
611
- - **Integration Tests**: All API endpoints and error paths
612
- - **Frontend Tests**: All user interactions and state changes
613
-
614
- ### Manual Testing Checklist
615
-
616
- - [ ] Deploy to Render/Railway and verify functionality
617
- - [ ] Test with various question types (in-scope, out-of-scope, edge cases)
618
- - [ ] Verify LLM responses have appropriate tone
619
- - [ ] Check confidence explanations are helpful
620
- - [ ] Test error scenarios (invalid API key, network issues)
621
- - [ ] Verify UI is functional on desktop and mobile browsers
622
-
623
- ## Implementation Notes
624
-
625
- ### Development Phases
626
-
627
- **Phase 1: Core RAG System (1 hour)**
628
- - Implement article loading and chunking
629
- - Set up FAISS vector store
630
- - Implement embedding generation
631
- - Test retrieval with sample questions
632
-
633
- **Phase 2: API and Escalation Logic (1 hour)**
634
- - Create FastAPI application
635
- - Implement escalation engine
636
- - Integrate LLM client
637
- - Add error handling
638
-
639
- **Phase 3: Frontend (0.5-1 hour)**
640
- - Create React chat interface
641
- - Implement API integration
642
- - Add loading states
643
- - Basic styling
644
-
645
- **Phase 4: Testing and Deployment (0.5 hour)**
646
- - Write unit and property tests
647
- - Deploy to hosting platform
648
- - End-to-end testing
649
-
650
- ### Configuration Management
651
-
652
- **Environment Variables**:
653
- ```bash
654
- # Required
655
- GOOGLE_API_KEY=AIza...
656
- EMBEDDING_MODEL=all-MiniLM-L6-v2
657
- RELEVANCE_THRESHOLD=0.5
658
-
659
- # Optional
660
- TOP_K_CHUNKS=3
661
- CHUNK_SIZE=500
662
- CHUNK_OVERLAP=50
663
- LLM_MODEL=gemini-1.5-flash
664
- LLM_TEMPERATURE=0.3
665
- ```
666
-
667
- **Configuration File** (`config.py`):
668
- ```python
669
- from pydantic_settings import BaseSettings
670
-
671
- class Settings(BaseSettings):
672
- google_api_key: str
673
- embedding_model: str = "all-MiniLM-L6-v2"
674
- relevance_threshold: float = 0.5
675
- top_k_chunks: int = 3
676
- chunk_size: int = 500
677
- chunk_overlap: int = 50
678
- llm_model: str = "gemini-1.5-flash"
679
- llm_temperature: float = 0.3
680
-
681
- class Config:
682
- env_file = ".env"
683
- ```
684
-
685
- ### Performance Considerations
686
-
687
- **Startup Time**:
688
- - Load articles and build vector store on startup (~2-3 seconds)
689
- - Cache embeddings to avoid recomputation
690
- - Use lazy loading for embedding model
691
-
692
- **Response Time**:
693
- - Target: <5 seconds for 95th percentile
694
- - Embedding generation: ~100ms
695
- - Vector search: ~50ms
696
- - LLM API call: 2-4 seconds (main bottleneck)
697
- - Total: ~3-5 seconds
698
-
699
- **Memory Usage**:
700
- - FAISS index: ~10-20 MB for 5 articles
701
- - Embedding model: ~100 MB
702
- - Total: ~150 MB (well within free tier limits)
703
-
704
- ### Security Considerations
705
-
706
- **API Key Management**:
707
- - Store LLM API keys in environment variables
708
- - Never commit keys to version control
709
- - Use `.env` file for local development
710
- - Use platform secrets for deployment
711
-
712
- **Input Validation**:
713
- - Sanitize user input to prevent injection attacks
714
- - Limit question length (max 500 characters)
715
- - Rate limiting (if needed): 10 requests/minute per IP
716
-
717
- **CORS Configuration**:
718
- - Allow only specific frontend origins in production
719
- - Use wildcard (`*`) only for development
720
-
721
- ### Deployment Instructions
722
-
723
- **Render Deployment**:
724
- 1. Create new Web Service
725
- 2. Connect GitHub repository
726
- 3. Set build command: `pip install -r requirements.txt`
727
- 4. Set start command: `uvicorn main:app --host 0.0.0.0 --port $PORT`
728
- 5. Add environment variables (API keys)
729
- 6. Deploy
730
-
731
- **Frontend Deployment** (Vercel):
732
- 1. Create new project
733
- 2. Connect GitHub repository
734
- 3. Set framework preset: React
735
- 4. Set build command: `npm run build`
736
- 5. Set output directory: `dist`
737
- 6. Add environment variable: `VITE_API_URL=<backend-url>`
738
- 7. Deploy
739
-
740
- ### Future Enhancements
741
-
742
- **Potential Improvements** (out of scope for initial implementation):
743
- - Conversation history tracking
744
- - Multi-turn dialogue support
745
- - Feedback collection (thumbs up/down)
746
- - Analytics dashboard for escalation rates
747
- - A/B testing different escalation thresholds
748
- - Persistent vector store (PostgreSQL with pgvector)
749
- - Streaming LLM responses
750
- - Multi-language support
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
.kiro/specs/smart-escalation-api/requirements.md DELETED
@@ -1,144 +0,0 @@
1
- # Requirements Document
2
-
3
- ## Introduction
4
-
5
- The Smart Escalation API is an AI-powered L1 customer support system that intelligently answers customer questions about TaskFlow (a fictional project management SaaS) or escalates to human agents when uncertain. The system uses RAG-based retrieval over 5 help articles to ground responses and includes a minimal React chat UI for interaction.
6
-
7
- ## Glossary
8
-
9
- - **API**: The FastAPI backend service that processes customer questions
10
- - **RAG_System**: The Retrieval-Augmented Generation system that searches help articles using embeddings and vector store
11
- - **LLM**: The Large Language Model (Claude, OpenAI, etc.) that generates responses
12
- - **Escalation_Logic**: The decision-making component that determines whether to answer or escalate
13
- - **Chat_UI**: The React-based frontend interface for customer interaction
14
- - **Help_Article**: One of 5 TaskFlow documentation articles stored in the knowledge base
15
- - **Confidence_Indicator**: An explanation of why the system answered or escalated
16
- - **Vector_Store**: In-memory storage for document embeddings used in retrieval
17
- - **Chunk**: A segment of a Help_Article used for embedding and retrieval
18
-
19
- ## Requirements
20
-
21
- ### Requirement 1: API Endpoint
22
-
23
- **User Story:** As a customer, I want to submit questions through an API endpoint, so that I can receive support answers or escalation messages.
24
-
25
- #### Acceptance Criteria
26
-
27
- 1. THE API SHALL expose a single POST endpoint that accepts customer questions
28
- 2. WHEN a customer question is received, THE API SHALL return a JSON response within 10 seconds
29
- 3. THE API SHALL include CORS headers to allow requests from the Chat_UI
30
- 4. WHEN the API receives an invalid request, THE API SHALL return a 400 status code with an error message
31
- 5. THE API SHALL be deployable to free hosting platforms (Render, Railway, or Vercel)
32
-
33
- ### Requirement 2: RAG-Based Retrieval
34
-
35
- **User Story:** As the system, I want to retrieve relevant help article content, so that I can ground responses in accurate documentation.
36
-
37
- #### Acceptance Criteria
38
-
39
- 1. THE RAG_System SHALL load 5 Help_Articles about TaskFlow on startup
40
- 2. THE RAG_System SHALL split Help_Articles into Chunks for embedding
41
- 3. WHEN a customer question is received, THE RAG_System SHALL generate an embedding for the question
42
- 4. THE RAG_System SHALL retrieve the top-k most relevant Chunks from the Vector_Store
43
- 5. THE RAG_System SHALL use an in-memory Vector_Store (no external database required)
44
- 6. FOR ALL Help_Articles, the chunking strategy SHALL preserve semantic coherence
45
-
46
- ### Requirement 3: Answer Generation
47
-
48
- **User Story:** As a customer, I want accurate answers grounded in help articles, so that I can resolve my issues quickly.
49
-
50
- #### Acceptance Criteria
51
-
52
- 1. WHEN the Escalation_Logic determines confidence is sufficient, THE LLM SHALL generate an answer using retrieved Chunks
53
- 2. THE LLM SHALL use a friendly customer support tone in all responses
54
- 3. THE LLM SHALL ground all answers in the retrieved Help_Article content
55
- 4. THE LLM SHALL NOT hallucinate information not present in the Help_Articles
56
- 5. WHEN no relevant information is found, THE Escalation_Logic SHALL trigger escalation
57
-
58
- ### Requirement 4: Escalation Decision
59
-
60
- **User Story:** As a support manager, I want the system to escalate uncertain questions to human agents, so that customers receive accurate help.
61
-
62
- #### Acceptance Criteria
63
-
64
- 1. THE Escalation_Logic SHALL decide whether to answer or escalate for each customer question
65
- 2. WHEN confidence is low, THE Escalation_Logic SHALL return an escalation message
66
- 3. WHEN the question is outside the scope of Help_Articles, THE Escalation_Logic SHALL escalate
67
- 4. WHEN retrieved Chunks have low relevance scores, THE Escalation_Logic SHALL escalate
68
- 5. THE Escalation_Logic SHALL prioritize escalation over providing uncertain answers
69
-
70
- ### Requirement 5: Confidence Indicator
71
-
72
- **User Story:** As a support manager, I want to understand why the system answered or escalated, so that I can evaluate system performance.
73
-
74
- #### Acceptance Criteria
75
-
76
- 1. THE API SHALL include a Confidence_Indicator in every response
77
- 2. THE Confidence_Indicator SHALL explain the reasoning behind the answer or escalation decision
78
- 3. WHEN answering, THE Confidence_Indicator SHALL reference the relevance of retrieved content
79
- 4. WHEN escalating, THE Confidence_Indicator SHALL explain why confidence was insufficient
80
- 5. THE Confidence_Indicator SHALL be human-readable and concise
81
-
82
- ### Requirement 6: Response Format
83
-
84
- **User Story:** As a frontend developer, I want a consistent JSON response format, so that I can reliably display results in the Chat_UI.
85
-
86
- #### Acceptance Criteria
87
-
88
- 1. THE API SHALL return responses with a consistent JSON schema
89
- 2. THE API SHALL include a response_type field indicating "answer" or "escalation"
90
- 3. THE API SHALL include a message field containing the answer or escalation text
91
- 4. THE API SHALL include a confidence_explanation field with the Confidence_Indicator
92
- 5. WHEN answering, THE API SHALL optionally include source references to Help_Articles
93
-
94
- ### Requirement 7: Chat UI
95
-
96
- **User Story:** As a customer, I want a simple chat interface, so that I can interact with the support system.
97
-
98
- #### Acceptance Criteria
99
-
100
- 1. THE Chat_UI SHALL provide a text input field for customer questions
101
- 2. WHEN a customer submits a question, THE Chat_UI SHALL display the question in the chat history
102
- 3. THE Chat_UI SHALL display API responses (answers or escalations) in the chat history
103
- 4. THE Chat_UI SHALL indicate loading state while waiting for API responses
104
- 5. THE Chat_UI SHALL be functional and deployable (visual polish is not required)
105
- 6. THE Chat_UI SHALL display the Confidence_Indicator for each response
106
-
107
- ### Requirement 8: Help Articles Knowledge Base
108
-
109
- **User Story:** As the system, I want to load TaskFlow help articles, so that I can answer customer questions accurately.
110
-
111
- #### Acceptance Criteria
112
-
113
- 1. THE RAG_System SHALL load 5 Help_Articles covering TaskFlow topics
114
- 2. THE Help_Articles SHALL include "Getting Started with TaskFlow"
115
- 3. THE Help_Articles SHALL include "Billing and Subscription Plans"
116
- 4. THE Help_Articles SHALL include "Integrations"
117
- 5. THE Help_Articles SHALL include "Managing Team Permissions"
118
- 6. THE Help_Articles SHALL include "Troubleshooting Common Issues"
119
- 7. THE Help_Articles SHALL be stored in a format accessible to the RAG_System
120
-
121
- ### Requirement 9: Prompt Design
122
-
123
- **User Story:** As a developer, I want well-designed LLM prompts, so that the system generates high-quality responses with appropriate guardrails.
124
-
125
- #### Acceptance Criteria
126
-
127
- 1. THE LLM SHALL receive instructions to use a friendly customer support tone
128
- 2. THE LLM SHALL receive instructions to ground answers only in provided context
129
- 3. THE LLM SHALL receive instructions to avoid hallucination
130
- 4. THE LLM SHALL receive the retrieved Chunks as context
131
- 5. THE LLM SHALL receive the customer question
132
- 6. THE LLM SHALL be instructed on when to indicate uncertainty
133
-
134
- ### Requirement 10: Code Quality
135
-
136
- **User Story:** As a developer, I want clean and maintainable code, so that the system is easy to understand and extend.
137
-
138
- #### Acceptance Criteria
139
-
140
- 1. THE API SHALL use clear function and variable names
141
- 2. THE API SHALL include comments explaining complex logic
142
- 3. THE API SHALL separate concerns (retrieval, generation, escalation logic)
143
- 4. THE API SHALL include a README with setup and deployment instructions
144
- 5. THE API SHALL use type hints (Python) or TypeScript types for clarity
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
.kiro/specs/smart-escalation-api/tasks.md DELETED
@@ -1,242 +0,0 @@
1
- # Implementation Plan: Smart Escalation API
2
-
3
- ## Overview
4
-
5
- This implementation plan follows a 4-phase approach to build a RAG-based customer support system with intelligent escalation. The system uses FastAPI, FAISS vector store, sentence-transformers embeddings, and an LLM API to answer customer questions about TaskFlow or escalate to human agents when uncertain.
6
-
7
- **Total Estimated Time**: 2-3.5 hours
8
-
9
- **Key Focus**: Escalation logic is the most critical component. Property-based testing ensures correctness of core behaviors.
10
-
11
- ## Tasks
12
-
13
- ### Phase 1: Core RAG System (1 hour)
14
-
15
- - [x] 1. Set up project structure and dependencies
16
- - Create project directory structure (`data/articles/`, `src/`, `tests/`)
17
- - Create `requirements.txt` with FastAPI, FAISS, sentence-transformers, pytest, hypothesis
18
- - Create `.env.example` file with required environment variables
19
- - Create `README.md` with setup instructions
20
- - _Requirements: 10.4_
21
-
22
- - [x] 2. Create help article knowledge base
23
- - [x] 2.1 Write 5 TaskFlow help articles as markdown files
24
- - Create `data/articles/getting-started.md` (TaskFlow basics, account setup, first project)
25
- - Create `data/articles/billing.md` (subscription plans, payment methods, billing cycles)
26
- - Create `data/articles/integrations.md` (Slack, GitHub, Jira integrations)
27
- - Create `data/articles/permissions.md` (team roles, access control, admin settings)
28
- - Create `data/articles/troubleshooting.md` (common errors, login issues, performance tips)
29
- - _Requirements: 8.1, 8.2, 8.3, 8.4, 8.5, 8.6, 8.7_
30
-
31
- - [x] 3. Implement article chunking logic
32
- - [x] 3.1 Create `src/chunking.py` with text chunking function
33
- - Implement fixed-size chunking with overlap (500 chars, 50 char overlap)
34
- - Preserve sentence boundaries where possible
35
- - Track source article metadata for each chunk
36
- - _Requirements: 2.2, 2.6_
37
-
38
- - [x] 3.2 Write property test for chunking semantic coherence
39
- - **Property 3: Chunking Semantic Coherence**
40
- - **Validates: Requirements 2.6**
41
- - Test that chunks don't split words mid-character
42
- - Test that overlap is maintained between consecutive chunks
43
- - Use Hypothesis to generate random article texts
44
-
45
- - [x] 4. Implement RAG system with FAISS vector store
46
- - [x] 4.1 Create `src/rag.py` with RAGSystem class
47
- - Implement article loading from `data/articles/` directory
48
- - Initialize sentence-transformers embedding model (all-MiniLM-L6-v2)
49
- - Generate embeddings for all chunks
50
- - Build in-memory FAISS index
51
- - Implement `retrieve()` method for similarity search
52
- - _Requirements: 2.1, 2.3, 2.4, 2.5_
53
-
54
- - [ ]* 4.2 Write property test for top-k retrieval behavior
55
- - **Property 2: Top-K Retrieval Behavior**
56
- - **Validates: Requirements 2.4**
57
- - Test that retrieval returns exactly k chunks (or fewer if total < k)
58
- - Test that chunks are ordered by descending relevance score
59
- - Use Hypothesis to generate different k values
60
-
61
- - [x] 5. Checkpoint - Test RAG system with sample questions
62
- - Ensure RAG system loads articles and retrieves relevant chunks, ask the user if questions arise.
63
-
64
- ### Phase 2: API and Escalation Logic (1 hour)
65
-
66
- - [x] 6. Implement LLM client
67
- - [x] 6.1 Create `src/llm_client.py` with LLMClient class
68
- - Implement Google Gemini API integration
69
- - Create prompt template with customer support instructions
70
- - Implement `generate_answer()` method
71
- - Parse LLM responses for uncertainty signals
72
- - Add error handling for API timeouts and rate limits
73
- - _Requirements: 3.1, 3.2, 3.3, 3.4, 9.1, 9.2, 9.3, 9.4, 9.5, 9.6_
74
-
75
- - [ ]* 6.2 Write property test for prompt construction completeness
76
- - **Property 6: Prompt Construction Completeness**
77
- - **Validates: Requirements 9.4, 9.5**
78
- - Test that constructed prompts include complete question text
79
- - Test that all retrieved chunk contents are included in prompt
80
- - Use Hypothesis to generate random questions and chunk lists
81
-
82
- - [x] 7. Implement escalation engine
83
- - [x] 7.1 Create `src/escalation.py` with EscalationEngine class
84
- - Implement relevance threshold checking (default 0.5)
85
- - Implement escalation decision logic
86
- - Generate confidence explanations for answers and escalations
87
- - Coordinate LLM calls when answering
88
- - _Requirements: 3.5, 4.1, 4.2, 4.3, 4.4, 4.5, 5.1, 5.2, 5.3, 5.4, 5.5_
89
-
90
- - [ ]* 7.2 Write property test for escalation on low confidence
91
- - **Property 4: Escalation on Low Confidence**
92
- - **Validates: Requirements 3.5, 4.2, 4.4**
93
- - Test that escalation occurs when max relevance score < threshold
94
- - Test that escalation occurs when LLM signals uncertainty
95
- - Use Hypothesis to generate relevance scores below threshold
96
-
97
- - [x] 8. Create FastAPI application
98
- - [x] 8.1 Create `src/main.py` with FastAPI app
99
- - Define Pydantic models (QuestionRequest, QuestionResponse)
100
- - Implement POST `/ask` endpoint
101
- - Configure CORS for frontend access
102
- - Wire together RAG system, escalation engine, and LLM client
103
- - Add request validation and error handling
104
- - _Requirements: 1.1, 1.2, 1.3, 1.4, 6.1, 6.2, 6.3, 6.4, 6.5_
105
-
106
- - [ ]* 8.2 Write property test for invalid request handling
107
- - **Property 1: Invalid Request Handling**
108
- - **Validates: Requirements 1.4**
109
- - Test that malformed requests return 400 status code
110
- - Test that missing required fields return error messages
111
- - Use Hypothesis to generate invalid request payloads
112
-
113
- - [ ]* 8.3 Write property test for response schema consistency
114
- - **Property 5: Response Schema Consistency**
115
- - **Validates: Requirements 5.1, 6.1, 6.2, 6.3, 6.4, 6.5**
116
- - Test that all responses conform to JSON schema
117
- - Test that response_type is either "answer" or "escalation"
118
- - Test that required fields have correct types
119
- - Use Hypothesis to generate random valid questions
120
-
121
- - [x] 9. Implement configuration management
122
- - [x] 9.1 Create `src/config.py` with Pydantic settings
123
- - Define Settings class with environment variables
124
- - Set default values for thresholds and parameters
125
- - Load configuration from `.env` file
126
- - _Requirements: 10.5_
127
-
128
- - [x] 10. Checkpoint - Test API end-to-end
129
- - Ensure API accepts questions and returns valid responses, ask the user if questions arise.
130
-
131
- ### Phase 3: Frontend (0.5-1 hour)
132
-
133
- - [ ] 11. Set up React frontend with Vite
134
- - [ ] 11.1 Create React project structure
135
- - Initialize Vite project with React and TypeScript
136
- - Install dependencies (axios for API calls)
137
- - Create `frontend/.env.example` with API URL
138
- - _Requirements: 7.5_
139
-
140
- - [ ] 12. Implement chat interface
141
- - [ ] 12.1 Create ChatInterface component
142
- - Implement message state management (user and assistant messages)
143
- - Create text input field for customer questions
144
- - Create submit button with loading state
145
- - Display chat history with user questions and API responses
146
- - Show confidence explanations for each response
147
- - Differentiate between answer and escalation response types
148
- - _Requirements: 7.1, 7.2, 7.3, 7.4, 7.6_
149
-
150
- - [ ]* 12.2 Write unit tests for ChatInterface component
151
- - Test that user questions appear in chat history
152
- - Test loading state during API calls
153
- - Test display of API responses
154
- - Use React Testing Library and Vitest
155
-
156
- - [ ] 13. Integrate frontend with backend API
157
- - [ ] 13.1 Implement API client
158
- - Create API service function to POST questions to `/ask` endpoint
159
- - Handle API errors gracefully
160
- - Configure CORS and API URL from environment variables
161
- - _Requirements: 1.3_
162
-
163
- - [ ] 14. Add basic styling
164
- - [ ] 14.1 Style chat interface
165
- - Add minimal CSS for chat layout
166
- - Style message bubbles (user vs assistant)
167
- - Add loading indicator
168
- - Ensure mobile responsiveness
169
- - _Requirements: 7.5_
170
-
171
- ### Phase 4: Testing and Deployment (0.5 hour)
172
-
173
- - [ ] 15. Write unit tests for core components
174
- - [ ]* 15.1 Write unit tests for chunking logic
175
- - Test chunk size and overlap calculations
176
- - Test sentence boundary preservation
177
- - _Requirements: 2.2, 2.6_
178
-
179
- - [ ]* 15.2 Write unit tests for escalation decision rules
180
- - Test threshold comparisons
181
- - Test confidence explanation generation
182
- - _Requirements: 4.1, 4.2, 4.3, 4.4, 4.5_
183
-
184
- - [ ]* 15.3 Write unit tests for response formatting
185
- - Test JSON construction
186
- - Test source reference inclusion
187
- - _Requirements: 6.1, 6.2, 6.3, 6.4, 6.5_
188
-
189
- - [ ]* 16. Write integration tests
190
- - [ ]* 16.1 Test end-to-end API flow
191
- - Test answerable questions return answer responses
192
- - Test out-of-scope questions trigger escalation
193
- - Mock LLM API responses
194
- - _Requirements: 1.1, 1.2_
195
-
196
- - [ ]* 16.2 Test error handling
197
- - Test LLM API timeout handling
198
- - Test invalid API key handling
199
- - Test network error handling
200
- - _Requirements: 1.4_
201
-
202
- - [ ]* 16.3 Test CORS configuration
203
- - Verify CORS headers are present
204
- - Test OPTIONS requests
205
- - _Requirements: 1.3_
206
-
207
- - [ ] 17. Create deployment documentation
208
- - [ ] 17.1 Update README with deployment instructions
209
- - Document Render/Railway deployment steps for backend
210
- - Document Vercel deployment steps for frontend
211
- - Include environment variable configuration
212
- - Add troubleshooting section
213
- - _Requirements: 1.5, 10.4_
214
-
215
- - [ ] 18. Deploy to hosting platforms
216
- - [ ] 18.1 Deploy backend to Render or Railway
217
- - Create web service
218
- - Configure build and start commands
219
- - Set environment variables (API keys, thresholds)
220
- - Verify deployment and test with sample requests
221
- - _Requirements: 1.5_
222
-
223
- - [ ] 18.2 Deploy frontend to Vercel
224
- - Create new project
225
- - Configure build settings
226
- - Set API URL environment variable
227
- - Verify deployment and test chat interface
228
- - _Requirements: 1.5, 7.5_
229
-
230
- - [ ] 19. Final checkpoint - End-to-end testing
231
- - Test deployed application with various question types (in-scope, out-of-scope, edge cases), verify LLM responses have appropriate tone, check confidence explanations are helpful, ask the user if questions arise.
232
-
233
- ## Notes
234
-
235
- - Tasks marked with `*` are optional and can be skipped for faster MVP
236
- - Each task references specific requirements for traceability
237
- - Property tests validate the 6 correctness properties defined in the design
238
- - Unit tests validate specific examples and edge cases
239
- - Checkpoints ensure incremental validation at key milestones
240
- - Focus on escalation logic as the most critical component
241
- - Use Python for backend (FastAPI, FAISS, sentence-transformers)
242
- - Use TypeScript for frontend (React, Vite)
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
.vercelignore ADDED
@@ -0,0 +1,41 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # Python
2
+ __pycache__/
3
+ *.py[cod]
4
+ *$py.class
5
+ *.so
6
+ .Python
7
+ venv/
8
+ env/
9
+ ENV/
10
+
11
+ # Testing
12
+ tests/
13
+ .pytest_cache/
14
+ .hypothesis/
15
+ .coverage
16
+ htmlcov/
17
+
18
+ # IDE
19
+ .vscode/
20
+ .idea/
21
+ *.swp
22
+ *.swo
23
+
24
+ # OS
25
+ .DS_Store
26
+ Thumbs.db
27
+
28
+ # Git
29
+ .git/
30
+ .gitignore
31
+
32
+ # Documentation
33
+ *.md
34
+ !README.md
35
+
36
+ # Benchmarks
37
+ .benchmarks/
38
+
39
+ # Local env files
40
+ .env
41
+ .env.local
DEPLOYMENT.md ADDED
@@ -0,0 +1,360 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # Vercel Deployment Guide
2
+
3
+ This guide walks you through deploying the Smart Escalation API to Vercel as a monorepo with both frontend and backend.
4
+
5
+ ## Quick Start
6
+
7
+ ### 1. Prerequisites
8
+
9
+ - GitHub account with this repository
10
+ - Google Gemini API key ([Get one here](https://makersuite.google.com/app/apikey))
11
+ - Vercel account ([Sign up free](https://vercel.com/signup))
12
+
13
+ ### 2. Deploy to Vercel
14
+
15
+ Click the button below to deploy:
16
+
17
+ [![Deploy with Vercel](https://vercel.com/button)](https://vercel.com/new/clone?repository-url=https://github.com/yourusername/smart-escalation-api)
18
+
19
+ Or follow manual steps:
20
+
21
+ #### Step 1: Import Project
22
+
23
+ 1. Go to [vercel.com](https://vercel.com)
24
+ 2. Click "Add New..." → "Project"
25
+ 3. Import your GitHub repository
26
+ 4. Vercel will auto-detect the configuration from `vercel.json`
27
+
28
+ #### Step 2: Configure Environment Variables
29
+
30
+ Add these environment variables in the Vercel dashboard:
31
+
32
+ | Variable | Value | Required |
33
+ |----------|-------|----------|
34
+ | `GOOGLE_API_KEY` | Your Gemini API key | ✅ Yes |
35
+ | `EMBEDDING_MODEL` | `all-MiniLM-L6-v2` | No (has default) |
36
+ | `RELEVANCE_THRESHOLD` | `0.5` | No (has default) |
37
+ | `TOP_K_CHUNKS` | `3` | No (has default) |
38
+ | `LLM_MODEL` | `gemini-1.5-flash` | No (has default) |
39
+ | `LLM_TEMPERATURE` | `0.3` | No (has default) |
40
+
41
+ **Important**: Set these for all environments (Production, Preview, Development)
42
+
43
+ #### Step 3: Deploy
44
+
45
+ 1. Click "Deploy"
46
+ 2. Wait 2-3 minutes for build
47
+ 3. Visit your deployment URL
48
+
49
+ ### 3. Test Your Deployment
50
+
51
+ 1. Visit `https://your-app.vercel.app`
52
+ 2. Try these test questions:
53
+ - "How do I create a new project in TaskFlow?" (should answer)
54
+ - "What's the weather today?" (should escalate)
55
+ 3. Check that responses appear correctly
56
+
57
+ ## Architecture
58
+
59
+ ### Monorepo Structure
60
+
61
+ ```
62
+ smart-escalation-api/
63
+ ├── api/ # Vercel Serverless Functions (Backend)
64
+ │ └── ask.py # POST /api/ask endpoint
65
+ ├── frontend/ # React Frontend
66
+ │ ├── src/
67
+ │ └── dist/ # Built static files
68
+ ├── src/ # Shared Python modules
69
+ │ ├── rag.py
70
+ │ ├── escalation.py
71
+ │ ├── llm_client.py
72
+ │ └── config.py
73
+ ├── data/articles/ # Help articles knowledge base
74
+ ├── vercel.json # Vercel configuration
75
+ └── package.json # Monorepo configuration
76
+ ```
77
+
78
+ ### How It Works
79
+
80
+ 1. **Frontend**: React app built with Vite, served as static files
81
+ 2. **Backend**: Python serverless function at `/api/ask`
82
+ 3. **Same Origin**: Frontend calls `/api/ask` (no CORS needed)
83
+ 4. **Automatic HTTPS**: Vercel provides SSL certificates
84
+ 5. **Global CDN**: Fast access worldwide
85
+
86
+ ## Configuration
87
+
88
+ ### vercel.json
89
+
90
+ The `vercel.json` file configures:
91
+ - Python runtime for API functions
92
+ - Static build for frontend
93
+ - Routing rules (API vs frontend)
94
+
95
+ ### Environment Variables
96
+
97
+ Set in Vercel Dashboard → Settings → Environment Variables
98
+
99
+ **Required:**
100
+ - `GOOGLE_API_KEY`: Your Gemini API key
101
+
102
+ **Optional (with defaults):**
103
+ - `EMBEDDING_MODEL`: Sentence transformer model
104
+ - `RELEVANCE_THRESHOLD`: Escalation threshold (0-1)
105
+ - `TOP_K_CHUNKS`: Number of chunks to retrieve
106
+ - `LLM_MODEL`: Gemini model to use
107
+ - `LLM_TEMPERATURE`: LLM temperature (0-1)
108
+
109
+ ## Local Development
110
+
111
+ ### Backend Only
112
+
113
+ ```bash
114
+ # Install Python dependencies
115
+ pip install -r requirements.txt
116
+
117
+ # Run FastAPI server
118
+ uvicorn src.main:app --reload
119
+ ```
120
+
121
+ API available at: `http://localhost:8000`
122
+
123
+ ### Frontend Only
124
+
125
+ ```bash
126
+ # Install Node dependencies
127
+ cd frontend
128
+ npm install
129
+
130
+ # Set API URL for local backend
131
+ echo "VITE_API_URL=http://localhost:8000" > .env
132
+
133
+ # Run dev server
134
+ npm run dev
135
+ ```
136
+
137
+ Frontend available at: `http://localhost:5173`
138
+
139
+ ### Full Stack (Vercel Dev)
140
+
141
+ ```bash
142
+ # Install Vercel CLI
143
+ npm install -g vercel
144
+
145
+ # Run local Vercel environment
146
+ vercel dev
147
+ ```
148
+
149
+ This simulates the Vercel environment locally.
150
+
151
+ ## Troubleshooting
152
+
153
+ ### Build Fails
154
+
155
+ **Problem**: "Module not found" error
156
+
157
+ **Solution**:
158
+ - Ensure `requirements.txt` includes all dependencies
159
+ - Check Python version is 3.9+ in `vercel.json`
160
+ - Verify all files are committed to git
161
+
162
+ ### API Returns 500
163
+
164
+ **Problem**: Serverless function crashes
165
+
166
+ **Solution**:
167
+ - Check function logs in Vercel Dashboard
168
+ - Verify `GOOGLE_API_KEY` is set correctly
169
+ - Ensure `data/articles/` directory exists
170
+ - Check that help articles are valid markdown
171
+
172
+ ### Frontend Can't Reach API
173
+
174
+ **Problem**: "Failed to fetch" errors
175
+
176
+ **Solution**:
177
+ - Verify API endpoint: `https://your-app.vercel.app/api/ask`
178
+ - Check browser console for errors
179
+ - Test API directly with curl:
180
+ ```bash
181
+ curl -X POST https://your-app.vercel.app/api/ask \
182
+ -H "Content-Type: application/json" \
183
+ -d '{"question":"test"}'
184
+ ```
185
+
186
+ ### Slow Response Times
187
+
188
+ **Problem**: Requests take >10 seconds
189
+
190
+ **Solution**:
191
+ - First request loads embedding model (2-3s cold start)
192
+ - Subsequent requests are faster (cached)
193
+ - Use `gemini-1.5-flash` for faster responses
194
+ - Consider Vercel Pro for better performance
195
+
196
+ ### Out of Memory
197
+
198
+ **Problem**: Function crashes with memory error
199
+
200
+ **Solution**:
201
+ - Vercel free tier: 1GB memory limit
202
+ - Reduce `TOP_K_CHUNKS` to use less memory
203
+ - Use smaller embedding model
204
+ - Upgrade to Pro plan for 3GB memory
205
+
206
+ ## Monitoring
207
+
208
+ ### Vercel Dashboard
209
+
210
+ Monitor your deployment:
211
+ 1. Go to Vercel Dashboard → Your Project
212
+ 2. View real-time logs in Deployments tab
213
+ 3. Check function execution time and errors
214
+ 4. Monitor bandwidth usage
215
+
216
+ ### Enable Analytics
217
+
218
+ 1. Go to Analytics tab in Vercel Dashboard
219
+ 2. Enable Web Analytics (free)
220
+ 3. Enable Speed Insights for performance monitoring
221
+
222
+ ### Key Metrics
223
+
224
+ Watch for:
225
+ - Function execution time (should be <5s)
226
+ - Error rate (should be <5%)
227
+ - Cold start frequency
228
+ - Memory usage
229
+
230
+ ## Updating
231
+
232
+ ### Update Code
233
+
234
+ ```bash
235
+ # Make changes
236
+ git add .
237
+ git commit -m "Update feature"
238
+ git push
239
+
240
+ # Vercel automatically deploys
241
+ ```
242
+
243
+ ### Update Help Articles
244
+
245
+ ```bash
246
+ # Edit articles
247
+ vim data/articles/getting-started.md
248
+
249
+ # Commit and push
250
+ git add data/articles/
251
+ git commit -m "Update help articles"
252
+ git push
253
+
254
+ # Vercel redeploys, vector store rebuilds
255
+ ```
256
+
257
+ ### Update Dependencies
258
+
259
+ **Python:**
260
+ ```bash
261
+ # Update requirements.txt
262
+ pip install new-package
263
+ pip freeze > requirements.txt
264
+
265
+ # Commit and push
266
+ git add requirements.txt
267
+ git commit -m "Add new dependency"
268
+ git push
269
+ ```
270
+
271
+ **Node:**
272
+ ```bash
273
+ cd frontend
274
+ npm install new-package
275
+ git add package.json package-lock.json
276
+ git commit -m "Add new dependency"
277
+ git push
278
+ ```
279
+
280
+ ## Cost Estimates
281
+
282
+ ### Vercel Free Tier
283
+
284
+ - ✅ 100GB bandwidth/month
285
+ - ✅ 100 hours function execution/month
286
+ - ✅ 6,000 function invocations/day
287
+ - ✅ Unlimited deployments
288
+ - ✅ Automatic HTTPS
289
+ - ✅ Global CDN
290
+
291
+ ### Expected Usage
292
+
293
+ For low-medium traffic (100-500 requests/day):
294
+ - Function time: ~1-2s per request
295
+ - Daily usage: 100-1000s = 0.03-0.3 hours/day
296
+ - Monthly usage: 1-9 hours/month
297
+ - **Well within free tier limits**
298
+
299
+ ### When to Upgrade
300
+
301
+ Upgrade to Pro ($20/month) if you need:
302
+ - >100 hours function execution
303
+ - >6,000 requests/day
304
+ - 60s timeout (vs 10s)
305
+ - 3GB memory (vs 1GB)
306
+ - Priority support
307
+
308
+ ### Additional Costs
309
+
310
+ - **Google Gemini API**: Free tier (15 req/min, 1500 req/day)
311
+ - **Total**: $0-20/month depending on traffic
312
+
313
+ ## Security
314
+
315
+ ### Best Practices
316
+
317
+ 1. ✅ Never commit API keys to git
318
+ 2. ✅ Use Vercel environment variables for secrets
319
+ 3. ✅ Rotate API keys periodically
320
+ 4. ✅ Monitor logs for suspicious activity
321
+ 5. ✅ Keep dependencies updated
322
+
323
+ ### Automatic Security
324
+
325
+ Vercel provides:
326
+ - ✅ Automatic HTTPS/SSL
327
+ - ✅ DDoS protection
328
+ - ✅ Secure environment variables
329
+ - ✅ Isolated function execution
330
+
331
+ ## Support
332
+
333
+ ### Resources
334
+
335
+ - [Vercel Documentation](https://vercel.com/docs)
336
+ - [Vercel Python Runtime](https://vercel.com/docs/functions/serverless-functions/runtimes/python)
337
+ - [Vercel Community](https://github.com/vercel/vercel/discussions)
338
+
339
+ ### Getting Help
340
+
341
+ 1. Check [README.md](README.md) for detailed documentation
342
+ 2. Review [Troubleshooting](#troubleshooting) section above
343
+ 3. Check Vercel function logs for errors
344
+ 4. Open GitHub issue for bugs
345
+ 5. Contact Vercel support (Pro plan)
346
+
347
+ ## Next Steps
348
+
349
+ After deployment:
350
+
351
+ 1. ✅ Test with various questions
352
+ 2. ✅ Monitor performance in Vercel Dashboard
353
+ 3. ✅ Enable Analytics for insights
354
+ 4. ✅ Set up custom domain (optional)
355
+ 5. ✅ Configure alerts for errors
356
+ 6. ✅ Share with users!
357
+
358
+ ---
359
+
360
+ **Need help?** Open an issue on GitHub or check the [README.md](README.md) for more details.
README.md CHANGED
@@ -2,6 +2,17 @@
2
 
3
  An AI-powered L1 customer support system that intelligently answers customer questions about TaskFlow (a fictional project management SaaS) or escalates to human agents when uncertain. The system uses RAG-based retrieval over help articles to ground responses.
4
 
 
 
 
 
 
 
 
 
 
 
 
5
  ## Features
6
 
7
  - **RAG-Based Retrieval**: Uses sentence-transformers and FAISS for semantic search over help articles
@@ -12,10 +23,11 @@ An AI-powered L1 customer support system that intelligently answers customer que
12
 
13
  ## Architecture
14
 
15
- - **Backend**: FastAPI + FAISS (in-memory vector store) + sentence-transformers
 
16
  - **LLM**: Google Gemini API (gemini-1.5-flash)
17
  - **Frontend**: React + Vite + TypeScript
18
- - **Deployment**: Render/Railway (backend) + Vercel (frontend)
19
 
20
  ## Quick Start
21
 
@@ -165,40 +177,565 @@ The system escalates to human agents when:
165
 
166
  ## Deployment
167
 
168
- ### Backend Deployment (Render/Railway)
169
-
170
- #### Render
171
-
172
- 1. Create a new Web Service on [Render](https://render.com)
173
- 2. Connect your GitHub repository
174
- 3. Configure build settings:
175
- - **Build Command**: `pip install -r requirements.txt`
176
- - **Start Command**: `uvicorn src.main:app --host 0.0.0.0 --port $PORT`
177
- 4. Add environment variables:
178
- - `GOOGLE_API_KEY`
179
- - Other configuration variables as needed
180
- 5. Deploy
181
-
182
- #### Railway
183
-
184
- 1. Create a new project on [Railway](https://railway.app)
185
- 2. Connect your GitHub repository
186
- 3. Add environment variables in the Variables tab
187
- 4. Railway will auto-detect Python and deploy
188
- 5. Set start command if needed: `uvicorn src.main:app --host 0.0.0.0 --port $PORT`
189
-
190
- ### Frontend Deployment (Vercel)
191
-
192
- 1. Create a new project on [Vercel](https://vercel.com)
193
- 2. Connect your GitHub repository
194
- 3. Configure build settings:
195
- - **Framework Preset**: Vite
196
- - **Root Directory**: `frontend`
197
- - **Build Command**: `npm run build`
198
- - **Output Directory**: `dist`
199
- 4. Add environment variable:
200
- - `VITE_API_URL`: Your backend URL (e.g., `https://your-app.onrender.com`)
201
- 5. Deploy
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
202
 
203
  ## Testing
204
 
 
2
 
3
  An AI-powered L1 customer support system that intelligently answers customer questions about TaskFlow (a fictional project management SaaS) or escalates to human agents when uncertain. The system uses RAG-based retrieval over help articles to ground responses.
4
 
5
+ ## 🚀 Quick Deploy to Vercel
6
+
7
+ [![Deploy with Vercel](https://vercel.com/button)](https://vercel.com/new)
8
+
9
+ 1. Click the button above or go to [vercel.com](https://vercel.com)
10
+ 2. Import this repository
11
+ 3. Add environment variable: `GOOGLE_API_KEY` (get from [Google AI Studio](https://makersuite.google.com/app/apikey))
12
+ 4. Deploy!
13
+
14
+ **See [DEPLOYMENT.md](DEPLOYMENT.md) for detailed deployment instructions.**
15
+
16
  ## Features
17
 
18
  - **RAG-Based Retrieval**: Uses sentence-transformers and FAISS for semantic search over help articles
 
23
 
24
  ## Architecture
25
 
26
+ - **Backend**: Python serverless functions on Vercel
27
+ - **Vector Store**: FAISS (in-memory) with sentence-transformers embeddings
28
  - **LLM**: Google Gemini API (gemini-1.5-flash)
29
  - **Frontend**: React + Vite + TypeScript
30
+ - **Deployment**: Vercel (monorepo with frontend + backend)
31
 
32
  ## Quick Start
33
 
 
177
 
178
  ## Deployment
179
 
180
+ This section provides step-by-step instructions for deploying the entire Smart Escalation API (both backend and frontend) to Vercel.
181
+
182
+ ### Why Vercel for Everything?
183
+
184
+ Vercel supports both:
185
+ - **Frontend**: Static React apps (traditional use case)
186
+ - **Backend**: Python serverless functions via API routes
187
+
188
+ This means you can deploy the entire application to a single platform with:
189
+ - ✅ Single deployment workflow
190
+ - ✅ Unified environment variables
191
+ - ✅ Automatic HTTPS
192
+ - ✅ Global CDN
193
+ - ✅ Free tier for both frontend and backend
194
+
195
+ ### Prerequisites
196
+
197
+ - GitHub account with your repository
198
+ - Google Gemini API key
199
+ - Vercel account (free tier available at [vercel.com](https://vercel.com))
200
+
201
+ ### Project Structure for Vercel
202
+
203
+ First, we need to restructure the project to work with Vercel's serverless functions:
204
+
205
+ ```
206
+ smart-escalation-api/
207
+ ├── api/ # Backend API (Vercel Serverless Functions)
208
+ │ └── ask.py # POST /api/ask endpoint
209
+ ├── frontend/ # React frontend
210
+ │ ├── src/
211
+ │ ├── package.json
212
+ │ └── vite.config.ts
213
+ ├── src/ # Shared Python modules
214
+ │ ├── rag.py
215
+ │ ├── escalation.py
216
+ │ ├── llm_client.py
217
+ │ ├── chunking.py
218
+ │ └── config.py
219
+ ├── data/
220
+ │ └── articles/ # Help articles
221
+ ├── requirements.txt # Python dependencies
222
+ ├── vercel.json # Vercel configuration
223
+ └── package.json # Root package.json for monorepo
224
+ ```
225
+
226
+ ### Step 1: Create Vercel Configuration
227
+
228
+ Create `vercel.json` in the project root:
229
+
230
+ ```json
231
+ {
232
+ "version": 2,
233
+ "builds": [
234
+ {
235
+ "src": "api/**/*.py",
236
+ "use": "@vercel/python"
237
+ },
238
+ {
239
+ "src": "frontend/package.json",
240
+ "use": "@vercel/static-build",
241
+ "config": {
242
+ "distDir": "frontend/dist"
243
+ }
244
+ }
245
+ ],
246
+ "routes": [
247
+ {
248
+ "src": "/api/(.*)",
249
+ "dest": "/api/$1"
250
+ },
251
+ {
252
+ "src": "/(.*)",
253
+ "dest": "/frontend/$1"
254
+ }
255
+ ],
256
+ "env": {
257
+ "PYTHON_VERSION": "3.9"
258
+ }
259
+ }
260
+ ```
261
+
262
+ ### Step 2: Create API Endpoint
263
+
264
+ Create `api/ask.py` to handle the `/api/ask` endpoint:
265
+
266
+ ```python
267
+ from fastapi import FastAPI, Request
268
+ from fastapi.responses import JSONResponse
269
+ from pydantic import BaseModel
270
+ import sys
271
+ import os
272
+
273
+ # Add parent directory to path to import src modules
274
+ sys.path.append(os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
275
+
276
+ from src.rag import RAGSystem
277
+ from src.escalation import EscalationEngine
278
+ from src.llm_client import LLMClient
279
+ from src.config import Settings
280
+
281
+ # Initialize components (cached across invocations)
282
+ settings = Settings()
283
+ rag_system = RAGSystem(articles_dir="data/articles")
284
+ llm_client = LLMClient(api_key=settings.google_api_key)
285
+ escalation_engine = EscalationEngine(llm_client=llm_client)
286
+
287
+ app = FastAPI()
288
+
289
+ class QuestionRequest(BaseModel):
290
+ question: str
291
+
292
+ @app.post("/api/ask")
293
+ async def ask_question(request: QuestionRequest):
294
+ """Process customer question and return answer or escalation."""
295
+ try:
296
+ # Retrieve relevant chunks
297
+ chunks = rag_system.retrieve(request.question, top_k=settings.top_k_chunks)
298
+
299
+ # Decide whether to answer or escalate
300
+ decision = escalation_engine.process_question(request.question, chunks)
301
+
302
+ return JSONResponse(content={
303
+ "response_type": decision.action,
304
+ "message": decision.message,
305
+ "confidence_explanation": decision.confidence_explanation,
306
+ "sources": decision.sources
307
+ })
308
+ except Exception as e:
309
+ return JSONResponse(
310
+ status_code=500,
311
+ content={
312
+ "response_type": "escalation",
313
+ "message": "I'm experiencing technical difficulties. Let me connect you with a human agent.",
314
+ "confidence_explanation": f"Error: {str(e)}",
315
+ "sources": None
316
+ }
317
+ )
318
+
319
+ # Vercel serverless function handler
320
+ async def handler(request: Request):
321
+ return await app(request)
322
+ ```
323
+
324
+ ### Step 3: Update Frontend API URL
325
+
326
+ Update `frontend/src/services/api.ts` to use relative API path:
327
+
328
+ ```typescript
329
+ const API_URL = import.meta.env.VITE_API_URL || '/api';
330
+
331
+ export async function askQuestion(question: string) {
332
+ const response = await fetch(`${API_URL}/ask`, {
333
+ method: 'POST',
334
+ headers: {
335
+ 'Content-Type': 'application/json',
336
+ },
337
+ body: JSON.stringify({ question }),
338
+ });
339
+
340
+ if (!response.ok) {
341
+ throw new Error('Failed to get response');
342
+ }
343
+
344
+ return response.json();
345
+ }
346
+ ```
347
+
348
+ ### Step 4: Deploy to Vercel
349
+
350
+ #### Option A: Deploy via Vercel Dashboard (Recommended)
351
+
352
+ 1. **Create a Vercel Account**
353
+ - Go to [vercel.com](https://vercel.com)
354
+ - Sign up with your GitHub account
355
+
356
+ 2. **Import Project**
357
+ - Click "Add New..." → "Project"
358
+ - Import your GitHub repository
359
+ - Vercel will auto-detect the configuration
360
+
361
+ 3. **Configure Environment Variables**
362
+
363
+ In the "Environment Variables" section, add:
364
+
365
+ | Variable | Value | Notes |
366
+ |----------|-------|-------|
367
+ | `GOOGLE_API_KEY` | `AIza...` | **Required** - Your Gemini API key |
368
+ | `EMBEDDING_MODEL` | `all-MiniLM-L6-v2` | Optional - defaults to this |
369
+ | `RELEVANCE_THRESHOLD` | `0.5` | Optional - adjust escalation sensitivity |
370
+ | `TOP_K_CHUNKS` | `3` | Optional - number of chunks to retrieve |
371
+ | `LLM_MODEL` | `gemini-1.5-flash` | Optional - use flash for speed |
372
+ | `LLM_TEMPERATURE` | `0.3` | Optional - lower = more deterministic |
373
+
374
+ **Important**: Set these for all environments (Production, Preview, Development)
375
+
376
+ 4. **Deploy**
377
+ - Click "Deploy"
378
+ - Wait 2-3 minutes for build and deployment
379
+ - Monitor build logs for any errors
380
+
381
+ 5. **Verify Deployment**
382
+ - Visit your Vercel URL (e.g., `https://smart-escalation-api.vercel.app`)
383
+ - Test the chat interface
384
+ - Submit a test question
385
+ - Check browser console for any errors
386
+
387
+ #### Option B: Deploy via Vercel CLI
388
+
389
+ 1. **Install Vercel CLI**
390
+ ```bash
391
+ npm install -g vercel
392
+ ```
393
+
394
+ 2. **Login to Vercel**
395
+ ```bash
396
+ vercel login
397
+ ```
398
+
399
+ 3. **Deploy**
400
+ ```bash
401
+ vercel
402
+ ```
403
+
404
+ Follow the prompts:
405
+ - Set up and deploy? **Y**
406
+ - Which scope? Select your account
407
+ - Link to existing project? **N**
408
+ - Project name? `smart-escalation-api`
409
+ - In which directory is your code located? `./`
410
+
411
+ 4. **Add Environment Variables**
412
+ ```bash
413
+ vercel env add GOOGLE_API_KEY
414
+ vercel env add EMBEDDING_MODEL
415
+ vercel env add RELEVANCE_THRESHOLD
416
+ # ... add other variables
417
+ ```
418
+
419
+ 5. **Deploy to Production**
420
+ ```bash
421
+ vercel --prod
422
+ ```
423
+
424
+ ### Step 5: Configure Custom Domain (Optional)
425
+
426
+ 1. Go to your project in Vercel Dashboard
427
+ 2. Navigate to Settings → Domains
428
+ 3. Add your custom domain (e.g., `support.yourdomain.com`)
429
+ 4. Follow DNS configuration instructions:
430
+ - Add CNAME record pointing to `cname.vercel-dns.com`
431
+ - Or add A record pointing to Vercel's IP
432
+ 5. Wait for DNS propagation (5-30 minutes)
433
+ 6. Vercel automatically provisions SSL certificate
434
+
435
+ ### Deployment Troubleshooting
436
+
437
+ #### Vercel Serverless Function Issues
438
+
439
+ **Problem: "Module not found" in API functions**
440
+ - **Solution**: Ensure `requirements.txt` is in project root
441
+ - Verify Python version is 3.9+ in `vercel.json`
442
+ - Check that `src/` modules are importable from `api/` directory
443
+ - Add `sys.path.append()` in `api/ask.py` if needed
444
+
445
+ **Problem: API function times out (10 second limit)**
446
+ - **Solution**: Vercel serverless functions have 10s timeout on free tier
447
+ - Optimize embedding model loading (cache between invocations)
448
+ - Consider upgrading to Pro plan for 60s timeout
449
+ - Use lighter embedding model if needed
450
+
451
+ **Problem: "Function size exceeds limit"**
452
+ - **Solution**: Vercel has 250MB limit for serverless functions
453
+ - Reduce dependencies in `requirements.txt`
454
+ - Use `.vercelignore` to exclude unnecessary files:
455
+ ```
456
+ tests/
457
+ .pytest_cache/
458
+ .hypothesis/
459
+ *.pyc
460
+ __pycache__/
461
+ ```
462
+
463
+ **Problem: Cold start takes too long**
464
+ - **Solution**: First request after inactivity loads embedding model (~2-3s)
465
+ - This is normal for serverless architecture
466
+ - Subsequent requests are fast (cached)
467
+ - Consider keeping function warm with periodic pings
468
+
469
+ **Problem: Environment variables not accessible**
470
+ - **Solution**: Ensure variables are set in Vercel dashboard
471
+ - Redeploy after adding new environment variables
472
+ - Check variable names match exactly (case-sensitive)
473
+ - Verify variables are set for correct environment (Production/Preview)
474
+
475
+ #### Frontend Issues
476
+
477
+ **Problem: Build fails with "Cannot find module"**
478
+ - **Solution**: Ensure `frontend/package.json` includes all dependencies
479
+ - Verify Node.js version (requires 18+)
480
+ - Run `npm install` locally to test
481
+ - Check `frontend/` is set as root directory in build settings
482
+
483
+ **Problem: Frontend loads but API calls fail with 404**
484
+ - **Solution**: Verify API routes are configured in `vercel.json`
485
+ - Check that `api/ask.py` exists and is properly structured
486
+ - Test API endpoint directly: `https://your-app.vercel.app/api/ask`
487
+ - Ensure frontend is calling `/api/ask` (not `/ask`)
488
+
489
+ **Problem: CORS errors in browser console**
490
+ - **Solution**: With Vercel monorepo, CORS shouldn't be needed (same origin)
491
+ - If using separate deployments, add CORS headers to API function
492
+ - Verify frontend and API are on same domain
493
+
494
+ **Problem: "Failed to fetch" errors**
495
+ - **Solution**: Check browser console for detailed error
496
+ - Verify API endpoint is accessible
497
+ - Test with curl: `curl -X POST https://your-app.vercel.app/api/ask -H "Content-Type: application/json" -d '{"question":"test"}'`
498
+ - Check network tab in browser dev tools
499
+
500
+ #### Build and Deployment Issues
501
+
502
+ **Problem: Build fails with Python errors**
503
+ - **Solution**: Check build logs in Vercel dashboard
504
+ - Verify all Python dependencies are in `requirements.txt`
505
+ - Ensure Python version compatibility (3.9+)
506
+ - Test locally: `pip install -r requirements.txt`
507
+
508
+ **Problem: Build succeeds but deployment fails**
509
+ - **Solution**: Check deployment logs for runtime errors
510
+ - Verify `vercel.json` configuration is correct
511
+ - Ensure all required files are committed to git
512
+ - Check that `data/articles/` directory exists with help articles
513
+
514
+ **Problem: Changes not reflecting after deployment**
515
+ - **Solution**: Clear browser cache (Ctrl+Shift+R or Cmd+Shift+R)
516
+ - Check that git changes are committed and pushed
517
+ - Verify Vercel deployed the correct branch
518
+ - Check deployment logs to confirm new version deployed
519
+
520
+ **Problem: Preview deployments work but production fails**
521
+ - **Solution**: Check environment variables are set for Production
522
+ - Verify production branch is correct (usually `main`)
523
+ - Compare environment variables between Preview and Production
524
+ - Check for hardcoded URLs or environment-specific code
525
+
526
+ #### Performance Issues
527
+
528
+ **Problem: Slow response times (>10 seconds)**
529
+ - **Solution**: Vercel free tier has 10s timeout for serverless functions
530
+ - Check LLM API latency in function logs
531
+ - Use `gemini-1.5-flash` instead of `gemini-1.5-pro`
532
+ - Optimize embedding model loading
533
+ - Consider upgrading to Pro plan for better performance
534
+
535
+ **Problem: Out of memory errors**
536
+ - **Solution**: Vercel serverless functions have 1GB memory limit (free tier)
537
+ - Reduce `TOP_K_CHUNKS` to decrease memory usage
538
+ - Use smaller embedding model
539
+ - Optimize chunk storage and retrieval
540
+ - Upgrade to Pro plan for 3GB memory
541
+
542
+ **Problem: Rate limiting from Vercel**
543
+ - **Solution**: Free tier has limits on function invocations
544
+ - Monitor usage in Vercel dashboard
545
+ - Implement client-side rate limiting
546
+ - Upgrade to Pro plan for higher limits
547
+
548
+ #### API Key and Authentication Issues
549
+
550
+ **Problem: "Invalid API key" errors**
551
+ - **Solution**: Verify `GOOGLE_API_KEY` is set correctly in Vercel
552
+ - Check API key has proper permissions
553
+ - Ensure quota is not exceeded
554
+ - Test API key with direct Gemini API call
555
+
556
+ **Problem: API key works locally but not on Vercel**
557
+ - **Solution**: Ensure environment variable is set in Vercel dashboard
558
+ - Redeploy after adding environment variables
559
+ - Check variable name matches exactly (case-sensitive)
560
+ - Verify no extra spaces or quotes in variable value
561
+
562
+ #### Data and File Issues
563
+
564
+ **Problem: Help articles not found**
565
+ - **Solution**: Ensure `data/articles/` directory is committed to git
566
+ - Verify file paths are correct (case-sensitive)
567
+ - Check that markdown files exist and are readable
568
+ - Test locally first: `ls data/articles/`
569
+
570
+ **Problem: Vector store fails to build**
571
+ - **Solution**: Check that all help articles are valid markdown
572
+ - Verify embedding model downloads successfully
573
+ - Check function logs for specific error messages
574
+ - Ensure sufficient memory for vector store creation
575
+
576
+ ### Monitoring and Maintenance
577
+
578
+ #### Vercel Analytics
579
+
580
+ Enable Vercel Analytics to monitor:
581
+ - Page views and user traffic
582
+ - API function invocations
583
+ - Response times and errors
584
+ - Bandwidth usage
585
+
586
+ **To enable:**
587
+ 1. Go to your project in Vercel Dashboard
588
+ 2. Navigate to Analytics tab
589
+ 3. Enable Web Analytics (free)
590
+ 4. Enable Speed Insights for performance monitoring
591
+
592
+ #### Function Logs
593
+
594
+ Monitor serverless function logs:
595
+ 1. Go to Vercel Dashboard → Your Project
596
+ 2. Click on Deployments → Select deployment
597
+ 3. View Function Logs for real-time monitoring
598
+ 4. Check for errors, warnings, and performance metrics
599
+
600
+ **Key metrics to watch:**
601
+ - Function execution time (should be <5s)
602
+ - Error rate (should be <5%)
603
+ - Cold start frequency
604
+ - Memory usage
605
+
606
+ #### Updating the Application
607
+
608
+ **To update help articles:**
609
+ 1. Edit markdown files in `data/articles/`
610
+ 2. Commit and push to GitHub
611
+ 3. Vercel automatically redeploys
612
+ 4. Vector store rebuilds on next function invocation
613
+
614
+ **To update code:**
615
+ 1. Make changes to Python or React code
616
+ 2. Test locally first
617
+ 3. Commit and push to GitHub
618
+ 4. Vercel automatically builds and deploys
619
+ 5. Monitor deployment logs for errors
620
+
621
+ **To update dependencies:**
622
+ 1. Update `requirements.txt` (Python) or `package.json` (Node)
623
+ 2. Test locally: `pip install -r requirements.txt` or `npm install`
624
+ 3. Commit and push
625
+ 4. Vercel rebuilds with new dependencies
626
+
627
+ ### Cost Estimates
628
+
629
+ **Vercel Free Tier Limits:**
630
+ - 100GB bandwidth per month
631
+ - 100 hours serverless function execution per month
632
+ - 6,000 function invocations per day
633
+ - 1GB function memory
634
+ - 10s function timeout
635
+ - Unlimited deployments
636
+
637
+ **Expected Usage for Low-Medium Traffic:**
638
+ - ~1-2s per API request
639
+ - ~100-500 requests/day = 100-500s function time/day
640
+ - ~3,000-15,000s function time/month = 0.8-4.2 hours/month
641
+ - Well within free tier limits
642
+
643
+ **When to Upgrade to Pro ($20/month):**
644
+ - Need >100 hours function execution
645
+ - Need >6,000 requests/day
646
+ - Need 60s timeout (vs 10s)
647
+ - Need 3GB memory (vs 1GB)
648
+ - Need priority support
649
+
650
+ **Additional Costs:**
651
+ - **Google Gemini API**: $0-10/month (free tier: 15 req/min, 1500 req/day)
652
+ - **Total**: $0-30/month depending on traffic
653
+
654
+ ### Security Best Practices
655
+
656
+ 1. **Environment Variables**
657
+ - Never commit API keys to git
658
+ - Use Vercel environment variables for all secrets
659
+ - Rotate API keys periodically
660
+
661
+ 2. **CORS Configuration**
662
+ - With Vercel monorepo, CORS is handled automatically (same origin)
663
+ - If using separate deployments, restrict CORS to specific domains
664
+
665
+ 3. **Input Validation**
666
+ - Already handled by Pydantic models
667
+ - Limit question length (max 500 characters)
668
+ - Sanitize user input
669
+
670
+ 4. **Rate Limiting**
671
+ - Implement client-side throttling
672
+ - Consider adding server-side rate limiting for production
673
+ - Monitor for abuse in Vercel logs
674
+
675
+ 5. **HTTPS**
676
+ - Vercel provides automatic HTTPS
677
+ - All traffic is encrypted by default
678
+ - SSL certificates auto-renewed
679
+
680
+ 6. **Monitoring**
681
+ - Enable Vercel Analytics
682
+ - Monitor function logs regularly
683
+ - Set up alerts for errors or unusual traffic
684
+
685
+ ### Advanced Configuration
686
+
687
+ #### Custom Build Configuration
688
+
689
+ Create `package.json` in project root for monorepo setup:
690
+
691
+ ```json
692
+ {
693
+ "name": "smart-escalation-api",
694
+ "version": "1.0.0",
695
+ "scripts": {
696
+ "build": "cd frontend && npm install && npm run build"
697
+ },
698
+ "workspaces": [
699
+ "frontend"
700
+ ]
701
+ }
702
+ ```
703
+
704
+ #### Optimizing Cold Starts
705
+
706
+ Add warming function to keep serverless functions active:
707
+
708
+ ```python
709
+ # api/health.py
710
+ def handler(request):
711
+ return {
712
+ "statusCode": 200,
713
+ "body": {"status": "healthy"}
714
+ }
715
+ ```
716
+
717
+ Set up external monitoring (e.g., UptimeRobot) to ping `/api/health` every 5 minutes.
718
+
719
+ #### Environment-Specific Configuration
720
+
721
+ Use Vercel's environment system:
722
+ - **Production**: Live environment with production API keys
723
+ - **Preview**: Staging environment for testing (auto-created for PRs)
724
+ - **Development**: Local development environment
725
+
726
+ Set different values for each environment in Vercel dashboard.
727
+
728
+ ### Getting Help
729
+
730
+ **Vercel Support:**
731
+ - Documentation: [vercel.com/docs](https://vercel.com/docs)
732
+ - Community: [github.com/vercel/vercel/discussions](https://github.com/vercel/vercel/discussions)
733
+ - Support: Available for Pro plan users
734
+
735
+ **Common Resources:**
736
+ - [Vercel Python Runtime](https://vercel.com/docs/functions/serverless-functions/runtimes/python)
737
+ - [Vercel Environment Variables](https://vercel.com/docs/projects/environment-variables)
738
+ - [Vercel Monorepo Guide](https://vercel.com/docs/monorepos)
739
 
740
  ## Testing
741
 
VERCEL_SETUP.md ADDED
@@ -0,0 +1,220 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # Vercel Setup - Quick Reference
2
+
3
+ ## 📋 Pre-Deployment Checklist
4
+
5
+ - [ ] GitHub repository with code
6
+ - [ ] Google Gemini API key ([Get one](https://makersuite.google.com/app/apikey))
7
+ - [ ] Vercel account ([Sign up free](https://vercel.com/signup))
8
+
9
+ ## 🚀 Deployment Steps
10
+
11
+ ### 1. Import Project to Vercel
12
+
13
+ ```
14
+ 1. Go to vercel.com
15
+ 2. Click "Add New..." → "Project"
16
+ 3. Select "Import Git Repository"
17
+ 4. Choose your GitHub repository
18
+ 5. Vercel auto-detects configuration from vercel.json
19
+ ```
20
+
21
+ ### 2. Configure Environment Variables
22
+
23
+ Add these in Vercel Dashboard → Settings → Environment Variables:
24
+
25
+ #### Required
26
+ ```
27
+ GOOGLE_API_KEY=AIza...your-key-here
28
+ ```
29
+
30
+ #### Optional (with defaults)
31
+ ```
32
+ EMBEDDING_MODEL=all-MiniLM-L6-v2
33
+ RELEVANCE_THRESHOLD=0.5
34
+ TOP_K_CHUNKS=3
35
+ LLM_MODEL=gemini-1.5-flash
36
+ LLM_TEMPERATURE=0.3
37
+ ```
38
+
39
+ **Important**: Set for all environments (Production, Preview, Development)
40
+
41
+ ### 3. Deploy
42
+
43
+ ```
44
+ 1. Click "Deploy"
45
+ 2. Wait 2-3 minutes
46
+ 3. Visit your URL: https://your-app.vercel.app
47
+ ```
48
+
49
+ ## ✅ Post-Deployment Testing
50
+
51
+ ### Test Questions
52
+
53
+ **Should Answer:**
54
+ ```
55
+ "How do I create a new project in TaskFlow?"
56
+ "How do I add a team member?"
57
+ "What are the subscription plans?"
58
+ ```
59
+
60
+ **Should Escalate:**
61
+ ```
62
+ "What's the weather today?"
63
+ "Tell me a joke"
64
+ "How do I cook pasta?"
65
+ ```
66
+
67
+ ### Verify
68
+
69
+ - [ ] Frontend loads correctly
70
+ - [ ] Can submit questions
71
+ - [ ] Answers appear in chat
72
+ - [ ] Confidence explanations show
73
+ - [ ] Escalations work for out-of-scope questions
74
+
75
+ ## 🔧 Common Issues
76
+
77
+ ### Build Fails
78
+
79
+ **Error**: "Module not found"
80
+ ```bash
81
+ # Solution: Ensure requirements.txt includes all dependencies
82
+ pip freeze > requirements.txt
83
+ git add requirements.txt
84
+ git commit -m "Update dependencies"
85
+ git push
86
+ ```
87
+
88
+ ### API Returns 500
89
+
90
+ **Error**: Function crashes
91
+ ```bash
92
+ # Solution: Check Vercel function logs
93
+ 1. Go to Vercel Dashboard → Deployments
94
+ 2. Click on latest deployment
95
+ 3. View Function Logs
96
+ 4. Look for Python errors
97
+ ```
98
+
99
+ ### Frontend Can't Reach API
100
+
101
+ **Error**: "Failed to fetch"
102
+ ```bash
103
+ # Solution: Test API directly
104
+ curl -X POST https://your-app.vercel.app/api/ask \
105
+ -H "Content-Type: application/json" \
106
+ -d '{"question":"test"}'
107
+
108
+ # Should return JSON response
109
+ ```
110
+
111
+ ### Slow Responses
112
+
113
+ **Issue**: First request takes 5-10 seconds
114
+ ```
115
+ This is normal! First request loads embedding model.
116
+ Subsequent requests are fast (1-3 seconds).
117
+ ```
118
+
119
+ ## 📊 Monitoring
120
+
121
+ ### View Logs
122
+
123
+ ```
124
+ Vercel Dashboard → Your Project → Deployments → Function Logs
125
+ ```
126
+
127
+ ### Key Metrics
128
+
129
+ - Function execution time: Should be <5s
130
+ - Error rate: Should be <5%
131
+ - Memory usage: Should be <1GB
132
+
133
+ ### Enable Analytics
134
+
135
+ ```
136
+ Vercel Dashboard → Analytics → Enable Web Analytics
137
+ ```
138
+
139
+ ## 🔄 Updating
140
+
141
+ ### Update Code
142
+
143
+ ```bash
144
+ git add .
145
+ git commit -m "Update feature"
146
+ git push
147
+ # Vercel auto-deploys
148
+ ```
149
+
150
+ ### Update Help Articles
151
+
152
+ ```bash
153
+ vim data/articles/getting-started.md
154
+ git add data/articles/
155
+ git commit -m "Update help articles"
156
+ git push
157
+ # Vector store rebuilds automatically
158
+ ```
159
+
160
+ ### Update Environment Variables
161
+
162
+ ```
163
+ 1. Vercel Dashboard → Settings → Environment Variables
164
+ 2. Edit variable
165
+ 3. Redeploy (automatic or manual)
166
+ ```
167
+
168
+ ## 💰 Cost Tracking
169
+
170
+ ### Free Tier Limits
171
+
172
+ - 100GB bandwidth/month
173
+ - 100 hours function execution/month
174
+ - 6,000 function invocations/day
175
+
176
+ ### Monitor Usage
177
+
178
+ ```
179
+ Vercel Dashboard → Usage
180
+ ```
181
+
182
+ ### Expected Usage (100-500 requests/day)
183
+
184
+ - Function time: ~1-2s per request
185
+ - Daily: 100-1000s = 0.03-0.3 hours
186
+ - Monthly: 1-9 hours
187
+ - **Well within free tier**
188
+
189
+ ## 🆘 Getting Help
190
+
191
+ ### Resources
192
+
193
+ - [Full Deployment Guide](DEPLOYMENT.md)
194
+ - [README](README.md)
195
+ - [Vercel Docs](https://vercel.com/docs)
196
+ - [Vercel Community](https://github.com/vercel/vercel/discussions)
197
+
198
+ ### Support Channels
199
+
200
+ 1. Check function logs in Vercel Dashboard
201
+ 2. Review [DEPLOYMENT.md](DEPLOYMENT.md) troubleshooting section
202
+ 3. Open GitHub issue
203
+ 4. Vercel support (Pro plan)
204
+
205
+ ## 🎯 Next Steps
206
+
207
+ After successful deployment:
208
+
209
+ - [ ] Test with various questions
210
+ - [ ] Enable Vercel Analytics
211
+ - [ ] Set up custom domain (optional)
212
+ - [ ] Monitor performance
213
+ - [ ] Share with users!
214
+
215
+ ---
216
+
217
+ **Quick Links:**
218
+ - [Vercel Dashboard](https://vercel.com/dashboard)
219
+ - [Google AI Studio](https://makersuite.google.com/app/apikey)
220
+ - [Full Documentation](DEPLOYMENT.md)
api/ask.py ADDED
@@ -0,0 +1,128 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """
2
+ Vercel Serverless Function for handling customer support questions.
3
+ Endpoint: POST /api/ask
4
+ """
5
+
6
+ from fastapi import FastAPI, Request
7
+ from fastapi.responses import JSONResponse
8
+ from fastapi.middleware.cors import CORSMiddleware
9
+ from pydantic import BaseModel
10
+ import sys
11
+ import os
12
+
13
+ # Add parent directory to path to import src modules
14
+ sys.path.append(os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
15
+
16
+ from src.rag import RAGSystem
17
+ from src.escalation import EscalationEngine
18
+ from src.llm_client import LLMClient
19
+ from src.config import Settings
20
+
21
+ # Initialize components (cached across invocations)
22
+ settings = Settings()
23
+ rag_system = None
24
+ llm_client = None
25
+ escalation_engine = None
26
+
27
+ def initialize_components():
28
+ """Initialize RAG system, LLM client, and escalation engine."""
29
+ global rag_system, llm_client, escalation_engine
30
+
31
+ if rag_system is None:
32
+ articles_dir = os.path.join(
33
+ os.path.dirname(os.path.dirname(os.path.abspath(__file__))),
34
+ "data",
35
+ "articles"
36
+ )
37
+ rag_system = RAGSystem(articles_dir=articles_dir)
38
+ llm_client = LLMClient(api_key=settings.google_api_key, model=settings.llm_model)
39
+ escalation_engine = EscalationEngine(
40
+ llm_client=llm_client,
41
+ relevance_threshold=settings.relevance_threshold
42
+ )
43
+
44
+ app = FastAPI(title="Smart Escalation API")
45
+
46
+ # Configure CORS
47
+ app.add_middleware(
48
+ CORSMiddleware,
49
+ allow_origins=["*"], # In production, restrict to your domain
50
+ allow_credentials=True,
51
+ allow_methods=["*"],
52
+ allow_headers=["*"],
53
+ )
54
+
55
+ class QuestionRequest(BaseModel):
56
+ question: str
57
+
58
+ class QuestionResponse(BaseModel):
59
+ response_type: str
60
+ message: str
61
+ confidence_explanation: str
62
+ sources: list = None
63
+
64
+ @app.post("/api/ask", response_model=QuestionResponse)
65
+ async def ask_question(request: QuestionRequest):
66
+ """
67
+ Process customer question and return answer or escalation.
68
+
69
+ Args:
70
+ request: QuestionRequest with 'question' field
71
+
72
+ Returns:
73
+ QuestionResponse with response_type, message, confidence_explanation, sources
74
+ """
75
+ try:
76
+ # Initialize components on first request
77
+ initialize_components()
78
+
79
+ # Validate question
80
+ if not request.question or len(request.question.strip()) == 0:
81
+ return JSONResponse(
82
+ status_code=400,
83
+ content={
84
+ "response_type": "error",
85
+ "message": "Question cannot be empty",
86
+ "confidence_explanation": "Invalid request",
87
+ "sources": None
88
+ }
89
+ )
90
+
91
+ # Retrieve relevant chunks
92
+ chunks = rag_system.retrieve(request.question, top_k=settings.top_k_chunks)
93
+
94
+ # Decide whether to answer or escalate
95
+ decision = escalation_engine.process_question(request.question, chunks)
96
+
97
+ return {
98
+ "response_type": decision.action,
99
+ "message": decision.message,
100
+ "confidence_explanation": decision.confidence_explanation,
101
+ "sources": decision.sources
102
+ }
103
+
104
+ except Exception as e:
105
+ # Log error and return escalation response
106
+ print(f"Error processing question: {str(e)}")
107
+ return JSONResponse(
108
+ status_code=500,
109
+ content={
110
+ "response_type": "escalation",
111
+ "message": "I'm experiencing technical difficulties. Let me connect you with a human agent.",
112
+ "confidence_explanation": f"System error occurred",
113
+ "sources": None
114
+ }
115
+ )
116
+
117
+ @app.get("/api/health")
118
+ async def health_check():
119
+ """Health check endpoint."""
120
+ return {
121
+ "status": "healthy",
122
+ "vector_store_loaded": rag_system is not None
123
+ }
124
+
125
+ # Vercel serverless function handler
126
+ def handler(request: Request):
127
+ """Main handler for Vercel serverless function."""
128
+ return app(request)
frontend/.env.example ADDED
@@ -0,0 +1,9 @@
 
 
 
 
 
 
 
 
 
 
1
+ # API Configuration
2
+ # For local development with separate backend server, use:
3
+ # VITE_API_URL=http://localhost:8000
4
+
5
+ # For Vercel deployment (monorepo with serverless functions), leave empty or use:
6
+ # VITE_API_URL=/api
7
+
8
+ # The default is /api which works for Vercel deployment
9
+ VITE_API_URL=/api
frontend/.gitignore ADDED
@@ -0,0 +1,24 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # Logs
2
+ logs
3
+ *.log
4
+ npm-debug.log*
5
+ yarn-debug.log*
6
+ yarn-error.log*
7
+ pnpm-debug.log*
8
+ lerna-debug.log*
9
+
10
+ node_modules
11
+ dist
12
+ dist-ssr
13
+ *.local
14
+
15
+ # Editor directories and files
16
+ .vscode/*
17
+ !.vscode/extensions.json
18
+ .idea
19
+ .DS_Store
20
+ *.suo
21
+ *.ntvs*
22
+ *.njsproj
23
+ *.sln
24
+ *.sw?
frontend/README.md ADDED
@@ -0,0 +1,73 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # React + TypeScript + Vite
2
+
3
+ This template provides a minimal setup to get React working in Vite with HMR and some ESLint rules.
4
+
5
+ Currently, two official plugins are available:
6
+
7
+ - [@vitejs/plugin-react](https://github.com/vitejs/vite-plugin-react/blob/main/packages/plugin-react) uses [Oxc](https://oxc.rs)
8
+ - [@vitejs/plugin-react-swc](https://github.com/vitejs/vite-plugin-react/blob/main/packages/plugin-react-swc) uses [SWC](https://swc.rs/)
9
+
10
+ ## React Compiler
11
+
12
+ The React Compiler is not enabled on this template because of its impact on dev & build performances. To add it, see [this documentation](https://react.dev/learn/react-compiler/installation).
13
+
14
+ ## Expanding the ESLint configuration
15
+
16
+ If you are developing a production application, we recommend updating the configuration to enable type-aware lint rules:
17
+
18
+ ```js
19
+ export default defineConfig([
20
+ globalIgnores(['dist']),
21
+ {
22
+ files: ['**/*.{ts,tsx}'],
23
+ extends: [
24
+ // Other configs...
25
+
26
+ // Remove tseslint.configs.recommended and replace with this
27
+ tseslint.configs.recommendedTypeChecked,
28
+ // Alternatively, use this for stricter rules
29
+ tseslint.configs.strictTypeChecked,
30
+ // Optionally, add this for stylistic rules
31
+ tseslint.configs.stylisticTypeChecked,
32
+
33
+ // Other configs...
34
+ ],
35
+ languageOptions: {
36
+ parserOptions: {
37
+ project: ['./tsconfig.node.json', './tsconfig.app.json'],
38
+ tsconfigRootDir: import.meta.dirname,
39
+ },
40
+ // other options...
41
+ },
42
+ },
43
+ ])
44
+ ```
45
+
46
+ You can also install [eslint-plugin-react-x](https://github.com/Rel1cx/eslint-react/tree/main/packages/plugins/eslint-plugin-react-x) and [eslint-plugin-react-dom](https://github.com/Rel1cx/eslint-react/tree/main/packages/plugins/eslint-plugin-react-dom) for React-specific lint rules:
47
+
48
+ ```js
49
+ // eslint.config.js
50
+ import reactX from 'eslint-plugin-react-x'
51
+ import reactDom from 'eslint-plugin-react-dom'
52
+
53
+ export default defineConfig([
54
+ globalIgnores(['dist']),
55
+ {
56
+ files: ['**/*.{ts,tsx}'],
57
+ extends: [
58
+ // Other configs...
59
+ // Enable lint rules for React
60
+ reactX.configs['recommended-typescript'],
61
+ // Enable lint rules for React DOM
62
+ reactDom.configs.recommended,
63
+ ],
64
+ languageOptions: {
65
+ parserOptions: {
66
+ project: ['./tsconfig.node.json', './tsconfig.app.json'],
67
+ tsconfigRootDir: import.meta.dirname,
68
+ },
69
+ // other options...
70
+ },
71
+ },
72
+ ])
73
+ ```
frontend/eslint.config.js ADDED
@@ -0,0 +1,23 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ import js from '@eslint/js'
2
+ import globals from 'globals'
3
+ import reactHooks from 'eslint-plugin-react-hooks'
4
+ import reactRefresh from 'eslint-plugin-react-refresh'
5
+ import tseslint from 'typescript-eslint'
6
+ import { defineConfig, globalIgnores } from 'eslint/config'
7
+
8
+ export default defineConfig([
9
+ globalIgnores(['dist']),
10
+ {
11
+ files: ['**/*.{ts,tsx}'],
12
+ extends: [
13
+ js.configs.recommended,
14
+ tseslint.configs.recommended,
15
+ reactHooks.configs.flat.recommended,
16
+ reactRefresh.configs.vite,
17
+ ],
18
+ languageOptions: {
19
+ ecmaVersion: 2020,
20
+ globals: globals.browser,
21
+ },
22
+ },
23
+ ])
frontend/index.html ADDED
@@ -0,0 +1,13 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ <!doctype html>
2
+ <html lang="en">
3
+ <head>
4
+ <meta charset="UTF-8" />
5
+ <link rel="icon" type="image/svg+xml" href="/favicon.svg" />
6
+ <meta name="viewport" content="width=device-width, initial-scale=1.0" />
7
+ <title>frontend</title>
8
+ </head>
9
+ <body>
10
+ <div id="root"></div>
11
+ <script type="module" src="/src/main.tsx"></script>
12
+ </body>
13
+ </html>
frontend/package-lock.json ADDED
The diff for this file is too large to render. See raw diff
 
frontend/package.json ADDED
@@ -0,0 +1,31 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "name": "frontend",
3
+ "private": true,
4
+ "version": "0.0.0",
5
+ "type": "module",
6
+ "scripts": {
7
+ "dev": "vite",
8
+ "build": "tsc -b && vite build",
9
+ "lint": "eslint .",
10
+ "preview": "vite preview"
11
+ },
12
+ "dependencies": {
13
+ "axios": "^1.15.0",
14
+ "react": "^19.2.4",
15
+ "react-dom": "^19.2.4"
16
+ },
17
+ "devDependencies": {
18
+ "@eslint/js": "^9.39.4",
19
+ "@types/node": "^24.12.2",
20
+ "@types/react": "^19.2.14",
21
+ "@types/react-dom": "^19.2.3",
22
+ "@vitejs/plugin-react": "^6.0.1",
23
+ "eslint": "^9.39.4",
24
+ "eslint-plugin-react-hooks": "^7.0.1",
25
+ "eslint-plugin-react-refresh": "^0.5.2",
26
+ "globals": "^17.4.0",
27
+ "typescript": "~6.0.2",
28
+ "typescript-eslint": "^8.58.0",
29
+ "vite": "^8.0.4"
30
+ }
31
+ }
frontend/public/favicon.svg ADDED
frontend/public/icons.svg ADDED
frontend/src/App.css ADDED
@@ -0,0 +1,246 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ /* Chat Container */
2
+ .chat-container {
3
+ display: flex;
4
+ flex-direction: column;
5
+ height: 100vh;
6
+ max-width: 900px;
7
+ margin: 0 auto;
8
+ background: var(--bg);
9
+ }
10
+
11
+ /* Chat Header */
12
+ .chat-header {
13
+ padding: 24px;
14
+ border-bottom: 1px solid var(--border);
15
+ text-align: center;
16
+ background: var(--bg);
17
+ }
18
+
19
+ .chat-header h1 {
20
+ margin: 0 0 8px 0;
21
+ font-size: 24px;
22
+ color: var(--text-h);
23
+ }
24
+
25
+ .chat-header p {
26
+ margin: 0;
27
+ font-size: 14px;
28
+ color: var(--text);
29
+ opacity: 0.8;
30
+ }
31
+
32
+ /* Chat Messages Area */
33
+ .chat-messages {
34
+ flex: 1;
35
+ overflow-y: auto;
36
+ padding: 24px;
37
+ display: flex;
38
+ flex-direction: column;
39
+ gap: 16px;
40
+ }
41
+
42
+ .empty-state {
43
+ display: flex;
44
+ align-items: center;
45
+ justify-content: center;
46
+ height: 100%;
47
+ color: var(--text);
48
+ opacity: 0.6;
49
+ font-size: 14px;
50
+ }
51
+
52
+ /* Message Styles */
53
+ .message {
54
+ display: flex;
55
+ flex-direction: column;
56
+ max-width: 80%;
57
+ animation: fadeIn 0.3s ease-in;
58
+ }
59
+
60
+ @keyframes fadeIn {
61
+ from {
62
+ opacity: 0;
63
+ transform: translateY(10px);
64
+ }
65
+ to {
66
+ opacity: 1;
67
+ transform: translateY(0);
68
+ }
69
+ }
70
+
71
+ .message.user {
72
+ align-self: flex-end;
73
+ }
74
+
75
+ .message.assistant {
76
+ align-self: flex-start;
77
+ }
78
+
79
+ .message-content {
80
+ padding: 12px 16px;
81
+ border-radius: 12px;
82
+ background: var(--social-bg);
83
+ }
84
+
85
+ .message.user .message-content {
86
+ background: var(--accent-bg);
87
+ border: 1px solid var(--accent-border);
88
+ }
89
+
90
+ .message.assistant .message-content {
91
+ background: var(--social-bg);
92
+ border: 1px solid var(--border);
93
+ }
94
+
95
+ .message.assistant.escalation .message-content {
96
+ background: #fff3cd;
97
+ border: 1px solid #ffc107;
98
+ }
99
+
100
+ .message-role {
101
+ display: block;
102
+ font-size: 12px;
103
+ font-weight: 600;
104
+ margin-bottom: 6px;
105
+ color: var(--text-h);
106
+ text-transform: uppercase;
107
+ letter-spacing: 0.5px;
108
+ }
109
+
110
+ .message-content p {
111
+ margin: 0;
112
+ font-size: 15px;
113
+ line-height: 1.5;
114
+ color: var(--text);
115
+ }
116
+
117
+ /* Confidence Indicator */
118
+ .confidence-indicator {
119
+ margin-top: 12px;
120
+ padding-top: 12px;
121
+ border-top: 1px solid var(--border);
122
+ font-size: 13px;
123
+ }
124
+
125
+ .confidence-label {
126
+ font-weight: 600;
127
+ color: var(--text-h);
128
+ margin-right: 6px;
129
+ }
130
+
131
+ .confidence-text {
132
+ color: var(--text);
133
+ opacity: 0.9;
134
+ }
135
+
136
+ /* Escalation Badge */
137
+ .escalation-badge {
138
+ margin-top: 8px;
139
+ padding: 6px 12px;
140
+ background: #ffc107;
141
+ color: #000;
142
+ border-radius: 6px;
143
+ font-size: 12px;
144
+ font-weight: 600;
145
+ display: inline-block;
146
+ }
147
+
148
+ /* Loading State */
149
+ .message.loading .message-content {
150
+ opacity: 0.7;
151
+ }
152
+
153
+ .loading-text {
154
+ font-style: italic;
155
+ }
156
+
157
+ /* Chat Input Form */
158
+ .chat-input-form {
159
+ display: flex;
160
+ gap: 12px;
161
+ padding: 24px;
162
+ border-top: 1px solid var(--border);
163
+ background: var(--bg);
164
+ }
165
+
166
+ .chat-input {
167
+ flex: 1;
168
+ padding: 12px 16px;
169
+ font-size: 15px;
170
+ border: 2px solid var(--border);
171
+ border-radius: 8px;
172
+ background: var(--bg);
173
+ color: var(--text);
174
+ transition: border-color 0.3s;
175
+ }
176
+
177
+ .chat-input:focus {
178
+ outline: none;
179
+ border-color: var(--accent);
180
+ }
181
+
182
+ .chat-input:disabled {
183
+ opacity: 0.5;
184
+ cursor: not-allowed;
185
+ }
186
+
187
+ .chat-submit {
188
+ padding: 12px 24px;
189
+ font-size: 15px;
190
+ font-weight: 600;
191
+ border: 2px solid var(--accent-border);
192
+ border-radius: 8px;
193
+ background: var(--accent-bg);
194
+ color: var(--accent);
195
+ cursor: pointer;
196
+ transition: all 0.3s;
197
+ }
198
+
199
+ .chat-submit:hover:not(:disabled) {
200
+ border-color: var(--accent);
201
+ box-shadow: 0 2px 8px rgba(0, 0, 0, 0.1);
202
+ }
203
+
204
+ .chat-submit:disabled {
205
+ opacity: 0.5;
206
+ cursor: not-allowed;
207
+ }
208
+
209
+ .chat-submit:focus-visible {
210
+ outline: 2px solid var(--accent);
211
+ outline-offset: 2px;
212
+ }
213
+
214
+ /* Responsive Design */
215
+ @media (max-width: 768px) {
216
+ .chat-header {
217
+ padding: 16px;
218
+ }
219
+
220
+ .chat-header h1 {
221
+ font-size: 20px;
222
+ }
223
+
224
+ .chat-messages {
225
+ padding: 16px;
226
+ }
227
+
228
+ .message {
229
+ max-width: 90%;
230
+ }
231
+
232
+ .chat-input-form {
233
+ padding: 16px;
234
+ gap: 8px;
235
+ }
236
+
237
+ .chat-input {
238
+ font-size: 14px;
239
+ }
240
+
241
+ .chat-submit {
242
+ padding: 12px 16px;
243
+ font-size: 14px;
244
+ }
245
+ }
246
+
frontend/src/App.tsx ADDED
@@ -0,0 +1,137 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ import { useState } from 'react'
2
+ import './App.css'
3
+ import { askQuestion, ApiError } from './services/api'
4
+
5
+ interface Message {
6
+ role: 'user' | 'assistant';
7
+ content: string;
8
+ confidenceExplanation?: string;
9
+ responseType?: 'answer' | 'escalation';
10
+ }
11
+
12
+ function App() {
13
+ const [messages, setMessages] = useState<Message[]>([]);
14
+ const [input, setInput] = useState('');
15
+ const [loading, setLoading] = useState(false);
16
+
17
+ const handleSubmit = async (e: React.FormEvent) => {
18
+ e.preventDefault();
19
+
20
+ if (!input.trim() || loading) return;
21
+
22
+ const userMessage: Message = {
23
+ role: 'user',
24
+ content: input.trim()
25
+ };
26
+
27
+ setMessages(prev => [...prev, userMessage]);
28
+ const questionText = input.trim();
29
+ setInput('');
30
+ setLoading(true);
31
+
32
+ try {
33
+ const response = await askQuestion(questionText);
34
+
35
+ const assistantMessage: Message = {
36
+ role: 'assistant',
37
+ content: response.message,
38
+ confidenceExplanation: response.confidence_explanation,
39
+ responseType: response.response_type
40
+ };
41
+
42
+ setMessages(prev => [...prev, assistantMessage]);
43
+ } catch (error) {
44
+ console.error('API Error:', error);
45
+
46
+ let errorMessage = 'Sorry, something went wrong. Please try again.';
47
+
48
+ if (error instanceof ApiError) {
49
+ errorMessage = error.message;
50
+ }
51
+
52
+ const errorResponse: Message = {
53
+ role: 'assistant',
54
+ content: errorMessage,
55
+ confidenceExplanation: 'Error occurred while processing your request',
56
+ responseType: 'escalation'
57
+ };
58
+
59
+ setMessages(prev => [...prev, errorResponse]);
60
+ } finally {
61
+ setLoading(false);
62
+ }
63
+ };
64
+
65
+ return (
66
+ <div className="chat-container">
67
+ <header className="chat-header">
68
+ <h1>TaskFlow Support Chat</h1>
69
+ <p>Ask questions about TaskFlow features and documentation</p>
70
+ </header>
71
+
72
+ <div className="chat-messages">
73
+ {messages.length === 0 && (
74
+ <div className="empty-state">
75
+ <p>Start a conversation by asking a question below</p>
76
+ </div>
77
+ )}
78
+
79
+ {messages.map((message, index) => (
80
+ <div
81
+ key={index}
82
+ className={`message ${message.role} ${message.responseType || ''}`}
83
+ >
84
+ <div className="message-content">
85
+ <strong className="message-role">
86
+ {message.role === 'user' ? 'You' : 'Assistant'}
87
+ </strong>
88
+ <p>{message.content}</p>
89
+
90
+ {message.confidenceExplanation && (
91
+ <div className="confidence-indicator">
92
+ <span className="confidence-label">Confidence:</span>
93
+ <span className="confidence-text">{message.confidenceExplanation}</span>
94
+ </div>
95
+ )}
96
+
97
+ {message.responseType === 'escalation' && (
98
+ <div className="escalation-badge">
99
+ Escalated to Human Support
100
+ </div>
101
+ )}
102
+ </div>
103
+ </div>
104
+ ))}
105
+
106
+ {loading && (
107
+ <div className="message assistant loading">
108
+ <div className="message-content">
109
+ <strong className="message-role">Assistant</strong>
110
+ <p className="loading-text">Thinking...</p>
111
+ </div>
112
+ </div>
113
+ )}
114
+ </div>
115
+
116
+ <form className="chat-input-form" onSubmit={handleSubmit}>
117
+ <input
118
+ type="text"
119
+ className="chat-input"
120
+ placeholder="Ask a question about TaskFlow..."
121
+ value={input}
122
+ onChange={(e) => setInput(e.target.value)}
123
+ disabled={loading}
124
+ />
125
+ <button
126
+ type="submit"
127
+ className="chat-submit"
128
+ disabled={loading || !input.trim()}
129
+ >
130
+ {loading ? 'Sending...' : 'Send'}
131
+ </button>
132
+ </form>
133
+ </div>
134
+ )
135
+ }
136
+
137
+ export default App
frontend/src/assets/hero.png ADDED
frontend/src/assets/react.svg ADDED
frontend/src/assets/vite.svg ADDED
frontend/src/index.css ADDED
@@ -0,0 +1,111 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ :root {
2
+ --text: #6b6375;
3
+ --text-h: #08060d;
4
+ --bg: #fff;
5
+ --border: #e5e4e7;
6
+ --code-bg: #f4f3ec;
7
+ --accent: #aa3bff;
8
+ --accent-bg: rgba(170, 59, 255, 0.1);
9
+ --accent-border: rgba(170, 59, 255, 0.5);
10
+ --social-bg: rgba(244, 243, 236, 0.5);
11
+ --shadow:
12
+ rgba(0, 0, 0, 0.1) 0 10px 15px -3px, rgba(0, 0, 0, 0.05) 0 4px 6px -2px;
13
+
14
+ --sans: system-ui, 'Segoe UI', Roboto, sans-serif;
15
+ --heading: system-ui, 'Segoe UI', Roboto, sans-serif;
16
+ --mono: ui-monospace, Consolas, monospace;
17
+
18
+ font: 18px/145% var(--sans);
19
+ letter-spacing: 0.18px;
20
+ color-scheme: light dark;
21
+ color: var(--text);
22
+ background: var(--bg);
23
+ font-synthesis: none;
24
+ text-rendering: optimizeLegibility;
25
+ -webkit-font-smoothing: antialiased;
26
+ -moz-osx-font-smoothing: grayscale;
27
+
28
+ @media (max-width: 1024px) {
29
+ font-size: 16px;
30
+ }
31
+ }
32
+
33
+ @media (prefers-color-scheme: dark) {
34
+ :root {
35
+ --text: #9ca3af;
36
+ --text-h: #f3f4f6;
37
+ --bg: #16171d;
38
+ --border: #2e303a;
39
+ --code-bg: #1f2028;
40
+ --accent: #c084fc;
41
+ --accent-bg: rgba(192, 132, 252, 0.15);
42
+ --accent-border: rgba(192, 132, 252, 0.5);
43
+ --social-bg: rgba(47, 48, 58, 0.5);
44
+ --shadow:
45
+ rgba(0, 0, 0, 0.4) 0 10px 15px -3px, rgba(0, 0, 0, 0.25) 0 4px 6px -2px;
46
+ }
47
+
48
+ #social .button-icon {
49
+ filter: invert(1) brightness(2);
50
+ }
51
+ }
52
+
53
+ #root {
54
+ width: 1126px;
55
+ max-width: 100%;
56
+ margin: 0 auto;
57
+ text-align: center;
58
+ border-inline: 1px solid var(--border);
59
+ min-height: 100svh;
60
+ display: flex;
61
+ flex-direction: column;
62
+ box-sizing: border-box;
63
+ }
64
+
65
+ body {
66
+ margin: 0;
67
+ }
68
+
69
+ h1,
70
+ h2 {
71
+ font-family: var(--heading);
72
+ font-weight: 500;
73
+ color: var(--text-h);
74
+ }
75
+
76
+ h1 {
77
+ font-size: 56px;
78
+ letter-spacing: -1.68px;
79
+ margin: 32px 0;
80
+ @media (max-width: 1024px) {
81
+ font-size: 36px;
82
+ margin: 20px 0;
83
+ }
84
+ }
85
+ h2 {
86
+ font-size: 24px;
87
+ line-height: 118%;
88
+ letter-spacing: -0.24px;
89
+ margin: 0 0 8px;
90
+ @media (max-width: 1024px) {
91
+ font-size: 20px;
92
+ }
93
+ }
94
+ p {
95
+ margin: 0;
96
+ }
97
+
98
+ code,
99
+ .counter {
100
+ font-family: var(--mono);
101
+ display: inline-flex;
102
+ border-radius: 4px;
103
+ color: var(--text-h);
104
+ }
105
+
106
+ code {
107
+ font-size: 15px;
108
+ line-height: 135%;
109
+ padding: 4px 8px;
110
+ background: var(--code-bg);
111
+ }
frontend/src/main.tsx ADDED
@@ -0,0 +1,10 @@
 
 
 
 
 
 
 
 
 
 
 
1
+ import { StrictMode } from 'react'
2
+ import { createRoot } from 'react-dom/client'
3
+ import './index.css'
4
+ import App from './App.tsx'
5
+
6
+ createRoot(document.getElementById('root')!).render(
7
+ <StrictMode>
8
+ <App />
9
+ </StrictMode>,
10
+ )
frontend/src/services/api.ts ADDED
@@ -0,0 +1,154 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ /**
2
+ * API client for Smart Escalation API
3
+ * Handles communication with the backend /ask endpoint
4
+ */
5
+
6
+ // API Response Types
7
+ export interface ApiResponse {
8
+ response_type: 'answer' | 'escalation';
9
+ message: string;
10
+ confidence_explanation: string;
11
+ sources: string[] | null;
12
+ }
13
+
14
+ // API Request Types
15
+ export interface QuestionRequest {
16
+ question: string;
17
+ }
18
+
19
+ // Error Types
20
+ export class ApiError extends Error {
21
+ statusCode?: number;
22
+ originalError?: unknown;
23
+
24
+ constructor(
25
+ message: string,
26
+ statusCode?: number,
27
+ originalError?: unknown
28
+ ) {
29
+ super(message);
30
+ this.name = 'ApiError';
31
+ this.statusCode = statusCode;
32
+ this.originalError = originalError;
33
+ }
34
+ }
35
+
36
+ /**
37
+ * Get API URL from environment variables
38
+ * For Vercel deployment, uses relative path /api
39
+ * For local development, uses localhost:8000
40
+ */
41
+ const getApiUrl = (): string => {
42
+ // In production (Vercel), use relative path to serverless function
43
+ // In development, use environment variable or default to localhost
44
+ return import.meta.env.VITE_API_URL || '/api';
45
+ };
46
+
47
+ /**
48
+ * POST a question to the /ask endpoint
49
+ *
50
+ * @param question - The customer question to submit
51
+ * @returns Promise resolving to the API response
52
+ * @throws ApiError on network errors, timeouts, or invalid responses
53
+ */
54
+ export const askQuestion = async (question: string): Promise<ApiResponse> => {
55
+ const apiUrl = getApiUrl();
56
+ const endpoint = `${apiUrl}/ask`;
57
+
58
+ // Validate input
59
+ if (!question || question.trim().length === 0) {
60
+ throw new ApiError('Question cannot be empty', 400);
61
+ }
62
+
63
+ const requestBody: QuestionRequest = {
64
+ question: question.trim()
65
+ };
66
+
67
+ try {
68
+ const controller = new AbortController();
69
+ const timeoutId = setTimeout(() => controller.abort(), 30000); // 30 second timeout
70
+
71
+ const response = await fetch(endpoint, {
72
+ method: 'POST',
73
+ headers: {
74
+ 'Content-Type': 'application/json',
75
+ },
76
+ body: JSON.stringify(requestBody),
77
+ signal: controller.signal,
78
+ });
79
+
80
+ clearTimeout(timeoutId);
81
+
82
+ // Handle HTTP errors
83
+ if (!response.ok) {
84
+ let errorMessage = `API request failed with status ${response.status}`;
85
+
86
+ try {
87
+ const errorData = await response.json();
88
+ errorMessage = errorData.detail || errorData.message || errorMessage;
89
+ } catch {
90
+ // If error response is not JSON, use status text
91
+ errorMessage = response.statusText || errorMessage;
92
+ }
93
+
94
+ throw new ApiError(errorMessage, response.status);
95
+ }
96
+
97
+ // Parse response
98
+ const data = await response.json();
99
+
100
+ // Validate response structure
101
+ if (!isValidApiResponse(data)) {
102
+ throw new ApiError('Invalid response format from API', 500);
103
+ }
104
+
105
+ return data;
106
+
107
+ } catch (error) {
108
+ // Handle network errors
109
+ if (error instanceof ApiError) {
110
+ throw error;
111
+ }
112
+
113
+ if (error instanceof Error) {
114
+ if (error.name === 'AbortError') {
115
+ throw new ApiError('Request timeout - please try again', 408, error);
116
+ }
117
+
118
+ if (error.message.includes('fetch')) {
119
+ throw new ApiError(
120
+ 'Unable to connect to the API. Please check your connection and try again.',
121
+ 0,
122
+ error
123
+ );
124
+ }
125
+
126
+ throw new ApiError(
127
+ `Network error: ${error.message}`,
128
+ 0,
129
+ error
130
+ );
131
+ }
132
+
133
+ throw new ApiError('An unexpected error occurred', 0, error);
134
+ }
135
+ };
136
+
137
+ /**
138
+ * Type guard to validate API response structure
139
+ */
140
+ function isValidApiResponse(data: unknown): data is ApiResponse {
141
+ if (typeof data !== 'object' || data === null) {
142
+ return false;
143
+ }
144
+
145
+ const response = data as Record<string, unknown>;
146
+
147
+ return (
148
+ typeof response.response_type === 'string' &&
149
+ (response.response_type === 'answer' || response.response_type === 'escalation') &&
150
+ typeof response.message === 'string' &&
151
+ typeof response.confidence_explanation === 'string' &&
152
+ (response.sources === null || Array.isArray(response.sources))
153
+ );
154
+ }
frontend/tsconfig.app.json ADDED
@@ -0,0 +1,25 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "compilerOptions": {
3
+ "tsBuildInfoFile": "./node_modules/.tmp/tsconfig.app.tsbuildinfo",
4
+ "target": "es2023",
5
+ "lib": ["ES2023", "DOM", "DOM.Iterable"],
6
+ "module": "esnext",
7
+ "types": ["vite/client"],
8
+ "skipLibCheck": true,
9
+
10
+ /* Bundler mode */
11
+ "moduleResolution": "bundler",
12
+ "allowImportingTsExtensions": true,
13
+ "verbatimModuleSyntax": true,
14
+ "moduleDetection": "force",
15
+ "noEmit": true,
16
+ "jsx": "react-jsx",
17
+
18
+ /* Linting */
19
+ "noUnusedLocals": true,
20
+ "noUnusedParameters": true,
21
+ "erasableSyntaxOnly": true,
22
+ "noFallthroughCasesInSwitch": true
23
+ },
24
+ "include": ["src"]
25
+ }
frontend/tsconfig.json ADDED
@@ -0,0 +1,7 @@
 
 
 
 
 
 
 
 
1
+ {
2
+ "files": [],
3
+ "references": [
4
+ { "path": "./tsconfig.app.json" },
5
+ { "path": "./tsconfig.node.json" }
6
+ ]
7
+ }
frontend/tsconfig.node.json ADDED
@@ -0,0 +1,24 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "compilerOptions": {
3
+ "tsBuildInfoFile": "./node_modules/.tmp/tsconfig.node.tsbuildinfo",
4
+ "target": "es2023",
5
+ "lib": ["ES2023"],
6
+ "module": "esnext",
7
+ "types": ["node"],
8
+ "skipLibCheck": true,
9
+
10
+ /* Bundler mode */
11
+ "moduleResolution": "bundler",
12
+ "allowImportingTsExtensions": true,
13
+ "verbatimModuleSyntax": true,
14
+ "moduleDetection": "force",
15
+ "noEmit": true,
16
+
17
+ /* Linting */
18
+ "noUnusedLocals": true,
19
+ "noUnusedParameters": true,
20
+ "erasableSyntaxOnly": true,
21
+ "noFallthroughCasesInSwitch": true
22
+ },
23
+ "include": ["vite.config.ts"]
24
+ }
frontend/vite.config.ts ADDED
@@ -0,0 +1,7 @@
 
 
 
 
 
 
 
 
1
+ import { defineConfig } from 'vite'
2
+ import react from '@vitejs/plugin-react'
3
+
4
+ // https://vite.dev/config/
5
+ export default defineConfig({
6
+ plugins: [react()],
7
+ })
package.json ADDED
@@ -0,0 +1,27 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "name": "smart-escalation-api",
3
+ "version": "1.0.0",
4
+ "description": "AI-powered L1 customer support system with intelligent escalation",
5
+ "private": true,
6
+ "scripts": {
7
+ "build": "cd frontend && npm install && npm run build",
8
+ "dev:frontend": "cd frontend && npm run dev",
9
+ "dev:backend": "uvicorn src.main:app --reload",
10
+ "test": "pytest -v",
11
+ "test:property": "pytest tests/property -v",
12
+ "lint": "cd frontend && npm run lint"
13
+ },
14
+ "workspaces": [
15
+ "frontend"
16
+ ],
17
+ "keywords": [
18
+ "customer-support",
19
+ "rag",
20
+ "llm",
21
+ "escalation",
22
+ "fastapi",
23
+ "react"
24
+ ],
25
+ "author": "",
26
+ "license": "MIT"
27
+ }
src/escalation.py CHANGED
@@ -223,13 +223,18 @@ class EscalationEngine:
223
  # If LLM call fails, escalate gracefully
224
  error_msg = str(e)
225
 
 
 
 
 
 
226
  # Provide specific error context in explanation
227
  if "timeout" in error_msg.lower():
228
  explanation = "Unable to generate response due to timeout, escalating to human agent"
229
  elif "rate limit" in error_msg.lower():
230
  explanation = "Service temporarily busy, escalating to human agent"
231
  else:
232
- explanation = "Unable to generate response, escalating to human agent"
233
 
234
  return EscalationDecision(
235
  action="escalation",
 
223
  # If LLM call fails, escalate gracefully
224
  error_msg = str(e)
225
 
226
+ # Log the actual error for debugging
227
+ print(f"ERROR in escalation engine: {error_msg}")
228
+ import traceback
229
+ traceback.print_exc()
230
+
231
  # Provide specific error context in explanation
232
  if "timeout" in error_msg.lower():
233
  explanation = "Unable to generate response due to timeout, escalating to human agent"
234
  elif "rate limit" in error_msg.lower():
235
  explanation = "Service temporarily busy, escalating to human agent"
236
  else:
237
+ explanation = f"Unable to generate response, escalating to human agent (Error: {error_msg})"
238
 
239
  return EscalationDecision(
240
  action="escalation",
tests/unit/test_escalation.py ADDED
@@ -0,0 +1,408 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """
2
+ Unit tests for the escalation engine module.
3
+
4
+ Tests escalation decision rules including threshold comparisons
5
+ and confidence explanation generation.
6
+
7
+ **Validates: Requirements 4.1, 4.2, 4.3, 4.4, 4.5**
8
+ """
9
+
10
+ import pytest
11
+ from unittest.mock import Mock, MagicMock
12
+ from src.escalation import EscalationEngine, EscalationDecision
13
+ from src.rag import RetrievedChunk
14
+ from src.llm_client import LLMClient, LLMResponse
15
+
16
+
17
+ class TestEscalationDecisionRules:
18
+ """Test cases for escalation decision logic."""
19
+
20
+ @pytest.fixture
21
+ def mock_llm_client(self):
22
+ """Create a mock LLM client."""
23
+ return Mock(spec=LLMClient)
24
+
25
+ @pytest.fixture
26
+ def engine(self, mock_llm_client):
27
+ """Create escalation engine with default threshold."""
28
+ return EscalationEngine(
29
+ llm_client=mock_llm_client,
30
+ relevance_threshold=0.5
31
+ )
32
+
33
+ def test_escalate_on_empty_chunks(self, engine):
34
+ """Should escalate when no chunks are retrieved."""
35
+ decision = engine.process_question("test question", [])
36
+
37
+ assert decision.action == "escalation"
38
+ assert "no relevant information" in decision.confidence_explanation.lower()
39
+
40
+ def test_escalate_on_low_relevance_score(self, engine):
41
+ """Should escalate when best relevance score is below threshold."""
42
+ chunks = [
43
+ RetrievedChunk(
44
+ content="Some content",
45
+ score=0.3, # Below 0.5 threshold
46
+ source="test.md",
47
+ chunk_id=0,
48
+ position=0
49
+ )
50
+ ]
51
+
52
+ decision = engine.process_question("test question", chunks)
53
+
54
+ assert decision.action == "escalation"
55
+ assert "0.30" in decision.confidence_explanation or "0.3" in decision.confidence_explanation
56
+ assert decision.sources is None
57
+
58
+ def test_escalate_on_llm_uncertainty(self, engine, mock_llm_client):
59
+ """Should escalate when LLM signals uncertainty."""
60
+ chunks = [
61
+ RetrievedChunk(
62
+ content="Relevant content",
63
+ score=0.8, # High score
64
+ source="test.md",
65
+ chunk_id=0,
66
+ position=0
67
+ )
68
+ ]
69
+
70
+ # Mock LLM to return uncertain response
71
+ mock_llm_client.generate_answer.return_value = LLMResponse(
72
+ answer="I'm not certain about this.",
73
+ uncertain=True
74
+ )
75
+
76
+ decision = engine.process_question("test question", chunks)
77
+
78
+ assert decision.action == "escalation"
79
+ assert "uncertainty" in decision.confidence_explanation.lower()
80
+ assert decision.sources is None
81
+
82
+ def test_answer_on_high_relevance(self, engine, mock_llm_client):
83
+ """Should answer when relevance score is above threshold and LLM is confident."""
84
+ chunks = [
85
+ RetrievedChunk(
86
+ content="Relevant content",
87
+ score=0.8,
88
+ source="billing.md",
89
+ chunk_id=0,
90
+ position=0
91
+ )
92
+ ]
93
+
94
+ # Mock LLM to return confident answer
95
+ mock_llm_client.generate_answer.return_value = LLMResponse(
96
+ answer="Here is the answer to your question.",
97
+ uncertain=False
98
+ )
99
+
100
+ decision = engine.process_question("test question", chunks)
101
+
102
+ assert decision.action == "answer"
103
+ assert decision.message == "Here is the answer to your question."
104
+ assert "0.8" in decision.confidence_explanation or "0.80" in decision.confidence_explanation
105
+ assert decision.sources == ["billing.md"]
106
+
107
+ def test_threshold_boundary_below(self, engine):
108
+ """Should escalate when score is exactly at threshold (0.5)."""
109
+ chunks = [
110
+ RetrievedChunk(
111
+ content="Content",
112
+ score=0.49, # Just below threshold
113
+ source="test.md",
114
+ chunk_id=0,
115
+ position=0
116
+ )
117
+ ]
118
+
119
+ decision = engine.process_question("test question", chunks)
120
+
121
+ assert decision.action == "escalation"
122
+
123
+ def test_threshold_boundary_above(self, engine, mock_llm_client):
124
+ """Should attempt to answer when score is above threshold."""
125
+ chunks = [
126
+ RetrievedChunk(
127
+ content="Content",
128
+ score=0.51, # Just above threshold
129
+ source="test.md",
130
+ chunk_id=0,
131
+ position=0
132
+ )
133
+ ]
134
+
135
+ mock_llm_client.generate_answer.return_value = LLMResponse(
136
+ answer="Answer",
137
+ uncertain=False
138
+ )
139
+
140
+ decision = engine.process_question("test question", chunks)
141
+
142
+ assert decision.action == "answer"
143
+
144
+ def test_custom_threshold(self, mock_llm_client):
145
+ """Should respect custom relevance threshold."""
146
+ engine = EscalationEngine(
147
+ llm_client=mock_llm_client,
148
+ relevance_threshold=0.7 # Higher threshold
149
+ )
150
+
151
+ chunks = [
152
+ RetrievedChunk(
153
+ content="Content",
154
+ score=0.6, # Would pass 0.5 but not 0.7
155
+ source="test.md",
156
+ chunk_id=0,
157
+ position=0
158
+ )
159
+ ]
160
+
161
+ decision = engine.process_question("test question", chunks)
162
+
163
+ assert decision.action == "escalation"
164
+ assert "0.7" in decision.confidence_explanation or "0.70" in decision.confidence_explanation
165
+
166
+
167
+ class TestConfidenceExplanationGeneration:
168
+ """Test cases for confidence explanation generation."""
169
+
170
+ @pytest.fixture
171
+ def mock_llm_client(self):
172
+ """Create a mock LLM client."""
173
+ return Mock(spec=LLMClient)
174
+
175
+ @pytest.fixture
176
+ def engine(self, mock_llm_client):
177
+ """Create escalation engine."""
178
+ return EscalationEngine(llm_client=mock_llm_client)
179
+
180
+ def test_explanation_includes_score_for_answer(self, engine, mock_llm_client):
181
+ """Confidence explanation for answers should include relevance score."""
182
+ chunks = [
183
+ RetrievedChunk(
184
+ content="Content",
185
+ score=0.85,
186
+ source="test.md",
187
+ chunk_id=0,
188
+ position=0
189
+ )
190
+ ]
191
+
192
+ mock_llm_client.generate_answer.return_value = LLMResponse(
193
+ answer="Answer",
194
+ uncertain=False
195
+ )
196
+
197
+ decision = engine.process_question("test", chunks)
198
+
199
+ assert "0.85" in decision.confidence_explanation
200
+ assert "confidence" in decision.confidence_explanation.lower()
201
+
202
+ def test_explanation_includes_source_for_answer(self, engine, mock_llm_client):
203
+ """Confidence explanation should mention source article."""
204
+ chunks = [
205
+ RetrievedChunk(
206
+ content="Content",
207
+ score=0.8,
208
+ source="billing.md",
209
+ chunk_id=0,
210
+ position=0
211
+ )
212
+ ]
213
+
214
+ mock_llm_client.generate_answer.return_value = LLMResponse(
215
+ answer="Answer",
216
+ uncertain=False
217
+ )
218
+
219
+ decision = engine.process_question("test", chunks)
220
+
221
+ assert "billing.md" in decision.confidence_explanation
222
+
223
+ def test_explanation_for_multiple_sources(self, engine, mock_llm_client):
224
+ """Should mention multiple sources in explanation."""
225
+ chunks = [
226
+ RetrievedChunk(content="C1", score=0.8, source="billing.md", chunk_id=0, position=0),
227
+ RetrievedChunk(content="C2", score=0.7, source="permissions.md", chunk_id=1, position=0)
228
+ ]
229
+
230
+ mock_llm_client.generate_answer.return_value = LLMResponse(
231
+ answer="Answer",
232
+ uncertain=False
233
+ )
234
+
235
+ decision = engine.process_question("test", chunks)
236
+
237
+ # Should mention multiple articles
238
+ assert "2" in decision.confidence_explanation or "multiple" in decision.confidence_explanation.lower()
239
+
240
+ def test_explanation_for_low_score_escalation(self, engine):
241
+ """Escalation explanation should mention low score."""
242
+ chunks = [
243
+ RetrievedChunk(
244
+ content="Content",
245
+ score=0.3,
246
+ source="test.md",
247
+ chunk_id=0,
248
+ position=0
249
+ )
250
+ ]
251
+
252
+ decision = engine.process_question("test", chunks)
253
+
254
+ assert decision.action == "escalation"
255
+ assert "0.3" in decision.confidence_explanation or "0.30" in decision.confidence_explanation
256
+ assert "threshold" in decision.confidence_explanation.lower()
257
+
258
+ def test_explanation_for_llm_uncertainty(self, engine, mock_llm_client):
259
+ """Escalation explanation should mention LLM uncertainty."""
260
+ chunks = [
261
+ RetrievedChunk(content="Content", score=0.8, source="test.md", chunk_id=0, position=0)
262
+ ]
263
+
264
+ mock_llm_client.generate_answer.return_value = LLMResponse(
265
+ answer="Not sure",
266
+ uncertain=True
267
+ )
268
+
269
+ decision = engine.process_question("test", chunks)
270
+
271
+ assert decision.action == "escalation"
272
+ assert "uncertainty" in decision.confidence_explanation.lower()
273
+
274
+ def test_explanation_for_no_chunks(self, engine):
275
+ """Escalation explanation should mention no relevant information."""
276
+ decision = engine.process_question("test", [])
277
+
278
+ assert decision.action == "escalation"
279
+ assert "no relevant" in decision.confidence_explanation.lower()
280
+
281
+
282
+ class TestSourceExtraction:
283
+ """Test cases for source reference extraction."""
284
+
285
+ @pytest.fixture
286
+ def mock_llm_client(self):
287
+ """Create a mock LLM client."""
288
+ return Mock(spec=LLMClient)
289
+
290
+ @pytest.fixture
291
+ def engine(self, mock_llm_client):
292
+ """Create escalation engine."""
293
+ return EscalationEngine(llm_client=mock_llm_client)
294
+
295
+ def test_sources_included_for_answers(self, engine, mock_llm_client):
296
+ """Answer responses should include source references."""
297
+ chunks = [
298
+ RetrievedChunk(content="C", score=0.8, source="billing.md", chunk_id=0, position=0)
299
+ ]
300
+
301
+ mock_llm_client.generate_answer.return_value = LLMResponse(
302
+ answer="Answer",
303
+ uncertain=False
304
+ )
305
+
306
+ decision = engine.process_question("test", chunks)
307
+
308
+ assert decision.sources is not None
309
+ assert "billing.md" in decision.sources
310
+
311
+ def test_sources_not_included_for_escalations(self, engine):
312
+ """Escalation responses should not include sources."""
313
+ chunks = [
314
+ RetrievedChunk(content="C", score=0.2, source="test.md", chunk_id=0, position=0)
315
+ ]
316
+
317
+ decision = engine.process_question("test", chunks)
318
+
319
+ assert decision.action == "escalation"
320
+ assert decision.sources is None
321
+
322
+ def test_multiple_sources_deduplicated(self, engine, mock_llm_client):
323
+ """Should deduplicate sources from multiple chunks."""
324
+ chunks = [
325
+ RetrievedChunk(content="C1", score=0.8, source="billing.md", chunk_id=0, position=0),
326
+ RetrievedChunk(content="C2", score=0.7, source="billing.md", chunk_id=1, position=100),
327
+ RetrievedChunk(content="C3", score=0.6, source="permissions.md", chunk_id=0, position=0)
328
+ ]
329
+
330
+ mock_llm_client.generate_answer.return_value = LLMResponse(
331
+ answer="Answer",
332
+ uncertain=False
333
+ )
334
+
335
+ decision = engine.process_question("test", chunks)
336
+
337
+ assert len(decision.sources) == 2
338
+ assert "billing.md" in decision.sources
339
+ assert "permissions.md" in decision.sources
340
+
341
+ def test_sources_sorted(self, engine, mock_llm_client):
342
+ """Sources should be sorted alphabetically."""
343
+ chunks = [
344
+ RetrievedChunk(content="C1", score=0.8, source="permissions.md", chunk_id=0, position=0),
345
+ RetrievedChunk(content="C2", score=0.7, source="billing.md", chunk_id=0, position=0)
346
+ ]
347
+
348
+ mock_llm_client.generate_answer.return_value = LLMResponse(
349
+ answer="Answer",
350
+ uncertain=False
351
+ )
352
+
353
+ decision = engine.process_question("test", chunks)
354
+
355
+ assert decision.sources == ["billing.md", "permissions.md"]
356
+
357
+
358
+ class TestErrorHandling:
359
+ """Test cases for error handling in escalation engine."""
360
+
361
+ @pytest.fixture
362
+ def mock_llm_client(self):
363
+ """Create a mock LLM client."""
364
+ return Mock(spec=LLMClient)
365
+
366
+ @pytest.fixture
367
+ def engine(self, mock_llm_client):
368
+ """Create escalation engine."""
369
+ return EscalationEngine(llm_client=mock_llm_client)
370
+
371
+ def test_escalate_on_llm_error(self, engine, mock_llm_client):
372
+ """Should escalate gracefully when LLM call fails."""
373
+ chunks = [
374
+ RetrievedChunk(content="C", score=0.8, source="test.md", chunk_id=0, position=0)
375
+ ]
376
+
377
+ mock_llm_client.generate_answer.side_effect = Exception("API error")
378
+
379
+ decision = engine.process_question("test", chunks)
380
+
381
+ assert decision.action == "escalation"
382
+ assert decision.sources is None
383
+
384
+ def test_escalate_on_timeout(self, engine, mock_llm_client):
385
+ """Should escalate with timeout message when LLM times out."""
386
+ chunks = [
387
+ RetrievedChunk(content="C", score=0.8, source="test.md", chunk_id=0, position=0)
388
+ ]
389
+
390
+ mock_llm_client.generate_answer.side_effect = RuntimeError("Request timeout")
391
+
392
+ decision = engine.process_question("test", chunks)
393
+
394
+ assert decision.action == "escalation"
395
+ assert "timeout" in decision.confidence_explanation.lower()
396
+
397
+ def test_escalate_on_rate_limit(self, engine, mock_llm_client):
398
+ """Should escalate with rate limit message."""
399
+ chunks = [
400
+ RetrievedChunk(content="C", score=0.8, source="test.md", chunk_id=0, position=0)
401
+ ]
402
+
403
+ mock_llm_client.generate_answer.side_effect = RuntimeError("Rate limit exceeded")
404
+
405
+ decision = engine.process_question("test", chunks)
406
+
407
+ assert decision.action == "escalation"
408
+ assert "rate limit" in decision.confidence_explanation.lower() or "busy" in decision.confidence_explanation.lower()
tests/unit/test_response_formatting.py ADDED
@@ -0,0 +1,381 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """
2
+ Unit tests for API response formatting.
3
+
4
+ Tests JSON construction and source reference inclusion in API responses.
5
+
6
+ **Validates: Requirements 6.1, 6.2, 6.3, 6.4, 6.5**
7
+ """
8
+
9
+ import pytest
10
+ from pydantic import ValidationError
11
+ from src.main import QuestionRequest, QuestionResponse
12
+
13
+
14
+ class TestQuestionRequestValidation:
15
+ """Test cases for QuestionRequest model validation."""
16
+
17
+ def test_valid_request(self):
18
+ """Should accept valid question."""
19
+ request = QuestionRequest(question="How do I reset my password?")
20
+
21
+ assert request.question == "How do I reset my password?"
22
+
23
+ def test_reject_empty_question(self):
24
+ """Should reject empty question string."""
25
+ with pytest.raises(ValidationError) as exc_info:
26
+ QuestionRequest(question="")
27
+
28
+ assert "question" in str(exc_info.value).lower()
29
+
30
+ def test_reject_whitespace_only(self):
31
+ """Should reject whitespace-only question."""
32
+ with pytest.raises(ValidationError) as exc_info:
33
+ QuestionRequest(question=" \n\t ")
34
+
35
+ assert "question" in str(exc_info.value).lower()
36
+
37
+ def test_reject_missing_question(self):
38
+ """Should reject request without question field."""
39
+ with pytest.raises(ValidationError):
40
+ QuestionRequest()
41
+
42
+ def test_reject_too_long_question(self):
43
+ """Should reject question exceeding max length."""
44
+ long_question = "a" * 501 # Max is 500
45
+
46
+ with pytest.raises(ValidationError) as exc_info:
47
+ QuestionRequest(question=long_question)
48
+
49
+ assert "500" in str(exc_info.value) or "length" in str(exc_info.value).lower()
50
+
51
+ def test_trim_whitespace(self):
52
+ """Should trim leading/trailing whitespace from question."""
53
+ request = QuestionRequest(question=" How do I reset? ")
54
+
55
+ assert request.question == "How do I reset?"
56
+ assert not request.question.startswith(" ")
57
+ assert not request.question.endswith(" ")
58
+
59
+ def test_accept_max_length_question(self):
60
+ """Should accept question at exactly max length."""
61
+ question = "a" * 500
62
+ request = QuestionRequest(question=question)
63
+
64
+ assert len(request.question) == 500
65
+
66
+
67
+ class TestQuestionResponseConstruction:
68
+ """Test cases for QuestionResponse model construction."""
69
+
70
+ def test_answer_response_construction(self):
71
+ """Should construct valid answer response."""
72
+ response = QuestionResponse(
73
+ response_type="answer",
74
+ message="Here is your answer.",
75
+ confidence_explanation="High confidence based on retrieval.",
76
+ sources=["billing.md"]
77
+ )
78
+
79
+ assert response.response_type == "answer"
80
+ assert response.message == "Here is your answer."
81
+ assert response.confidence_explanation == "High confidence based on retrieval."
82
+ assert response.sources == ["billing.md"]
83
+
84
+ def test_escalation_response_construction(self):
85
+ """Should construct valid escalation response."""
86
+ response = QuestionResponse(
87
+ response_type="escalation",
88
+ message="Let me connect you with a human agent.",
89
+ confidence_explanation="Low confidence - score below threshold.",
90
+ sources=None
91
+ )
92
+
93
+ assert response.response_type == "escalation"
94
+ assert response.message == "Let me connect you with a human agent."
95
+ assert response.confidence_explanation == "Low confidence - score below threshold."
96
+ assert response.sources is None
97
+
98
+ def test_sources_optional(self):
99
+ """Sources field should be optional."""
100
+ response = QuestionResponse(
101
+ response_type="answer",
102
+ message="Answer",
103
+ confidence_explanation="Explanation"
104
+ )
105
+
106
+ assert response.sources is None
107
+
108
+ def test_reject_invalid_response_type(self):
109
+ """Should reject invalid response_type values."""
110
+ with pytest.raises(ValidationError) as exc_info:
111
+ QuestionResponse(
112
+ response_type="invalid",
113
+ message="Message",
114
+ confidence_explanation="Explanation"
115
+ )
116
+
117
+ assert "response_type" in str(exc_info.value).lower()
118
+
119
+ def test_reject_missing_required_fields(self):
120
+ """Should reject response missing required fields."""
121
+ with pytest.raises(ValidationError):
122
+ QuestionResponse(response_type="answer")
123
+
124
+ def test_multiple_sources(self):
125
+ """Should accept multiple source references."""
126
+ response = QuestionResponse(
127
+ response_type="answer",
128
+ message="Answer",
129
+ confidence_explanation="Explanation",
130
+ sources=["billing.md", "permissions.md", "troubleshooting.md"]
131
+ )
132
+
133
+ assert len(response.sources) == 3
134
+ assert "billing.md" in response.sources
135
+
136
+ def test_empty_sources_list(self):
137
+ """Should accept empty sources list."""
138
+ response = QuestionResponse(
139
+ response_type="answer",
140
+ message="Answer",
141
+ confidence_explanation="Explanation",
142
+ sources=[]
143
+ )
144
+
145
+ assert response.sources == []
146
+
147
+
148
+ class TestResponseJSONSerialization:
149
+ """Test cases for JSON serialization of responses."""
150
+
151
+ def test_answer_json_schema(self):
152
+ """Answer response should serialize to correct JSON schema."""
153
+ response = QuestionResponse(
154
+ response_type="answer",
155
+ message="Your answer here.",
156
+ confidence_explanation="High confidence.",
157
+ sources=["billing.md"]
158
+ )
159
+
160
+ json_data = response.model_dump()
161
+
162
+ assert "response_type" in json_data
163
+ assert "message" in json_data
164
+ assert "confidence_explanation" in json_data
165
+ assert "sources" in json_data
166
+ assert json_data["response_type"] == "answer"
167
+ assert json_data["message"] == "Your answer here."
168
+ assert json_data["confidence_explanation"] == "High confidence."
169
+ assert json_data["sources"] == ["billing.md"]
170
+
171
+ def test_escalation_json_schema(self):
172
+ """Escalation response should serialize to correct JSON schema."""
173
+ response = QuestionResponse(
174
+ response_type="escalation",
175
+ message="Connecting you with an agent.",
176
+ confidence_explanation="Low confidence.",
177
+ sources=None
178
+ )
179
+
180
+ json_data = response.model_dump()
181
+
182
+ assert json_data["response_type"] == "escalation"
183
+ assert json_data["message"] == "Connecting you with an agent."
184
+ assert json_data["confidence_explanation"] == "Low confidence."
185
+ assert json_data["sources"] is None
186
+
187
+ def test_json_field_types(self):
188
+ """JSON fields should have correct types."""
189
+ response = QuestionResponse(
190
+ response_type="answer",
191
+ message="Answer",
192
+ confidence_explanation="Explanation",
193
+ sources=["test.md"]
194
+ )
195
+
196
+ json_data = response.model_dump()
197
+
198
+ assert isinstance(json_data["response_type"], str)
199
+ assert isinstance(json_data["message"], str)
200
+ assert isinstance(json_data["confidence_explanation"], str)
201
+ assert isinstance(json_data["sources"], list)
202
+
203
+ def test_json_roundtrip(self):
204
+ """Should be able to serialize and deserialize."""
205
+ original = QuestionResponse(
206
+ response_type="answer",
207
+ message="Test answer",
208
+ confidence_explanation="Test explanation",
209
+ sources=["test.md"]
210
+ )
211
+
212
+ # Serialize to dict
213
+ json_data = original.model_dump()
214
+
215
+ # Deserialize back
216
+ restored = QuestionResponse(**json_data)
217
+
218
+ assert restored.response_type == original.response_type
219
+ assert restored.message == original.message
220
+ assert restored.confidence_explanation == original.confidence_explanation
221
+ assert restored.sources == original.sources
222
+
223
+
224
+ class TestResponseFieldContent:
225
+ """Test cases for response field content validation."""
226
+
227
+ def test_message_not_empty(self):
228
+ """Message field should not be empty."""
229
+ # Pydantic doesn't enforce non-empty strings by default,
230
+ # but we can test that empty strings are technically allowed
231
+ # (business logic should prevent this)
232
+ response = QuestionResponse(
233
+ response_type="answer",
234
+ message="",
235
+ confidence_explanation="Explanation"
236
+ )
237
+
238
+ assert response.message == ""
239
+
240
+ def test_confidence_explanation_not_empty(self):
241
+ """Confidence explanation should not be empty."""
242
+ response = QuestionResponse(
243
+ response_type="answer",
244
+ message="Message",
245
+ confidence_explanation=""
246
+ )
247
+
248
+ assert response.confidence_explanation == ""
249
+
250
+ def test_response_type_literal_values(self):
251
+ """Response type should only accept 'answer' or 'escalation'."""
252
+ # Valid values
253
+ answer_response = QuestionResponse(
254
+ response_type="answer",
255
+ message="M",
256
+ confidence_explanation="E"
257
+ )
258
+ assert answer_response.response_type == "answer"
259
+
260
+ escalation_response = QuestionResponse(
261
+ response_type="escalation",
262
+ message="M",
263
+ confidence_explanation="E"
264
+ )
265
+ assert escalation_response.response_type == "escalation"
266
+
267
+ # Invalid value should raise error
268
+ with pytest.raises(ValidationError):
269
+ QuestionResponse(
270
+ response_type="unknown",
271
+ message="M",
272
+ confidence_explanation="E"
273
+ )
274
+
275
+
276
+ class TestSourceReferenceFormatting:
277
+ """Test cases for source reference formatting."""
278
+
279
+ def test_sources_as_list_of_strings(self):
280
+ """Sources should be a list of strings."""
281
+ response = QuestionResponse(
282
+ response_type="answer",
283
+ message="Answer",
284
+ confidence_explanation="Explanation",
285
+ sources=["billing.md", "permissions.md"]
286
+ )
287
+
288
+ assert isinstance(response.sources, list)
289
+ assert all(isinstance(s, str) for s in response.sources)
290
+
291
+ def test_sources_preserve_order(self):
292
+ """Sources list should preserve order."""
293
+ sources = ["c.md", "a.md", "b.md"]
294
+ response = QuestionResponse(
295
+ response_type="answer",
296
+ message="Answer",
297
+ confidence_explanation="Explanation",
298
+ sources=sources
299
+ )
300
+
301
+ assert response.sources == sources
302
+
303
+ def test_sources_allow_duplicates(self):
304
+ """Sources list should allow duplicates (though business logic may prevent this)."""
305
+ response = QuestionResponse(
306
+ response_type="answer",
307
+ message="Answer",
308
+ confidence_explanation="Explanation",
309
+ sources=["test.md", "test.md"]
310
+ )
311
+
312
+ assert len(response.sources) == 2
313
+
314
+ def test_escalation_typically_has_no_sources(self):
315
+ """Escalation responses typically have no sources."""
316
+ response = QuestionResponse(
317
+ response_type="escalation",
318
+ message="Escalating",
319
+ confidence_explanation="Low confidence",
320
+ sources=None
321
+ )
322
+
323
+ assert response.sources is None
324
+
325
+ def test_answer_can_have_sources(self):
326
+ """Answer responses can include sources."""
327
+ response = QuestionResponse(
328
+ response_type="answer",
329
+ message="Answer",
330
+ confidence_explanation="High confidence",
331
+ sources=["billing.md"]
332
+ )
333
+
334
+ assert response.sources is not None
335
+ assert len(response.sources) > 0
336
+
337
+
338
+ class TestResponseConsistency:
339
+ """Test cases for response schema consistency."""
340
+
341
+ def test_all_responses_have_required_fields(self):
342
+ """All responses must have response_type, message, and confidence_explanation."""
343
+ # Answer response
344
+ answer = QuestionResponse(
345
+ response_type="answer",
346
+ message="A",
347
+ confidence_explanation="E"
348
+ )
349
+ assert hasattr(answer, "response_type")
350
+ assert hasattr(answer, "message")
351
+ assert hasattr(answer, "confidence_explanation")
352
+ assert hasattr(answer, "sources")
353
+
354
+ # Escalation response
355
+ escalation = QuestionResponse(
356
+ response_type="escalation",
357
+ message="A",
358
+ confidence_explanation="E"
359
+ )
360
+ assert hasattr(escalation, "response_type")
361
+ assert hasattr(escalation, "message")
362
+ assert hasattr(escalation, "confidence_explanation")
363
+ assert hasattr(escalation, "sources")
364
+
365
+ def test_response_type_determines_content(self):
366
+ """Response type should indicate whether it's an answer or escalation."""
367
+ answer = QuestionResponse(
368
+ response_type="answer",
369
+ message="Here is your answer",
370
+ confidence_explanation="High confidence"
371
+ )
372
+
373
+ escalation = QuestionResponse(
374
+ response_type="escalation",
375
+ message="Let me connect you with an agent",
376
+ confidence_explanation="Low confidence"
377
+ )
378
+
379
+ assert answer.response_type == "answer"
380
+ assert escalation.response_type == "escalation"
381
+ assert answer.response_type != escalation.response_type
vercel.json ADDED
@@ -0,0 +1,29 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "version": 2,
3
+ "builds": [
4
+ {
5
+ "src": "api/**/*.py",
6
+ "use": "@vercel/python"
7
+ },
8
+ {
9
+ "src": "frontend/package.json",
10
+ "use": "@vercel/static-build",
11
+ "config": {
12
+ "distDir": "frontend/dist"
13
+ }
14
+ }
15
+ ],
16
+ "routes": [
17
+ {
18
+ "src": "/api/(.*)",
19
+ "dest": "/api/$1"
20
+ },
21
+ {
22
+ "src": "/(.*)",
23
+ "dest": "/frontend/$1"
24
+ }
25
+ ],
26
+ "env": {
27
+ "PYTHON_VERSION": "3.9"
28
+ }
29
+ }