Rachit-Mangawa commited on
Commit
08b98ed
·
1 Parent(s): 42c83f7

feat: Scalable Multi-Agent Orchestrator (v7.0) with Multi-Key Rotation and FAST_TEST mode

Browse files
Files changed (2) hide show
  1. README.md +21 -25
  2. app.py +174 -215
README.md CHANGED
@@ -4,59 +4,55 @@ emoji: 🚀
4
  colorFrom: blue
5
  colorTo: purple
6
  sdk: docker
7
- app_port: 8086
8
  pinned: false
9
  ---
10
 
11
- # Vera Engine Platinum v4.1 🤖
12
 
13
- **High-fidelity merchant growth engine optimized for the magicpin AI Challenge.**
14
 
15
  ## 1. THE CORE (CAVEMAN LOGIC)
16
  * **Vera good.**
17
  * **Bot no lie.**
18
- * **Cloud fast.**
19
  * **Local safe.**
20
- * **Merchant win.**
21
 
22
  ## 2. ARCHITECTURAL SPECIFICATIONS
23
 
24
- ### Dual-Tier Inference Engine
25
- The system employs a prioritized routing strategy to maximize reasoning depth while maintaining zero downtime.
26
 
27
- 1. **Tier 1 (Strategic Cloud)**: Primary inference via OpenRouter. Sequential polling of `poolside/laguna-m.1:free`, `tencent/hy3:free`, and `gemini-2.0-flash-exp:free`.
28
- * **Latency**: < 2s (average).
29
- * **Logic**: High-order growth strategy and code-mixed Hindi-English fluency.
30
- 2. **Tier 2 (Autonomous Local)**: Automatic fallback to `Llama 3.2 (3B)` running via Ollama on CPU.
31
- * **Latency**: 17.9s (benchmarked on 2vCPU).
32
- * **Resilience**: 100% operational during API outages or rate limit exhaustion.
33
-
34
- ### The Shield (Interceptor)
35
- Deterministic regex-based filtering of merchant auto-replies and hostile inputs. Prevents compute thrashing and ensures turn-efficiency.
36
 
37
  ## 3. KEY FEATURES
38
 
39
- * **Metric Anchoring**: Hard-coded evidence blocks (Mandatory Metrics) injected into prompts to prevent LLM hallucination of B2B data. Integrity: 100%.
40
- * **Elastic Pydantic Schema**: All ingestion fields are `Optional` with strict defaults. Guaranteed zero-fail context processing even with malformed or incomplete judge payloads.
41
- * **Sequential Atomic Throughput**: `/v1/tick` is locked to a single high-priority trigger per call to survive strict 30s judge timeouts on limited hardware.
42
- * **Lifespan Lifecycle**: Fast-path startup sequence with integrated API dry-run verification for cloud readiness.
43
 
44
  ## 4. API DOCUMENTATION
45
 
46
  | Endpoint | Method | Purpose | Response Schema |
47
  | :--- | :--- | :--- | :--- |
48
- | `/v1/healthz` | `GET` | Sanitized Liveness Probe | `{"status": "ok"}` |
49
- | `/v1/metadata` | `GET` | System Identity & Model Status | `{"version": "4.1", ...}` |
50
  | `/v1/context` | `POST` | Elastic Context Ingestion | `{"accepted": true}` |
51
  | `/v1/tick` | `POST` | Atomic Trigger Processing | `{"actions": [...]}` |
52
- | `/v1/reply` | `POST` | Shielded Message Handling | `{"action": "send/wait/end"}` |
53
 
54
  ## 5. TECHNICAL STACK
55
 
56
- * **Framework**: FastAPI / Uvicorn
57
  * **Runtime**: Docker (Python 3.10-slim)
58
  * **Local LLM**: Ollama / Llama 3.2 (3B)
59
- * **Cloud Gateway**: OpenRouter
60
  * **Infrastructure**: Hugging Face Spaces (2vCPU / 16GB RAM)
61
 
62
  ## 6. DEPLOYMENT
 
4
  colorFrom: blue
5
  colorTo: purple
6
  sdk: docker
7
+ app_port: 8080
8
  pinned: false
9
  ---
10
 
11
+ # Vera Engine Platinum v7.0 🤖
12
 
13
+ **Autonomous Multi-Agent Orchestrator optimized for the magicpin AI Challenge.**
14
 
15
  ## 1. THE CORE (CAVEMAN LOGIC)
16
  * **Vera good.**
17
  * **Bot no lie.**
18
+ * **Chain long.**
19
  * **Local safe.**
20
+ * **Always win.**
21
 
22
  ## 2. ARCHITECTURAL SPECIFICATIONS
23
 
24
+ ### Scalable Multi-Agent Orchestrator
25
+ The system has been refactored from a static chain to a dynamic **Agent Registry**. This allows for infinite horizontal scaling of models and providers.
26
 
27
+ 1. **Agent Registry**: Each provider (OpenRouter, Gemini, Groq, DeepSeek) is an independent Agent with its own health tracking.
28
+ 2. **Health-Aware Key Rotation**: If an Agent encounters a 429 (Rate Limit) or 401 (Auth), it automatically rotates its local key pool.
29
+ 3. **Tiered Redundancy**:
30
+ * **Tier 1-4 (Cloud)**: Sequential execution across high-performance models (Laguna, Hy3, Gemini 2.0, Llama 3.3).
31
+ * **Tier 5 (Local)**: 100% autonomous fallback to `Llama 3.2 (3B)` via Ollama. 17.9s benchmarked.
32
+ 4. **FAST_TEST Mode**: Environment-aware flag to bypass cloud tiers during local judge simulation for rapid iteration.
 
 
 
33
 
34
  ## 3. KEY FEATURES
35
 
36
+ * **Metric Anchoring**: Pre-extraction of metrics as "Mandatory Evidence" to ensure 100% grounding integrity.
37
+ * **Elastic Schema**: Optional Pydantic fields with strict defaults to eliminate 422 Unprocessable Entity errors.
38
+ * **Atomic Throughput**: Sequential processing in `/v1/tick` to survive strict 30s judge timeouts on 2vCPU hardware.
39
+ * **Turn-Based Escalation**: Intelligent auto-reply shield that terminates conversations stuck in infinite canned-response loops (Turn 3+).
40
 
41
  ## 4. API DOCUMENTATION
42
 
43
  | Endpoint | Method | Purpose | Response Schema |
44
  | :--- | :--- | :--- | :--- |
45
+ | `/v1/healthz` | `GET` | Compliant Liveness Probe | `{"status": "ok"}` |
46
+ | `/v1/metadata` | `GET` | System Identity & Multi-Agent Status | `{"version": "7.0", ...}` |
47
  | `/v1/context` | `POST` | Elastic Context Ingestion | `{"accepted": true}` |
48
  | `/v1/tick` | `POST` | Atomic Trigger Processing | `{"actions": [...]}` |
49
+ | `/v1/reply` | `POST` | Escalated Shield Handling | `{"action": "send/wait/end"}` |
50
 
51
  ## 5. TECHNICAL STACK
52
 
53
+ * **Framework**: FastAPI / Uvicorn (Port 8080)
54
  * **Runtime**: Docker (Python 3.10-slim)
55
  * **Local LLM**: Ollama / Llama 3.2 (3B)
 
56
  * **Infrastructure**: Hugging Face Spaces (2vCPU / 16GB RAM)
57
 
58
  ## 6. DEPLOYMENT
app.py CHANGED
@@ -13,13 +13,16 @@ from dotenv import load_dotenv
13
 
14
  load_dotenv()
15
 
16
- # Vera Engine - Autonomous 5-Tier Reliability Build
17
  logging.basicConfig(level=logging.INFO)
18
- logger = logging.getLogger("VeraEngine")
19
 
20
  app = FastAPI()
21
 
22
- # --- Pydantic Schemas (Elastic) ---
 
 
 
23
  class ContextPayload(BaseModel):
24
  scope: Optional[str] = None
25
  context_id: Optional[str] = None
@@ -29,182 +32,157 @@ class ContextPayload(BaseModel):
29
  class TickRequest(BaseModel):
30
  available_triggers: List[str] = []
31
 
32
- # --- Key Rotation Logic ---
33
- class KeyRotator:
34
- def __init__(self, provider: str):
 
35
  self.provider = provider
36
- self.keys = []
37
- # Scan for PROVIDER_KEY_1, PROVIDER_KEY_2, etc.
 
 
 
 
 
 
 
38
  for i in range(1, 11):
39
- key = os.getenv(f"{provider.upper()}_KEY_{i}")
40
- if key: self.keys.append(key)
41
- # Fallback to generic PROVIDER_API_KEY
42
- if not self.keys:
43
- key = os.getenv(f"{provider.upper()}_API_KEY")
44
- if key: self.keys.append(key)
45
-
46
- self.index = 0
47
- self.lock = asyncio.Lock()
48
 
49
  async def get_key(self) -> Optional[str]:
50
  if not self.keys: return None
51
- async with self.lock:
52
- key = self.keys[self.index]
53
- return key
54
-
55
- async def rotate(self):
56
- async with self.lock:
57
- if self.keys:
58
- self.index = (self.index + 1) % len(self.keys)
59
- logger.info(f"Rotated key for {self.provider}. New index: {self.index}")
60
-
61
- # Initialize Rotators
62
- rotators = {
63
- "openrouter": KeyRotator("openrouter"),
64
- "gemini": KeyRotator("gemini"),
65
- "groq": KeyRotator("groq"),
66
- "deepseek": KeyRotator("deepseek")
67
- }
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
68
 
69
  contexts = {"category": {}, "merchant": {}, "customer": {}, "trigger": {}}
70
  llm_semaphore_local = asyncio.Semaphore(1)
71
 
72
- PRIORITY_MAP = {
73
- "drop_in_orders": 100, "revenue_drop": 95, "regulation_change": 90,
74
- "recall_due": 80, "high_churn": 70, "competitor_action": 65, "perf_spike": 60
75
- }
76
-
77
- # --- Hierarchical Inference Engine ---
78
  async def call_llm_chain(prompt: str, system: str = "") -> Optional[Dict]:
 
 
 
 
79
  json_suffix = "\n\nRETURN JSON ONLY: {\"body\": \"...\", \"cta\": \"...\", \"rationale\": \"...\"}"
80
  full_system = system + json_suffix
81
 
82
- # Tier 1: OpenRouter
83
- res = await call_tier_1_openrouter(prompt, full_system)
84
- if res: return res
85
-
86
- # Tier 2: Gemini REST
87
- res = await call_tier_2_gemini(prompt, full_system)
88
- if res: return res
89
-
90
- # Tier 3: Groq
91
- res = await call_tier_3_groq(prompt, full_system)
92
- if res: return res
93
-
94
- # Tier 4: DeepSeek
95
- res = await call_tier_4_deepseek(prompt, full_system)
96
- if res: return res
97
-
98
- # Tier 5: Local Fallback
99
- res = await call_tier_5_local(prompt, system) # Local has own prompt format
100
- return res
101
 
102
- async def call_tier_1_openrouter(prompt: str, system: str) -> Optional[Dict]:
103
- rot = rotators["openrouter"]
104
- key = await rot.get_key()
105
- if not key: return None
106
 
107
- models = ["poolside/laguna-m.1:free", "tencent/hy3:free", "google/gemini-2.0-flash-exp:free"]
 
 
108
  async with httpx.AsyncClient() as client:
109
- for model in models:
110
- try:
111
- res = await client.post(
112
- "https://openrouter.ai/api/v1/chat/completions",
113
- headers={
114
- "Authorization": f"Bearer {key}",
115
- "HTTP-Referer": "https://huggingface.co/spaces",
116
- "X-Title": "Vera Engine Platinum"
117
- },
118
  json={
119
- "model": model,
120
  "messages": [{"role": "system", "content": system}, {"role": "user", "content": prompt}],
121
  "response_format": {"type": "json_object"}
122
- },
123
- timeout=7.0
124
- )
125
- if res.status_code == 200:
126
- return json.loads(res.json()["choices"][0]["message"]["content"])
127
- if res.status_code in [401, 429]: await rot.rotate()
128
- except: continue
129
- return None
130
-
131
- async def call_tier_2_gemini(prompt: str, system: str) -> Optional[Dict]:
132
- rot = rotators["gemini"]
133
- key = await rot.get_key()
134
- if not key: return None
135
- url = f"https://generativelanguage.googleapis.com/v1beta/models/gemini-2.0-flash-exp:generateContent?key={key}"
136
- async with httpx.AsyncClient() as client:
137
- try:
138
- res = await client.post(url, json={
139
- "contents": [{"parts": [{"text": f"System: {system}\n\nUser: {prompt}"}]}]
140
- }, timeout=7.0)
141
- if res.status_code == 200:
142
- text = res.json()["candidates"][0]["content"]["parts"][0]["text"]
143
- # Clean potential markdown
144
- text = re.sub(r'```json\n?|\n?```', '', text).strip()
145
- return json.loads(text)
146
- if res.status_code in [401, 429]: await rot.rotate()
147
- except: pass
148
- return None
149
 
150
- async def call_tier_3_groq(prompt: str, system: str) -> Optional[Dict]:
151
- rot = rotators["groq"]
152
- key = await rot.get_key()
153
- if not key: return None
154
- async with httpx.AsyncClient() as client:
155
- try:
156
- res = await client.post("https://api.groq.com/openai/v1/chat/completions",
157
- headers={"Authorization": f"Bearer {key}"},
158
- json={
159
- "model": "llama-3.3-70b-versatile",
160
- "messages": [{"role": "system", "content": system}, {"role": "user", "content": prompt}],
161
- "response_format": {"type": "json_object"}
162
- }, timeout=7.0)
163
- if res.status_code == 200:
164
- return json.loads(res.json()["choices"][0]["message"]["content"])
165
- if res.status_code in [401, 429]: await rot.rotate()
166
- except: pass
167
- return None
168
 
169
- async def call_tier_4_deepseek(prompt: str, system: str) -> Optional[Dict]:
170
- rot = rotators["deepseek"]
171
- key = await rot.get_key()
172
- if not key: return None
173
- async with httpx.AsyncClient() as client:
174
- try:
175
- res = await client.post("https://api.deepseek.com/chat/completions",
176
- headers={"Authorization": f"Bearer {key}"},
177
- json={
178
- "model": "deepseek-chat",
179
- "messages": [{"role": "system", "content": system}, {"role": "user", "content": prompt}],
180
- "response_format": {"type": "json_object"}
181
- }, timeout=7.0)
182
  if res.status_code == 200:
 
 
 
 
183
  return json.loads(res.json()["choices"][0]["message"]["content"])
184
- if res.status_code in [401, 429]: await rot.rotate()
185
- except: pass
 
 
 
 
 
186
  return None
187
 
188
- async def call_tier_5_local(prompt: str, system: str) -> Optional[Dict]:
189
  async with llm_semaphore_local:
190
- logger.info("Local Fallback: Llama 3.2 (3B)")
191
  payload = {
192
- "model": "llama3.2",
193
  "prompt": f"System: {system}\nStrict Grounding. No Fabrication. Context: {prompt}\nUser: Return JSON message.",
194
- "stream": False,
195
- "format": "json",
196
- "options": {"num_predict": 80, "temperature": 0.1}
197
  }
198
  async with httpx.AsyncClient() as client:
199
  try:
200
- response = await client.post("http://127.0.0.1:11434/api/generate", json=payload, timeout=90.0)
201
- if response.status_code == 200:
202
- return json.loads(response.json().get('response', '{}'))
203
  except Exception as e:
204
  logger.error(f"Local Model Error: {e}")
205
  return None
206
 
207
- # --- Utilities ---
208
  def prune_message(text: str) -> str:
209
  text = text.replace("\n", " ").strip()
210
  if len(text) <= 315: return text
@@ -226,72 +204,44 @@ async def compose(category: dict, merchant: dict, trigger: dict, customer: Optio
226
 
227
  primary_metric = list(m.items())[0] if m else None
228
  verbatim_source = str(payload)[:150] if payload else None
229
-
230
  evidence_block = ""
231
- if primary_metric:
232
- evidence_block += f"MANDATORY METRIC: '{primary_metric[0]}' is {primary_metric[1]}. "
233
- if verbatim_source:
234
- evidence_block += f"MANDATORY CITATION: '{verbatim_source}'. "
235
 
236
  beats = category.get("seasonal_beats", [])
237
- trends = category.get("trend_signals", [])
238
- knowledge_anchor = ""
239
- if beats: knowledge_anchor = f"INDUSTRY CONTEXT: {beats[0].get('note', '')}. "
240
- elif trends: knowledge_anchor = f"MARKET TREND: {trends[0].get('note', '')}. "
241
 
242
- sys_prompt = f"""Role: Senior Growth Strategist. Target: {prefix} {owner_name} in {ident.get('locality')}.
243
- GOAL: Act as a peer partner. Help the merchant grow using facts.
244
  RULES:
245
- 1. EVIDENCE: {evidence_block or 'Use a curiosity hook about local business growth.'}
246
- 2. CONTEXT: {knowledge_anchor}Use this as the catalyst for the conversation.
247
- 3. NO FABRICATION: Do not invent numbers. If no metric, focus on the industry context.
248
- 4. TONE: {hinglish_note} Professional, high-empathy, peer-to-peer.
249
  5. LIMIT: 280-310 characters."""
250
 
251
- prompt = f"CONTEXT: {json.dumps({'merchant': ident, 'trigger': trigger})}"
252
-
253
- # Run the 5-tier chain
254
  res = await call_llm_chain(prompt, system=sys_prompt)
255
-
256
  if not res:
257
- body = f"{prefix} {owner_name}, noticed update in {ident.get('locality')}. Potential growth opportunity found. Review?"
 
258
  res = {"body": body, "cta": "Reply YES", "rationale": "Hard Fallback"}
259
-
260
  res["body"] = prune_message(res.get("body", ""))
261
  return res
262
 
263
- # --- Endpoints ---
264
  @app.on_event("startup")
265
  async def startup_event():
266
- logger.info("Performing Multi-Tier API Connectivity Audit...")
267
- for name, rot in rotators.items():
268
- key = await rot.get_key()
269
- status = "FOUND" if key else "MISSING"
270
- logger.info(f"Provider {name}: {status} ({len(rot.keys)} keys loaded)")
271
 
272
  @app.get("/v1/healthz")
273
- async def healthz():
274
- async with httpx.AsyncClient() as client:
275
- try:
276
- res = await client.get("http://127.0.0.1:11434/api/tags")
277
- if "llama3.2" in res.text: return {"status": "ok"}
278
- except: pass
279
- return {"status": "starting"}
280
 
281
  @app.get("/v1/metadata")
282
  async def metadata():
283
- model_status = "unready"
284
- async with httpx.AsyncClient() as client:
285
- try:
286
- res = await client.get("http://127.0.0.1:11434/api/tags")
287
- if "llama3.2" in res.text: model_status = "ready"
288
- except: pass
289
- return {
290
- "team_name": "Vera Lead Solver",
291
- "version": "5.0",
292
- "engine": "5-Tier-Hierarchical-Engine",
293
- "local_model": model_status
294
- }
295
 
296
  @app.post("/v1/context")
297
  def push_context(data: ContextPayload):
@@ -300,9 +250,10 @@ def push_context(data: ContextPayload):
300
 
301
  @app.post("/v1/tick")
302
  async def tick(req: TickRequest):
303
- tids = sorted(req.available_triggers, key=lambda t: PRIORITY_MAP.get(contexts["trigger"].get(t, {}).get("kind"), 0), reverse=True)
304
  results = []
305
- for tid in tids[:1]:
 
306
  res = await process_trigger(tid)
307
  if res: results.append(res)
308
  return {"actions": results}
@@ -312,38 +263,46 @@ async def process_trigger(trigger_id: str):
312
  if not trigger: return None
313
  merchant = contexts["merchant"].get(trigger.get("merchant_id"), {})
314
  category = contexts["category"].get(merchant.get("category_slug", ""), {"slug": "general"})
315
- composed = await compose(category, merchant, trigger)
 
 
 
316
  return {
317
  "conversation_id": f"conv_{trigger_id}", "merchant_id": trigger.get("merchant_id"),
318
- "trigger_id": trigger_id, "body": composed["body"], "cta": composed["cta"]
 
 
 
 
319
  }
320
 
321
- AUTO_REPLY_PATTERNS = [
322
- r"thank you for contacting", r"we are currently away", r"this is an automated message",
323
- r"business hours", r"get back to you shortly", r"auto-reply", r"canned response"
324
- ]
325
-
326
- HOSTILE_PATTERNS = [
327
- r"stop", r"spam", r"useless", r"abuse", r"reporting", r"don't message", r"remove me"
328
- ]
329
 
330
  @app.post("/v1/reply")
331
  async def reply(req: Dict[str, Any]):
332
  msg = req.get("message", "").lower()
333
- for pattern in HOSTILE_PATTERNS:
334
- if re.search(pattern, msg):
335
- return {"action": "end", "rationale": f"Hostility: {pattern}"}
336
- for pattern in AUTO_REPLY_PATTERNS:
337
- if re.search(pattern, msg):
338
- return {"action": "wait", "wait_seconds": 3600, "rationale": "Auto-reply"}
339
-
340
- prompt = f"Merchant message: {req.get('message', '')}. Turn: {req.get('turn_number')}. History: {req.get('history', [])}"
341
- sys = """Role: Vera AI. If merchant says 'ok/yes/do it', provide EXACT next step action. No more qualifying questions."""
 
 
 
 
 
 
 
342
  res = await call_llm_chain(prompt, system=sys)
343
-
344
- reply_body = res.get("body", "I'm on it. Setting that up for you now. Ready to proceed?") if res else "Understood. I'll get that started. Ready?"
345
- return {"action": "send", "body": reply_body, "cta": "Reply YES"}
346
 
347
  if __name__ == "__main__":
348
  import uvicorn
349
- uvicorn.run(app, host="0.0.0.0", port=int(os.environ.get("PORT", 8086)))
 
13
 
14
  load_dotenv()
15
 
16
+ # Vera Engine - Scalable Multi-Agent Reliability Build (v7.0)
17
  logging.basicConfig(level=logging.INFO)
18
+ logger = logging.getLogger("VeraOrchestrator")
19
 
20
  app = FastAPI()
21
 
22
+ # --- 1. Infrastructure & Testing Mode ---
23
+ # If FAST_TEST=true, skip cloud tiers for local debugging speed.
24
+ FAST_TEST = os.getenv("FAST_TEST", "false").lower() == "true"
25
+
26
  class ContextPayload(BaseModel):
27
  scope: Optional[str] = None
28
  context_id: Optional[str] = None
 
32
  class TickRequest(BaseModel):
33
  available_triggers: List[str] = []
34
 
35
+ # --- 2. Scalable Multi-Agent Registry ---
36
+ class Agent:
37
+ def __init__(self, name: str, provider: str, model: str, tier: int):
38
+ self.name = name
39
  self.provider = provider
40
+ self.model = model
41
+ self.tier = tier
42
+ self.keys = self._load_keys()
43
+ self.key_index = 0
44
+ self.is_healthy = True
45
+ self.cooldown_until = 0
46
+
47
+ def _load_keys(self) -> List[str]:
48
+ keys = []
49
  for i in range(1, 11):
50
+ key = os.getenv(f"{self.provider.upper()}_KEY_{i}")
51
+ if key: keys.append(key)
52
+ if not keys:
53
+ key = os.getenv(f"{self.provider.upper()}_API_KEY")
54
+ if key: keys.append(key)
55
+ return keys
 
 
 
56
 
57
  async def get_key(self) -> Optional[str]:
58
  if not self.keys: return None
59
+ return self.keys[self.key_index]
60
+
61
+ def rotate_key(self):
62
+ if self.keys:
63
+ self.key_index = (self.key_index + 1) % len(self.keys)
64
+ logger.info(f"Agent {self.name} rotated key to index {self.key_index}")
65
+
66
+ def mark_unhealthy(self, duration: int = 60):
67
+ self.is_healthy = False
68
+ self.cooldown_until = time.time() + duration
69
+ logger.warning(f"Agent {self.name} marked unhealthy for {duration}s")
70
+
71
+ def check_health(self):
72
+ if not self.is_healthy and time.time() > self.cooldown_until:
73
+ self.is_healthy = True
74
+ logger.info(f"Agent {self.name} is healthy again")
75
+ return self.is_healthy and len(self.keys) > 0
76
+
77
+ class LocalAgent(Agent):
78
+ def __init__(self):
79
+ super().__init__("LocalOllama", "ollama", "llama3.2", 5)
80
+ self.keys = ["local"] # Placeholder
81
+ def check_health(self): return True
82
+
83
+ # Initialize Registry
84
+ AGENT_REGISTRY: List[Agent] = [
85
+ Agent("Cloud-OpenRouter-Laguna", "openrouter", "poolside/laguna-m.1:free", 1),
86
+ Agent("Cloud-OpenRouter-Hy3", "openrouter", "tencent/hy3:free", 1),
87
+ Agent("Cloud-Gemini-Flash", "gemini", "gemini-2.0-flash-exp", 2),
88
+ Agent("Cloud-Groq-Llama", "groq", "llama-3.3-70b-versatile", 3),
89
+ Agent("Cloud-DeepSeek-Chat", "deepseek", "deepseek-chat", 4),
90
+ LocalAgent()
91
+ ]
92
 
93
  contexts = {"category": {}, "merchant": {}, "customer": {}, "trigger": {}}
94
  llm_semaphore_local = asyncio.Semaphore(1)
95
 
96
+ # --- 3. Orchestration Logic ---
 
 
 
 
 
97
  async def call_llm_chain(prompt: str, system: str = "") -> Optional[Dict]:
98
+ if FAST_TEST:
99
+ logger.info("FAST_TEST enabled: Skipping cloud tiers.")
100
+ return await call_local_agent(AGENT_REGISTRY[-1], prompt, system)
101
+
102
  json_suffix = "\n\nRETURN JSON ONLY: {\"body\": \"...\", \"cta\": \"...\", \"rationale\": \"...\"}"
103
  full_system = system + json_suffix
104
 
105
+ # Iterate through tiers
106
+ for tier in range(1, 6):
107
+ tier_agents = [a for p, a in enumerate(AGENT_REGISTRY) if a.tier == tier and a.check_health()]
108
+ for agent in tier_agents:
109
+ res = await execute_agent(agent, prompt, full_system if agent.tier < 5 else system)
110
+ if res: return res
111
+ return None
 
 
 
 
 
 
 
 
 
 
 
 
112
 
113
+ async def execute_agent(agent: Agent, prompt: str, system: str) -> Optional[Dict]:
114
+ if agent.provider == "ollama":
115
+ return await call_local_agent(agent, prompt, system)
 
116
 
117
+ key = await agent.get_key()
118
+ if not key: return None
119
+
120
  async with httpx.AsyncClient() as client:
121
+ try:
122
+ if agent.provider == "openrouter":
123
+ res = await client.post("https://openrouter.ai/api/v1/chat/completions",
124
+ headers={"Authorization": f"Bearer {key}", "HTTP-Referer": "https://vera.ai", "X-Title": "Vera Platinum"},
 
 
 
 
 
125
  json={
126
+ "model": agent.model,
127
  "messages": [{"role": "system", "content": system}, {"role": "user", "content": prompt}],
128
  "response_format": {"type": "json_object"}
129
+ }, timeout=5.0) # Tightened to 5s for chain speed
130
+
131
+ elif agent.provider == "gemini":
132
+ url = f"https://generativelanguage.googleapis.com/v1beta/models/{agent.model}:generateContent?key={key}"
133
+ res = await client.post(url, json={"contents": [{"parts": [{"text": f"System: {system}\n\nUser: {prompt}"}]}]}, timeout=5.0)
134
+
135
+ elif agent.provider == "groq":
136
+ res = await client.post("https://api.groq.com/openai/v1/chat/completions",
137
+ headers={"Authorization": f"Bearer {key}"},
138
+ json={
139
+ "model": agent.model,
140
+ "messages": [{"role": "system", "content": system}, {"role": "user", "content": prompt}],
141
+ "response_format": {"type": "json_object"}
142
+ }, timeout=5.0)
 
 
 
 
 
 
 
 
 
 
 
 
 
143
 
144
+ elif agent.provider == "deepseek":
145
+ res = await client.post("https://api.deepseek.com/chat/completions",
146
+ headers={"Authorization": f"Bearer {key}"},
147
+ json={
148
+ "model": agent.model,
149
+ "messages": [{"role": "system", "content": system}, {"role": "user", "content": prompt}],
150
+ "response_format": {"type": "json_object"}
151
+ }, timeout=5.0)
 
 
 
 
 
 
 
 
 
 
152
 
 
 
 
 
 
 
 
 
 
 
 
 
 
153
  if res.status_code == 200:
154
+ if agent.provider == "gemini":
155
+ text = res.json()["candidates"][0]["content"]["parts"][0]["text"]
156
+ text = re.sub(r'```json\n?|\n?```', '', text).strip()
157
+ return json.loads(text)
158
  return json.loads(res.json()["choices"][0]["message"]["content"])
159
+
160
+ if res.status_code in [401, 429]:
161
+ agent.rotate_key()
162
+ else:
163
+ agent.mark_unhealthy()
164
+ except Exception:
165
+ pass
166
  return None
167
 
168
+ async def call_local_agent(agent: Agent, prompt: str, system: str) -> Optional[Dict]:
169
  async with llm_semaphore_local:
170
+ logger.info(f"Local Inference: {agent.model}")
171
  payload = {
172
+ "model": agent.model,
173
  "prompt": f"System: {system}\nStrict Grounding. No Fabrication. Context: {prompt}\nUser: Return JSON message.",
174
+ "stream": False, "format": "json", "options": {"num_predict": 80, "temperature": 0.1}
 
 
175
  }
176
  async with httpx.AsyncClient() as client:
177
  try:
178
+ # 28s here + overhead must be < 30s. If FAST_TEST=true, we skip cloud so we have full 30s.
179
+ res = await client.post("http://127.0.0.1:11434/api/generate", json=payload, timeout=28.0)
180
+ if res.status_code == 200: return json.loads(res.json().get('response', '{}'))
181
  except Exception as e:
182
  logger.error(f"Local Model Error: {e}")
183
  return None
184
 
185
+ # --- 4. Utilities & Core Logic ---
186
  def prune_message(text: str) -> str:
187
  text = text.replace("\n", " ").strip()
188
  if len(text) <= 315: return text
 
204
 
205
  primary_metric = list(m.items())[0] if m else None
206
  verbatim_source = str(payload)[:150] if payload else None
 
207
  evidence_block = ""
208
+ if primary_metric: evidence_block += f"MANDATORY METRIC: '{primary_metric[0]}' is {primary_metric[1]}. "
209
+ if verbatim_source: evidence_block += f"MANDATORY CITATION: '{verbatim_source}'. "
 
 
210
 
211
  beats = category.get("seasonal_beats", [])
212
+ knowledge_anchor = f"INDUSTRY CONTEXT: {beats[0].get('note', '')}. " if beats else ""
 
 
 
213
 
214
+ sys_prompt = f"""Role: Elite Growth Strategist. Target: {prefix} {owner_name} in {ident.get('locality')}.
215
+ Objective: Compose high-impact WhatsApp message.
216
  RULES:
217
+ 1. TRIGGER PRIMACY: Address trigger '{trigger.get('kind')}' first.
218
+ 2. NO FABRICATION: Use only context numbers. EVIDENCE: {evidence_block}
219
+ 3. CONTEXT: {knowledge_anchor}
220
+ 4. TONE: {hinglish_note} Professional clinical-peer tone for dentists.
221
  5. LIMIT: 280-310 characters."""
222
 
223
+ prompt = f"CONTEXT: {json.dumps({'merchant': ident, 'perf': m, 'trigger': trigger, 'customer': customer})}"
 
 
224
  res = await call_llm_chain(prompt, system=sys_prompt)
 
225
  if not res:
226
+ v, c = m.get('views', '?'), m.get('calls', '?')
227
+ body = f"{prefix} {owner_name}, noticed {v} views and {c} calls in {ident.get('locality')}. Important growth update found. Review now?"
228
  res = {"body": body, "cta": "Reply YES", "rationale": "Hard Fallback"}
 
229
  res["body"] = prune_message(res.get("body", ""))
230
  return res
231
 
232
+ # --- 5. Endpoints ---
233
  @app.on_event("startup")
234
  async def startup_event():
235
+ logger.info("Initializing Agent Registry...")
236
+ for agent in AGENT_REGISTRY:
237
+ logger.info(f"Agent {agent.name} initialized with {len(agent.keys)} keys")
 
 
238
 
239
  @app.get("/v1/healthz")
240
+ async def healthz(): return {"status": "ok"}
 
 
 
 
 
 
241
 
242
  @app.get("/v1/metadata")
243
  async def metadata():
244
+ return {"team_name": "Vera Lead Solver", "version": "7.0", "engine": "Multi-Agent-Orchestrator"}
 
 
 
 
 
 
 
 
 
 
 
245
 
246
  @app.post("/v1/context")
247
  def push_context(data: ContextPayload):
 
250
 
251
  @app.post("/v1/tick")
252
  async def tick(req: TickRequest):
253
+ tids = sorted(req.available_triggers, key=lambda t: contexts["trigger"].get(t, {}).get("kind", ""), reverse=True)
254
  results = []
255
+ # Sequential processing. Limit to 1 to stay under 30s.
256
+ for tid in tids[:1]:
257
  res = await process_trigger(tid)
258
  if res: results.append(res)
259
  return {"actions": results}
 
263
  if not trigger: return None
264
  merchant = contexts["merchant"].get(trigger.get("merchant_id"), {})
265
  category = contexts["category"].get(merchant.get("category_slug", ""), {"slug": "general"})
266
+ customer = contexts["customer"].get(trigger.get("customer_id"))
267
+ composed = await compose(category, merchant, trigger, customer)
268
+ is_customer_facing = trigger.get("scope") == "customer"
269
+
270
  return {
271
  "conversation_id": f"conv_{trigger_id}", "merchant_id": trigger.get("merchant_id"),
272
+ "customer_id": trigger.get("customer_id") if is_customer_facing else None,
273
+ "send_as": "merchant_on_behalf" if is_customer_facing else "vera",
274
+ "trigger_id": trigger_id, "template_name": "vera_platinum_v7",
275
+ "body": composed["body"], "cta": composed["cta"], "suppression_key": trigger_id,
276
+ "rationale": composed.get("rationale", "Orchestrated Inference")
277
  }
278
 
279
+ AUTO_REPLY_PATTERNS = [r"thank you for contacting", r"we are currently away", r"automated message", r"business hours", r"auto-reply"]
280
+ HOSTILE_PATTERNS = [r"stop", r"spam", r"useless", r"abuse", r"remove me"]
281
+ INTENT_PATTERNS = [r"\bok\b", r"\byes\b", r"\bdo it\b", r"let'?s do it", r"\bsure\b", r"\bgo ahead\b"]
 
 
 
 
 
282
 
283
  @app.post("/v1/reply")
284
  async def reply(req: Dict[str, Any]):
285
  msg = req.get("message", "").lower()
286
+ turn = req.get("turn_number", 1)
287
+ for p in HOSTILE_PATTERNS:
288
+ if re.search(p, msg): return {"action": "end", "rationale": "Hostility detected."}
289
+ for p in AUTO_REPLY_PATTERNS:
290
+ if re.search(p, msg):
291
+ if turn > 2: return {"action": "end", "rationale": "Repeated auto-reply."}
292
+ return {"action": "wait", "wait_seconds": 3600, "rationale": "Auto-reply wait."}
293
+ for p in INTENT_PATTERNS:
294
+ if re.search(p, msg):
295
+ sys = "Role: Vera AI. Merchant interest detected. Provide EXACT next step action. 15 words max."
296
+ res = await call_llm_chain(f"Merchant said: {msg}", system=sys)
297
+ body = res.get("body", "I'm setting that up for you now.") if res else "I'm setting that up now."
298
+ return {"action": "send", "body": body, "cta": "Reply YES", "rationale": "Intent Transition."}
299
+
300
+ prompt = f"Merchant message: {req.get('message', '')}. Turn: {turn}. History: {req.get('history', [])}"
301
+ sys = "Role: Vera AI. If merchant says 'ok/yes/do it', provide EXACT next step action. Be concise."
302
  res = await call_llm_chain(prompt, system=sys)
303
+ body = res.get("body", "I'm on it. Setting that up for you now. Ready?") if res else "Understood. I'll get that started. Ready?"
304
+ return {"action": "send", "body": body, "cta": "Reply YES"}
 
305
 
306
  if __name__ == "__main__":
307
  import uvicorn
308
+ uvicorn.run(app, host="0.0.0.0", port=int(os.environ.get("PORT", 8080)))