Spaces:
Running
Running
Rachit-Mangawa commited on
Commit ·
08b98ed
1
Parent(s): 42c83f7
feat: Scalable Multi-Agent Orchestrator (v7.0) with Multi-Key Rotation and FAST_TEST mode
Browse files
README.md
CHANGED
|
@@ -4,59 +4,55 @@ emoji: 🚀
|
|
| 4 |
colorFrom: blue
|
| 5 |
colorTo: purple
|
| 6 |
sdk: docker
|
| 7 |
-
app_port:
|
| 8 |
pinned: false
|
| 9 |
---
|
| 10 |
|
| 11 |
-
# Vera Engine Platinum
|
| 12 |
|
| 13 |
-
**
|
| 14 |
|
| 15 |
## 1. THE CORE (CAVEMAN LOGIC)
|
| 16 |
* **Vera good.**
|
| 17 |
* **Bot no lie.**
|
| 18 |
-
* **
|
| 19 |
* **Local safe.**
|
| 20 |
-
* **
|
| 21 |
|
| 22 |
## 2. ARCHITECTURAL SPECIFICATIONS
|
| 23 |
|
| 24 |
-
###
|
| 25 |
-
The system
|
| 26 |
|
| 27 |
-
1. **
|
| 28 |
-
|
| 29 |
-
|
| 30 |
-
|
| 31 |
-
* **
|
| 32 |
-
|
| 33 |
-
|
| 34 |
-
### The Shield (Interceptor)
|
| 35 |
-
Deterministic regex-based filtering of merchant auto-replies and hostile inputs. Prevents compute thrashing and ensures turn-efficiency.
|
| 36 |
|
| 37 |
## 3. KEY FEATURES
|
| 38 |
|
| 39 |
-
* **Metric Anchoring**:
|
| 40 |
-
* **Elastic
|
| 41 |
-
* **
|
| 42 |
-
* **
|
| 43 |
|
| 44 |
## 4. API DOCUMENTATION
|
| 45 |
|
| 46 |
| Endpoint | Method | Purpose | Response Schema |
|
| 47 |
| :--- | :--- | :--- | :--- |
|
| 48 |
-
| `/v1/healthz` | `GET` |
|
| 49 |
-
| `/v1/metadata` | `GET` | System Identity &
|
| 50 |
| `/v1/context` | `POST` | Elastic Context Ingestion | `{"accepted": true}` |
|
| 51 |
| `/v1/tick` | `POST` | Atomic Trigger Processing | `{"actions": [...]}` |
|
| 52 |
-
| `/v1/reply` | `POST` |
|
| 53 |
|
| 54 |
## 5. TECHNICAL STACK
|
| 55 |
|
| 56 |
-
* **Framework**: FastAPI / Uvicorn
|
| 57 |
* **Runtime**: Docker (Python 3.10-slim)
|
| 58 |
* **Local LLM**: Ollama / Llama 3.2 (3B)
|
| 59 |
-
* **Cloud Gateway**: OpenRouter
|
| 60 |
* **Infrastructure**: Hugging Face Spaces (2vCPU / 16GB RAM)
|
| 61 |
|
| 62 |
## 6. DEPLOYMENT
|
|
|
|
| 4 |
colorFrom: blue
|
| 5 |
colorTo: purple
|
| 6 |
sdk: docker
|
| 7 |
+
app_port: 8080
|
| 8 |
pinned: false
|
| 9 |
---
|
| 10 |
|
| 11 |
+
# Vera Engine Platinum v7.0 🤖
|
| 12 |
|
| 13 |
+
**Autonomous Multi-Agent Orchestrator optimized for the magicpin AI Challenge.**
|
| 14 |
|
| 15 |
## 1. THE CORE (CAVEMAN LOGIC)
|
| 16 |
* **Vera good.**
|
| 17 |
* **Bot no lie.**
|
| 18 |
+
* **Chain long.**
|
| 19 |
* **Local safe.**
|
| 20 |
+
* **Always win.**
|
| 21 |
|
| 22 |
## 2. ARCHITECTURAL SPECIFICATIONS
|
| 23 |
|
| 24 |
+
### Scalable Multi-Agent Orchestrator
|
| 25 |
+
The system has been refactored from a static chain to a dynamic **Agent Registry**. This allows for infinite horizontal scaling of models and providers.
|
| 26 |
|
| 27 |
+
1. **Agent Registry**: Each provider (OpenRouter, Gemini, Groq, DeepSeek) is an independent Agent with its own health tracking.
|
| 28 |
+
2. **Health-Aware Key Rotation**: If an Agent encounters a 429 (Rate Limit) or 401 (Auth), it automatically rotates its local key pool.
|
| 29 |
+
3. **Tiered Redundancy**:
|
| 30 |
+
* **Tier 1-4 (Cloud)**: Sequential execution across high-performance models (Laguna, Hy3, Gemini 2.0, Llama 3.3).
|
| 31 |
+
* **Tier 5 (Local)**: 100% autonomous fallback to `Llama 3.2 (3B)` via Ollama. 17.9s benchmarked.
|
| 32 |
+
4. **FAST_TEST Mode**: Environment-aware flag to bypass cloud tiers during local judge simulation for rapid iteration.
|
|
|
|
|
|
|
|
|
|
| 33 |
|
| 34 |
## 3. KEY FEATURES
|
| 35 |
|
| 36 |
+
* **Metric Anchoring**: Pre-extraction of metrics as "Mandatory Evidence" to ensure 100% grounding integrity.
|
| 37 |
+
* **Elastic Schema**: Optional Pydantic fields with strict defaults to eliminate 422 Unprocessable Entity errors.
|
| 38 |
+
* **Atomic Throughput**: Sequential processing in `/v1/tick` to survive strict 30s judge timeouts on 2vCPU hardware.
|
| 39 |
+
* **Turn-Based Escalation**: Intelligent auto-reply shield that terminates conversations stuck in infinite canned-response loops (Turn 3+).
|
| 40 |
|
| 41 |
## 4. API DOCUMENTATION
|
| 42 |
|
| 43 |
| Endpoint | Method | Purpose | Response Schema |
|
| 44 |
| :--- | :--- | :--- | :--- |
|
| 45 |
+
| `/v1/healthz` | `GET` | Compliant Liveness Probe | `{"status": "ok"}` |
|
| 46 |
+
| `/v1/metadata` | `GET` | System Identity & Multi-Agent Status | `{"version": "7.0", ...}` |
|
| 47 |
| `/v1/context` | `POST` | Elastic Context Ingestion | `{"accepted": true}` |
|
| 48 |
| `/v1/tick` | `POST` | Atomic Trigger Processing | `{"actions": [...]}` |
|
| 49 |
+
| `/v1/reply` | `POST` | Escalated Shield Handling | `{"action": "send/wait/end"}` |
|
| 50 |
|
| 51 |
## 5. TECHNICAL STACK
|
| 52 |
|
| 53 |
+
* **Framework**: FastAPI / Uvicorn (Port 8080)
|
| 54 |
* **Runtime**: Docker (Python 3.10-slim)
|
| 55 |
* **Local LLM**: Ollama / Llama 3.2 (3B)
|
|
|
|
| 56 |
* **Infrastructure**: Hugging Face Spaces (2vCPU / 16GB RAM)
|
| 57 |
|
| 58 |
## 6. DEPLOYMENT
|
app.py
CHANGED
|
@@ -13,13 +13,16 @@ from dotenv import load_dotenv
|
|
| 13 |
|
| 14 |
load_dotenv()
|
| 15 |
|
| 16 |
-
# Vera Engine -
|
| 17 |
logging.basicConfig(level=logging.INFO)
|
| 18 |
-
logger = logging.getLogger("
|
| 19 |
|
| 20 |
app = FastAPI()
|
| 21 |
|
| 22 |
-
# ---
|
|
|
|
|
|
|
|
|
|
| 23 |
class ContextPayload(BaseModel):
|
| 24 |
scope: Optional[str] = None
|
| 25 |
context_id: Optional[str] = None
|
|
@@ -29,182 +32,157 @@ class ContextPayload(BaseModel):
|
|
| 29 |
class TickRequest(BaseModel):
|
| 30 |
available_triggers: List[str] = []
|
| 31 |
|
| 32 |
-
# ---
|
| 33 |
-
class
|
| 34 |
-
def __init__(self, provider: str):
|
|
|
|
| 35 |
self.provider = provider
|
| 36 |
-
self.
|
| 37 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 38 |
for i in range(1, 11):
|
| 39 |
-
key = os.getenv(f"{provider.upper()}_KEY_{i}")
|
| 40 |
-
if key:
|
| 41 |
-
|
| 42 |
-
|
| 43 |
-
key
|
| 44 |
-
|
| 45 |
-
|
| 46 |
-
self.index = 0
|
| 47 |
-
self.lock = asyncio.Lock()
|
| 48 |
|
| 49 |
async def get_key(self) -> Optional[str]:
|
| 50 |
if not self.keys: return None
|
| 51 |
-
|
| 52 |
-
|
| 53 |
-
|
| 54 |
-
|
| 55 |
-
|
| 56 |
-
|
| 57 |
-
|
| 58 |
-
|
| 59 |
-
|
| 60 |
-
|
| 61 |
-
|
| 62 |
-
|
| 63 |
-
|
| 64 |
-
|
| 65 |
-
|
| 66 |
-
|
| 67 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 68 |
|
| 69 |
contexts = {"category": {}, "merchant": {}, "customer": {}, "trigger": {}}
|
| 70 |
llm_semaphore_local = asyncio.Semaphore(1)
|
| 71 |
|
| 72 |
-
|
| 73 |
-
"drop_in_orders": 100, "revenue_drop": 95, "regulation_change": 90,
|
| 74 |
-
"recall_due": 80, "high_churn": 70, "competitor_action": 65, "perf_spike": 60
|
| 75 |
-
}
|
| 76 |
-
|
| 77 |
-
# --- Hierarchical Inference Engine ---
|
| 78 |
async def call_llm_chain(prompt: str, system: str = "") -> Optional[Dict]:
|
|
|
|
|
|
|
|
|
|
|
|
|
| 79 |
json_suffix = "\n\nRETURN JSON ONLY: {\"body\": \"...\", \"cta\": \"...\", \"rationale\": \"...\"}"
|
| 80 |
full_system = system + json_suffix
|
| 81 |
|
| 82 |
-
#
|
| 83 |
-
|
| 84 |
-
|
| 85 |
-
|
| 86 |
-
|
| 87 |
-
|
| 88 |
-
|
| 89 |
-
|
| 90 |
-
# Tier 3: Groq
|
| 91 |
-
res = await call_tier_3_groq(prompt, full_system)
|
| 92 |
-
if res: return res
|
| 93 |
-
|
| 94 |
-
# Tier 4: DeepSeek
|
| 95 |
-
res = await call_tier_4_deepseek(prompt, full_system)
|
| 96 |
-
if res: return res
|
| 97 |
-
|
| 98 |
-
# Tier 5: Local Fallback
|
| 99 |
-
res = await call_tier_5_local(prompt, system) # Local has own prompt format
|
| 100 |
-
return res
|
| 101 |
|
| 102 |
-
async def
|
| 103 |
-
|
| 104 |
-
|
| 105 |
-
if not key: return None
|
| 106 |
|
| 107 |
-
|
|
|
|
|
|
|
| 108 |
async with httpx.AsyncClient() as client:
|
| 109 |
-
|
| 110 |
-
|
| 111 |
-
res = await client.post(
|
| 112 |
-
"https://
|
| 113 |
-
headers={
|
| 114 |
-
"Authorization": f"Bearer {key}",
|
| 115 |
-
"HTTP-Referer": "https://huggingface.co/spaces",
|
| 116 |
-
"X-Title": "Vera Engine Platinum"
|
| 117 |
-
},
|
| 118 |
json={
|
| 119 |
-
"model": model,
|
| 120 |
"messages": [{"role": "system", "content": system}, {"role": "user", "content": prompt}],
|
| 121 |
"response_format": {"type": "json_object"}
|
| 122 |
-
},
|
| 123 |
-
|
| 124 |
-
|
| 125 |
-
|
| 126 |
-
|
| 127 |
-
|
| 128 |
-
|
| 129 |
-
|
| 130 |
-
|
| 131 |
-
|
| 132 |
-
|
| 133 |
-
|
| 134 |
-
|
| 135 |
-
|
| 136 |
-
async with httpx.AsyncClient() as client:
|
| 137 |
-
try:
|
| 138 |
-
res = await client.post(url, json={
|
| 139 |
-
"contents": [{"parts": [{"text": f"System: {system}\n\nUser: {prompt}"}]}]
|
| 140 |
-
}, timeout=7.0)
|
| 141 |
-
if res.status_code == 200:
|
| 142 |
-
text = res.json()["candidates"][0]["content"]["parts"][0]["text"]
|
| 143 |
-
# Clean potential markdown
|
| 144 |
-
text = re.sub(r'```json\n?|\n?```', '', text).strip()
|
| 145 |
-
return json.loads(text)
|
| 146 |
-
if res.status_code in [401, 429]: await rot.rotate()
|
| 147 |
-
except: pass
|
| 148 |
-
return None
|
| 149 |
|
| 150 |
-
|
| 151 |
-
|
| 152 |
-
|
| 153 |
-
|
| 154 |
-
|
| 155 |
-
|
| 156 |
-
|
| 157 |
-
|
| 158 |
-
json={
|
| 159 |
-
"model": "llama-3.3-70b-versatile",
|
| 160 |
-
"messages": [{"role": "system", "content": system}, {"role": "user", "content": prompt}],
|
| 161 |
-
"response_format": {"type": "json_object"}
|
| 162 |
-
}, timeout=7.0)
|
| 163 |
-
if res.status_code == 200:
|
| 164 |
-
return json.loads(res.json()["choices"][0]["message"]["content"])
|
| 165 |
-
if res.status_code in [401, 429]: await rot.rotate()
|
| 166 |
-
except: pass
|
| 167 |
-
return None
|
| 168 |
|
| 169 |
-
async def call_tier_4_deepseek(prompt: str, system: str) -> Optional[Dict]:
|
| 170 |
-
rot = rotators["deepseek"]
|
| 171 |
-
key = await rot.get_key()
|
| 172 |
-
if not key: return None
|
| 173 |
-
async with httpx.AsyncClient() as client:
|
| 174 |
-
try:
|
| 175 |
-
res = await client.post("https://api.deepseek.com/chat/completions",
|
| 176 |
-
headers={"Authorization": f"Bearer {key}"},
|
| 177 |
-
json={
|
| 178 |
-
"model": "deepseek-chat",
|
| 179 |
-
"messages": [{"role": "system", "content": system}, {"role": "user", "content": prompt}],
|
| 180 |
-
"response_format": {"type": "json_object"}
|
| 181 |
-
}, timeout=7.0)
|
| 182 |
if res.status_code == 200:
|
|
|
|
|
|
|
|
|
|
|
|
|
| 183 |
return json.loads(res.json()["choices"][0]["message"]["content"])
|
| 184 |
-
|
| 185 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 186 |
return None
|
| 187 |
|
| 188 |
-
async def
|
| 189 |
async with llm_semaphore_local:
|
| 190 |
-
logger.info("Local
|
| 191 |
payload = {
|
| 192 |
-
"model":
|
| 193 |
"prompt": f"System: {system}\nStrict Grounding. No Fabrication. Context: {prompt}\nUser: Return JSON message.",
|
| 194 |
-
"stream": False,
|
| 195 |
-
"format": "json",
|
| 196 |
-
"options": {"num_predict": 80, "temperature": 0.1}
|
| 197 |
}
|
| 198 |
async with httpx.AsyncClient() as client:
|
| 199 |
try:
|
| 200 |
-
|
| 201 |
-
|
| 202 |
-
|
| 203 |
except Exception as e:
|
| 204 |
logger.error(f"Local Model Error: {e}")
|
| 205 |
return None
|
| 206 |
|
| 207 |
-
# --- Utilities ---
|
| 208 |
def prune_message(text: str) -> str:
|
| 209 |
text = text.replace("\n", " ").strip()
|
| 210 |
if len(text) <= 315: return text
|
|
@@ -226,72 +204,44 @@ async def compose(category: dict, merchant: dict, trigger: dict, customer: Optio
|
|
| 226 |
|
| 227 |
primary_metric = list(m.items())[0] if m else None
|
| 228 |
verbatim_source = str(payload)[:150] if payload else None
|
| 229 |
-
|
| 230 |
evidence_block = ""
|
| 231 |
-
if primary_metric:
|
| 232 |
-
|
| 233 |
-
if verbatim_source:
|
| 234 |
-
evidence_block += f"MANDATORY CITATION: '{verbatim_source}'. "
|
| 235 |
|
| 236 |
beats = category.get("seasonal_beats", [])
|
| 237 |
-
|
| 238 |
-
knowledge_anchor = ""
|
| 239 |
-
if beats: knowledge_anchor = f"INDUSTRY CONTEXT: {beats[0].get('note', '')}. "
|
| 240 |
-
elif trends: knowledge_anchor = f"MARKET TREND: {trends[0].get('note', '')}. "
|
| 241 |
|
| 242 |
-
sys_prompt = f"""Role:
|
| 243 |
-
|
| 244 |
RULES:
|
| 245 |
-
1.
|
| 246 |
-
2.
|
| 247 |
-
3.
|
| 248 |
-
4. TONE: {hinglish_note} Professional
|
| 249 |
5. LIMIT: 280-310 characters."""
|
| 250 |
|
| 251 |
-
prompt = f"CONTEXT: {json.dumps({'merchant': ident, 'trigger': trigger})}"
|
| 252 |
-
|
| 253 |
-
# Run the 5-tier chain
|
| 254 |
res = await call_llm_chain(prompt, system=sys_prompt)
|
| 255 |
-
|
| 256 |
if not res:
|
| 257 |
-
|
|
|
|
| 258 |
res = {"body": body, "cta": "Reply YES", "rationale": "Hard Fallback"}
|
| 259 |
-
|
| 260 |
res["body"] = prune_message(res.get("body", ""))
|
| 261 |
return res
|
| 262 |
|
| 263 |
-
# --- Endpoints ---
|
| 264 |
@app.on_event("startup")
|
| 265 |
async def startup_event():
|
| 266 |
-
logger.info("
|
| 267 |
-
for
|
| 268 |
-
|
| 269 |
-
status = "FOUND" if key else "MISSING"
|
| 270 |
-
logger.info(f"Provider {name}: {status} ({len(rot.keys)} keys loaded)")
|
| 271 |
|
| 272 |
@app.get("/v1/healthz")
|
| 273 |
-
async def healthz():
|
| 274 |
-
async with httpx.AsyncClient() as client:
|
| 275 |
-
try:
|
| 276 |
-
res = await client.get("http://127.0.0.1:11434/api/tags")
|
| 277 |
-
if "llama3.2" in res.text: return {"status": "ok"}
|
| 278 |
-
except: pass
|
| 279 |
-
return {"status": "starting"}
|
| 280 |
|
| 281 |
@app.get("/v1/metadata")
|
| 282 |
async def metadata():
|
| 283 |
-
|
| 284 |
-
async with httpx.AsyncClient() as client:
|
| 285 |
-
try:
|
| 286 |
-
res = await client.get("http://127.0.0.1:11434/api/tags")
|
| 287 |
-
if "llama3.2" in res.text: model_status = "ready"
|
| 288 |
-
except: pass
|
| 289 |
-
return {
|
| 290 |
-
"team_name": "Vera Lead Solver",
|
| 291 |
-
"version": "5.0",
|
| 292 |
-
"engine": "5-Tier-Hierarchical-Engine",
|
| 293 |
-
"local_model": model_status
|
| 294 |
-
}
|
| 295 |
|
| 296 |
@app.post("/v1/context")
|
| 297 |
def push_context(data: ContextPayload):
|
|
@@ -300,9 +250,10 @@ def push_context(data: ContextPayload):
|
|
| 300 |
|
| 301 |
@app.post("/v1/tick")
|
| 302 |
async def tick(req: TickRequest):
|
| 303 |
-
tids = sorted(req.available_triggers, key=lambda t:
|
| 304 |
results = []
|
| 305 |
-
|
|
|
|
| 306 |
res = await process_trigger(tid)
|
| 307 |
if res: results.append(res)
|
| 308 |
return {"actions": results}
|
|
@@ -312,38 +263,46 @@ async def process_trigger(trigger_id: str):
|
|
| 312 |
if not trigger: return None
|
| 313 |
merchant = contexts["merchant"].get(trigger.get("merchant_id"), {})
|
| 314 |
category = contexts["category"].get(merchant.get("category_slug", ""), {"slug": "general"})
|
| 315 |
-
|
|
|
|
|
|
|
|
|
|
| 316 |
return {
|
| 317 |
"conversation_id": f"conv_{trigger_id}", "merchant_id": trigger.get("merchant_id"),
|
| 318 |
-
"
|
|
|
|
|
|
|
|
|
|
|
|
|
| 319 |
}
|
| 320 |
|
| 321 |
-
AUTO_REPLY_PATTERNS = [
|
| 322 |
-
|
| 323 |
-
|
| 324 |
-
]
|
| 325 |
-
|
| 326 |
-
HOSTILE_PATTERNS = [
|
| 327 |
-
r"stop", r"spam", r"useless", r"abuse", r"reporting", r"don't message", r"remove me"
|
| 328 |
-
]
|
| 329 |
|
| 330 |
@app.post("/v1/reply")
|
| 331 |
async def reply(req: Dict[str, Any]):
|
| 332 |
msg = req.get("message", "").lower()
|
| 333 |
-
|
| 334 |
-
|
| 335 |
-
|
| 336 |
-
for
|
| 337 |
-
if re.search(
|
| 338 |
-
return {"action": "
|
| 339 |
-
|
| 340 |
-
|
| 341 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 342 |
res = await call_llm_chain(prompt, system=sys)
|
| 343 |
-
|
| 344 |
-
|
| 345 |
-
return {"action": "send", "body": reply_body, "cta": "Reply YES"}
|
| 346 |
|
| 347 |
if __name__ == "__main__":
|
| 348 |
import uvicorn
|
| 349 |
-
uvicorn.run(app, host="0.0.0.0", port=int(os.environ.get("PORT",
|
|
|
|
| 13 |
|
| 14 |
load_dotenv()
|
| 15 |
|
| 16 |
+
# Vera Engine - Scalable Multi-Agent Reliability Build (v7.0)
|
| 17 |
logging.basicConfig(level=logging.INFO)
|
| 18 |
+
logger = logging.getLogger("VeraOrchestrator")
|
| 19 |
|
| 20 |
app = FastAPI()
|
| 21 |
|
| 22 |
+
# --- 1. Infrastructure & Testing Mode ---
|
| 23 |
+
# If FAST_TEST=true, skip cloud tiers for local debugging speed.
|
| 24 |
+
FAST_TEST = os.getenv("FAST_TEST", "false").lower() == "true"
|
| 25 |
+
|
| 26 |
class ContextPayload(BaseModel):
|
| 27 |
scope: Optional[str] = None
|
| 28 |
context_id: Optional[str] = None
|
|
|
|
| 32 |
class TickRequest(BaseModel):
|
| 33 |
available_triggers: List[str] = []
|
| 34 |
|
| 35 |
+
# --- 2. Scalable Multi-Agent Registry ---
|
| 36 |
+
class Agent:
|
| 37 |
+
def __init__(self, name: str, provider: str, model: str, tier: int):
|
| 38 |
+
self.name = name
|
| 39 |
self.provider = provider
|
| 40 |
+
self.model = model
|
| 41 |
+
self.tier = tier
|
| 42 |
+
self.keys = self._load_keys()
|
| 43 |
+
self.key_index = 0
|
| 44 |
+
self.is_healthy = True
|
| 45 |
+
self.cooldown_until = 0
|
| 46 |
+
|
| 47 |
+
def _load_keys(self) -> List[str]:
|
| 48 |
+
keys = []
|
| 49 |
for i in range(1, 11):
|
| 50 |
+
key = os.getenv(f"{self.provider.upper()}_KEY_{i}")
|
| 51 |
+
if key: keys.append(key)
|
| 52 |
+
if not keys:
|
| 53 |
+
key = os.getenv(f"{self.provider.upper()}_API_KEY")
|
| 54 |
+
if key: keys.append(key)
|
| 55 |
+
return keys
|
|
|
|
|
|
|
|
|
|
| 56 |
|
| 57 |
async def get_key(self) -> Optional[str]:
|
| 58 |
if not self.keys: return None
|
| 59 |
+
return self.keys[self.key_index]
|
| 60 |
+
|
| 61 |
+
def rotate_key(self):
|
| 62 |
+
if self.keys:
|
| 63 |
+
self.key_index = (self.key_index + 1) % len(self.keys)
|
| 64 |
+
logger.info(f"Agent {self.name} rotated key to index {self.key_index}")
|
| 65 |
+
|
| 66 |
+
def mark_unhealthy(self, duration: int = 60):
|
| 67 |
+
self.is_healthy = False
|
| 68 |
+
self.cooldown_until = time.time() + duration
|
| 69 |
+
logger.warning(f"Agent {self.name} marked unhealthy for {duration}s")
|
| 70 |
+
|
| 71 |
+
def check_health(self):
|
| 72 |
+
if not self.is_healthy and time.time() > self.cooldown_until:
|
| 73 |
+
self.is_healthy = True
|
| 74 |
+
logger.info(f"Agent {self.name} is healthy again")
|
| 75 |
+
return self.is_healthy and len(self.keys) > 0
|
| 76 |
+
|
| 77 |
+
class LocalAgent(Agent):
|
| 78 |
+
def __init__(self):
|
| 79 |
+
super().__init__("LocalOllama", "ollama", "llama3.2", 5)
|
| 80 |
+
self.keys = ["local"] # Placeholder
|
| 81 |
+
def check_health(self): return True
|
| 82 |
+
|
| 83 |
+
# Initialize Registry
|
| 84 |
+
AGENT_REGISTRY: List[Agent] = [
|
| 85 |
+
Agent("Cloud-OpenRouter-Laguna", "openrouter", "poolside/laguna-m.1:free", 1),
|
| 86 |
+
Agent("Cloud-OpenRouter-Hy3", "openrouter", "tencent/hy3:free", 1),
|
| 87 |
+
Agent("Cloud-Gemini-Flash", "gemini", "gemini-2.0-flash-exp", 2),
|
| 88 |
+
Agent("Cloud-Groq-Llama", "groq", "llama-3.3-70b-versatile", 3),
|
| 89 |
+
Agent("Cloud-DeepSeek-Chat", "deepseek", "deepseek-chat", 4),
|
| 90 |
+
LocalAgent()
|
| 91 |
+
]
|
| 92 |
|
| 93 |
contexts = {"category": {}, "merchant": {}, "customer": {}, "trigger": {}}
|
| 94 |
llm_semaphore_local = asyncio.Semaphore(1)
|
| 95 |
|
| 96 |
+
# --- 3. Orchestration Logic ---
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 97 |
async def call_llm_chain(prompt: str, system: str = "") -> Optional[Dict]:
|
| 98 |
+
if FAST_TEST:
|
| 99 |
+
logger.info("FAST_TEST enabled: Skipping cloud tiers.")
|
| 100 |
+
return await call_local_agent(AGENT_REGISTRY[-1], prompt, system)
|
| 101 |
+
|
| 102 |
json_suffix = "\n\nRETURN JSON ONLY: {\"body\": \"...\", \"cta\": \"...\", \"rationale\": \"...\"}"
|
| 103 |
full_system = system + json_suffix
|
| 104 |
|
| 105 |
+
# Iterate through tiers
|
| 106 |
+
for tier in range(1, 6):
|
| 107 |
+
tier_agents = [a for p, a in enumerate(AGENT_REGISTRY) if a.tier == tier and a.check_health()]
|
| 108 |
+
for agent in tier_agents:
|
| 109 |
+
res = await execute_agent(agent, prompt, full_system if agent.tier < 5 else system)
|
| 110 |
+
if res: return res
|
| 111 |
+
return None
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 112 |
|
| 113 |
+
async def execute_agent(agent: Agent, prompt: str, system: str) -> Optional[Dict]:
|
| 114 |
+
if agent.provider == "ollama":
|
| 115 |
+
return await call_local_agent(agent, prompt, system)
|
|
|
|
| 116 |
|
| 117 |
+
key = await agent.get_key()
|
| 118 |
+
if not key: return None
|
| 119 |
+
|
| 120 |
async with httpx.AsyncClient() as client:
|
| 121 |
+
try:
|
| 122 |
+
if agent.provider == "openrouter":
|
| 123 |
+
res = await client.post("https://openrouter.ai/api/v1/chat/completions",
|
| 124 |
+
headers={"Authorization": f"Bearer {key}", "HTTP-Referer": "https://vera.ai", "X-Title": "Vera Platinum"},
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 125 |
json={
|
| 126 |
+
"model": agent.model,
|
| 127 |
"messages": [{"role": "system", "content": system}, {"role": "user", "content": prompt}],
|
| 128 |
"response_format": {"type": "json_object"}
|
| 129 |
+
}, timeout=5.0) # Tightened to 5s for chain speed
|
| 130 |
+
|
| 131 |
+
elif agent.provider == "gemini":
|
| 132 |
+
url = f"https://generativelanguage.googleapis.com/v1beta/models/{agent.model}:generateContent?key={key}"
|
| 133 |
+
res = await client.post(url, json={"contents": [{"parts": [{"text": f"System: {system}\n\nUser: {prompt}"}]}]}, timeout=5.0)
|
| 134 |
+
|
| 135 |
+
elif agent.provider == "groq":
|
| 136 |
+
res = await client.post("https://api.groq.com/openai/v1/chat/completions",
|
| 137 |
+
headers={"Authorization": f"Bearer {key}"},
|
| 138 |
+
json={
|
| 139 |
+
"model": agent.model,
|
| 140 |
+
"messages": [{"role": "system", "content": system}, {"role": "user", "content": prompt}],
|
| 141 |
+
"response_format": {"type": "json_object"}
|
| 142 |
+
}, timeout=5.0)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 143 |
|
| 144 |
+
elif agent.provider == "deepseek":
|
| 145 |
+
res = await client.post("https://api.deepseek.com/chat/completions",
|
| 146 |
+
headers={"Authorization": f"Bearer {key}"},
|
| 147 |
+
json={
|
| 148 |
+
"model": agent.model,
|
| 149 |
+
"messages": [{"role": "system", "content": system}, {"role": "user", "content": prompt}],
|
| 150 |
+
"response_format": {"type": "json_object"}
|
| 151 |
+
}, timeout=5.0)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 152 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 153 |
if res.status_code == 200:
|
| 154 |
+
if agent.provider == "gemini":
|
| 155 |
+
text = res.json()["candidates"][0]["content"]["parts"][0]["text"]
|
| 156 |
+
text = re.sub(r'```json\n?|\n?```', '', text).strip()
|
| 157 |
+
return json.loads(text)
|
| 158 |
return json.loads(res.json()["choices"][0]["message"]["content"])
|
| 159 |
+
|
| 160 |
+
if res.status_code in [401, 429]:
|
| 161 |
+
agent.rotate_key()
|
| 162 |
+
else:
|
| 163 |
+
agent.mark_unhealthy()
|
| 164 |
+
except Exception:
|
| 165 |
+
pass
|
| 166 |
return None
|
| 167 |
|
| 168 |
+
async def call_local_agent(agent: Agent, prompt: str, system: str) -> Optional[Dict]:
|
| 169 |
async with llm_semaphore_local:
|
| 170 |
+
logger.info(f"Local Inference: {agent.model}")
|
| 171 |
payload = {
|
| 172 |
+
"model": agent.model,
|
| 173 |
"prompt": f"System: {system}\nStrict Grounding. No Fabrication. Context: {prompt}\nUser: Return JSON message.",
|
| 174 |
+
"stream": False, "format": "json", "options": {"num_predict": 80, "temperature": 0.1}
|
|
|
|
|
|
|
| 175 |
}
|
| 176 |
async with httpx.AsyncClient() as client:
|
| 177 |
try:
|
| 178 |
+
# 28s here + overhead must be < 30s. If FAST_TEST=true, we skip cloud so we have full 30s.
|
| 179 |
+
res = await client.post("http://127.0.0.1:11434/api/generate", json=payload, timeout=28.0)
|
| 180 |
+
if res.status_code == 200: return json.loads(res.json().get('response', '{}'))
|
| 181 |
except Exception as e:
|
| 182 |
logger.error(f"Local Model Error: {e}")
|
| 183 |
return None
|
| 184 |
|
| 185 |
+
# --- 4. Utilities & Core Logic ---
|
| 186 |
def prune_message(text: str) -> str:
|
| 187 |
text = text.replace("\n", " ").strip()
|
| 188 |
if len(text) <= 315: return text
|
|
|
|
| 204 |
|
| 205 |
primary_metric = list(m.items())[0] if m else None
|
| 206 |
verbatim_source = str(payload)[:150] if payload else None
|
|
|
|
| 207 |
evidence_block = ""
|
| 208 |
+
if primary_metric: evidence_block += f"MANDATORY METRIC: '{primary_metric[0]}' is {primary_metric[1]}. "
|
| 209 |
+
if verbatim_source: evidence_block += f"MANDATORY CITATION: '{verbatim_source}'. "
|
|
|
|
|
|
|
| 210 |
|
| 211 |
beats = category.get("seasonal_beats", [])
|
| 212 |
+
knowledge_anchor = f"INDUSTRY CONTEXT: {beats[0].get('note', '')}. " if beats else ""
|
|
|
|
|
|
|
|
|
|
| 213 |
|
| 214 |
+
sys_prompt = f"""Role: Elite Growth Strategist. Target: {prefix} {owner_name} in {ident.get('locality')}.
|
| 215 |
+
Objective: Compose high-impact WhatsApp message.
|
| 216 |
RULES:
|
| 217 |
+
1. TRIGGER PRIMACY: Address trigger '{trigger.get('kind')}' first.
|
| 218 |
+
2. NO FABRICATION: Use only context numbers. EVIDENCE: {evidence_block}
|
| 219 |
+
3. CONTEXT: {knowledge_anchor}
|
| 220 |
+
4. TONE: {hinglish_note} Professional clinical-peer tone for dentists.
|
| 221 |
5. LIMIT: 280-310 characters."""
|
| 222 |
|
| 223 |
+
prompt = f"CONTEXT: {json.dumps({'merchant': ident, 'perf': m, 'trigger': trigger, 'customer': customer})}"
|
|
|
|
|
|
|
| 224 |
res = await call_llm_chain(prompt, system=sys_prompt)
|
|
|
|
| 225 |
if not res:
|
| 226 |
+
v, c = m.get('views', '?'), m.get('calls', '?')
|
| 227 |
+
body = f"{prefix} {owner_name}, noticed {v} views and {c} calls in {ident.get('locality')}. Important growth update found. Review now?"
|
| 228 |
res = {"body": body, "cta": "Reply YES", "rationale": "Hard Fallback"}
|
|
|
|
| 229 |
res["body"] = prune_message(res.get("body", ""))
|
| 230 |
return res
|
| 231 |
|
| 232 |
+
# --- 5. Endpoints ---
|
| 233 |
@app.on_event("startup")
|
| 234 |
async def startup_event():
|
| 235 |
+
logger.info("Initializing Agent Registry...")
|
| 236 |
+
for agent in AGENT_REGISTRY:
|
| 237 |
+
logger.info(f"Agent {agent.name} initialized with {len(agent.keys)} keys")
|
|
|
|
|
|
|
| 238 |
|
| 239 |
@app.get("/v1/healthz")
|
| 240 |
+
async def healthz(): return {"status": "ok"}
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 241 |
|
| 242 |
@app.get("/v1/metadata")
|
| 243 |
async def metadata():
|
| 244 |
+
return {"team_name": "Vera Lead Solver", "version": "7.0", "engine": "Multi-Agent-Orchestrator"}
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 245 |
|
| 246 |
@app.post("/v1/context")
|
| 247 |
def push_context(data: ContextPayload):
|
|
|
|
| 250 |
|
| 251 |
@app.post("/v1/tick")
|
| 252 |
async def tick(req: TickRequest):
|
| 253 |
+
tids = sorted(req.available_triggers, key=lambda t: contexts["trigger"].get(t, {}).get("kind", ""), reverse=True)
|
| 254 |
results = []
|
| 255 |
+
# Sequential processing. Limit to 1 to stay under 30s.
|
| 256 |
+
for tid in tids[:1]:
|
| 257 |
res = await process_trigger(tid)
|
| 258 |
if res: results.append(res)
|
| 259 |
return {"actions": results}
|
|
|
|
| 263 |
if not trigger: return None
|
| 264 |
merchant = contexts["merchant"].get(trigger.get("merchant_id"), {})
|
| 265 |
category = contexts["category"].get(merchant.get("category_slug", ""), {"slug": "general"})
|
| 266 |
+
customer = contexts["customer"].get(trigger.get("customer_id"))
|
| 267 |
+
composed = await compose(category, merchant, trigger, customer)
|
| 268 |
+
is_customer_facing = trigger.get("scope") == "customer"
|
| 269 |
+
|
| 270 |
return {
|
| 271 |
"conversation_id": f"conv_{trigger_id}", "merchant_id": trigger.get("merchant_id"),
|
| 272 |
+
"customer_id": trigger.get("customer_id") if is_customer_facing else None,
|
| 273 |
+
"send_as": "merchant_on_behalf" if is_customer_facing else "vera",
|
| 274 |
+
"trigger_id": trigger_id, "template_name": "vera_platinum_v7",
|
| 275 |
+
"body": composed["body"], "cta": composed["cta"], "suppression_key": trigger_id,
|
| 276 |
+
"rationale": composed.get("rationale", "Orchestrated Inference")
|
| 277 |
}
|
| 278 |
|
| 279 |
+
AUTO_REPLY_PATTERNS = [r"thank you for contacting", r"we are currently away", r"automated message", r"business hours", r"auto-reply"]
|
| 280 |
+
HOSTILE_PATTERNS = [r"stop", r"spam", r"useless", r"abuse", r"remove me"]
|
| 281 |
+
INTENT_PATTERNS = [r"\bok\b", r"\byes\b", r"\bdo it\b", r"let'?s do it", r"\bsure\b", r"\bgo ahead\b"]
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 282 |
|
| 283 |
@app.post("/v1/reply")
|
| 284 |
async def reply(req: Dict[str, Any]):
|
| 285 |
msg = req.get("message", "").lower()
|
| 286 |
+
turn = req.get("turn_number", 1)
|
| 287 |
+
for p in HOSTILE_PATTERNS:
|
| 288 |
+
if re.search(p, msg): return {"action": "end", "rationale": "Hostility detected."}
|
| 289 |
+
for p in AUTO_REPLY_PATTERNS:
|
| 290 |
+
if re.search(p, msg):
|
| 291 |
+
if turn > 2: return {"action": "end", "rationale": "Repeated auto-reply."}
|
| 292 |
+
return {"action": "wait", "wait_seconds": 3600, "rationale": "Auto-reply wait."}
|
| 293 |
+
for p in INTENT_PATTERNS:
|
| 294 |
+
if re.search(p, msg):
|
| 295 |
+
sys = "Role: Vera AI. Merchant interest detected. Provide EXACT next step action. 15 words max."
|
| 296 |
+
res = await call_llm_chain(f"Merchant said: {msg}", system=sys)
|
| 297 |
+
body = res.get("body", "I'm setting that up for you now.") if res else "I'm setting that up now."
|
| 298 |
+
return {"action": "send", "body": body, "cta": "Reply YES", "rationale": "Intent Transition."}
|
| 299 |
+
|
| 300 |
+
prompt = f"Merchant message: {req.get('message', '')}. Turn: {turn}. History: {req.get('history', [])}"
|
| 301 |
+
sys = "Role: Vera AI. If merchant says 'ok/yes/do it', provide EXACT next step action. Be concise."
|
| 302 |
res = await call_llm_chain(prompt, system=sys)
|
| 303 |
+
body = res.get("body", "I'm on it. Setting that up for you now. Ready?") if res else "Understood. I'll get that started. Ready?"
|
| 304 |
+
return {"action": "send", "body": body, "cta": "Reply YES"}
|
|
|
|
| 305 |
|
| 306 |
if __name__ == "__main__":
|
| 307 |
import uvicorn
|
| 308 |
+
uvicorn.run(app, host="0.0.0.0", port=int(os.environ.get("PORT", 8080)))
|