Spaces:
Running
Running
Commit Β·
00235a3
1
Parent(s): 0cfbac9
docs: update README to version 1.0 and document 4-tier architecture
Browse files
README.md
CHANGED
|
@@ -1,5 +1,5 @@
|
|
| 1 |
---
|
| 2 |
-
title: Vera Engine
|
| 3 |
emoji: π
|
| 4 |
colorFrom: blue
|
| 5 |
colorTo: purple
|
|
@@ -8,56 +8,41 @@ app_port: 8080
|
|
| 8 |
pinned: false
|
| 9 |
---
|
| 10 |
|
| 11 |
-
# Vera Engine
|
| 12 |
|
| 13 |
-
**
|
| 14 |
|
| 15 |
-
## 1. THE
|
| 16 |
-
|
| 17 |
-
* **Bot no lie.**
|
| 18 |
-
* **Chain long.**
|
| 19 |
-
* **Local safe.**
|
| 20 |
-
* **Always win.**
|
| 21 |
|
| 22 |
-
##
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 23 |
|
| 24 |
-
###
|
| 25 |
-
|
|
|
|
| 26 |
|
| 27 |
-
|
| 28 |
-
|
| 29 |
-
|
| 30 |
-
|
| 31 |
-
* **Tier 5 (Local)**: 100% autonomous fallback to `Llama 3.2 (3B)` via Ollama. 17.9s benchmarked.
|
| 32 |
-
4. **FAST_TEST Mode**: Environment-aware flag to bypass cloud tiers during local judge simulation for rapid iteration.
|
| 33 |
|
| 34 |
-
## 3.
|
| 35 |
-
|
| 36 |
-
* **Metric Anchoring**: Pre-extraction of metrics as "Mandatory Evidence" to ensure 100% grounding integrity.
|
| 37 |
-
* **Elastic Schema**: Optional Pydantic fields with strict defaults to eliminate 422 Unprocessable Entity errors.
|
| 38 |
-
* **Atomic Throughput**: Sequential processing in `/v1/tick` to survive strict 30s judge timeouts on 2vCPU hardware.
|
| 39 |
-
* **Turn-Based Escalation**: Intelligent auto-reply shield that terminates conversations stuck in infinite canned-response loops (Turn 3+).
|
| 40 |
-
|
| 41 |
-
## 4. API DOCUMENTATION
|
| 42 |
|
| 43 |
| Endpoint | Method | Purpose | Response Schema |
|
| 44 |
| :--- | :--- | :--- | :--- |
|
| 45 |
-
|
|
| 46 |
-
|
|
| 47 |
-
|
|
| 48 |
-
|
|
| 49 |
-
|
|
| 50 |
-
|
| 51 |
-
##
|
| 52 |
-
|
| 53 |
-
*
|
| 54 |
-
*
|
| 55 |
-
*
|
| 56 |
-
* **Infrastructure**: Hugging Face Spaces (2vCPU / 16GB RAM)
|
| 57 |
-
|
| 58 |
-
## 6. DEPLOYMENT
|
| 59 |
-
|
| 60 |
-
```bash
|
| 61 |
-
git push origin main
|
| 62 |
-
# Auto-sync to Hugging Face Space
|
| 63 |
-
```
|
|
|
|
| 1 |
---
|
| 2 |
+
title: Vera Engine
|
| 3 |
emoji: π
|
| 4 |
colorFrom: blue
|
| 5 |
colorTo: purple
|
|
|
|
| 8 |
pinned: false
|
| 9 |
---
|
| 10 |
|
| 11 |
+
# Vera Engine v1.0 π€
|
| 12 |
|
| 13 |
+
**High-Performance AI Orchestrator optimized for the magicpin Vera AI Challenge.**
|
| 14 |
|
| 15 |
+
## 1. THE ARCHITECTURE
|
| 16 |
+
The system operates on a highly optimized, deterministic fail-fast fallback cascade to guarantee sub-30-second responses on Hugging Face Spaces (2vCPU / 16GB RAM) without compromising on reasoning quality.
|
|
|
|
|
|
|
|
|
|
|
|
|
| 17 |
|
| 18 |
+
### 4-Tier Fail-Fast Cascade
|
| 19 |
+
Instead of relying on a single provider, the engine delegates composition to a 4-tier chain of LLM providers.
|
| 20 |
+
1. **Tier 1 (Speed & Intelligence)**: NVIDIA NIM (`llama-3.3-70b-instruct`) - 5s timeout.
|
| 21 |
+
2. **Tier 2 (Fast Failover)**: Groq (`llama-3.3-70b-versatile`) - 4s timeout.
|
| 22 |
+
3. **Tier 3 (Secondary Failover)**: Gemini (`gemini-2.5-flash`) - 4s timeout.
|
| 23 |
+
4. **Tier 4 (Local Airgap)**: Natively executed `qwen2.5-3b-instruct` via `llama-cpp-python`.
|
| 24 |
|
| 25 |
+
### Dynamic Time Budgeting & Fail-Fast
|
| 26 |
+
- **Fail-Fast**: If an API returns a 401 (Auth Error) or 429 (Rate Limit), the orchestrator immediately short-circuits to the next tier without waiting for the timeout, saving crucial seconds.
|
| 27 |
+
- **Budget Reallocation**: The local model inherently requires the most time on a CPU. By aggressively capping the cloud tier timeouts (13s combined worst-case), the orchestrator guarantees a minimum of 14 seconds for the local `qwen2.5-3b` model to complete execution before the hard 30s deadline.
|
| 28 |
|
| 29 |
+
## 2. PIPELINE STRATEGY
|
| 30 |
+
1. **Context Distiller (Python)**: Instantly processes 500KB+ JSON payloads into dense factual bullet points in < 0.001s.
|
| 31 |
+
2. **LLM Expander**: Takes the strict facts and expands them into compelling, Hinglish-supported, natively conversational WhatsApp replies.
|
| 32 |
+
3. **Strict JSON Schema**: Native `response_format={"type": "json_object"}` constraints across all adapters ensure zero JSON-parsing failures.
|
|
|
|
|
|
|
| 33 |
|
| 34 |
+
## 3. API DOCUMENTATION
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 35 |
|
| 36 |
| Endpoint | Method | Purpose | Response Schema |
|
| 37 |
| :--- | :--- | :--- | :--- |
|
| 38 |
+
| /v1/healthz | GET | Compliant Liveness Probe | {"status": "ok"} |
|
| 39 |
+
| /v1/metadata | GET | System Identity | {"version": "1.0", "model": "Fallback Chain...", ...} |
|
| 40 |
+
| /v1/context | POST | Elastic Context Ingestion | {"accepted": true} |
|
| 41 |
+
| /v1/tick | POST | Atomic Trigger Processing | {"actions": [...]} |
|
| 42 |
+
| /v1/reply | POST | Escalated Shield Handling | {"action": "send/wait/end"} |
|
| 43 |
+
|
| 44 |
+
## 4. TECHNICAL STACK
|
| 45 |
+
* **Framework**: FastAPI / Uvicorn (Port 8080)
|
| 46 |
+
* **Local LLM Execution**: `llama-cpp-python` (Pre-compiled CPU Wheels)
|
| 47 |
+
* **Models**: Llama 3.3 70B, Gemini 2.5 Flash, Qwen2.5 3B
|
| 48 |
+
* **Infrastructure**: Dockerized Python 3.10-slim on Hugging Face Spaces
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|