Spaces:
Running
Running
| # Vera Engagement Framework — Research: Current Merchant Data Access | |
| **Status**: Research notes — companion to `engagement-design.md`. | |
| **Last updated**: 2026-04-26 | |
| **Scope**: How the existing system loads merchant + customer data at conversation init and during a turn. Used to inform what the new framework must build vs. adapt. | |
| --- | |
| ## TL;DR | |
| Two distinct paths exist today — **merchant-facing** (Vera ↔ Dr. Meera) and **customer-facing** (a customer asks Vera *about* Dr. Meera). They share infrastructure (vera-mcp + merchant-support-mcp + Redis) but compose context very differently. | |
| Most of what the proposed `MerchantContext` needs **already exists in scattered form** (`_merchant_snapshot`, `_behavioral_profile`, `_session_scenario`). The genuinely new pieces are: | |
| - `CategoryContext` (no existing equivalent) | |
| - `TriggerContext` (no normalized abstraction today — every cron emits ad-hoc payloads) | |
| - `CustomerContext.relationship` (visit history aggregation doesn't exist) | |
| - `EngagementComposer` (no shared composer — each agent has its own prompt-builder) | |
| The aryan dependency for `category` and `locality` is the soft underbelly — every customer-info-pack call hits aryan synchronously. | |
| --- | |
| ## Path 1 — Merchant-facing (`VeraMerchantAgent`) | |
| ### Init (per session, instance-level) | |
| `agents/vera/merchant_agent.py:402` — `__init__`: | |
| - Spawns its own `vera_mcp_client` (`HTTPMCPClient` → `VERA_MCP_SERVER_URL`, default `vera-mcp:8000`). | |
| - Inherits `BaseAgent._merchant_mcp_client` — class-level shared singleton pointing to `merchant-support-mcp` (`https://search5.magicpin.com/staging/mcp-server/mcp`). | |
| - **No merchant data is loaded at init** — agent doesn't know who it's talking to yet. | |
| ### Per-turn (every inbound message) | |
| `agents/vera/merchant_agent.py:2171` — `handle_input(agent_input)`: | |
| 1. Extract `merchant_id` from `metadata.context_data.merchant_id`, falling back to regex on the message body (`\d{7,}`). | |
| 2. Fire 2 parallel tasks: | |
| - `_detect_state(merchant_id)` — GBP connection + subscription status checks. | |
| - `_prefetch_merchant_context(merchant_id)` — full snapshot + behavioral profile. | |
| ### `_prefetch_merchant_context` — `merchant_agent.py:740` | |
| Cache-first design: | |
| 1. **Redis check**: `vera:merchant_ctx:{merchant_id}` — TTL **30 min**. Hit → return immediately, skip everything below. | |
| 2. Parallel via `asyncio.gather`: | |
| - `vera_merchant_snapshot` (vera-mcp tool) | |
| - `vera_get_merchant_profile` (vera-mcp tool) | |
| 3. If snapshot is empty: fall back to `get_aggregated_unassociated_merchant_data` (merchant-support-mcp) — basic GBP health, no commercial data. Wrap it in a snapshot-shaped envelope. | |
| 4. Sequential enrichments: | |
| - `_prefetch_product_context` — fans out to `vera_get_subscription_context`, `vera_get_performance_summary`, and (only if DA subscribed) `da_get_campaign_context`. ~2KB total, embedded into snapshot. | |
| - `_enrich_snapshot_with_pricing` — direct HTTP to `https://vera.magicpin.com/api/v1/merchant/pricing/get?mid=...` (vera-mcp's pricing path is unreachable locally). Mounted into snapshot as `pricing_recommendation`. | |
| 5. Stuff result into Redis at the same key, TTL 30 min. | |
| Final state: `self._merchant_snapshot` and `self._behavioral_profile` are populated. `_get_system_prompt()` reads from these and serializes the snapshot directly into the LLM system prompt (`merchant_agent.py:996`). | |
| ### What `vera_merchant_snapshot` actually fetches | |
| `vera-mcp/src/services/merchant_snapshot.py:51` — `build_merchant_snapshot()`: | |
| 1. **Resolve identity** via `gbp_resolve_merchant` — gets `place_id`, `location_name`, `merchant_title`. | |
| 2. **One async fan-out** of up to 13 tool calls in parallel: | |
| | Bucket | Tools | | |
| |---|---| | |
| | Merchant-level (no GBP needed) | `vera_get_subscription_status`, `vera_get_pricing_by_merchant`, `vera_get_merchant_pain_points`, `vera_get_merchant_offer`, `vera_get_merchant_config`, `vera_get_onboarding_status`, `vera_get_enhancement_suggestions` | | |
| | Location-level (skipped if no place_id) | `gbp_get_location`, `gbp_get_profile_completeness`, `gbp_get_performance_summary`, `gbp_get_search_keywords`, `gbp_get_review_stats`, `gbp_list_posts` | | |
| 3. Composes 7 sections: `identity`, `profile`, `reputation`, `growth`, `commercial`, `conversation_hooks`, `issues`. | |
| > **Note**: `category` and `locality` are not first-class fields on the snapshot — they're buried inside `profile.business_info` (from `gbp_get_location`) and surfaced via `aryan_client.get_merchant_v2()` only when `vera_get_customer_info_pack` is called. | |
| ### Mid-conversation tool calls | |
| The LLM gets the full toolset from `MERCHANT_MCP_ALLOWED_TOOLS` (declared on the agent class) merged with `_vera_mcp_tools_cache`. Mid-turn tool calls flow through: | |
| - `self.vera_mcp_client.call_tool(name, args)` — for vera-mcp tools (HTTP) | |
| - `BaseAgent._merchant_mcp_client.call_tool(name, args)` — for merchant-support-mcp tools (HTTP) | |
| No re-prefetch of the snapshot mid-turn. The agent relies on the prefetched snapshot + LLM-initiated lookups when it needs fresh data. | |
| --- | |
| ## Path 2 — Customer-facing (`CustomerIncomingAgent`) | |
| `agents/vera/customer_incoming_agent.py:91` — different agent, different state model. | |
| ### Init + per-turn | |
| The instance carries: | |
| - `_merchant_id` — the merchant the customer is asking about | |
| - `_merchant_data`, `_merchant_name`, `_category`, `_offers`, `_jd_info` — about the **business** | |
| - `_customer_phone`, `_customer_profile` — about the **customer** (the one chatting) | |
| - `_chat_history`, `_session_state` — conversation continuity | |
| ### Merchant data load — `_load_merchant_data` (`customer_incoming_agent.py:221`) | |
| Single shape, no Redis cache: | |
| 1. **Primary call**: `vera_get_customer_info_pack(merchant_id)` — one MCP call returns `business_info` + `reviews` + `photos` + `offers` + `metadata` in one shot. The aggregated customer-facing endpoint. | |
| 2. **Fallback chain** if `info_pack` is empty: | |
| - `get_unassociated_merchant_data` — basic merchant info | |
| - `vera_list_merchant_offers` — separate offer list | |
| 3. **Supplementary** (non-blocking): `vera_get_merchant_jd_info` — JustDial crawl data for additional name / address / phone. | |
| Caching is at the request level (in-memory on the agent instance) — re-using the same `merchant_id` in a session avoids re-fetching. | |
| ### What `vera_get_customer_info_pack` does | |
| `vera-mcp/src/tools/merchant_info.py:188`: | |
| 1. **Resolve place_id**: `_resolve_place_id(merchant_id)` — chain of MongoDB (`gbp_status`) → local JSON → `aryan_client.get_mapping()`. | |
| 2. **Parallel fetch**: | |
| - GBP data via `get_or_fetch(place_id)` — 24h cache in MongoDB, falls through to Google Places API | |
| - `_get_active_offers(merchant_id)` — direct MongoDB read of `offers` collection where `status=active` | |
| - `_get_merchant_metadata(merchant_id)` — `aryan_client.get_merchant_v2()` → returns `name`, `category`, `locality` | |
| 3. **Last-resort fallback**: `_fetch_embed_data(place_id)` — vera REST `/embed` endpoint if everything else came back empty. | |
| ### Customer profile (the *caller*) | |
| `_customer_profile` is loaded separately — populated in `_init_customer_data` from past conversation tickets keyed off `_customer_phone`. There's no rich CRM behind it today; it's mostly conversation continuity (last visit, last topic). | |
| --- | |
| ## Cross-cutting infrastructure | |
| | Concern | Implementation | | |
| |---|---| | |
| | MCP transport | `HTTPMCPClient` (vera client) — keeps a session, calls `/mcp/tools/{name}` HTTP POST | | |
| | Auth | Not required for challenge bot endpoints | | |
| | Cache key for merchant context | `vera:merchant_ctx:{merchant_id}` — Redis, TTL 30 min, written by `_prefetch_merchant_context` | | |
| | Cache key for GBP data | `gbp_health_report:{place_id}` — MongoDB, TTL 24h, in vera-mcp | | |
| | Snapshot freshness for sends | Whatever's in Redis — not refreshed on send unless agent is in active conversation | | |
| | Source of truth for `category` | aryan `get_merchant_v2` API (via `aryan_client`) — used in `_get_merchant_metadata` | | |
| | Source of truth for `name` | aryan first, GBP `business_info` second | | |
| --- | |
| ## Observations relevant to the engagement framework | |
| ### What already exists | |
| 1. **Most of `MerchantContext` already exists** — spread across `_merchant_snapshot`, `_behavioral_profile`, `_session_scenario`, `_jd_info`. A `MerchantContext.from_existing(agent_state)` adapter could load ~80% of the fields without any new fetching. | |
| 2. **The customer agent already has half a `CustomerContext`** — `_customer_phone` + `_customer_profile` give us identity + conversation continuity. Missing: visit history, services received, lapse state. | |
| 3. **Two MCP servers, one orchestration** — vera-mcp (instance-level) for vera tools, merchant-support-mcp (class-level shared) for fallback merchant tools. The composer can just consume whatever `MerchantContext` already collected — no new MCP wiring required. | |
| 4. **Cache TTL of 30 min is fine** for engagement nudges that fire daily/weekly. Redis hits during conversation are plenty fresh for composition. | |
| ### What does NOT exist | |
| 1. **No `CategoryContext`.** Category is just a string buried in metadata. Voice rules, peer benchmarks, knowledge digests — none of it exists. Biggest greenfield area, but also the most leveraged (one CategoryContext serves all merchants in the vertical). | |
| 2. **No `customer_aggregate` field on the merchant snapshot.** No pipeline today aggregates per-merchant customer roster stats (active count, lapsed count, retention rate). | |
| 3. **No `TriggerContext` abstraction.** Every nudge type today has its own cron + its own fetch logic + its own send code. The proposed `TriggerContext` is the genuinely new architectural primitive; everything else is reorganization. | |
| 4. **No visit-history aggregation per (merchant, customer_phone).** Required for `CustomerContext.relationship`. BOTOPS chat history has the raw data; no derived view exists. | |
| 5. **No shared composer.** Each agent has its own prompt builder embedded in `_get_system_prompt()`. The proposed `EngagementComposer` is net new. | |
| ### Operational risks to mitigate | |
| 1. **Aryan is the synchronous bottleneck.** `aryan_client.get_merchant_v2()` is the only path to category and locality, and it's a remote HTTP call. If aryan is slow, every customer-info-pack call is slow. Worth caching aryan responses per merchant for ~24h before scaling engagement frequency. | |
| 2. **The 30-min Redis cache is keyed per-merchant** — fine for in-conversation reuse, but a daily engagement cron will miss this cache 100% of the time and pay the full snapshot-build cost per send. Consider a longer-TTL background-warmed cache for the engagement loop specifically. | |
| 3. **No version tracking on prompts today.** Every send loses the prompt-version provenance. The composer should record prompt version + context hash on every send so we can replay and A/B. | |
| --- | |
| ## Concrete recommendation for Phase 1 of the framework | |
| Phase 1 is mostly an **adapter layer**, not new infrastructure: | |
| | Layer | Effort | What it actually does | | |
| |---|---|---| | |
| | `CategoryContext` | Net new | Build for dentistry first (offer catalog, voice, peer stats, weekly digest, patient-content seed) | | |
| | `MerchantContext` | Adapter | Wrap existing `_merchant_snapshot` + `_behavioral_profile` + a new `_customer_aggregate` derived from BOTOPS chat history | | |
| | `TriggerContext` | Net new abstraction | Normalize the payloads existing crons emit into a single shape; new triggers (research_digest, recall_due) emit it natively | | |
| | `CustomerContext` | Partial adapter | Wrap existing `_customer_profile`, plus a new visit-history aggregator | | |
| | `EngagementComposer` | Net new | Single LLM-prompted module with versioned prompt; consumes the above | | |
| Phase 1 should NOT require modifying any existing agent. The adapter reads from existing state; the composer is a new module that runs in parallel; the new triggers (research digest, recall) are new crons that don't touch the existing matrix-followup or campaign-engagement loops. | |
| --- | |
| ## Appendix: file pointers for follow-up implementation | |
| - `agents/vera/merchant_agent.py:402` — VeraMerchantAgent init | |
| - `agents/vera/merchant_agent.py:740` — `_prefetch_merchant_context` (the main load) | |
| - `agents/vera/merchant_agent.py:899` — `_prefetch_product_context` (subscription + DA campaign + perf summary) | |
| - `agents/vera/merchant_agent.py:996` — system-prompt builder reads `_merchant_snapshot` | |
| - `agents/vera/merchant_agent.py:2171` — `handle_input` per-turn entry | |
| - `agents/vera/customer_incoming_agent.py:91` — CustomerIncomingAgent class | |
| - `agents/vera/customer_incoming_agent.py:221` — `_load_merchant_data` (single info-pack call + fallbacks) | |
| - `agents/base_agent.py:80-82` — MERCHANT_MCP_SERVER_URL config | |
| - `agents/base_agent.py:237-244` — class-level shared `_merchant_mcp_client` | |
| - `vera-mcp/src/tools/merchant_snapshot.py` — `vera_merchant_snapshot` tool | |
| - `vera-mcp/src/services/merchant_snapshot.py:51` — `build_merchant_snapshot` (the 13-call fan-out) | |
| - `vera-mcp/src/tools/merchant_info.py:188` — `vera_get_customer_info_pack` tool | |
| - `vera-mcp/src/tools/merchant_info.py:30` — `_resolve_place_id` chain (MongoDB → JSON → aryan) | |
| - `vera-mcp/src/services/aryan_client.py` — aryan HTTP client (`get_merchant_v2`, `get_mapping`) | |