docs/publishing_roadmap.md

Production Readiness & PyPI Publishing Roadmap

This document outlines the systematic, step-by-step transition of agentcache from a local development tool to an enterprise-grade, highly-available, production-ready service and distributable Python library.

As a persistent memory server for AI agents, agentcache must guarantee data consistency, fast search latency, high concurrent write handling, and secure operation under load. This roadmap addresses the structural, architectural, and operational improvements required to achieve this level of maturity.

---

graph TD
    A[Phase 1: Package Structure & API Gateway] -->|Done| B[Phase 2: Code Hygiene & Warnings]
    B -->|Done| C[Phase 3: Database & Concurrency Hardening]
    C -->|Next| D[Phase 4: Multi-Worker & Task Queue Safety]
    D --> E[Phase 5: Pluggable Storage & Vector Abstractions]
    E --> F[Phase 6: Security & Enterprise Observability]
    F --> G[Phase 7: Packaging, CI/CD & SDK Ecosystem]

    style A fill:#4D7C0F,stroke:#1E293B,color:#fff
    style B fill:#4D7C0F,stroke:#1E293B,color:#fff
    style C fill:#0369A1,stroke:#1E293B,color:#fff
    style D fill:#EA580C,stroke:#1E293B,color:#fff
    style E fill:#6D28D9,stroke:#1E293B,color:#fff
    style F fill:#B91C1C,stroke:#1E293B,color:#fff
    style G fill:#6B21A8,stroke:#1E293B,color:#fff

---

Phase 1: Package Restructuring & Packaging Design (Completed)

Adopted the standard Python src/ layout to ensure clean namespace isolation and prevent installation path collisions.

• Namespace Migration: Relocated core modules from flat src/ to src/agentcache/.
• Top-Level Exports: Configured src/agentcache/__init__.py to expose programmatic endpoints (StateKV, create_app, folder_observe, remember, etc.) for clean client imports.
• Package Resources: Updated app.py to use importlib.resources.files to resolve the static HTML viewer directory path, avoiding fragile __file__ calculations.
• Command Line Entrypoint: Configured agentcache = agentcache.cli:main as the official console script entrypoint in pyproject.toml.

---

Phase 2: Code Hygiene & Warning Resolution (Completed)

Addressed deprecation warnings and database queries that would cause operational errors under load.

• Python 3.12+ Datetime Compliance: Replaced naive datetime.datetime.utcnow() with timezone-aware datetime.datetime.now(datetime.timezone.utc) and standard formatters, eliminating 200+ deprecation warnings during testing.
• SQLite Parameterization Fix: Cleaned up SQL syntax parameter placeholders from %s (Postgres/MySQL) to ? (SQLite) inside index cleanup operations, resolving silent cleanup failures and preventing database bloat.
• Test Path Cleanup: Eliminated manual sys.path.insert hacks in the tests/ directory, moving to package-level imports utilizing editable install hooks (pip install -e .).

---

Phase 3: Database Concurrency & SQLite Hardening (Active)

To run in production with multiple concurrent agents writing logs, the database layer must prevent locking bottlenecks and handle concurrent access gracefully.

1. Advanced SQLite WAL Tuning

SQLite WAL (Write-Ahead Logging) is already enabled, but it needs specific pragmas optimized for concurrent reads/writes:

• PRAGMA synchronous = NORMAL; (Reduces disk I/O, extremely safe in WAL mode because WAL is transactional and checksummed).
• PRAGMA journal_size_limit = 67108864; (Caps WAL file size at 64MB, preventing unbounded WAL growth and optimizing read scan performance).
• PRAGMA mmap_size = 268435456; (Enables memory-mapped I/O up to 256MB, greatly accelerating read queries and search operations).

2. Lock & Busy State Handlers

SQLite throws SQLITE_BUSY when a write connection blocks another. We must configure a longer timeout and implement a pythonic backoff decorator for database transactions.

# In src/agentcache/db.py connection setup
conn = sqlite3.connect(self.db_path, check_same_thread=False, timeout=30.0)

• Transaction Retry Decorator: Wrap database updates and transaction commits in a retry loop with randomized exponential backoff (e.g., tenacity or custom implementation) to handle transient database locks under high concurrent write loads.

3. Explicit SQLite Write Transaction Lock

Ensure all methods executing writes (set, delete, update, commit_version) acquire an internal thread lock (self._lock) to serialize write requests inside the Python process before they reach the database driver.

---

Phase 4: Multi-Worker & Task Queue Safety

When deployed in a production WSGI container (e.g., Gunicorn with multiple worker processes), background threads running inside the Flask app (src/agentcache/workers.py for indexing and auto-forget sweeps) spawn multiple times, causing race conditions and database locks.

       [ Client Request ]
               │
      ┌────────┴────────┐
      ▼ (WSGI Master)   ▼
 ┌─────────────┐   ┌─────────────┐
 │  Worker P1  │   │  Worker P2  │
 ├─────────────┤   ├─────────────┤
 │ - App Code  │   │ - App Code  │
 │ - Thread:   │   │ - Thread:   │
 │   Index/    │   │   Index/    │
 │   AutoForget│   │   AutoForget│
 └──────┬──────┘   └──────┬──────┘
        └────────┬────────┘
                 ▼
          [ sqlite3.db ] (Locks & Collision Risk)

Action Items

1. Single-Worker Sweep Locking: Implement a database-backed distributed lock (stored in a mem:locks metadata scope) to ensure that only one worker runs index-rebuild or auto-forget sweeps at any given time.
2. Process-Level CLI Commands: Create CLI commands specifically for running background maintenance, allowing users to run workers as distinct sidecar containers:

   # Run server without background worker threads
   agentcache serve --no-workers
   
   # Run background workers in a dedicated process
   agentcache worker --tasks=index,forget
   

3. Graceful Shutdown Hook (SIGTERM/SIGINT): Ensure the WSGI worker captures shutdown signals and flushes dirty BM25 and Vector indices to SQLite before terminating, preventing index corruption.

---

Phase 5: Pluggable Storage & Vector Abstractions

A pure SQLite and in-memory index design is perfect for single-user desktops, but production deployment requires scaling to millions of observations and sharing memories across cluster deployments.

1. Abstract Storage Layer (BaseStorage)

Define a clean abstract base class (or Protocol) for key-value storage:

class BaseStorage(abc.ABC):
    @abc.abstractmethod
    def get(self, scope: str, key: str) -> Optional[Any]: ...
    @abc.abstractmethod
    def set(self, scope: str, key: str, value: Any) -> Any: ...
    @abc.abstractmethod
    def delete(self, scope: str, key: str) -> bool: ...
    @abc.abstractmethod
    def list(self, scope: str) -> List[Any]: ...

• Implement SQLiteStorage (current default).
• Implement PostgreSQLStorage (utilizing JSONB for production scale).
• Implement RedisStorage (for high-speed caching environments).

2. Pluggable Vector Indexing

Move beyond keeping all vector embeddings in memory as float32 lists. Create a BaseVectorIndex abstraction supporting external vector databases:

• InMemoryVectorIndex (current default for local setups).
• ChromaVectorIndex / QdrantVectorIndex (stores and indexes embeddings out-of-process, enabling fast hybrid search query times and low memory footprints).

---

Phase 6: Security & Enterprise Observability

To operate safely as a shared team backend or internet-facing service, authentication and telemetry must be hardened.

1. Timing-Safe Multi-Token Authentication

• Enhance the current AGENTCACHE_SECRET system to support timing-safe HMAC verification against multiple authorized API keys (e.g. read-only vs. read-write access tokens).
• Add support for standard Bearer Token Authorization headers in addition to query parameter fallbacks.

2. Structured JSON Logging & Observability

• Replace all raw print statements with standard Python logging.
• Configure structured JSON layout formatting (e.g. python-json-logger) for simple ingestion into centralized log aggregators (Datadog, ELK, AWS CloudWatch).
• Add Prometheus-style /metrics endpoint exposing:
  • Total observations/memories counts
  • Query execution durations (BM25 vs. Vector vs. Hybrid search)
  • Active WebSocket connections
  • Database connection pool sizes and WAL sizes

3. API Rate Limiting

• Integrate Flask-Limiter to protect search and write routes from spam or runaway loops originating from local agent workflows.

---

Phase 7: Packaging, CI/CD & SDK Ecosystem

Once the codebase is robust, we must establish automation to compile, test, and release the library seamlessly.

1. Trusted Publishing Setup (OIDC)

By configuring PyPI OIDC Trusted Publishing, we authorize our GitHub Actions repository to request short-lived access tokens from PyPI, eliminating the need to save long-lived PyPI API tokens in GitHub Secrets.

PyPI Configuration Steps:

1. Log in to pypi.org.
2. Navigate to Account Settings $\to$ Publishing.
3. Create a GitHub publisher:
   • Repository Owner: Yash030
   • Repository Name: agentcache-python
   • Workflow Name: automation.yml (matching the GHA file)
   • Environment Name: (Leave empty)

2. SDK Development & Hook Scripts

A raw REST API requires clients to implement boilerplate connection logic.

• Python Client SDK: Separate the connection client src/agentcache/connect.py into a lightweight, standalone pip package agentcache-client or bundle it as a high-level programmatic wrapper:

  from agentcache import AgentCacheClient
  client = AgentCacheClient(url="http://localhost:3111", secret="...")
  client.observe(folder_path="./src", agent_id="coder-1", message="Refactored imports")
  

• TypeScript Client SDK: Build a corresponding JavaScript/TypeScript client library (@agentcache/client) to simplify integration with Node.js-based agent tools (like Cline, Cursor, or AutoGPT).
• IDE Launch Scripts: Build pre-packaged scripts and configuration recipes for Cursor (.cursorrules), Cline (mcpSettings.json), and Claude CLI (config.json) to automate the setup process for new users.

---

Execution Checklist & Prioritization

Phase	Milestone	Priority	Risk	Effort
3	Add transaction retry backoff logic & WAL tuning pragmas	Critical	Low	Low
4	Multi-worker safety (worker locks & background thread flags)	High	Medium	Medium
6	Replace print statements with structured standard logging	High	Low	Medium
5	Implement storage interface abstraction layers	Medium	High	High
6	Implement Prometheus /metrics and HTTP rate limiting	Medium	Low	Low
7	Configure PyPI OIDC & publish package	High	Low	Low
7	Create npm/pip client SDK wrapper libraries	Low	Low	High