brain / deploy /README.md
Leon4gr45's picture
Deploy complete BrainAPI source 42cde77950db
6663ad4 verified
|
Raw History Blame Contribute Delete
7.94 kB

BrainAPI deploy profiles

Choose heavy or light when bringing up a new deployment.

Heavy Light
Compose docker-compose.heavy.yaml docker-compose.light.yaml
Env template env.heavy.example env.light.example
Graph Neo4j NetworkX on Postgres
Data MongoDB PostgreSQL
Vectors Milvus PostgreSQL + pgvector
LLMs GCP Vertex (small) + Azure (large) DeepSeek (small + large)
Embeddings Azure OpenAI (text-embedding-3-large)
Container limits Full (current production) Smaller footprint (mem_limit / cpus)
Suggested VM ~8–16 GB RAM / 4+ vCPU ~2 GB RAM / 1 vCPU

Quick start

# 1. Copy a profile template outside the repository and replace every secret.
install -m 600 deploy/env.light.example /etc/brainapi/light.env
# Set BRAINAPI_ENV_FILE to the absolute file path in /etc/brainapi/light.env.

# 2. Validate first, then start and wait for health.
docker compose --env-file /etc/brainapi/light.env \
  -f deploy/docker-compose.light.yaml config -q
docker compose --env-file /etc/brainapi/light.env \
  -f deploy/docker-compose.light.yaml up -d --wait
# or
docker compose --env-file /etc/brainapi/heavy.env \
  -f deploy/docker-compose.heavy.yaml up -d --wait

Only nginx publishes host ports (80 and 443). The API, embedded Console, and MCP are reached through /, /console/, and /mcp. The checked-in TLS server rejects handshakes until an operator mounts a certificate-specific nginx server file; do not expose port 80 beyond a trusted proxy without HTTPS.

Light sizing

Service Heavy Light
nginx 128m / 0.2 CPU 64m / 0.1 CPU
redis 2048m / 0.5 CPU (maxmemory 1500mb) 256m / 0.1 CPU (maxmemory 192mb)
postgres — (not in heavy compose) 768m / 0.35 CPU
neo4j / milvus / mongo / etcd / minio present omitted
api / worker / mcp 2–4g / 1–2 CPU 768m / 0.5 CPU each
celery --concurrency 2 1
CELERY_WORKER_CONCURRENCY 4 1

Notes

  • BRAINAPI_IMAGE, BRAINAPI_ENV_FILE, Redis credentials, and profile backend credentials are required and fail during Compose interpolation when absent.
  • Mount cloud credentials with a private operator-owned Compose override; no host-specific credential path is present in the production profiles.
  • Do not mix profiles under the same Compose project name. Their volume layouts and restore order differ.

Backup and restore

Backups default to /srv/brainapi/backups and are created with mode 0700. They contain sensitive application data (including stored brain credentials), so encrypt and access-control the backup directory. The generated manifest contains only profile/version metadata, service names, restore order, and checksums; it never contains secrets.

deploy/brainapi-backup backup --profile light \
  --env-file /etc/brainapi/light.env --project-name brainapi
deploy/brainapi-backup verify --profile light \
  --env-file /etc/brainapi/light.env --archive /srv/brainapi/backups/light-UTCSTAMP
deploy/brainapi-backup restore --profile light \
  --env-file /etc/brainapi/light.env --archive /srv/brainapi/backups/light-UTCSTAMP

Backup stops nginx, API, MCP, and workers before capture. Restore refuses a running stack, a profile/image mismatch, a checksum mismatch, or any non-empty target volume. Use a documented maintenance window and retain the generated manifest with the release artifacts. Light backups include the system registry plus an individual custom-format dump for every brain_* PostgreSQL database.

Production validation

The CI-only Compose override supplies a deterministic OpenAI-compatible model stub; it is not a production deployment file. It lets both profiles exercise the complete storage and gateway path without external model credentials:

docker compose -f deploy/docker-compose.light.yaml \
  -f deploy/docker-compose.ci.yaml up -d --wait
python scripts/production_smoke.py exercise --profile light \
  --system-token "$BRAINPAT_TOKEN" --artifact-dir release-artifacts

After backup, clean-volume restore, and restart, run the same tool with verify-restore. CI retains the profile smoke, latency, restore, backup manifest, image audit, and SBOM evidence. The tag workflow downloads the exact commit's successful light and heavy artifacts and runs the release-readiness gate before any image can be published. Quality CI exports one linux/amd64 candidate archive; the heavy runner loads that same archive, and the tag workflow publishes it unchanged after verifying its recorded image ID.

Public documentation sandbox

The anonymous sandbox is an opt-in extension of the light profile. Merge env.public-demo.example into the deployment's private environment, deploy the current API and worker image, and seed only the dedicated agentdemo brain:

docker compose --env-file /etc/brainapi/light.env -f deploy/docker-compose.light.yaml up -d
docker compose --env-file /etc/brainapi/light.env -f deploy/docker-compose.light.yaml exec brainapi \
  python scripts/seed_public_demo.py --api-url http://localhost:8000

The seed command downloads the published V2 documentation, records its SHA-256 marker, skips an already-current corpus, and polls the asynchronous ingestion task. It requires the private system BRAINPAT_TOKEN inside the deployment; do not expose that token through the gateway. The seed command explicitly creates the brain; BRAIN_CREATION_ALLOWED does not enable implicit API provisioning.

Expose the API origin as api.brain-api.dev, with TLS, and put the Cloudflare gateway in front of root-domain /api/*. The application itself deliberately does not implement client rate limiting; the checked-in gateway owns the 30-request/60-second demo limit.

Brain registry contract

The persisted registry is authoritative. Retrieval, query, agent, and MCP access cannot provision an unknown brain, even with BRAIN_CREATION_ALLOWED=true. Explicit unknown API scopes return 404 BRAIN_NOT_FOUND. A missing scope can resolve to default only when DEFAULT_BRAIN_FALLBACK=true and that brain is already registered. Create brains with POST /system/brains or POST /system/workspaces; duplicate brain creation returns 409 BRAIN_ALREADY_EXISTS, including concurrent inserts.

Brain IDs accept 1–128 ASCII letters, digits, hyphens, and underscores, starting with a letter or digit. system is reserved (case insensitive). The same rules apply to create, clear, and delete; workspace slugs retain their narrower syntax. /system/brains-list returns persisted registry identities only. Orphan physical databases never appear as registered brains or acquire generated PATs.

Clear preserves registered identity and physical storage. Explicit delete also removes the PostgreSQL database, including a named orphan without a registry row. These are destructive operations: target only an intended identifier.

Run the deterministic gate and the opt-in real integration gate:

PYTHONPATH=.:tests python -m pytest -q tests/test_brain_registry_contract.py tests/test_brain_lifecycle.py
BRAINAPI_REGISTRY_TEST_CONTAINER=<disposable-local-container> \
  PYTHONPATH=.:tests python -m pytest -q tests/integration/test_brain_registry_contract.py

The integration gate requires a disposable local HF CPU container with the candidate source and its normal entrypoint environment. It invokes bash /app/entrypoint.sh --qualify-registry, compares actual PostgreSQL catalogs and application Redis state around unknown GET/search/context requests, tests concurrent storage inserts, and deletes only its own collision-checked temporary namespaces. It does not clean production probe artifacts or other existing brains.