Spaces:

Benaah
/

MicProtocol

Running

App Files Files Community

MicProtocol / docs /self-hosted-stack.md

Benaah

Upload folder using huggingface_hub

b17baae verified about 20 hours ago

preview code

Raw History Blame Contribute Delete

17.9 kB

# Self-Hosted Stack

MicProtocol ships a complete self-hostable integration fleet in `docker-compose.yml`. Every service replaces a paid SaaS equivalent, giving you maximum data ownership, no per-seat licensing, and zero vendor lock-in.

This doc is the unified operations reference. For per-integration API behavior and webhook side effects, see [Third-Party & Self-Hosted Integrations](./third-party-integrations.md) and [Webhooks](./webhooks.md).

## Architecture

```mermaid
graph TD
    API[MicProtocol NestJS API :4000] --> POSTGRES[(PostgreSQL pgvector :5432)]
    API --> REDIS[(Redis :6379)]
    API --> MEILI[(Meilisearch :7700)]
    API --> MINIO[(MinIO :9001)]
    API --> MICRT[mic-realtime :4001]
    API -. webhook .-> CALCOM[Cal.com :3002]
    API -. webhook .-> FORMBRICKS[Formbricks stack :3001]
    API -. webhook .-> CHATWOOT[Chatwoot :3005]
    API -. webhook .-> DOCUMENSO[Documenso :3004]
    API -. webhook .-> LISTMONK[Listmonk :9000]
    API -. webhook .-> LIVEKIT[LiveKit :7880]
    CALCOM --> POSTGRES
    FORMBRICKS --> POSTGRES
    CHATWOOT --> POSTGRES
    DOCUMENSO --> POSTGRES
    LISTMONK --> POSTGRES
    UMAMI --> POSTGRES
    CALCOM --> REDIS
    CHATWOOT --> REDIS
    FORMBRICKS --> REDIS
    CALCOM --> MINIO
    DOCUMENSO --> MINIO
    FORMBRICKS --> MINIO
    MAILHOG[Mailhog :8025] -. SMTP .-> ALL[All services]
    style API fill:#e1f5e1
    style MICRT fill:#fff3e0
```

## Service Catalog

### Core Infrastructure

| Service | Container | Image | Port(s) | Database |
|---|---|---|---|---|
| PostgreSQL | `micprotocol-postgres` | `pgvector/pgvector:pg16` | 5432 | All databases (via `init-databases.sql`) |
| Redis | `micprotocol-redis` | `redis:7-alpine` | 6379 | n/a (in-memory) |
| Meilisearch | `micprotocol-meilisearch` | `getmeili/meilisearch:v1.6` | 7700 | n/a (own volume) |
| MinIO (S3 storage) | `micprotocol-minio` | `minio/minio:latest` | 9001 (API), 9090 (Console) | n/a (own volume) |
| MinIO bucket creator | `micprotocol-minio-mc` | `minio/mc:latest` | - (one-shot) | n/a |
| Mailhog (dev SMTP) | `micprotocol-mailhog` | `mailhog/mailhog:latest` | 1025 (SMTP), 8025 (UI) | n/a |

### MicProtocol Apps

| Service | Container | Port | Notes |
|---|---|---|---|
| API | `micprotocol-api` | 4000 | tRPC + REST + Swagger at `/api/docs` |
| Background worker | `micprotocol-worker` | - | Same app image as the API (no published ports); runs BullMQ WorkerHosts incl. the Python clip-scoring subprocess |
| mic-realtime (Erlang/OTP) | `micprotocol-mic-realtime` | 4001 | Chat fanout + presence |
| Web | `micprotocol-web` | 3000 | Next.js 14 |
| Admin | `micprotocol-admin` | 3006 | Next.js platform admin |

### Self-Hosted Integrations

| Service | Container(s) | Port | Replacement | DB schema |
|---|---|---|---|---|
| Cal.com | `micprotocol-calcom` | 3002 | Calendly | `calcom` |
| Formbricks v4 | `micprotocol-formbricks` + `-hub` + `-cube` + 2 migrate jobs | 3001 (web), 8080 (hub), 4000 (cube) | Typeform / SurveyMonkey | `formbricks` (pgvector) |
| Documenso | `micprotocol-documenso` | 3004 | DocuSign / PandaDoc | `documenso` |
| Chatwoot | `micprotocol-chatwoot-web` + `-worker` | 3005 | Intercom / Zendesk | `chatwoot` (pgvector) |
| Listmonk | `micprotocol-listmonk` | 9000 | Mailchimp / ConvertKit | `listmonk` |
| Umami | `micprotocol-umami` | 3003 | Google Analytics / Plausible cloud | `umami` |
| LiveKit | `micprotocol-livekit` | 7880 (HTTP), 7881 (TCP), 7882 (UDP) | Twilio Video / Zoom SDK | n/a |

### Formbricks v4 substack

```mermaid
graph LR
    MIGRATE1[migrate job] --> DB[(Postgres formbricks DB)]
    MIGRATE2[hub migrate job] --> DB
    HUB[formbricks-hub :8080] --> DB
    HUB --> CUBE[cube.js :4000]
    CUBE --> DB
    WEB[formbricks :3001] --> HUB
    WEB --> CUBE
    WEB --> DB
    WEB --> REDIS[(Redis)]
```

Containers in the Formbricks substack:

1. `micprotocol-formbricks-migrate` (one-shot, runs Prisma migrations)
2. `micprotocol-formbricks` (web app, `:3001`)
3. `micprotocol-formbricks-hub-migrate` (one-shot, runs goose + river migrations)
4. `micprotocol-formbricks-hub` (Hub API, `:8080`)
5. `micprotocol-formbricks-cube` (Cube.js analytics, `:4000`)

Formbricks-specific runbook: [`formbricks-runbook.md`](./formbricks-runbook.md).

## Port Map

Fixed ports in `docker-compose.yml` — change in `docker-compose.yml` and matching env vars together:

| Port | Service |
|---|---|
| 3000 | Web (also guest portal) |
| 3001 | Formbricks |
| 3002 | Cal.com |
| 3003 | Umami |
| 3004 | Documenso |
| 3005 | Chatwoot |
| 3006 | Admin |
| 4000 | API |
| 4001 | mic-realtime |
| 5432 | Postgres |
| 6379 | Redis |
| 7700 | Meilisearch |
| 7880 | LiveKit HTTP |
| 7881 | LiveKit TCP |
| 7882 | LiveKit UDP |
| 8025 | Mailhog UI |
| 9000 | Listmonk |
| 9001 | MinIO API |
| 9090 | MinIO Console |
| 1025 | Mailhog SMTP |

## Database Layout

`init-databases.sql` runs on first Postgres boot and creates one database per integration:

| Database | Owner | pgvector | pgcrypto |
|---|---|---|---|
| `micprotocol` (default) | MicProtocol API | - | yes |
| `chatwoot` | Chatwoot | yes | yes |
| `formbricks` | Formbricks | yes | yes |
| `calcom` | Cal.com | - | yes |
| `documenso` | Documenso | - | yes |
| `listmonk` | Listmonk | - | yes |
| `umami` | Umami | - | yes |

If a volume already exists, `init-databases.sql` will not run. To bootstrap missing databases on an existing volume:

```bash
docker exec -i micprotocol-postgres psql -U micprotocol -d micprotocol < scripts/init-databases.sql
```

## Shared Object Storage (MinIO)

MinIO is the default storage backend for every self-hosted integration that uploads files. `minio-createbuckets` is a one-shot service that creates:

| Bucket | Owner |
|---|---|
| `micprotocol-storage` | MicProtocol API (media, headshots, clips) |
| `calcom-uploads` | Cal.com |
| `documenso-uploads` | Documenso |
| `formbricks-uploads` | Formbricks |

`micprotocol-storage` is set to public-read (used for guest-facing media). The others are private.

In production, point MicProtocol at Cloudflare R2 instead of MinIO by changing `R2_ENDPOINT`, `R2_ACCESS_KEY_ID`, `R2_SECRET_ACCESS_KEY`, `R2_BUCKET`, and `STORAGE_PUBLIC_URL` — no code changes required, both speak the S3 API.

## Shared SMTP (Mailhog in dev)

Every self-hosted integration sends outbound mail through the same SMTP settings:

```
SMTP_HOST=mailhog          # in compose network
SMTP_PORT=1025
SMTP_USER=                 # empty for Mailhog
SMTP_PASSWORD=             # empty for Mailhog
EMAIL_FROM=noreply@<yourdomain>
```

In production, swap Mailhog for your transactional relay (Resend, SES, Postmark, or self-hosted Postal/Haraka) by setting `SMTP_HOST`, `SMTP_PORT`, `SMTP_USER`, `SMTP_PASSWORD`, and `EMAIL_FROM` to the provider's values. The integrations read these env vars directly — no code changes.

## Shared Redis (namespaced)

Chatwoot, Formbricks, and Cal.com all share the `micprotocol-redis` instance. Chatwoot and Cal.com namespace keys internally; Formbricks uses Redis for rate-limiting and Cube.js queue. There is currently no isolation beyond provider namespacing — for high-throughput production, split into one Redis per integration.

## Co-located Secrets

Every self-hosted service reads its secrets from environment variables that the API also needs. The complete list lives in [`.env.example`](../.env.example). Critical guidance:

| Category | Examples | Rotation policy |
|---|---|---|
| Postgres / Redis passwords | `POSTGRES_PASSWORD`, `REDIS_PASSWORD` | Rotatable; requires service restart |
| Encryption keys (immutable) | `FORMBRICKS_ENCRYPTION_KEY`, `DOCUMENSO_ENCRYPTION_KEY`, `CALCOM_ENCRYPTION_KEY`, `NEXTAUTH_SECRET` variants | Never rotate after first start; old encrypted data becomes unrecoverable |
| Webhook signatures | `*_WEBHOOK_SECRET` | Rotatable; update on both sides simultaneously |
| Admin / API tokens | `CALCOM_API_KEY`, `LISTMONK_API_TOKEN`, `CHATWOOT_API_KEY` | Rotatable via the admin UI of each integration |
| App secrets | `UMAMI_APP_SECRET`, `FORMBRICKS_CRON_SECRET` | Rotatable |

Use `bash scripts/generate-secrets.sh` to produce strong random values for every variable in one shot. See [Getting Started → Environment Configuration](./getting-started.md#3-environment-configuration) for details.

## Service-Specific Configuration Notes

### Cal.com

- Uses its own `calcom` database (Prisma migrations need a clean schema).
- `NEXT_PUBLIC_SINGLE_ORG_SLUG=micprotocol` and `ORGANIZATIONS_ENABLED=0` disable paid multi-org features for self-hosted single-tenant use.
- Generate an API key in Cal.com Settings → API Keys; set `CALCOM_API_KEY` in `.env`.
- Encrypts DB columns with `CALCOM_ENCRYPTION_KEY` (24-char hex, **never rotate** after init).

### Formbricks v4

- Five containers, must start in order: `formbricks-migrate` → `formbricks-hub-migrate` → `formbricks-cube` → `formbricks-hub` → `formbricks`. Compose `depends_on` chains handle this automatically.
- `SKIP_STARTUP_MIGRATION=true` prevents the web container from re-migrating (the dedicated migrate job already did it).
- `SKIP_STARTUP_MIGRATION` must remain `true`; running migrations inside the web container will race with the migrate job.
- Analytics require Cube.js (`CUBEJS_API_SECRET` must match between web and cube containers).
- Sign-up is disabled without an invite (`EMAIL_VERIFICATION_DISABLED=1`).

### Documenso

- Requires a signing certificate. Generate with the commands in [`scripts/documenso-cert/README.md`](../scripts/documenso-cert/README.md).
- Mount the cert directory read-only at `/opt/documenso/cert`; the app looks for `cert.p12` inside.
- `DOCUMENSO_ENCRYPTION_KEY` and `DOCUMENSO_ENCRYPTION_SECONDARY_KEY` are both required and **never rotated**.

### Chatwoot

- Rails + Sidekiq worker pair (`chatwoot-web` + `chatwoot-worker`).
- Uses the `chatwoot` database with `pgvector` (required by Chatwoot itself).
- Storage is local (`RAILS_STORAGE_LOCATION=/app/storage`, mounted as `chatwoot_storage` volume). For larger deployments, switch `ACTIVE_STORAGE_SERVICE` to S3/MinIO.
- `FRONTEND_URL` must match the public URL or the live-chat widget will fail to load.
- Generate `SECRET_KEY_BASE` with `openssl rand -hex 64`.

### Listmonk

- Container runs `./listmonk --install --idempotent --yes && ./listmonk --upgrade --yes && ./listmonk --config ''` on first boot — idempotent, safe on restarts.
- `LISTMONK_db__database=listmonk` (must match the name in `init-databases.sql`).
- API token is generated in Listmonk Admin → Settings → API; set `LISTMONK_API_TOKEN` in `.env`.
- `LISTMONK_ADMIN_USER`/`LISTMONK_ADMIN_PASSWORD` bootstrap the super-admin on first run.

### Umami

- Single container, Postgres-backed.
- `UMAMI_APP_SECRET` is required; generate with `openssl rand -base64 32`.
- Add a website in the Umami UI and copy the website ID to `NEXT_PUBLIC_UMAMI_WEBSITE_ID`.

### LiveKit

- SFU for the in-app recording studio. MicProtocol owns the browser client (`apps/web`); LiveKit owns the WebRTC media routing.
- Dev defaults (`devkey` / `devsecret`) work in Compose but **must be changed for production**.
- Webhooks from LiveKit back to the API call `POST /integrations/webhooks/livekit` and trigger `RecordingSession` state transitions.
- For internet-facing deployments, expose UDP `7882` and configure `rtc.use_external_ip: true` (or run behind a TURN server).

### mic-realtime

Erlang/OTP service that owns:

- Chat fanout (per-workspace conversation messages)
- Presence (who is online in a workspace)
- Live domain events (published by the NestJS API via `/internal/dispatch`)

Key env vars:

| Var | Default | Notes |
|---|---|---|
| `MIC_REALTIME_PORT` | 4001 | HTTP listener |
| `MIC_REALTIME_HOST` | 0.0.0.0 | Bind address |
| `MICPROTOCOL_API_URL` | http://api:4000 | Used by mic-realtime to call back into the API |
| `MIC_REALTIME_INTERNAL_TOKEN` | (change me) | Bearer used by API to publish events |
| `MIC_REALTIME_SHARED_SECRET` | (change me) | JWT signing secret for WS auth tokens |
| `MIC_REALTIME_PUBLIC_URL` | ws://localhost:4001 | Public WS URL handed to web clients |
| `MIC_REALTIME_MAX_MSG` | 65536 | Max message size in bytes |
| `MIC_REALTIME_RATE_LIMIT` | 120 | Messages per minute per connection |
| `MIC_REALTIME_PRESENCE_TTL` | 90 | Presence TTL (seconds) |

Both `MIC_REALTIME_INTERNAL_TOKEN` and `MIC_REALTIME_SHARED_SECRET` must be strong random strings in production.

#### Production deployment

`mic-realtime` ships in every deployment topology:

| Topology | Where | Public route |
|---|---|---|
| Local dev | `docker-compose.yml` (service `mic-realtime`) | `ws://localhost:4001` |
| Coolify | `docker-compose.coolify.yml` (service `mic-realtime`, internal `:4001`) | `wss://realtime.<root>` via Caddy |
| Render | `render.yaml` (`micprotocol-realtime`, `rootDir: services/mic-realtime`) | `wss://micprotocol-realtime.onrender.com` (or custom domain) |

The [Caddyfile](../Caddyfile) routes `realtime.<root>` to `mic-realtime:4001`
(the specific host block wins over the `*.` wildcard). Caddy handles the
WebSocket upgrade transparently on the same route, so one URL serves both the
`GET /healthz` probe and `GET /ws?token=...`.

Secrets are **fail-closed**: `RealtimeClient`
(`apps/api/src/chat/realtime.client.ts`) and `mic_config.erl` both refuse to
start in production when `MIC_REALTIME_INTERNAL_TOKEN` /
`MIC_REALTIME_SHARED_SECRET` are missing or equal to the known defaults.
Generate strong values (`openssl rand -hex 32`) and set
`MIC_REALTIME_PUBLIC_URL` to the public `wss://` URL; the web/admin frontends
receive it as `NEXT_PUBLIC_MIC_REALTIME_URL`. The Coolify stack defaults the
API→realtime base URL to the internal `http://mic-realtime:4001`
(`MIC_REALTIME_URL`), so API→realtime callbacks never leave the Docker
network.

> ⚠️ Render **free** instances sleep after ~15 min of inactivity and cap at
> 750 hours/month — a sleeping WebSocket engine drops every live chat/presence
> socket. Use `plan: starter` or higher for realtime in production.

## Start / Stop / Update

### Bring up the full stack

```bash
docker compose up -d
```

### Bring up only infra (for local app dev)

```bash
docker compose up -d postgres redis meilisearch minio minio-createbuckets
```

### Rolling update a single integration

```bash
docker compose pull formbricks
docker compose up -d --no-deps formbricks
```

Formbricks v4 migrations need the migrate jobs to re-run after a version bump:

```bash
docker compose up formbricks-migrate
docker compose up formbricks-hub-migrate
docker compose up -d --no-deps formbricks formbricks-hub formbricks-cube
```

### Tear down

```bash
docker compose down            # keep data
docker compose down -v         # wipe all volumes (destructive)
```

## Health Endpoints

| Service | Endpoint |
|---|---|
| API | `GET http://localhost:4000/health` |
| Cal.com | `GET http://localhost:3002/api/health` |
| Formbricks web | `GET http://localhost:3001/api/health` |
| Formbricks Hub | `GET http://localhost:8080/health` |
| Formbricks Cube | `GET http://localhost:4000/formbricks/readyz` (via cube container) |
| Chatwoot | `GET http://localhost:3005/health` |
| Listmonk | `GET http://localhost:9000/health` |
| Documenso | `GET http://localhost:3004/api/health` |
| Umami | `GET http://localhost:3003/api/health` |
| MinIO | `GET http://localhost:9001/minio/health/live` |
| Meilisearch | `GET http://localhost:7700/health` |
| mic-realtime | `GET http://localhost:4001/healthz` |
| LiveKit | `GET http://localhost:7880/health` (HTTP only; UDP port 7882 for WebRTC) |
| Mailhog | UI at `:8025` always returns 200 |

The API exposes a consolidated integration health check at:

```bash
curl -s http://localhost:4000/integrations/health | jq
```

This pings each self-hosted service's health endpoint in parallel and reports reachability.

### Integration Management Dashboard

The Admin panel (`apps/admin`) provides a full management interface for all self-hosted integrations:

| Feature | Description |
|---|---|
| **Health Monitoring** | Real-time health status, latency, and version for Chatwoot, Cal.com, Formbricks, Listmonk, Documenso, Umami, and LiveKit |
| **Webhook Secret Management** | View and rotate shared webhook secrets directly from the admin panel |
| **Admin Panel Links** | Direct "Open Admin" buttons that open each integration's native admin UI in a new tab |
| **Credential Management** | Connect, update, or disconnect API keys for self-hosted integrations |
| **Webhook Delivery Log** | Recent webhook deliveries with status, latency, and error details |
| **Provider Metrics** | Per-provider request counts, average latency, and error rates |

Access these features from the **Monitoring** and **Integrations** tabs in the admin navigation sidebar.

## Resource Limits

Default `deploy.resources.limits` in `docker-compose.yml`:

| Service | CPUs | Memory |
|---|---|---|
| API | 2 | 2G |
| Worker | 2 | 2G |
| mic-realtime | 1.5 | 1G |

Other services have no explicit limits — tune for your hardware. CLIP scoring (Python ML subprocess) runs inside the worker and is by far the most CPU- and memory-hungry: see [CLIP Scoring Engine](./clip-scoring.md) for scale guidance.

## Alternatives if You Skip Self-Hosting

If you don't want to run a particular self-hosted service, simply remove its block from `docker-compose.yml` and omit its env vars from `.env`. The MicProtocol API degrades gracefully — each integration's API client is called only when its corresponding URL env var is set. Missing integrations return `false` from the health check and the API logs a warning rather than 500-ing.

For the externally-hosted (non-self-hosted) integrations (Clerk, Resend, Cloudflare R2, NVIDIA NIM, DeepSeek, Moonshot, Stripe, Paystack, M-Pesa, Africa's Talking, Twilio, Google/Outlook calendars), see [Third-Party & Self-Hosted Integrations](./third-party-integrations.md).

	# Self-Hosted Stack

	MicProtocol ships a complete self-hostable integration fleet in `docker-compose.yml`. Every service replaces a paid SaaS equivalent, giving you maximum data ownership, no per-seat licensing, and zero vendor lock-in.

	This doc is the unified operations reference. For per-integration API behavior and webhook side effects, see [Third-Party & Self-Hosted Integrations](./third-party-integrations.md) and [Webhooks](./webhooks.md).

	## Architecture

	```mermaid
	graph TD
	API[MicProtocol NestJS API :4000] --> POSTGRES[(PostgreSQL pgvector :5432)]
	API --> REDIS[(Redis :6379)]
	API --> MEILI[(Meilisearch :7700)]
	API --> MINIO[(MinIO :9001)]
	API --> MICRT[mic-realtime :4001]

	API -. webhook .-> CALCOM[Cal.com :3002]
	API -. webhook .-> FORMBRICKS[Formbricks stack :3001]
	API -. webhook .-> CHATWOOT[Chatwoot :3005]
	API -. webhook .-> DOCUMENSO[Documenso :3004]
	API -. webhook .-> LISTMONK[Listmonk :9000]
	API -. webhook .-> LIVEKIT[LiveKit :7880]

	CALCOM --> POSTGRES
	FORMBRICKS --> POSTGRES
	CHATWOOT --> POSTGRES
	DOCUMENSO --> POSTGRES
	LISTMONK --> POSTGRES
	UMAMI --> POSTGRES

	CALCOM --> REDIS
	CHATWOOT --> REDIS
	FORMBRICKS --> REDIS

	CALCOM --> MINIO
	DOCUMENSO --> MINIO
	FORMBRICKS --> MINIO

	MAILHOG[Mailhog :8025] -. SMTP .-> ALL[All services]

	style API fill:#e1f5e1
	style MICRT fill:#fff3e0
	```

	## Service Catalog

	### Core Infrastructure

	\| Service \| Container \| Image \| Port(s) \| Database \|
	\|---\|---\|---\|---\|---\|
	\| PostgreSQL \| `micprotocol-postgres` \| `pgvector/pgvector:pg16` \| 5432 \| All databases (via `init-databases.sql`) \|
	\| Redis \| `micprotocol-redis` \| `redis:7-alpine` \| 6379 \| n/a (in-memory) \|
	\| Meilisearch \| `micprotocol-meilisearch` \| `getmeili/meilisearch:v1.6` \| 7700 \| n/a (own volume) \|
	\| MinIO (S3 storage) \| `micprotocol-minio` \| `minio/minio:latest` \| 9001 (API), 9090 (Console) \| n/a (own volume) \|
	\| MinIO bucket creator \| `micprotocol-minio-mc` \| `minio/mc:latest` \| - (one-shot) \| n/a \|
	\| Mailhog (dev SMTP) \| `micprotocol-mailhog` \| `mailhog/mailhog:latest` \| 1025 (SMTP), 8025 (UI) \| n/a \|

	### MicProtocol Apps

	\| Service \| Container \| Port \| Notes \|
	\|---\|---\|---\|---\|
	\| API \| `micprotocol-api` \| 4000 \| tRPC + REST + Swagger at `/api/docs` \|
	\| Background worker \| `micprotocol-worker` \| - \| Same app image as the API (no published ports); runs BullMQ WorkerHosts incl. the Python clip-scoring subprocess \|
	\| mic-realtime (Erlang/OTP) \| `micprotocol-mic-realtime` \| 4001 \| Chat fanout + presence \|
	\| Web \| `micprotocol-web` \| 3000 \| Next.js 14 \|
	\| Admin \| `micprotocol-admin` \| 3006 \| Next.js platform admin \|

	### Self-Hosted Integrations

	\| Service \| Container(s) \| Port \| Replacement \| DB schema \|
	\|---\|---\|---\|---\|---\|
	\| Cal.com \| `micprotocol-calcom` \| 3002 \| Calendly \| `calcom` \|
	\| Formbricks v4 \| `micprotocol-formbricks` + `-hub` + `-cube` + 2 migrate jobs \| 3001 (web), 8080 (hub), 4000 (cube) \| Typeform / SurveyMonkey \| `formbricks` (pgvector) \|
	\| Documenso \| `micprotocol-documenso` \| 3004 \| DocuSign / PandaDoc \| `documenso` \|
	\| Chatwoot \| `micprotocol-chatwoot-web` + `-worker` \| 3005 \| Intercom / Zendesk \| `chatwoot` (pgvector) \|
	\| Listmonk \| `micprotocol-listmonk` \| 9000 \| Mailchimp / ConvertKit \| `listmonk` \|
	\| Umami \| `micprotocol-umami` \| 3003 \| Google Analytics / Plausible cloud \| `umami` \|
	\| LiveKit \| `micprotocol-livekit` \| 7880 (HTTP), 7881 (TCP), 7882 (UDP) \| Twilio Video / Zoom SDK \| n/a \|

	### Formbricks v4 substack

	```mermaid
	graph LR
	MIGRATE1[migrate job] --> DB[(Postgres formbricks DB)]
	MIGRATE2[hub migrate job] --> DB
	HUB[formbricks-hub :8080] --> DB
	HUB --> CUBE[cube.js :4000]
	CUBE --> DB
	WEB[formbricks :3001] --> HUB
	WEB --> CUBE
	WEB --> DB
	WEB --> REDIS[(Redis)]
	```

	Containers in the Formbricks substack:

	1. `micprotocol-formbricks-migrate` (one-shot, runs Prisma migrations)
	2. `micprotocol-formbricks` (web app, `:3001`)
	3. `micprotocol-formbricks-hub-migrate` (one-shot, runs goose + river migrations)
	4. `micprotocol-formbricks-hub` (Hub API, `:8080`)
	5. `micprotocol-formbricks-cube` (Cube.js analytics, `:4000`)

	Formbricks-specific runbook: [`formbricks-runbook.md`](./formbricks-runbook.md).

	## Port Map

	Fixed ports in `docker-compose.yml` — change in `docker-compose.yml` and matching env vars together:

	\| Port \| Service \|
	\|---\|---\|
	\| 3000 \| Web (also guest portal) \|
	\| 3001 \| Formbricks \|
	\| 3002 \| Cal.com \|
	\| 3003 \| Umami \|
	\| 3004 \| Documenso \|
	\| 3005 \| Chatwoot \|
	\| 3006 \| Admin \|
	\| 4000 \| API \|
	\| 4001 \| mic-realtime \|
	\| 5432 \| Postgres \|
	\| 6379 \| Redis \|
	\| 7700 \| Meilisearch \|
	\| 7880 \| LiveKit HTTP \|
	\| 7881 \| LiveKit TCP \|
	\| 7882 \| LiveKit UDP \|
	\| 8025 \| Mailhog UI \|
	\| 9000 \| Listmonk \|
	\| 9001 \| MinIO API \|
	\| 9090 \| MinIO Console \|
	\| 1025 \| Mailhog SMTP \|

	## Database Layout

	`init-databases.sql` runs on first Postgres boot and creates one database per integration:

	\| Database \| Owner \| pgvector \| pgcrypto \|
	\|---\|---\|---\|---\|
	\| `micprotocol` (default) \| MicProtocol API \| - \| yes \|
	\| `chatwoot` \| Chatwoot \| yes \| yes \|
	\| `formbricks` \| Formbricks \| yes \| yes \|
	\| `calcom` \| Cal.com \| - \| yes \|
	\| `documenso` \| Documenso \| - \| yes \|
	\| `listmonk` \| Listmonk \| - \| yes \|
	\| `umami` \| Umami \| - \| yes \|

	If a volume already exists, `init-databases.sql` will not run. To bootstrap missing databases on an existing volume:

	```bash
	docker exec -i micprotocol-postgres psql -U micprotocol -d micprotocol < scripts/init-databases.sql
	```

	## Shared Object Storage (MinIO)

	MinIO is the default storage backend for every self-hosted integration that uploads files. `minio-createbuckets` is a one-shot service that creates:

	\| Bucket \| Owner \|
	\|---\|---\|
	\| `micprotocol-storage` \| MicProtocol API (media, headshots, clips) \|
	\| `calcom-uploads` \| Cal.com \|
	\| `documenso-uploads` \| Documenso \|
	\| `formbricks-uploads` \| Formbricks \|

	`micprotocol-storage` is set to public-read (used for guest-facing media). The others are private.

	In production, point MicProtocol at Cloudflare R2 instead of MinIO by changing `R2_ENDPOINT`, `R2_ACCESS_KEY_ID`, `R2_SECRET_ACCESS_KEY`, `R2_BUCKET`, and `STORAGE_PUBLIC_URL` — no code changes required, both speak the S3 API.

	## Shared SMTP (Mailhog in dev)

	Every self-hosted integration sends outbound mail through the same SMTP settings:

	```
	SMTP_HOST=mailhog # in compose network
	SMTP_PORT=1025
	SMTP_USER= # empty for Mailhog
	SMTP_PASSWORD= # empty for Mailhog
	EMAIL_FROM=noreply@<yourdomain>
	```

	In production, swap Mailhog for your transactional relay (Resend, SES, Postmark, or self-hosted Postal/Haraka) by setting `SMTP_HOST`, `SMTP_PORT`, `SMTP_USER`, `SMTP_PASSWORD`, and `EMAIL_FROM` to the provider's values. The integrations read these env vars directly — no code changes.

	## Shared Redis (namespaced)

	Chatwoot, Formbricks, and Cal.com all share the `micprotocol-redis` instance. Chatwoot and Cal.com namespace keys internally; Formbricks uses Redis for rate-limiting and Cube.js queue. There is currently no isolation beyond provider namespacing — for high-throughput production, split into one Redis per integration.

	## Co-located Secrets

	Every self-hosted service reads its secrets from environment variables that the API also needs. The complete list lives in [`.env.example`](../.env.example). Critical guidance:

	\| Category \| Examples \| Rotation policy \|
	\|---\|---\|---\|
	\| Postgres / Redis passwords \| `POSTGRES_PASSWORD`, `REDIS_PASSWORD` \| Rotatable; requires service restart \|
	\| Encryption keys (immutable) \| `FORMBRICKS_ENCRYPTION_KEY`, `DOCUMENSO_ENCRYPTION_KEY`, `CALCOM_ENCRYPTION_KEY`, `NEXTAUTH_SECRET` variants \| Never rotate after first start; old encrypted data becomes unrecoverable \|
	\| Webhook signatures \| `*_WEBHOOK_SECRET` \| Rotatable; update on both sides simultaneously \|
	\| Admin / API tokens \| `CALCOM_API_KEY`, `LISTMONK_API_TOKEN`, `CHATWOOT_API_KEY` \| Rotatable via the admin UI of each integration \|
	\| App secrets \| `UMAMI_APP_SECRET`, `FORMBRICKS_CRON_SECRET` \| Rotatable \|

	Use `bash scripts/generate-secrets.sh` to produce strong random values for every variable in one shot. See [Getting Started → Environment Configuration](./getting-started.md#3-environment-configuration) for details.

	## Service-Specific Configuration Notes

	### Cal.com

	- Uses its own `calcom` database (Prisma migrations need a clean schema).
	- `NEXT_PUBLIC_SINGLE_ORG_SLUG=micprotocol` and `ORGANIZATIONS_ENABLED=0` disable paid multi-org features for self-hosted single-tenant use.
	- Generate an API key in Cal.com Settings → API Keys; set `CALCOM_API_KEY` in `.env`.
	- Encrypts DB columns with `CALCOM_ENCRYPTION_KEY` (24-char hex, never rotate after init).

	### Formbricks v4

	- Five containers, must start in order: `formbricks-migrate` → `formbricks-hub-migrate` → `formbricks-cube` → `formbricks-hub` → `formbricks`. Compose `depends_on` chains handle this automatically.
	- `SKIP_STARTUP_MIGRATION=true` prevents the web container from re-migrating (the dedicated migrate job already did it).
	- `SKIP_STARTUP_MIGRATION` must remain `true`; running migrations inside the web container will race with the migrate job.
	- Analytics require Cube.js (`CUBEJS_API_SECRET` must match between web and cube containers).
	- Sign-up is disabled without an invite (`EMAIL_VERIFICATION_DISABLED=1`).

	### Documenso

	- Requires a signing certificate. Generate with the commands in [`scripts/documenso-cert/README.md`](../scripts/documenso-cert/README.md).
	- Mount the cert directory read-only at `/opt/documenso/cert`; the app looks for `cert.p12` inside.
	- `DOCUMENSO_ENCRYPTION_KEY` and `DOCUMENSO_ENCRYPTION_SECONDARY_KEY` are both required and never rotated.

	### Chatwoot

	- Rails + Sidekiq worker pair (`chatwoot-web` + `chatwoot-worker`).
	- Uses the `chatwoot` database with `pgvector` (required by Chatwoot itself).
	- Storage is local (`RAILS_STORAGE_LOCATION=/app/storage`, mounted as `chatwoot_storage` volume). For larger deployments, switch `ACTIVE_STORAGE_SERVICE` to S3/MinIO.
	- `FRONTEND_URL` must match the public URL or the live-chat widget will fail to load.
	- Generate `SECRET_KEY_BASE` with `openssl rand -hex 64`.

	### Listmonk

	- Container runs `./listmonk --install --idempotent --yes && ./listmonk --upgrade --yes && ./listmonk --config ''` on first boot — idempotent, safe on restarts.
	- `LISTMONK_db__database=listmonk` (must match the name in `init-databases.sql`).
	- API token is generated in Listmonk Admin → Settings → API; set `LISTMONK_API_TOKEN` in `.env`.
	- `LISTMONK_ADMIN_USER`/`LISTMONK_ADMIN_PASSWORD` bootstrap the super-admin on first run.

	### Umami

	- Single container, Postgres-backed.
	- `UMAMI_APP_SECRET` is required; generate with `openssl rand -base64 32`.
	- Add a website in the Umami UI and copy the website ID to `NEXT_PUBLIC_UMAMI_WEBSITE_ID`.

	### LiveKit

	- SFU for the in-app recording studio. MicProtocol owns the browser client (`apps/web`); LiveKit owns the WebRTC media routing.
	- Dev defaults (`devkey` / `devsecret`) work in Compose but must be changed for production.
	- Webhooks from LiveKit back to the API call `POST /integrations/webhooks/livekit` and trigger `RecordingSession` state transitions.
	- For internet-facing deployments, expose UDP `7882` and configure `rtc.use_external_ip: true` (or run behind a TURN server).

	### mic-realtime

	Erlang/OTP service that owns:

	- Chat fanout (per-workspace conversation messages)
	- Presence (who is online in a workspace)
	- Live domain events (published by the NestJS API via `/internal/dispatch`)

	Key env vars:

	\| Var \| Default \| Notes \|
	\|---\|---\|---\|
	\| `MIC_REALTIME_PORT` \| 4001 \| HTTP listener \|
	\| `MIC_REALTIME_HOST` \| 0.0.0.0 \| Bind address \|
	\| `MICPROTOCOL_API_URL` \| http://api:4000 \| Used by mic-realtime to call back into the API \|
	\| `MIC_REALTIME_INTERNAL_TOKEN` \| (change me) \| Bearer used by API to publish events \|
	\| `MIC_REALTIME_SHARED_SECRET` \| (change me) \| JWT signing secret for WS auth tokens \|
	\| `MIC_REALTIME_PUBLIC_URL` \| ws://localhost:4001 \| Public WS URL handed to web clients \|
	\| `MIC_REALTIME_MAX_MSG` \| 65536 \| Max message size in bytes \|
	\| `MIC_REALTIME_RATE_LIMIT` \| 120 \| Messages per minute per connection \|
	\| `MIC_REALTIME_PRESENCE_TTL` \| 90 \| Presence TTL (seconds) \|

	Both `MIC_REALTIME_INTERNAL_TOKEN` and `MIC_REALTIME_SHARED_SECRET` must be strong random strings in production.

	#### Production deployment

	`mic-realtime` ships in every deployment topology:

	\| Topology \| Where \| Public route \|
	\|---\|---\|---\|
	\| Local dev \| `docker-compose.yml` (service `mic-realtime`) \| `ws://localhost:4001` \|
	\| Coolify \| `docker-compose.coolify.yml` (service `mic-realtime`, internal `:4001`) \| `wss://realtime.<root>` via Caddy \|
	\| Render \| `render.yaml` (`micprotocol-realtime`, `rootDir: services/mic-realtime`) \| `wss://micprotocol-realtime.onrender.com` (or custom domain) \|

	The [Caddyfile](../Caddyfile) routes `realtime.<root>` to `mic-realtime:4001`
	(the specific host block wins over the `*.` wildcard). Caddy handles the
	WebSocket upgrade transparently on the same route, so one URL serves both the
	`GET /healthz` probe and `GET /ws?token=...`.

	Secrets are fail-closed: `RealtimeClient`
	(`apps/api/src/chat/realtime.client.ts`) and `mic_config.erl` both refuse to
	start in production when `MIC_REALTIME_INTERNAL_TOKEN` /
	`MIC_REALTIME_SHARED_SECRET` are missing or equal to the known defaults.
	Generate strong values (`openssl rand -hex 32`) and set
	`MIC_REALTIME_PUBLIC_URL` to the public `wss://` URL; the web/admin frontends
	receive it as `NEXT_PUBLIC_MIC_REALTIME_URL`. The Coolify stack defaults the
	API→realtime base URL to the internal `http://mic-realtime:4001`
	(`MIC_REALTIME_URL`), so API→realtime callbacks never leave the Docker
	network.

	> ⚠️ Render free instances sleep after ~15 min of inactivity and cap at
	> 750 hours/month — a sleeping WebSocket engine drops every live chat/presence
	> socket. Use `plan: starter` or higher for realtime in production.

	## Start / Stop / Update

	### Bring up the full stack

	```bash
	docker compose up -d
	```

	### Bring up only infra (for local app dev)

	```bash
	docker compose up -d postgres redis meilisearch minio minio-createbuckets
	```

	### Rolling update a single integration

	```bash
	docker compose pull formbricks
	docker compose up -d --no-deps formbricks
	```

	Formbricks v4 migrations need the migrate jobs to re-run after a version bump:

	```bash
	docker compose up formbricks-migrate
	docker compose up formbricks-hub-migrate
	docker compose up -d --no-deps formbricks formbricks-hub formbricks-cube
	```

	### Tear down

	```bash
	docker compose down # keep data
	docker compose down -v # wipe all volumes (destructive)
	```

	## Health Endpoints

	\| Service \| Endpoint \|
	\|---\|---\|
	\| API \| `GET http://localhost:4000/health` \|
	\| Cal.com \| `GET http://localhost:3002/api/health` \|
	\| Formbricks web \| `GET http://localhost:3001/api/health` \|
	\| Formbricks Hub \| `GET http://localhost:8080/health` \|
	\| Formbricks Cube \| `GET http://localhost:4000/formbricks/readyz` (via cube container) \|
	\| Chatwoot \| `GET http://localhost:3005/health` \|
	\| Listmonk \| `GET http://localhost:9000/health` \|
	\| Documenso \| `GET http://localhost:3004/api/health` \|
	\| Umami \| `GET http://localhost:3003/api/health` \|
	\| MinIO \| `GET http://localhost:9001/minio/health/live` \|
	\| Meilisearch \| `GET http://localhost:7700/health` \|
	\| mic-realtime \| `GET http://localhost:4001/healthz` \|
	\| LiveKit \| `GET http://localhost:7880/health` (HTTP only; UDP port 7882 for WebRTC) \|
	\| Mailhog \| UI at `:8025` always returns 200 \|

	The API exposes a consolidated integration health check at:

	```bash
	curl -s http://localhost:4000/integrations/health \| jq
	```

	This pings each self-hosted service's health endpoint in parallel and reports reachability.

	### Integration Management Dashboard

	The Admin panel (`apps/admin`) provides a full management interface for all self-hosted integrations:

	\| Feature \| Description \|
	\|---\|---\|
	\| Health Monitoring \| Real-time health status, latency, and version for Chatwoot, Cal.com, Formbricks, Listmonk, Documenso, Umami, and LiveKit \|
	\| Webhook Secret Management \| View and rotate shared webhook secrets directly from the admin panel \|
	\| Admin Panel Links \| Direct "Open Admin" buttons that open each integration's native admin UI in a new tab \|
	\| Credential Management \| Connect, update, or disconnect API keys for self-hosted integrations \|
	\| Webhook Delivery Log \| Recent webhook deliveries with status, latency, and error details \|
	\| Provider Metrics \| Per-provider request counts, average latency, and error rates \|

	Access these features from the Monitoring and Integrations tabs in the admin navigation sidebar.

	## Resource Limits

	Default `deploy.resources.limits` in `docker-compose.yml`:

	\| Service \| CPUs \| Memory \|
	\|---\|---\|---\|
	\| API \| 2 \| 2G \|
	\| Worker \| 2 \| 2G \|
	\| mic-realtime \| 1.5 \| 1G \|

	Other services have no explicit limits — tune for your hardware. CLIP scoring (Python ML subprocess) runs inside the worker and is by far the most CPU- and memory-hungry: see [CLIP Scoring Engine](./clip-scoring.md) for scale guidance.

	## Alternatives if You Skip Self-Hosting

	If you don't want to run a particular self-hosted service, simply remove its block from `docker-compose.yml` and omit its env vars from `.env`. The MicProtocol API degrades gracefully — each integration's API client is called only when its corresponding URL env var is set. Missing integrations return `false` from the health check and the API logs a warning rather than 500-ing.

	For the externally-hosted (non-self-hosted) integrations (Clerk, Resend, Cloudflare R2, NVIDIA NIM, DeepSeek, Moonshot, Stripe, Paystack, M-Pesa, Africa's Talking, Twilio, Google/Outlook calendars), see [Third-Party & Self-Hosted Integrations](./third-party-integrations.md).