docs/getting-started.md

Getting Started

This guide takes you from a clean machine to a running MicProtocol stack with all self-hosted integrations. Read end-to-end on first install; use the checklists for re-installs.

1. Prerequisites

Tool	Version	Purpose
Node.js	>= 20.0.0	API, Web, Admin, Mobile tooling
pnpm	>= 9.15.0 (locked to pnpm@10.30.3)	Workspace package manager
Docker + Docker Compose	latest stable	Infra (Postgres, Redis, Meilisearch, integrations)
Python	>= 3.11	CLIP scoring engine
uv	latest	Python package manager for services/clip-scorer
FFmpeg	latest	Audio/video frame extraction (also installed in API image)
PostgreSQL client	15+	Optional local DB without Docker
Redis CLI	7+	Optional local Redis without Docker
podman + podman-compose	latest	Required only for pnpm agents:run

Install uv:

curl -LsSf https://astral.sh/uv/install.sh | sh

2. Repository

git clone <repo-url> micprotocol
cd micprotocol

The repo is a Turborepo monorepo. Top-level layout:

micprotocol/
├── apps/
│   ├── api/        NestJS backend (tRPC + REST + Socket.io)
│   ├── web/        Next.js 14 host dashboard + guest portal
│   ├── admin/      Next.js platform-admin app
│   └── mobile/     Expo React Native app
├── packages/
│   ├── shared-types/   Zod schemas + domain enums
│   ├── ui/             Cross-platform UI primitives
│   ├── tsconfig/       Shared TS configs
│   └── eslint-config/  Shared ESLint config
├── services/
│   ├── clip-scorer/  Python ML engine (OpenCLIP + pyloudnorm + bart-large-mnli)
│   └── mic-realtime/ Erlang/OTP chat fanout + presence
├── scripts/
│   ├── init-databases.sql          Per-integration Postgres databases
│   ├── generate-secrets.sh         Generates all self-hosted secrets
│   ├── documenso-cert/README.md    How to generate the Documenso signing cert
│   └── synthetic-agents/           Synthetic agent test framework
├── docs/             This documentation set
├── docker-compose.yml           Production Docker Compose (full stack)
├── docker-compose.coolify.yml   Coolify override
├── turbo.json        Turborepo pipeline
└── pnpm-workspace.yaml

3. Environment Configuration

cp .env.example .env
cp .env.example.coolify .env.coolify   # only if deploying via Coolify

Generate strong secrets for the self-hosted integrations:

bash scripts/generate-secrets.sh > .env.production.formbricks
# Manually merge generated values into your .env

scripts/generate-secrets.sh produces:

• Core infrastructure (Postgres, Redis, Meilisearch, MinIO passwords)
• Encryption keys (never rotate after initial setup)
• Chatwoot, Cal.com, Formbricks, Documenso, Listmonk secrets
• LiveKit API key/secret/webhook secret
• Umami app secret

For each *_ENCRYPTION_KEY and *_NEXTAUTH_SECRET, treat them as immutable after first start — rotating will invalidate sessions and unrecoverable encrypted data.

Generate the Documenso signing certificate (one-time)

openssl req -x509 -newkey rsa:4096 -keyout key.pem -out cert.pem \
  -days 365 -nodes -subj "/CN=MicProtocol Documenso"
openssl pkcs12 -export -out scripts/documenso-cert/cert.p12 \
  -inkey key.pem -in cert.pem \
  -passout pass:"$(openssl rand -hex 32)"
rm key.pem cert.pem

Set DOCUMENSO_SIGNING_PASSPHRASE in .env to the passphrase used above. Production should use a CA-issued cert, never the self-signed one.

4. First Boot — Full Stack (Docker)

# 4.1. Start core infra (Postgres, Redis, Meilisearch)
docker compose up -d postgres redis meilisearch

# 4.2. Generate Prisma client and push schema
pnpm db:generate
pnpm db:push

# 4.3. Start the entire stack (API + worker + CLIP scorer + web + admin
#       + all self-hosted integrations + realtime)
docker compose up -d

# 4.4. Verify
docker compose ps
curl -s http://localhost:4000/health

Default ports:

Service	Port	URL
Web (host dashboard + guest portal)	3000	http://localhost:3000
Admin	3006	http://localhost:3006
API	4000	http://localhost:4000
Swagger (API)	4000	http://localhost:4000/api/docs
Meilisearch	7700	http://localhost:7700
mic-realtime (WS)	4001	ws://localhost:4001
Cal.com	3002	http://localhost:3002
Formbricks	3001	http://localhost:3001
Umami	3003	http://localhost:3003
Documenso	3004	http://localhost:3004
Chatwoot	3005	http://localhost:3005
Listmonk	9000	http://localhost:9000
MinIO API	9000 (internal)	http://localhost:9001
MinIO Console	9090	http://localhost:9090
LiveKit	7880 (HTTP), 7881 (TCP), 7882 (UDP)	http://localhost:7880
Mailhog UI	8025	http://localhost:8025
Mailhog SMTP	1025	n/a

5. First Boot — Local Dev (No Docker for App Code)

Useful when iterating on the API or web app and you have the infra running in Docker.

# 5.1. Install all JS deps
pnpm install

# 5.2. Install Python deps for CLIP scorer
cd services/clip-scorer
uv sync
cd ../..

# 5.3. Start only infra in Docker
docker compose up -d postgres redis meilisearch

# 5.4. Generate Prisma client and push schema
pnpm db:generate
pnpm db:push

# 5.5. Start all apps in dev mode
pnpm dev

The Turborepo pipeline runs api, web, admin (if configured), and any app with a dev script. Hot reload is enabled for both the API (NestJS) and Web (Next.js).

6. Verifying the Stack

API

# Liveness
curl -s http://localhost:4000/health

# Detailed
curl -s http://localhost:4000/health/detailed

# Readiness (K8s style)
curl -s http://localhost:4000/health/ready

Database

pnpm db:studio   # opens Prisma Studio on http://localhost:5555

Self-hosted integrations

After first boot, complete these one-time setup steps inside each UI:

Service	Action
Cal.com	Create org micprotocol, generate an API key in Settings → API Keys; set CALCOM_API_KEY in .env and restart API
Formbricks	Create Environment, copy ID to NEXT_PUBLIC_FORMBRICKS_ENVIRONMENT_ID; generate API key in Settings → API Keys
Listmonk	Log in with LISTMONK_ADMIN_USER/LISTMONK_ADMIN_PASSWORD, generate API token in Settings → API Tokens
Documenso	Upload signing cert (already generated in step 3) and configure SMTP
Chatwoot	Create user, create inbox (Website), copy inbox ID and website token to .env
Umami	Add website, copy website ID to NEXT_PUBLIC_UMAMI_WEBSITE_ID
MinIO	Log in with MINIO_ROOT_USER/MINIO_ROOT_PASSWORD; buckets are auto-created by minio-createbuckets

Webhook smoke tests

Each self-hosted integration has a webhook the API consumes. Use the Integration Health controller to confirm reachability:

curl -s http://localhost:4000/integrations/health | jq

See Webhooks for the full endpoint catalog and signature headers.

7. Running Tests

API unit tests (Jest)

pnpm --filter @micprotocol/api test

API E2E tests

pnpm --filter @micprotocol/api test:e2e

Synthetic agents (full-stack scenario sweep)

pnpm agents:run   # full pipeline (pod up + seed + API + scenarios)
pnpm agents:orchestrator   # orchestrator only (API must be running)
pnpm db:seed:agents        # seed only

Requires podman + podman-compose. Outputs:

• scripts/synthetic-agents/test-report.txt
• scripts/synthetic-agents/weaknesses.json

8. Common Day-to-Day Commands

Task	Command
Start everything	pnpm dev (JS apps) or docker compose up -d (full stack)
Rebuild after schema change	pnpm db:generate && pnpm db:push
Create a real migration	pnpm --filter @micprotocol/api prisma:migrate
Open Prisma Studio	pnpm db:studio
Run lint across the monorepo	pnpm lint
Run type check	pnpm type-check
Build for production	pnpm build
Tail logs	docker compose logs -f api worker clip-scorer web
Restart a single service	docker compose restart <name>
Stop everything	docker compose down
Wipe volumes (destructive)	docker compose down -v

9. Troubleshooting First Run

Postgres won't start

docker compose logs postgres — most often it's a port conflict (5432 already in use). Either stop the local Postgres or change the host port mapping in docker-compose.yml.

pnpm db:push complains about missing tables for chatwoot, calcom, etc.

You must run the init script that creates per-integration databases. The init-databases.sql is mounted into the Postgres container on first boot only — if the volume already exists, run manually:

docker exec -i micprotocol-postgres psql -U micprotocol -d micprotocol < scripts/init-databases.sql

API returns 500 on first hit

Check that CLERK_SECRET_KEY and CLERK_PUBLISHABLE_KEY are set; without them the auth guard rejects all requests. For local-only development without Clerk, see Security & Compliance for the dev-mode x-user-id header bypass.

CLIP scorer stuck in "Scoring…"

See CLIP Scoring Engine → Troubleshooting.

Webhook 401s

Every webhook signature is verified with crypto.timingSafeEqual(). See Security & Compliance and Webhooks for the exact header names per provider.

Port already in use

The compose file uses fixed ports for every integration. Stop the conflicting process or remap the host port in docker-compose.yml.

10. Next Steps

• Read Architecture for the bounded-context breakdown.
• Read Self-Hosted Stack before changing any container config.
• Read Webhooks before wiring a new integration.
• Read Operations Runbook before going to production.