Spaces:

Benaah
/

MicProtocol

Running

App Files Files Community

MicProtocol / docs /deployment.md

Benaah

Upload folder using huggingface_hub

b17baae verified about 20 hours ago

preview code

Raw History Blame Contribute Delete

7.67 kB

# MicProtocol – Deployment & Scaling Strategy

## Deployment Architecture

```mermaid
graph LR
    subgraph "Vercel Edge"
        WEB[Next.js Web App]
        PORTAL[Guest Portal]
    end
    
    subgraph "AWS ECS/Fargate"
        API[NestJS API]
        WORKER[Background Workers]
        CLIP_SCORER[CLIP Scorer<br/>Python + ML]
    end
    
    subgraph "Managed Services"
        DB[RDS PostgreSQL]
        CACHE[ElastiCache Redis]
        SEARCH[Meilisearch Cloud]
        OBJ[Cloudflare R2]
    end
    
    subgraph "CI/CD"
        GHA[GitHub Actions]
        TURBO[Turborepo Cache]
    end
    
    GHA -->|Docker Build| API
    GHA -->|Deploy| WEB
    WEB -->|tRPC| API
    API --> DB
    WORKER --> DB
    WORKER --> CACHE
    WORKER -->|subprocess| CLIP_SCORER
    CLIP_SCORER --> DB
```

## Production Stacks (Coolify / Render)

Beyond the ECS layout above, the repo ships two ready-to-deploy production
blueprints that include the Erlang/OTP real-time engine (`mic-realtime`).

### Coolify (`docker-compose.coolify.yml`)

Full self-hosted stack: Postgres, Redis, API, worker, **mic-realtime**, web,
admin, and the self-hosted integration fleet. Caddy terminates TLS (wildcard
cert via DNS-01) and routes:

| Host | Upstream |
|---|---|
| `<root>` / `*.` (workspace subdomains) | web :3000 |
| `api.<root>` | api :4000 |
| `admin.<root>` | admin :3000 |
| `realtime.<root>` | mic-realtime :4001 (WebSocket) |

### Render (`render.yaml`)

Backend-only blueprint (web + admin deploy on Vercel):

- `micprotocol-api` — NestJS API
- `micprotocol-redis` — Redis add-on
- `micprotocol-realtime` — Erlang/OTP engine (`env: docker`,
  `rootDir: services/mic-realtime`, health at `/healthz`)

### Real-time engine & required secrets

The `mic-realtime` service is a first-class citizen of both stacks and ships
**fail-closed**: both it and the API (`RealtimeClient` in
`apps/api/src/chat/realtime.client.ts`) refuse to boot in production when the
secrets are missing or equal to the known defaults.

| Env var | Required | Notes |
|---|---|---|
| `MIC_REALTIME_INTERNAL_TOKEN` | ✅ | Bearer used by the API to publish events via `/internal/dispatch` |
| `MIC_REALTIME_SHARED_SECRET` | ✅ | HMAC secret for WebSocket auth tokens |
| `MIC_REALTIME_PUBLIC_URL` | ✅ | Browser-facing `wss://realtime.<root>` URL; forwarded to web/admin as `NEXT_PUBLIC_MIC_REALTIME_URL` |
| `MIC_REALTIME_URL` | API only | API→realtime base URL (Coolify default `http://mic-realtime:4001`) |

Generate both secrets with `openssl rand -hex 32` and set
`MIC_REALTIME_PUBLIC_URL` to a `wss://` URL **before first boot** — the
services will not start with missing or default values.

## Environment Strategy

| Environment | Web Frontend | API Backend | Database |
|-------------|-------------|-------------|----------|
| **Development** | `localhost:3000` | `localhost:4000` | Local PostgreSQL |
| **Preview** | Vercel Preview | ECS Staging | RDS Staging |
| **Production** | Vercel Edge | ECS Production | RDS Multi-AZ |

## Scaling Triggers

### < 1,000 Podcasts (Startup Phase)

- Single ECS task for API + Workers
- Shared Redis instance (ElastiCache `t3.micro`)
- Cloudflare R2 for all media storage
- Single PostgreSQL instance (RDS `db.t3.medium`)

### > 1,000 Podcasts (Growth Phase)

- Split Workers into separate ECS service
- Dedicated Redis for BullMQ vs. caching
- Read replicas for PostgreSQL
- CDN for static assets
- Migrate complex workflows to **Temporal.io**

### > 10,000 Podcasts (Scale Phase)

- Extract Outreach Domain into standalone microservice
- Outreach gets own MongoDB cluster (high write volume on email events)
- Event bus (AWS SNS/SQS) for cross-service communication
- Meilisearch cluster for search scaling
- Multi-region deployment for global guest portal latency

## CI/CD Pipeline

```mermaid
flowchart TD
    PUSH[Git Push] --> LINT[Lint & Type Check]
    LINT --> TEST[Unit Tests]
    TEST --> BUILD[Turbo Build]
    
    BUILD --> |Main Branch| DEPLOY_STAGING[Deploy to Staging]
    BUILD --> |Release Tag| DEPLOY_PROD[Deploy to Production]
    BUILD --> |PR| PREVIEW[Vercel Preview]
    
    DEPLOY_STAGING --> INTEGRATION[Integration Tests]
    INTEGRATION --> |Pass| PROMOTE[Promote to Production]
    
    DEPLOY_PROD --> MIGRATE[Run DB Migrations]
    MIGRATE --> HEALTH[Health Checks]
    HEALTH --> |Healthy| COMPLETE([Deployment Complete])
    HEALTH --> |Unhealthy| ROLLBACK[Automatic Rollback]
```

## Infrastructure Configuration

### Docker (API)

```dockerfile
FROM node:20-alpine AS base
RUN corepack enable && corepack prepare pnpm@9.15.0 --activate
FROM base AS deps
WORKDIR /app
COPY package.json pnpm-lock.yaml pnpm-workspace.yaml ./
COPY apps/api/package.json apps/api/
COPY packages/shared-types/package.json packages/shared-types/
RUN pnpm install --frozen-lockfile
FROM base AS build
WORKDIR /app
COPY --from=deps /app/node_modules ./node_modules
COPY . .
RUN pnpm --filter @micprotocol/api prisma generate
RUN pnpm --filter @micprotocol/api build
FROM base AS runtime
WORKDIR /app
COPY --from=build /app/apps/api/dist ./dist
COPY --from=build /app/node_modules ./node_modules
COPY --from=build /app/apps/api/prisma ./prisma
RUN apk add --no-cache ffmpeg
EXPOSE 4000
CMD ["node", "dist/main.js"]
```

### Docker (CLIP Scorer)

Canonical source of truth is `services/clip-scorer/Dockerfile`. It builds a one-shot CLI image:

```dockerfile
FROM python:3.11-slim AS base
RUN apt-get update && apt-get install -y --no-install-recommends ffmpeg && rm -rf /var/lib/apt/lists/*
COPY --from=ghcr.io/astral-sh/uv:latest /uv /usr/local/bin/uv
WORKDIR /app
RUN uv venv /opt/venv \
    && VIRTUAL_ENV=/opt/venv uv pip install --index-url https://download.pytorch.org/whl/cpu "torch>=2.0.0" \
    && VIRTUAL_ENV=/opt/venv uv pip install "open-clip-torch>=2.20.0" "transformers>=4.30.0" "Pillow>=10.0.0" "soundfile>=0.12.0" "pyloudnorm>=0.1.0" "numpy>=1.24.0"
ENV PATH="/opt/venv/bin:$PATH"
RUN python3 -c "import open_clip; open_clip.create_model_and_transforms('ViT-B-32', pretrained='laion2b_s34b_b79k')"
COPY scoring/ scoring/
COPY pipeline.py .
ENTRYPOINT ["python3", "pipeline.py"]
```

In production the same Python stack is baked into the **worker** image (`apps/api/Dockerfile`), where the BullMQ clip-scoring processor executes the pipeline via `execFile`.

### Docker Compose Configuration

The worker service requires `ipc: host` for `/dev/shm` access (needed by ffmpeg shared memory). CLIP scoring runs inside the worker as a subprocess of the processor — there is no separate clip-scorer container or shared volume:

```yaml
worker:
  container_name: micprotocol-worker
  command: node /app/apps/api/dist/main.js
  ipc: host
  deploy:
    resources:
      limits:
        cpus: "2"
        memory: 2G
```

### Scaling Considerations for CLIP Scorer

| Metric | Value | Notes |
|--------|-------|-------|
| CPU per clip | ~0.5 cores | ~30–120s for CLIP inference (CPU) |
| Memory per clip | ~1.5GB | CLIP model + frame buffers |
| Concurrent clips | 2 per worker (set in processor) | Bump cautiously — Python ML is memory-heavy |
| Horizontal scaling | Add worker containers | Each container handles 2 concurrent clips |
| GPU acceleration | Optional | CLIP runs on CPU by default; GPU cuts time by 5-10x |

## Security

- **GDPR/CCPA**: Right-to-erasure cascading through all related records
- **Email Compliance**: CAN-SPAM footer injection, automatic unsubscribe
- **Data Isolation**: Row-level scoping through workspace → podcast hierarchy
- **Upload Security**: Presigned URLs (15-min expiry), MIME verification, virus scanning
- **Auth**: Clerk JWT verification with workspace-scoped claims

	# MicProtocol – Deployment & Scaling Strategy

	## Deployment Architecture

	```mermaid
	graph LR
	subgraph "Vercel Edge"
	WEB[Next.js Web App]
	PORTAL[Guest Portal]
	end

	subgraph "AWS ECS/Fargate"
	API[NestJS API]
	WORKER[Background Workers]
	CLIP_SCORER[CLIP Scorer<br/>Python + ML]
	end

	subgraph "Managed Services"
	DB[RDS PostgreSQL]
	CACHE[ElastiCache Redis]
	SEARCH[Meilisearch Cloud]
	OBJ[Cloudflare R2]
	end

	subgraph "CI/CD"
	GHA[GitHub Actions]
	TURBO[Turborepo Cache]
	end

	GHA -->\|Docker Build\| API
	GHA -->\|Deploy\| WEB
	WEB -->\|tRPC\| API
	API --> DB
	WORKER --> DB
	WORKER --> CACHE
	WORKER -->\|subprocess\| CLIP_SCORER
	CLIP_SCORER --> DB
	```

	## Production Stacks (Coolify / Render)

	Beyond the ECS layout above, the repo ships two ready-to-deploy production
	blueprints that include the Erlang/OTP real-time engine (`mic-realtime`).

	### Coolify (`docker-compose.coolify.yml`)

	Full self-hosted stack: Postgres, Redis, API, worker, mic-realtime, web,
	admin, and the self-hosted integration fleet. Caddy terminates TLS (wildcard
	cert via DNS-01) and routes:

	\| Host \| Upstream \|
	\|---\|---\|
	\| `<root>` / `*.` (workspace subdomains) \| web :3000 \|
	\| `api.<root>` \| api :4000 \|
	\| `admin.<root>` \| admin :3000 \|
	\| `realtime.<root>` \| mic-realtime :4001 (WebSocket) \|

	### Render (`render.yaml`)

	Backend-only blueprint (web + admin deploy on Vercel):

	- `micprotocol-api` — NestJS API
	- `micprotocol-redis` — Redis add-on
	- `micprotocol-realtime` — Erlang/OTP engine (`env: docker`,
	`rootDir: services/mic-realtime`, health at `/healthz`)

	### Real-time engine & required secrets

	The `mic-realtime` service is a first-class citizen of both stacks and ships
	fail-closed: both it and the API (`RealtimeClient` in
	`apps/api/src/chat/realtime.client.ts`) refuse to boot in production when the
	secrets are missing or equal to the known defaults.

	\| Env var \| Required \| Notes \|
	\|---\|---\|---\|
	\| `MIC_REALTIME_INTERNAL_TOKEN` \| ✅ \| Bearer used by the API to publish events via `/internal/dispatch` \|
	\| `MIC_REALTIME_SHARED_SECRET` \| ✅ \| HMAC secret for WebSocket auth tokens \|
	\| `MIC_REALTIME_PUBLIC_URL` \| ✅ \| Browser-facing `wss://realtime.<root>` URL; forwarded to web/admin as `NEXT_PUBLIC_MIC_REALTIME_URL` \|
	\| `MIC_REALTIME_URL` \| API only \| API→realtime base URL (Coolify default `http://mic-realtime:4001`) \|

	Generate both secrets with `openssl rand -hex 32` and set
	`MIC_REALTIME_PUBLIC_URL` to a `wss://` URL before first boot — the
	services will not start with missing or default values.

	## Environment Strategy

	\| Environment \| Web Frontend \| API Backend \| Database \|
	\|-------------\|-------------\|-------------\|----------\|
	\| Development \| `localhost:3000` \| `localhost:4000` \| Local PostgreSQL \|
	\| Preview \| Vercel Preview \| ECS Staging \| RDS Staging \|
	\| Production \| Vercel Edge \| ECS Production \| RDS Multi-AZ \|

	## Scaling Triggers

	### < 1,000 Podcasts (Startup Phase)

	- Single ECS task for API + Workers
	- Shared Redis instance (ElastiCache `t3.micro`)
	- Cloudflare R2 for all media storage
	- Single PostgreSQL instance (RDS `db.t3.medium`)

	### > 1,000 Podcasts (Growth Phase)

	- Split Workers into separate ECS service
	- Dedicated Redis for BullMQ vs. caching
	- Read replicas for PostgreSQL
	- CDN for static assets
	- Migrate complex workflows to Temporal.io

	### > 10,000 Podcasts (Scale Phase)

	- Extract Outreach Domain into standalone microservice
	- Outreach gets own MongoDB cluster (high write volume on email events)
	- Event bus (AWS SNS/SQS) for cross-service communication
	- Meilisearch cluster for search scaling
	- Multi-region deployment for global guest portal latency

	## CI/CD Pipeline

	```mermaid
	flowchart TD
	PUSH[Git Push] --> LINT[Lint & Type Check]
	LINT --> TEST[Unit Tests]
	TEST --> BUILD[Turbo Build]

	BUILD --> \|Main Branch\| DEPLOY_STAGING[Deploy to Staging]
	BUILD --> \|Release Tag\| DEPLOY_PROD[Deploy to Production]
	BUILD --> \|PR\| PREVIEW[Vercel Preview]

	DEPLOY_STAGING --> INTEGRATION[Integration Tests]
	INTEGRATION --> \|Pass\| PROMOTE[Promote to Production]

	DEPLOY_PROD --> MIGRATE[Run DB Migrations]
	MIGRATE --> HEALTH[Health Checks]
	HEALTH --> \|Healthy\| COMPLETE([Deployment Complete])
	HEALTH --> \|Unhealthy\| ROLLBACK[Automatic Rollback]
	```

	## Infrastructure Configuration

	### Docker (API)

	```dockerfile
	FROM node:20-alpine AS base
	RUN corepack enable && corepack prepare pnpm@9.15.0 --activate

	FROM base AS deps
	WORKDIR /app
	COPY package.json pnpm-lock.yaml pnpm-workspace.yaml ./
	COPY apps/api/package.json apps/api/
	COPY packages/shared-types/package.json packages/shared-types/
	RUN pnpm install --frozen-lockfile

	FROM base AS build
	WORKDIR /app
	COPY --from=deps /app/node_modules ./node_modules
	COPY . .
	RUN pnpm --filter @micprotocol/api prisma generate
	RUN pnpm --filter @micprotocol/api build

	FROM base AS runtime
	WORKDIR /app
	COPY --from=build /app/apps/api/dist ./dist
	COPY --from=build /app/node_modules ./node_modules
	COPY --from=build /app/apps/api/prisma ./prisma
	RUN apk add --no-cache ffmpeg
	EXPOSE 4000
	CMD ["node", "dist/main.js"]
	```

	### Docker (CLIP Scorer)

	Canonical source of truth is `services/clip-scorer/Dockerfile`. It builds a one-shot CLI image:

	```dockerfile
	FROM python:3.11-slim AS base
	RUN apt-get update && apt-get install -y --no-install-recommends ffmpeg && rm -rf /var/lib/apt/lists/*
	COPY --from=ghcr.io/astral-sh/uv:latest /uv /usr/local/bin/uv
	WORKDIR /app
	RUN uv venv /opt/venv \
	&& VIRTUAL_ENV=/opt/venv uv pip install --index-url https://download.pytorch.org/whl/cpu "torch>=2.0.0" \
	&& VIRTUAL_ENV=/opt/venv uv pip install "open-clip-torch>=2.20.0" "transformers>=4.30.0" "Pillow>=10.0.0" "soundfile>=0.12.0" "pyloudnorm>=0.1.0" "numpy>=1.24.0"
	ENV PATH="/opt/venv/bin:$PATH"
	RUN python3 -c "import open_clip; open_clip.create_model_and_transforms('ViT-B-32', pretrained='laion2b_s34b_b79k')"
	COPY scoring/ scoring/
	COPY pipeline.py .
	ENTRYPOINT ["python3", "pipeline.py"]
	```

	In production the same Python stack is baked into the worker image (`apps/api/Dockerfile`), where the BullMQ clip-scoring processor executes the pipeline via `execFile`.

	### Docker Compose Configuration

	The worker service requires `ipc: host` for `/dev/shm` access (needed by ffmpeg shared memory). CLIP scoring runs inside the worker as a subprocess of the processor — there is no separate clip-scorer container or shared volume:

	```yaml
	worker:
	container_name: micprotocol-worker
	command: node /app/apps/api/dist/main.js
	ipc: host
	deploy:
	resources:
	limits:
	cpus: "2"
	memory: 2G
	```

	### Scaling Considerations for CLIP Scorer

	\| Metric \| Value \| Notes \|
	\|--------\|-------\|-------\|
	\| CPU per clip \| ~0.5 cores \| ~30–120s for CLIP inference (CPU) \|
	\| Memory per clip \| ~1.5GB \| CLIP model + frame buffers \|
	\| Concurrent clips \| 2 per worker (set in processor) \| Bump cautiously — Python ML is memory-heavy \|
	\| Horizontal scaling \| Add worker containers \| Each container handles 2 concurrent clips \|
	\| GPU acceleration \| Optional \| CLIP runs on CPU by default; GPU cuts time by 5-10x \|

	## Security

	- GDPR/CCPA: Right-to-erasure cascading through all related records
	- Email Compliance: CAN-SPAM footer injection, automatic unsubscribe
	- Data Isolation: Row-level scoping through workspace → podcast hierarchy
	- Upload Security: Presigned URLs (15-min expiry), MIME verification, virus scanning
	- Auth: Clerk JWT verification with workspace-scoped claims