Spaces:
Runtime error
Runtime error
Merge pull request #90 from dewitt4/docs
Browse files- README.md +121 -0
- docs/README.md +555 -1
README.md
CHANGED
|
@@ -301,6 +301,127 @@ docker run -p 8000:8000 \
|
|
| 301 |
|
| 302 |
See [docker/README.md](docker/README.md) for complete Docker documentation.
|
| 303 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 304 |
## ⚙️ Configuration
|
| 305 |
|
| 306 |
### Environment Variables
|
|
|
|
| 301 |
|
| 302 |
See [docker/README.md](docker/README.md) for complete Docker documentation.
|
| 303 |
|
| 304 |
+
## ☁️ Cloud Deployment
|
| 305 |
+
|
| 306 |
+
LLMGuardian can be deployed on all major cloud platforms. Below are quick start guides for each provider. For detailed step-by-step instructions, see [PROJECT.md - Cloud Deployment Guides](PROJECT.md#cloud-deployment-guides).
|
| 307 |
+
|
| 308 |
+
### AWS Deployment
|
| 309 |
+
|
| 310 |
+
**Option 1: ECS with Fargate (Recommended)**
|
| 311 |
+
```bash
|
| 312 |
+
# Push to ECR and deploy
|
| 313 |
+
aws ecr get-login-password --region us-east-1 | docker login --username AWS --password-stdin YOUR_ACCOUNT_ID.dkr.ecr.us-east-1.amazonaws.com
|
| 314 |
+
aws ecr create-repository --repository-name llmguardian
|
| 315 |
+
docker tag llmguardian:latest YOUR_ACCOUNT_ID.dkr.ecr.us-east-1.amazonaws.com/llmguardian:latest
|
| 316 |
+
docker push YOUR_ACCOUNT_ID.dkr.ecr.us-east-1.amazonaws.com/llmguardian:latest
|
| 317 |
+
```
|
| 318 |
+
|
| 319 |
+
**Other AWS Options:**
|
| 320 |
+
- AWS Lambda with Docker containers
|
| 321 |
+
- Elastic Beanstalk for PaaS deployment
|
| 322 |
+
- EKS for Kubernetes orchestration
|
| 323 |
+
|
| 324 |
+
### Google Cloud Platform
|
| 325 |
+
|
| 326 |
+
**Cloud Run (Recommended)**
|
| 327 |
+
```bash
|
| 328 |
+
# Build and deploy to Cloud Run
|
| 329 |
+
gcloud auth configure-docker
|
| 330 |
+
docker tag llmguardian:latest gcr.io/YOUR_PROJECT_ID/llmguardian:latest
|
| 331 |
+
docker push gcr.io/YOUR_PROJECT_ID/llmguardian:latest
|
| 332 |
+
|
| 333 |
+
gcloud run deploy llmguardian \
|
| 334 |
+
--image gcr.io/YOUR_PROJECT_ID/llmguardian:latest \
|
| 335 |
+
--platform managed \
|
| 336 |
+
--region us-central1 \
|
| 337 |
+
--allow-unauthenticated \
|
| 338 |
+
--memory 2Gi \
|
| 339 |
+
--port 8000
|
| 340 |
+
```
|
| 341 |
+
|
| 342 |
+
**Other GCP Options:**
|
| 343 |
+
- Google Kubernetes Engine (GKE)
|
| 344 |
+
- App Engine for PaaS deployment
|
| 345 |
+
|
| 346 |
+
### Microsoft Azure
|
| 347 |
+
|
| 348 |
+
**Azure Container Instances**
|
| 349 |
+
```bash
|
| 350 |
+
# Create resource group and deploy
|
| 351 |
+
az group create --name llmguardian-rg --location eastus
|
| 352 |
+
az acr create --resource-group llmguardian-rg --name llmguardianacr --sku Basic
|
| 353 |
+
az acr login --name llmguardianacr
|
| 354 |
+
|
| 355 |
+
docker tag llmguardian:latest llmguardianacr.azurecr.io/llmguardian:latest
|
| 356 |
+
docker push llmguardianacr.azurecr.io/llmguardian:latest
|
| 357 |
+
|
| 358 |
+
az container create \
|
| 359 |
+
--resource-group llmguardian-rg \
|
| 360 |
+
--name llmguardian-container \
|
| 361 |
+
--image llmguardianacr.azurecr.io/llmguardian:latest \
|
| 362 |
+
--cpu 2 --memory 4 --ports 8000
|
| 363 |
+
```
|
| 364 |
+
|
| 365 |
+
**Other Azure Options:**
|
| 366 |
+
- Azure App Service (Web App for Containers)
|
| 367 |
+
- Azure Kubernetes Service (AKS)
|
| 368 |
+
- Azure Functions
|
| 369 |
+
|
| 370 |
+
### Vercel
|
| 371 |
+
|
| 372 |
+
**Serverless Deployment**
|
| 373 |
+
```bash
|
| 374 |
+
# Install Vercel CLI and deploy
|
| 375 |
+
npm i -g vercel
|
| 376 |
+
vercel login
|
| 377 |
+
vercel --prod
|
| 378 |
+
```
|
| 379 |
+
|
| 380 |
+
Create `vercel.json`:
|
| 381 |
+
```json
|
| 382 |
+
{
|
| 383 |
+
"version": 2,
|
| 384 |
+
"builds": [{"src": "src/llmguardian/api/app.py", "use": "@vercel/python"}],
|
| 385 |
+
"routes": [{"src": "/(.*)", "dest": "src/llmguardian/api/app.py"}]
|
| 386 |
+
}
|
| 387 |
+
```
|
| 388 |
+
|
| 389 |
+
### DigitalOcean
|
| 390 |
+
|
| 391 |
+
**App Platform (Easiest)**
|
| 392 |
+
```bash
|
| 393 |
+
# Using doctl CLI
|
| 394 |
+
doctl auth init
|
| 395 |
+
doctl apps create --spec .do/app.yaml
|
| 396 |
+
```
|
| 397 |
+
|
| 398 |
+
**Other DigitalOcean Options:**
|
| 399 |
+
- DigitalOcean Kubernetes (DOKS)
|
| 400 |
+
- Droplets with Docker
|
| 401 |
+
|
| 402 |
+
### Platform Comparison
|
| 403 |
+
|
| 404 |
+
| Platform | Best For | Ease of Setup | Estimated Cost |
|
| 405 |
+
|----------|----------|---------------|----------------|
|
| 406 |
+
| **GCP Cloud Run** | Startups, Auto-scaling | ⭐⭐⭐⭐⭐ Easy | $30-150/mo |
|
| 407 |
+
| **AWS ECS** | Enterprise, Flexibility | ⭐⭐⭐ Medium | $50-200/mo |
|
| 408 |
+
| **Azure ACI** | Microsoft Ecosystem | ⭐⭐⭐⭐ Easy | $50-200/mo |
|
| 409 |
+
| **Vercel** | API Routes, Serverless | ⭐⭐⭐⭐⭐ Very Easy | $20-100/mo |
|
| 410 |
+
| **DigitalOcean** | Simple, Predictable | ⭐⭐⭐⭐ Easy | $24-120/mo |
|
| 411 |
+
|
| 412 |
+
### Prerequisites for Cloud Deployment
|
| 413 |
+
|
| 414 |
+
Before deploying to any cloud:
|
| 415 |
+
|
| 416 |
+
1. **Prepare Environment Variables**: Copy `.env.example` to `.env` and configure
|
| 417 |
+
2. **Build Docker Image**: `docker build -t llmguardian:latest -f docker/dockerfile .`
|
| 418 |
+
3. **Set Up Cloud CLI**: Install and authenticate with your chosen provider
|
| 419 |
+
4. **Configure Secrets**: Use cloud secret managers (AWS Secrets Manager, Azure Key Vault, GCP Secret Manager)
|
| 420 |
+
5. **Enable HTTPS**: Configure SSL/TLS certificates
|
| 421 |
+
6. **Set Up Monitoring**: Enable cloud-native monitoring and logging
|
| 422 |
+
|
| 423 |
+
For complete deployment guides with step-by-step instructions, configuration examples, and best practices, see **[PROJECT.md - Cloud Deployment Guides](PROJECT.md#cloud-deployment-guides)**.
|
| 424 |
+
|
| 425 |
## ⚙️ Configuration
|
| 426 |
|
| 427 |
### Environment Variables
|
docs/README.md
CHANGED
|
@@ -1605,4 +1605,558 @@ response = requests.post(
|
|
| 1605 |
## API Status
|
| 1606 |
Check status at: https://status.llmguardian.com # replace llmguardian.com with your domain
|
| 1607 |
|
| 1608 |
-
Rate limits and API metrics available in dashboard.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1605 |
## API Status
|
| 1606 |
Check status at: https://status.llmguardian.com # replace llmguardian.com with your domain
|
| 1607 |
|
| 1608 |
+
Rate limits and API metrics available in dashboard.
|
| 1609 |
+
|
| 1610 |
+
---
|
| 1611 |
+
|
| 1612 |
+
## ☁️ Cloud Deployment Guides
|
| 1613 |
+
|
| 1614 |
+
LLMGuardian can be deployed on all major cloud platforms. This section provides comprehensive deployment guides for AWS, Google Cloud, Azure, Vercel, and DigitalOcean.
|
| 1615 |
+
|
| 1616 |
+
> **📘 For complete step-by-step instructions with all configuration details, see [PROJECT.md - Cloud Deployment Guides](../PROJECT.md#cloud-deployment-guides)**
|
| 1617 |
+
|
| 1618 |
+
### Quick Start by Platform
|
| 1619 |
+
|
| 1620 |
+
#### AWS Deployment
|
| 1621 |
+
|
| 1622 |
+
**Recommended: ECS with Fargate**
|
| 1623 |
+
|
| 1624 |
+
```bash
|
| 1625 |
+
# Push to ECR
|
| 1626 |
+
aws ecr get-login-password --region us-east-1 | docker login --username AWS --password-stdin YOUR_ACCOUNT_ID.dkr.ecr.us-east-1.amazonaws.com
|
| 1627 |
+
aws ecr create-repository --repository-name llmguardian --region us-east-1
|
| 1628 |
+
|
| 1629 |
+
# Tag and push
|
| 1630 |
+
docker tag llmguardian:latest YOUR_ACCOUNT_ID.dkr.ecr.us-east-1.amazonaws.com/llmguardian:latest
|
| 1631 |
+
docker push YOUR_ACCOUNT_ID.dkr.ecr.us-east-1.amazonaws.com/llmguardian:latest
|
| 1632 |
+
|
| 1633 |
+
# Create ECS cluster and deploy
|
| 1634 |
+
aws ecs create-cluster --cluster-name llmguardian-cluster --region us-east-1
|
| 1635 |
+
aws ecs register-task-definition --cli-input-json file://task-definition.json
|
| 1636 |
+
aws ecs create-service --cluster llmguardian-cluster --service-name llmguardian-service --task-definition llmguardian --desired-count 2
|
| 1637 |
+
```
|
| 1638 |
+
|
| 1639 |
+
**Other AWS Options:**
|
| 1640 |
+
- **Lambda**: Serverless function deployment with Docker containers
|
| 1641 |
+
- **Elastic Beanstalk**: PaaS deployment with auto-scaling
|
| 1642 |
+
- **EKS**: Kubernetes orchestration for large-scale deployments
|
| 1643 |
+
|
| 1644 |
+
**Key Features:**
|
| 1645 |
+
- Auto-scaling with CloudWatch metrics
|
| 1646 |
+
- Load balancing with ALB/NLB
|
| 1647 |
+
- Secrets management with Secrets Manager
|
| 1648 |
+
- CloudWatch logging and monitoring
|
| 1649 |
+
|
| 1650 |
+
#### Google Cloud Platform
|
| 1651 |
+
|
| 1652 |
+
**Recommended: Cloud Run**
|
| 1653 |
+
|
| 1654 |
+
```bash
|
| 1655 |
+
# Configure Docker for GCP
|
| 1656 |
+
gcloud auth configure-docker
|
| 1657 |
+
|
| 1658 |
+
# Build and push to GCR
|
| 1659 |
+
docker tag llmguardian:latest gcr.io/YOUR_PROJECT_ID/llmguardian:latest
|
| 1660 |
+
docker push gcr.io/YOUR_PROJECT_ID/llmguardian:latest
|
| 1661 |
+
|
| 1662 |
+
# Deploy to Cloud Run
|
| 1663 |
+
gcloud run deploy llmguardian \
|
| 1664 |
+
--image gcr.io/YOUR_PROJECT_ID/llmguardian:latest \
|
| 1665 |
+
--platform managed \
|
| 1666 |
+
--region us-central1 \
|
| 1667 |
+
--allow-unauthenticated \
|
| 1668 |
+
--memory 2Gi \
|
| 1669 |
+
--cpu 2 \
|
| 1670 |
+
--port 8000 \
|
| 1671 |
+
--min-instances 1 \
|
| 1672 |
+
--max-instances 10
|
| 1673 |
+
```
|
| 1674 |
+
|
| 1675 |
+
**Other GCP Options:**
|
| 1676 |
+
- **GKE (Google Kubernetes Engine)**: Full Kubernetes control
|
| 1677 |
+
- **App Engine**: PaaS with automatic scaling
|
| 1678 |
+
- **Cloud Functions**: Event-driven serverless
|
| 1679 |
+
|
| 1680 |
+
**Key Features:**
|
| 1681 |
+
- Automatic HTTPS and custom domains
|
| 1682 |
+
- Built-in auto-scaling
|
| 1683 |
+
- Secret Manager integration
|
| 1684 |
+
- Cloud Logging and Monitoring
|
| 1685 |
+
|
| 1686 |
+
#### Microsoft Azure
|
| 1687 |
+
|
| 1688 |
+
**Recommended: Container Instances**
|
| 1689 |
+
|
| 1690 |
+
```bash
|
| 1691 |
+
# Create resource group and registry
|
| 1692 |
+
az group create --name llmguardian-rg --location eastus
|
| 1693 |
+
az acr create --resource-group llmguardian-rg --name llmguardianacr --sku Basic
|
| 1694 |
+
az acr login --name llmguardianacr
|
| 1695 |
+
|
| 1696 |
+
# Push image
|
| 1697 |
+
docker tag llmguardian:latest llmguardianacr.azurecr.io/llmguardian:latest
|
| 1698 |
+
docker push llmguardianacr.azurecr.io/llmguardian:latest
|
| 1699 |
+
|
| 1700 |
+
# Deploy container instance
|
| 1701 |
+
az container create \
|
| 1702 |
+
--resource-group llmguardian-rg \
|
| 1703 |
+
--name llmguardian-container \
|
| 1704 |
+
--image llmguardianacr.azurecr.io/llmguardian:latest \
|
| 1705 |
+
--cpu 2 \
|
| 1706 |
+
--memory 4 \
|
| 1707 |
+
--dns-name-label llmguardian \
|
| 1708 |
+
--ports 8000 \
|
| 1709 |
+
--environment-variables LOG_LEVEL=INFO
|
| 1710 |
+
```
|
| 1711 |
+
|
| 1712 |
+
**Other Azure Options:**
|
| 1713 |
+
- **App Service**: Web App for Containers with built-in CI/CD
|
| 1714 |
+
- **AKS (Azure Kubernetes Service)**: Managed Kubernetes
|
| 1715 |
+
- **Azure Functions**: Serverless with Python support
|
| 1716 |
+
|
| 1717 |
+
**Key Features:**
|
| 1718 |
+
- Azure Key Vault for secrets
|
| 1719 |
+
- Application Insights monitoring
|
| 1720 |
+
- Azure CDN integration
|
| 1721 |
+
- Auto-scaling capabilities
|
| 1722 |
+
|
| 1723 |
+
#### Vercel Deployment
|
| 1724 |
+
|
| 1725 |
+
**Serverless API Deployment**
|
| 1726 |
+
|
| 1727 |
+
```bash
|
| 1728 |
+
# Install Vercel CLI
|
| 1729 |
+
npm i -g vercel
|
| 1730 |
+
|
| 1731 |
+
# Login and deploy
|
| 1732 |
+
vercel login
|
| 1733 |
+
vercel --prod
|
| 1734 |
+
```
|
| 1735 |
+
|
| 1736 |
+
**Configuration** (`vercel.json`):
|
| 1737 |
+
```json
|
| 1738 |
+
{
|
| 1739 |
+
"version": 2,
|
| 1740 |
+
"builds": [
|
| 1741 |
+
{
|
| 1742 |
+
"src": "src/llmguardian/api/app.py",
|
| 1743 |
+
"use": "@vercel/python"
|
| 1744 |
+
}
|
| 1745 |
+
],
|
| 1746 |
+
"routes": [
|
| 1747 |
+
{
|
| 1748 |
+
"src": "/(.*)",
|
| 1749 |
+
"dest": "src/llmguardian/api/app.py"
|
| 1750 |
+
}
|
| 1751 |
+
],
|
| 1752 |
+
"env": {
|
| 1753 |
+
"LOG_LEVEL": "INFO",
|
| 1754 |
+
"ENVIRONMENT": "production"
|
| 1755 |
+
}
|
| 1756 |
+
}
|
| 1757 |
+
```
|
| 1758 |
+
|
| 1759 |
+
**Key Features:**
|
| 1760 |
+
- Automatic HTTPS and custom domains
|
| 1761 |
+
- Edge network deployment
|
| 1762 |
+
- Environment variable management
|
| 1763 |
+
- GitHub integration for auto-deploy
|
| 1764 |
+
|
| 1765 |
+
**Limitations:**
|
| 1766 |
+
- 10s execution time (Hobby), 60s (Pro)
|
| 1767 |
+
- Better for API routes than long-running processes
|
| 1768 |
+
|
| 1769 |
+
#### DigitalOcean Deployment
|
| 1770 |
+
|
| 1771 |
+
**Recommended: App Platform**
|
| 1772 |
+
|
| 1773 |
+
```bash
|
| 1774 |
+
# Install doctl
|
| 1775 |
+
brew install doctl # or download from DigitalOcean
|
| 1776 |
+
|
| 1777 |
+
# Authenticate
|
| 1778 |
+
doctl auth init
|
| 1779 |
+
|
| 1780 |
+
# Create app from spec
|
| 1781 |
+
doctl apps create --spec .do/app.yaml
|
| 1782 |
+
```
|
| 1783 |
+
|
| 1784 |
+
**Configuration** (`.do/app.yaml`):
|
| 1785 |
+
```yaml
|
| 1786 |
+
name: llmguardian
|
| 1787 |
+
services:
|
| 1788 |
+
- name: api
|
| 1789 |
+
github:
|
| 1790 |
+
repo: dewitt4/LLMGuardian
|
| 1791 |
+
branch: main
|
| 1792 |
+
deploy_on_push: true
|
| 1793 |
+
dockerfile_path: docker/dockerfile
|
| 1794 |
+
http_port: 8000
|
| 1795 |
+
instance_count: 2
|
| 1796 |
+
instance_size_slug: professional-s
|
| 1797 |
+
routes:
|
| 1798 |
+
- path: /
|
| 1799 |
+
envs:
|
| 1800 |
+
- key: LOG_LEVEL
|
| 1801 |
+
value: INFO
|
| 1802 |
+
- key: ENVIRONMENT
|
| 1803 |
+
value: production
|
| 1804 |
+
health_check:
|
| 1805 |
+
http_path: /health
|
| 1806 |
+
```
|
| 1807 |
+
|
| 1808 |
+
**Other DigitalOcean Options:**
|
| 1809 |
+
- **DOKS (DigitalOcean Kubernetes)**: Managed Kubernetes
|
| 1810 |
+
- **Droplets**: Traditional VMs with Docker
|
| 1811 |
+
|
| 1812 |
+
**Key Features:**
|
| 1813 |
+
- Simple pricing and scaling
|
| 1814 |
+
- Built-in monitoring
|
| 1815 |
+
- Automatic HTTPS
|
| 1816 |
+
- GitHub integration
|
| 1817 |
+
|
| 1818 |
+
### Platform Comparison
|
| 1819 |
+
|
| 1820 |
+
| Feature | AWS | GCP | Azure | Vercel | DigitalOcean |
|
| 1821 |
+
|---------|-----|-----|-------|--------|--------------|
|
| 1822 |
+
| **Ease of Setup** | ⭐⭐⭐ | ⭐⭐⭐⭐⭐ | ⭐⭐⭐ | ⭐⭐⭐⭐⭐ | ⭐⭐⭐⭐ |
|
| 1823 |
+
| **Auto-Scaling** | Excellent | Excellent | Excellent | Automatic | Good |
|
| 1824 |
+
| **Cost (Monthly)** | $50-200 | $30-150 | $50-200 | $20-100 | $24-120 |
|
| 1825 |
+
| **Best For** | Enterprise | Startups | Enterprise | API/JAMstack | Simple Apps |
|
| 1826 |
+
| **Container Support** | ✅ ECS/EKS | ✅ Cloud Run/GKE | ✅ ACI/AKS | ❌ | ✅ App Platform |
|
| 1827 |
+
| **Serverless** | ✅ Lambda | ✅ Functions | ✅ Functions | ✅ Functions | Limited |
|
| 1828 |
+
| **Kubernetes** | ✅ EKS | ✅ GKE | ✅ AKS | ❌ | ✅ DOKS |
|
| 1829 |
+
| **Free Tier** | Yes | Yes | Yes | Yes | No |
|
| 1830 |
+
|
| 1831 |
+
### Deployment Prerequisites
|
| 1832 |
+
|
| 1833 |
+
Before deploying to any cloud platform:
|
| 1834 |
+
|
| 1835 |
+
#### 1. Prepare Environment Configuration
|
| 1836 |
+
|
| 1837 |
+
```bash
|
| 1838 |
+
# Copy and configure environment variables
|
| 1839 |
+
cp .env.example .env
|
| 1840 |
+
|
| 1841 |
+
# Edit with your settings
|
| 1842 |
+
nano .env
|
| 1843 |
+
```
|
| 1844 |
+
|
| 1845 |
+
Key variables to set:
|
| 1846 |
+
- `SECURITY_RISK_THRESHOLD`
|
| 1847 |
+
- `API_SERVER_PORT`
|
| 1848 |
+
- `LOG_LEVEL`
|
| 1849 |
+
- `ENVIRONMENT` (production, staging, development)
|
| 1850 |
+
- API keys and secrets
|
| 1851 |
+
|
| 1852 |
+
#### 2. Build Docker Image
|
| 1853 |
+
|
| 1854 |
+
```bash
|
| 1855 |
+
# Build from project root
|
| 1856 |
+
docker build -t llmguardian:latest -f docker/dockerfile .
|
| 1857 |
+
|
| 1858 |
+
# Test locally
|
| 1859 |
+
docker run -p 8000:8000 --env-file .env llmguardian:latest
|
| 1860 |
+
```
|
| 1861 |
+
|
| 1862 |
+
#### 3. Set Up Cloud CLI Tools
|
| 1863 |
+
|
| 1864 |
+
**AWS:**
|
| 1865 |
+
```bash
|
| 1866 |
+
# Install AWS CLI
|
| 1867 |
+
curl "https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip" -o "awscliv2.zip"
|
| 1868 |
+
unzip awscliv2.zip
|
| 1869 |
+
sudo ./aws/install
|
| 1870 |
+
|
| 1871 |
+
# Configure credentials
|
| 1872 |
+
aws configure
|
| 1873 |
+
```
|
| 1874 |
+
|
| 1875 |
+
**GCP:**
|
| 1876 |
+
```bash
|
| 1877 |
+
# Install gcloud SDK
|
| 1878 |
+
curl https://sdk.cloud.google.com | bash
|
| 1879 |
+
exec -l $SHELL
|
| 1880 |
+
|
| 1881 |
+
# Authenticate
|
| 1882 |
+
gcloud init
|
| 1883 |
+
gcloud auth login
|
| 1884 |
+
```
|
| 1885 |
+
|
| 1886 |
+
**Azure:**
|
| 1887 |
+
```bash
|
| 1888 |
+
# Install Azure CLI
|
| 1889 |
+
curl -sL https://aka.ms/InstallAzureCLIDeb | sudo bash
|
| 1890 |
+
|
| 1891 |
+
# Login
|
| 1892 |
+
az login
|
| 1893 |
+
```
|
| 1894 |
+
|
| 1895 |
+
**Vercel:**
|
| 1896 |
+
```bash
|
| 1897 |
+
# Install Vercel CLI
|
| 1898 |
+
npm i -g vercel
|
| 1899 |
+
|
| 1900 |
+
# Login
|
| 1901 |
+
vercel login
|
| 1902 |
+
```
|
| 1903 |
+
|
| 1904 |
+
**DigitalOcean:**
|
| 1905 |
+
```bash
|
| 1906 |
+
# Install doctl
|
| 1907 |
+
brew install doctl # macOS
|
| 1908 |
+
# or download from https://github.com/digitalocean/doctl
|
| 1909 |
+
|
| 1910 |
+
# Authenticate
|
| 1911 |
+
doctl auth init
|
| 1912 |
+
```
|
| 1913 |
+
|
| 1914 |
+
#### 4. Configure Secrets Management
|
| 1915 |
+
|
| 1916 |
+
**AWS Secrets Manager:**
|
| 1917 |
+
```bash
|
| 1918 |
+
aws secretsmanager create-secret \
|
| 1919 |
+
--name llmguardian-api-key \
|
| 1920 |
+
--secret-string "your-secret-key"
|
| 1921 |
+
```
|
| 1922 |
+
|
| 1923 |
+
**GCP Secret Manager:**
|
| 1924 |
+
```bash
|
| 1925 |
+
echo -n "your-secret-key" | gcloud secrets create llmguardian-api-key --data-file=-
|
| 1926 |
+
```
|
| 1927 |
+
|
| 1928 |
+
**Azure Key Vault:**
|
| 1929 |
+
```bash
|
| 1930 |
+
az keyvault create --name llmguardian-vault --resource-group llmguardian-rg
|
| 1931 |
+
az keyvault secret set --vault-name llmguardian-vault --name api-key --value "your-secret-key"
|
| 1932 |
+
```
|
| 1933 |
+
|
| 1934 |
+
**Vercel:**
|
| 1935 |
+
```bash
|
| 1936 |
+
vercel env add API_KEY
|
| 1937 |
+
# Enter secret when prompted
|
| 1938 |
+
```
|
| 1939 |
+
|
| 1940 |
+
**DigitalOcean:**
|
| 1941 |
+
```bash
|
| 1942 |
+
# Via App Platform dashboard or doctl
|
| 1943 |
+
doctl apps update YOUR_APP_ID --spec .do/app.yaml
|
| 1944 |
+
```
|
| 1945 |
+
|
| 1946 |
+
### Best Practices for Cloud Deployment
|
| 1947 |
+
|
| 1948 |
+
#### Security Hardening
|
| 1949 |
+
|
| 1950 |
+
1. **Use Secret Managers**
|
| 1951 |
+
- Never hardcode secrets in code or environment files
|
| 1952 |
+
- Rotate secrets regularly
|
| 1953 |
+
- Use least-privilege IAM roles
|
| 1954 |
+
|
| 1955 |
+
2. **Enable HTTPS/TLS**
|
| 1956 |
+
- Use cloud-provided certificates (free with most platforms)
|
| 1957 |
+
- Force HTTPS redirects
|
| 1958 |
+
- Configure SSL/TLS termination at load balancer
|
| 1959 |
+
|
| 1960 |
+
3. **Implement WAF (Web Application Firewall)**
|
| 1961 |
+
- AWS: AWS WAF
|
| 1962 |
+
- Azure: Azure Application Gateway WAF
|
| 1963 |
+
- GCP: Cloud Armor
|
| 1964 |
+
- Vercel: Built-in DDoS protection
|
| 1965 |
+
- DigitalOcean: Cloud Firewalls
|
| 1966 |
+
|
| 1967 |
+
4. **Network Security**
|
| 1968 |
+
- Configure VPCs/VNets for isolation
|
| 1969 |
+
- Use security groups/firewall rules
|
| 1970 |
+
- Implement least-privilege network policies
|
| 1971 |
+
|
| 1972 |
+
#### Monitoring & Logging
|
| 1973 |
+
|
| 1974 |
+
1. **Enable Cloud-Native Monitoring**
|
| 1975 |
+
- AWS: CloudWatch
|
| 1976 |
+
- GCP: Cloud Monitoring & Logging
|
| 1977 |
+
- Azure: Application Insights
|
| 1978 |
+
- Vercel: Analytics
|
| 1979 |
+
- DigitalOcean: Built-in monitoring
|
| 1980 |
+
|
| 1981 |
+
2. **Configure Alerts**
|
| 1982 |
+
```bash
|
| 1983 |
+
# Example: AWS CloudWatch alarm
|
| 1984 |
+
aws cloudwatch put-metric-alarm \
|
| 1985 |
+
--alarm-name llmguardian-high-cpu \
|
| 1986 |
+
--alarm-description "Alert when CPU exceeds 80%" \
|
| 1987 |
+
--metric-name CPUUtilization \
|
| 1988 |
+
--threshold 80
|
| 1989 |
+
```
|
| 1990 |
+
|
| 1991 |
+
3. **Set Up Log Aggregation**
|
| 1992 |
+
- Centralize logs for analysis
|
| 1993 |
+
- Implement log retention policies
|
| 1994 |
+
- Enable audit logging
|
| 1995 |
+
|
| 1996 |
+
#### Performance Optimization
|
| 1997 |
+
|
| 1998 |
+
1. **Auto-Scaling Configuration**
|
| 1999 |
+
- Set appropriate min/max instances
|
| 2000 |
+
- Configure based on CPU/memory metrics
|
| 2001 |
+
- Implement graceful shutdown
|
| 2002 |
+
|
| 2003 |
+
2. **Caching**
|
| 2004 |
+
- Use Redis/Memcached for response caching
|
| 2005 |
+
- Implement CDN for static content
|
| 2006 |
+
- Cache embeddings and common queries
|
| 2007 |
+
|
| 2008 |
+
3. **Database Optimization**
|
| 2009 |
+
- Use managed database services
|
| 2010 |
+
- Implement connection pooling
|
| 2011 |
+
- Regular performance monitoring
|
| 2012 |
+
|
| 2013 |
+
#### Cost Optimization
|
| 2014 |
+
|
| 2015 |
+
1. **Right-Sizing**
|
| 2016 |
+
- Start small and scale based on metrics
|
| 2017 |
+
- Use spot/preemptible instances for non-critical workloads
|
| 2018 |
+
- Monitor and optimize resource usage
|
| 2019 |
+
|
| 2020 |
+
2. **Reserved Instances**
|
| 2021 |
+
- Purchase reserved capacity for predictable workloads
|
| 2022 |
+
- 1-year or 3-year commitments for savings
|
| 2023 |
+
|
| 2024 |
+
3. **Cost Alerts**
|
| 2025 |
+
```bash
|
| 2026 |
+
# AWS Budget alert
|
| 2027 |
+
aws budgets create-budget \
|
| 2028 |
+
--account-id YOUR_ACCOUNT_ID \
|
| 2029 |
+
--budget file://budget.json
|
| 2030 |
+
```
|
| 2031 |
+
|
| 2032 |
+
### CI/CD Integration
|
| 2033 |
+
|
| 2034 |
+
**GitHub Actions Example** (`.github/workflows/deploy-cloud.yml`):
|
| 2035 |
+
|
| 2036 |
+
```yaml
|
| 2037 |
+
name: Deploy to Cloud
|
| 2038 |
+
|
| 2039 |
+
on:
|
| 2040 |
+
push:
|
| 2041 |
+
branches: [main]
|
| 2042 |
+
workflow_dispatch:
|
| 2043 |
+
|
| 2044 |
+
jobs:
|
| 2045 |
+
deploy-aws:
|
| 2046 |
+
runs-on: ubuntu-latest
|
| 2047 |
+
steps:
|
| 2048 |
+
- uses: actions/checkout@v4
|
| 2049 |
+
|
| 2050 |
+
- name: Configure AWS credentials
|
| 2051 |
+
uses: aws-actions/configure-aws-credentials@v1
|
| 2052 |
+
with:
|
| 2053 |
+
aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
|
| 2054 |
+
aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
|
| 2055 |
+
aws-region: us-east-1
|
| 2056 |
+
|
| 2057 |
+
- name: Login to Amazon ECR
|
| 2058 |
+
run: |
|
| 2059 |
+
aws ecr get-login-password --region us-east-1 | docker login --username AWS --password-stdin ${{ secrets.AWS_ACCOUNT_ID }}.dkr.ecr.us-east-1.amazonaws.com
|
| 2060 |
+
|
| 2061 |
+
- name: Build and push
|
| 2062 |
+
run: |
|
| 2063 |
+
docker build -t llmguardian:latest -f docker/dockerfile .
|
| 2064 |
+
docker tag llmguardian:latest ${{ secrets.AWS_ACCOUNT_ID }}.dkr.ecr.us-east-1.amazonaws.com/llmguardian:latest
|
| 2065 |
+
docker push ${{ secrets.AWS_ACCOUNT_ID }}.dkr.ecr.us-east-1.amazonaws.com/llmguardian:latest
|
| 2066 |
+
|
| 2067 |
+
- name: Deploy to ECS
|
| 2068 |
+
run: |
|
| 2069 |
+
aws ecs update-service --cluster llmguardian-cluster --service llmguardian-service --force-new-deployment
|
| 2070 |
+
|
| 2071 |
+
deploy-gcp:
|
| 2072 |
+
runs-on: ubuntu-latest
|
| 2073 |
+
steps:
|
| 2074 |
+
- uses: actions/checkout@v4
|
| 2075 |
+
|
| 2076 |
+
- uses: google-github-actions/setup-gcloud@v0
|
| 2077 |
+
with:
|
| 2078 |
+
service_account_key: ${{ secrets.GCP_SA_KEY }}
|
| 2079 |
+
project_id: ${{ secrets.GCP_PROJECT_ID }}
|
| 2080 |
+
|
| 2081 |
+
- name: Deploy to Cloud Run
|
| 2082 |
+
run: |
|
| 2083 |
+
gcloud auth configure-docker
|
| 2084 |
+
docker build -t llmguardian:latest -f docker/dockerfile .
|
| 2085 |
+
docker tag llmguardian:latest gcr.io/${{ secrets.GCP_PROJECT_ID }}/llmguardian:latest
|
| 2086 |
+
docker push gcr.io/${{ secrets.GCP_PROJECT_ID }}/llmguardian:latest
|
| 2087 |
+
gcloud run deploy llmguardian --image gcr.io/${{ secrets.GCP_PROJECT_ID }}/llmguardian:latest --region us-central1
|
| 2088 |
+
|
| 2089 |
+
deploy-azure:
|
| 2090 |
+
runs-on: ubuntu-latest
|
| 2091 |
+
steps:
|
| 2092 |
+
- uses: actions/checkout@v4
|
| 2093 |
+
|
| 2094 |
+
- uses: azure/login@v1
|
| 2095 |
+
with:
|
| 2096 |
+
creds: ${{ secrets.AZURE_CREDENTIALS }}
|
| 2097 |
+
|
| 2098 |
+
- name: Deploy to Azure
|
| 2099 |
+
run: |
|
| 2100 |
+
az acr login --name llmguardianacr
|
| 2101 |
+
docker build -t llmguardian:latest -f docker/dockerfile .
|
| 2102 |
+
docker tag llmguardian:latest llmguardianacr.azurecr.io/llmguardian:latest
|
| 2103 |
+
docker push llmguardianacr.azurecr.io/llmguardian:latest
|
| 2104 |
+
az container restart --resource-group llmguardian-rg --name llmguardian-container
|
| 2105 |
+
```
|
| 2106 |
+
|
| 2107 |
+
### Troubleshooting Common Issues
|
| 2108 |
+
|
| 2109 |
+
#### Port Binding Issues
|
| 2110 |
+
```bash
|
| 2111 |
+
# Ensure correct port exposure
|
| 2112 |
+
docker run -p 8000:8000 llmguardian:latest
|
| 2113 |
+
|
| 2114 |
+
# Check health endpoint
|
| 2115 |
+
curl http://localhost:8000/health
|
| 2116 |
+
```
|
| 2117 |
+
|
| 2118 |
+
#### Memory/CPU Limits
|
| 2119 |
+
```bash
|
| 2120 |
+
# Increase container resources
|
| 2121 |
+
# AWS ECS: Update task definition
|
| 2122 |
+
# GCP Cloud Run: Use --memory and --cpu flags
|
| 2123 |
+
# Azure: Update container instance specs
|
| 2124 |
+
```
|
| 2125 |
+
|
| 2126 |
+
#### Environment Variables Not Loading
|
| 2127 |
+
```bash
|
| 2128 |
+
# Verify environment variables
|
| 2129 |
+
docker run llmguardian:latest env | grep LOG_LEVEL
|
| 2130 |
+
|
| 2131 |
+
# Check cloud secret access
|
| 2132 |
+
# AWS: Verify IAM role permissions
|
| 2133 |
+
# GCP: Check service account permissions
|
| 2134 |
+
# Azure: Verify Key Vault access policies
|
| 2135 |
+
```
|
| 2136 |
+
|
| 2137 |
+
#### Image Pull Failures
|
| 2138 |
+
```bash
|
| 2139 |
+
# Authenticate with registry
|
| 2140 |
+
aws ecr get-login-password | docker login --username AWS --password-stdin YOUR_REGISTRY
|
| 2141 |
+
gcloud auth configure-docker
|
| 2142 |
+
az acr login --name YOUR_REGISTRY
|
| 2143 |
+
```
|
| 2144 |
+
|
| 2145 |
+
### Additional Resources
|
| 2146 |
+
|
| 2147 |
+
- **[PROJECT.md - Complete Cloud Deployment Guides](../PROJECT.md#cloud-deployment-guides)**: Full step-by-step instructions with all configuration details
|
| 2148 |
+
- **[Docker README](../docker/README.md)**: Docker-specific documentation
|
| 2149 |
+
- **[Environment Variables](.env.example)**: All configuration options
|
| 2150 |
+
- **[GitHub Actions Workflows](../.github/workflows/README.md)**: CI/CD automation
|
| 2151 |
+
|
| 2152 |
+
### Support
|
| 2153 |
+
|
| 2154 |
+
For deployment issues:
|
| 2155 |
+
1. Check the [GitHub Issues](https://github.com/dewitt4/LLMGuardian/issues)
|
| 2156 |
+
2. Review cloud provider documentation
|
| 2157 |
+
3. Enable debug logging: `LOG_LEVEL=DEBUG`
|
| 2158 |
+
4. Check health endpoint: `curl http://your-deployment/health`
|
| 2159 |
+
|
| 2160 |
+
---
|
| 2161 |
+
|
| 2162 |
+
**Ready to deploy? Choose your platform above and follow the deployment guide!** 🚀
|