nayara-v1 / source /deploy /aws /README.md
sriharithebest's picture
Upload folder using huggingface_hub
0990c3f verified
|
Raw History Blame Contribute Delete
4.1 kB
# AWS API deployment (ECS Fargate)
This is the supported AWS path for the CPU API. It uses one ECS Fargate task with 4 vCPU/8 GB, an ECR image, an S3 checkpoint, Secrets Manager for the bearer token, CloudWatch logs, and an Application Load Balancer (ALB). Do not use Lambda for this checkpoint: model startup and memory are a poor fit.
## 1. Upload the final checkpoint
The S3 prefix must contain `config.json`, `model.safetensors`, `tokenizer.json`, and preferably `model-int8.pt`.
```bash
export AWS_REGION=ap-south-1
export ACCOUNT_ID=$(aws sts get-caller-identity --query Account --output text)
export BUCKET=your-private-nayara-model-bucket
export CHECKPOINT=/path/to/final/checkpoint
aws s3 sync "$CHECKPOINT" "s3://$BUCKET/nayara/v1/" --exclude 'trainer.pt'
```
Keep the bucket private and in the same Region as ECS. The task role needs only `s3:GetObject` on `arn:aws:s3:::$BUCKET/nayara/v1/*` and `s3:ListBucket` scoped to the prefix.
## 2. Build and push the CPU image
```bash
aws ecr create-repository --repository-name nayara-api --region "$AWS_REGION" 2>/dev/null || true
aws ecr get-login-password --region "$AWS_REGION" | \
docker login --username AWS --password-stdin "$ACCOUNT_ID.dkr.ecr.$AWS_REGION.amazonaws.com"
docker build --platform linux/amd64 -t nayara-api:1.0 .
docker tag nayara-api:1.0 "$ACCOUNT_ID.dkr.ecr.$AWS_REGION.amazonaws.com/nayara-api:1.0"
docker push "$ACCOUNT_ID.dkr.ecr.$AWS_REGION.amazonaws.com/nayara-api:1.0"
```
The image uses the reviewed versions in `constraints.txt`. Update them intentionally and rerun the API/INT8 tests before rebuilding.
## 3. IAM and secret
- Execution role: trust `ecs-tasks.amazonaws.com`; attach `AmazonECSTaskExecutionRolePolicy` and permission to read the API-key secret.
- Task role: trust `ecs-tasks.amazonaws.com`; grant only the S3 prefix permissions above.
- Create a Secrets Manager secret containing only the API key string:
```bash
openssl rand -hex 32 > nayara-api-key.txt
aws secretsmanager create-secret --name nayara/api-key --secret-string file://nayara-api-key.txt
```
Delete the local key file after storing it securely. Never put the key in the task-definition environment array.
## 4. Register and run
Create the log group, replace all uppercase placeholders in `task-definition.template.json`, then register it:
```bash
aws logs create-log-group --log-group-name /ecs/nayara-api --region "$AWS_REGION" 2>/dev/null || true
aws ecs register-task-definition \
--cli-input-json file://deploy/aws/task-definition.json \
--region "$AWS_REGION"
```
In ECS, create a Fargate service using platform `LATEST`, private subnets with an S3 gateway endpoint or controlled NAT egress, the task security group, and an ALB target group on port 8080. Configure the ALB health check as:
- Path: `/readyz`
- Success code: `200`
- Interval: 30 seconds
- Timeout: 5 seconds
- Healthy/unhealthy threshold: 2/3
- Deregistration delay: at least 30 seconds
Use HTTPS only with an ACM certificate. Allow port 8080 only from the ALB security group. Start with desired count 1 and autoscale on CPU and request count only after measuring memory and latency. Each worker loads a complete model; keep `--workers 1`.
The task template sets `NAYARA_REQUIRE_API_KEY=1`; startup fails closed if Secrets Manager does not inject the key.
## 5. Verify before traffic
```bash
export API_URL=https://your-api.example.com
export NAYARA_API_KEY='the-secret-value'
curl -fsS "$API_URL/healthz"
curl -fsS "$API_URL/readyz"
curl -fsS "$API_URL/v1/models" -H "Authorization: Bearer $NAYARA_API_KEY"
curl -fsS "$API_URL/v1/chat/completions" \
-H "Authorization: Bearer $NAYARA_API_KEY" \
-H 'Content-Type: application/json' \
-d '{"model":"nayara-v1","messages":[{"role":"user","content":"Write a Python binary search."}],"max_tokens":96}'
```
Run a load test before production. The API serializes generation inside each task to prevent CPU oversubscription; scale out tasks for concurrency. Add AWS WAF, ALB access logs, alarms for 5xx/latency/task restarts, and a deployment rollback policy before exposing it publicly.