|
Download source/deploy/aws/README.md from sriharithebest/nayara-v1: direct link, hf CLI and curl.
- Browser
- Download file 4.1 kB
-
https://huggingface.co/sriharithebest/nayara-v1/resolve/main/source/deploy/aws/README.md
- Command line
-
hf download hf://sriharithebest/nayara-v1/source/deploy/aws/README.md
-
curl -L -o README.md https://huggingface.co/sriharithebest/nayara-v1/resolve/main/source/deploy/aws/README.md
4.1 kB
| # AWS API deployment (ECS Fargate) | |
| This is the supported AWS path for the CPU API. It uses one ECS Fargate task with 4 vCPU/8 GB, an ECR image, an S3 checkpoint, Secrets Manager for the bearer token, CloudWatch logs, and an Application Load Balancer (ALB). Do not use Lambda for this checkpoint: model startup and memory are a poor fit. | |
| ## 1. Upload the final checkpoint | |
| The S3 prefix must contain `config.json`, `model.safetensors`, `tokenizer.json`, and preferably `model-int8.pt`. | |
| ```bash | |
| export AWS_REGION=ap-south-1 | |
| export ACCOUNT_ID=$(aws sts get-caller-identity --query Account --output text) | |
| export BUCKET=your-private-nayara-model-bucket | |
| export CHECKPOINT=/path/to/final/checkpoint | |
| aws s3 sync "$CHECKPOINT" "s3://$BUCKET/nayara/v1/" --exclude 'trainer.pt' | |
| ``` | |
| Keep the bucket private and in the same Region as ECS. The task role needs only `s3:GetObject` on `arn:aws:s3:::$BUCKET/nayara/v1/*` and `s3:ListBucket` scoped to the prefix. | |
| ## 2. Build and push the CPU image | |
| ```bash | |
| aws ecr create-repository --repository-name nayara-api --region "$AWS_REGION" 2>/dev/null || true | |
| aws ecr get-login-password --region "$AWS_REGION" | \ | |
| docker login --username AWS --password-stdin "$ACCOUNT_ID.dkr.ecr.$AWS_REGION.amazonaws.com" | |
| docker build --platform linux/amd64 -t nayara-api:1.0 . | |
| docker tag nayara-api:1.0 "$ACCOUNT_ID.dkr.ecr.$AWS_REGION.amazonaws.com/nayara-api:1.0" | |
| docker push "$ACCOUNT_ID.dkr.ecr.$AWS_REGION.amazonaws.com/nayara-api:1.0" | |
| ``` | |
| The image uses the reviewed versions in `constraints.txt`. Update them intentionally and rerun the API/INT8 tests before rebuilding. | |
| ## 3. IAM and secret | |
| - Execution role: trust `ecs-tasks.amazonaws.com`; attach `AmazonECSTaskExecutionRolePolicy` and permission to read the API-key secret. | |
| - Task role: trust `ecs-tasks.amazonaws.com`; grant only the S3 prefix permissions above. | |
| - Create a Secrets Manager secret containing only the API key string: | |
| ```bash | |
| openssl rand -hex 32 > nayara-api-key.txt | |
| aws secretsmanager create-secret --name nayara/api-key --secret-string file://nayara-api-key.txt | |
| ``` | |
| Delete the local key file after storing it securely. Never put the key in the task-definition environment array. | |
| ## 4. Register and run | |
| Create the log group, replace all uppercase placeholders in `task-definition.template.json`, then register it: | |
| ```bash | |
| aws logs create-log-group --log-group-name /ecs/nayara-api --region "$AWS_REGION" 2>/dev/null || true | |
| aws ecs register-task-definition \ | |
| --cli-input-json file://deploy/aws/task-definition.json \ | |
| --region "$AWS_REGION" | |
| ``` | |
| In ECS, create a Fargate service using platform `LATEST`, private subnets with an S3 gateway endpoint or controlled NAT egress, the task security group, and an ALB target group on port 8080. Configure the ALB health check as: | |
| - Path: `/readyz` | |
| - Success code: `200` | |
| - Interval: 30 seconds | |
| - Timeout: 5 seconds | |
| - Healthy/unhealthy threshold: 2/3 | |
| - Deregistration delay: at least 30 seconds | |
| Use HTTPS only with an ACM certificate. Allow port 8080 only from the ALB security group. Start with desired count 1 and autoscale on CPU and request count only after measuring memory and latency. Each worker loads a complete model; keep `--workers 1`. | |
| The task template sets `NAYARA_REQUIRE_API_KEY=1`; startup fails closed if Secrets Manager does not inject the key. | |
| ## 5. Verify before traffic | |
| ```bash | |
| export API_URL=https://your-api.example.com | |
| export NAYARA_API_KEY='the-secret-value' | |
| curl -fsS "$API_URL/healthz" | |
| curl -fsS "$API_URL/readyz" | |
| curl -fsS "$API_URL/v1/models" -H "Authorization: Bearer $NAYARA_API_KEY" | |
| curl -fsS "$API_URL/v1/chat/completions" \ | |
| -H "Authorization: Bearer $NAYARA_API_KEY" \ | |
| -H 'Content-Type: application/json' \ | |
| -d '{"model":"nayara-v1","messages":[{"role":"user","content":"Write a Python binary search."}],"max_tokens":96}' | |
| ``` | |
| Run a load test before production. The API serializes generation inside each task to prevent CPU oversubscription; scale out tasks for concurrency. Add AWS WAF, ALB access logs, alarms for 5xx/latency/task restarts, and a deployment rollback policy before exposing it publicly. | |