Download source/deploy/aws/README.md from sriharithebest/nayara-v1: direct link, hf CLI and curl.
- Browser
- Download file 4.1 kB
-
https://huggingface.co/sriharithebest/nayara-v1/resolve/main/source/deploy/aws/README.md
- Command line
-
hf download hf://sriharithebest/nayara-v1/source/deploy/aws/README.md
-
curl -L -o README.md https://huggingface.co/sriharithebest/nayara-v1/resolve/main/source/deploy/aws/README.md
AWS API deployment (ECS Fargate)
This is the supported AWS path for the CPU API. It uses one ECS Fargate task with 4 vCPU/8 GB, an ECR image, an S3 checkpoint, Secrets Manager for the bearer token, CloudWatch logs, and an Application Load Balancer (ALB). Do not use Lambda for this checkpoint: model startup and memory are a poor fit.
1. Upload the final checkpoint
The S3 prefix must contain config.json, model.safetensors, tokenizer.json, and preferably model-int8.pt.
export AWS_REGION=ap-south-1
export ACCOUNT_ID=$(aws sts get-caller-identity --query Account --output text)
export BUCKET=your-private-nayara-model-bucket
export CHECKPOINT=/path/to/final/checkpoint
aws s3 sync "$CHECKPOINT" "s3://$BUCKET/nayara/v1/" --exclude 'trainer.pt'
Keep the bucket private and in the same Region as ECS. The task role needs only s3:GetObject on arn:aws:s3:::$BUCKET/nayara/v1/* and s3:ListBucket scoped to the prefix.
2. Build and push the CPU image
aws ecr create-repository --repository-name nayara-api --region "$AWS_REGION" 2>/dev/null || true
aws ecr get-login-password --region "$AWS_REGION" | \
docker login --username AWS --password-stdin "$ACCOUNT_ID.dkr.ecr.$AWS_REGION.amazonaws.com"
docker build --platform linux/amd64 -t nayara-api:1.0 .
docker tag nayara-api:1.0 "$ACCOUNT_ID.dkr.ecr.$AWS_REGION.amazonaws.com/nayara-api:1.0"
docker push "$ACCOUNT_ID.dkr.ecr.$AWS_REGION.amazonaws.com/nayara-api:1.0"
The image uses the reviewed versions in constraints.txt. Update them intentionally and rerun the API/INT8 tests before rebuilding.
3. IAM and secret
- Execution role: trust
ecs-tasks.amazonaws.com; attachAmazonECSTaskExecutionRolePolicyand permission to read the API-key secret. - Task role: trust
ecs-tasks.amazonaws.com; grant only the S3 prefix permissions above. - Create a Secrets Manager secret containing only the API key string:
openssl rand -hex 32 > nayara-api-key.txt
aws secretsmanager create-secret --name nayara/api-key --secret-string file://nayara-api-key.txt
Delete the local key file after storing it securely. Never put the key in the task-definition environment array.
4. Register and run
Create the log group, replace all uppercase placeholders in task-definition.template.json, then register it:
aws logs create-log-group --log-group-name /ecs/nayara-api --region "$AWS_REGION" 2>/dev/null || true
aws ecs register-task-definition \
--cli-input-json file://deploy/aws/task-definition.json \
--region "$AWS_REGION"
In ECS, create a Fargate service using platform LATEST, private subnets with an S3 gateway endpoint or controlled NAT egress, the task security group, and an ALB target group on port 8080. Configure the ALB health check as:
- Path:
/readyz - Success code:
200 - Interval: 30 seconds
- Timeout: 5 seconds
- Healthy/unhealthy threshold: 2/3
- Deregistration delay: at least 30 seconds
Use HTTPS only with an ACM certificate. Allow port 8080 only from the ALB security group. Start with desired count 1 and autoscale on CPU and request count only after measuring memory and latency. Each worker loads a complete model; keep --workers 1.
The task template sets NAYARA_REQUIRE_API_KEY=1; startup fails closed if Secrets Manager does not inject the key.
5. Verify before traffic
export API_URL=https://your-api.example.com
export NAYARA_API_KEY='the-secret-value'
curl -fsS "$API_URL/healthz"
curl -fsS "$API_URL/readyz"
curl -fsS "$API_URL/v1/models" -H "Authorization: Bearer $NAYARA_API_KEY"
curl -fsS "$API_URL/v1/chat/completions" \
-H "Authorization: Bearer $NAYARA_API_KEY" \
-H 'Content-Type: application/json' \
-d '{"model":"nayara-v1","messages":[{"role":"user","content":"Write a Python binary search."}],"max_tokens":96}'
Run a load test before production. The API serializes generation inside each task to prevent CPU oversubscription; scale out tasks for concurrency. Add AWS WAF, ALB access logs, alarms for 5xx/latency/task restarts, and a deployment rollback policy before exposing it publicly.