copilot_work_3 / README.md
arshvinw5's picture
add rate limit to submittions
416bdd1
|
Raw History Blame
5.54 kB
# Chainkit
Chainkit is an SEO-friendly directory for discovering and sharing public, open-source AI tools used in supply chain planning, procurement, logistics, warehousing, and visibility. Tool detail pages are rendered on the server and included in the generated sitemap.
## Features
- Search and filter the directory by supply-chain category.
- Sort tools by rating, review count, and recency.
- Submit a project using its public GitHub repository URL.
- Rate tools from one to five stars; a browser can update its own rating.
- Per-tool metadata, canonical URLs, Open Graph tags, `robots.txt`, and `sitemap.xml`.
- BigQuery persistence on Google Cloud, with an in-memory sample directory for local development.
- Container deployment to Cloud Run.
The local demo directory uses real public GitHub repository links as examples; they are marked as starter data and have no pre-filled community ratings.
## Run locally
Requirements: Node.js 22+ and npm.
```sh
npm ci
copy .env.example .env.local
npm run dev
```
Open [http://localhost:3000](http://localhost:3000). Without BigQuery settings the app starts in local demo mode. Local submissions and ratings are held in server memory and reset when the server restarts.
## BigQuery
1. Create a BigQuery dataset in the region you plan to use.
2. Replace `YOUR_PROJECT_ID` and `YOUR_DATASET` in `infra/bigquery_schema.sql`, then run the SQL in the BigQuery console.
3. Configure Application Default Credentials locally, or attach a runtime service account to Cloud Run.
4. Set `BIGQUERY_PROJECT_ID`, `BIGQUERY_DATASET`, and `NEXT_PUBLIC_SITE_URL`.
The runtime service account needs `roles/bigquery.jobUser` on the project and `roles/bigquery.dataEditor` on the dataset. The app uses parameterized values for BigQuery writes; project and dataset identifiers are validated before being inserted into query text.
## Deploy to Cloud Run
Build and deploy the container from the repository root. Supply your canonical
domain if you already have one; otherwise omit `NEXT_PUBLIC_SITE_URL` on the
first deployment:
```sh
gcloud run deploy chainkit \
--source . \
--region us-central1 \
--allow-unauthenticated \
--service-account chainkit-runtime@YOUR_PROJECT_ID.iam.gserviceaccount.com \
--set-env-vars BIGQUERY_PROJECT_ID=YOUR_PROJECT_ID,BIGQUERY_DATASET=YOUR_DATASET \
--set-secrets VOTER_SECRET=chainkit-voter-secret:latest,ADMIN_TOKEN=chainkit-admin-token:latest,GITHUB_TOKEN=chainkit-github-token:latest
```
Create the three secrets in Secret Manager first and grant the runtime service
account `roles/secretmanager.secretAccessor`. If you are upgrading an existing
deployment, run the migration at the bottom of `infra/bigquery_schema.sql` before
deploying.
### Upgrading an existing BigQuery table
Run the three statements at the bottom of `infra/bigquery_schema.sql` once, either
in the BigQuery console or with `bq query --use_legacy_sql=false`. They add the
`status` and `repo_key` columns and mark existing rows as `approved`. Use an account
with `roles/bigquery.dataEditor` on the dataset and `roles/bigquery.jobUser` on the project.
### Moderation
New submissions are stored as `pending` and are not listed until approved:
```sh
curl -H "Authorization: Bearer $ADMIN_TOKEN" "$URL/api/admin/tools?status=pending"
curl -X POST -H "Authorization: Bearer $ADMIN_TOKEN" -H "Content-Type: application/json" \
-d '{"action":"approve"}' "$URL/api/admin/tools/TOOL_ID"
```
After deployment, set `NEXT_PUBLIC_SITE_URL` to the Cloud Run URL or your
canonical domain so canonical links and the sitemap match the live site:
```sh
gcloud run services describe chainkit --region us-central1 --format="value(status.url)"
gcloud run services update chainkit \
--region us-central1 \
--update-env-vars NEXT_PUBLIC_SITE_URL=https://YOUR_CANONICAL_DOMAIN
```
The deployer needs permission to build/deploy the service and to act as the
runtime service account.
## Important production considerations
- Submissions are held as `pending` until approved through `/api/admin/tools`. The GitHub API is used to confirm the repository exists and is public; a repository that cannot be verified because GitHub is unavailable stays pending. Whether a project is actually about supply chains is a moderator decision.
- Duplicate repositories are rejected (case-insensitive, ignoring `.git` and a trailing slash). BigQuery cannot enforce uniqueness, so the check runs inside the insert; two exactly simultaneous submissions could still both succeed.
- `/api/tools` (3 per hour) and `/api/ratings` (30 per hour) are rate limited per client IP in application memory. On Cloud Run this is per instance and resets on cold start, so it is best-effort. Put Cloud Armor in front of the service for a hard limit.
- Ratings use a server-signed cookie (`VOTER_SECRET`), so voter IDs cannot be made up. Clearing cookies still gives a new identity; the IP rate limit is the second layer. Add sign-in before treating ratings as verified.
- The repository URL is validated and stored as a link. Chainkit does not clone, fetch, or execute submitted code.
- BigQuery is optimized for analytics rather than high-volume transactional writes. For substantial rating or submission traffic, consider a transactional store and retain BigQuery for analytics.
## Project structure
```text
src/app/ Next.js pages, SEO routes, and API endpoints
src/lib/tools.ts BigQuery adapter and local demo data
infra/ BigQuery table schema
Dockerfile Cloud Run container
```