--- title: Attestation Store emoji: 🛡️ colorFrom: gray colorTo: blue sdk: docker app_port: 7860 pinned: true docker_build_args: BUILD_DOCS: false --- # Attestation Store ## Setup ## Environment Variables Below is a list of environment variables that can be set to configure the Attestation Store application. You can set these variables in your shell or in a `.env` file if you're using Docker Compose. | Variable | Default | Description | |----------|---------|-------------| | `SESSION_SECRET` | `dev-secret-key-change-in-production` | Flask session secret used for signing cookies; change in production. | | `JWT_SECRET_KEY` | `jwt-secret-change-in-production` | Secret key for signing JWT access tokens returned by `/v1/login`. | | `DATABASE_URL` | `sqlite:///attestation_store.db` | SQLAlchemy connection string for the metadata store (supports PostgreSQL via standard URI). | | `PORT` | `5003` | Port used by the built-in server and to build default download URLs when running locally. | | `STORAGE_TYPE` | `FILE_MOUNT` | Selects evidence storage backend: `FILE_MOUNT` for local disk or `CLOUD_STORAGE` for S3/MinIO. | | `FILE_STORAGE_PATH` | `./storage` | Root directory for evidence files when `STORAGE_TYPE=FILE_MOUNT`; created automatically if missing. | | `FILE_STORAGE_BASE_URL` | `http://localhost:/download` | Public base URL clients use to reach files stored on disk (override when running behind a reverse proxy). | | `CLOUD_STORAGE_PROVIDER` | `AWS` | Cloud storage provider when `STORAGE_TYPE=CLOUD_STORAGE`; valid values are `AWS` and `MINIO`. | | `AWS_ACCESS_KEY_ID` | _none_ | AWS access key for S3 uploads when `CLOUD_STORAGE_PROVIDER=AWS`. | | `AWS_SECRET_ACCESS_KEY` | _none_ | AWS secret key for S3 uploads when `CLOUD_STORAGE_PROVIDER=AWS`. | | `AWS_SESSION_TOKEN` | _none_ | Optional AWS session token for temporary credentials. | | `AWS_DEFAULT_REGION` | `il-central-1` | AWS region used for S3 requests. | | `S3_BUCKET` | _none_ | Target S3 bucket name when using AWS storage. | | `PUBLIC_BASE_URL` | `http://localhost:` | Public HTTPS base URL used to calculate SAML ACS/SLO endpoints. | | `SAML_SP_CERTIFICATE` | _none_ | Optional PEM certificate for signing SAML Authn/SLO requests. | | `SAML_SP_PRIVATE_KEY` | _none_ | Private key paired with `SAML_SP_CERTIFICATE`. | | `SAML_DEFAULT_NAMEID_FORMAT` | `urn:oasis:names:tc:SAML:1.1:nameid-format:emailAddress` | Overrides the NameID format published in SP metadata. | ## SAML / Enterprise SSO Multi-tenant SAML 2.0 support is available out of the box. Use the `/api/saml/*` admin endpoints to create tenants and upload IdP metadata, then direct users to `/login` where tenant-specific SSO buttons appear automatically. Detailed setup instructions (including a Google Workspace walkthrough) live in [`docs/SAML_SETUP.md`](docs/SAML_SETUP.md). ## AWS S3 Storage Setup When using AWS S3 for storage (`STORAGE_TYPE=CLOUD_STORAGE` and `CLOUD_STORAGE_PROVIDER=AWS`), you need to create and configure an S3 bucket. ### Creating an S3 Bucket 1. **Create the bucket:** ```bash aws s3 mb s3://your-bucket-name --region your-region ``` 2. **Block public access** (recommended for security): - In AWS Console: S3 → your bucket → Permissions → Block public access → Enable all settings - Or via CLI: ```bash aws s3api put-public-access-block \ --bucket your-bucket-name \ --public-access-block-configuration \ "BlockPublicAcls=true,IgnorePublicAcls=true,BlockPublicPolicy=true,RestrictPublicBuckets=true" ``` 3. **Configure CORS** (required for web UI uploads): The web UI uploads files directly from the browser to S3 using presigned URLs. Browsers enforce CORS (Cross-Origin Resource Sharing) and will block these uploads unless the bucket has proper CORS configuration. Create a file named `cors.json`: ```json { "CORSRules": [ { "AllowedHeaders": ["*"], "AllowedMethods": ["GET", "PUT", "POST"], "AllowedOrigins": ["*"], "ExposeHeaders": ["ETag"], "MaxAgeSeconds": 3000 } ] } ``` Apply the CORS configuration: ```bash aws s3api put-bucket-cors --bucket your-bucket-name --cors-configuration file://cors.json ``` **Note:** CORS is only needed for browser-based uploads. CLI tools (like valint) don't enforce CORS and will work without this configuration. However, if you want the web UI to work, CORS configuration is mandatory. 4. **Create IAM credentials:** Create an IAM user with S3 access or use existing credentials. The minimum required permissions are: ```json { "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Action": [ "s3:PutObject", "s3:GetObject", "s3:DeleteObject" ], "Resource": "arn:aws:s3:::your-bucket-name/*" } ] } ``` ### Environment Variables Set the following environment variables in your `.env` file: ```bash STORAGE_TYPE=CLOUD_STORAGE CLOUD_STORAGE_PROVIDER=AWS AWS_ACCESS_KEY_ID=AKIA... AWS_SECRET_ACCESS_KEY=... AWS_DEFAULT_REGION=us-east-1 # or your preferred region S3_BUCKET=your-bucket-name ``` ### Verifying the Setup After configuration, you can verify that files are being stored in S3: ```bash # List all objects in the bucket aws s3 ls s3://your-bucket-name/attestations/ --recursive --region your-region # Check a specific attestation aws s3 ls s3://your-bucket-name/attestations/1/ --region your-region ``` ### Troubleshooting **Web UI uploads fail with "Network Error":** - Verify CORS configuration is applied to the bucket - Check that you're using the correct AWS region - Ensure the bucket name matches the `S3_BUCKET` environment variable **Files upload but don't appear in AWS Console:** - Verify you're viewing the correct region in the AWS Console - Try refreshing the console or navigating away and back - List objects via AWS CLI to confirm they exist - Check IAM permissions for the console user **Valint uploads fail with signature errors:** - Ensure `AWS_ACCESS_KEY_ID` and `AWS_SECRET_ACCESS_KEY` are correct - Verify the bucket exists in the specified region - Check that the IAM credentials have the required permissions ### Helm Chart Deployment with S3 To deploy the Attestation Store using Helm with S3 storage and SQLite database, first set your AWS credentials as environment variables: ```bash export AWS_ACCESS_KEY_ID=AKIA... export AWS_SECRET_ACCESS_KEY=... export AWS_DEFAULT_REGION=us-east-1 export S3_BUCKET=your-bucket-name ``` Then install with: ```bash helm install attstore ./helm/attstore \ --set storage.type=CLOUD_STORAGE \ --set storage.cloudStorage.provider=AWS \ --set storage.cloudStorage.aws.accessKeyId=$AWS_ACCESS_KEY_ID \ --set storage.cloudStorage.aws.secretAccessKey=$AWS_SECRET_ACCESS_KEY \ --set storage.cloudStorage.aws.region=$AWS_DEFAULT_REGION \ --set storage.cloudStorage.aws.bucket=$S3_BUCKET \ --set database.type=sqlite ``` **To use a specific image tag (e.g., for testing a branch):** ```bash helm install attstore ./helm/attstore \ --set image.tag=latest \ --set storage.type=CLOUD_STORAGE \ --set storage.cloudStorage.provider=AWS \ --set storage.cloudStorage.aws.accessKeyId=$AWS_ACCESS_KEY_ID \ --set storage.cloudStorage.aws.secretAccessKey=$AWS_SECRET_ACCESS_KEY \ --set storage.cloudStorage.aws.region=$AWS_DEFAULT_REGION \ --set storage.cloudStorage.aws.bucket=$S3_BUCKET \ --set database.type=sqlite ``` Or create a `values-s3.yaml` file: ```yaml storage: type: CLOUD_STORAGE cloudStorage: provider: AWS aws: accessKeyId: "" # Will be read from AWS_ACCESS_KEY_ID env var if empty secretAccessKey: "" # Will be read from AWS_SECRET_ACCESS_KEY env var if empty region: "us-east-1" bucket: "your-bucket-name" database: type: sqlite ``` Then install with: ```bash helm install attstore ./helm/attstore -f values-s3.yaml ``` **Note:** The default configuration uses SQLite, which stores the database in a persistent volume. For production deployments, consider using PostgreSQL instead by setting `database.type=postgresql` and `database.postgresql.enabled=true`. | `MINIO_ENDPOINT` | _none_ | MinIO server endpoint (e.g., `http://minio:9000`) when `CLOUD_STORAGE_PROVIDER=MINIO`. | | `MINIO_ACCESS_KEY` | _none_ | MinIO access key when `CLOUD_STORAGE_PROVIDER=MINIO`. | | `MINIO_SECRET_KEY` | _none_ | MinIO secret key when `CLOUD_STORAGE_PROVIDER=MINIO`. | | `MINIO_BUCKET` | _none_ | MinIO bucket used for storing evidence objects. | | `MINIO_SECURE` | `true` | Set to `false` to disable TLS when connecting to MinIO. | | `PRESIGNED_URL_EXPIRATION` | `3600` | Lifetime (seconds) of generated upload/download presigned URLs. | | `SYSLOG_FACILITY` | `16` | RFC5424 facility value recorded for syslog events generated by the service. | | `SYSLOG_HOSTNAME` | `attestation-store` | Hostname field used when formatting syslog messages. | | `SYSLOG_APP_NAME` | `attestation-store` | App name field used when formatting syslog messages. | | `LOG_RETENTION_DAYS` | `10` | Number of days to retain system logs; logs older than this are automatically deleted. Set to `0` to disable cleanup. See [Log Retention](docs/LOG_RETENTION.md) for details. | | `DEFAULT_ADMIN_USERNAME` | `admin` | Username for the bootstrapped administrator account. | | `DEFAULT_ADMIN_PASSWORD` | `admin#admin` | Password for the bootstrapped administrator account; update immediately in production. | | `DEFAULT_ADMIN_EMAIL` | `admin@example.com` | Email associated with the bootstrapped administrator account. | ## Docker Containers To run the Attestation Store using Docker, you can use the provided `docker-compose.yml` file. This setup includes the application, a PostgreSQL database, and a Minio object storage service. Run: ```bash docker compose -f 'docker-compose.yml' up -d ``` The application will be accessible at `http://localhost:5003`. The default admin user is `admin` with password `admin#admin`. You should change this password immediately after logging in. ### Documentation The Attestation Store includes integrated Scribe Security documentation built from the source repository at [https://github.com/scribe-security/docs](https://github.com/scribe-security/docs). **Documentation Integration Details:** - Documentation is built from source (Docusaurus) during Docker builds and via a local script, ensuring all internal links work at `/docs`. - For airgapped environments, docs can be pre-built and transferred. - Troubleshooting and maintenance tips are included. See [docs/DOCUMENTATION_INTEGRATION.md](docs/DOCUMENTATION_INTEGRATION.md) for a full implementation guide and troubleshooting tips. **For Local Development:** Requirements: Node.js 18+ and Git ```bash # Build documentation make docs # or bash scripts/download-docs.sh # Start development server (automatically builds docs if missing) make dev ``` The documentation will be available at `http://localhost:5003/docs` when running the application. **For Production/Docker:** Documentation is automatically built from source during the Docker build process using a multi-stage build - no additional steps required. All internal links are configured to work correctly at the `/docs` path. **For Airgapped Environments:** Pre-build the documentation before transferring to airgapped environment: ```bash bash scripts/download-docs.sh # Transfer the entire project including static/docs/ to airgapped environment docker build -t attstore:latest . ``` ### Scaling attstore with Docker It is possible to run multiple instances of the attestation store application behind a load balancer. To do this, simply run ```bash docker compose scale attstore=3 ``` or add `--scale` flag to the `docker compose up` command: ```bash docker compose -f 'docker-compose.yml' up -d --scale attstore=3 ``` Load balancing is performed by `traefik` as defined in the `docker-compose.yml` file. You can access the application at `http://localhost:5003` no matter how many instances are running. ### Running locally with a single image You can run Attestation Store as a single Docker container without using docker-compose. This approach is simpler but doesn't include PostgreSQL database or MinIO object storage - it will use SQLite and local file storage instead. #### Authentication to GitHub Container Registry Before pulling the image, you need to authenticate to the GitHub Container Registry: ```bash # Login to GitHub Container Registry # When prompted, use your GitHub username and a personal access token with read:packages scope docker login ghcr.io ``` If you don't have a GitHub personal access token, you can create one at GitHub Settings → Developer settings → Personal access tokens → Tokens (classic). Make sure to give it at least the `read:packages` scope. #### Running the container After authentication, run the container: **Default (Gunicorn - Production):** ```bash docker run -it \ --platform linux/amd64 \ --name attstore \ -p 5003:5003 \ -v "$(pwd)/storage:/app/storage" \ -v "$(pwd)/instance:/app/instance" \ -e STORAGE_TYPE=FILE_MOUNT \ ghcr.io/scribe-security/attstore:latest ``` By default, the container runs with Gunicorn (4 workers, 120s timeout). This is recommended for production use. **Flask Development Server:** To use the Flask development server instead: ```bash docker run -it \ --platform linux/amd64 \ --name attstore \ -p 5003:5003 \ -v "$(pwd)/storage:/app/storage" \ -v "$(pwd)/instance:/app/instance" \ -e STORAGE_TYPE=FILE_MOUNT \ ghcr.io/scribe-security/attstore:latest \ python main.py ``` **Flask Development Server with Debug Mode:** ```bash docker run -it \ --platform linux/amd64 \ --name attstore \ -p 5003:5003 \ -v "$(pwd)/storage:/app/storage" \ -v "$(pwd)/instance:/app/instance" \ -e STORAGE_TYPE=FILE_MOUNT \ -e FLASK_DEBUG=true \ ghcr.io/scribe-security/attstore:latest \ python main.py ``` **Gunicorn with Custom Parameters:** ```bash docker run -it \ --platform linux/amd64 \ --name attstore \ -p 5003:5003 \ -v "$(pwd)/storage:/app/storage" \ -v "$(pwd)/instance:/app/instance" \ -e STORAGE_TYPE=FILE_MOUNT \ ghcr.io/scribe-security/attstore:latest \ gunicorn -c gunicorn.conf.py --workers 8 --timeout 300 --log-level debug gunicorn_app:app ``` > **Note**: We're using the `:latest` tag instead of `:main` as it's more commonly available. This command: - Maps port 5003 to access the application - Creates persistent volumes for your data: - `./storage`: Stores the attestation files - `./instance`: Stores the SQLite database - Uses the same image from the docker-compose file but with the `latest` tag - Configures the app to use local file storage instead of MinIO The application will be available at http://localhost:5003 with the default admin credentials (username: `admin`, password: `admin#admin`). ### Running locally with SQLite #### Production Mode (Gunicorn) By default, the application uses Gunicorn as the WSGI server for better performance and stability: ```bash gunicorn -c gunicorn.conf.py gunicorn_app:app ``` The default configuration (`gunicorn.conf.py`): - Uses 4 worker processes for handling concurrent requests - Sets a 120-second timeout for long-running attestation operations - Logs all access and errors to stdout - Runs migrations and setup automatically in the master process before forking workers **To customize Gunicorn parameters:** ```bash # Override workers via command line gunicorn -c gunicorn.conf.py --workers 8 gunicorn_app:app # Override timeout gunicorn -c gunicorn.conf.py --timeout 300 gunicorn_app:app # Enable detailed logging gunicorn -c gunicorn.conf.py --log-level debug gunicorn_app:app # Or run without config file (not recommended - migrations run in each worker) gunicorn --bind 0.0.0.0:5003 --workers 4 --timeout 120 gunicorn_app:app ``` #### Development Mode (Flask) For development with auto-reload and better debugging, use the Flask development server: ```bash python main.py ``` Then create a token through the UX. Optionally create users and tokens for them. #### Debug Mode By default, the Flask development server runs with debug mode disabled. To enable debug mode: ```bash export FLASK_DEBUG=true python main.py ``` Or run it inline: ```bash FLASK_DEBUG=true python main.py ``` Debug mode accepts the following values (case-insensitive): - `true`, `1`, `yes` - Enable debug mode - `false`, `0`, `no` or any other value - Disable debug mode (default) **Note:** The `FLASK_DEBUG` variable only affects `main.py` (Flask development server). When using Gunicorn, debug mode is controlled by `--log-level debug`. ## Usage examples Before running following examples, generate a token through the UI on the `Settings -> API Keys` page and set it as `TOKEN` environment variable. ### Basic Evidence Collection ```bash valint bom file:README.md -u http://127.0.0.1:5003 -P $TOKEN -vv -n GoodProduct -V 0.6 ``` ### Policy Verification with Git Bundle Attestation Store includes a built-in Git HTTP server that serves the [scribe-public/sample-policies](https://github.com/scribe-public/sample-policies) repository. This allows valint to fetch policy bundles directly from the attestation store without requiring external internet access. **Available endpoint:** `http://localhost:5003/git/sample-policies.git` **Features:** - Smart HTTP protocol support (supports shallow clones with `--depth 1`) - All version tags available (e.g., `2.0.7`, `2.0.8`, `v1.4.0`) - Special `latest` tag always points to the most recent version - Read-only access (no authentication required) - Secure against directory traversal and path injection attacks **Example with specific version:** ```bash valint verify file:README.md -n GoodProduct -V 0.6 \ -u http://127.0.0.1:5003 -P $TOKEN -vv \ --bundle http://localhost:5003/git/sample-policies.git \ --initiative slsa.l1@v2 \ --bundle-tag 2.0.7 ``` **Example output:** ``` ... [2025-11-26 23:00:59] DEBUG git: cloning tmp dir: /var/folders/h9/75hf9vcx6995j5csf3rdjzvr0000gn/T/git_tmp_243415826 [2025-11-26 23:00:59] INFO git: stat default-branch: 14b5ace03bbdf729f8351554b884c87eb1fb9c71 refs/heads/main [2025-11-26 23:00:59] INFO git: cloning remote repository: http://localhost:5003/git/sample-policies.git @ refs/tags/2.0.7, Depth: 1 Enumerating objects: 2105, done. Counting objects: 100% (2105/2105), done. Compressing objects: 100% (2055/2055), done. Total 2105 (delta 1478), reused 0 (delta 0), pack-reused 0 (from 0) [2025-11-26 23:00:59] INFO git: cloning success: http://localhost:5003/git/sample-policies.git @ refs/tags/2.0.7, Depth: 1 ... ``` **Example with latest version:** ```bash valint verify file:README.md -n GoodProduct -V 0.6 \ -u http://127.0.0.1:5003 -P $TOKEN -vv \ --bundle http://localhost:5003/git/sample-policies.git \ --initiative slsa.l1@v2 \ --bundle-tag latest ``` **Manual Git clone (for testing):** ```bash # Clone specific version git clone --depth 1 --branch 2.0.7 http://localhost:5003/git/sample-policies.git # Clone latest version git clone --depth 1 --branch latest http://localhost:5003/git/sample-policies.git # Clone full repository git clone http://localhost:5003/git/sample-policies.git ``` ## API Endpoints ### Evidence and Attestation Management | Endpoint | Method | Auth | Description | |----------|--------|------|-------------| | `/v1/login` | POST | None | Authenticate with API key, username/password, or client credentials. Returns JWT. | | `/v1/logout` | POST | JWT | Logout current user. | | `/v1/profile` | GET | JWT | Get current user profile. | | `/v1/users` | GET | JWT | List users (admin only). | | `/v1/users` | POST | JWT | Create new user (admin only). | | `/v1/users/bulk` | GET | JWT | Get all users for bulk creation (admin only). | | `/v1/users/bulk` | POST | JWT | Create users in batch (admin only). | | `/v1/generate-api-key` | POST | JWT | Generate API key for a user (admin only). | | `/v1/api-key` | POST | JWT | Create API key for current user. | | `/v1/api-keys` | GET | JWT | List API keys (admin sees all, user sees own). | | `/products/download-json` | GET | JWT | Download products as JSON, supports filtering. | | `/products/download-csv` | GET | JWT | Download products as CSV, supports filtering. | | `/products` | GET | JWT | Products page (HTML, for web UI). | | `/attestations` | GET | JWT | List attestations. | | `/attestations//file` | GET | JWT | Download attestation file. | | `/ux/policy-results` | GET | JWT | List policy results (web UI, supports filtering). | | `/ux/policy-results/` | GET | JWT | View a single policy result (web UI). | | `/ux/policy-results/download-csv` | GET | JWT | Download policy results as CSV, supports filtering. | | `/evidence/policy-results/csv` | GET | JWT | Download policy results as CSV (API, supports filtering). | | `/status` | GET | None | Health/status check. | | `/api/webhooks/test-receiver` | POST | X-Webhook-Secret | Webhook test endpoint (external, no JWT). | | `/api/webhooks/test/` | POST | JWT | Internal webhook test endpoint. | ### Git Policy Repository Serving | Endpoint | Method | Auth | Description | |----------|--------|------|-------------| | `/git/sample-policies.git/info/refs` | GET | None | Git smart HTTP protocol - repository advertisement. | | `/git/sample-policies.git/git-upload-pack` | POST | None | Git smart HTTP protocol - fetch/clone operations. | | `/git/sample-policies.git/` | GET | None | Static git repository files (objects, refs, HEAD, etc.). | The Git endpoints serve the [scribe-public/sample-policies](https://github.com/scribe-public/sample-policies) repository via smart HTTP protocol, supporting shallow clones (`--depth`) and all version tags. The `latest` tag automatically points to the most recent version. > **Note:** Most endpoints require JWT authentication, which can be obtained via `/v1/login` using an API key or credentials. Unless noted, endpoints exchange JSON and expect a `Content-Type: application/json` request header. Authenticated calls require a JWT access token obtained from `POST /v1/login` (tokens expire after one hour). Include it in the `Authorization` header: ``` Authorization: Bearer ``` ### Authentication & User Management (`/v1`) #### POST /v1/login Authenticate using one of the supported credential types and receive a short-lived JWT. - **Auth**: none - **Request body**: supply exactly one of the following payloads: - API token / legacy API key: ```json {"api_token": ""} ``` - Client credentials (base64 or plain secret): ```json {"client_id": "ci", "client_secret": "encoded-secret"} ``` - Username and password: ```json {"username": "admin", "password": "admin#admin"} ``` - **Success response (200)**: JSON object with - `access_token` and `token`: the JWT (identical value). - `user`: user profile (`id`, `username`, `email`, status flags, timestamps). - `auth_method`: `"api_token"`, `"client_credentials"`, or `"password"`. - **Common errors**: 400 missing body, 401 invalid credentials, 404 unknown user, 500 server error. #### POST /v1/logout Invalidate the caller's session in the audit log. - **Auth**: JWT - **Response (200)**: ```json {"message": "Logged out successfully"} ``` #### GET /v1/profile Return the authenticated user's profile. - **Auth**: JWT - **Response (200)**: same shape as the `user` object from `/v1/login`. #### POST /v1/users Create a new user (admin only). - **Auth**: JWT for `admin` - **Request body fields**: - `username` (string, required) - `email` (string, required) - `password` (string, optional; default is generated and sets `password_change_required`). - **Response (201)**: user object. #### POST /v1/users/bulk Create multiple users in one request (admin only). See usage example in `scripts/create_batch_users.py`. - **Auth**: JWT for `admin` - **Request body**: JSON array of user objects, each containing `username`, `email`, optional `password`. - **Response (201)**: JSON object with - `success`: list of created users (`id`, `username`, `email`, `password_change_required`). - `failed`: list of entries that were skipped with `data` echo and `error` message. #### GET /v1/users/bulk Download users in a CSV-friendly JSON layout (admin only). See usage example in `scripts/get_users.py`. - **Auth**: JWT for `admin` - **Query parameters**: `include_inactive=true` to include disabled accounts. - **Response (200)**: ```json { "users": [ { "username": "user1", "email": "user1@example.com", "password": "", "require_password_change": false, "is_active": true, "is_admin": false } ] } ``` #### POST /v1/generate-api-key Generate an API key by authenticating with admin credentials in the request body (does not require JWT). Returns a base64 client credential that can later be used with `/v1/login`. - **Auth**: none (body must contain valid admin credentials) - **Request body fields**: - `username` and `password` (admin's credentials, required) - `target_user_id` (int, required) - `name` (string, optional label) - `expiration_days` (int, optional; 0 or `null` means no expiration; default 365) - **Response (200)**: ```json { "api_key": "", "name": "build-bot", "user_id": 5, "expiration_date": "2025-05-12T12:00:00", "expires_in_days": 365 } ``` #### POST /v1/api-key Generate an API key using an existing JWT. Admins can create keys for other users. - **Auth**: JWT - **Request body fields**: - `name` (string, optional) - `user_id` (int, optional; defaults to current user, requires admin to set) - `expiration_days` (int, optional; 0 or `null` for no expiration) - **Response (201)**: same format as `/v1/generate-api-key`. #### GET /v1/users List users (admin only). - **Auth**: JWT for `admin` - **Response (200)**: ```json {"users": [, ...]} ``` #### GET /v1/api-keys List API keys. Admins see all keys; other users see their own. - **Auth**: JWT - **Response (200)**: ```json {"api_keys": [, ...]} ``` ### Evidence & Attestation Workflow (`/evidence`) Uploading evidence is a three-step process: 1. `POST /evidence` to register metadata and obtain a `file_id` plus upload URL. 2. Upload the artifact either to the returned `presigned_url` or via `PUT /evidence/upload/` when using local filesystem storage. 3. `POST /evidence/finish` to mark the attestation complete (and optionally report upload errors). All evidence endpoints require a JWT unless noted. #### POST /evidence Register evidence metadata and create an upload session. - **Auth**: JWT - **Request body fields**: - `key` (string, required): unique identifier supplied by the client. - `content_type` (string, required): MIME type for the artifact. - `context_type` (string, required). - `context` (object, optional) influencing deduplication; common keys include `product_key`, `product_version`, `labels`, `target_type`, `input_name`, `input_version`, `tool`, `signed`, `team_id`, `job_ids`, `pipeline_run`. - **Response (200)**: ```json { "file_id": 123, "presigned_url": "https://storage/upload/...", "message": "Existing attestation will be replaced" } ``` The `message` key is only present when an older attestation is being superseded. #### PUT /evidence/upload/ Upload the artifact bytes to local storage. When `FILE_MOUNT` storage is enabled, this endpoint replaces the presigned URL returned by `POST /evidence`. - **Auth**: none (validated via the upload session created earlier) - **Request body**: raw file contents. Set `Content-Type` to match `content_type`. - **Response (200)**: empty body. Errors include 404 (session not found) and 400 (session not pending). #### POST /evidence/finish Finalize an upload or report a failure. - **Auth**: JWT - **Request body fields**: - `file_id` (int, required) - `error` (string, optional). When provided, the attestation is soft-deleted and the upload session is marked as failed. - **Response (200)**: ```json {"delete_candidates": []} ``` #### POST /evidence/list Return attestations owned by the caller (completed uploads only). - **Auth**: JWT - **Request body**: optional filters keyed by metadata such as `product_key`, `product_version`, `target_name`, `target_type`, `content_type`, `context_type`, `tool`, or `labels` (array). Unknown keys are matched against attestation context using exact comparison. - **Response (200)**: ```json { "evidences": [ { "id": 123, "key": "cyclonedx-123", "targetname": "service-a", "targettype": "policy-results", "contenttype": "application/json", "context": {...}, "sigstatus": "signed", "deliverable": false, "userid": 5, "timestamp": {"Time": "2024-05-12T20:17:04.805938", "Valid": true}, "deleted": false, ... } ], "all_processed": true } ``` #### GET /evidence/ Fetch a single attestation plus a fresh download URL. - **Auth**: JWT - **Response (200)**: attestation object (`to_dict()` fields) with an additional `presigned_url`. #### DELETE /evidence/ Delete an attestation owned by the caller. The backing file is removed and related policy results are marked deleted. - **Auth**: JWT - **Response (204)**: empty body. #### GET /evidence/download/ Download an artifact by its storage path. - **Auth**: none - **Response**: - For remote storage backends, returns status `302` with body `{"download_url": ""}`. - For local storage, streams the file directly (content type inferred from extension). Returns 404 if the file does not exist. #### GET /evidence/policy-results/csv Export policy results as CSV. Mirrors the filters exposed in the UI. `scripts/test_policy_csv.py` demonstrates usage. - **Auth**: JWT (admin may add `show_all_users=true`) - **Query parameters**: - `rule_id`, `product`, `result_kind`, `result_level` (optional filters) - `show_all_users=true` (admin only) - `filename` (custom file name) - **Response (200)**: `text/csv` attachment. #### GET /evidence/team Placeholder endpoint reporting that teams are not supported on the on-prem service. - **Auth**: JWT - **Response (200)**: ```json {"team_id": 1, "team_name": "Teams not supported on on-prem attestation service"} ``` ### Product Data Exports (`/products`) These endpoints live under the web blueprint but provide machine-consumable exports. #### GET /products/download-json Return product metadata, badge counts, and summary statistics. Demonstrated in `scripts/test_download_products_json.py`. - **Auth**: JWT - **Query parameters**: `filter` (optional substring match on product key). - **Response (200)**: ```json { "products": [ { "product": "GoodProduct", "version": "0.1", "attestation_count": 3, "policy_result_count": 12, "badges": [ {"name": "delivery-ready", "count": 1, "description": "", "positive": true} ], "product_metadata": {...}, "user_id": 5 } ] } ``` ### Syslog & Audit Logging #### POST /syslog Ingest RFC 5424-formatted syslog messages. JWT is optional; when provided the log entry is attributed to the user. - **Auth**: optional JWT - **Supported payloads**: - JSON: `{"syslog_message": ""}` with `Content-Type: application/json`. - Raw text body with `Content-Type: text/plain`. - **Response (200)**: JSON acknowledging how many messages were processed. Invalid lines produce structured errors while continuing to process remaining entries. #### GET /logs Retrieve stored log entries with pagination and filtering. Include a JWT to scope the results to your account (admin users can see all entries). - **Auth**: optional JWT - **Query parameters**: `page`, `per_page` (<=100), `severity`, `app_name`, `event_type`, `hostname`, `search`. - **Response (200)**: ```json { "logs": [ { "id": 10, "facility": 1, "severity": 6, "timestamp": "2024-05-12T20:16:00.123456", "app_name": "webhook-sender", "event_type": "api_key_created", "message": "...", "username": "admin" } ], "total": 42, "pages": 5, "current_page": 1, "per_page": 10 } ``` #### GET /logs/stats Aggregate log counts for dashboards. - **Auth**: optional JWT - **Response (200)**: ```json { "total_logs": 42, "severity_stats": [{"severity": 6, "count": 30}], "event_stats": [{"event_type": "login_success", "count": 12}], "app_stats": [{"app_name": "webhook-sender", "count": 9}] } ``` ### Utility Endpoints #### GET /healthz Lightweight health check used by orchestrators. No authentication required. - **Response (200)**: ```json { "status": "ok", "db": "ok", "jobs_running_or_queued": 0 } ``` Returns HTTP 503 with `{"db": "unavailable"}` when the database check fails. ## Webhook support The webhook system ("webhookd") allows you to send outgoing HTTP requests when specific events occur in the Attestation Store. You can configure webhooks via the UI or API. Creating & Managing Webhooks Go to Webhooks in the UI to add, edit, delete, or test webhooks. - Name: Unique name for the webhook. - Event Type: The event that triggers the webhook (e.g., attestation_finalized). - Product Key: (Optional) Restrict to a specific product. - Attestation Type: (Optional) Restrict to a specific attestation type. - URL: The endpoint to send the webhook to. - HTTP Method: POST, GET, PUT, or DELETE. - Headers: JSON object for custom headers (e.g., { "Authorization": "Bearer ..." }). Supports templating. - Content Template: JSON object for the request body. Supports templating. - Secret: (Optional) Used for authentication with the receiver. Can be injected into headers or sent as X-Webhook-Secret. - Active: Enable/disable the webhook. ![Webhook Diagram](docs/Webhook.png) Webhook headers and content support simple templating, for example: - Use --secret-- in headers to inject the webhook's secret value. - Use --attestation_details-- in the content template to inject the attestation data (as JSON). See other placeholders in the image above. Note: The Example above shows an integration with [dependency-track](https://docs.dependencytrack.org/). you can run it locally using docker-compose: ```bash # Downloads the latest Docker Compose file curl -LO https://dependencytrack.org/docker-compose.yml # Starts the stack using Docker Compose docker compose up -d ``` And replace the secret with a token generated in the dependency-track UI. Then upload an SBOM attestation using `valint`: ```bash valint bom postgres:15 --product-key postgres --product-version 15 -u http://localhost:5003 -P ``` ### Webhook Management (`/api/webhooks`) All webhook management endpoints require a JWT. Webhook payload templating supports `{secret}`, `{attestation_details}`, `{event}`, and `{webhook_id}` placeholders. #### POST /api/webhooks Create a webhook for the current user. - **Auth**: JWT - **Request body fields**: - `name` (string, required, unique per user) - `event_type` (string, required, e.g. `attestation_finalized`) - `product_key` (string or `"all"` to apply to every product) - `attestation_type` (string or `"all"`) - `url` (string, required) - `secret` (string, optional) - `is_active` (bool, default `true`) - `headers` (JSON string, optional) - `content` (JSON string/template, optional; defaults to `{"attestation": "{attestation_details}"}`) - `method` (HTTP verb, default `"POST"`) - **Response (201)**: ```json {"webhook": } ``` #### GET /api/webhooks List webhooks created by the caller. - **Auth**: JWT - **Response (200)**: ```json {"webhooks": [, ...]} ``` #### GET /api/webhooks//json Return a single webhook definition. - **Auth**: JWT - **Response (200)**: webhook object. #### PUT /api/webhooks/ Update a webhook definition. - **Auth**: JWT - **Request body**: same fields as creation (all optional). - **Response (200)**: ```json {"webhook": } ``` #### DELETE /api/webhooks/ Delete a webhook. - **Auth**: JWT - **Response (200)**: ```json {"result": "deleted"} ``` #### GET /api/webhooks/settings/download Export the caller's webhook definitions as JSON. - **Auth**: JWT - **Response (200)**: `application/json` attachment containing a `{name: definition}` mapping. #### POST /api/webhooks/settings/upload Import webhook settings from JSON. - **Auth**: JWT - **Request body**: - Provide JSON in the request body, or upload a `file` form field containing a JSON document. - Each entry must include `event_type`, `product_key`, `attestation_type`, `url`, `secret`, `is_active`, `headers`, `content`, and `method`. - **Response (200)**: ```json { "accepted": [{"name": "prod-finalized", "action": "updated"}], "rejected": [{"name": "bad-webhook", "reason": "Missing required fields"}] } ``` --- ## Webhook Test Receiver You can test webhooks by pointing them to the following endpoint of this service: ``` POST /api/webhooks/test-receiver ``` **Use this endpoint when testing from external services (e.g., Zapier, Postman, or your webhook provider).** It does NOT require JWT authentication, only the `X-Webhook-Secret` header. ### Required Webhook Configuration - **URL**: `https:///api/webhooks/test-receiver` - **Method**: `POST` - **Headers**: - `Content-Type: application/json` - `X-Webhook-Secret: "{secret}"` (This will replace `{secret}` with a token generated via the token interface of this app and you provide to the secred field of the webhook configuration in the app) - **Content**: (JSON body, can be any payload; will be logged) Example payload: ```json { "event": "webhook_test", "data": { "example": "value" } } ``` ### POST /api/webhooks/test-receiver Public endpoint that logs incoming webhook payloads. Supply the webhook secret in the headers. - **Auth**: header `X-Webhook-Secret: ` - **Request body**: JSON payload to log. - **Response (200)**: ```json {"result": "Webhook received and logged"} ``` ## Internal Webhook Test Endpoint The `/api/webhooks/test/` endpoint is for testing webhooks *from within the app* and requires JWT authentication (i.e., you must be logged in via the app). If you try to use this endpoint from an external service without a valid JWT, you will get an authentication error. ### POST /api/webhooks/test/ Send a sample payload to the specified webhook. - **Auth**: JWT - **Response (200)**: ```json {"status_code": 200, "response": "..."} ``` # Badges Badge Ownership Summary ## Badge Creation Logic: A badge is always owned by the user who created it (owner_id = current user's ID) Admin users can optionally set their badges as global (visible to all users) Regular users can only create personal badges (never global) Added explicit name uniqueness check to prevent duplicate badge names ## Badge List Logic: All users (including admins) only see their own badges plus global badges Removed any logic that gave admins special access to other users' badges ## Badge Edit Logic: Users can only edit badges they own (owner_id matches their user ID) Even admins cannot edit badges created by other users Admins can edit the global flag for their own badges Added name uniqueness check to prevent duplicate badge names during edits ## Badge Delete Logic: Users can only delete badges they own Even admins cannot delete badges created by other users ## Badge JSON API: Users can only access JSON data for their own badges and global badges Simplified the JSON response to match the new ownership model ## Badge Settings Export/Import: Export only includes the user's own badges and global badges Import only allows updating the user's own badges Import rejects attempts to update badges owned by other users Import enforces name uniqueness for new badges