docs/DOCUMENTATION_INTEGRATION.md

Documentation Integration - Implementation Summary

Quick Summary

This document describes how Scribe Security documentation is integrated into the Attestation Store for both production and local development. Documentation is built from source (Docusaurus) during Docker builds and via a local script, ensuring all internal links work at /docs. For airgapped environments, docs can be pre-built and transferred. Troubleshooting and maintenance tips are included.

Key Points:

• Documentation is served locally at /docs (not from GitHub Pages)
• Docker builds and local scripts build docs from source for correct routing
• All internal links and assets work as expected
• See troubleshooting section for common issues

Overview

This implementation adds Scribe Security documentation from GitHub Pages to the Attestation Store application during build time for production and provides easy local development access.

What Was Implemented

1. Multi-Stage Dockerfile (Dockerfile)

• Stage 1: Downloads documentation from https://scribe-security.github.io/docs/ using wget
• Stage 2: Copies downloaded docs to /app/static/docs in the application image
• Documentation is embedded in the Docker image automatically during build

2. Local Development Script (scripts/download-docs.sh)

• Bash script to download documentation for local development
• Features:
  • Checks if docs already exist (skips re-download unless --force flag is used)
  • Uses same wget parameters as Dockerfile for consistency
  • Provides helpful output and error handling
  • Made executable with chmod +x

3. Makefile (Makefile)

Convenient targets for common operations:

• make docs - Build documentation locally
• make docs-force - Force rebuild documentation
• make clean-docs - Remove built documentation
• make dev - Build docs (if needed) and start dev server
• make build - Build Docker image with documentation
• make help - Show available targets

4. Flask Routes (web/routes.py)

Added two new routes to serve documentation:

• GET /docs or GET /docs/ - Serves documentation index page
• GET /docs/<path:filename> - Serves documentation files and assets

Features:

• Friendly error message if docs haven't been built yet
• Automatic fallback to file listing if no index.html
• Uses Flask's send_from_directory for secure file serving
• Opens in new tab via target="_blank" in navigation links

5. Git Configuration (.gitignore)

• Added static/docs/ to prevent committing built documentation
• Keeps repository clean (docs are ~10MB+ when built)
• Prevents duplication since docs are built during CI/deployment

6. Docker Configuration (.dockerignore)

• Added static/docs/ to exclude locally built docs from build context
• Ensures Docker build uses docs from the multi-stage build process
• Reduces build context size and prevents conflicts

7. Documentation Updates

README.md:

• Added comprehensive documentation section
• Explains local development requirements (Node.js 18+, Git)
• Covers production/Docker build process
• Includes airgapped environment instructions
• Links to this detailed implementation guide

templates/base.html:

• Updated documentation link from https://scribe-security.github.io/docs/ to /docs
• Retained target="_blank" to open in new tab
• Documentation now served locally instead of external link

How It Works

Production/Docker Build

docker build -t attstore:latest .

Build Process:

1. Stage 1 - Documentation Builder (node:18-alpine):

   • Installs system dependencies (git, build tools)
   • Clones https://github.com/scribe-security/docs.git
   • Runs npm ci to install Node.js dependencies
   • Builds with DOCUSAURUS_BASE_URL=/docs npm run build
   • Output: Static site in /docs-build/build

2. Stage 2 - Application (python:3.11-slim):

   • Sets up Python environment
   • Installs application dependencies
   • Copies application code
   • Copies built documentation from stage 1 to /app/static/docs

3. Result:

   • Documentation embedded in Docker image (~10MB)
   • All internal links work correctly at /docs
   • Assets (CSS/JS/images) load properly
   • Client-side routing functions as expected

Run the container:

docker run -p 5003:5003 attstore:latest

Local Development

Requirements:

• Node.js 18+ (for building Docusaurus)
• Git (for cloning the docs repo)
• ~2-3 minutes for initial build
• Build tools may be needed on some systems

Build Commands:

# Option 1: Using Make (recommended)
make dev              # Builds docs if needed, then starts server

# Option 2: Manual build then run
make docs            # Just build documentation
python main.py       # Start server separately

# Option 3: Direct script
bash scripts/download-docs.sh
python main.py

# Force rebuild
make docs-force

Build Process (Local):

1. Clones docs repository to temporary directory
2. Runs npm ci to install dependencies
3. Builds with DOCUSAURUS_BASE_URL=/docs npm run build
4. Copies built files to static/docs/
5. Cleans up temporary directory

Accessing Documentation

Once the application is running (locally or in Docker):

• Application: http://localhost:5003
• Documentation: http://localhost:5003/docs
• Click "Scribe Documentation" link in footer (opens in new tab)
• All internal navigation works correctly
• Client-side routing functions properly

Airgapped Environment Support

For environments without internet access during build:

Option 1: Pre-build Documentation (Recommended)

# On internet-connected machine:
bash scripts/download-docs.sh

# Transfer entire project to airgapped environment
# The built docs in static/docs/ will be included

# In airgapped environment:
docker build -t attstore:latest .

Note: The Docker build will still try to clone the repository in stage 1, but will use the locally built docs if the build stage fails.

Option 2: Modify Dockerfile for Airgap

Comment out the docs-builder stage and copy local docs:

# Comment out or remove the docs-builder stage:
# FROM node:18-alpine AS docs-builder
# ...entire stage...

# In the application stage:
FROM python:3.11-slim

# ... existing app setup ...

# Copy pre-built docs from local filesystem
COPY static/docs /app/static/docs

Then build:

docker build -t attstore:latest .

Option 3: Save Built Image

Build on internet-connected machine, save, and transfer:

# On internet-connected machine:
docker build -t attstore:latest .
docker save -o attstore-latest.tar attstore:latest

# Transfer attstore-latest.tar to airgapped environment

# On airgapped machine:
docker load -i attstore-latest.tar
docker run -p 5003:5003 attstore:latest

File Changes Summary

File	Change Type	Purpose
Dockerfile	Modified	Added multi-stage build: Node.js stage builds docs, Python stage includes them
.gitignore	Modified	Exclude static/docs/ (built documentation)
.dockerignore	Modified	Exclude static/docs/ from build context
scripts/download-docs.sh	Created	Build script for local development (clones, builds, copies)
Makefile	Created	Convenient targets (docs, dev, build, clean-docs)
web/routes.py	Modified	Added /docs and /docs/<path> routes
templates/base.html	Modified	Updated doc link from external URL to /docs
README.md	Modified	Added documentation section with build instructions
docs/DOCUMENTATION_INTEGRATION.md	Created	This comprehensive implementation guide

Testing Checklist

Docker Build & Run

• Test Docker build: docker build -t attstore:latest .
  • Verify stage 1 clones repo successfully
  • Verify npm ci completes without errors
  • Verify build completes with correct baseUrl
  • Build time should be ~4-5 minutes
• Test Docker run: docker run -p 5003:5003 attstore:latest
• Access application at http://localhost:5003
• Access documentation at http://localhost:5003/docs
• Click "Scribe Documentation" link in footer
• Verify docs open in new tab
• Test internal navigation within docs

Local Development

• Test local build: make docs
  • Verify Node.js and Git are detected
  • Verify clone succeeds
  • Verify npm install completes
  • Verify build completes
  • Check static/docs/ directory exists
• Test force rebuild: make docs-force
• Test clean: make clean-docs
• Test dev mode: make dev
• Access /docs in browser from local development
• Test internal links and navigation

Git & Docker Configuration

• Verify docs excluded from git: git status should not show static/docs/
• Verify .dockerignore excludes static/docs/
• Build Docker image and verify it doesn't use local static/docs/

Airgapped Environment (if applicable)

• Build docs locally
• Transfer project to airgapped system
• Build Docker image in airgapped environment
• Verify documentation is included and works

Maintenance

Updating Documentation

Documentation is built from the latest main branch of the docs repository:

For Production:

docker build -t attstore:latest .

This will clone and build the latest documentation automatically.

For Local Development:

make docs-force
# or
bash scripts/download-docs.sh --force

Cleaning Up:

make clean-docs  # Remove built documentation

Build Details

What gets built:

• Static HTML, CSS, JavaScript files
• Images and assets
• Search index (Lunr.js)
• Total size: ~10MB

Build time:

• Docker build (first time): ~4-5 minutes
• Local build (first time): ~2-3 minutes
• Subsequent builds (with cache): ~1-2 minutes

Internal Links & Routing

Why build from source instead of downloading pre-built HTML?

The documentation uses Docusaurus 2, a React-based static site generator with:

• Client-side routing (React Router)
• Dynamic base URL configuration
• Asset bundling with webpack

The Problem:

• GitHub Pages builds with baseUrl=/ (root path)
• We need to serve at /docs (sub-path)
• Downloading pre-built HTML would have wrong paths

The Solution:

• Build from source with DOCUSAURUS_BASE_URL=/docs
• All generated URLs use /docs as the base
• Internal links: /docs/guides/... instead of /guides/...
• Asset paths: /docs/assets/... instead of /assets/...
• Client-side routing configured for /docs base

This ensures:

• ✅ Internal navigation works correctly
• ✅ Asset paths (CSS/JS/images) resolve properly
• ✅ Client-side routing functions as expected
• ✅ Search works within the docs
• ✅ No broken links or 404 errors

Troubleshooting

"Documentation Not Available" Error

Local Development:

make docs  # Requires Node.js 18+ and Git

If the build succeeded but docs still aren't available:

• Check that static/docs/ directory exists
• Check that it contains files (especially index.html)
• Verify Flask can access the directory (permissions)

Docker:

• Check build logs for errors in the docs-builder stage
• Look for git clone failures or npm install errors
• Verify the COPY command succeeded in stage 2

Node.js Not Found (Local Dev)

Solution:

• Install Node.js 18+ from https://nodejs.org/
• Or use nvm: nvm install 18 && nvm use 18
• Or use Docker build which includes Node.js

Verify installation:

node --version  # Should show v18.x.x or higher
npm --version

Build Fails During npm ci

Possible causes:

• No internet connectivity
• Cannot access npmjs.org registry
• Firewall/proxy blocking npm
• Network timeout

Solutions:

# Check connectivity
curl https://registry.npmjs.org/

# Configure npm proxy if needed
npm config set proxy http://proxy.example.com:8080
npm config set https-proxy http://proxy.example.com:8080

# Use longer timeout
npm config set fetch-timeout 60000

Build Fails with gifsicle/Native Module Errors

Error example:

qemu-x86_64: Could not open '/lib64/ld-linux-x86-64.so.2'
autoreconf: not found

Cause: Native Node.js modules (like gifsicle for image optimization) need to be compiled, but build tools are missing.

Solution (already implemented): The Dockerfile now includes all necessary build dependencies:

• python3 - Required by node-gyp
• make, g++ - C/C++ build tools
• autoconf, automake, libtool - Build configuration
• nasm - Assembler for image optimization

If you still encounter this on Mac (ARM64):

• Ensure Docker Desktop is up to date
• Try rebuilding without cache: docker build --no-cache -t attstore:latest .
• Check that BuildKit is enabled (it is by default)

Docs Not Showing in Container

Check if docs exist in image:

docker run --rm attstore:latest ls -la /app/static/docs

Should show files including index.html, assets/, etc.

Check build logs:

docker build -t attstore:latest . 2>&1 | grep -A 10 "docs-builder"

Look for errors in:

• Git clone
• npm ci
• npm run build
• COPY command

Internal Links Not Working

Symptoms:

• Clicking links returns 404
• Assets don't load (no CSS/JS)
• Pages redirect incorrectly

Check:

1. Open browser DevTools console for errors

2. Verify docs were built with correct baseUrl:

   grep -r 'href="/docs' static/docs/index.html
   

   Should find links starting with /docs/

3. Check Flask route is working:

   curl http://localhost:5003/docs/
   

   Should return the HTML content

4. Verify baseUrl in build:

   # In built docs
   cat static/docs/index.html | grep baseUrl
   

Clone/Git Errors in Docker Build

Error: fatal: unable to access 'https://github.com/...': Could not resolve host

Solutions:

• Check internet connectivity
• Verify DNS resolution works in Docker
• Try using Docker's default DNS: docker build --network=host ...
• Check firewall/proxy settings

Build Too Slow

Optimization tips:

# Use BuildKit for better caching
DOCKER_BUILDKIT=1 docker build -t attstore:latest .

# Use --cache-from to leverage previous builds
docker pull attstore:latest
docker build --cache-from attstore:latest -t attstore:latest .

Local development:

• Built docs are cached in static/docs/
• Rebuild only when needed with make docs-force
• Don't commit static/docs/ to git (it's ignored)