VigneshBaraniS24's picture
Initial clean deployment for Hugging Face
7009aba
|
Raw History Blame Contribute Delete
7.55 kB

πŸ› οΈ Development Guide

Quick Start

πŸš€ Initial Setup

# Clone and install
git clone <repo-url>
cd sap-docs-mcp
npm install

# Run enhanced setup (submodules + build)
npm run setup

# Start development server
npm run start:http

πŸ§ͺ Run Tests

npm run test:smoke    # Quick validation
npm run test:fast     # Skip build, test only
npm run test          # Full build + test

Common Commands

πŸ“¦ Build Commands

npm run build:tsc       # Compile TypeScript
npm run build:index     # Build documentation index
npm run build:fts       # Build FTS5 search database  
npm run build           # Complete build pipeline (tsc + index + fts)

πŸ–₯️ Server Commands

npm start                    # MCP stdio server (for Claude)
npm run start:http           # HTTP development server (port 3001)
npm run start:streamable     # Streamable HTTP server (port 3122)

πŸ§ͺ Test Commands

npm run test:smoke      # Quick smoke tests
npm run test:fast       # Test without rebuild
npm run test            # Full test suite
npm run test:community  # SAP Community search tests
npm run inspect         # MCP protocol inspector

Environment Variables

πŸ”§ Core Configuration

RETURN_K=25                    # Number of search results (default: 25)
LOG_LEVEL=INFO                 # Logging level (ERROR, WARN, INFO, DEBUG)
LOG_FORMAT=json                # Log format (json or text)
NODE_ENV=production            # Environment mode

πŸ—„οΈ Database & Paths

DB_PATH=dist/data/docs.sqlite  # FTS5 database path
METADATA_PATH=src/metadata.json # Metadata configuration path

🌐 Server Configuration

PORT=3001                      # HTTP server port
MCP_PORT=3122                  # Streamable HTTP MCP port

Development Servers

πŸ“‘ 1. Stdio MCP Server (Main)

npm run start:stdio
# For Claude/LLM integration via stdio transport

🌐 2. HTTP Development Server

npm run start:http
# Access: http://localhost:3001
# Endpoints: /status, /healthz, /readyz, /mcp

πŸ”„ 3. Streamable HTTP Server

npm run start:streamable  
# Access: http://localhost:3122
# Endpoints: /mcp, /health

Where to Change Things

πŸ” Search Behavior

  • Query Processing: src/lib/searchDb.ts β†’ toMatchQuery()
  • Search Logic: src/lib/search.ts β†’ search()
  • Result Formatting: src/lib/localDocs.ts β†’ searchLibraries()

βš™οΈ Configuration

  • Source Settings: src/metadata.json β†’ Add/modify sources
  • Core Config: src/lib/config.ts β†’ System settings
  • Metadata APIs: src/lib/metadata.ts β†’ Configuration access

πŸ› οΈ MCP Tools

  • Tool Definitions: src/server.ts β†’ ListToolsRequestSchema
  • Tool Handlers: src/server.ts β†’ CallToolRequestSchema
  • HTTP Endpoints: src/http-server.ts β†’ /mcp handler

πŸ—οΈ Build Process

  • Index Building: scripts/build-index.ts
  • FTS Database: scripts/build-fts.ts
  • Source Processing: Modify build scripts for new source types

πŸ§ͺ Tests

  • Test Cases: test/tools/search/ β†’ Add new test files
  • Test Runner: test/tools/run-tests.js β†’ Modify test execution
  • Output Parsing: test/_utils/parseResults.js β†’ Update format expectations

πŸš€ Deployment

  • PM2 Config: ecosystem.config.cjs β†’ Process configuration
  • GitHub Actions: .github/workflows/deploy-mcp-sap-docs.yml
  • Setup Script: setup.sh β†’ Deployment automation

Adding New Documentation Sources

1. Update Metadata (src/metadata.json)

{
  "id": "new-source",
  "type": "documentation",
  "libraryId": "/new-source",
  "sourcePath": "new-source/docs",
  "baseUrl": "https://example.com/docs",
  "pathPattern": "/{file}",
  "anchorStyle": "github",
  "boost": 0.05,
  "tags": ["new", "documentation"],
  "description": "New documentation source"
}

2. Add Context Boosts (if needed)

"contextBoosts": {
  "New Context": {
    "/new-source": 1.0,
    "/other-source": 0.3
  }
}

3. Add Library Mapping (if needed)

"libraryMappings": {
  "new-source-alias": "new-source"
}

4. No Code Changes Required!

The metadata APIs automatically handle the new source.

Debugging

πŸ” Search Issues

# Test specific queries
node -e "
import { search } from './dist/src/lib/search.js';
const results = await search('your query');
console.log(JSON.stringify(results, null, 2));
"

# Check FTS database
sqlite3 dist/data/docs.sqlite "SELECT * FROM docs WHERE docs MATCH 'your query' LIMIT 5;"

πŸ“Š Metadata Issues

# Test metadata loading
node -e "
import { loadMetadata, getSourceBoosts } from './dist/src/lib/metadata.js';
loadMetadata();
console.log('Boosts:', getSourceBoosts());
"

🌐 Server Issues

# Check server health
curl http://localhost:3001/status
curl http://localhost:3122/health

# Test search endpoint
curl -X POST http://localhost:3001/mcp \
  -H "Content-Type: application/json" \
  -d '{"role": "user", "content": "wizard"}'

Performance Optimization

⚑ Search Performance

  • FTS5 Tuning: Modify scripts/build-fts.ts for different indexing strategies
  • Query Optimization: Adjust toMatchQuery() in src/lib/searchDb.ts
  • Result Limits: Configure RETURN_K environment variable

πŸ’Ύ Memory Usage

  • Index Size: Monitor dist/data/ artifact sizes
  • Metadata Loading: Lazy loading in src/lib/metadata.ts
  • Process Monitoring: Use PM2 monitoring features

Common Issues

❌ Build Failures

# Clean and rebuild
rm -rf dist/
npm run build:all

❌ Search Returns No Results

# Check if database exists
ls -la dist/data/docs.sqlite

# Verify index content
sqlite3 dist/data/docs.sqlite "SELECT COUNT(*) FROM docs;"

❌ Metadata Loading Errors

# Validate JSON syntax
node -e "JSON.parse(require('fs').readFileSync('src/metadata.json', 'utf8'))"

# Check file permissions
ls -la src/metadata.json

❌ Server Won't Start

# Check port availability
lsof -i :3001
lsof -i :3122

# Kill conflicting processes
lsof -ti:3001 | xargs kill -9

Best Practices

πŸ“ Code Changes

  1. Update Cursor Rules: Modify .cursor/rules/ when changing architecture
  2. Test First: Run smoke tests before committing
  3. Metadata Over Code: Use metadata.json for configuration changes
  4. Type Safety: Use metadata APIs, never direct JSON access

πŸ§ͺ Testing

  1. Smoke Tests: Always run before deployment
  2. Integration Tests: Test full MCP tool workflows
  3. Performance Tests: Monitor search response times
  4. Output Validation: Ensure format consistency

πŸš€ Deployment

  1. Build Validation: Ensure all artifacts generated
  2. Health Checks: Verify all endpoints after deployment
  3. Rollback Plan: Keep previous artifacts for quick rollback
  4. Monitoring: Watch logs and performance metrics

Useful Development Tools

πŸ”§ VS Code Extensions

  • REST Client: Use test-search.http for API testing
  • SQLite Viewer: Inspect FTS5 database content
  • JSON Schema: Validate metadata.json structure

πŸ“Š Monitoring

# PM2 monitoring
pm2 monit

# Log streaming
pm2 logs mcp-sap-http --lines 100

# Process status
pm2 status