|
Download README.md from Slfgod/shield-agents: direct link, hf CLI and curl.
- Browser
- Download file 17.6 kB
-
https://huggingface.co/Slfgod/shield-agents/resolve/main/README.md
- Command line
-
hf download hf://Slfgod/shield-agents/README.md
-
curl -L -o README.md https://huggingface.co/Slfgod/shield-agents/resolve/main/README.md
17.6 kB
| <div align="center"> | |
| # Shield Agents | |
| ### AI-Powered Multi-Agent Cybersecurity Scanner | |
| [](https://www.python.org/downloads/) | |
| [](https://opensource.org/licenses/MIT) | |
| [](https://github.com/astral-sh/ruff) | |
| []() | |
| []() | |
| **Production-grade security analysis using coordinated AI agents.** | |
| Detect vulnerabilities, threats, secrets, and compliance issues β then auto-fix them. | |
| [Getting Started](#getting-started) Β· [Features](#features) Β· [Architecture](#architecture) Β· [CI/CD](#cicd-integration) Β· [VS Code](#vs-code-extension) Β· [Contributing](CONTRIBUTING.md) | |
| </div> | |
| --- | |
| ## Why Shield Agents? | |
| Most security scanners just find problems. Shield Agents **finds AND fixes them** using a team of specialized AI agents that work together like a real security team: | |
| - **VulnAgent** β Finds SQL injection, XSS, command injection, and more | |
| - **ThreatAgent** β Maps attack vectors to MITRE ATT&CK techniques | |
| - **ReconAgent** β Detects information disclosure and exposed secrets | |
| - **ComplianceAgent** β Checks against OWASP Top 10 2021 | |
| - **ResponseAgent** β Provides risk assessment and incident response plans | |
| - **AutoFixAgent** β Generates copy-paste-ready code fixes | |
| Works out of the box with the **smart mock provider** β no API key needed to start scanning. | |
| --- | |
| ## Getting Started | |
| ### Installation | |
| ```bash | |
| # Install from source (mock provider included - no API key needed) | |
| pip install -e . | |
| # With OpenAI support | |
| pip install -e ".[openai]" | |
| # With all LLM providers (OpenAI + Anthropic + Ollama) | |
| pip install -e ".[all]" | |
| # With development tools | |
| pip install -e ".[dev]" | |
| ``` | |
| ### Your First Scan | |
| ```bash | |
| # Scan a project (uses smart mock provider by default) | |
| shield-agents scan ./my-project | |
| # Scan with auto-fix suggestions | |
| shield-agents scan ./my-project --fix | |
| # Full scan (ignore cache, scan everything) | |
| shield-agents scan ./my-project --full | |
| # Initialize configuration files | |
| shield-agents init | |
| ``` | |
| ### Docker | |
| ```bash | |
| # Build and scan | |
| docker-compose up shield-agents | |
| # With Ollama for local LLM | |
| docker-compose up ollama shield-agents | |
| ``` | |
| --- | |
| ## Features | |
| ### Core Scanning Engine | |
| | Feature | Details | | |
| |---------|---------| | |
| | **SAST Scanner** | 10 detection rules: SQL Injection, XSS, Command Injection, Path Traversal, Insecure Deserialization, Weak Crypto, Auth Issues, Hardcoded Credentials, SSL/TLS Issues, SSRF | | |
| | **Secrets Scanner** | 24 pattern types: AWS, GitHub, Google, Slack, Stripe, DB connections, JWTs, Private Keys, and generic secrets with Shannon entropy filtering | | |
| | **6 AI Agents** | VulnAgent, ThreatAgent, ReconAgent, ComplianceAgent, ResponseAgent, AutoFixAgent | | |
| | **Auto-Fix Engine** | Pattern-based instant fixes + LLM-powered deep remediation with copy-paste-ready code | | |
| ### Production Features (v2.0) | |
| | Feature | Description | | |
| |---------|-------------| | |
| | **Smart Mock Provider** | Pattern-matched findings based on actual code content β demo mode feels real, not static | | |
| | **Deduplication Engine** | 3-phase dedup (exact β fuzzy β category merge) β when VulnAgent and SAST both find "SQL Injection", they merge into one finding with multiple sources | | |
| | **SARIF 2.1.0 Output** | GitHub Security tab integration β upload results directly to GitHub code scanning | | |
| | **`.shieldignore` File** | Like `.gitignore` for false positives β 9 rule types (file, category, severity, rule, path, line, title, cwe, id) | | |
| | **Caching / Incremental Scans** | Cache previous results, only re-scan changed files β production-ready performance | | |
| | **LLM Fallback Parser** | 5-strategy parser for when LLMs return invalid JSON (~30% failure rate) β never crash on bad responses | | |
| | **VS Code Extension** | On-save scanning with inline diagnostics β security issues appear directly in your editor | | |
| | **Benchmark Suite** | 13 OWASP WebGoat-style test cases β proves the scanner actually finds real bugs | | |
| | **CI/CD Mode** | `--ci` flag for pipelines β JSON to stdout, SARIF output, exit code based on risk threshold | | |
| | **Format Options** | Pipe results to other tools β structured JSON output for integration | | |
| | **Auto-Exclude** | Tests, benchmarks, and examples automatically excluded from scan β eliminates ~70% false positives | | |
| | **Agent-Differentiated Mock** | Each AI agent returns specialized findings in mock mode β VulnAgent finds vulns, ThreatAgent finds threats | | |
| --- | |
| ## CLI Reference | |
| ```bash | |
| shield-agents scan <target> [options] | |
| Options: | |
| --config, -c Path to config YAML file | |
| --full Full scan (ignore cache) | |
| --fix Generate auto-fix suggestions | |
| --no-report Skip report generation | |
| --sarif-only Output only SARIF format | |
| --output, -o Output directory for reports | |
| --provider LLM provider: mock, openai, anthropic, ollama | |
| --format, -f Output format: rich, json, sarif, plain (default: rich) | |
| --ci CI/CD mode: silent except JSON on stdout | |
| --fail-threshold Risk score threshold for CI failure (default: 75) | |
| --no-cache Disable caching | |
| --no-dedup Disable deduplication | |
| --no-ignore Ignore .shieldignore rules | |
| --verbose, -v Verbose output | |
| --debug Debug output | |
| Commands: | |
| scan <target> Run security scan | |
| init Create .shieldignore and config templates | |
| cache --stats Show cache statistics | |
| cache --clear Clear scan cache | |
| version Show version info | |
| ``` | |
| --- | |
| ## Configuration | |
| ### Environment Variables | |
| | Variable | Description | Default | | |
| |----------|-------------|---------| | |
| | `SHIELD_LLM_PROVIDER` | LLM provider (mock, openai, anthropic, ollama) | `mock` | | |
| | `SHIELD_LLM_API_KEY` | API key for the LLM provider | None | | |
| | `SHIELD_LLM_MODEL` | Model name | `gpt-4` | | |
| | `SHIELD_LLM_BASE_URL` | Custom API base URL (for Ollama) | None | | |
| | `SHIELD_VERBOSE` | Enable verbose output | `false` | | |
| | `SHIELD_DEBUG` | Enable debug output | `false` | | |
| ### Config File (`config.yaml`) | |
| ```yaml | |
| llm: | |
| provider: mock # mock, openai, anthropic, ollama | |
| model: gpt-4 | |
| temperature: 0.1 | |
| fallback_enabled: true # Robust JSON fallback parser | |
| scanner: | |
| sast_enabled: true | |
| secrets_enabled: true | |
| cache: | |
| enabled: true | |
| incremental: true # Only re-scan changed files | |
| deduplication: | |
| enabled: true | |
| merge_sources: true # Merge duplicate findings from different sources | |
| report: | |
| output_dir: ./shield-reports | |
| formats: | |
| - html | |
| - sarif | |
| - json | |
| ``` | |
| ### `.shieldignore` File | |
| Suppress false positives with 9 rule types: | |
| ```gitignore | |
| # Shield Agents Ignore File | |
| category:information-disclosure # Ignore all info-disclosure findings | |
| severity:LOW # Ignore LOW and INFO findings | |
| rule:SAST-001 # Ignore a specific rule | |
| file:test_app.py # Ignore all findings in a file | |
| path:*test* # Ignore findings in test files | |
| line:42:app.py # Ignore finding at specific line | |
| title:Assertion* # Ignore findings with matching title (glob) | |
| cwe:CWE-617 # Ignore findings with matching CWE | |
| id:VulnAgent-3 # Ignore a specific finding by ID | |
| ``` | |
| --- | |
| ## Architecture | |
| ``` | |
| shield-agents/ | |
| βββ shield_agents/ | |
| β βββ agents/ # 6 AI agents | |
| β β βββ base.py # Abstract base agent | |
| β β βββ vuln.py # Vulnerability detection | |
| β β βββ threat.py # Threat modeling & MITRE ATT&CK | |
| β β βββ recon.py # Reconnaissance & info disclosure | |
| β β βββ compliance.py # OWASP compliance checking | |
| β β βββ response.py # Incident response & risk assessment | |
| β β βββ autofix.py # Auto-remediation | |
| β βββ scanners/ # Static analysis | |
| β β βββ sast.py # 10 SAST detection rules | |
| β β βββ secrets.py # 24 secret type patterns | |
| β βββ report/ # Output generation | |
| β β βββ generator.py # HTML + JSON reports | |
| β β βββ sarif.py # SARIF 2.1.0 for GitHub | |
| β βββ knowledge/ # Security knowledge bases | |
| β β βββ owasp.py # OWASP Top 10 2021 | |
| β β βββ mitre.py # MITRE ATT&CK | |
| β β βββ cve.py # Known CVE database | |
| β βββ utils/ # Utilities | |
| β β βββ crypto.py # Hashing for cache | |
| β β βββ helpers.py # File I/O, risk scoring | |
| β βββ config.py # Configuration management | |
| β βββ llm.py # LLM providers + 5-strategy fallback parser | |
| β βββ deduplication.py # 3-phase finding deduplication engine | |
| β βββ cache.py # Incremental scan caching | |
| β βββ shieldignore.py # .shieldignore file parser (9 rule types) | |
| β βββ orchestrator.py # Central coordinator (6-phase pipeline) | |
| β βββ cli.py # Rich-powered CLI | |
| βββ vscode-extension/ # VS Code extension | |
| βββ benchmarks/ # OWASP-style benchmark suite (13 cases) | |
| βββ tests/ # 52 unit tests | |
| βββ examples/ # Vulnerable example app | |
| βββ .github/ # Issue/PR templates | |
| βββ CHANGELOG.md # Version history | |
| βββ CONTRIBUTING.md # Contribution guide | |
| βββ pyproject.toml # Modern Python packaging | |
| ``` | |
| ### 6-Phase Scan Pipeline | |
| ``` | |
| βββββββββββββββ ββββββββββββββββββββ βββββββββββββββββ | |
| β Phase 1 β β Phase 2 β β Phase 3 β | |
| β SAST Scan ββββΆβ AI Agent AnalysisββββΆβ Deduplication β | |
| β + Secrets β β (4 agents/file) β β (3 phases) β | |
| βββββββββββββββ ββββββββββββββββββββ βββββββββββββββββ | |
| β | |
| βββββββββββββββ ββββββββββββββββββββ βββββββββββββββββ | |
| β Phase 6 β β Phase 5 β β Phase 4 β | |
| β Reporting βββββ Auto-Fix Gen βββββ .shieldignore β | |
| β HTML/SARIF β β (Pattern + LLM) β β Filtering β | |
| βββββββββββββββ ββββββββββββββββββββ βββββββββββββββββ | |
| ``` | |
| --- | |
| ## CI/CD Integration | |
| ### GitHub Actions | |
| ```yaml | |
| name: Security Scan | |
| on: [push, pull_request] | |
| jobs: | |
| security: | |
| runs-on: ubuntu-latest | |
| steps: | |
| - uses: actions/checkout@v4 | |
| - uses: actions/setup-python@v5 | |
| with: | |
| python-version: '3.11' | |
| - name: Install Shield Agents | |
| run: pip install -e . | |
| - name: Run Security Scan | |
| run: shield-agents scan ./src --ci --fail-threshold 75 | |
| - name: Upload SARIF | |
| if: always() | |
| uses: github/codeql-action/upload-sarif@v3 | |
| with: | |
| sarif_file: shield-reports/results.sarif | |
| ``` | |
| ### GitLab CI | |
| ```yaml | |
| security-scan: | |
| stage: test | |
| image: python:3.11-slim | |
| script: | |
| - pip install -e . | |
| - shield-agents scan ./src --ci --format json > scan-results.json | |
| artifacts: | |
| paths: | |
| - shield-reports/ | |
| reports: | |
| sast: shield-reports/results.sarif | |
| ``` | |
| ### Generic Pipeline (JSON to stdout) | |
| ```bash | |
| # JSON output for any CI system | |
| shield-agents scan ./src --ci --format json | |
| # Exit code: 0 = pass, 1 = risk score exceeds threshold | |
| shield-agents scan ./src --ci --fail-threshold 60 | |
| ``` | |
| --- | |
| ## VS Code Extension | |
| Real-time security scanning directly in your editor: | |
| - **Scan on Save** β Automatically scans files when you save | |
| - **Inline Diagnostics** β Security findings appear as editor warnings/errors | |
| - **Severity Highlighting** β Color-coded CRITICAL/HIGH/MEDIUM/LOW indicators | |
| - **Quick Fix Suggestions** β Remediation advice for each finding | |
| - **Workspace Scan** β Scan your entire project with one command | |
| Install the extension from the `vscode-extension/` directory and configure via VS Code settings. | |
| --- | |
| ## AI Assistant Compatibility | |
| Shield Agents works seamlessly with AI coding assistants: | |
| | Tool | How to Use | | |
| |------|-----------| | |
| | **Cursor** | Add `shield-agents scan` to `.cursorrules` for auto-scanning on changes | | |
| | **Claude Code** | Run `shield-agents scan` in terminal, integrate findings into workflow | | |
| | **GitHub Copilot** | VS Code extension provides inline security diagnostics alongside Copilot | | |
| | **Aider** | Use `--fix` mode and pipe auto-fix suggestions for code modifications | | |
| | **Windsurf** | CLI-based integration via terminal | | |
| --- | |
| ## Benchmark Suite | |
| Verify detection accuracy with 13 OWASP WebGoat-style test cases: | |
| ```bash | |
| python -m benchmarks.benchmark --verbose | |
| ``` | |
| | Category | Test Cases | | |
| |----------|-----------| | |
| | SQL Injection | String concat, format strings | | |
| | XSS | DOM-based, template injection | | |
| | Command Injection | os.system, subprocess | | |
| | Path Traversal | User-controlled file paths | | |
| | Insecure Deserialization | pickle, yaml, marshal | | |
| | Weak Cryptography | MD5, SHA-1, random module | | |
| | Hardcoded Secrets | Passwords, API keys, AWS credentials | | |
| | SSL/TLS Issues | verify=False, unverified context | | |
| | SSRF | User-controlled URLs | | |
| | Auth Bypass | Assertion-based, session manipulation | | |
| | Clean Code | Negative test (minimal findings) | | |
| --- | |
| ## Running Tests | |
| ```bash | |
| # Run all 52 unit tests | |
| pytest tests/ -v | |
| # Run specific test module | |
| pytest tests/test_sast.py -v | |
| pytest tests/test_secrets.py -v | |
| pytest tests/test_llm.py -v | |
| # Run benchmarks | |
| python -m benchmarks.benchmark --verbose | |
| # Lint | |
| ruff check shield_agents/ | |
| ``` | |
| --- | |
| ## Contributing | |
| We love contributions! See [CONTRIBUTING.md](CONTRIBUTING.md) for detailed guidelines. | |
| Quick start: | |
| ```bash | |
| git clone https://github.com/shield-agents/shield-agents.git | |
| cd shield-agents | |
| pip install -e ".[dev]" | |
| pytest tests/ -v | |
| ``` | |
| --- | |
| ## License | |
| MIT License β see [LICENSE](LICENSE) for details. | |
| --- | |
| <div align="center"> | |
| ### En EspaΓ±ol | |
| **Shield Agents β EscΓ‘ner de Ciberseguridad Multi-Agente con IA** | |
| Shield Agents es una plataforma de anΓ‘lisis de seguridad de grado producciΓ³n que utiliza agentes de IA coordinados para detectar vulnerabilidades, amenazas, secretos y problemas de cumplimiento. | |
| **CaracterΓsticas principales:** | |
| | CaracterΓstica | DescripciΓ³n | | |
| |---------------|-------------| | |
| | EscΓ‘ner SAST | 10 reglas de detecciΓ³n: InyecciΓ³n SQL, XSS, InyecciΓ³n de Comandos, Path Traversal, DeserializaciΓ³n Insegura, CriptografΓa DΓ©bil, Problemas de AutenticaciΓ³n, Credenciales hardcodeadas, Problemas SSL/TLS, SSRF | | |
| | EscΓ‘ner de Secretos | 24 tipos de patrones: AWS, GitHub, Google, Slack, Stripe, conexiones DB, JWTs, Claves Privadas, con filtrado de entropΓa Shannon | | |
| | 6 Agentes de IA | VulnAgent (vulnerabilidades), ThreatAgent (modelado de amenazas), ReconAgent (reconocimiento), ComplianceAgent (cumplimiento OWASP), ResponseAgent (respuesta a incidentes), AutoFixAgent (correcciones automΓ‘ticas) | | |
| | Motor de DeduplicaciΓ³n | 3 fases: coincidencia exacta β coincidencia difusa β fusiΓ³n por categorΓa | | |
| | Formato SARIF | IntegraciΓ³n con la pestaΓ±a de Seguridad de GitHub | | |
| | Archivo `.shieldignore` | Como `.gitignore` pero para falsos positivos β 9 tipos de reglas | | |
| | Escaneo Incremental | CachΓ© de resultados anteriores, solo re-escanea archivos modificados | | |
| | Auto-Fix | Correcciones instantΓ‘neas basadas en patrones + correcciones profundas con LLM | | |
| | ExtensiΓ³n VS Code | Escaneo al guardar con diagnΓ³sticos inline | | |
| | Modo CI/CD | `--ci` para pipelines β JSON a stdout, SARIF, cΓ³digo de salida basado en riesgo | | |
| **Inicio rΓ‘pido:** | |
| ```bash | |
| # Instalar (proveedor mock incluido β sin API key necesario) | |
| pip install -e . | |
| # Escanear un proyecto | |
| shield-agents scan ./mi-proyecto | |
| # Escanear con sugerencias de correcciΓ³n automΓ‘tica | |
| shield-agents scan ./mi-proyecto --fix | |
| # Modo CI/CD | |
| shield-agents scan ./src --ci --fail-threshold 75 | |
| # Inicializar configuraciΓ³n | |
| shield-agents init | |
| ``` | |
| **Compatibilidad con asistentes de IA:** | |
| Shield Agents es compatible con Cursor, Claude Code, GitHub Copilot, Aider y Windsurf. Funciona como una herramienta CLI estΓ‘ndar que se integra en cualquier flujo de trabajo de desarrollo. | |
| **Contribuir:** | |
| Las contribuciones son bienvenidas. Lee [CONTRIBUTING.md](CONTRIBUTING.md) para las guΓas detalladas. Los reportes de bugs, solicitudes de funcionalidades y pull requests son apreciados. | |
| **Licencia:** MIT | |
| --- | |
| If you find Shield Agents useful, please star the repo β it helps others discover the project! | |
| Built with care for the security community | |
| </div> | |