Spaces:
Paused
Paused
Commit History
-
fix: probe backend directly, bypass nginx for health checks
health-server.js was probing nginx:7861/health → nginx proxied to uvicorn:8001 → uvicorn logged every probe. Mirroring HuggingClip's pattern of probing the app backend directly (no nginx hop). Now: probe(127.0.0.1, 8001, /health) — nginx never sees it. Uvicorn filter (prev commit) handles any residual access logs. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>c81db10 -
fix: filter /health hits from uvicorn access log
Custom HealthEndpointFilter drops /health from uvicorn.access handler. Nginx proxy-passes health probes to backend, causing log spam every 30s. Filter reads ASGI scope.path so it catches it before message formatting. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>dc0726e -
fix: silence noisy backend logs + fix gemini key field name
- Fix API_KEY_FIELD google→google_api_key (was gemini_api_key): eliminates 4x UserWarning per LLM call from langchain-google-genai - Remove duplicate GEMINI_API_KEY export (keep only GOOGLE_API_KEY): eliminates "Both GOOGLE_API_KEY and GEMINI_API_KEY are set" warning - Add uvicorn log-config: suppress INFO noise from google_genai, httpx, primp, ddgs loggers (silence AFC/retry/response spam) - Suppress pydantic UserWarnings via PYTHONWARNINGS - nginx: access_log off for /health (suppress keepalive ping noise) Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>28f94e0
-
feat(sync): skip backup when no state changes detected
Add metadata marker (file count/size/mtime) and SHA-256 fingerprint checks before uploading. State persisted to /tmp so subsequent sync intervals within the same container run skip unnecessary HF uploads. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>d89d9d5
-
docs: add duplicate space button, secrets frontmatter, support section
- Frontmatter: add secrets: block (LLM_MODEL, LLM_API_KEY, HF_TOKEN, SERPER_API_KEY, AUTH_JWT_SECRET, CLOUDFLARE_WORKERS_TOKEN) — pre-fills secret names and descriptions in the HF Spaces duplicate dialog - Frontmatter: add license: mit field - Quick Start: replace text step with clickable Duplicate Space SVG badge + add GitHub-style TIP callout for HF_TOKEN recommendation - More Projects: expand to full table with HF Space + GitHub links for all @somratpro projects (HuggingClip, HuggingClaw, HuggingMes, Hugging8n, HuggingPost) - Add Support / USDT section matching HuggingClip style Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>0b075c6 -
fix: 12-point production audit — bugs, security, perf, reliability
start.sh: - BACKUP_DATASET_NAME/SYNC_INTERVAL defaulted before export — empty string was passed to deerflow-sync.py causing invalid repo name (bug) - Export FRONTEND_PORT and BACKEND_PORT so health-server.js and child processes receive correct values from environment - openssl JWT generation now has Python fallback — openssl CLI not guaranteed in python:3.12-slim-bookworm slim image - Remove dead os.environ["CORS_ORIGINS"] in Python heredoc — Python subprocess env never propagates back to parent bash (dead code) - Graceful shutdown uses explicit PID list instead of $(jobs -p) which is unreliable in bash subshells - Truncate log files on startup to prevent unbounded disk growth health-server.js: - sync "error" status now renders as "off" (red) not "neutral" (grey) - Dashboard response includes Cache-Control: no-store header nginx.conf: - gzip compression enabled for JSON/HTML/JS — large research reports can be 60-80% smaller over the wire - server_tokens off — stop leaking…b6e5beb -
fix: robust backup — retry on HF 500, SQLite WAL checkpoint, fix tar deprecation
- _upload_with_retry(): 4 attempts with 3/6/12s exponential backoff — handles transient HF 500 errors (was failing permanently on single attempt) - _checkpoint_sqlite(): PRAGMA wal_checkpoint(TRUNCATE) before archiving so the snapshot is consistent; WAL/SHM files excluded from archive entirely - _make_archive(): explicit per-file iteration with _should_exclude() filter instead of blind tar.add() which dragged in .db-wal and .db-shm - _extract_archive(): use tar.extractall(filter='data') for Python 3.12+ to suppress DeprecationWarning about unfiltered tar extraction Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>1ca1144 -
feat: auto-generate and persist AUTH_JWT_SECRET — sessions survive restarts
Pattern from HuggingPost: check env var → load from disk → generate fresh. Generated secret saved to DATA_DIR/.secrets/auth-jwt-secret (chmod 600). .secrets dir added to deerflow-sync.py BACKUP_TARGETS so the secret survives HF Dataset backup/restore cycles (i.e. persists across Space rebuilds too). Priority: HF Space secret (AUTH_JWT_SECRET) > saved file > auto-generate. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>31f0064 -
fix: switch frontend probe to TCP — eliminates 429 rate-limit spam on /api/v1/auth/setup-status
HTTP probing /workspace or / via nginx triggers Next.js SSR which makes server-side calls to /api/v1/auth/setup-status every 30s (dashboard auto-refresh). DeerFlow rate-limits that endpoint → flood of 429s in logs. Replace all three frontend probes (health, status, dashboard handlers) with a TCP socket check on FRONTEND_PORT (3000). TCP confirms Next.js is listening without issuing any HTTP request, so no SSR runs and no auth endpoints are hit. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>53b20c2 -
fix: enable agents_api in generated config — custom agent management was disabled
DeerFlow defaults agents_api.enabled=false. The UI /workspace/agents/new was returning "Custom agent management is not enabled on this server." because the generated config.yaml didn't override the default. Set agents_api.enabled=true in the Python config generation block in start.sh. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>a7898a7 -
fix: dashboard probe /api/health → /health — was stuck on STARTING
/api/health requires JWT auth → 401 → backendUp=false → tile shows STARTING forever. /health is the public unauthenticated endpoint (nginx routes it to backend:8001/health → 200 OK). Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>a603910 -
fix: replace dashboard footer with custom copyright — Built by @somratpro
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>0c04730 -
chore: production-readiness pass — add missing release files, clean up code
New files: - .env.example: full configuration reference with inline docs - .gitattributes: normalize line endings (LF), mark binaries - CHANGELOG.md: v1.0.0 release history + unreleased roadmap - CODE_OF_CONDUCT.md: community standards - SECURITY.md: vulnerability reporting, secrets mgmt, known limitations - docker-compose.yml: local development with hot-reload volume mounts Code fixes: - health-server.js: remove dead .replace() call on template literal output - Dockerfile: remove dashboard.html COPY (nginx no longer serves it) - dashboard.html: delete orphaned file Docs: - CONTRIBUTING.md: full rewrite with file overview table, docker-compose workflow, hot-reload note - README.md: fix stale worker count caveat (1 worker, not 2) Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>0ff29e3 -
feat: dynamic dashboard at / + rewrite README following HuggingClip style
- health-server.js: serve live dashboard at / (not just /dashboard); hero → /workspace; probe /api/health + /workspace - nginx.conf: remove static location = / block (health-server handles root now) - README.md: full rewrite — badges, ToC, Features, Quick Start, LLM Providers, Search, Cloudflare, Backup, Keep Alive, Architecture, Troubleshooting, More Projects Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>534b16f -
fix: set DEER_FLOW_TRUSTED_ORIGINS for Next.js — fixes Application error
The pre-built frontend image (af6e48cc) uses an older gateway-config.ts that has NO defaults in production mode. Both DEER_FLOW_INTERNAL_GATEWAY_BASE_URL and DEER_FLOW_TRUSTED_ORIGINS must be set explicitly — otherwise zod schema validation throws, getServerSideUser() returns {tag:"config_error"}, workspace/auth layouts throw new Error(), and Next.js shows "Application error: a server-side exception has occurred" on every page. We were setting INTERNAL_GATEWAY_BASE_URL but not TRUSTED_ORIGINS. TRUSTED_ORIGINS set to localhost:3000 + localhost:7860 + SPACE_HOST. With this fix the SSR no longer throws. /workspace redirects to /login (unauthenticated) or /setup (first boot). /setup renders the admin account creation form. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>1e9ba78 -
fix: serve dashboard.html with try_files instead of alias
nginx alias on a file path appends index.html → 500. Use root + try_files /dashboard.html which resolves correctly to /app/dashboard.html without directory index lookup. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>b2d6ac5 -
feat: custom dashboard at / and route DeerFlow under /app
- Add dashboard.html: status page with links to workspace, setup, API docs - nginx: location = / serves dashboard.html (alias, no-cache) - nginx: location = /app redirects 302 → /workspace - Dockerfile: COPY dashboard.html to /app/dashboard.html To use DeerFlow: visit /workspace (or /app as alias). First boot: visit /setup to create admin account before /workspace. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>939f343 -
fix: drop uvicorn to 1 worker to prevent SQLite CREATE TABLE race
With --workers 2, both processes run Base.metadata.create_all simultaneously. SQLite has no IF NOT EXISTS guard in DeerFlow's engine.py, so the second worker always fails with "table feedback already exists" and gets killed/restarted. Single worker eliminates the race; SQLite is single-writer anyway. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>e778d47 -
fix: execute next shim as shell script, not node script
pnpm's node_modules/.bin/next is a shell shim — running it with `node` causes SyntaxError. Drop the `node` prefix; NODE_OPTIONS env var still propagates to the child node process spawned by the shim. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>9eceb2e -
refactor: replace source compilation with official pre-built GHCR images
Use ghcr.io/bytedance/deer-flow-backend:latest and ghcr.io/bytedance/deer-flow-frontend:latest as source stages instead of cloning + building from source. Eliminates: - pnpm install + Next.js Turbopack build (OOM risk, 15-20 min) - uv sync + grpcio/sympy native compilation (stall risk, 10-15 min) Build now pulls pre-built images (~5 min) instead of compiling (~30 min). Retain minimal alpine/git clone only for skills/ and config.example.yaml which are not bundled in the official images. Image structure verified from upstream Dockerfiles: backend: /app/backend/ (source + .venv) matches start.sh cd /app/backend frontend: /app/frontend/ (built .next + node_modules) matches start.sh cd /app/frontend Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>45fe8f0 -
fix: split runtime RUN layers and add pip timeout/retries
Split the monolithic apt+pip RUN into three separate layers so BuildKit caches each independently — a pip network stall no longer forces Node.js to reinstall. Add --timeout 120 --retries 5 and a 3-attempt retry loop to pip install huggingface_hub/pyyaml (mirrors the uv sync retry pattern). Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>6cd3e11 -
fix: prevent OOMKilled by serializing build stages and capping Node heap
- Add NODE_OPTIONS=--max-old-space-size=3072 to Next.js build step (caps Node heap at 3 GB, prevents OOM during Turbopack compilation) - Serialize backend-builder after frontend-builder via COPY --from trick (BuildKit parallelizes independent stages; running Next.js + grpcio compilation simultaneously exhausted HF Spaces build server memory) - Reduce UV_CONCURRENT_DOWNLOADS from 4 to 2 to lower peak memory Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>0e52d6d -
fix: align with DeerFlow's official Docker pattern — BuildKit cache + --no-install-package
Comprehensive overhaul based on reading deer-flow/backend/Dockerfile: 1. Add # syntax=docker/dockerfile:1 (enables BuildKit features) 2. uv sync uses --mount=type=cache,target=/root/.cache/uv (their pattern) — wheel cache persists across retries within the RUN, so each retry only re-fetches the one wheel that failed — was the missing piece causing every retry to start from zero 3. Use --no-install-package onnxruntime/magika/speechrecognition/pdfminer-six — clean uv flag, no need to patch pyproject.toml or delete uv.lock — skips ~65MB of markitdown[all] extras not needed for web research 4. Set UV_INDEX_URL=https://pypi.org/simple (DeerFlow's official setting) UV_LINK_MODE=copy (avoid hardlink issues on cache mount) 5. pnpm install also gets BuildKit cache + retry, in case it stalls Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>b73c012 -
fix: keep uv.lock — deleting it caused silent hang during full re-resolution
Without uv.lock, uv fetches metadata for all 200+ packages from PyPI to re-resolve the entire dependency tree. On HF Spaces flaky network this hangs silently for 10+ minutes with no output. With the lockfile present, uv only re-resolves the one changed package (markitdown extras) — one metadata fetch, then straight to downloading. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>5847bce -
fix: reduce UV_HTTP_TIMEOUT 600→120s so stalled connections fail fast
600s was the bug: a stalled grpcio download sat for 10 full minutes before timing out. 120s is enough for any 6-30MB package at >=100KB/s, but stalls fail in 2 minutes. Combined with 10 retries + uv wheel cache, each retry only re-fetches the one stalled package. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>702035a -
fix: correct markitdown patch string — actual format is markitdown[all,xlsx]
Debug output revealed the exact dependency string in harness/pyproject.toml: 'markitdown[all,xlsx]>=0.0.1a2' Replace with markitdown[xlsx] to keep Excel support but drop heavy extras: speechrecognition (31MB), pdfminer-six (6MB), magika (15MB), onnxruntime (13MB) Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>28b0f1d -
fix: UV_HTTP_TIMEOUT=600 + 10 retries + markitdown debug output
HF Spaces build network is genuinely slow (~100KB/s), so 300s timeout was cutting off large packages (speechrecognition 31MB needs 310s at 100KB/s). 600s gives enough headroom. 10 retries with uv cache mean each retry only re-fetches the one package that timed out — converges quickly. Also add debug output for markitdown line format so we can see the exact string to patch (previous patch not confirmed working). Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>8a2127b -
fix: remove invalid --concurrent-downloads flag, use UV_CONCURRENT_DOWNLOADS env var only
The flag does not exist in uv CLI at all. UV_CONCURRENT_DOWNLOADS=4 env var is the correct mechanism to limit parallel downloads. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>7ebd244 -
fix: --concurrent-downloads is global uv flag, goes before subcommand
uv sync --concurrent-downloads 4 → WRONG (subcommand flag) uv --concurrent-downloads 4 sync → CORRECT (global flag) Also add UV_CONCURRENT_DOWNLOADS=4 env var as belt-and-suspenders. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>ae7e909 -
fix: reliable markitdown patch + --concurrent-downloads CLI flag
Previous attempt had two silent failures: - sed escaping didn't match markitdown[all] in pyproject.toml - UV_CONCURRENT_DOWNLOADS env var not recognized by uv 0.7.x Fix: - Use Python heredoc for reliable string replacement (prints warning if not found) - Delete uv.lock in separate RUN so resolver picks up patched deps - Pass --concurrent-downloads directly to uv sync CLI (4 → 2 → 1 on retry) Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>876a7c3 -
fix: limit uv concurrency + strip markitdown[all] heavy extras
Two root causes of build stalls on HF Spaces flaky network: 1. uv default concurrent downloads saturate the connection — cap at 4 2. markitdown[all] pulls speechrecognition (31MB), pdfminer-six (6MB), magika (15MB), onnxruntime (13MB) — none needed for web research Patch pyproject.toml via sed before uv sync, drop uv.lock so it resolves fresh without the heavy extras (~65MB less to download). Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>30b5b3a -
fix: retry uv sync up to 3x for flaky HF Spaces build network
Transient I/O errors during wheel extraction (speechrecognition, pdfminer-six) cause spurious build failures. uv caches successful downloads so retries only re-fetch the failed package. Also raise UV_HTTP_TIMEOUT 120s → 300s. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>da4628c -
fix: raise UV_HTTP_TIMEOUT to 120s in backend-builder to prevent download timeouts
speechrecognition wheel (via markitdown[all] → deerflow-harness) was timing out at the default 30s limit during HF Spaces build. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>4c171b7 -
feat: restructure to HuggingClaw architecture with dashboard + Cloudflare features
Port architecture: health-server.js:7860 (public) → nginx:7861 → backend:8001/frontend:3000 - Add health-server.js: Node.js status dashboard at /dashboard + reverse proxy to nginx - Add cloudflare-proxy.js: patches Node.js HTTP/fetch/undici for outbound CF proxy - Add cloudflare-proxy-setup.py: deploys CF Worker for outbound proxy routing - Add cloudflare-keepalive-setup.py: deploys CF cron Worker to ping /health - Rename flow-sync.py → deerflow-sync.py; write /tmp/huggingflow-sync-status.json after each sync - Update nginx.conf: listen on 7861 (internal only) - Update start.sh: CF proxy/keepalive setup, health-server start, NODE_OPTIONS for CF proxy - Update Dockerfile: COPY all new runtime scripts, updated HEALTHCHECK path - Add LICENSE (MIT), CONTRIBUTING.md, .gitignore Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>6a3762e -
rename: HuggingDeer → HuggingFlow
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>c13c2a7 -
fix: add HF Spaces YAML metadata to README
Required by HF hub for Docker SDK space detection and port routing. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>14dfaa4 -
init: HuggingDeer — DeerFlow on Hugging Face Spaces
Single-container Docker deployment of DeerFlow (frontend + backend + nginx). Clones deer-flow source at build time, builds Next.js and Python backend, runs all three services inside one container on port 7860. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>5d8d23e