Spaces:
Running
Running
Commit History
-
Require the API key on /v1 and default thinking off Running
Verified signatureeac299b
-
Use LLAMA_API_KEY (not gateway's own API_KEY) to authenticate to llama-server
Claude +1: Claude Sonnet 5Unverified signatureThe previous fix reused the gateway's own API_KEY secret, but that's unset on this Space and, more fundamentally, is the wrong secret: llama.cpp maps --api-key to the env var LLAMA_API_KEY specifically (not the generic LLAMA_ARG_* pattern used for other flags), and this Space already has a LLAMA_API_KEY secret configured for llama-server that nothing was wiring up. Read that env var directly and attach it to every gateway -> llama-server request instead. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01XvAfE7bsSLrQm1VVBswYnx490468e -
Forward the API key to llama-server on proxied requests
Claude +1: Claude Sonnet 5Unverified signaturegateway.py never sent any credentials when it proxied /v1/models and /v1/chat/completions through to llama-server, so once llama-server has its own API-key protection configured it rejects every request with "Invalid API Key" regardless of what key the caller supplied to the gateway. Attach the same Bearer API_KEY the gateway already uses for its own auth check when calling llama-server. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01XvAfE7bsSLrQm1VVBswYnx30eec79 -
Fix llama-server failing to start: set LD_LIBRARY_PATH
Claude +1: Claude Sonnet 5Unverified signatureThe deployed Space's runtime logs showed llama-server crashing on every start with: /app/llama-server: error while loading shared libraries: libllama-server-impl.so: cannot open shared object file Per ggml-org/llama.cpp's own server.Dockerfile, the "server" stage copies its shared libraries flat into /app next to the llama-server binary and relies on RPATH to find them there, which isn't reliably resolving. start.sh backgrounds the process with "&" so this failure was silent: uvicorn always started anyway, gateway.py's proxy always got a connection refused/unavailable, and the UI/API never worked. Export LD_LIBRARY_PATH=/app before launching it so the dynamic linker can find the libraries directly. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01XvAfE7bsSLrQm1VVBswYnx5cf1cf9 -
Serve llama.cpp web UI on port 7860 via gateway proxy
Claude +1: Claude Sonnet 5Unverified signaturegateway.py was the only process exposed on 7860 (HF Spaces only route one port), but it had no route for "/" or static assets, and llama-server was started with --no-webui, so every UI request 404'd. Enable the built-in webui in llama-server and add a catch-all reverse proxy in gateway.py that forwards any unmatched request to it. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01XvAfE7bsSLrQm1VVBswYnx08e31f7 -
Update gateway.py
Verified signatureac524df -
Rename gateway (1).py to gateway.py
Verified signature26d58c0 -
Delete gateway.py
Verified signaturec27b76e -
Upload gateway (1).py
Verified signatured2436c2 -
Upload 2 files
Verified signature9feb615 -
Update gateway.py
Verified signaturec5d0cc7 -
Update start.sh
Verified signatured18c573 -
Update gateway.py
Verified signature17f3c93 -
Upload 4 files
Verified signaturee841cbd -
Update Dockerfile
Verified signature88f4721 -
Create Dockerfile
Verified signature1fe2763 -
Create README.md
Verified signatured449a1c -
initial commit
Verified signature3cb4823