| --- |
| title: BioPatch Backend |
| emoji: 🩺 |
| colorFrom: green |
| colorTo: blue |
| sdk: docker |
| app_port: 7860 |
| suggested_hardware: cpu-upgrade |
| suggested_storage: small |
| short_description: Jarvis medical AI for SSI surveillance |
| pinned: false |
| --- |
| |
| # BioPatch Backend (Hugging Face Spaces deployment) |
|
|
| This Space hosts the FastAPI backend, the SQLite store, and the local |
| medical chat brain that powers |
| [BioPatch](https://github.com/Shwagger/Claudecodepam). The React |
| frontend lives on Netlify and talks to this Space over HTTPS. |
|
|
| ## What's running |
|
|
| - **FastAPI** on port 7860. |
| - **scikit-learn RandomForest** — trained at image build time, |
| deterministic, forced to `n_jobs=1` for thread isolation. |
| - **Jarvis** explainable medical AI (ontology, RAG, recommendations, |
| case-retrieval, three-source detection). |
| - **BioMistral-7B Q4_K_M** via `llama-cpp-python`, downloaded on first |
| `POST /api/jarvis/llm/load`. ~4.4 GB on disk, ~6 GB RAM with 8K |
| context. Cached on persistent `/data/models` between restarts. |
|
|
| ## Required hardware |
|
|
| **cpu-upgrade** tier (8 vCPU / 32 GB RAM) — required for BioMistral-7B. |
| The free tier (2 vCPU / 16 GB) OOMs on model load; if you must run on |
| free, override `JARVIS_LLM_REPO`, `JARVIS_LLM_FILE` and |
| `JARVIS_LLM_CHAT_FORMAT` to a 1B–3B model (see fallback below). |
|
|
| ## Persistent storage (`/data`) |
|
|
| Enable **Persistent storage → small** in the Space settings so the |
| following survive container restarts: |
|
|
| - `/data/biopatch.db` — patients, vitals, wound profiles, doctor |
| corrections, Jarvis conversation history. |
| - `/data/wound_photos/` — uploaded wound photos. |
| - `/data/models/` — the BioMistral GGUF (no re-download after restart). |
| - `/data/.cache/huggingface/` — HF Hub cache for any other downloads. |
|
|
| The Dockerfile defaults already point to these paths. The lifespan |
| hook in `main.py` (`_ensure_persistent_dirs`) creates the dirs on |
| startup so the first boot doesn't error out. |
|
|
| ## Required secrets |
|
|
| None for the chat to work — local llama needs no token. |
|
|
| Optional: |
|
|
| ``` |
| HF_TOKEN = <Read scope> # only if you flip JARVIS_LLM_BACKEND=hf |
| JWT_SECRET = <long random> # auto-generated if missing; pin it in prod |
| ``` |
|
|
| ## Space Variables (overrides) |
|
|
| The Dockerfile defaults work as-is on cpu-upgrade with persistent |
| storage. To swap the model or tune perf, set these in |
| **Settings → Variables**: |
|
|
| | Variable | Default | When to override | |
| |---|---|---| |
| | `JARVIS_LLM_REPO` | `MaziyarPanahi/BioMistral-7B-GGUF` | swap model | |
| | `JARVIS_LLM_FILE` | `BioMistral-7B.Q4_K_M.gguf` | swap quant | |
| | `JARVIS_LLM_CHAT_FORMAT` | `mistral-instruct` | must match the GGUF (`llama-3`, `chatml`, …) | |
| | `JARVIS_LLM_CTX` | `8192` | `4096` for tight RAM, `16384` for richer RAG | |
| | `JARVIS_LLM_THREADS` | `6` | match vCPU count − 2 | |
| | `JARVIS_LLM_CACHE` | `/data/models` | `/app/ai/artifacts/models` if no persistent storage | |
| | `DATABASE_URL` | `sqlite:////data/biopatch.db` | `sqlite:////app/data/biopatch.db` if no persistent storage | |
| | `WOUND_PHOTO_DIR` | `/data/wound_photos` | `/app/data/wound_photos` if no persistent storage | |
|
|
| ### Free-tier fallback (no persistent storage) |
|
|
| ``` |
| JARVIS_LLM_REPO=bartowski/Llama-3.2-1B-Instruct-GGUF |
| JARVIS_LLM_FILE=Llama-3.2-1B-Instruct-Q4_K_M.gguf |
| JARVIS_LLM_CHAT_FORMAT=llama-3 |
| JARVIS_LLM_CTX=4096 |
| JARVIS_LLM_THREADS=2 |
| JARVIS_LLM_CACHE=/app/ai/artifacts/models |
| DATABASE_URL=sqlite:////app/data/biopatch.db |
| WOUND_PHOTO_DIR=/app/data/wound_photos |
| ``` |
|
|
| ## Endpoints |
|
|
| - `GET /api/health` |
| - `POST /api/auth/login/json` |
| - `GET /api/jarvis/health` |
| - `POST /api/jarvis/llm/load` (downloads the GGUF on first call) |
| - `POST /api/jarvis/chat` (SSE streaming) |
| - `WS /ws/stream` |
|
|
| Full surface documented in the GitHub repo. |
|
|
| ## Pre-warm the LLM after a restart |
|
|
| ```bash |
| TOKEN=$(curl -s -X POST https://<USER>-biopatch.hf.space/api/auth/login/json \ |
| -H 'Content-Type: application/json' \ |
| -d '{"email":"admin@biopatch.io","password":"BioPatch!Demo2025"}' \ |
| | python3 -c 'import sys,json;print(json.load(sys.stdin)["access_token"])') |
| |
| curl -X POST https://<USER>-biopatch.hf.space/api/jarvis/llm/load \ |
| -H "Authorization: Bearer $TOKEN" |
| ``` |
|
|
| First load downloads ~4.4 GB into `/data/models/`. Subsequent restarts |
| re-use the cached file (~5 s warm start). `GET /api/jarvis/llm/status` |
| reports `loaded: true` once Llama is ready. |
|
|
