imkrish commited on
Commit
8003382
Β·
verified Β·
1 Parent(s): 0ebe358

Deploy: postgres + ngrok tunnel + gradio watcher + backups

Browse files
Files changed (7) hide show
  1. .env.example +23 -0
  2. Dockerfile +43 -0
  3. README.md +121 -2
  4. app/backup.py +153 -0
  5. app/main.py +233 -0
  6. app/requirements.txt +5 -0
  7. start.sh +110 -0
.env.example ADDED
@@ -0,0 +1,23 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # Copy to .env for local runs, or set these as Hugging Face Space secrets.
2
+
3
+ # Required for a public connection URL (free: https://dashboard.ngrok.com)
4
+ NGROK_AUTHTOKEN=
5
+
6
+ # Recommended: locks the web UI behind Basic auth (username: admin)
7
+ APP_PASSWORD=
8
+
9
+ # Optional: fixed Postgres credentials (otherwise password is auto-generated each boot)
10
+ POSTGRES_USER=demo
11
+ POSTGRES_DB=demo
12
+ POSTGRES_PASSWORD=
13
+
14
+ # --- Backups (so data survives restarts) ---
15
+ # Off-Space durable backups to a PRIVATE HF Dataset repo (works on the free tier).
16
+ # Create a write token at https://huggingface.co/settings/tokens
17
+ HF_TOKEN=
18
+ HF_BACKUP_REPO=your-username/pg-backups
19
+
20
+ # Tuning (optional)
21
+ BACKUP_INTERVAL_MIN=30
22
+ BACKUP_KEEP=24
23
+ # BACKUP_DIR is auto-chosen: /data/backups if persistent storage is on, else local.
Dockerfile ADDED
@@ -0,0 +1,43 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ FROM python:3.11-slim
2
+
3
+ ENV DEBIAN_FRONTEND=noninteractive
4
+
5
+ # PostgreSQL + tools we need at runtime
6
+ RUN apt-get update && apt-get install -y --no-install-recommends \
7
+ postgresql postgresql-contrib \
8
+ curl ca-certificates bash \
9
+ && rm -rf /var/lib/apt/lists/*
10
+
11
+ # ngrok v3 (gives us a public TCP endpoint for Postgres) β€” pick the right arch
12
+ RUN ARCH="$(dpkg --print-architecture)" \
13
+ && curl -fsSL "https://bin.equinox.io/c/bNyj1mQVY4c/ngrok-v3-stable-linux-${ARCH}.tgz" \
14
+ -o /tmp/ngrok.tgz \
15
+ && tar -xzf /tmp/ngrok.tgz -C /usr/local/bin \
16
+ && rm /tmp/ngrok.tgz \
17
+ && chmod +x /usr/local/bin/ngrok
18
+
19
+ # HF Spaces runs the container as uid 1000 β€” create a matching user
20
+ RUN useradd -m -u 1000 appuser
21
+ ENV HOME=/home/appuser
22
+
23
+ WORKDIR /app
24
+ COPY app/requirements.txt /app/requirements.txt
25
+ RUN pip install --no-cache-dir -r /app/requirements.txt
26
+
27
+ COPY app /app
28
+ COPY start.sh /app/start.sh
29
+ RUN chmod +x /app/start.sh \
30
+ && mkdir -p /home/appuser/data \
31
+ && chown -R appuser:appuser /home/appuser /app
32
+
33
+ USER appuser
34
+
35
+ ENV PGDATA=/home/appuser/data/pgdata \
36
+ POSTGRES_USER=demo \
37
+ POSTGRES_DB=demo \
38
+ PGPORT=5432 \
39
+ APP_PORT=7860
40
+
41
+ EXPOSE 7860
42
+
43
+ CMD ["/app/start.sh"]
README.md CHANGED
@@ -1,10 +1,129 @@
1
  ---
2
  title: Remote Postgres
3
- emoji: 🐠
4
  colorFrom: indigo
5
  colorTo: blue
6
  sdk: docker
 
7
  pinned: false
8
  ---
9
 
10
- Check out the configuration reference at https://huggingface.co/docs/hub/spaces-config-reference
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
  title: Remote Postgres
3
+ emoji: 🐘
4
  colorFrom: indigo
5
  colorTo: blue
6
  sdk: docker
7
+ app_port: 7860
8
  pinned: false
9
  ---
10
 
11
+ # Remote Postgres on Hugging Face Spaces
12
+
13
+ A single Docker container that runs **PostgreSQL** + a small **web UI** that hands you a
14
+ **public connection URL** to drop into any demo project that needs a Postgres database.
15
+
16
+ Because Hugging Face Spaces only exposes one HTTP port (no raw TCP), the container opens an
17
+ outbound **ngrok TCP tunnel** so Postgres is reachable from anywhere with a normal
18
+ `postgresql://…` URL and any client/ORM.
19
+
20
+ > ⚠️ **Ephemeral & throwaway.** Data is wiped on every restart/rebuild, and the public URL
21
+ > changes each time. Great for demos; do not use for anything you need to keep.
22
+
23
+ ---
24
+
25
+ ## Deploy (5 minutes)
26
+
27
+ 1. **Get a free ngrok authtoken** β€” sign up at <https://dashboard.ngrok.com>, copy *Your Authtoken*.
28
+ 2. **Create a Space** β†’ New Space β†’ **SDK: Docker** β†’ (recommended) **Visibility: Private**.
29
+ 3. **Push these files** to the Space repo (or upload them in the web UI):
30
+ `Dockerfile`, `start.sh`, `README.md`, and the `app/` folder.
31
+ 4. **Add Space secrets** (Settings β†’ *Variables and secrets*):
32
+ | Secret | Required | Purpose |
33
+ |---|---|---|
34
+ | `NGROK_AUTHTOKEN` | βœ… | Opens the public TCP tunnel for Postgres |
35
+ | `APP_PASSWORD` | recommended | Locks the web UI (Basic auth, user `admin`) so only you see the URL |
36
+ | `POSTGRES_PASSWORD` | optional | Fixed DB password (otherwise one is generated per boot) |
37
+ | `POSTGRES_USER` / `POSTGRES_DB` | optional | Defaults: `demo` / `demo` |
38
+ 5. The Space builds and starts. Open it β†’ the UI shows your **connection URL** + a Copy button.
39
+
40
+ ## Use it
41
+
42
+ Open the Space, copy the `postgresql://…` URL, and use it anywhere:
43
+
44
+ ```bash
45
+ psql "postgresql://demo:PASSWORD@0.tcp.ngrok.io:12345/demo"
46
+ ```
47
+
48
+ ```python
49
+ import psycopg
50
+ conn = psycopg.connect("postgresql://demo:PASSWORD@0.tcp.ngrok.io:12345/demo")
51
+ ```
52
+
53
+ Works the same with SQLAlchemy, Prisma, Drizzle, node-postgres, etc.
54
+
55
+ ## Endpoints
56
+
57
+ - `/` β€” **Gradio uptime-watcher dashboard**: live Postgres/tunnel status, container uptime,
58
+ availability %, recent-checks table, keep-alive counter, and the copyable connection URL.
59
+ Login-protected (user `admin`) when `APP_PASSWORD` is set.
60
+ - `/api/connection` β€” JSON with the URL + fields (also protected).
61
+ - `/keepalive?src=<label>` β€” open; bumps the keep-alive counter and logs the hit. Hit by the
62
+ in-container loop (`src=self`) and the external cron (`src=github-actions`).
63
+ - `/health` β€” open status check: `{"status","postgres","tunnel","uptime"}`.
64
+
65
+ ## Don't lose the data (persistence)
66
+
67
+ The container stacks two independent safeguards β€” use either or both:
68
+
69
+ 1. **Persistent volume (cleanest).** Enable **persistent storage** on the Space
70
+ (Settings β†’ Storage). It mounts a writable `/data`; on boot the container detects it and
71
+ stores the live Postgres cluster at `/data/pgdata`, so data survives restarts as-is. Zero
72
+ extra config. (Paid HF add-on.)
73
+ 2. **Backups + auto-restore (works on free tier).** Every `BACKUP_INTERVAL_MIN` minutes the
74
+ DB is `pg_dump`ed to `BACKUP_DIR`, and β€” if you set `HF_TOKEN` + `HF_BACKUP_REPO` β€” mirrored
75
+ to a **private HF Dataset repo** (durable off-Space). On a fresh boot the container
76
+ **auto-restores the latest dump**, so even a wiped ephemeral Space comes back with your data
77
+ (you lose at most the last interval). There's also a **Back up now** button on the dashboard.
78
+
79
+ To enable off-Space backups, add Space secrets:
80
+ | Secret | Example | Purpose |
81
+ |---|---|---|
82
+ | `HF_TOKEN` | `hf_xxx` (write) | Lets the Space push/pull dumps β€” [create one](https://huggingface.co/settings/tokens) |
83
+ | `HF_BACKUP_REPO` | `your-name/pg-backups` | Private dataset repo (auto-created) holding the dumps |
84
+
85
+ Optional: `BACKUP_INTERVAL_MIN` (default 30), `BACKUP_KEEP` (default 24 local dumps).
86
+
87
+ > Recommended combo for "I don't want to risk it" on free tier: **HF Dataset backups every
88
+ > 15–30 min** + the external hourly keep-alive below. Data is safe across restarts; only the
89
+ > ngrok URL still rotates (re-copy it from the dashboard after a restart).
90
+
91
+ ## Keeping the Space awake
92
+
93
+ Free Spaces sleep after **48h with no HTTP traffic to the web app**. Your Postgres queries go
94
+ through ngrok and do **not** count β€” only requests to this app's web port do. Two layers keep
95
+ it awake:
96
+
97
+ 1. **In-container self-ping (automatic).** A loop hits `https://$SPACE_HOST/keepalive` every
98
+ 20 min and logs each hit. Keeps the Space from ever going idle *while the container runs* β€”
99
+ but it cannot wake a Space that already slept (it's asleep too).
100
+ 2. **External hourly cron (recommended).** An outside request also **wakes** a slept Space.
101
+ Pick one:
102
+ - **GitHub Actions** β€” [`.github/workflows/keepalive.yml`](.github/workflows/keepalive.yml)
103
+ is included. Push this repo to GitHub, add a repo secret **`SPACE_URL`** =
104
+ `https://<your-space>.hf.space`, and it pings every hour. (Note: GitHub disables scheduled
105
+ workflows after 60 days with no commits β€” push occasionally, or use the option below.)
106
+ - **No-code:** [cron-job.org](https://cron-job.org) or [UptimeRobot](https://uptimerobot.com)
107
+ β†’ GET `https://<your-space>.hf.space/keepalive` every hour. Truly set-and-forget.
108
+
109
+ You can watch the **Keep-alive hits** counter on the dashboard to confirm pings are landing.
110
+
111
+ > A sleep/wake or any rebuild still wipes data and rotates the URL β€” keep-alive only keeps the
112
+ > Space *running*; it does not make the data or the URL permanent.
113
+
114
+ ## Keep it secure
115
+
116
+ - Set the Space to **Private** and set **`APP_PASSWORD`** so the URL isn't world-readable.
117
+ - The DB itself is internet-exposed via the tunnel β€” anyone with the URL **and** password can
118
+ connect. Use a strong `POSTGRES_PASSWORD` and treat the data as disposable.
119
+
120
+ ## Local test
121
+
122
+ ```bash
123
+ docker build -t remote-pg .
124
+ docker run -p 7860:7860 \
125
+ -e NGROK_AUTHTOKEN=your_token \
126
+ -e APP_PASSWORD=letmein \
127
+ remote-pg
128
+ # open http://localhost:7860
129
+ ```
app/backup.py ADDED
@@ -0,0 +1,153 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """Backup / restore for the in-container Postgres.
2
+
3
+ Two durable targets, either or both:
4
+ * BACKUP_DIR β€” local dir (durable only if it's on HF persistent /data)
5
+ * HF_BACKUP_REPO β€” a private HF Dataset repo (durable even on the free tier)
6
+
7
+ A fresh boot calls `restore()` to pull the latest dump back; a background scheduler
8
+ calls `backup()` every BACKUP_INTERVAL_MIN minutes.
9
+ """
10
+ import os
11
+ import sys
12
+ import glob
13
+ import gzip
14
+ import time
15
+ import threading
16
+ import subprocess
17
+ from datetime import datetime
18
+
19
+ PG_USER = os.environ.get("POSTGRES_USER", "demo")
20
+ PG_DB = os.environ.get("POSTGRES_DB", "demo")
21
+ PG_PORT = os.environ.get("PGPORT", "5432")
22
+ PGHOST = "/tmp"
23
+
24
+ BACKUP_DIR = os.environ.get("BACKUP_DIR", "/home/appuser/backups")
25
+ KEEP = int(os.environ.get("BACKUP_KEEP", "24"))
26
+ INTERVAL_MIN = int(os.environ.get("BACKUP_INTERVAL_MIN", "30"))
27
+
28
+ HF_REPO = os.environ.get("HF_BACKUP_REPO", "").strip() # e.g. "user/pg-backups"
29
+ HF_TOKEN = os.environ.get("HF_TOKEN", "").strip()
30
+ HF_LATEST = "backups/latest.sql.gz"
31
+
32
+ # Surfaced on the dashboard.
33
+ STATE = {"count": 0, "last": None, "last_name": None, "error": None,
34
+ "dir": BACKUP_DIR, "hf": bool(HF_REPO and HF_TOKEN)}
35
+
36
+
37
+ def _ts():
38
+ return datetime.now().strftime("%Y%m%d-%H%M%S")
39
+
40
+
41
+ def _hf_api():
42
+ from huggingface_hub import HfApi
43
+ return HfApi(token=HF_TOKEN)
44
+
45
+
46
+ def ensure_repo():
47
+ if not (HF_REPO and HF_TOKEN):
48
+ return
49
+ try:
50
+ _hf_api().create_repo(repo_id=HF_REPO, repo_type="dataset",
51
+ private=True, exist_ok=True)
52
+ print(f"[backup] HF dataset ready: {HF_REPO}", flush=True)
53
+ except Exception as e:
54
+ print(f"[backup] create_repo failed: {e}", flush=True)
55
+
56
+
57
+ def backup():
58
+ """Dump the DB to a gzip file locally, rotate, and mirror to HF (latest)."""
59
+ os.makedirs(BACKUP_DIR, exist_ok=True)
60
+ name = f"backup-{_ts()}.sql.gz"
61
+ path = os.path.join(BACKUP_DIR, name)
62
+ try:
63
+ dump = subprocess.run(
64
+ ["pg_dump", "-h", PGHOST, "-p", PG_PORT, "-U", PG_USER,
65
+ "-d", PG_DB, "--clean", "--if-exists"],
66
+ stdout=subprocess.PIPE, check=True,
67
+ ).stdout
68
+ with gzip.open(path, "wb") as f:
69
+ f.write(dump)
70
+
71
+ # keep only the newest KEEP local dumps
72
+ for old in sorted(glob.glob(os.path.join(BACKUP_DIR, "backup-*.sql.gz")))[:-KEEP]:
73
+ try:
74
+ os.remove(old)
75
+ except OSError:
76
+ pass
77
+
78
+ if HF_REPO and HF_TOKEN:
79
+ try:
80
+ _hf_api().upload_file(
81
+ path_or_fileobj=path, path_in_repo=HF_LATEST,
82
+ repo_id=HF_REPO, repo_type="dataset",
83
+ commit_message=f"backup {name}",
84
+ )
85
+ except Exception as e:
86
+ print(f"[backup] HF upload failed: {e}", flush=True)
87
+
88
+ STATE.update(count=STATE["count"] + 1, last=datetime.now(),
89
+ last_name=name, error=None)
90
+ print(f"[backup] wrote {name} ({os.path.getsize(path)} bytes)", flush=True)
91
+ return path
92
+ except Exception as e:
93
+ STATE["error"] = str(e)
94
+ print(f"[backup] FAILED: {e}", flush=True)
95
+ raise
96
+
97
+
98
+ def restore():
99
+ """Restore the most recent dump (prefers HF, falls back to local). Returns bool."""
100
+ src = None
101
+ if HF_REPO and HF_TOKEN:
102
+ try:
103
+ from huggingface_hub import hf_hub_download
104
+ src = hf_hub_download(HF_REPO, HF_LATEST, repo_type="dataset",
105
+ token=HF_TOKEN)
106
+ print(f"[backup] fetched latest dump from HF dataset {HF_REPO}", flush=True)
107
+ except Exception as e:
108
+ print(f"[backup] no HF backup to restore ({e})", flush=True)
109
+ if src is None:
110
+ local = sorted(glob.glob(os.path.join(BACKUP_DIR, "backup-*.sql.gz")))
111
+ src = local[-1] if local else None
112
+ if not src:
113
+ print("[backup] no backup found β€” starting empty", flush=True)
114
+ return False
115
+
116
+ print(f"[backup] restoring from {src}", flush=True)
117
+ with gzip.open(src, "rb") as f:
118
+ subprocess.run(
119
+ ["psql", "-h", PGHOST, "-p", PG_PORT, "-U", PG_USER,
120
+ "-d", PG_DB, "-v", "ON_ERROR_STOP=0", "-q"],
121
+ input=f.read(), check=True,
122
+ )
123
+ print("[backup] restore complete", flush=True)
124
+ return True
125
+
126
+
127
+ def _loop():
128
+ while True:
129
+ time.sleep(INTERVAL_MIN * 60)
130
+ try:
131
+ backup()
132
+ except Exception:
133
+ pass
134
+
135
+
136
+ def start_scheduler():
137
+ """Called from the web app: ensure the HF repo exists and start periodic backups."""
138
+ ensure_repo()
139
+ threading.Thread(target=_loop, daemon=True).start()
140
+ print(f"[backup] scheduler started β€” every {INTERVAL_MIN} min, "
141
+ f"dir={BACKUP_DIR}, hf={'on' if STATE['hf'] else 'off'}", flush=True)
142
+
143
+
144
+ if __name__ == "__main__":
145
+ cmd = sys.argv[1] if len(sys.argv) > 1 else ""
146
+ if cmd == "backup":
147
+ backup()
148
+ elif cmd == "restore":
149
+ restore()
150
+ elif cmd == "ensure-repo":
151
+ ensure_repo()
152
+ else:
153
+ print("usage: backup.py [backup|restore|ensure-repo]")
app/main.py ADDED
@@ -0,0 +1,233 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ import os
2
+ import time
3
+ import secrets
4
+ import threading
5
+ from datetime import datetime, timedelta
6
+
7
+ import httpx
8
+ import psycopg
9
+ import gradio as gr
10
+
11
+ import backup
12
+ from fastapi import FastAPI, Depends, HTTPException, status
13
+ from fastapi.responses import JSONResponse
14
+ from fastapi.security import HTTPBasic, HTTPBasicCredentials
15
+
16
+ PG_USER = os.environ.get("POSTGRES_USER", "demo")
17
+ PG_PASSWORD = os.environ.get("POSTGRES_PASSWORD", "")
18
+ PG_DB = os.environ.get("POSTGRES_DB", "demo")
19
+ PG_PORT = os.environ.get("PGPORT", "5432")
20
+
21
+ # If set, the Gradio UI and the JSON API require this password (username: admin).
22
+ APP_PASSWORD = os.environ.get("APP_PASSWORD", "")
23
+ APP_USER = os.environ.get("APP_USER", "admin")
24
+
25
+ NGROK_API = "http://127.0.0.1:4040/api/tunnels"
26
+
27
+ BOOT = datetime.now()
28
+ HISTORY: list[tuple[str, bool, bool]] = [] # (HH:MM:SS, postgres_up, tunnel_up)
29
+ HISTORY_MAX = 500
30
+
31
+ # Keep-alive counter (incremented by the in-container loop AND any external cron).
32
+ KEEPALIVE = {"count": 0, "last": None, "src": "β€”"}
33
+
34
+
35
+ # ---------------------------------------------------------------- helpers ----
36
+ def get_tunnel():
37
+ """Return (host, port) of the public ngrok TCP tunnel, or None."""
38
+ try:
39
+ r = httpx.get(NGROK_API, timeout=3)
40
+ for t in r.json().get("tunnels", []):
41
+ if t.get("proto") == "tcp":
42
+ host, port = t["public_url"].split("://", 1)[1].split(":")
43
+ return host, port
44
+ except Exception:
45
+ return None
46
+ return None
47
+
48
+
49
+ def pg_ok() -> bool:
50
+ try:
51
+ with psycopg.connect(
52
+ host="/tmp", port=PG_PORT, user=PG_USER,
53
+ dbname=PG_DB, password=PG_PASSWORD, connect_timeout=3,
54
+ ):
55
+ return True
56
+ except Exception:
57
+ return False
58
+
59
+
60
+ def build_url(tunnel):
61
+ if not tunnel:
62
+ return None
63
+ host, port = tunnel
64
+ return f"postgresql://{PG_USER}:{PG_PASSWORD}@{host}:{port}/{PG_DB}"
65
+
66
+
67
+ def humanize(delta: timedelta) -> str:
68
+ s = int(delta.total_seconds())
69
+ d, s = divmod(s, 86400)
70
+ h, s = divmod(s, 3600)
71
+ m, s = divmod(s, 60)
72
+ if d:
73
+ return f"{d}d {h}h {m}m"
74
+ if h:
75
+ return f"{h}h {m}m {s}s"
76
+ return f"{m}m {s}s"
77
+
78
+
79
+ def record_check():
80
+ ok = pg_ok()
81
+ tn = get_tunnel() is not None
82
+ HISTORY.append((datetime.now().strftime("%H:%M:%S"), ok, tn))
83
+ del HISTORY[:-HISTORY_MAX]
84
+ return ok, tn
85
+
86
+
87
+ def _watcher_loop():
88
+ while True:
89
+ try:
90
+ record_check()
91
+ except Exception:
92
+ pass
93
+ time.sleep(30)
94
+
95
+
96
+ # Background uptime watcher β€” records a sample every 30s while the container lives.
97
+ threading.Thread(target=_watcher_loop, daemon=True).start()
98
+
99
+ # Periodic Postgres backups (to BACKUP_DIR and/or a private HF Dataset).
100
+ backup.start_scheduler()
101
+
102
+
103
+ # ------------------------------------------------------------- JSON / API ----
104
+ api = FastAPI(title="Remote Postgres", docs_url=None, redoc_url=None)
105
+ security = HTTPBasic(auto_error=False)
106
+
107
+
108
+ def require_auth(credentials: HTTPBasicCredentials | None = Depends(security)):
109
+ if not APP_PASSWORD:
110
+ return
111
+ if credentials is None or not (
112
+ secrets.compare_digest(credentials.username, APP_USER)
113
+ and secrets.compare_digest(credentials.password, APP_PASSWORD)
114
+ ):
115
+ raise HTTPException(
116
+ status_code=status.HTTP_401_UNAUTHORIZED,
117
+ detail="Authentication required",
118
+ headers={"WWW-Authenticate": "Basic"},
119
+ )
120
+
121
+
122
+ @api.get("/health")
123
+ async def health():
124
+ """Open endpoint β€” used by the self keep-alive and external uptime monitors."""
125
+ tunnel = get_tunnel()
126
+ ok = pg_ok()
127
+ return JSONResponse({
128
+ "status": "ok" if ok else "degraded",
129
+ "postgres": ok,
130
+ "tunnel": bool(tunnel),
131
+ "uptime": humanize(datetime.now() - BOOT),
132
+ })
133
+
134
+
135
+ @api.get("/keepalive")
136
+ async def keepalive(src: str = "self"):
137
+ """Hit to reset HF's idle timer (by the in-container loop AND any external cron).
138
+ Open + logged so you can confirm pings are landing."""
139
+ KEEPALIVE["count"] += 1
140
+ KEEPALIVE["last"] = datetime.now()
141
+ KEEPALIVE["src"] = src
142
+ print(f"[keepalive] hit #{KEEPALIVE['count']} from '{src}' at "
143
+ f"{KEEPALIVE['last'].strftime('%Y-%m-%d %H:%M:%S')}", flush=True)
144
+ return {"ok": True, "count": KEEPALIVE["count"], "src": src}
145
+
146
+
147
+ @api.get("/api/connection")
148
+ async def connection(_: None = Depends(require_auth)):
149
+ tunnel = get_tunnel()
150
+ host, port = tunnel if tunnel else (None, None)
151
+ return {
152
+ "postgres_up": pg_ok(),
153
+ "connection_url": build_url(tunnel),
154
+ "host": host, "port": port,
155
+ "user": PG_USER, "password": PG_PASSWORD, "database": PG_DB,
156
+ }
157
+
158
+
159
+ # ------------------------------------------------- Gradio uptime watcher ----
160
+ def snapshot():
161
+ tunnel = get_tunnel()
162
+ up = pg_ok()
163
+ url = build_url(tunnel) or "β€” no public tunnel yet (set NGROK_AUTHTOKEN) β€”"
164
+ host, port = tunnel if tunnel else ("β€”", "οΏ½οΏ½")
165
+
166
+ samples = len(HISTORY)
167
+ pct = (100 * sum(1 for _, o, _ in HISTORY if o) / samples) if samples else 100.0
168
+ status_md = (
169
+ f"### Status\n"
170
+ f"- **Postgres:** {'🟒 up' if up else 'πŸ”΄ down'}\n"
171
+ f"- **Public tunnel:** {'🟒 up' if tunnel else '🟑 off'}\n"
172
+ f"- **Container uptime:** {humanize(datetime.now() - BOOT)}\n"
173
+ f"- **PG availability (this container):** {pct:.1f}% Β· {samples} checks\n"
174
+ f"- **Keep-alive hits:** {KEEPALIVE['count']} Β· last from "
175
+ f"`{KEEPALIVE['src']}` at "
176
+ f"{KEEPALIVE['last'].strftime('%H:%M:%S') if KEEPALIVE['last'] else 'never'}\n"
177
+ f"- **Backups:** {backup.STATE['count']} taken Β· last: "
178
+ f"{backup.STATE['last'].strftime('%H:%M:%S') if backup.STATE['last'] else 'none yet'}"
179
+ f" Β· HF off-Space: {'on' if backup.STATE['hf'] else 'off'}\n"
180
+ f"- **Last checked:** {datetime.now().strftime('%H:%M:%S')}"
181
+ )
182
+ table = [
183
+ [t, "βœ…" if o else "❌", "βœ…" if n else "❌"]
184
+ for (t, o, n) in reversed(HISTORY[-25:])
185
+ ]
186
+ return status_md, url, host, port, PG_USER, PG_PASSWORD, PG_DB, table
187
+
188
+
189
+ with gr.Blocks(title="Remote Postgres β€” Uptime Watcher", theme=gr.themes.Soft()) as demo:
190
+ gr.Markdown("# 🐘 Remote Postgres β€” Uptime Watcher")
191
+ gr.Markdown(
192
+ "Throwaway Postgres for demo projects, exposed over an ngrok TCP tunnel. "
193
+ "Data is **ephemeral** and the URL **changes on every restart**."
194
+ )
195
+ status_box = gr.Markdown()
196
+ url_box = gr.Textbox(
197
+ label="Connection URL β€” paste into your projects",
198
+ interactive=False, show_copy_button=True,
199
+ )
200
+ with gr.Row():
201
+ host_box = gr.Textbox(label="Host", interactive=False)
202
+ port_box = gr.Textbox(label="Port", interactive=False)
203
+ with gr.Row():
204
+ user_box = gr.Textbox(label="User", interactive=False)
205
+ pw_box = gr.Textbox(label="Password", interactive=False)
206
+ db_box = gr.Textbox(label="Database", interactive=False)
207
+ history_box = gr.Dataframe(
208
+ headers=["time", "postgres", "tunnel"],
209
+ label="Recent checks (newest first)",
210
+ interactive=False,
211
+ )
212
+ with gr.Row():
213
+ refresh_btn = gr.Button("Refresh now", variant="primary")
214
+ backup_btn = gr.Button("Back up now")
215
+
216
+ def backup_now():
217
+ try:
218
+ backup.backup()
219
+ except Exception as e:
220
+ print(f"[ui] manual backup failed: {e}", flush=True)
221
+ return snapshot()
222
+
223
+ outputs = [status_box, url_box, host_box, port_box,
224
+ user_box, pw_box, db_box, history_box]
225
+ timer = gr.Timer(10)
226
+ timer.tick(snapshot, outputs=outputs)
227
+ refresh_btn.click(snapshot, outputs=outputs)
228
+ backup_btn.click(backup_now, outputs=outputs)
229
+ demo.load(snapshot, outputs=outputs)
230
+
231
+
232
+ auth = (APP_USER, APP_PASSWORD) if APP_PASSWORD else None
233
+ app = gr.mount_gradio_app(api, demo, path="/", auth=auth)
app/requirements.txt ADDED
@@ -0,0 +1,5 @@
 
 
 
 
 
 
1
+ gradio==4.44.1
2
+ huggingface_hub==0.25.2
3
+ fastapi==0.112.2
4
+ psycopg[binary]==3.1.19
5
+ httpx==0.27.0
start.sh ADDED
@@ -0,0 +1,110 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ #!/usr/bin/env bash
2
+ set -euo pipefail
3
+
4
+ # Postgres client/server binaries live in /usr/lib/postgresql/<ver>/bin on Debian
5
+ export PATH="$(ls -d /usr/lib/postgresql/*/bin 2>/dev/null | head -n1):$PATH"
6
+
7
+ : "${POSTGRES_USER:=demo}"
8
+ : "${POSTGRES_DB:=demo}"
9
+ : "${PGPORT:=5432}"
10
+ : "${APP_PORT:=7860}"
11
+ export PGHOST=/tmp # local clients connect over the unix socket in /tmp (trust auth)
12
+
13
+ # ---- pick data + backup locations ----
14
+ # HF persistent storage (if enabled) mounts a writable /data owned by uid 1000.
15
+ # When present we keep BOTH the live cluster AND backups there so they survive restarts.
16
+ if [ -d /data ] && [ -w /data ]; then
17
+ echo "[start] Persistent /data detected β€” using it for the cluster and backups."
18
+ export PGDATA="${PGDATA:-/data/pgdata}"
19
+ export BACKUP_DIR="${BACKUP_DIR:-/data/backups}"
20
+ else
21
+ echo "[start] No persistent /data β€” cluster is EPHEMERAL; relying on backup/restore."
22
+ export PGDATA="${PGDATA:-/home/appuser/data/pgdata}"
23
+ export BACKUP_DIR="${BACKUP_DIR:-/home/appuser/backups}"
24
+ fi
25
+ mkdir -p "$BACKUP_DIR"
26
+
27
+ # Generate a strong password if the user didn't supply one as a Space secret
28
+ if [ -z "${POSTGRES_PASSWORD:-}" ]; then
29
+ POSTGRES_PASSWORD="$(python3 -c 'import secrets;print(secrets.token_urlsafe(18))')"
30
+ echo "[start] No POSTGRES_PASSWORD secret set β€” generated one for this session."
31
+ fi
32
+ export POSTGRES_PASSWORD POSTGRES_USER POSTGRES_DB PGPORT BACKUP_DIR
33
+
34
+ FRESH_INIT=0
35
+
36
+ # ---- initialize the cluster (only when the data dir is empty) ----
37
+ if [ ! -s "$PGDATA/PG_VERSION" ]; then
38
+ FRESH_INIT=1
39
+ echo "[start] Initializing PostgreSQL cluster at $PGDATA"
40
+ mkdir -p "$PGDATA"
41
+ chmod 700 "$PGDATA"
42
+ PWFILE="$(mktemp)"
43
+ printf '%s' "$POSTGRES_PASSWORD" > "$PWFILE"
44
+ initdb -D "$PGDATA" \
45
+ --auth-host=scram-sha-256 \
46
+ --auth-local=trust \
47
+ -U "$POSTGRES_USER" \
48
+ --pwfile="$PWFILE" >/dev/null
49
+ rm -f "$PWFILE"
50
+
51
+ {
52
+ echo "listen_addresses = '*'"
53
+ echo "port = $PGPORT"
54
+ echo "unix_socket_directories = '/tmp'"
55
+ } >> "$PGDATA/postgresql.conf"
56
+
57
+ {
58
+ echo "host all all 0.0.0.0/0 scram-sha-256"
59
+ echo "host all all ::/0 scram-sha-256"
60
+ } >> "$PGDATA/pg_hba.conf"
61
+ fi
62
+
63
+ # ---- start postgres ----
64
+ echo "[start] Starting PostgreSQL on port $PGPORT"
65
+ pg_ctl -D "$PGDATA" -o "-p $PGPORT" -w -l "$PGDATA/server.log" start
66
+
67
+ # ensure the target database exists
68
+ if ! psql -p "$PGPORT" -U "$POSTGRES_USER" -d postgres -tAc \
69
+ "SELECT 1 FROM pg_database WHERE datname='$POSTGRES_DB'" | grep -q 1; then
70
+ echo "[start] Creating database '$POSTGRES_DB'"
71
+ createdb -p "$PGPORT" -U "$POSTGRES_USER" "$POSTGRES_DB"
72
+ fi
73
+
74
+ # ---- restore the latest backup onto a freshly-initialized cluster ----
75
+ # On a fresh boot (ephemeral disk, or first run) pull back the most recent dump so the
76
+ # data "survives" restarts. Skipped when the cluster already had data (persistent disk).
77
+ if [ "$FRESH_INIT" = "1" ]; then
78
+ echo "[start] Fresh cluster β€” checking for a backup to restore"
79
+ python /app/backup.py restore || echo "[start] no restore performed"
80
+ fi
81
+
82
+ # ---- expose Postgres publicly via ngrok TCP tunnel ----
83
+ if [ -n "${NGROK_AUTHTOKEN:-}" ]; then
84
+ echo "[start] Configuring ngrok and opening TCP tunnel for port $PGPORT"
85
+ ngrok config add-authtoken "$NGROK_AUTHTOKEN" >/dev/null 2>&1 || true
86
+ ngrok tcp "$PGPORT" --log=stdout > "$HOME/ngrok.log" 2>&1 &
87
+ else
88
+ echo "[start] NGROK_AUTHTOKEN not set β€” Postgres is only reachable INSIDE the Space."
89
+ echo "[start] Add a free ngrok authtoken as a Space secret to get a public URL."
90
+ fi
91
+
92
+ # ---- keep the Space awake (free Spaces sleep after 48h with NO http traffic) ----
93
+ # Postgres/ngrok traffic does NOT count β€” only requests to this web app do. So we hit
94
+ # the PUBLIC url (goes through HF's router => resets the idle timer). SPACE_HOST is
95
+ # injected by HF, e.g. user-space.hf.space. Each hit is logged to the Space logs.
96
+ if [ -n "${SPACE_HOST:-}" ]; then
97
+ echo "[start] Self keep-alive enabled β€” pinging https://${SPACE_HOST}/keepalive every 20m"
98
+ ( while true; do
99
+ sleep 1200
100
+ ts="$(date '+%Y-%m-%d %H:%M:%S')"
101
+ code="$(curl -fsS -o /dev/null -w '%{http_code}' \
102
+ "https://${SPACE_HOST}/keepalive?src=self" 2>/dev/null || echo 000)"
103
+ echo "[keepalive] self ${ts} -> ${code}"
104
+ done ) &
105
+ fi
106
+
107
+ # ---- start the web UI (foreground; keeps the container alive) ----
108
+ echo "[start] Starting web UI on port $APP_PORT"
109
+ cd /app
110
+ exec uvicorn main:app --host 0.0.0.0 --port "$APP_PORT"