Spaces:
Running
Running
Hermes Agent commited on
Commit Β·
98d2cb8
1
Parent(s): c3b879f
Refine architecture: Add Redis server, OmniRoute multi-port listeners (20128/20129/20132), safe SQLite backup, and OMNIROUTE_BASE_PATH
Browse files- ARCHITECTURE.md +55 -69
- Dockerfile +6 -3
- entrypoint.sh +24 -14
- gateway/omniroute.py +12 -5
- gateway/utils.py +15 -11
ARCHITECTURE.md
CHANGED
|
@@ -6,17 +6,21 @@ This document details the software architecture, network routing, multi-service
|
|
| 6 |
|
| 7 |
## 1. System Overview
|
| 8 |
|
| 9 |
-
**Opencode-Cli** is
|
| 10 |
-
|
| 11 |
-
1. **FastAPI Reverse Proxy & Gateway Router**: Ingress routing,
|
| 12 |
-
2. **Open WebUI**: Primary ChatGPT-style
|
| 13 |
-
3. **OmniRoute AI Gateway**:
|
| 14 |
-
|
| 15 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
| 16 |
|
| 17 |
---
|
| 18 |
|
| 19 |
-
## 2.
|
| 20 |
|
| 21 |
```mermaid
|
| 22 |
flowchart TD
|
|
@@ -24,19 +28,22 @@ flowchart TD
|
|
| 24 |
|
| 25 |
subgraph Container ["π³ Docker Container (Debian Bookworm Slim)"]
|
| 26 |
Ingress -->|Path: / | OWUI[Open WebUI - Port 8098]
|
| 27 |
-
Ingress -->|Path: /omniroute |
|
| 28 |
-
Ingress -->|Path: /v1 , /v1beta |
|
| 29 |
-
Ingress -->|Path: /
|
| 30 |
-
Ingress -->|Path: /
|
| 31 |
-
|
| 32 |
-
|
|
|
|
|
|
|
|
|
|
| 33 |
TG -->|Auto-Generate .strm Files| JF
|
| 34 |
end
|
| 35 |
|
| 36 |
-
subgraph Persistence ["πΎ Persistence
|
| 37 |
-
Ext4["Local Container Ext4 (/root/.omniroute)"] <-->|
|
| 38 |
-
|
| 39 |
-
OWUI -->|User
|
| 40 |
JF -->|Media & Metadata| HFData
|
| 41 |
end
|
| 42 |
```
|
|
@@ -49,37 +56,42 @@ flowchart TD
|
|
| 49 |
- **Port**: `4096` (Exposed as container entrypoint)
|
| 50 |
- **Role**:
|
| 51 |
- Handles single-port ingress on Hugging Face Spaces.
|
| 52 |
-
-
|
| 53 |
-
-
|
| 54 |
-
- Proxies HTTP requests and bidirectional WebSockets (`/ws/socket.io`).
|
| 55 |
|
| 56 |
### 3.2 Open WebUI (`open-webui`)
|
| 57 |
- **Port**: `8098`
|
| 58 |
- **Path**: `/`
|
| 59 |
- **Environment**:
|
| 60 |
-
- `OPENAI_API_BASE_URL="http://127.0.0.1:
|
| 61 |
- `ENABLE_OPENAI_API="true"`
|
| 62 |
- `RAG_AUTO_UPDATE_INDEX="false"`
|
| 63 |
-
- **Optimization**: SentenceTransformer embeddings (`all-MiniLM-L6-v2`)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 64 |
|
| 65 |
-
### 3.
|
| 66 |
-
- **Port**: `
|
| 67 |
-
- **
|
| 68 |
-
- **API Endpoints**: `/v1` (OpenAI format), `/v1beta` (Gemini format), and 21 specific backend management routes (`/api/providers`, `/api/combos`, etc.).
|
| 69 |
-
- **Engine**: Node.js 22 LTS + Next.js.
|
| 70 |
-
- **Auto-Fix**: `fix_omniroute.py` auto-resolves numeric migration version collisions (e.g., version 143 collisions) at build time and container boot.
|
| 71 |
|
| 72 |
-
### 3.
|
| 73 |
- **Port**: `8096`
|
| 74 |
- **Path**: `/jellyfin`
|
| 75 |
-
- **Data Location**: `/data/jellyfin/data`, `/data/jellyfin/media/Movies`
|
| 76 |
- **Engine**: Jellyfin 10.11 + FFmpeg 5.1.
|
| 77 |
|
| 78 |
-
### 3.
|
| 79 |
- **Port**: `8080`
|
| 80 |
- **Path**: `/tg-stream`
|
| 81 |
- **Engine**: Python Pyrogram MTProto Client.
|
| 82 |
-
- **Function**: Automatically syncs Telegram channel media into `.strm` files inside Jellyfin's movie directory.
|
| 83 |
|
| 84 |
---
|
| 85 |
|
|
@@ -88,13 +100,10 @@ flowchart TD
|
|
| 88 |
| Public Request Path | Target Upstream Port | Target Service | Function |
|
| 89 |
| :--- | :--- | :--- | :--- |
|
| 90 |
| `/` | `8098` | Open WebUI | Main Web UI |
|
| 91 |
-
| `/
|
| 92 |
-
| `/
|
| 93 |
-
| `/
|
| 94 |
-
| `/
|
| 95 |
-
| `/v1beta/*` | `20128` | OmniRoute AI Gateway | Gemini API Endpoint |
|
| 96 |
-
| `/api/providers`, `/api/combos`, ... | `20128` | OmniRoute AI Gateway | OmniRoute Management APIs |
|
| 97 |
-
| `/api/config`, `/api/v1/*` | `8098` | Open WebUI | Open WebUI Core APIs |
|
| 98 |
| `/jellyfin/*` | `8096` | Jellyfin Media Server | Media Streaming |
|
| 99 |
| `/tg-stream/*` | `8080` | TG Stream Proxy | Telegram Direct Streamer |
|
| 100 |
|
|
@@ -102,30 +111,8 @@ flowchart TD
|
|
| 102 |
|
| 103 |
## 5. Storage & Persistence Model
|
| 104 |
|
| 105 |
-
```
|
| 106 |
-
|
| 107 |
-
β Hugging Face Network Mount (/data) β
|
| 108 |
-
β (FUSE Volume - Persistent Across Boots) β
|
| 109 |
-
ββββββββββββββββββββββββ¬ββββββββββββββββββββββββ
|
| 110 |
-
β
|
| 111 |
-
ββββββββββββββββββββ΄βββββββββββββββββββ
|
| 112 |
-
β β
|
| 113 |
-
ββββββββββββΌβββββββββββ ββββββββββββΌβββββββββββ
|
| 114 |
-
β /data/open-webui β β /data/omniroute β
|
| 115 |
-
β (webui.db) β β (storage.sqlite) β
|
| 116 |
-
βββββββββββββββββββββββ ββββββββββββ²βββββββββββ
|
| 117 |
-
β
|
| 118 |
-
(Background Sync 25s)
|
| 119 |
-
β
|
| 120 |
-
ββββββββββββ΄βββββββββββ
|
| 121 |
-
β /root/.omniroute β
|
| 122 |
-
β (Active Local Ext4) β
|
| 123 |
-
βββββββββββββββββββββββ
|
| 124 |
-
```
|
| 125 |
-
|
| 126 |
-
- **HF FUSE Mount (`/data`)**: Persistent volume provided by Hugging Face Spaces.
|
| 127 |
-
- **Local Ext4 Disk (`/root/.omniroute`)**: Stores OmniRoute's active `better-sqlite3` database file to prevent SQLite `disk I/O error` on FUSE mounts.
|
| 128 |
-
- **Background Sync Daemon**: Runs every 25 seconds in `entrypoint.sh` to copy `/root/.omniroute/storage.sqlite` and `server.env` to `/data/omniroute/`.
|
| 129 |
|
| 130 |
---
|
| 131 |
|
|
@@ -133,15 +120,14 @@ flowchart TD
|
|
| 133 |
|
| 134 |
```
|
| 135 |
.
|
| 136 |
-
βββ Dockerfile # Multi-stage Docker image
|
| 137 |
-
βββ ARCHITECTURE.md # Architecture specification (this
|
| 138 |
βββ README.md # User quickstart guide
|
| 139 |
-
βββ entrypoint.sh # Concurrent service startup
|
| 140 |
βββ fix_omniroute.py # Migration version collision auto-repair script
|
| 141 |
-
βββ nginx.conf # Nginx configuration
|
| 142 |
βββ proxy.py # FastAPI gateway entrypoint script
|
| 143 |
βββ tg_streamer.py # Pyrogram Telegram 5G stream proxy
|
| 144 |
-
βββ index.html # Custom splash landing page
|
| 145 |
βββ gateway/ # FastAPI Modular Router Package
|
| 146 |
βββ __init__.py
|
| 147 |
βββ main.py # App router aggregator
|
|
|
|
| 6 |
|
| 7 |
## 1. System Overview
|
| 8 |
|
| 9 |
+
**Opencode-Cli** is an enterprise-grade multi-service web container running on Debian Bookworm Slim. It orchestrates six integrated services into a unified web application exposed via a single public ingress port (`4096`):
|
| 10 |
+
|
| 11 |
+
1. **FastAPI Reverse Proxy & Gateway Router**: Ingress path-routing, CORS, WebSocket proxying, and service isolation.
|
| 12 |
+
2. **Open WebUI**: Primary ChatGPT-style frontend pre-configured to use OmniRoute's dedicated API server (`127.0.0.1:20129/v1`).
|
| 13 |
+
3. **OmniRoute v3.8.50 Multi-Port AI Gateway**:
|
| 14 |
+
- **Dashboard**: Port `20128` (Web UI & settings at `/omniroute`)
|
| 15 |
+
- **Dedicated API Server**: Port `20129` (OpenAI `/v1` & Gemini `/v1beta` compatibility)
|
| 16 |
+
- **Live WebSocket Server**: Port `20132` (Real-time monitoring)
|
| 17 |
+
4. **Redis Cache & Rate Limiter**: Port `6379` (In-memory cache and distributed rate limiting for OmniRoute).
|
| 18 |
+
5. **Jellyfin Media Server**: Port `8096` (Media streaming server at `/jellyfin`).
|
| 19 |
+
6. **Telegram 5G Stream Proxy**: Port `8080` (High-speed MTProto streaming daemon at `/tg-stream`).
|
| 20 |
|
| 21 |
---
|
| 22 |
|
| 23 |
+
## 2. Production Architecture Diagram
|
| 24 |
|
| 25 |
```mermaid
|
| 26 |
flowchart TD
|
|
|
|
| 28 |
|
| 29 |
subgraph Container ["π³ Docker Container (Debian Bookworm Slim)"]
|
| 30 |
Ingress -->|Path: / | OWUI[Open WebUI - Port 8098]
|
| 31 |
+
Ingress -->|Path: /omniroute/* | OmniDash[OmniRoute Dashboard - Port 20128]
|
| 32 |
+
Ingress -->|Path: /v1 , /v1beta | OmniAPI[OmniRoute Dedicated API - Port 20129]
|
| 33 |
+
Ingress -->|Path: /omniroute/live-ws/* | OmniWS[OmniRoute Live WS - Port 20132]
|
| 34 |
+
Ingress -->|Path: /jellyfin/* | JF[Jellyfin Media Server - Port 8096]
|
| 35 |
+
Ingress -->|Path: /tg-stream/* | TG[Telegram 5G Streamer - Port 8080]
|
| 36 |
+
|
| 37 |
+
OWUI -->|Internal OpenAI API /v1| OmniAPI
|
| 38 |
+
OmniDash -->|Distributed Cache & Limits| Redis[(Redis Server - Port 6379)]
|
| 39 |
+
OmniAPI -->|Distributed Cache & Limits| Redis
|
| 40 |
TG -->|Auto-Generate .strm Files| JF
|
| 41 |
end
|
| 42 |
|
| 43 |
+
subgraph Persistence ["πΎ Persistence & Storage Model"]
|
| 44 |
+
Ext4["Local Container Ext4 (/root/.omniroute)"] <-->|SQLite .backup API (Every 5 mins)| HFData["HF FUSE Persistent Mount (/data)"]
|
| 45 |
+
OmniDash -->|Fast POSIX SQLite Locks| Ext4
|
| 46 |
+
OWUI -->|User Accounts & DB| HFData
|
| 47 |
JF -->|Media & Metadata| HFData
|
| 48 |
end
|
| 49 |
```
|
|
|
|
| 56 |
- **Port**: `4096` (Exposed as container entrypoint)
|
| 57 |
- **Role**:
|
| 58 |
- Handles single-port ingress on Hugging Face Spaces.
|
| 59 |
+
- Namespace path-routing with strict boundary enforcement.
|
| 60 |
+
- Proxies HTTP requests and bidirectional WebSockets (`/ws/socket.io` & `/omniroute/live-ws`).
|
|
|
|
| 61 |
|
| 62 |
### 3.2 Open WebUI (`open-webui`)
|
| 63 |
- **Port**: `8098`
|
| 64 |
- **Path**: `/`
|
| 65 |
- **Environment**:
|
| 66 |
+
- `OPENAI_API_BASE_URL="http://127.0.0.1:20129/v1"`
|
| 67 |
- `ENABLE_OPENAI_API="true"`
|
| 68 |
- `RAG_AUTO_UPDATE_INDEX="false"`
|
| 69 |
+
- **Optimization**: SentenceTransformer embeddings (`all-MiniLM-L6-v2`) pre-cached in Docker image for 1-second startup.
|
| 70 |
+
|
| 71 |
+
### 3.3 OmniRoute AI Gateway v3.8.50
|
| 72 |
+
- **Ports**:
|
| 73 |
+
- **Dashboard**: `20128` (Path `/omniroute`)
|
| 74 |
+
- **Dedicated API**: `20129` (Endpoints `/v1`, `/v1beta`)
|
| 75 |
+
- **Live WebSocket**: `20132` (Path `/omniroute/live-ws`)
|
| 76 |
+
- **Environment**:
|
| 77 |
+
- `OMNIROUTE_BASE_PATH="/omniroute"`
|
| 78 |
+
- `REDIS_URL="redis://127.0.0.1:6379"`
|
| 79 |
+
- `DATA_DIR="/root/.omniroute"`
|
| 80 |
+
- **Auto-Fix**: `fix_omniroute.py` auto-resolves numeric migration version collisions (e.g. version 143 collisions) at build time and container boot.
|
| 81 |
|
| 82 |
+
### 3.4 Redis Server (`redis-server`)
|
| 83 |
+
- **Port**: `6379`
|
| 84 |
+
- **Role**: In-memory rate limiting and distributed caching for OmniRoute.
|
|
|
|
|
|
|
|
|
|
| 85 |
|
| 86 |
+
### 3.5 Jellyfin Media Server (`jellyfin`)
|
| 87 |
- **Port**: `8096`
|
| 88 |
- **Path**: `/jellyfin`
|
|
|
|
| 89 |
- **Engine**: Jellyfin 10.11 + FFmpeg 5.1.
|
| 90 |
|
| 91 |
+
### 3.6 Telegram Direct Stream Proxy (`tg_streamer.py`)
|
| 92 |
- **Port**: `8080`
|
| 93 |
- **Path**: `/tg-stream`
|
| 94 |
- **Engine**: Python Pyrogram MTProto Client.
|
|
|
|
| 95 |
|
| 96 |
---
|
| 97 |
|
|
|
|
| 100 |
| Public Request Path | Target Upstream Port | Target Service | Function |
|
| 101 |
| :--- | :--- | :--- | :--- |
|
| 102 |
| `/` | `8098` | Open WebUI | Main Web UI |
|
| 103 |
+
| `/omniroute/*` | `20128` | OmniRoute Dashboard | Management Panel |
|
| 104 |
+
| `/v1/*` | `20129` | OmniRoute API Server | Dedicated OpenAI API |
|
| 105 |
+
| `/v1beta/*` | `20129` | OmniRoute API Server | Dedicated Gemini API |
|
| 106 |
+
| `/omniroute/live-ws/*` | `20132` | OmniRoute WebSocket | Live Monitoring WS |
|
|
|
|
|
|
|
|
|
|
| 107 |
| `/jellyfin/*` | `8096` | Jellyfin Media Server | Media Streaming |
|
| 108 |
| `/tg-stream/*` | `8080` | TG Stream Proxy | Telegram Direct Streamer |
|
| 109 |
|
|
|
|
| 111 |
|
| 112 |
## 5. Storage & Persistence Model
|
| 113 |
|
| 114 |
+
- **Active Execution (`/root/.omniroute/storage.sqlite`)**: Runs on local Ext4 container disk to support `better-sqlite3` POSIX file locking.
|
| 115 |
+
- **Safe SQLite Backup (`/data/omniroute/storage.sqlite`)**: Background daemon uses `sqlite3 .backup` API every 5 minutes and on shutdown to write consistent snapshots to the HF FUSE mount.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 116 |
|
| 117 |
---
|
| 118 |
|
|
|
|
| 120 |
|
| 121 |
```
|
| 122 |
.
|
| 123 |
+
βββ Dockerfile # Multi-stage Docker image with Redis & pre-cached models
|
| 124 |
+
βββ ARCHITECTURE.md # Architecture specification (this document)
|
| 125 |
βββ README.md # User quickstart guide
|
| 126 |
+
βββ entrypoint.sh # Concurrent service startup, Redis daemon & SQLite backup
|
| 127 |
βββ fix_omniroute.py # Migration version collision auto-repair script
|
| 128 |
+
βββ nginx.conf # Nginx configuration reference
|
| 129 |
βββ proxy.py # FastAPI gateway entrypoint script
|
| 130 |
βββ tg_streamer.py # Pyrogram Telegram 5G stream proxy
|
|
|
|
| 131 |
βββ gateway/ # FastAPI Modular Router Package
|
| 132 |
βββ __init__.py
|
| 133 |
βββ main.py # App router aggregator
|
Dockerfile
CHANGED
|
@@ -24,6 +24,7 @@ RUN apt-get update && apt-get install -y --no-install-recommends \
|
|
| 24 |
g++ \
|
| 25 |
sqlite3 \
|
| 26 |
libsqlite3-dev \
|
|
|
|
| 27 |
&& rm -rf /var/lib/apt/lists/*
|
| 28 |
|
| 29 |
# 2. Add Official Jellyfin APT Repository & Install Jellyfin Server, Web UI, and FFmpeg
|
|
@@ -44,12 +45,14 @@ RUN python3 -c "from sentence_transformers import SentenceTransformer; SentenceT
|
|
| 44 |
RUN curl -fsSL https://nodejs.org/dist/v22.22.2/node-v22.22.2-linux-x64.tar.gz \
|
| 45 |
| tar -xz -C /usr/local --strip-components=1
|
| 46 |
|
| 47 |
-
# 5. Clone and
|
| 48 |
WORKDIR /omniroute
|
| 49 |
ENV NEXT_TELEMETRY_DISABLED=1
|
| 50 |
ENV OMNIROUTE_USE_TURBOPACK=0
|
| 51 |
-
ENV NODE_OPTIONS="--max-old-space-size=
|
| 52 |
ENV DISABLE_ESLINT_PLUGIN=true
|
|
|
|
|
|
|
| 53 |
|
| 54 |
COPY fix_omniroute.py /fix_omniroute.py
|
| 55 |
|
|
@@ -57,7 +60,7 @@ RUN git clone --depth 1 https://github.com/diegosouzapw/OmniRoute.git /omniroute
|
|
| 57 |
&& python3 /fix_omniroute.py /omniroute \
|
| 58 |
&& npm install --legacy-peer-deps --no-audit --no-fund --prefer-offline \
|
| 59 |
&& npm rebuild better-sqlite3 --build-from-source \
|
| 60 |
-
&& NEXT_TELEMETRY_DISABLED=1 OMNIROUTE_USE_TURBOPACK=0 NODE_OPTIONS="--max-old-space-size=2560" npm run build || true
|
| 61 |
|
| 62 |
RUN mkdir -p /root/.cache /data/cache /data/omniroute /data/open-webui
|
| 63 |
RUN chmod -R 777 /root/.cache /data/cache /omniroute
|
|
|
|
| 24 |
g++ \
|
| 25 |
sqlite3 \
|
| 26 |
libsqlite3-dev \
|
| 27 |
+
redis-server \
|
| 28 |
&& rm -rf /var/lib/apt/lists/*
|
| 29 |
|
| 30 |
# 2. Add Official Jellyfin APT Repository & Install Jellyfin Server, Web UI, and FFmpeg
|
|
|
|
| 45 |
RUN curl -fsSL https://nodejs.org/dist/v22.22.2/node-v22.22.2-linux-x64.tar.gz \
|
| 46 |
| tar -xz -C /usr/local --strip-components=1
|
| 47 |
|
| 48 |
+
# 5. Clone and Build OmniRoute AI Gateway (with OMNIROUTE_BASE_PATH=/omniroute)
|
| 49 |
WORKDIR /omniroute
|
| 50 |
ENV NEXT_TELEMETRY_DISABLED=1
|
| 51 |
ENV OMNIROUTE_USE_TURBOPACK=0
|
| 52 |
+
ENV NODE_OPTIONS="--max-old-space-size=2560"
|
| 53 |
ENV DISABLE_ESLINT_PLUGIN=true
|
| 54 |
+
ENV OMNIROUTE_BASE_PATH="/omniroute"
|
| 55 |
+
ENV NEXT_PUBLIC_BASE_PATH="/omniroute"
|
| 56 |
|
| 57 |
COPY fix_omniroute.py /fix_omniroute.py
|
| 58 |
|
|
|
|
| 60 |
&& python3 /fix_omniroute.py /omniroute \
|
| 61 |
&& npm install --legacy-peer-deps --no-audit --no-fund --prefer-offline \
|
| 62 |
&& npm rebuild better-sqlite3 --build-from-source \
|
| 63 |
+
&& OMNIROUTE_BASE_PATH="/omniroute" NEXT_PUBLIC_BASE_PATH="/omniroute" NEXT_TELEMETRY_DISABLED=1 OMNIROUTE_USE_TURBOPACK=0 NODE_OPTIONS="--max-old-space-size=2560" npm run build || true
|
| 64 |
|
| 65 |
RUN mkdir -p /root/.cache /data/cache /data/omniroute /data/open-webui
|
| 66 |
RUN chmod -R 777 /root/.cache /data/cache /omniroute
|
entrypoint.sh
CHANGED
|
@@ -32,24 +32,34 @@ fi
|
|
| 32 |
# Clean up stale lock files
|
| 33 |
rm -f /root/.omniroute/*.sqlite-wal /root/.omniroute/*.sqlite-shm /root/.omniroute/*.lock 2>/dev/null || true
|
| 34 |
|
| 35 |
-
#
|
|
|
|
|
|
|
|
|
|
|
|
|
| 36 |
(
|
| 37 |
while true; do
|
| 38 |
-
sleep
|
| 39 |
-
if [ -f "/root/.omniroute/storage.sqlite" ]; then
|
| 40 |
-
|
| 41 |
-
fi
|
| 42 |
-
if [ -f "/root/.omniroute/server.env" ]; then
|
| 43 |
-
cp -f /root/.omniroute/server.env /data/omniroute/server.env 2>/dev/null || true
|
| 44 |
fi
|
| 45 |
done
|
| 46 |
) &
|
| 47 |
|
| 48 |
-
# Start OmniRoute AI Gateway
|
| 49 |
-
echo "[INIT] Starting OmniRoute AI Gateway
|
| 50 |
export PORT=20128
|
|
|
|
|
|
|
| 51 |
export HOSTNAME="127.0.0.1"
|
| 52 |
export DATA_DIR="/root/.omniroute"
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 53 |
if [ -d "/omniroute" ]; then
|
| 54 |
cd /omniroute
|
| 55 |
# Fix any upstream migration version collisions (e.g. version 143 collisions)
|
|
@@ -63,11 +73,11 @@ if [ -d "/omniroute" ]; then
|
|
| 63 |
npm run start -- --port 20128 &
|
| 64 |
else
|
| 65 |
echo "[INIT] Building OmniRoute production assets to resolve WASM modules..."
|
| 66 |
-
NEXT_TELEMETRY_DISABLED=1 OMNIROUTE_USE_TURBOPACK=0 NODE_OPTIONS="--max-old-space-size=2560" npm run build 2>/dev/null || true
|
| 67 |
if [ -d ".build/next" ] || [ -d ".next" ]; then
|
| 68 |
npm run start -- --port 20128 &
|
| 69 |
else
|
| 70 |
-
npx next dev --port 20128 -H 127.0.0.1 &
|
| 71 |
fi
|
| 72 |
fi
|
| 73 |
echo "[INIT] OmniRoute AI Gateway started in background."
|
|
@@ -102,11 +112,11 @@ mkdir -p /root/.cache /data/cache 2>/dev/null || true
|
|
| 102 |
chmod -R 777 /root/.cache /data/cache 2>/dev/null || true
|
| 103 |
|
| 104 |
# Pre-configure & Start Open WebUI on port 8098 (Mounted at / via FastAPI proxy)
|
| 105 |
-
echo "[INIT] Starting Open WebUI on port 8098 pre-configured with OmniRoute..."
|
| 106 |
if command -v open-webui >/dev/null 2>&1; then
|
| 107 |
-
#
|
| 108 |
export WEBUI_URL="${WEBUI_URL:-https://jishnupg-opencode-cli.hf.space}"
|
| 109 |
-
export OPENAI_API_BASE_URL="http://127.0.0.1:
|
| 110 |
export OPENAI_API_KEY="omniroute"
|
| 111 |
export WEBUI_SECRET_KEY="${WEBUI_SECRET_KEY:-opencode_webui_jwt_secret_2026}"
|
| 112 |
export ENABLE_OLLAMA_API="${ENABLE_OLLAMA_API:-false}"
|
|
|
|
| 32 |
# Clean up stale lock files
|
| 33 |
rm -f /root/.omniroute/*.sqlite-wal /root/.omniroute/*.sqlite-shm /root/.omniroute/*.lock 2>/dev/null || true
|
| 34 |
|
| 35 |
+
# Start Redis server for OmniRoute distributed caching and rate limiting
|
| 36 |
+
echo "[INIT] Starting Redis server on port 6379..."
|
| 37 |
+
redis-server --daemonize yes --bind 127.0.0.1 --port 6379 2>/dev/null || echo "[WARN] Redis server startup warning"
|
| 38 |
+
|
| 39 |
+
# Safe SQLite Backup Process (Uses sqlite3 .backup API every 5 mins to prevent live DB corruption)
|
| 40 |
(
|
| 41 |
while true; do
|
| 42 |
+
sleep 300
|
| 43 |
+
if [ -f "/root/.omniroute/storage.sqlite" ] && command -v sqlite3 >/dev/null 2>&1; then
|
| 44 |
+
sqlite3 /root/.omniroute/storage.sqlite ".backup /data/omniroute/storage.sqlite" 2>/dev/null || true
|
|
|
|
|
|
|
|
|
|
| 45 |
fi
|
| 46 |
done
|
| 47 |
) &
|
| 48 |
|
| 49 |
+
# Start OmniRoute AI Gateway (Multi-Port: Dashboard: 20128, Dedicated API: 20129, Live WS: 20132)
|
| 50 |
+
echo "[INIT] Starting OmniRoute AI Gateway (Dashboard: 20128, API: 20129, WS: 20132)..."
|
| 51 |
export PORT=20128
|
| 52 |
+
export API_PORT=20129
|
| 53 |
+
export LIVE_WS_PORT=20132
|
| 54 |
export HOSTNAME="127.0.0.1"
|
| 55 |
export DATA_DIR="/root/.omniroute"
|
| 56 |
+
export REDIS_URL="redis://127.0.0.1:6379"
|
| 57 |
+
export OMNIROUTE_BASE_PATH="/omniroute"
|
| 58 |
+
export NEXT_PUBLIC_BASE_URL="${WEBUI_URL:-https://jishnupg-opencode-cli.hf.space}/omniroute"
|
| 59 |
+
export OMNIROUTE_WS_BRIDGE_SECRET="${OMNIROUTE_WS_BRIDGE_SECRET:-opencode_ws_bridge_secret_2026}"
|
| 60 |
+
export JWT_SECRET="${JWT_SECRET:-opencode_jwt_secret_2026}"
|
| 61 |
+
export API_KEY_SECRET="${API_KEY_SECRET:-opencode_api_key_secret_2026}"
|
| 62 |
+
|
| 63 |
if [ -d "/omniroute" ]; then
|
| 64 |
cd /omniroute
|
| 65 |
# Fix any upstream migration version collisions (e.g. version 143 collisions)
|
|
|
|
| 73 |
npm run start -- --port 20128 &
|
| 74 |
else
|
| 75 |
echo "[INIT] Building OmniRoute production assets to resolve WASM modules..."
|
| 76 |
+
OMNIROUTE_BASE_PATH="/omniroute" NEXT_PUBLIC_BASE_PATH="/omniroute" NEXT_TELEMETRY_DISABLED=1 OMNIROUTE_USE_TURBOPACK=0 NODE_OPTIONS="--max-old-space-size=2560" npm run build 2>/dev/null || true
|
| 77 |
if [ -d ".build/next" ] || [ -d ".next" ]; then
|
| 78 |
npm run start -- --port 20128 &
|
| 79 |
else
|
| 80 |
+
OMNIROUTE_BASE_PATH="/omniroute" npx next dev --port 20128 -H 127.0.0.1 &
|
| 81 |
fi
|
| 82 |
fi
|
| 83 |
echo "[INIT] OmniRoute AI Gateway started in background."
|
|
|
|
| 112 |
chmod -R 777 /root/.cache /data/cache 2>/dev/null || true
|
| 113 |
|
| 114 |
# Pre-configure & Start Open WebUI on port 8098 (Mounted at / via FastAPI proxy)
|
| 115 |
+
echo "[INIT] Starting Open WebUI on port 8098 pre-configured with OmniRoute API..."
|
| 116 |
if command -v open-webui >/dev/null 2>&1; then
|
| 117 |
+
# Point OpenAI API to OmniRoute dedicated API port 20129
|
| 118 |
export WEBUI_URL="${WEBUI_URL:-https://jishnupg-opencode-cli.hf.space}"
|
| 119 |
+
export OPENAI_API_BASE_URL="http://127.0.0.1:20129/v1"
|
| 120 |
export OPENAI_API_KEY="omniroute"
|
| 121 |
export WEBUI_SECRET_KEY="${WEBUI_SECRET_KEY:-opencode_webui_jwt_secret_2026}"
|
| 122 |
export ENABLE_OLLAMA_API="${ENABLE_OLLAMA_API:-false}"
|
gateway/omniroute.py
CHANGED
|
@@ -89,17 +89,24 @@ async def omniroute_main_route(request: Request, path: str = ""):
|
|
| 89 |
|
| 90 |
return res
|
| 91 |
|
| 92 |
-
# ββ 2. OpenAI & Gemini Compatibility APIs ββββββββββ
|
| 93 |
@router.api_route("/v1", methods=["GET", "POST", "PUT", "DELETE", "PATCH", "HEAD", "OPTIONS"])
|
| 94 |
@router.api_route("/v1/{path:path}", methods=["GET", "POST", "PUT", "DELETE", "PATCH", "HEAD", "OPTIONS"])
|
| 95 |
async def omniroute_v1_api(request: Request, path: str = ""):
|
| 96 |
-
target = f"http://127.0.0.1:{
|
| 97 |
-
return await proxy_http_request(target, request, default_prefix="
|
| 98 |
|
| 99 |
@router.api_route("/v1beta/{path:path}", methods=["GET", "POST", "PUT", "DELETE", "PATCH", "HEAD", "OPTIONS"])
|
| 100 |
async def omniroute_v1beta_api(request: Request, path: str = ""):
|
| 101 |
-
target = f"http://127.0.0.1:{
|
| 102 |
-
return await proxy_http_request(target, request, default_prefix="
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 103 |
|
| 104 |
# ββ 3. Next.js Static Asset Routing ββββββββββββββββββββββββββββββββββββββββββ
|
| 105 |
@router.api_route("/_next/{path:path}", methods=["GET", "POST", "PUT", "DELETE", "PATCH", "HEAD", "OPTIONS"])
|
|
|
|
| 89 |
|
| 90 |
return res
|
| 91 |
|
| 92 |
+
# ββ 2. OpenAI & Gemini Compatibility APIs (Dedicated API Port 20129) ββββββββββ
|
| 93 |
@router.api_route("/v1", methods=["GET", "POST", "PUT", "DELETE", "PATCH", "HEAD", "OPTIONS"])
|
| 94 |
@router.api_route("/v1/{path:path}", methods=["GET", "POST", "PUT", "DELETE", "PATCH", "HEAD", "OPTIONS"])
|
| 95 |
async def omniroute_v1_api(request: Request, path: str = ""):
|
| 96 |
+
target = f"http://127.0.0.1:{OMNIROUTE_API_PORT}/v1/{path}" if path else f"http://127.0.0.1:{OMNIROUTE_API_PORT}/v1"
|
| 97 |
+
return await proxy_http_request(target, request, default_prefix="")
|
| 98 |
|
| 99 |
@router.api_route("/v1beta/{path:path}", methods=["GET", "POST", "PUT", "DELETE", "PATCH", "HEAD", "OPTIONS"])
|
| 100 |
async def omniroute_v1beta_api(request: Request, path: str = ""):
|
| 101 |
+
target = f"http://127.0.0.1:{OMNIROUTE_API_PORT}/v1beta/{path}"
|
| 102 |
+
return await proxy_http_request(target, request, default_prefix="")
|
| 103 |
+
|
| 104 |
+
# ββ 3. OmniRoute Live Monitoring WebSocket (Dedicated WS Port 20132) ββββββββββ
|
| 105 |
+
@router.websocket("/omniroute/live-ws")
|
| 106 |
+
@router.websocket("/omniroute/live-ws/{path:path}")
|
| 107 |
+
async def omniroute_live_ws(websocket: Request, path: str = ""):
|
| 108 |
+
target_ws = f"ws://127.0.0.1:{OMNIROUTE_WS_PORT}/live-ws/{path}" if path else f"ws://127.0.0.1:{OMNIROUTE_WS_PORT}/live-ws"
|
| 109 |
+
await proxy_websocket_stream(websocket, target_ws)
|
| 110 |
|
| 111 |
# ββ 3. Next.js Static Asset Routing ββββββββββββββββββββββββββββββββββββββββββ
|
| 112 |
@router.api_route("/_next/{path:path}", methods=["GET", "POST", "PUT", "DELETE", "PATCH", "HEAD", "OPTIONS"])
|
gateway/utils.py
CHANGED
|
@@ -19,24 +19,28 @@ from fastapi.responses import StreamingResponse
|
|
| 19 |
logger = logging.getLogger("GatewayUtils")
|
| 20 |
|
| 21 |
# Port definitions
|
| 22 |
-
WEBUI_PORT
|
| 23 |
-
OMNIROUTE_PORT
|
| 24 |
-
|
| 25 |
-
|
| 26 |
-
|
|
|
|
|
|
|
| 27 |
|
| 28 |
PUBLIC_HOST = os.environ.get("PUBLIC_HOST", "jishnupg-opencode-cli.hf.space")
|
| 29 |
PUBLIC_ORIGIN = f"https://{PUBLIC_HOST}"
|
| 30 |
|
| 31 |
-
_INTERNAL_PORTS = {GATEWAY_PORT, WEBUI_PORT, OMNIROUTE_PORT, JELLYFIN_PORT, TG_PORT}
|
| 32 |
_PORT_STRIP_RE = re.compile(r"(https?://[^/:]+):(" + "|".join(str(p) for p in _INTERNAL_PORTS) + r")")
|
| 33 |
|
| 34 |
_PORT_PREFIX_MAP = {
|
| 35 |
-
GATEWAY_PORT:
|
| 36 |
-
OMNIROUTE_PORT:
|
| 37 |
-
|
| 38 |
-
|
| 39 |
-
|
|
|
|
|
|
|
| 40 |
}
|
| 41 |
|
| 42 |
# Shared Async HTTPX Client Instance
|
|
|
|
| 19 |
logger = logging.getLogger("GatewayUtils")
|
| 20 |
|
| 21 |
# Port definitions
|
| 22 |
+
WEBUI_PORT = int(os.environ.get("WEBUI_PORT", 8098))
|
| 23 |
+
OMNIROUTE_PORT = int(os.environ.get("OMNIROUTE_PORT", 20128))
|
| 24 |
+
OMNIROUTE_API_PORT = int(os.environ.get("OMNIROUTE_API_PORT", 20129))
|
| 25 |
+
OMNIROUTE_WS_PORT = int(os.environ.get("OMNIROUTE_WS_PORT", 20132))
|
| 26 |
+
JELLYFIN_PORT = int(os.environ.get("JELLYFIN_PORT", 8096))
|
| 27 |
+
TG_PORT = int(os.environ.get("TG_PORT", 8080))
|
| 28 |
+
GATEWAY_PORT = int(os.environ.get("PORT", 4096))
|
| 29 |
|
| 30 |
PUBLIC_HOST = os.environ.get("PUBLIC_HOST", "jishnupg-opencode-cli.hf.space")
|
| 31 |
PUBLIC_ORIGIN = f"https://{PUBLIC_HOST}"
|
| 32 |
|
| 33 |
+
_INTERNAL_PORTS = {GATEWAY_PORT, WEBUI_PORT, OMNIROUTE_PORT, OMNIROUTE_API_PORT, OMNIROUTE_WS_PORT, JELLYFIN_PORT, TG_PORT}
|
| 34 |
_PORT_STRIP_RE = re.compile(r"(https?://[^/:]+):(" + "|".join(str(p) for p in _INTERNAL_PORTS) + r")")
|
| 35 |
|
| 36 |
_PORT_PREFIX_MAP = {
|
| 37 |
+
GATEWAY_PORT: "",
|
| 38 |
+
OMNIROUTE_PORT: "/omniroute",
|
| 39 |
+
OMNIROUTE_API_PORT: "",
|
| 40 |
+
OMNIROUTE_WS_PORT: "/live-ws",
|
| 41 |
+
WEBUI_PORT: "/openwebui",
|
| 42 |
+
JELLYFIN_PORT: "/jellyfin",
|
| 43 |
+
TG_PORT: "/tg-stream",
|
| 44 |
}
|
| 45 |
|
| 46 |
# Shared Async HTTPX Client Instance
|