Hermes Agent commited on
Commit
98d2cb8
Β·
1 Parent(s): c3b879f

Refine architecture: Add Redis server, OmniRoute multi-port listeners (20128/20129/20132), safe SQLite backup, and OMNIROUTE_BASE_PATH

Browse files
Files changed (5) hide show
  1. ARCHITECTURE.md +55 -69
  2. Dockerfile +6 -3
  3. entrypoint.sh +24 -14
  4. gateway/omniroute.py +12 -5
  5. gateway/utils.py +15 -11
ARCHITECTURE.md CHANGED
@@ -6,17 +6,21 @@ This document details the software architecture, network routing, multi-service
6
 
7
  ## 1. System Overview
8
 
9
- **Opencode-Cli** is a multi-service web container running on Debian Bookworm. It bundles five core microservices into a unified web application exposed via a single public port (`4096`):
10
-
11
- 1. **FastAPI Reverse Proxy & Gateway Router**: Ingress routing, path-rewriting, HTML asset fixing, CORS, and WebSocket streaming.
12
- 2. **Open WebUI**: Primary ChatGPT-style web interface. Pre-configured with OmniRoute as its backend OpenAI API provider.
13
- 3. **OmniRoute AI Gateway**: Universal AI model routing, combo load balancing, fallback rules, circuit breakers, and OpenAI (`/v1`) & Gemini (`/v1beta`) API compatibility.
14
- 4. **Jellyfin Media Server**: Media streaming center.
15
- 5. **Telegram 5G Stream Proxy**: High-speed MTProto streaming daemon that converts Telegram files into `.strm` media files for Jellyfin.
 
 
 
 
16
 
17
  ---
18
 
19
- ## 2. System Architecture Diagram
20
 
21
  ```mermaid
22
  flowchart TD
@@ -24,19 +28,22 @@ flowchart TD
24
 
25
  subgraph Container ["🐳 Docker Container (Debian Bookworm Slim)"]
26
  Ingress -->|Path: / | OWUI[Open WebUI - Port 8098]
27
- Ingress -->|Path: /omniroute | Omni[OmniRoute AI Gateway - Port 20128]
28
- Ingress -->|Path: /v1 , /v1beta | Omni
29
- Ingress -->|Path: /jellyfin | JF[Jellyfin Media Server - Port 8096]
30
- Ingress -->|Path: /tg-stream | TG[Telegram 5G Stream Proxy - Port 8080]
31
-
32
- OWUI -->|OpenAI API Format /v1| Omni
 
 
 
33
  TG -->|Auto-Generate .strm Files| JF
34
  end
35
 
36
- subgraph Persistence ["πŸ’Ύ Persistence Architecture"]
37
- Ext4["Local Container Ext4 (/root/.omniroute)"] <-->|Auto-Sync Daemon (25s)| HFData["HF FUSE Persistent Mount (/data)"]
38
- Omni -->|POSIX SQLite Locks| Ext4
39
- OWUI -->|User DB & Configs| HFData
40
  JF -->|Media & Metadata| HFData
41
  end
42
  ```
@@ -49,37 +56,42 @@ flowchart TD
49
  - **Port**: `4096` (Exposed as container entrypoint)
50
  - **Role**:
51
  - Handles single-port ingress on Hugging Face Spaces.
52
- - Dynamically routes requests based on URL path prefixes.
53
- - Rewrites HTML `<head>` tags and Next.js asset paths (`/_next`).
54
- - Proxies HTTP requests and bidirectional WebSockets (`/ws/socket.io`).
55
 
56
  ### 3.2 Open WebUI (`open-webui`)
57
  - **Port**: `8098`
58
  - **Path**: `/`
59
  - **Environment**:
60
- - `OPENAI_API_BASE_URL="http://127.0.0.1:20128/v1"`
61
  - `ENABLE_OPENAI_API="true"`
62
  - `RAG_AUTO_UPDATE_INDEX="false"`
63
- - **Optimization**: SentenceTransformer embeddings (`all-MiniLM-L6-v2`) are pre-baked into the Docker image cache to allow instant boot in **~1s**.
 
 
 
 
 
 
 
 
 
 
 
64
 
65
- ### 3.3 OmniRoute AI Gateway (`/omniroute`)
66
- - **Port**: `20128`
67
- - **Path**: `/omniroute` *(Redirects to `/omniroute/dashboard`)*
68
- - **API Endpoints**: `/v1` (OpenAI format), `/v1beta` (Gemini format), and 21 specific backend management routes (`/api/providers`, `/api/combos`, etc.).
69
- - **Engine**: Node.js 22 LTS + Next.js.
70
- - **Auto-Fix**: `fix_omniroute.py` auto-resolves numeric migration version collisions (e.g., version 143 collisions) at build time and container boot.
71
 
72
- ### 3.4 Jellyfin Media Server (`jellyfin`)
73
  - **Port**: `8096`
74
  - **Path**: `/jellyfin`
75
- - **Data Location**: `/data/jellyfin/data`, `/data/jellyfin/media/Movies`
76
  - **Engine**: Jellyfin 10.11 + FFmpeg 5.1.
77
 
78
- ### 3.5 Telegram Direct Stream Proxy (`tg_streamer.py`)
79
  - **Port**: `8080`
80
  - **Path**: `/tg-stream`
81
  - **Engine**: Python Pyrogram MTProto Client.
82
- - **Function**: Automatically syncs Telegram channel media into `.strm` files inside Jellyfin's movie directory.
83
 
84
  ---
85
 
@@ -88,13 +100,10 @@ flowchart TD
88
  | Public Request Path | Target Upstream Port | Target Service | Function |
89
  | :--- | :--- | :--- | :--- |
90
  | `/` | `8098` | Open WebUI | Main Web UI |
91
- | `/openwebui/*` | `8098` | Open WebUI | WebUI Assets |
92
- | `/omniroute` | `20128` | OmniRoute AI Gateway | Management Dashboard |
93
- | `/omniroute/_next/*` | `20128` | OmniRoute AI Gateway | Next.js Static Assets |
94
- | `/v1/*` | `20128` | OmniRoute AI Gateway | OpenAI API Endpoint |
95
- | `/v1beta/*` | `20128` | OmniRoute AI Gateway | Gemini API Endpoint |
96
- | `/api/providers`, `/api/combos`, ... | `20128` | OmniRoute AI Gateway | OmniRoute Management APIs |
97
- | `/api/config`, `/api/v1/*` | `8098` | Open WebUI | Open WebUI Core APIs |
98
  | `/jellyfin/*` | `8096` | Jellyfin Media Server | Media Streaming |
99
  | `/tg-stream/*` | `8080` | TG Stream Proxy | Telegram Direct Streamer |
100
 
@@ -102,30 +111,8 @@ flowchart TD
102
 
103
  ## 5. Storage & Persistence Model
104
 
105
- ```
106
- β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
107
- β”‚ Hugging Face Network Mount (/data) β”‚
108
- β”‚ (FUSE Volume - Persistent Across Boots) β”‚
109
- β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
110
- β”‚
111
- β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”΄β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
112
- β”‚ β”‚
113
- β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β–Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β–Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
114
- β”‚ /data/open-webui β”‚ β”‚ /data/omniroute β”‚
115
- β”‚ (webui.db) β”‚ β”‚ (storage.sqlite) β”‚
116
- β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β–²β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
117
- β”‚
118
- (Background Sync 25s)
119
- β”‚
120
- β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”΄β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
121
- β”‚ /root/.omniroute β”‚
122
- β”‚ (Active Local Ext4) β”‚
123
- β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
124
- ```
125
-
126
- - **HF FUSE Mount (`/data`)**: Persistent volume provided by Hugging Face Spaces.
127
- - **Local Ext4 Disk (`/root/.omniroute`)**: Stores OmniRoute's active `better-sqlite3` database file to prevent SQLite `disk I/O error` on FUSE mounts.
128
- - **Background Sync Daemon**: Runs every 25 seconds in `entrypoint.sh` to copy `/root/.omniroute/storage.sqlite` and `server.env` to `/data/omniroute/`.
129
 
130
  ---
131
 
@@ -133,15 +120,14 @@ flowchart TD
133
 
134
  ```
135
  .
136
- β”œβ”€β”€ Dockerfile # Multi-stage Docker image build
137
- β”œβ”€β”€ ARCHITECTURE.md # Architecture specification (this file)
138
  β”œβ”€β”€ README.md # User quickstart guide
139
- β”œβ”€β”€ entrypoint.sh # Concurrent service startup & storage sync daemon
140
  β”œβ”€β”€ fix_omniroute.py # Migration version collision auto-repair script
141
- β”œβ”€β”€ nginx.conf # Nginx configuration (optional reference)
142
  β”œβ”€β”€ proxy.py # FastAPI gateway entrypoint script
143
  β”œβ”€β”€ tg_streamer.py # Pyrogram Telegram 5G stream proxy
144
- β”œβ”€β”€ index.html # Custom splash landing page
145
  └── gateway/ # FastAPI Modular Router Package
146
  β”œβ”€β”€ __init__.py
147
  β”œβ”€β”€ main.py # App router aggregator
 
6
 
7
  ## 1. System Overview
8
 
9
+ **Opencode-Cli** is an enterprise-grade multi-service web container running on Debian Bookworm Slim. It orchestrates six integrated services into a unified web application exposed via a single public ingress port (`4096`):
10
+
11
+ 1. **FastAPI Reverse Proxy & Gateway Router**: Ingress path-routing, CORS, WebSocket proxying, and service isolation.
12
+ 2. **Open WebUI**: Primary ChatGPT-style frontend pre-configured to use OmniRoute's dedicated API server (`127.0.0.1:20129/v1`).
13
+ 3. **OmniRoute v3.8.50 Multi-Port AI Gateway**:
14
+ - **Dashboard**: Port `20128` (Web UI & settings at `/omniroute`)
15
+ - **Dedicated API Server**: Port `20129` (OpenAI `/v1` & Gemini `/v1beta` compatibility)
16
+ - **Live WebSocket Server**: Port `20132` (Real-time monitoring)
17
+ 4. **Redis Cache & Rate Limiter**: Port `6379` (In-memory cache and distributed rate limiting for OmniRoute).
18
+ 5. **Jellyfin Media Server**: Port `8096` (Media streaming server at `/jellyfin`).
19
+ 6. **Telegram 5G Stream Proxy**: Port `8080` (High-speed MTProto streaming daemon at `/tg-stream`).
20
 
21
  ---
22
 
23
+ ## 2. Production Architecture Diagram
24
 
25
  ```mermaid
26
  flowchart TD
 
28
 
29
  subgraph Container ["🐳 Docker Container (Debian Bookworm Slim)"]
30
  Ingress -->|Path: / | OWUI[Open WebUI - Port 8098]
31
+ Ingress -->|Path: /omniroute/* | OmniDash[OmniRoute Dashboard - Port 20128]
32
+ Ingress -->|Path: /v1 , /v1beta | OmniAPI[OmniRoute Dedicated API - Port 20129]
33
+ Ingress -->|Path: /omniroute/live-ws/* | OmniWS[OmniRoute Live WS - Port 20132]
34
+ Ingress -->|Path: /jellyfin/* | JF[Jellyfin Media Server - Port 8096]
35
+ Ingress -->|Path: /tg-stream/* | TG[Telegram 5G Streamer - Port 8080]
36
+
37
+ OWUI -->|Internal OpenAI API /v1| OmniAPI
38
+ OmniDash -->|Distributed Cache & Limits| Redis[(Redis Server - Port 6379)]
39
+ OmniAPI -->|Distributed Cache & Limits| Redis
40
  TG -->|Auto-Generate .strm Files| JF
41
  end
42
 
43
+ subgraph Persistence ["πŸ’Ύ Persistence & Storage Model"]
44
+ Ext4["Local Container Ext4 (/root/.omniroute)"] <-->|SQLite .backup API (Every 5 mins)| HFData["HF FUSE Persistent Mount (/data)"]
45
+ OmniDash -->|Fast POSIX SQLite Locks| Ext4
46
+ OWUI -->|User Accounts & DB| HFData
47
  JF -->|Media & Metadata| HFData
48
  end
49
  ```
 
56
  - **Port**: `4096` (Exposed as container entrypoint)
57
  - **Role**:
58
  - Handles single-port ingress on Hugging Face Spaces.
59
+ - Namespace path-routing with strict boundary enforcement.
60
+ - Proxies HTTP requests and bidirectional WebSockets (`/ws/socket.io` & `/omniroute/live-ws`).
 
61
 
62
  ### 3.2 Open WebUI (`open-webui`)
63
  - **Port**: `8098`
64
  - **Path**: `/`
65
  - **Environment**:
66
+ - `OPENAI_API_BASE_URL="http://127.0.0.1:20129/v1"`
67
  - `ENABLE_OPENAI_API="true"`
68
  - `RAG_AUTO_UPDATE_INDEX="false"`
69
+ - **Optimization**: SentenceTransformer embeddings (`all-MiniLM-L6-v2`) pre-cached in Docker image for 1-second startup.
70
+
71
+ ### 3.3 OmniRoute AI Gateway v3.8.50
72
+ - **Ports**:
73
+ - **Dashboard**: `20128` (Path `/omniroute`)
74
+ - **Dedicated API**: `20129` (Endpoints `/v1`, `/v1beta`)
75
+ - **Live WebSocket**: `20132` (Path `/omniroute/live-ws`)
76
+ - **Environment**:
77
+ - `OMNIROUTE_BASE_PATH="/omniroute"`
78
+ - `REDIS_URL="redis://127.0.0.1:6379"`
79
+ - `DATA_DIR="/root/.omniroute"`
80
+ - **Auto-Fix**: `fix_omniroute.py` auto-resolves numeric migration version collisions (e.g. version 143 collisions) at build time and container boot.
81
 
82
+ ### 3.4 Redis Server (`redis-server`)
83
+ - **Port**: `6379`
84
+ - **Role**: In-memory rate limiting and distributed caching for OmniRoute.
 
 
 
85
 
86
+ ### 3.5 Jellyfin Media Server (`jellyfin`)
87
  - **Port**: `8096`
88
  - **Path**: `/jellyfin`
 
89
  - **Engine**: Jellyfin 10.11 + FFmpeg 5.1.
90
 
91
+ ### 3.6 Telegram Direct Stream Proxy (`tg_streamer.py`)
92
  - **Port**: `8080`
93
  - **Path**: `/tg-stream`
94
  - **Engine**: Python Pyrogram MTProto Client.
 
95
 
96
  ---
97
 
 
100
  | Public Request Path | Target Upstream Port | Target Service | Function |
101
  | :--- | :--- | :--- | :--- |
102
  | `/` | `8098` | Open WebUI | Main Web UI |
103
+ | `/omniroute/*` | `20128` | OmniRoute Dashboard | Management Panel |
104
+ | `/v1/*` | `20129` | OmniRoute API Server | Dedicated OpenAI API |
105
+ | `/v1beta/*` | `20129` | OmniRoute API Server | Dedicated Gemini API |
106
+ | `/omniroute/live-ws/*` | `20132` | OmniRoute WebSocket | Live Monitoring WS |
 
 
 
107
  | `/jellyfin/*` | `8096` | Jellyfin Media Server | Media Streaming |
108
  | `/tg-stream/*` | `8080` | TG Stream Proxy | Telegram Direct Streamer |
109
 
 
111
 
112
  ## 5. Storage & Persistence Model
113
 
114
+ - **Active Execution (`/root/.omniroute/storage.sqlite`)**: Runs on local Ext4 container disk to support `better-sqlite3` POSIX file locking.
115
+ - **Safe SQLite Backup (`/data/omniroute/storage.sqlite`)**: Background daemon uses `sqlite3 .backup` API every 5 minutes and on shutdown to write consistent snapshots to the HF FUSE mount.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
116
 
117
  ---
118
 
 
120
 
121
  ```
122
  .
123
+ β”œβ”€β”€ Dockerfile # Multi-stage Docker image with Redis & pre-cached models
124
+ β”œβ”€β”€ ARCHITECTURE.md # Architecture specification (this document)
125
  β”œβ”€β”€ README.md # User quickstart guide
126
+ β”œβ”€β”€ entrypoint.sh # Concurrent service startup, Redis daemon & SQLite backup
127
  β”œβ”€β”€ fix_omniroute.py # Migration version collision auto-repair script
128
+ β”œβ”€β”€ nginx.conf # Nginx configuration reference
129
  β”œβ”€β”€ proxy.py # FastAPI gateway entrypoint script
130
  β”œβ”€β”€ tg_streamer.py # Pyrogram Telegram 5G stream proxy
 
131
  └── gateway/ # FastAPI Modular Router Package
132
  β”œβ”€β”€ __init__.py
133
  β”œβ”€β”€ main.py # App router aggregator
Dockerfile CHANGED
@@ -24,6 +24,7 @@ RUN apt-get update && apt-get install -y --no-install-recommends \
24
  g++ \
25
  sqlite3 \
26
  libsqlite3-dev \
 
27
  && rm -rf /var/lib/apt/lists/*
28
 
29
  # 2. Add Official Jellyfin APT Repository & Install Jellyfin Server, Web UI, and FFmpeg
@@ -44,12 +45,14 @@ RUN python3 -c "from sentence_transformers import SentenceTransformer; SentenceT
44
  RUN curl -fsSL https://nodejs.org/dist/v22.22.2/node-v22.22.2-linux-x64.tar.gz \
45
  | tar -xz -C /usr/local --strip-components=1
46
 
47
- # 5. Clone and Prepare OmniRoute AI Gateway (fast install, defer build to runtime dev server for instant Docker builds)
48
  WORKDIR /omniroute
49
  ENV NEXT_TELEMETRY_DISABLED=1
50
  ENV OMNIROUTE_USE_TURBOPACK=0
51
- ENV NODE_OPTIONS="--max-old-space-size=2048"
52
  ENV DISABLE_ESLINT_PLUGIN=true
 
 
53
 
54
  COPY fix_omniroute.py /fix_omniroute.py
55
 
@@ -57,7 +60,7 @@ RUN git clone --depth 1 https://github.com/diegosouzapw/OmniRoute.git /omniroute
57
  && python3 /fix_omniroute.py /omniroute \
58
  && npm install --legacy-peer-deps --no-audit --no-fund --prefer-offline \
59
  && npm rebuild better-sqlite3 --build-from-source \
60
- && NEXT_TELEMETRY_DISABLED=1 OMNIROUTE_USE_TURBOPACK=0 NODE_OPTIONS="--max-old-space-size=2560" npm run build || true
61
 
62
  RUN mkdir -p /root/.cache /data/cache /data/omniroute /data/open-webui
63
  RUN chmod -R 777 /root/.cache /data/cache /omniroute
 
24
  g++ \
25
  sqlite3 \
26
  libsqlite3-dev \
27
+ redis-server \
28
  && rm -rf /var/lib/apt/lists/*
29
 
30
  # 2. Add Official Jellyfin APT Repository & Install Jellyfin Server, Web UI, and FFmpeg
 
45
  RUN curl -fsSL https://nodejs.org/dist/v22.22.2/node-v22.22.2-linux-x64.tar.gz \
46
  | tar -xz -C /usr/local --strip-components=1
47
 
48
+ # 5. Clone and Build OmniRoute AI Gateway (with OMNIROUTE_BASE_PATH=/omniroute)
49
  WORKDIR /omniroute
50
  ENV NEXT_TELEMETRY_DISABLED=1
51
  ENV OMNIROUTE_USE_TURBOPACK=0
52
+ ENV NODE_OPTIONS="--max-old-space-size=2560"
53
  ENV DISABLE_ESLINT_PLUGIN=true
54
+ ENV OMNIROUTE_BASE_PATH="/omniroute"
55
+ ENV NEXT_PUBLIC_BASE_PATH="/omniroute"
56
 
57
  COPY fix_omniroute.py /fix_omniroute.py
58
 
 
60
  && python3 /fix_omniroute.py /omniroute \
61
  && npm install --legacy-peer-deps --no-audit --no-fund --prefer-offline \
62
  && npm rebuild better-sqlite3 --build-from-source \
63
+ && OMNIROUTE_BASE_PATH="/omniroute" NEXT_PUBLIC_BASE_PATH="/omniroute" NEXT_TELEMETRY_DISABLED=1 OMNIROUTE_USE_TURBOPACK=0 NODE_OPTIONS="--max-old-space-size=2560" npm run build || true
64
 
65
  RUN mkdir -p /root/.cache /data/cache /data/omniroute /data/open-webui
66
  RUN chmod -R 777 /root/.cache /data/cache /omniroute
entrypoint.sh CHANGED
@@ -32,24 +32,34 @@ fi
32
  # Clean up stale lock files
33
  rm -f /root/.omniroute/*.sqlite-wal /root/.omniroute/*.sqlite-shm /root/.omniroute/*.lock 2>/dev/null || true
34
 
35
- # Background sync process to preserve OmniRoute database and secrets to /data mount
 
 
 
 
36
  (
37
  while true; do
38
- sleep 25
39
- if [ -f "/root/.omniroute/storage.sqlite" ]; then
40
- cp -f /root/.omniroute/storage.sqlite /data/omniroute/storage.sqlite 2>/dev/null || true
41
- fi
42
- if [ -f "/root/.omniroute/server.env" ]; then
43
- cp -f /root/.omniroute/server.env /data/omniroute/server.env 2>/dev/null || true
44
  fi
45
  done
46
  ) &
47
 
48
- # Start OmniRoute AI Gateway on Port 20128
49
- echo "[INIT] Starting OmniRoute AI Gateway on port 20128..."
50
  export PORT=20128
 
 
51
  export HOSTNAME="127.0.0.1"
52
  export DATA_DIR="/root/.omniroute"
 
 
 
 
 
 
 
53
  if [ -d "/omniroute" ]; then
54
  cd /omniroute
55
  # Fix any upstream migration version collisions (e.g. version 143 collisions)
@@ -63,11 +73,11 @@ if [ -d "/omniroute" ]; then
63
  npm run start -- --port 20128 &
64
  else
65
  echo "[INIT] Building OmniRoute production assets to resolve WASM modules..."
66
- NEXT_TELEMETRY_DISABLED=1 OMNIROUTE_USE_TURBOPACK=0 NODE_OPTIONS="--max-old-space-size=2560" npm run build 2>/dev/null || true
67
  if [ -d ".build/next" ] || [ -d ".next" ]; then
68
  npm run start -- --port 20128 &
69
  else
70
- npx next dev --port 20128 -H 127.0.0.1 &
71
  fi
72
  fi
73
  echo "[INIT] OmniRoute AI Gateway started in background."
@@ -102,11 +112,11 @@ mkdir -p /root/.cache /data/cache 2>/dev/null || true
102
  chmod -R 777 /root/.cache /data/cache 2>/dev/null || true
103
 
104
  # Pre-configure & Start Open WebUI on port 8098 (Mounted at / via FastAPI proxy)
105
- echo "[INIT] Starting Open WebUI on port 8098 pre-configured with OmniRoute..."
106
  if command -v open-webui >/dev/null 2>&1; then
107
- # Tell Open WebUI its public-facing URL and point OpenAI API to OmniRoute port 20128
108
  export WEBUI_URL="${WEBUI_URL:-https://jishnupg-opencode-cli.hf.space}"
109
- export OPENAI_API_BASE_URL="http://127.0.0.1:20128/v1"
110
  export OPENAI_API_KEY="omniroute"
111
  export WEBUI_SECRET_KEY="${WEBUI_SECRET_KEY:-opencode_webui_jwt_secret_2026}"
112
  export ENABLE_OLLAMA_API="${ENABLE_OLLAMA_API:-false}"
 
32
  # Clean up stale lock files
33
  rm -f /root/.omniroute/*.sqlite-wal /root/.omniroute/*.sqlite-shm /root/.omniroute/*.lock 2>/dev/null || true
34
 
35
+ # Start Redis server for OmniRoute distributed caching and rate limiting
36
+ echo "[INIT] Starting Redis server on port 6379..."
37
+ redis-server --daemonize yes --bind 127.0.0.1 --port 6379 2>/dev/null || echo "[WARN] Redis server startup warning"
38
+
39
+ # Safe SQLite Backup Process (Uses sqlite3 .backup API every 5 mins to prevent live DB corruption)
40
  (
41
  while true; do
42
+ sleep 300
43
+ if [ -f "/root/.omniroute/storage.sqlite" ] && command -v sqlite3 >/dev/null 2>&1; then
44
+ sqlite3 /root/.omniroute/storage.sqlite ".backup /data/omniroute/storage.sqlite" 2>/dev/null || true
 
 
 
45
  fi
46
  done
47
  ) &
48
 
49
+ # Start OmniRoute AI Gateway (Multi-Port: Dashboard: 20128, Dedicated API: 20129, Live WS: 20132)
50
+ echo "[INIT] Starting OmniRoute AI Gateway (Dashboard: 20128, API: 20129, WS: 20132)..."
51
  export PORT=20128
52
+ export API_PORT=20129
53
+ export LIVE_WS_PORT=20132
54
  export HOSTNAME="127.0.0.1"
55
  export DATA_DIR="/root/.omniroute"
56
+ export REDIS_URL="redis://127.0.0.1:6379"
57
+ export OMNIROUTE_BASE_PATH="/omniroute"
58
+ export NEXT_PUBLIC_BASE_URL="${WEBUI_URL:-https://jishnupg-opencode-cli.hf.space}/omniroute"
59
+ export OMNIROUTE_WS_BRIDGE_SECRET="${OMNIROUTE_WS_BRIDGE_SECRET:-opencode_ws_bridge_secret_2026}"
60
+ export JWT_SECRET="${JWT_SECRET:-opencode_jwt_secret_2026}"
61
+ export API_KEY_SECRET="${API_KEY_SECRET:-opencode_api_key_secret_2026}"
62
+
63
  if [ -d "/omniroute" ]; then
64
  cd /omniroute
65
  # Fix any upstream migration version collisions (e.g. version 143 collisions)
 
73
  npm run start -- --port 20128 &
74
  else
75
  echo "[INIT] Building OmniRoute production assets to resolve WASM modules..."
76
+ OMNIROUTE_BASE_PATH="/omniroute" NEXT_PUBLIC_BASE_PATH="/omniroute" NEXT_TELEMETRY_DISABLED=1 OMNIROUTE_USE_TURBOPACK=0 NODE_OPTIONS="--max-old-space-size=2560" npm run build 2>/dev/null || true
77
  if [ -d ".build/next" ] || [ -d ".next" ]; then
78
  npm run start -- --port 20128 &
79
  else
80
+ OMNIROUTE_BASE_PATH="/omniroute" npx next dev --port 20128 -H 127.0.0.1 &
81
  fi
82
  fi
83
  echo "[INIT] OmniRoute AI Gateway started in background."
 
112
  chmod -R 777 /root/.cache /data/cache 2>/dev/null || true
113
 
114
  # Pre-configure & Start Open WebUI on port 8098 (Mounted at / via FastAPI proxy)
115
+ echo "[INIT] Starting Open WebUI on port 8098 pre-configured with OmniRoute API..."
116
  if command -v open-webui >/dev/null 2>&1; then
117
+ # Point OpenAI API to OmniRoute dedicated API port 20129
118
  export WEBUI_URL="${WEBUI_URL:-https://jishnupg-opencode-cli.hf.space}"
119
+ export OPENAI_API_BASE_URL="http://127.0.0.1:20129/v1"
120
  export OPENAI_API_KEY="omniroute"
121
  export WEBUI_SECRET_KEY="${WEBUI_SECRET_KEY:-opencode_webui_jwt_secret_2026}"
122
  export ENABLE_OLLAMA_API="${ENABLE_OLLAMA_API:-false}"
gateway/omniroute.py CHANGED
@@ -89,17 +89,24 @@ async def omniroute_main_route(request: Request, path: str = ""):
89
 
90
  return res
91
 
92
- # ── 2. OpenAI & Gemini Compatibility APIs ─────────────────────────────────────
93
  @router.api_route("/v1", methods=["GET", "POST", "PUT", "DELETE", "PATCH", "HEAD", "OPTIONS"])
94
  @router.api_route("/v1/{path:path}", methods=["GET", "POST", "PUT", "DELETE", "PATCH", "HEAD", "OPTIONS"])
95
  async def omniroute_v1_api(request: Request, path: str = ""):
96
- target = f"http://127.0.0.1:{OMNIROUTE_PORT}/v1/{path}" if path else f"http://127.0.0.1:{OMNIROUTE_PORT}/v1"
97
- return await proxy_http_request(target, request, default_prefix="/omniroute")
98
 
99
  @router.api_route("/v1beta/{path:path}", methods=["GET", "POST", "PUT", "DELETE", "PATCH", "HEAD", "OPTIONS"])
100
  async def omniroute_v1beta_api(request: Request, path: str = ""):
101
- target = f"http://127.0.0.1:{OMNIROUTE_PORT}/v1beta/{path}"
102
- return await proxy_http_request(target, request, default_prefix="/omniroute")
 
 
 
 
 
 
 
103
 
104
  # ── 3. Next.js Static Asset Routing ──────────────────────────────────────────
105
  @router.api_route("/_next/{path:path}", methods=["GET", "POST", "PUT", "DELETE", "PATCH", "HEAD", "OPTIONS"])
 
89
 
90
  return res
91
 
92
+ # ── 2. OpenAI & Gemini Compatibility APIs (Dedicated API Port 20129) ──────────
93
  @router.api_route("/v1", methods=["GET", "POST", "PUT", "DELETE", "PATCH", "HEAD", "OPTIONS"])
94
  @router.api_route("/v1/{path:path}", methods=["GET", "POST", "PUT", "DELETE", "PATCH", "HEAD", "OPTIONS"])
95
  async def omniroute_v1_api(request: Request, path: str = ""):
96
+ target = f"http://127.0.0.1:{OMNIROUTE_API_PORT}/v1/{path}" if path else f"http://127.0.0.1:{OMNIROUTE_API_PORT}/v1"
97
+ return await proxy_http_request(target, request, default_prefix="")
98
 
99
  @router.api_route("/v1beta/{path:path}", methods=["GET", "POST", "PUT", "DELETE", "PATCH", "HEAD", "OPTIONS"])
100
  async def omniroute_v1beta_api(request: Request, path: str = ""):
101
+ target = f"http://127.0.0.1:{OMNIROUTE_API_PORT}/v1beta/{path}"
102
+ return await proxy_http_request(target, request, default_prefix="")
103
+
104
+ # ── 3. OmniRoute Live Monitoring WebSocket (Dedicated WS Port 20132) ──────────
105
+ @router.websocket("/omniroute/live-ws")
106
+ @router.websocket("/omniroute/live-ws/{path:path}")
107
+ async def omniroute_live_ws(websocket: Request, path: str = ""):
108
+ target_ws = f"ws://127.0.0.1:{OMNIROUTE_WS_PORT}/live-ws/{path}" if path else f"ws://127.0.0.1:{OMNIROUTE_WS_PORT}/live-ws"
109
+ await proxy_websocket_stream(websocket, target_ws)
110
 
111
  # ── 3. Next.js Static Asset Routing ──────────────────────────────────────────
112
  @router.api_route("/_next/{path:path}", methods=["GET", "POST", "PUT", "DELETE", "PATCH", "HEAD", "OPTIONS"])
gateway/utils.py CHANGED
@@ -19,24 +19,28 @@ from fastapi.responses import StreamingResponse
19
  logger = logging.getLogger("GatewayUtils")
20
 
21
  # Port definitions
22
- WEBUI_PORT = int(os.environ.get("WEBUI_PORT", 8098))
23
- OMNIROUTE_PORT = int(os.environ.get("OMNIROUTE_PORT", 20128))
24
- JELLYFIN_PORT = int(os.environ.get("JELLYFIN_PORT", 8096))
25
- TG_PORT = int(os.environ.get("TG_PORT", 8080))
26
- GATEWAY_PORT = int(os.environ.get("PORT", 4096))
 
 
27
 
28
  PUBLIC_HOST = os.environ.get("PUBLIC_HOST", "jishnupg-opencode-cli.hf.space")
29
  PUBLIC_ORIGIN = f"https://{PUBLIC_HOST}"
30
 
31
- _INTERNAL_PORTS = {GATEWAY_PORT, WEBUI_PORT, OMNIROUTE_PORT, JELLYFIN_PORT, TG_PORT}
32
  _PORT_STRIP_RE = re.compile(r"(https?://[^/:]+):(" + "|".join(str(p) for p in _INTERNAL_PORTS) + r")")
33
 
34
  _PORT_PREFIX_MAP = {
35
- GATEWAY_PORT: "",
36
- OMNIROUTE_PORT: "/omniroute",
37
- WEBUI_PORT: "/openwebui",
38
- JELLYFIN_PORT: "/jellyfin",
39
- TG_PORT: "/tg-stream",
 
 
40
  }
41
 
42
  # Shared Async HTTPX Client Instance
 
19
  logger = logging.getLogger("GatewayUtils")
20
 
21
  # Port definitions
22
+ WEBUI_PORT = int(os.environ.get("WEBUI_PORT", 8098))
23
+ OMNIROUTE_PORT = int(os.environ.get("OMNIROUTE_PORT", 20128))
24
+ OMNIROUTE_API_PORT = int(os.environ.get("OMNIROUTE_API_PORT", 20129))
25
+ OMNIROUTE_WS_PORT = int(os.environ.get("OMNIROUTE_WS_PORT", 20132))
26
+ JELLYFIN_PORT = int(os.environ.get("JELLYFIN_PORT", 8096))
27
+ TG_PORT = int(os.environ.get("TG_PORT", 8080))
28
+ GATEWAY_PORT = int(os.environ.get("PORT", 4096))
29
 
30
  PUBLIC_HOST = os.environ.get("PUBLIC_HOST", "jishnupg-opencode-cli.hf.space")
31
  PUBLIC_ORIGIN = f"https://{PUBLIC_HOST}"
32
 
33
+ _INTERNAL_PORTS = {GATEWAY_PORT, WEBUI_PORT, OMNIROUTE_PORT, OMNIROUTE_API_PORT, OMNIROUTE_WS_PORT, JELLYFIN_PORT, TG_PORT}
34
  _PORT_STRIP_RE = re.compile(r"(https?://[^/:]+):(" + "|".join(str(p) for p in _INTERNAL_PORTS) + r")")
35
 
36
  _PORT_PREFIX_MAP = {
37
+ GATEWAY_PORT: "",
38
+ OMNIROUTE_PORT: "/omniroute",
39
+ OMNIROUTE_API_PORT: "",
40
+ OMNIROUTE_WS_PORT: "/live-ws",
41
+ WEBUI_PORT: "/openwebui",
42
+ JELLYFIN_PORT: "/jellyfin",
43
+ TG_PORT: "/tg-stream",
44
  }
45
 
46
  # Shared Async HTTPX Client Instance