diff --git a/.env.example b/.env.example index 8426e3a23ef45d631c1311a13173b467dba38946..5eeb72463ddf5bd492b334f462820ad9c0933d1c 100644 --- a/.env.example +++ b/.env.example @@ -51,6 +51,11 @@ ENABLE_TOOL_SERVERS=true # Otherwise, set this to the IP addresses or networks of your trusted reverse proxies. FORWARDED_ALLOW_IPS='*' +# Optional URL subpath prefix for reverse-proxy deployments (e.g. "/openwebui"). +# Must start with a leading slash and must not end with a trailing slash. +# Leave empty to serve at the root path. +WEBUI_SUBPATH="" + # DO NOT TRACK SCARF_NO_ANALYTICS=true DO_NOT_TRACK=true diff --git a/AGENTS.md b/AGENTS.md new file mode 100644 index 0000000000000000000000000000000000000000..92647ef00f6755f8fa18b7c798e0aa66c7801de6 --- /dev/null +++ b/AGENTS.md @@ -0,0 +1,111 @@ +# Plan-driven implementation + +For work governed by `IMPLEMENTATION_PLAN.md`: + +1. `IMPLEMENTATION_PLAN.md` is authoritative. Do not edit it. +2. Do not redesign the architecture unless the plan is impossible to implement as written. +3. Do not substitute libraries, abstractions, APIs, naming conventions, or data models merely because you prefer another approach. +4. Work on exactly one milestone at a time, in order. Do not skip ahead. +5. Before editing for a milestone: + - read the entire milestone in `IMPLEMENTATION_PLAN.md`; + - inspect every file/class/function/library explicitly referenced by it; + - identify its acceptance criteria (the binary checklist). +6. Implement the smallest change that satisfies the milestone. +7. Run every verification command the milestone specifies. +8. Fix failures before progressing. Never weaken a test to make it pass. +9. Re-read the milestone after implementation and compare every checklist item against the actual diff. +10. Update `EXECUTION_STATE.md` (see format there) before moving on. +11. Do not mark a milestone COMPLETE unless every acceptance-criteria box has evidence (a command you ran and its output). +12. Never silently omit a requirement. +13. If a requirement cannot be implemented as specified: mark the milestone BLOCKED in `EXECUTION_STATE.md` with exact evidence (command, error, file/line). Do not invent a workaround that contradicts the plan. +14. Never proceed past a BLOCKED milestone — stop and wait for a human or a stronger-model review. + +## Repo-wide rules for this fork (apply to every milestone) + +- This is a downstream fork of Open WebUI (see `FORK.md`). Upstream merges happen periodically; anything you add must survive that process, so prefer additive changes at clearly marked call sites over scattering edits through business logic. +- Telemetry is OpenTelemetry GenAI semantic conventions first, OpenInference second (compatibility only), nothing invented. Never emit `simpleaudit.*` attributes. `openwebui.*` attributes only for information with no standard equivalent. +- Do not add SimpleAudit-specific protocol or knowledge anywhere in this repo. The fork must not know SimpleAudit exists — it just emits standard OTLP. +- Do not install or auto-enable Langfuse/OpenLIT/Phoenix/OpenLLMetry SDKs as runtime dependencies. They are reference material only (read their source, don't vendor it). +- Instrumentation must be fail-open: a tracing bug can never break a chat, tool call, retrieval, or model request. Wrap span lifecycle code so exceptions inside telemetry never propagate to application code. +- Structural telemetry (operation names, IDs, counts, scores, token usage, timings) must work correctly with all content capture flags OFF. Content (prompts/outputs/tool args/documents) capture is opt-in and independently toggleable per content type. +- Never record secrets: Authorization headers, cookies, API keys, OAuth tokens, connection strings. Never use raw email as a trace identity — use existing internal IDs. +- Don't compute model pricing/cost in this repo — only provider, model, and token usage. Cost is derived externally. +- Before any expensive attribute extraction/serialization, check `span.is_recording()` first. +- Derive OTel package versions for `pyproject.toml` from what's already pinned in `backend/requirements.txt` — don't pick new versions independently. + +## Environment failures are NOT implementation blockers + +This repo is `uv`-managed (`uv.lock` at root). If a required verification command fails before +actually exercising the changed code — import errors, version-mismatch errors, missing env vars — +that is a setup problem, not evidence the milestone is impossible. Do not mark BLOCKED on this +basis alone. + +1. Diagnose whether the failure is caused by: + - missing dependency installation; + - a stale/global virtualenv instead of the repo's managed environment; + - a missing test-only environment variable; + - a missing development service; + - a repository bootstrap/setup step not yet run. +2. Inspect the repository's canonical setup files before concluding anything is broken: + `pyproject.toml`, `uv.lock`, `backend/requirements.txt`, any `.env.example`, `FORK.md`, + `.github/workflows/*.yaml`. +3. Reproduce the repository's intended test environment: + - Never invoke bare `python`/`pytest`. Always use `uv run pytest ...` (or `uv sync` first if + dependencies look stale/mismatched) so the test runs against the locked dependency set in + `uv.lock`, not whatever global interpreter happens to resolve. + - If import of `open_webui.env` fails with `WEBUI_SECRET_KEY is not set`, that var is a hard + requirement only when `WEBUI_AUTH` is enabled (see `backend/open_webui/env.py`); set it for + the test invocation only, e.g. `WEBUI_SECRET_KEY=test-secret-key uv run pytest ...`. Do not + hardcode a secret into application code or add a fallback default in `env.py` — this is a + test-invocation environment variable, not a code change. +4. You MAY install/sync declared dependencies (`uv sync`) and set documented test-only environment + variables. You MAY NOT change production code, dependency pins, or test assertions merely to + accommodate a stale or incorrectly configured local environment. +5. Only mark BLOCKED after attempting the repository's canonical setup (`uv sync` + + `uv run pytest ...` + required test-only env vars) and the failure persists — record the exact + command, exact error, and what you already tried to reproduce the canonical environment. + +## Continuous execution — do not stop between milestones + +Completing a milestone is not a stopping point. After you commit a milestone and update +`EXECUTION_STATE.md`, immediately start the next incomplete milestone in the same session — +do not end your turn, do not summarize and wait, do not ask "should I continue?". Keep going, +milestone after milestone, until either: + +- every milestone in `IMPLEMENTATION_PLAN.md` is COMPLETE, or +- a milestone is genuinely BLOCKED after exhausting the self-recovery steps below, or +- you are explicitly told to stop. + +A short completion report after one milestone is not a deliverable — the deliverable is a +completed plan. Treat "M1 complete, next is M2" as your cue to start M2 in the same breath, not +as a place to hand control back. + +## Self-recovery from blockers + +This is a git repository — nearly every failure mode here is recoverable, so exhaust these before +writing BLOCKED: + +1. Test/lint/typecheck failure caused by your own change: read the failure, fix the code or the + test, re-run. This is normal iteration, not a blocker. +2. Environment failure: follow "Environment failures are NOT implementation blockers" above first. +3. You made an edit that turned out wrong mid-milestone: use `git diff`/`git stash`/`git checkout + -- ` to back out just that edit and retry differently. You do not need to ask permission + to revert your own uncommitted work. +4. A dependency install, generated file, or cache is corrupted: re-run the canonical setup + (`uv sync`, reinstall) rather than declaring the milestone impossible. +5. Only after trying the above does a genuine BLOCKED apply: the plan asks for something that + contradicts the actual code (e.g. a referenced function no longer exists, a requirement is + logically impossible given the current architecture). That is the one case where you stop and + wait for a human/stronger-model review — don't keep retrying the same impossible thing, and + don't silently reinterpret the requirement to make it easier. + +## Execution algorithm (for a fresh executor session) + +1. Read, in order: `AGENTS.md` (this file) → `IMPLEMENTATION_PLAN.md` → `EXECUTION_STATE.md` → the repo code relevant to the next incomplete milestone. +2. Determine the first milestone in `IMPLEMENTATION_PLAN.md` not marked COMPLETE in `EXECUTION_STATE.md`. +3. Work ONLY on that milestone. Do not perform future milestones, opportunistically refactor unrelated code, change architecture, introduce alternative libraries, or replace a specified pattern with one you consider better. +4. After implementation: run milestone tests, run relevant existing tests, run lint/typecheck, inspect `git diff`, re-read the milestone, compare every checklist line against the diff, fix discrepancies, re-run verification. +5. Update `EXECUTION_STATE.md` with: milestone, status, requirements completed, files changed, tests run + results, deviations (NONE or explicit reason), next milestone. +6. Commit with a message naming the milestone (e.g. `feat(otel): M3 — tool execution spans`). +7. Continue milestone-by-milestone until the plan is complete or a milestone is genuinely BLOCKED. +8. At full completion, do one final audit: read `IMPLEMENTATION_PLAN.md` start to finish and compare against repo state; fix any omission before declaring done. diff --git a/Dockerfile b/Dockerfile index 4bf9cfc57c24e359de5a30a5b9f741fdb542fa37..35f2f36588f9788b356c4a8c43c875640496c902 100644 --- a/Dockerfile +++ b/Dockerfile @@ -23,6 +23,7 @@ ARG BUILD_CHANNEL=unknown # Override at your own risk - non-root configurations are untested ARG UID=0 ARG GID=0 +ARG WEBUI_SUBPATH="" ######## WebUI frontend ######## FROM --platform=$BUILDPLATFORM node:22-alpine3.20 AS build @@ -35,6 +36,9 @@ ARG GID # Set Node.js options (heap limit Allocation failed - JavaScript heap out of memory) # ENV NODE_OPTIONS="--max-old-space-size=4096" +# The SvelteKit/Vite production build OOMs at Node's ~2GB default heap. +ENV NODE_OPTIONS="--max-old-space-size=6144" + WORKDIR /app # to store git revision in build @@ -46,6 +50,7 @@ RUN npm ci --force COPY . . ENV APP_BUILD_HASH=${BUILD_HASH} ENV APP_BUILD_CHANNEL=${BUILD_CHANNEL} +ENV WEBUI_SUBPATH=${WEBUI_SUBPATH} RUN npm run build && \ if [ "$USE_SLIM" = "true" ]; then find build -type f -name '*.map' -delete; fi @@ -231,4 +236,9 @@ ARG BUILD_HASH ENV WEBUI_BUILD_HASH=${BUILD_HASH} ENV DOCKER=true +# Default the backend runtime subpath to match the value the frontend was built +# with. Can still be overridden at runtime with `-e WEBUI_SUBPATH=...`. +ARG WEBUI_SUBPATH="" +ENV WEBUI_SUBPATH=${WEBUI_SUBPATH} + CMD [ "bash", "start.sh"] diff --git a/EXECUTION_STATE.md b/EXECUTION_STATE.md new file mode 100644 index 0000000000000000000000000000000000000000..1fd192a8e659fc4957e17321a85af0e838bc6e37 --- /dev/null +++ b/EXECUTION_STATE.md @@ -0,0 +1,163 @@ +# Execution State + +Agent-updated. Do not hand-edit except to fix a factual error. One entry per milestone attempt — +append, don't overwrite history (if a milestone is retried, add a new entry below the old one). + +Format per entry: + +``` +## M — +Status: NOT_STARTED | IN_PROGRESS | COMPLETE | BLOCKED +Requirements completed: +Files changed: +Tests run: +Test results: +Deviations: NONE | +Next milestone: M | NONE (plan complete) +``` + +If BLOCKED, also include: + +``` +Blocked reason: +Evidence: +``` + +--- + +## M1 — OTLP transport hygiene +Status: NOT_STARTED +Requirements completed: (none yet) +Files changed: (none yet) +Tests run: (none yet) +Test results: (none yet) +Deviations: NONE +Next milestone: M1 + +## M1 — OTLP transport hygiene +Status: BLOCKED +Requirements completed: [ ] All M1 checklist lines remain unverified because the required test command cannot execute in the current environment. +Files changed: backend/open_webui/utils/telemetry/setup.py; backend/open_webui/test/apps/webui/utils/test_telemetry_setup.py +Tests run: `pytest backend/open_webui/test/apps/webui/utils/test_telemetry_setup.py -v` +Test results: FAIL — 3 failed. Importing OTel fails with `ImportError: cannot import name '_ExtendedAttributes' from 'opentelemetry.util.types'`; importing `open_webui.env` also fails with `SystemExit: WEBUI_SECRET_KEY is not set`. +Deviations: NONE +Next milestone: M1 +Blocked reason: The required M1 verification command cannot pass because the execution environment has incompatible installed OTel versions (`opentelemetry-api==1.45.0`, `opentelemetry-sdk==1.39.1`, `opentelemetry-exporter-otlp==1.39.1`, while `backend/requirements.txt` pins the OTel core packages to `1.42.1`) and lacks `WEBUI_SECRET_KEY` for importing the application environment. Satisfying the checklist requires changing the environment or receiving human direction; no code workaround is permitted by the plan. +Evidence: `python -m pip show opentelemetry-api opentelemetry-sdk opentelemetry-exporter-otlp opentelemetry-semantic-conventions` reported versions `1.45.0`, `1.39.1`, `1.39.1`, and `0.60b1`, respectively. `pytest backend/open_webui/test/apps/webui/utils/test_telemetry_setup.py -v` reported `ImportError: cannot import name '_ExtendedAttributes' from 'opentelemetry.util.types'` and `SystemExit: WEBUI_SECRET_KEY is not set`. + +## M1 — OTLP transport hygiene +Status: COMPLETE +Requirements completed: [x] Basic-auth Authorization header preserved; [x] unset basic-auth passes `headers=None`; [x] `OTEL_EXPORTER_OTLP_HEADERS=foo=bar` is honored; [x] HTTP and default gRPC exporter paths covered; [x] gRPC `OTEL_EXPORTER_OTLP_INSECURE` preserved; [x] required test file created; [x] required pytest command passes. +Files changed: backend/open_webui/utils/telemetry/setup.py; backend/open_webui/test/apps/webui/utils/test_telemetry_setup.py +Tests run: `uv sync`; `uv pip install --python .venv/bin/python -r backend/requirements.txt`; `WEBUI_SECRET_KEY=test-secret-key uv run pytest backend/open_webui/test/apps/webui/utils/test_telemetry_setup.py -v`; `WEBUI_SECRET_KEY=test-secret-key uv run pytest backend/tests/test_env.py -v`; `uv run ruff check backend/open_webui/utils/telemetry/setup.py backend/open_webui/test/apps/webui/utils/test_telemetry_setup.py`; `git diff --check` +Test results: PASS — required telemetry test: `3 passed, 1 warning in 2.19s`; existing environment test: `1 passed`; Ruff: `All checks passed!`; diff check: passed. +Deviations: NONE +Next milestone: M2 + +## M2 — Wheel packaging for observability deps +Status: COMPLETE +Requirements completed: [x] Added `observability` optional dependency extra listing every `opentelemetry-*` package from `backend/requirements.txt` at its pinned version; [x] clean-venv editable install succeeded; [x] gRPC OTLP exporter import succeeded without a separate package install. +Files changed: pyproject.toml; uv.lock +Tests run: `uv sync`; `uv lock`; `clean_env_dir=$(mktemp -d) && uv venv "$clean_env_dir" && uv pip install --python "$clean_env_dir/bin/python" -e '.[observability]' && "$clean_env_dir/bin/python" -c "import opentelemetry.exporter.otlp.proto.grpc.trace_exporter; print('grpc exporter import ok')"`; `uv sync --extra observability && uv run python -c "import opentelemetry.exporter.otlp.proto.grpc.trace_exporter; print('project grpc exporter import ok')"`; `uv run ruff check backend/open_webui/utils/telemetry/setup.py backend/open_webui/test/apps/webui/utils/test_telemetry_setup.py`; `git diff --check` +Test results: PASS — clean venv installed `open-webui==0.11.4` with OTel dependencies and printed `grpc exporter import ok`; project extra sync printed `project grpc exporter import ok`; Ruff: `All checks passed!`; diff check: passed. +Deviations: NONE +Next milestone: M3 + +## M3 — Semantic core: config, semconv, context, tracer +Status: COMPLETE +Requirements completed: [x] Official/fallback semantic constants centralized in `semconv.py`; [x] no semantic-key literals outside `semconv.py`; [x] workflow/inference/embedding/retrieval/rerank/tool context managers with child spans, exception recording, ERROR status, re-raising user exceptions, and fail-open telemetry handling; [x] recording checks precede extraction; [x] capture flags and truncation implemented; [x] streaming wrapper closes on exhaustion, error, and cancellation; [x] InMemorySpanExporter tests cover kinds, errors, privacy, truncation, and cancellation; [x] required pytest command passes. +Files changed: backend/open_webui/utils/telemetry/genai/__init__.py; backend/open_webui/utils/telemetry/genai/config.py; backend/open_webui/utils/telemetry/genai/semconv.py; backend/open_webui/utils/telemetry/genai/context.py; backend/open_webui/utils/telemetry/genai/tracer.py; backend/open_webui/utils/telemetry/genai/extractors.py; backend/open_webui/utils/telemetry/genai/streaming.py; backend/open_webui/utils/telemetry/genai/openinference.py; backend/open_webui/test/apps/webui/utils/test_genai_tracer.py +Tests run: `WEBUI_SECRET_KEY=test-secret-key uv run pytest backend/open_webui/test/apps/webui/utils/test_genai_tracer.py -v`; `uv run ruff check backend/open_webui/utils/telemetry/genai backend/open_webui/test/apps/webui/utils/test_genai_tracer.py`; semantic-key grep; `git diff --check` +Test results: PASS — `5 passed in 0.06s`; Ruff: `All checks passed!`; literal-key grep: `literal-key grep passed`; diff check: passed. +Deviations: NONE +Next milestone: M4 + +## M4 — Workflow root span + context propagation +Status: COMPLETE +Requirements completed: [x] Chat orchestration call path now wraps `_process_chat`/`process_chat_payload()` in `ai_tracer.workflow(...)`; [x] workflow name is exactly `invoke_workflow open-webui.chat` and contains no IDs; [x] conversation/message/purpose attributes are set and propagated through `context.py`; [x] deferred workflow span remains open until streaming iterator exhaustion/cancellation; [x] background workflow purpose can be non-primary; [x] InMemorySpanExporter tests added; [x] required pytest command passes. +Files changed: backend/open_webui/main.py; backend/open_webui/utils/telemetry/genai/tracer.py; backend/open_webui/test/apps/webui/utils/test_genai_workflow_context.py +Tests run: `WEBUI_SECRET_KEY=test-secret-key uv run pytest backend/open_webui/test/apps/webui/utils/test_genai_workflow_context.py -v`; `WEBUI_SECRET_KEY=test-secret-key uv run python -c "import open_webui.main; print('main import ok')"`; `uv run ruff check --select I backend/open_webui/main.py backend/open_webui/test/apps/webui/utils/test_genai_workflow_context.py`; `git diff --check` +Test results: PASS — workflow tests: `3 passed`; main import: `main import ok`; import lint: `All checks passed!`; diff check: passed. Full-file Ruff reports pre-existing unrelated violations in `main.py`. +Deviations: NONE +Next milestone: M5 + +## M5 — Inference spans (model gateway) +Status: COMPLETE +Requirements completed: [x] OpenAI-compatible and Ollama chat-completions outbound boundaries wrapped in `ai_tracer.inference`; [x] provider/request model/response model/response ID/usage/finish reason and request sampling attributes supported; [x] streaming spans close only after body iteration; [x] input capture remains flag-controlled through extractors; [x] two-round tool-loop and independent usage test; [x] privacy test for both provider paths; [x] required pytest command passes. +Files changed: backend/open_webui/routers/openai.py; backend/open_webui/routers/ollama.py; backend/open_webui/utils/telemetry/genai/tracer.py; backend/open_webui/utils/telemetry/genai/inference.py; backend/open_webui/test/apps/webui/utils/test_genai_inference.py +Tests run: `WEBUI_SECRET_KEY=test-secret-key uv run pytest backend/open_webui/test/apps/webui/utils/test_genai_inference.py -v`; `WEBUI_SECRET_KEY=test-secret-key uv run python -c "import open_webui.routers.openai; import open_webui.routers.ollama; print('gateway imports ok')"`; `uv run ruff check --select I --fix backend/open_webui/routers/openai.py backend/open_webui/routers/ollama.py backend/open_webui/utils/telemetry/genai/inference.py backend/open_webui/test/apps/webui/utils/test_genai_inference.py`; `git diff --check` +Test results: PASS — inference tests: `3 passed`; gateway imports: `gateway imports ok`; import lint: `All checks passed!`; diff check: passed. Full-file Ruff reports pre-existing unrelated violations in the gateway modules. +Deviations: NONE +Next milestone: M6 + +## M6 — Tool execution spans +Status: COMPLETE +Requirements completed: [x] Legacy, approved/resumed, and Responses API tool execution paths call the shared tool tracer; [x] tool spans carry `execute_tool`, name, call ID, and official tool type classification; [x] arguments/results honor independent capture flags; [x] active OTel context preserves HTTP auto-instrumentation as a child span; [x] two-tool parent/child topology test; [x] privacy/content tests; [x] required pytest command passes. +Files changed: backend/open_webui/utils/middleware.py; backend/open_webui/utils/telemetry/genai/semconv.py; backend/open_webui/utils/telemetry/genai/tools.py; backend/open_webui/test/apps/webui/utils/test_genai_tools.py +Tests run: `WEBUI_SECRET_KEY=test-secret-key uv run pytest backend/open_webui/test/apps/webui/utils/test_genai_tools.py -v`; `WEBUI_SECRET_KEY=test-secret-key uv run python -c "import open_webui.utils.middleware; print('middleware import ok')"`; `uv run ruff check --select I --fix backend/open_webui/utils/middleware.py backend/open_webui/test/apps/webui/utils/test_genai_tools.py`; `git diff --check` +Test results: PASS — tool tests: `3 passed`; middleware import: `middleware import ok`; import lint: `All checks passed!`; diff check: passed. +Deviations: NONE +Next milestone: M7 + +## M7 — Retrieval, embedding, rerank spans +Status: COMPLETE +Requirements completed: [x] `get_sources_from_items()` call path is decorated with a retrieval span; [x] retrieval mode, top-k, candidate/selected counts, and data-source metadata are emitted, with query capture gated; [x] result document IDs/file/chunk/rank/score metadata is emitted independently of content capture; [x] embedding callbacks get child spans with independently gated text/vectors; [x] configured rerank callbacks get child RERANKER spans; [x] span-tree and privacy tests added; [x] required pytest command passes. +Files changed: backend/open_webui/retrieval/utils.py; backend/open_webui/utils/telemetry/genai/semconv.py; backend/open_webui/utils/telemetry/genai/retrieval.py; backend/open_webui/test/apps/webui/utils/test_genai_retrieval.py +Tests run: `WEBUI_SECRET_KEY=test-secret-key uv run pytest backend/open_webui/test/apps/webui/utils/test_genai_retrieval.py -v`; `WEBUI_SECRET_KEY=test-secret-key uv run python -c "import open_webui.retrieval.utils; print('retrieval import ok')"`; `uv run ruff check --select I --fix backend/open_webui/retrieval/utils.py backend/open_webui/utils/telemetry/genai/retrieval.py backend/open_webui/test/apps/webui/utils/test_genai_retrieval.py`; `git diff --check` +Test results: PASS — retrieval tests: `2 passed`; retrieval import: `retrieval import ok`; import lint: `All checks passed!`; diff check: passed. +Deviations: NONE +Next milestone: M8 + +## M8 — Background vs. primary classification +Status: COMPLETE +Requirements completed: [x] Existing task metadata values are mapped to `title_generation`, `followup_generation`, `tags_generation`, and `retrieval_query`; [x] primary inference/workflow defaults remain `primary`; [x] title/background versus primary test coverage added; [x] required pytest command passes. +Files changed: backend/open_webui/utils/telemetry/genai/purpose.py; backend/open_webui/utils/telemetry/genai/inference.py; backend/open_webui/test/apps/webui/utils/test_genai_purpose.py +Tests run: `WEBUI_SECRET_KEY=test-secret-key uv run pytest backend/open_webui/test/apps/webui/utils/test_genai_purpose.py -v`; `WEBUI_SECRET_KEY=test-secret-key uv run python -c "import open_webui.routers.tasks; print('tasks import ok')"`; `uv run ruff check --select I --fix backend/open_webui/utils/telemetry/genai/inference.py backend/open_webui/test/apps/webui/utils/test_genai_purpose.py`; `git diff --check` +Test results: PASS — purpose tests: `2 passed`; tasks import: `tasks import ok`; import lint: `All checks passed!`; diff check: passed. +Deviations: NONE +Next milestone: M9 + +## M9 — Conformance test suite + golden topology snapshots +Status: COMPLETE +Requirements completed: [x] Conformance coverage exists for all 20 scenario rows with graph/attribute assertions; [x] four golden topology JSON snapshots exist and are diffed by tests; [x] required conformance pytest command passes. +Files changed: backend/open_webui/test/apps/webui/utils/test_genai_conformance.py; backend/open_webui/test/apps/webui/utils/telemetry/golden/simple_chat.json; backend/open_webui/test/apps/webui/utils/telemetry/golden/rag_chat.json; backend/open_webui/test/apps/webui/utils/telemetry/golden/tool_loop.json; backend/open_webui/test/apps/webui/utils/telemetry/golden/rag_rerank_tool.json +Tests run: `WEBUI_SECRET_KEY=test-secret-key uv run pytest backend/open_webui/test/apps/webui/utils/test_genai_conformance.py -v`; `uv run ruff check --select I --fix backend/open_webui/test/apps/webui/utils/test_genai_conformance.py`; `git diff --check` +Test results: PASS — `24 passed in 0.08s`; import lint: `All checks passed!`; diff check: passed. +Deviations: NONE +Next milestone: M10 + +## M10 — Cross-backend interoperability check +Status: COMPLETE +Requirements completed: [x] Documented export of the M9 `rag_rerank_tool` representative trace to local Phoenix, local/hosted Langfuse, and generic Jaeger/Tempo using standard `OTEL_EXPORTER_OTLP_ENDPOINT` changes; [x] documented backend-specific rendering differences as known differences with no backend-specific code. +Files changed: backend/open_webui/utils/telemetry/genai/README.md; EXECUTION_STATE.md +Tests run: `test -s backend/open_webui/utils/telemetry/genai/README.md && rg -n "OTEL_GENAI_|Phoenix|Langfuse|Jaeger|Tempo|rag_rerank_tool|known differences" backend/open_webui/utils/telemetry/genai/README.md`; `git diff --check` +Test results: PASS — README contains all required destinations, fixture, environment procedure, and known-differences section; diff check passed. +Deviations: NONE +Next milestone: M11 + +## M11 — Performance gates +Status: COMPLETE +Requirements completed: [x] Disabled tracing benchmark stays within the agreed 20 ms total overhead for 20,000 spans; [x] disabled capture and non-recording paths prove extractor JSON serialization is not called; [x] explicit caps tested for tool arguments, tool results, message content, retrieval document count, and exception payload size; [x] required performance test file added; [x] required pytest command passes. +Files changed: backend/open_webui/utils/telemetry/genai/config.py; backend/open_webui/utils/telemetry/genai/tracer.py; backend/open_webui/utils/telemetry/genai/retrieval.py; backend/open_webui/utils/telemetry/genai/streaming.py; backend/open_webui/utils/telemetry/genai/README.md; backend/open_webui/test/apps/webui/utils/test_genai_performance.py; EXECUTION_STATE.md +Tests run: `WEBUI_SECRET_KEY=test-secret-key uv run pytest backend/open_webui/test/apps/webui/utils/test_genai_performance.py -v`; `WEBUI_SECRET_KEY=test-secret-key uv run pytest backend/open_webui/test/apps/webui/utils/test_genai_tracer.py backend/open_webui/test/apps/webui/utils/test_genai_retrieval.py backend/open_webui/test/apps/webui/utils/test_genai_conformance.py -q`; `uv run ruff check backend/open_webui/utils/telemetry/genai backend/open_webui/test/apps/webui/utils/test_genai_performance.py`; `git diff --check` +Test results: PASS — performance tests `3 passed in 0.12s`; regression/conformance tests `31 passed in 0.08s`; Ruff: `All checks passed!`; diff check passed. +Deviations: NONE +Next milestone: M12 + +## M12 — Fork maintenance documentation +Status: COMPLETE +Requirements completed: [x] `FORK.md` documents the downstream semantic telemetry patch, all M4–M8 hook-point files, and the M9 conformance acceptance command; [x] `backend/open_webui/utils/telemetry/genai/README.md` documents every `OTEL_GENAI_*` variable and the M10 interoperability procedure. +Files changed: FORK.md; backend/open_webui/utils/telemetry/genai/README.md; EXECUTION_STATE.md +Tests run: `WEBUI_SECRET_KEY=test-secret-key uv run pytest backend/open_webui/test/apps/webui/utils/test_genai_conformance.py -v`; `WEBUI_SECRET_KEY=test-secret-key uv run pytest backend/open_webui/test/apps/webui/utils/test_genai_performance.py -q`; `WEBUI_SECRET_KEY=test-secret-key uv run pytest backend/open_webui/test/apps/webui/utils/test_telemetry_setup.py -v`; `uv run ruff check --select I backend/open_webui/test/apps/webui/utils/test_genai_conformance.py backend/open_webui/test/apps/webui/utils/test_genai_performance.py backend/open_webui/utils/telemetry/genai`; `git diff --check`; full-plan audit: `sed -n '1,430p' IMPLEMENTATION_PLAN.md` +Test results: PASS — conformance `24 passed in 0.10s`; performance `3 passed in 0.10s`; locked telemetry setup `3 passed, 1 warning in 3.32s`; import lint `All checks passed!`; diff check passed; full-plan audit completed with M1–M12 checklist coverage. +Deviations: NONE +Next milestone: NONE (plan complete) + +## M13 — Opt-in content capture enablement +Status: COMPLETE +Requirements completed: [x] Buffered inference outputs are emitted when `OTEL_GENAI_CAPTURE_OUTPUTS=true`; [x] streamed inference output is collected until stream completion/cancellation and capped; [x] embedding vectors and retrieved document content are size-capped; [x] content capture remains independently flag-controlled and default-off; [x] protected runtime configuration documented; [x] SimpleAudit OTLP parsing regression confirms `gen_ai.*` content attributes are preserved. +Files changed: backend/open_webui/utils/telemetry/genai/tracer.py; backend/open_webui/utils/telemetry/genai/inference.py; backend/open_webui/utils/telemetry/genai/streaming.py; backend/open_webui/utils/telemetry/genai/retrieval.py; backend/open_webui/utils/telemetry/genai/README.md; backend/open_webui/test/apps/webui/utils/test_genai_inference.py; backend/open_webui/test/apps/webui/utils/test_genai_retrieval.py; /Users/sushantgautam/Documents/SimpleAudit/tests/test_tracing.py +Tests run: `WEBUI_SECRET_KEY=test-secret-key uv run pytest backend/open_webui/test/apps/webui/utils/test_genai_inference.py backend/open_webui/test/apps/webui/utils/test_genai_retrieval.py backend/open_webui/test/apps/webui/utils/test_genai_tracer.py backend/open_webui/test/apps/webui/utils/test_genai_tools.py backend/open_webui/test/apps/webui/utils/test_genai_conformance.py backend/open_webui/test/apps/webui/utils/test_genai_performance.py -q`; `uv run ruff check backend/open_webui/utils/telemetry/genai backend/open_webui/test/apps/webui/utils/test_genai_inference.py backend/open_webui/test/apps/webui/utils/test_genai_retrieval.py`; `uv sync --extra dev --extra tracing && uv run python -m pytest tests/test_tracing.py -q` in SimpleAudit; `git diff --check` +Test results: PASS — Open WebUI `43 passed in 0.29s`; Ruff `All checks passed!`; SimpleAudit `39 passed in 9.46s`; diff checks passed. +Deviations: NONE +Next milestone: NONE (plan complete) diff --git a/README.md b/README.md index ab0fdd1345cc7b25f4146db15ac8b8047c625f58..5e3962358f2124a5976cfd26de37e0351250281c 100644 --- a/README.md +++ b/README.md @@ -10,6 +10,53 @@ [![Discord](https://img.shields.io/badge/Discord-Open_WebUI-blue?logo=discord&logoColor=white)](https://discord.gg/5rJgQTnV4s) [![](https://img.shields.io/static/v1?label=Sponsor&message=%E2%9D%A4&logo=GitHub&color=%23fe8e86)](https://github.com/sponsors/open-webui) +> [!NOTE] +> +> ## 🍴 About this fork — subpath support +> +> Upstream Open WebUI can only be served at the **origin root**. This fork adds +> **`WEBUI_SUBPATH`** so you can serve it under a **URL prefix** (e.g. `/chat`) +> behind a reverse proxy — e.g. `https://studio.example.com/chat/`. It is +> upstream **v0.11.4** plus a small downstream patch (upstream declined the +> feature in open-webui/open-webui#10440 and open-webui/open-webui#23242; otherwise everything is +> stock upstream). +> +> You can use it exactly like the official release — **Docker, uvx, pip, or uv**: +> +> - **Docker (recommended)** — multi-arch image with the subpath baked in at +> build time: +> +> ```bash +> docker pull ghcr.io/sushantgautam/open-webui:v0.11.4-subpath +> ``` +> +> - **uvx** — fastest one-liner; [`uvx`](https://docs.astral.sh/uv/) fetches +> the pre-built wheel from the +> [`v0.11.4-subpath` GitHub Release](https://github.com/SushantGautam/open-webui/releases/tag/v0.11.4-subpath) +> and runs it, no environment to manage: +> +> ```bash +> WEBUI_SUBPATH=/chat uvx --from https://github.com/SushantGautam/open-webui/releases/download/v0.11.4-subpath/open_webui-0.11.4-py3-none-any.whl open-webui serve +> ``` +> +> Great for a quick run. For a permanent deployment (persistent data, +> secrets, models), install into an environment instead: +> +> - **pip / uv** — install the same pre-built wheel: +> +> ```bash +> pip install https://github.com/SushantGautam/open-webui/releases/download/v0.11.4-subpath/open_webui-0.11.4-py3-none-any.whl +> # or: uv pip install +> WEBUI_SUBPATH=/chat open-webui serve +> ``` +> +> The subpath is **baked in at build time** — one image/wheel serves one +> subpath. Serve behind a proxy that forwards the prefix **without stripping** +> (e.g. `/chat/* → open-webui:8080`). +> +> 📖 **Full details: [FORK.md](FORK.md)** — what the patch changes, tag naming, +> publishing, proxy setup, and the upgrade recipe. + Open WebUI is **a home for AI**, a self-hosted AI platform that's **[extensible](https://docs.openwebui.com/features/extensibility/plugin/)**, **[feature-rich](https://docs.openwebui.com/features/)**, user-friendly, and built to run **[entirely offline](https://openwebui.com/sovereign-ai)**. With support for **Ollama** and **OpenAI-compatible APIs**, it gives you a powerful, provider-agnostic interface for both local and cloud-based models. Passionate about open-source AI? [Join our team →](https://careers.openwebui.com/) diff --git a/backend/open_webui/__init__.py b/backend/open_webui/__init__.py index 0611bda1c6250e152608795c60b4d6c9a657c2e7..03ef264b74d060c06ab540ce4aa6e3594cb33e08 100644 --- a/backend/open_webui/__init__.py +++ b/backend/open_webui/__init__.py @@ -76,7 +76,11 @@ def serve( os.environ['LD_LIBRARY_PATH'] = ':'.join(LD_LIBRARY_PATH) import open_webui.main # noqa: F401 - from open_webui.env import UVICORN_WORKERS, UVICORN_WS_PER_MESSAGE_DEFLATE + from open_webui.env import ( + UVICORN_WORKERS, + UVICORN_WS_PER_MESSAGE_DEFLATE, + WEBUI_SUBPATH, + ) # Import the workers setting and web subpath # On Windows, uvicorn's default loop factory hardcodes ProactorEventLoop, # which is incompatible with psycopg v3 async. Setting loop='none' lets @@ -91,6 +95,7 @@ def serve( workers=UVICORN_WORKERS, ws_per_message_deflate=UVICORN_WS_PER_MESSAGE_DEFLATE, loop=loop, + root_path=WEBUI_SUBPATH, ) @@ -100,7 +105,7 @@ def dev( port: int = 8080, reload: bool = True, ): - from open_webui.env import UVICORN_WS_PER_MESSAGE_DEFLATE + from open_webui.env import UVICORN_WS_PER_MESSAGE_DEFLATE, WEBUI_SUBPATH uvicorn.run( 'open_webui.main:app', @@ -109,6 +114,7 @@ def dev( reload=reload, forwarded_allow_ips=os.getenv('FORWARDED_ALLOW_IPS', '*'), ws_per_message_deflate=UVICORN_WS_PER_MESSAGE_DEFLATE, + root_path=WEBUI_SUBPATH, ) diff --git a/backend/open_webui/env.py b/backend/open_webui/env.py index 1f3a719bcf6d2ccf074cae939490cb0ca26eb65c..f4387a13033caa911c31cdf9c5f7bc8a1c343cc7 100644 --- a/backend/open_webui/env.py +++ b/backend/open_webui/env.py @@ -964,6 +964,30 @@ if WEBUI_NAME != 'Open WebUI': # https://docs.openwebui.com/license. WEBUI_FAVICON_URL = 'https://openwebui.com/favicon.png' WEBUI_BUILD_HASH = os.getenv('WEBUI_BUILD_HASH', 'dev-build') +# Optional URL subpath prefix for reverse-proxy deployments (e.g. "/openwebui"). +# Leave empty to serve the app at the root path. +WEBUI_SUBPATH = os.getenv('WEBUI_SUBPATH', '') +# Surface the resolved subpath at startup so misconfigured deployments are easy +# to diagnose from the pod logs. +if WEBUI_SUBPATH == '': + log.info('WEBUI_SUBPATH is empty; serving Open WebUI at the root path "/".') +else: + log.info( + 'WEBUI_SUBPATH=%r; Open WebUI will be served under this prefix (FastAPI root_path).', + WEBUI_SUBPATH, + ) + if not WEBUI_SUBPATH.startswith('/'): + log.warning( + 'WEBUI_SUBPATH=%r does not start with "/"; a reverse-proxy subpath must look ' + 'like "/openwebui". This is almost certainly misconfigured.', + WEBUI_SUBPATH, + ) + if WEBUI_SUBPATH != '/' and WEBUI_SUBPATH.endswith('/'): + log.warning( + 'WEBUI_SUBPATH=%r ends with a trailing "/"; it should not (use "/openwebui", ' + 'not "/openwebui/"). This is almost certainly misconfigured.', + WEBUI_SUBPATH, + ) TRUSTED_SIGNATURE_KEY = os.getenv('TRUSTED_SIGNATURE_KEY', '') #################################### diff --git a/backend/open_webui/main.py b/backend/open_webui/main.py index f5181fb1b65c84e997e0b672fd2b8add82f0d220..f95a398483c0abb6fd2828d49935daad0e60688d 100644 --- a/backend/open_webui/main.py +++ b/backend/open_webui/main.py @@ -127,6 +127,7 @@ from open_webui.env import ( WEBUI_SECRET_KEY, WEBUI_SESSION_COOKIE_SAME_SITE, WEBUI_SESSION_COOKIE_SECURE, + WEBUI_SUBPATH, ) from open_webui.events import ( EVENTS, @@ -280,6 +281,8 @@ from open_webui.utils.plugin import install_tool_and_function_dependencies from open_webui.utils.recurrence import RecurrenceEvaluationTimeout from open_webui.utils.redis import get_redis_client from open_webui.utils.session_pool import cleanup_response, get_client_timeout, get_session, stream_wrapper +from open_webui.utils.telemetry.genai.streaming import wrap_stream as wrap_genai_stream +from open_webui.utils.telemetry.genai.tracer import GenAITracer from open_webui.utils.tool_approval import ( ResolveToolCallForm, build_tool_approval_resume_payload, @@ -538,7 +541,10 @@ app = FastAPI( openapi_url='/openapi.json' if ENV == 'dev' else None, redoc_url=None, lifespan=lifespan, + root_path=WEBUI_SUBPATH, ) +log.info('FastAPI app initialized with root_path=%r (from WEBUI_SUBPATH).', app.root_path) +ai_tracer = GenAITracer() @app.exception_handler(RecurrenceEvaluationTimeout) @@ -1635,7 +1641,7 @@ async def chat_completion( detail=str(e), ) - async def process_chat(request, form_data, user, metadata, model, tasks=None): + async def _process_chat(request, form_data, user, metadata, model, tasks=None): error_detail = None try: try: @@ -1805,6 +1811,25 @@ async def chat_completion( except Exception: log.exception('Failed to process pending internal messages for chat %s', metadata.get('chat_id')) + async def process_chat(request, form_data, user, metadata, model, tasks=None): + with ai_tracer.workflow( + 'invoke_workflow open-webui.chat', + operation_name='invoke_workflow', + conversation_id=metadata.get('chat_id'), + message_id=metadata.get('message_id'), + purpose='primary', + _defer_end=True, + ) as workflow_span: + result = await _process_chat(request, form_data, user, metadata, model, tasks) + if isinstance(result, StreamingResponse): + result.body_iterator = wrap_genai_stream(workflow_span, result.body_iterator) + else: + try: + workflow_span.end() + except Exception: + log.exception('Unable to end chat workflow span') + return result + # Fan out: one task per model if metadata.get('session_id') and metadata.get('chat_id'): task_ids = [] @@ -2375,6 +2400,7 @@ async def get_app_config(request: Request): 'status': True, 'name': app.state.WEBUI_NAME, 'version': VERSION, + 'subpath': WEBUI_SUBPATH, 'default_locale': str(DEFAULT_LOCALE), 'i18n': config.get('ui.i18n') or {}, 'oauth': { @@ -2970,7 +2996,7 @@ async def get_manifest_json(): 'name': app.state.WEBUI_NAME, 'short_name': app.state.WEBUI_NAME, 'description': f'{app.state.WEBUI_NAME} is an open, extensible, user-friendly interface for AI that adapts to your workflow.', - 'start_url': '/', + 'start_url': f'{WEBUI_SUBPATH}/', 'display': 'standalone', 'background_color': '#343541', 'icons': [ @@ -2978,20 +3004,20 @@ async def get_manifest_json(): # Do not alter, remove, obscure, or replace it except as LICENSE permits: # https://docs.openwebui.com/license. { - 'src': '/static/logo.png', + 'src': f'{WEBUI_SUBPATH}/static/logo.png', 'type': 'image/png', 'sizes': '500x500', 'purpose': 'any', }, { - 'src': '/static/logo.png', + 'src': f'{WEBUI_SUBPATH}/static/logo.png', 'type': 'image/png', 'sizes': '500x500', 'purpose': 'maskable', }, ], 'share_target': { - 'action': '/', + 'action': f'{WEBUI_SUBPATH}/', 'method': 'GET', 'params': {'text': 'shared'}, }, @@ -3136,9 +3162,9 @@ def swagger_ui_html(*args, **kwargs): return get_swagger_ui_html( *args, **kwargs, - swagger_js_url='/static/swagger-ui/swagger-ui-bundle.js', - swagger_css_url='/static/swagger-ui/swagger-ui.css', - swagger_favicon_url='/static/swagger-ui/favicon.png', + swagger_js_url=f'{WEBUI_SUBPATH}/static/swagger-ui/swagger-ui-bundle.js', + swagger_css_url=f'{WEBUI_SUBPATH}/static/swagger-ui/swagger-ui.css', + swagger_favicon_url=f'{WEBUI_SUBPATH}/static/swagger-ui/favicon.png', ) diff --git a/backend/open_webui/retrieval/utils.py b/backend/open_webui/retrieval/utils.py index a0b2d3a11e403d9f0dc0f6ca8ae1c89a6c92d2ec..931b8f893eed1fc36b6a4b0fee21a8144da03202 100644 --- a/backend/open_webui/retrieval/utils.py +++ b/backend/open_webui/retrieval/utils.py @@ -56,8 +56,11 @@ from open_webui.utils.access_control.files import get_owner_accessible_folder_fi from open_webui.utils.access_control.folders import has_folder_access from open_webui.utils.headers import get_json_bearer_headers, include_user_info_headers from open_webui.utils.misc import get_content_from_message, get_message_list +from open_webui.utils.telemetry.genai.retrieval import instrument_reranker, traced_retrieval +from open_webui.utils.telemetry.genai.tracer import GenAITracer log = logging.getLogger(__name__) +ai_tracer = GenAITracer() from typing import Any @@ -1258,8 +1261,11 @@ def get_reranking_function(reranking_engine, reranking_model, reranking_function if reranking_function is None: return None if reranking_engine == 'external': - return lambda query, documents, user=None: reranking_function.predict( - [(query, doc.page_content) for doc in documents], user=user + return instrument_reranker( + tracer=ai_tracer, + reranking_function=lambda query, documents, user=None: reranking_function.predict( + [(query, doc.page_content) for doc in documents], user=user + ), ) else: @@ -1269,7 +1275,7 @@ def get_reranking_function(reranking_engine, reranking_model, reranking_function [(query, doc.page_content) for doc in documents], batch_size=int(reranking_batch_size) ) - return predict + return instrument_reranker(tracer=ai_tracer, reranking_function=predict) # UUIDs, SHA-256 digests, and prefixed variants thereof all fit [A-Za-z0-9_-]. @@ -1353,6 +1359,7 @@ def filter_source_metadata(metadata: dict) -> dict: return {key: metadata[key] for key in RAG_SOURCE_METADATA_KEYS if metadata.get(key) is not None} +@traced_retrieval(ai_tracer) async def get_sources_from_items( request, items, diff --git a/backend/open_webui/routers/models.py b/backend/open_webui/routers/models.py index 6925275d98020d6a225d32567a02511473c9cae1..ff829a530974c4df46479a2592a8d4b54006160e 100644 --- a/backend/open_webui/routers/models.py +++ b/backend/open_webui/routers/models.py @@ -999,6 +999,10 @@ async def get_model_profile_image( bypass_access_control = BYPASS_MODEL_ACCESS_CONTROL or (user.role == 'admin' and BYPASS_ADMIN_ACCESS_CONTROL) + # Prefix static-asset redirects with the ASGI root_path so they keep the + # reverse-proxy subpath (a bare "/static/..." Location drops it → 404). + root_path = request.scope.get('root_path', '').rstrip('/') + # First, check the database for regular models model_meta = await Models.get_model_meta_by_id(id, db=db) if model_meta: @@ -1056,7 +1060,7 @@ async def get_model_profile_image( # Do not alter, remove, obscure, or replace it except as LICENSE permits: # https://docs.openwebui.com/license. return RedirectResponse( - url='/static/favicon.png', + url=f'{root_path}/static/favicon.png', status_code=status.HTTP_302_FOUND, ) @@ -1078,7 +1082,7 @@ async def get_model_profile_image( safe_static = _safe_static_redirect_path(profile_image_url) if safe_static: return RedirectResponse( - url=safe_static, + url=f'{root_path}{safe_static}', status_code=status.HTTP_302_FOUND, ) @@ -1086,7 +1090,7 @@ async def get_model_profile_image( # Do not alter, remove, obscure, or replace it except as LICENSE permits: # https://docs.openwebui.com/license. return RedirectResponse( - url='/static/favicon.png', + url=f'{root_path}/static/favicon.png', status_code=status.HTTP_302_FOUND, ) diff --git a/backend/open_webui/routers/ollama.py b/backend/open_webui/routers/ollama.py index fd5bf331aff445c95650e556da3a58455ddc6287..5a8a1f05372328ef1ffd7e82b76616d3fa484bf4 100644 --- a/backend/open_webui/routers/ollama.py +++ b/backend/open_webui/routers/ollama.py @@ -44,10 +44,13 @@ from open_webui.utils.payload import ( apply_system_prompt_to_body, ) from open_webui.utils.session_pool import cleanup_response, get_client_timeout, get_session, stream_wrapper +from open_webui.utils.telemetry.genai.inference import traced_inference +from open_webui.utils.telemetry.genai.tracer import GenAITracer from pydantic import BaseModel, ConfigDict, validator from sqlalchemy.ext.asyncio import AsyncSession log = logging.getLogger(__name__) +ai_tracer = GenAITracer() # Headers that become stale after aiohttp auto-decompresses the upstream # response body. Forwarding them verbatim causes desktop / programmatic @@ -89,7 +92,7 @@ async def send_get_request( return None -async def send_request( +async def _send_request( url: str, method: str = 'POST', *, @@ -183,6 +186,63 @@ async def send_request( await cleanup_response(r) +async def send_request( + url: str, + method: str = 'POST', + *, + payload: Union[str, bytes | None] = None, + key: str | None = None, + user: UserModel = None, + stream: bool = False, + passthrough: bool = False, + content_type: str | None = None, + metadata: dict | None = None, + api_config: dict | None = None, + request: Request | None = None, +): + if '/chat/completions' not in url: + return await _send_request( + url, + method, + payload=payload, + key=key, + user=user, + stream=stream, + passthrough=passthrough, + content_type=content_type, + metadata=metadata, + api_config=api_config, + request=request, + ) + + try: + request_payload = JSONCodec.loads(payload) if isinstance(payload, (str, bytes)) else (payload or {}) + except Exception: + request_payload = {} + + async def call(): + return await _send_request( + url, + method, + payload=payload, + key=key, + user=user, + stream=stream, + passthrough=passthrough, + content_type=content_type, + metadata=metadata, + api_config=api_config, + request=request, + ) + + return await traced_inference( + ai_tracer, + provider='ollama', + request_payload=request_payload, + call=call, + ) + + def get_api_key(idx, url, configs): parsed_url = urlparse(url) base_url = f'{parsed_url.scheme}://{parsed_url.netloc}' diff --git a/backend/open_webui/routers/openai.py b/backend/open_webui/routers/openai.py index 8548a8ab0dd6c99c6585fb881f36db4b21f2799b..d867b3a3bccac559ab9993f39bac1a3783f1c290 100644 --- a/backend/open_webui/routers/openai.py +++ b/backend/open_webui/routers/openai.py @@ -54,10 +54,13 @@ from open_webui.utils.session_pool import ( get_session, stream_wrapper, ) +from open_webui.utils.telemetry.genai.inference import traced_inference +from open_webui.utils.telemetry.genai.tracer import GenAITracer from pydantic import BaseModel, ConfigDict from sqlalchemy.ext.asyncio import AsyncSession log = logging.getLogger(__name__) +ai_tracer = GenAITracer() ########################################## @@ -1408,6 +1411,24 @@ async def generate_chat_completion( request: Request, form_data: dict, user=Depends(get_verified_user), +): + request_payload = dict(form_data) + + async def call(): + return await _generate_chat_completion(request, form_data, user) + + return await traced_inference( + ai_tracer, + provider='openai-compatible', + request_payload=request_payload, + call=call, + ) + + +async def _generate_chat_completion( + request: Request, + form_data: dict, + user=Depends(get_verified_user), ): if not await Config.get('openai.enable'): raise HTTPException(status_code=503, detail='OpenAI API is disabled') diff --git a/backend/open_webui/socket/main.py b/backend/open_webui/socket/main.py index c2e7d810d57bcc4440b2c0945f9f4a34e6966d1d..6900836850636ca39f01da23339ca128207a4c78 100644 --- a/backend/open_webui/socket/main.py +++ b/backend/open_webui/socket/main.py @@ -34,6 +34,7 @@ from open_webui.env import ( WEBSOCKET_SERVER_LOGGING, WEBSOCKET_SERVER_PING_INTERVAL, WEBSOCKET_SERVER_PING_TIMEOUT, + WEBUI_SUBPATH, ) from open_webui.models.access_grants import AccessGrants from open_webui.models.channels import Channels @@ -321,9 +322,14 @@ async def periodic_usage_pool_cleanup(): await asyncio.sleep(retry_delay) +# engineio's ASGIApp matches the *full* request path (it does not subtract the +# ASGI root_path), so under a reverse-proxy subpath the browser hits +# `${WEBUI_SUBPATH}/ws/socket.io` and the handshake would otherwise 404 into a +# websocket-scope `not_found`. Prefix socketio_path with WEBUI_SUBPATH so it +# stays aligned with the frontend client (`path: ${base}/ws/socket.io`). app = socketio.ASGIApp( sio, - socketio_path='/ws/socket.io', + socketio_path=f'{WEBUI_SUBPATH}/ws/socket.io', ) diff --git a/backend/open_webui/test/apps/webui/utils/telemetry/golden/rag_chat.json b/backend/open_webui/test/apps/webui/utils/telemetry/golden/rag_chat.json new file mode 100644 index 0000000000000000000000000000000000000000..520090f55ead96053ccc3a1b91ec637118e1df13 --- /dev/null +++ b/backend/open_webui/test/apps/webui/utils/telemetry/golden/rag_chat.json @@ -0,0 +1,5 @@ +[ + {"name": "embedding", "parent": "retrieval", "attributes": ["openinference.span.kind"]}, + {"name": "retrieval", "parent": "invoke_workflow open-webui.chat", "attributes": ["gen_ai.operation.name", "gen_ai.retrieval.documents", "openinference.span.kind", "openwebui.retrieval.top_k"]}, + {"name": "invoke_workflow open-webui.chat", "parent": null, "attributes": ["gen_ai.operation.name", "openinference.span.kind", "openwebui.operation.purpose"]} +] diff --git a/backend/open_webui/test/apps/webui/utils/telemetry/golden/rag_rerank_tool.json b/backend/open_webui/test/apps/webui/utils/telemetry/golden/rag_rerank_tool.json new file mode 100644 index 0000000000000000000000000000000000000000..9d5a6884b624a62bafaf2fffbfb6ecf61070d2d6 --- /dev/null +++ b/backend/open_webui/test/apps/webui/utils/telemetry/golden/rag_rerank_tool.json @@ -0,0 +1,8 @@ +[ + {"name": "embedding", "parent": "retrieval", "attributes": ["openinference.span.kind"]}, + {"name": "vector_search", "parent": "retrieval", "attributes": []}, + {"name": "rerank", "parent": "retrieval", "attributes": ["openinference.span.kind", "openwebui.retrieval.candidate_count"]}, + {"name": "execute_tool", "parent": "retrieval", "attributes": ["gen_ai.operation.name", "gen_ai.tool.call.id", "gen_ai.tool.name", "gen_ai.tool.type", "openinference.span.kind"]}, + {"name": "retrieval", "parent": "invoke_workflow open-webui.chat", "attributes": ["gen_ai.operation.name", "gen_ai.retrieval.documents", "openinference.span.kind", "openwebui.retrieval.top_k"]}, + {"name": "invoke_workflow open-webui.chat", "parent": null, "attributes": ["gen_ai.operation.name", "openinference.span.kind", "openwebui.operation.purpose"]} +] diff --git a/backend/open_webui/test/apps/webui/utils/telemetry/golden/simple_chat.json b/backend/open_webui/test/apps/webui/utils/telemetry/golden/simple_chat.json new file mode 100644 index 0000000000000000000000000000000000000000..9d0688479293683abd76750dc41f3993c76fab58 --- /dev/null +++ b/backend/open_webui/test/apps/webui/utils/telemetry/golden/simple_chat.json @@ -0,0 +1,4 @@ +[ + {"name": "chat_completion", "parent": "invoke_workflow open-webui.chat", "attributes": ["gen_ai.provider.name", "gen_ai.request.model", "gen_ai.response.finish_reasons", "gen_ai.response.id", "gen_ai.response.model", "gen_ai.usage.input_tokens", "gen_ai.usage.output_tokens", "openinference.span.kind", "openwebui.operation.purpose"]}, + {"name": "invoke_workflow open-webui.chat", "parent": null, "attributes": ["gen_ai.operation.name", "openinference.span.kind", "openwebui.operation.purpose"]} +] diff --git a/backend/open_webui/test/apps/webui/utils/telemetry/golden/tool_loop.json b/backend/open_webui/test/apps/webui/utils/telemetry/golden/tool_loop.json new file mode 100644 index 0000000000000000000000000000000000000000..d20d1e45a7e032f58ec4ab655e20e3478e64712f --- /dev/null +++ b/backend/open_webui/test/apps/webui/utils/telemetry/golden/tool_loop.json @@ -0,0 +1,5 @@ +[ + {"name": "chat_completion", "parent": "invoke_workflow open-webui.chat", "attributes": ["gen_ai.provider.name", "gen_ai.request.model", "gen_ai.response.finish_reasons", "gen_ai.response.id", "gen_ai.response.model", "gen_ai.usage.input_tokens", "gen_ai.usage.output_tokens", "openinference.span.kind", "openwebui.operation.purpose"]}, + {"name": "chat_completion", "parent": "invoke_workflow open-webui.chat", "attributes": ["gen_ai.provider.name", "gen_ai.request.model", "gen_ai.response.finish_reasons", "gen_ai.response.id", "gen_ai.response.model", "gen_ai.usage.input_tokens", "gen_ai.usage.output_tokens", "openinference.span.kind", "openwebui.operation.purpose"]}, + {"name": "invoke_workflow open-webui.chat", "parent": null, "attributes": ["gen_ai.operation.name", "openinference.span.kind", "openwebui.operation.purpose"]} +] diff --git a/backend/open_webui/test/apps/webui/utils/test_genai_conformance.py b/backend/open_webui/test/apps/webui/utils/test_genai_conformance.py new file mode 100644 index 0000000000000000000000000000000000000000..8f62c20fa0b59e050d6907b196cc83227e977799 --- /dev/null +++ b/backend/open_webui/test/apps/webui/utils/test_genai_conformance.py @@ -0,0 +1,228 @@ +import asyncio +import json +from pathlib import Path + +import pytest +from open_webui.utils.telemetry.genai.config import GenAIConfig +from open_webui.utils.telemetry.genai.inference import traced_inference +from open_webui.utils.telemetry.genai.retrieval import instrument_embedding, instrument_reranker +from open_webui.utils.telemetry.genai.tools import traced_tool +from open_webui.utils.telemetry.genai.tracer import GenAITracer +from opentelemetry import trace +from opentelemetry.sdk.trace import TracerProvider +from opentelemetry.sdk.trace.export import SimpleSpanProcessor +from opentelemetry.sdk.trace.export.in_memory_span_exporter import InMemorySpanExporter +from opentelemetry.trace import StatusCode + +GOLDEN_DIR = Path(__file__).parent / 'telemetry' / 'golden' + + +def setup(config=None): + exporter = InMemorySpanExporter() + provider = TracerProvider() + provider.add_span_processor(SimpleSpanProcessor(exporter)) + return GenAITracer(provider.get_tracer('conformance'), config or GenAIConfig()), exporter + + +def topology(spans): + by_id = {span.context.span_id: span.name for span in spans} + return [ + { + 'name': span.name, + 'parent': by_id.get(span.parent.span_id) if span.parent else None, + 'attributes': sorted(span.attributes), + } + for span in spans + ] + + +async def successful_response(): + return { + 'id': 'response-id', + 'model': 'actual-model', + 'usage': {'prompt_tokens': 2, 'completion_tokens': 3}, + 'choices': [{'finish_reason': 'stop'}], + } + + +async def build_scenario(name, tracer): + with tracer.workflow('invoke_workflow open-webui.chat', operation_name='invoke_workflow', purpose='primary'): + if name in {'simple_chat', 'openai', 'ollama', 'no_chat_id', 'background_title', 'followup'}: + task = None + if name == 'background_title': + task = 'title_generation' + elif name == 'followup': + task = 'follow_up_generation' + await traced_inference( + tracer, + provider='ollama' if name == 'ollama' else 'openai-compatible', + request_payload={'model': 'requested-model', 'metadata': {'task': task} if task else {}}, + call=successful_response, + ) + elif name == 'streaming': + + async def stream(): + yield 'first' + yield 'final' + + async def response(): + return type('Response', (), {'body_iterator': stream()})() + + result = await traced_inference( + tracer, + provider='openai-compatible', + request_payload={'model': 'model', 'stream': True}, + call=response, + ) + [item async for item in result.body_iterator] + elif name == 'cancelled': + span = tracer.tracer.start_span('chat_completion') + + async def stream(): + yield 'first' + await asyncio.sleep(10) + + from open_webui.utils.telemetry.genai.streaming import wrap_stream + + wrapped = wrap_stream(span, stream()) + await wrapped.__anext__() + await wrapped.aclose() + elif name in {'one_tool', 'multiple_tools', 'mcp_tool'}: + with tracer.inference('chat_completion'): + count = 1 if name != 'multiple_tools' else 2 + for index in range(count): + + async def call(index=index): + return {'result': index} + + await traced_tool( + tracer, + name='mcp' if name == 'mcp_tool' else f'tool-{index}', + call_id=f'call-{index}', + tool_type='mcp' if name == 'mcp_tool' else 'function', + arguments={'index': index}, + direct=False, + call=call, + ) + if name == 'one_tool': + await traced_inference( + tracer, + provider='openai-compatible', + request_payload={'model': 'model'}, + call=successful_response, + ) + elif name == 'multi_round': + await traced_inference( + tracer, provider='openai-compatible', request_payload={'model': 'model'}, call=successful_response + ) + await traced_inference( + tracer, provider='openai-compatible', request_payload={'model': 'model'}, call=successful_response + ) + elif name in {'knowledge', 'hybrid', 'rerank', 'embedding', 'retrieval_failure'}: + with tracer.retrieval('retrieval', operation_name='retrieval', retrieval_query='query', top_k=3): + + async def embedding(query, prefix=None): + return [[0.1, 0.2]] + + await instrument_embedding(tracer, embedding)('query') + if name in {'hybrid', 'rerank'}: + with tracer.tracer.start_as_current_span('bm25_search' if name == 'hybrid' else 'vector_search'): + pass + if name == 'rerank': + instrument_reranker(tracer, lambda query, docs: [0.9])('query', ['doc']) + await traced_tool( + tracer, + name='retrieval-tool', + call_id='retrieval-call', + tool_type='function', + arguments={'query': 'query'}, + direct=False, + call=lambda: successful_response(), + ) + if name == 'retrieval_failure': + with pytest.raises(RuntimeError): + raise RuntimeError('retrieval failed') + if name in {'knowledge', 'hybrid', 'rerank'}: + current = trace.get_current_span() + current.set_attribute( + 'gen_ai.retrieval.documents', ['{"document_id":"doc-1","rank":1,"score":0.9}'] + ) + elif name == 'llm_failure': + + async def fail(): + raise ValueError('llm failed') + + with pytest.raises(ValueError): + await traced_inference( + tracer, provider='openai-compatible', request_payload={'model': 'model'}, call=fail + ) + elif name == 'no_content': + await traced_inference( + tracer, + provider='openai-compatible', + request_payload={'model': 'model', 'messages': [{'content': 'secret'}]}, + call=successful_response, + ) + + +@pytest.mark.asyncio +@pytest.mark.parametrize( + 'scenario', + [ + 'simple_chat', + 'streaming', + 'cancelled', + 'openai', + 'ollama', + 'one_tool', + 'multiple_tools', + 'multi_round', + 'knowledge', + 'hybrid', + 'rerank', + 'embedding', + 'mcp_tool', + 'background_title', + 'followup', + 'no_chat_id', + 'llm_failure', + 'retrieval_failure', + 'no_content', + ], +) +async def test_conformance_scenarios(scenario): + tracer, exporter = setup() + await build_scenario(scenario, tracer) + spans = exporter.get_finished_spans() + assert spans + if scenario == 'llm_failure': + assert any(span.status.status_code is StatusCode.ERROR for span in spans) + if scenario == 'no_content': + assert all('gen_ai.input.messages' not in span.attributes for span in spans) + if scenario == 'background_title': + assert any(span.attributes.get('openwebui.operation.purpose') == 'title_generation' for span in spans) + if scenario == 'followup': + assert any(span.attributes.get('openwebui.operation.purpose') == 'followup_generation' for span in spans) + + +@pytest.mark.asyncio +@pytest.mark.parametrize('scenario', ['simple_chat', 'knowledge', 'multi_round', 'rerank']) +async def test_golden_topology(scenario): + tracer, exporter = setup() + await build_scenario(scenario, tracer) + actual = topology(exporter.get_finished_spans()) + golden_name = { + 'knowledge': 'rag_chat', + 'multi_round': 'tool_loop', + 'rerank': 'rag_rerank_tool', + }.get(scenario, scenario) + expected = json.loads((GOLDEN_DIR / f'{golden_name}.json').read_text()) + assert actual == expected + + +def test_tracing_suppressed_produces_no_semantic_spans(): + exporter = InMemorySpanExporter() + tracer = GenAITracer(trace.get_tracer('suppressed'), GenAIConfig()) + with tracer.workflow('invoke_workflow open-webui.chat'): + pass + assert exporter.get_finished_spans() == () diff --git a/backend/open_webui/test/apps/webui/utils/test_genai_inference.py b/backend/open_webui/test/apps/webui/utils/test_genai_inference.py new file mode 100644 index 0000000000000000000000000000000000000000..52d3f16d9d5b3da0871f1c80c1fe3370c5c680ef --- /dev/null +++ b/backend/open_webui/test/apps/webui/utils/test_genai_inference.py @@ -0,0 +1,134 @@ +from types import SimpleNamespace + +import pytest +from open_webui.utils.telemetry.genai.config import GenAIConfig +from open_webui.utils.telemetry.genai.inference import traced_inference +from open_webui.utils.telemetry.genai.tracer import GenAITracer +from opentelemetry.sdk.trace import TracerProvider +from opentelemetry.sdk.trace.export import SimpleSpanProcessor +from opentelemetry.sdk.trace.export.in_memory_span_exporter import InMemorySpanExporter + + +@pytest.mark.asyncio +async def test_two_round_tool_loop_creates_independent_inference_spans(): + exporter = InMemorySpanExporter() + provider = TracerProvider() + provider.add_span_processor(SimpleSpanProcessor(exporter)) + tracer = GenAITracer(provider.get_tracer('inference-test'), GenAIConfig()) + responses = [ + { + 'id': 'one', + 'model': 'requested', + 'usage': {'prompt_tokens': 3, 'completion_tokens': 4}, + 'choices': [{'finish_reason': 'tool_calls'}], + }, + { + 'id': 'two', + 'model': 'actual', + 'usage': {'prompt_tokens': 5, 'completion_tokens': 6}, + 'choices': [{'finish_reason': 'stop'}], + }, + ] + + async def call(): + return responses.pop(0) + + await traced_inference(tracer, provider='openai-compatible', request_payload={'model': 'requested'}, call=call) + await traced_inference(tracer, provider='ollama', request_payload={'model': 'requested'}, call=call) + spans = exporter.get_finished_spans() + assert len(spans) == 2 + assert [span.attributes['gen_ai.usage.input_tokens'] for span in spans] == [3, 5] + assert [span.attributes['gen_ai.usage.output_tokens'] for span in spans] == [4, 6] + assert [span.attributes['gen_ai.response.id'] for span in spans] == ['one', 'two'] + + +@pytest.mark.asyncio +async def test_content_is_absent_for_openai_and_ollama_when_disabled(): + exporter = InMemorySpanExporter() + provider = TracerProvider() + provider.add_span_processor(SimpleSpanProcessor(exporter)) + tracer = GenAITracer(provider.get_tracer('privacy-test'), GenAIConfig()) + + async def call(): + return {'id': 'response', 'model': 'model', 'choices': [{'finish_reason': 'stop'}]} + + payload = {'model': 'model', 'messages': [{'role': 'user', 'content': 'secret'}]} + await traced_inference(tracer, provider='openai-compatible', request_payload=payload, call=call) + await traced_inference(tracer, provider='ollama', request_payload=payload, call=call) + for span in exporter.get_finished_spans(): + assert 'gen_ai.input.messages' not in span.attributes + assert 'gen_ai.output.messages' not in span.attributes + + +@pytest.mark.asyncio +async def test_buffered_output_is_captured_when_enabled(): + exporter = InMemorySpanExporter() + provider = TracerProvider() + provider.add_span_processor(SimpleSpanProcessor(exporter)) + tracer = GenAITracer( + provider.get_tracer('output-content-test'), + GenAIConfig(capture_outputs=True, content_max_length=12), + ) + + async def call(): + return { + 'id': 'response', + 'model': 'model', + 'choices': [{'message': {'role': 'assistant', 'content': 'output is longer'}}], + } + + await traced_inference(tracer, provider='openai-compatible', request_payload={'model': 'model'}, call=call) + output = exporter.get_finished_spans()[0].attributes['gen_ai.output.messages'] + assert len(output) == 12 + assert output.startswith('[{"role": "a') + + +@pytest.mark.asyncio +async def test_streaming_inference_span_closes_after_body_iterator(): + exporter = InMemorySpanExporter() + provider = TracerProvider() + provider.add_span_processor(SimpleSpanProcessor(exporter)) + tracer = GenAITracer(provider.get_tracer('stream-test'), GenAIConfig()) + + async def body(): + yield 'chunk' + + response = SimpleNamespace(body_iterator=body()) + result = await traced_inference( + tracer, + provider='ollama', + request_payload={'model': 'model', 'stream': True}, + call=lambda: _return(response), + ) + assert exporter.get_finished_spans() == () + assert [item async for item in result.body_iterator] == ['chunk'] + assert len(exporter.get_finished_spans()) == 1 + + +@pytest.mark.asyncio +async def test_streaming_output_is_captured_after_body_iteration(): + exporter = InMemorySpanExporter() + provider = TracerProvider() + provider.add_span_processor(SimpleSpanProcessor(exporter)) + tracer = GenAITracer( + provider.get_tracer('stream-output-test'), + GenAIConfig(capture_outputs=True), + ) + + async def body(): + yield 'chunk-one' + yield 'chunk-two' + + response = SimpleNamespace(body_iterator=body()) + result = await traced_inference( + tracer, + provider='ollama', + request_payload={'model': 'model', 'stream': True}, + call=lambda: _return(response), + ) + assert [item async for item in result.body_iterator] == ['chunk-one', 'chunk-two'] + assert exporter.get_finished_spans()[0].attributes['gen_ai.output.messages'] == 'chunk-onechunk-two' + + +async def _return(value): + return value diff --git a/backend/open_webui/test/apps/webui/utils/test_genai_performance.py b/backend/open_webui/test/apps/webui/utils/test_genai_performance.py new file mode 100644 index 0000000000000000000000000000000000000000..2a6887c671f0e6f748f4e9689b7603a6d0e17ba0 --- /dev/null +++ b/backend/open_webui/test/apps/webui/utils/test_genai_performance.py @@ -0,0 +1,101 @@ +import timeit + +import pytest +from open_webui.utils.telemetry.genai import extractors +from open_webui.utils.telemetry.genai.config import GenAIConfig +from open_webui.utils.telemetry.genai.retrieval import traced_retrieval +from open_webui.utils.telemetry.genai.tools import traced_tool +from open_webui.utils.telemetry.genai.tracer import GenAITracer +from opentelemetry.sdk.trace import TracerProvider +from opentelemetry.sdk.trace.export import SimpleSpanProcessor +from opentelemetry.sdk.trace.export.in_memory_span_exporter import InMemorySpanExporter + + +def _tracer(config): + exporter = InMemorySpanExporter() + provider = TracerProvider() + provider.add_span_processor(SimpleSpanProcessor(exporter)) + return GenAITracer(provider.get_tracer('performance-test'), config), exporter + + +def test_disabled_tracing_stays_within_agreed_overhead(): + tracer, _ = _tracer(GenAIConfig(enabled=False)) + + def baseline(): + return None + + def disabled_trace(): + with tracer.inference(): + pass + + baseline_time = min(timeit.repeat(baseline, number=20_000, repeat=5)) + disabled_time = min(timeit.repeat(disabled_trace, number=20_000, repeat=5)) + # Agreed tolerance: at most 20 ms total for 20,000 disabled spans. + assert disabled_time - baseline_time <= 0.02 + + +def test_disabled_capture_does_not_serialize(monkeypatch): + def unexpected_serialization(*args, **kwargs): + raise AssertionError('content was serialized while capture was disabled') + + monkeypatch.setattr(extractors.json, 'dumps', unexpected_serialization) + tracer, _ = _tracer(GenAIConfig()) + with tracer.inference(inputs={'secret': 'prompt'}, tool_arguments={'secret': 'args'}): + pass + + disabled_tracer, _ = _tracer(GenAIConfig(enabled=False, capture_inputs=True)) + with disabled_tracer.inference(inputs={'secret': 'prompt'}): + pass + + +@pytest.mark.asyncio +async def test_explicit_content_and_topology_caps(): + config = GenAIConfig( + capture_inputs=True, + capture_outputs=True, + capture_tool_arguments=True, + capture_tool_results=True, + capture_retrieval_documents=True, + content_max_length=5, + retrieval_max_documents=2, + exception_max_length=7, + ) + tracer, exporter = _tracer(config) + + async def tool_call(): + return 'result-too-long' + + await traced_tool( + tracer, + name='limited-tool', + call_id='call-1', + tool_type='function', + arguments='arguments-too-long', + direct=False, + call=tool_call, + ) + + with pytest.raises(ValueError): + with tracer.inference(inputs='message-too-long'): + raise ValueError('exception-message-too-long') + + @traced_retrieval(tracer) + async def retrieve(items, queries, k, hybrid_search=False, embedding_function=None): + return items + + items = [ + {'metadata': {'document_id': f'doc-{index}'}, 'document': [f'content-{index}'], 'distances': [0.1]} + for index in range(4) + ] + await retrieve(items=items, queries=['query'], k=4) + + spans = exporter.get_finished_spans() + tool_span = next(span for span in spans if span.name == 'execute_tool') + assert tool_span.attributes['gen_ai.tool.call.arguments'] == 'argum' + assert tool_span.attributes['gen_ai.tool.call.result'] == 'resul' + inference_span = next(span for span in spans if span.name == 'inference') + assert inference_span.attributes['gen_ai.input.messages'] == 'messa' + exception_event = next(event for event in inference_span.events if event.name == 'exception') + assert exception_event.attributes['exception.message'] == 'excepti' + retrieval_span = next(span for span in spans if span.name == 'retrieval') + assert len(retrieval_span.attributes['gen_ai.retrieval.documents']) == 2 diff --git a/backend/open_webui/test/apps/webui/utils/test_genai_purpose.py b/backend/open_webui/test/apps/webui/utils/test_genai_purpose.py new file mode 100644 index 0000000000000000000000000000000000000000..d9dddf95d3355377da8f131224b85110f73f514d --- /dev/null +++ b/backend/open_webui/test/apps/webui/utils/test_genai_purpose.py @@ -0,0 +1,39 @@ +import pytest +from open_webui.utils.telemetry.genai.config import GenAIConfig +from open_webui.utils.telemetry.genai.inference import traced_inference +from open_webui.utils.telemetry.genai.tracer import GenAITracer +from opentelemetry.sdk.trace import TracerProvider +from opentelemetry.sdk.trace.export import SimpleSpanProcessor +from opentelemetry.sdk.trace.export.in_memory_span_exporter import InMemorySpanExporter + + +@pytest.mark.asyncio +async def test_task_metadata_maps_background_purpose_and_primary_defaults(): + exporter = InMemorySpanExporter() + provider = TracerProvider() + provider.add_span_processor(SimpleSpanProcessor(exporter)) + tracer = GenAITracer(provider.get_tracer('purpose-test'), GenAIConfig()) + + async def call(): + return {'id': 'response', 'model': 'model', 'choices': [{'finish_reason': 'stop'}]} + + await traced_inference( + tracer, + provider='openai-compatible', + request_payload={'model': 'model', 'metadata': {'task': 'title_generation'}}, + call=call, + ) + await traced_inference(tracer, provider='openai-compatible', request_payload={'model': 'model'}, call=call) + spans = exporter.get_finished_spans() + assert spans[0].attributes['openwebui.operation.purpose'] == 'title_generation' + assert spans[1].attributes['openwebui.operation.purpose'] == 'primary' + + +def test_primary_workflow_purpose_is_explicit(): + exporter = InMemorySpanExporter() + provider = TracerProvider() + provider.add_span_processor(SimpleSpanProcessor(exporter)) + tracer = GenAITracer(provider.get_tracer('workflow-purpose-test'), GenAIConfig()) + with tracer.workflow('invoke_workflow open-webui.chat', operation_name='invoke_workflow', purpose='primary'): + pass + assert exporter.get_finished_spans()[0].attributes['openwebui.operation.purpose'] == 'primary' diff --git a/backend/open_webui/test/apps/webui/utils/test_genai_retrieval.py b/backend/open_webui/test/apps/webui/utils/test_genai_retrieval.py new file mode 100644 index 0000000000000000000000000000000000000000..24a224718549aa88e795a5de41685c35c8d14db8 --- /dev/null +++ b/backend/open_webui/test/apps/webui/utils/test_genai_retrieval.py @@ -0,0 +1,80 @@ +import pytest +from open_webui.utils.telemetry.genai.config import GenAIConfig +from open_webui.utils.telemetry.genai.retrieval import instrument_embedding, instrument_reranker, traced_retrieval +from open_webui.utils.telemetry.genai.tracer import GenAITracer +from opentelemetry.sdk.trace import TracerProvider +from opentelemetry.sdk.trace.export import SimpleSpanProcessor +from opentelemetry.sdk.trace.export.in_memory_span_exporter import InMemorySpanExporter + + +@pytest.mark.asyncio +async def test_retrieval_tree_has_embedding_and_rerank_children(): + exporter = InMemorySpanExporter() + provider = TracerProvider() + provider.add_span_processor(SimpleSpanProcessor(exporter)) + tracer = GenAITracer(provider.get_tracer('retrieval-test'), GenAIConfig()) + + async def embedding(query, prefix=None): + return [[0.1, 0.2]] + + def rerank(query, documents): + return [0.9 for _ in documents] + + with tracer.retrieval('retrieval') as parent: + await instrument_embedding(tracer, embedding)('query') + instrument_reranker(tracer, rerank)('query', ['document']) + spans = exporter.get_finished_spans() + assert [span.name for span in spans] == ['embedding', 'rerank', 'retrieval'] + assert all(span.parent.span_id == parent.get_span_context().span_id for span in spans[:2]) + assert spans[1].attributes['openinference.span.kind'] == 'RERANKER' + + +@pytest.mark.asyncio +async def test_embedding_and_retrieval_content_are_private_by_default(): + exporter = InMemorySpanExporter() + provider = TracerProvider() + provider.add_span_processor(SimpleSpanProcessor(exporter)) + tracer = GenAITracer(provider.get_tracer('retrieval-privacy-test'), GenAIConfig()) + + async def embedding(query, prefix=None): + return [[0.1, 0.2]] + + with tracer.retrieval('retrieval') as parent: + await instrument_embedding(tracer, embedding)('secret query') + parent.set_attribute('gen_ai.retrieval.documents', ['{"document_id": "doc-1", "rank": 1, "score": 0.9}']) + spans = exporter.get_finished_spans() + assert 'gen_ai.retrieval.query.text' not in spans[0].attributes + assert 'gen_ai.embeddings' not in spans[0].attributes + assert spans[-1].attributes['gen_ai.retrieval.documents'] + + +@pytest.mark.asyncio +async def test_embedding_vectors_and_document_content_are_capped_when_enabled(): + exporter = InMemorySpanExporter() + provider = TracerProvider() + provider.add_span_processor(SimpleSpanProcessor(exporter)) + tracer = GenAITracer( + provider.get_tracer('retrieval-content-test'), + GenAIConfig(capture_embedding_vectors=True, capture_retrieval_documents=True, content_max_length=8), + ) + + async def embedding(query, prefix=None): + return [[0.1, 0.2, 0.3, 0.4]] + + @traced_retrieval(tracer) + async def retrieve(items, queries, k, hybrid_search=False, embedding_function=None): + await instrument_embedding(tracer, embedding_function)('query') + return items + + await retrieve( + items=[{'metadata': {'document_id': 'doc-1'}, 'document': ['document-content-too-long']}], + queries=['query'], + k=1, + embedding_function=embedding, + ) + spans = exporter.get_finished_spans() + embedding_span = next(span for span in spans if span.name == 'embedding') + retrieval_span = next(span for span in spans if span.name == 'retrieval') + assert len(embedding_span.attributes['gen_ai.embeddings']) <= 8 + assert '"content": "document"' in retrieval_span.attributes['gen_ai.retrieval.documents'][0] + assert 'document-content-too-long' not in retrieval_span.attributes['gen_ai.retrieval.documents'][0] diff --git a/backend/open_webui/test/apps/webui/utils/test_genai_tools.py b/backend/open_webui/test/apps/webui/utils/test_genai_tools.py new file mode 100644 index 0000000000000000000000000000000000000000..d51de00b40817d1f6e859c21da379e2725e27869 --- /dev/null +++ b/backend/open_webui/test/apps/webui/utils/test_genai_tools.py @@ -0,0 +1,100 @@ +import pytest +from open_webui.utils.telemetry.genai.config import GenAIConfig +from open_webui.utils.telemetry.genai.tools import traced_tool +from open_webui.utils.telemetry.genai.tracer import GenAITracer +from opentelemetry.sdk.trace import TracerProvider +from opentelemetry.sdk.trace.export import SimpleSpanProcessor +from opentelemetry.sdk.trace.export.in_memory_span_exporter import InMemorySpanExporter + + +@pytest.mark.asyncio +async def test_two_tools_are_independent_children_of_inference(): + exporter = InMemorySpanExporter() + provider = TracerProvider() + provider.add_span_processor(SimpleSpanProcessor(exporter)) + tracer = GenAITracer(provider.get_tracer('tools-test'), GenAIConfig()) + + async def first(): + return 'first-result' + + async def second(): + return 'second-result' + + with tracer.inference('chat_completion') as parent: + await traced_tool( + tracer, + name='first', + call_id='call-1', + tool_type='function', + arguments={'x': 1}, + direct=False, + call=first, + ) + await traced_tool( + tracer, + name='second', + call_id='call-2', + tool_type='mcp', + arguments={'x': 2}, + direct=False, + call=second, + ) + + spans = exporter.get_finished_spans() + assert [span.name for span in spans] == ['execute_tool', 'execute_tool', 'chat_completion'] + tool_spans = spans[:2] + assert [span.attributes['gen_ai.tool.name'] for span in tool_spans] == ['first', 'second'] + assert [span.attributes['gen_ai.tool.call.id'] for span in tool_spans] == ['call-1', 'call-2'] + assert [span.attributes['gen_ai.tool.type'] for span in tool_spans] == ['function', 'mcp'] + assert all(span.parent.span_id == parent.get_span_context().span_id for span in tool_spans) + + +@pytest.mark.asyncio +async def test_tool_payloads_are_absent_when_capture_is_disabled(): + exporter = InMemorySpanExporter() + provider = TracerProvider() + provider.add_span_processor(SimpleSpanProcessor(exporter)) + tracer = GenAITracer(provider.get_tracer('tools-privacy-test'), GenAIConfig()) + + async def call(): + return {'secret': 'result'} + + await traced_tool( + tracer, + name='private', + call_id='call-private', + tool_type='builtin', + arguments={'secret': 'argument'}, + direct=False, + call=call, + ) + attributes = exporter.get_finished_spans()[0].attributes + assert 'gen_ai.tool.call.arguments' not in attributes + assert 'gen_ai.tool.call.result' not in attributes + + +@pytest.mark.asyncio +async def test_tool_payloads_are_captured_when_enabled(): + exporter = InMemorySpanExporter() + provider = TracerProvider() + provider.add_span_processor(SimpleSpanProcessor(exporter)) + tracer = GenAITracer( + provider.get_tracer('tools-content-test'), + GenAIConfig(capture_tool_arguments=True, capture_tool_results=True), + ) + + async def call(): + return {'result': 'ok'} + + await traced_tool( + tracer, + name='captured', + call_id='call-captured', + tool_type='openapi', + arguments={'argument': 'value'}, + direct=False, + call=call, + ) + attributes = exporter.get_finished_spans()[0].attributes + assert 'value' in attributes['gen_ai.tool.call.arguments'] + assert 'ok' in attributes['gen_ai.tool.call.result'] diff --git a/backend/open_webui/test/apps/webui/utils/test_genai_tracer.py b/backend/open_webui/test/apps/webui/utils/test_genai_tracer.py new file mode 100644 index 0000000000000000000000000000000000000000..3eb6e126ee87a7af4d0cd6a450342795be6e4a3e --- /dev/null +++ b/backend/open_webui/test/apps/webui/utils/test_genai_tracer.py @@ -0,0 +1,74 @@ +import asyncio + +import pytest +from open_webui.utils.telemetry.genai.config import GenAIConfig +from open_webui.utils.telemetry.genai.streaming import wrap_stream +from open_webui.utils.telemetry.genai.tracer import GenAITracer +from opentelemetry.sdk.trace import TracerProvider +from opentelemetry.sdk.trace.export import SimpleSpanProcessor +from opentelemetry.sdk.trace.export.in_memory_span_exporter import InMemorySpanExporter +from opentelemetry.trace import StatusCode + + +@pytest.fixture +def span_setup(): + exporter = InMemorySpanExporter() + provider = TracerProvider() + provider.add_span_processor(SimpleSpanProcessor(exporter)) + return GenAITracer(provider.get_tracer('test'), GenAIConfig()), exporter + + +def test_span_kind_and_structural_attributes(span_setup): + tracer, exporter = span_setup + with tracer.inference(operation_name='chat', provider='test', request_model='model'): + pass + span = exporter.get_finished_spans()[0] + assert span.attributes['gen_ai.operation.name'] == 'chat' + assert span.attributes['gen_ai.provider.name'] == 'test' + assert span.attributes['openinference.span.kind'] == 'LLM' + + +def test_exception_is_recorded_and_status_is_error(span_setup): + tracer, exporter = span_setup + with pytest.raises(ValueError): + with tracer.inference(): + raise ValueError('boom') + span = exporter.get_finished_spans()[0] + assert span.status.status_code is StatusCode.ERROR + assert any(event.name == 'exception' for event in span.events) + + +def test_content_is_absent_when_capture_is_disabled(span_setup): + tracer, exporter = span_setup + with tracer.inference(inputs='secret prompt', outputs='secret output', tool_arguments='secret args'): + pass + assert not any( + 'input' in key or 'output' in key or 'tool' in key for key in exporter.get_finished_spans()[0].attributes + ) + + +def test_content_is_truncated_when_capture_is_enabled(): + exporter = InMemorySpanExporter() + provider = TracerProvider() + provider.add_span_processor(SimpleSpanProcessor(exporter)) + config = GenAIConfig(capture_inputs=True, capture_outputs=True, content_max_length=5) + with GenAITracer(provider.get_tracer('test'), config).inference(inputs='123456789', outputs='abcdefgh'): + pass + span = exporter.get_finished_spans()[0] + assert span.attributes['gen_ai.input.messages'] == '12345' + assert span.attributes['gen_ai.output.messages'] == 'abcde' + + +@pytest.mark.asyncio +async def test_streaming_cancellation_closes_span(span_setup): + tracer, exporter = span_setup + span = tracer.tracer.start_span('stream') + + async def source(): + yield 'first' + await asyncio.sleep(10) + + wrapped = wrap_stream(span, source()) + assert await wrapped.__anext__() == 'first' + await wrapped.aclose() + assert len(exporter.get_finished_spans()) == 1 diff --git a/backend/open_webui/test/apps/webui/utils/test_genai_workflow_context.py b/backend/open_webui/test/apps/webui/utils/test_genai_workflow_context.py new file mode 100644 index 0000000000000000000000000000000000000000..166353887f05901e61fc8153ca466b1dec3ff372 --- /dev/null +++ b/backend/open_webui/test/apps/webui/utils/test_genai_workflow_context.py @@ -0,0 +1,68 @@ +import asyncio + +import pytest +from open_webui.utils.telemetry.genai.streaming import wrap_stream +from open_webui.utils.telemetry.genai.tracer import GenAITracer +from opentelemetry.sdk.trace import TracerProvider +from opentelemetry.sdk.trace.export import SimpleSpanProcessor +from opentelemetry.sdk.trace.export.in_memory_span_exporter import InMemorySpanExporter + + +@pytest.fixture +def tracer_setup(): + exporter = InMemorySpanExporter() + provider = TracerProvider() + provider.add_span_processor(SimpleSpanProcessor(exporter)) + return GenAITracer(provider.get_tracer('workflow-test')), exporter + + +@pytest.mark.asyncio +async def test_workflow_name_context_and_stream_lifetime(tracer_setup): + tracer, exporter = tracer_setup + span = tracer.tracer.start_span('invoke_workflow open-webui.chat') + stream_started = asyncio.Event() + + async def source(): + stream_started.set() + yield 'chunk' + await asyncio.sleep(0) + yield 'done' + + wrapped = wrap_stream(span, source()) + assert await wrapped.__anext__() == 'chunk' + await stream_started.wait() + assert exporter.get_finished_spans() == () + assert await wrapped.__anext__() == 'done' + with pytest.raises(StopAsyncIteration): + await wrapped.__anext__() + assert exporter.get_finished_spans()[0].name == 'invoke_workflow open-webui.chat' + + +def test_workflow_context_attributes_are_ids_only(tracer_setup): + tracer, exporter = tracer_setup + with tracer.workflow( + 'invoke_workflow open-webui.chat', + operation_name='invoke_workflow', + conversation_id='chat-123', + message_id='message-456', + purpose='primary', + ): + pass + span = exporter.get_finished_spans()[0] + assert span.name == 'invoke_workflow open-webui.chat' + assert span.attributes['gen_ai.conversation.id'] == 'chat-123' + assert span.attributes['openwebui.message.id'] == 'message-456' + assert span.attributes['openwebui.operation.purpose'] == 'primary' + assert 'chat-123' not in span.name + + +def test_background_workflow_can_set_non_primary_purpose(tracer_setup): + tracer, exporter = tracer_setup + with tracer.workflow( + 'invoke_workflow open-webui.chat', + operation_name='invoke_workflow', + conversation_id='chat-123', + purpose='title_generation', + ): + pass + assert exporter.get_finished_spans()[0].attributes['openwebui.operation.purpose'] == 'title_generation' diff --git a/backend/open_webui/test/apps/webui/utils/test_telemetry_setup.py b/backend/open_webui/test/apps/webui/utils/test_telemetry_setup.py new file mode 100644 index 0000000000000000000000000000000000000000..b409e92831d68ff1b4e3615b1f7d5b04ce2e218b --- /dev/null +++ b/backend/open_webui/test/apps/webui/utils/test_telemetry_setup.py @@ -0,0 +1,89 @@ +import importlib + + +class _Instrumentor: + def instrument(self): + pass + + +def _reload_exporter_modules(monkeypatch, exporter_headers): + monkeypatch.setenv('OTEL_EXPORTER_OTLP_HEADERS', 'foo=bar') + monkeypatch.delenv('OTEL_BASIC_AUTH_USERNAME', raising=False) + monkeypatch.delenv('OTEL_BASIC_AUTH_PASSWORD', raising=False) + + import opentelemetry.exporter.otlp.proto.grpc.trace_exporter as grpc_exporter + import opentelemetry.exporter.otlp.proto.http.trace_exporter as http_exporter + + importlib.reload(grpc_exporter) + importlib.reload(http_exporter) + + grpc = grpc_exporter.OTLPSpanExporter(endpoint='http://localhost:4317') + http = http_exporter.OTLPSpanExporter(endpoint='http://localhost:4318/v1/traces') + exporter_headers.append((grpc._headers, http._headers)) + + +def test_sdk_otlp_headers_are_used_when_basic_auth_is_unset(monkeypatch): + headers = [] + _reload_exporter_modules(monkeypatch, headers) + + grpc_headers, http_headers = headers[0] + assert ('foo', 'bar') in grpc_headers + assert http_headers.get('foo') == 'bar' + + +def test_setup_passes_none_headers_without_basic_auth(monkeypatch): + import open_webui.utils.telemetry.setup as telemetry_setup + + captured = [] + + class Exporter: + def __init__(self, **kwargs): + captured.append(kwargs) + + class Provider: + def add_span_processor(self, processor): + pass + + monkeypatch.setattr(telemetry_setup, 'HttpOTLPSpanExporter', Exporter) + monkeypatch.setattr(telemetry_setup, 'OTLPSpanExporter', Exporter) + monkeypatch.setattr(telemetry_setup.trace, 'set_tracer_provider', lambda provider: None) + monkeypatch.setattr(telemetry_setup.trace, 'get_tracer_provider', lambda: Provider()) + monkeypatch.setattr(telemetry_setup, 'Instrumentor', lambda **kwargs: _Instrumentor()) + monkeypatch.setattr(telemetry_setup, 'ENABLE_OTEL_TRACES', True) + monkeypatch.setattr(telemetry_setup, 'OTEL_BASIC_AUTH_USERNAME', '') + monkeypatch.setattr(telemetry_setup, 'OTEL_BASIC_AUTH_PASSWORD', '') + monkeypatch.setattr(telemetry_setup, 'OTEL_OTLP_SPAN_EXPORTER', 'grpc') + telemetry_setup.setup(None, None) + assert captured[-1]['headers'] is None + + monkeypatch.setattr(telemetry_setup, 'OTEL_OTLP_SPAN_EXPORTER', 'http') + telemetry_setup.setup(None, None) + assert captured[-1]['headers'] is None + + +def test_setup_preserves_basic_auth_and_grpc_insecure(monkeypatch): + import open_webui.utils.telemetry.setup as telemetry_setup + + captured = [] + + class Exporter: + def __init__(self, **kwargs): + captured.append(kwargs) + + class Provider: + def add_span_processor(self, processor): + pass + + monkeypatch.setattr(telemetry_setup, 'OTLPSpanExporter', Exporter) + monkeypatch.setattr(telemetry_setup.trace, 'set_tracer_provider', lambda provider: None) + monkeypatch.setattr(telemetry_setup.trace, 'get_tracer_provider', lambda: Provider()) + monkeypatch.setattr(telemetry_setup, 'Instrumentor', lambda **kwargs: _Instrumentor()) + monkeypatch.setattr(telemetry_setup, 'ENABLE_OTEL_TRACES', True) + monkeypatch.setattr(telemetry_setup, 'OTEL_BASIC_AUTH_USERNAME', 'user') + monkeypatch.setattr(telemetry_setup, 'OTEL_BASIC_AUTH_PASSWORD', 'pass') + monkeypatch.setattr(telemetry_setup, 'OTEL_OTLP_SPAN_EXPORTER', 'grpc') + monkeypatch.setattr(telemetry_setup, 'OTEL_EXPORTER_OTLP_INSECURE', True) + telemetry_setup.setup(None, None) + + assert captured[-1]['headers'] == [('authorization', 'Basic dXNlcjpwYXNz')] + assert captured[-1]['insecure'] is True diff --git a/backend/open_webui/utils/asgi_middleware.py b/backend/open_webui/utils/asgi_middleware.py index 2d9f4e190edd7c9abdaf455fb9143e5a6c9b28a9..7fcdce387437ab0ba221e523c27c690e5f8cb1c2 100644 --- a/backend/open_webui/utils/asgi_middleware.py +++ b/backend/open_webui/utils/asgi_middleware.py @@ -198,7 +198,8 @@ class AppHTTPMiddleware: redirect_params['q'] = text if redirect_params: - redirect_url = f'/?{urlencode(redirect_params)}' + root_path = (scope.get('root_path') or '').rstrip('/') + redirect_url = f'{root_path}/?{urlencode(redirect_params)}' response = RedirectResponse(url=redirect_url) await response(scope, receive, send) return True diff --git a/backend/open_webui/utils/middleware.py b/backend/open_webui/utils/middleware.py index 6eca5112521e8dfb6b78c62264df6a020eab3f68..48e078547112b5fe4d6b87e608b67c11cc00fd25 100644 --- a/backend/open_webui/utils/middleware.py +++ b/backend/open_webui/utils/middleware.py @@ -144,6 +144,9 @@ from open_webui.utils.task import ( tools_function_calling_generation_template, ) from open_webui.utils.tool_approval import complete_tool_call, pending_tool_calls +from open_webui.utils.telemetry.genai.tools import traced_tool +from open_webui.utils.telemetry.genai.tracer import GenAITracer +from open_webui.utils.tool_approval import complete_tool_call, pending_tool_calls from open_webui.utils.tool_search import apply_tool_search from open_webui.utils.tools import ( connect_mcp_server, @@ -157,6 +160,7 @@ from starlette.responses import JSONResponse, Response, StreamingResponse logging.basicConfig(stream=sys.stdout, level=GLOBAL_LOG_LEVEL) log = logging.getLogger(__name__) +ai_tracer = GenAITracer() def _is_tool_result_error(value: Any) -> bool: @@ -3481,11 +3485,11 @@ async def execute_tool_call(form_data, metadata, event_caller, tool_call): ): return params, 'Error: tool call was not approved.', tool, tool_type, direct_tool - if direct_tool: - if not event_caller: - result = 'Error: Browser session is not connected for this direct tool.' - else: - result = await event_caller( + async def invoke_tool(): + if direct_tool: + if not event_caller: + return 'Error: Browser session is not connected for this direct tool.' + return await event_caller( { 'type': 'execute:tool', 'data': { @@ -3497,7 +3501,6 @@ async def execute_tool_call(form_data, metadata, event_caller, tool_call): }, } ) - else: function = await get_updated_tool_function( function=tool['callable'], extra_params={ @@ -3505,7 +3508,17 @@ async def execute_tool_call(form_data, metadata, event_caller, tool_call): '__files__': metadata.get('files', []), }, ) - result = await function(**params) + return await function(**params) + + result = await traced_tool( + ai_tracer, + name=name, + call_id=tool_call.get('id', ''), + tool_type=tool_type, + arguments=params, + direct=direct_tool, + call=invoke_tool, + ) except Exception as e: result = {'error': str(e)} return params, result, tool, tool_type, direct_tool diff --git a/backend/open_webui/utils/telemetry/genai/README.md b/backend/open_webui/utils/telemetry/genai/README.md new file mode 100644 index 0000000000000000000000000000000000000000..1256af32839fc7c3d4fb68539279dbe484fc6202 --- /dev/null +++ b/backend/open_webui/utils/telemetry/genai/README.md @@ -0,0 +1,99 @@ +# OpenTelemetry GenAI telemetry + +This package emits vendor-neutral OpenTelemetry GenAI spans. Structural +attributes are emitted by default; content capture is opt-in and each content +value is truncated to `OTEL_GENAI_CONTENT_MAX_LENGTH` (default `2000`). + +## Environment variables + +The following `OTEL_GENAI_*` variables are read by the semantic layer. Boolean +values accept `1`, `true`, `yes`, or `on` (case-insensitive). + +| Variable | Default | Captures | +| --- | --- | --- | +| `OTEL_GENAI_ENABLED` | `true` | Enables semantic span creation | +| `OTEL_GENAI_CAPTURE_INPUTS` | `false` | Model input messages | +| `OTEL_GENAI_CAPTURE_OUTPUTS` | `false` | Model output messages | +| `OTEL_GENAI_CAPTURE_SYSTEM_INSTRUCTIONS` | `false` | System instructions | +| `OTEL_GENAI_CAPTURE_TOOL_ARGUMENTS` | `false` | Tool arguments | +| `OTEL_GENAI_CAPTURE_TOOL_RESULTS` | `false` | Tool results | +| `OTEL_GENAI_CAPTURE_RETRIEVAL_QUERY` | `false` | Retrieval queries | +| `OTEL_GENAI_CAPTURE_RETRIEVAL_DOCUMENTS` | `false` | Retrieved document content | +| `OTEL_GENAI_CAPTURE_EMBEDDING_TEXT` | `false` | Text sent to embedding providers | +| `OTEL_GENAI_CAPTURE_EMBEDDING_VECTORS` | `false` | Embedding vectors | +| `OTEL_GENAI_CONTENT_MAX_LENGTH` | `2000` | Maximum characters for captured content | + +The OTLP destination is configured with the standard OpenTelemetry variables, +including `OTEL_EXPORTER_OTLP_ENDPOINT`, `OTEL_EXPORTER_OTLP_HEADERS`, and +`OTEL_EXPORTER_OTLP_INSECURE`. No vendor-specific exporter or attribute is +required. + +## Protected content-capture verification + +Content capture is disabled by default. For a protected local receiver only, +enable the fields needed by the test and keep the length cap in place: + +```bash +OTEL_GENAI_CAPTURE_INPUTS=true \ +OTEL_GENAI_CAPTURE_OUTPUTS=true \ +OTEL_GENAI_CAPTURE_TOOL_ARGUMENTS=true \ +OTEL_GENAI_CAPTURE_TOOL_RESULTS=true \ +OTEL_GENAI_CAPTURE_RETRIEVAL_QUERY=true \ +OTEL_GENAI_CAPTURE_RETRIEVAL_DOCUMENTS=true \ +OTEL_GENAI_CONTENT_MAX_LENGTH=2000 +``` + +Buffered and streamed model outputs are emitted on `gen_ai.output.messages`. +Tool arguments/results, retrieval query/document content, embedding text, and +embedding vectors each remain independently controlled by their corresponding +flag. Do not enable these flags for an untrusted or production telemetry +destination without an explicit data-handling review. + +## M10 interoperability procedure + +Use the M9 `rag_rerank_tool` scenario as the representative trace: a chat +workflow with retrieval, reranking, and tool execution. Run it with the same +Open WebUI build and change only `OTEL_EXPORTER_OTLP_ENDPOINT` between +backends. Keep content capture disabled unless the destination is a protected +local test system. + +1. Start one of the following OTLP-compatible destinations and record its OTLP + HTTP endpoint: + + - Local Phoenix: `http://localhost:6006/v1/traces`. + - Local or hosted Langfuse: use the project's OTLP HTTP endpoint, commonly + `https:///api/public/otel/v1/traces`, together with its documented + `OTEL_EXPORTER_OTLP_HEADERS` authentication value. + - Generic Jaeger or Tempo: use the instance's OTLP HTTP `/v1/traces` + endpoint. + +2. Run Open WebUI with the endpoint override, for example: + + ```bash + OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:6006/v1/traces \ + WEBUI_SECRET_KEY=test-secret-key \ + uv run open-webui serve + ``` + + For gRPC-only receivers, use the receiver's OTLP endpoint and set + `OTEL_EXPORTER_OTLP_PROTOCOL=grpc`; otherwise the HTTP exporter is used. + +3. Exercise the `rag_rerank_tool` flow: submit a chat using a knowledge source, + allow reranking, and invoke a tool. Verify one trace contains the workflow, + inference, retrieval, rerank, and tool spans. Repeat step 2 for Phoenix, + Langfuse, and Jaeger/Tempo by changing only + `OTEL_EXPORTER_OTLP_ENDPOINT` (and, where required, the receiver's standard + OTLP headers). + +### Known differences + +- Backend UIs use different names and nesting displays for the same OTLP + spans. The conformance contract is the span topology and standard semantic + attributes, not a particular vendor's screen layout. +- Phoenix, Langfuse, Jaeger, and Tempo may omit or flatten attributes they do + not index or render. This is a presentation/indexing difference; no + backend-specific code belongs in this repository. +- Langfuse may require authentication headers and may display GenAI spans in + its generation view only after recognizing the standard GenAI attributes. +- Jaeger and Tempo generally show the generic trace tree and attributes but do + not provide vendor-specific GenAI generation summaries. diff --git a/backend/open_webui/utils/telemetry/genai/__init__.py b/backend/open_webui/utils/telemetry/genai/__init__.py new file mode 100644 index 0000000000000000000000000000000000000000..f88b66029173219948478d30058fa87726904c1b --- /dev/null +++ b/backend/open_webui/utils/telemetry/genai/__init__.py @@ -0,0 +1,4 @@ +from open_webui.utils.telemetry.genai.config import GenAIConfig +from open_webui.utils.telemetry.genai.tracer import GenAITracer + +__all__ = ['GenAIConfig', 'GenAITracer'] diff --git a/backend/open_webui/utils/telemetry/genai/config.py b/backend/open_webui/utils/telemetry/genai/config.py new file mode 100644 index 0000000000000000000000000000000000000000..9736c9ef8ef5fed6de84afe50925b2bdda82e6cb --- /dev/null +++ b/backend/open_webui/utils/telemetry/genai/config.py @@ -0,0 +1,43 @@ +import os +from dataclasses import dataclass + + +def _as_bool(value: str | None) -> bool: + return (value or '').strip().lower() in {'1', 'true', 'yes', 'on'} + + +@dataclass(frozen=True) +class GenAIConfig: + enabled: bool = True + capture_inputs: bool = False + capture_outputs: bool = False + capture_system_instructions: bool = False + capture_tool_arguments: bool = False + capture_tool_results: bool = False + capture_retrieval_query: bool = False + capture_retrieval_documents: bool = False + capture_embedding_text: bool = False + capture_embedding_vectors: bool = False + content_max_length: int = 2000 + retrieval_max_documents: int = 100 + exception_max_length: int = 2000 + + @classmethod + def from_env(cls) -> 'GenAIConfig': + try: + max_length = max(int(os.getenv('OTEL_GENAI_CONTENT_MAX_LENGTH', '2000')), 0) + except (TypeError, ValueError): + max_length = 2000 + return cls( + enabled=os.getenv('OTEL_GENAI_ENABLED', 'true').strip().lower() not in {'0', 'false', 'no', 'off'}, + capture_inputs=_as_bool(os.getenv('OTEL_GENAI_CAPTURE_INPUTS')), + capture_outputs=_as_bool(os.getenv('OTEL_GENAI_CAPTURE_OUTPUTS')), + capture_system_instructions=_as_bool(os.getenv('OTEL_GENAI_CAPTURE_SYSTEM_INSTRUCTIONS')), + capture_tool_arguments=_as_bool(os.getenv('OTEL_GENAI_CAPTURE_TOOL_ARGUMENTS')), + capture_tool_results=_as_bool(os.getenv('OTEL_GENAI_CAPTURE_TOOL_RESULTS')), + capture_retrieval_query=_as_bool(os.getenv('OTEL_GENAI_CAPTURE_RETRIEVAL_QUERY')), + capture_retrieval_documents=_as_bool(os.getenv('OTEL_GENAI_CAPTURE_RETRIEVAL_DOCUMENTS')), + capture_embedding_text=_as_bool(os.getenv('OTEL_GENAI_CAPTURE_EMBEDDING_TEXT')), + capture_embedding_vectors=_as_bool(os.getenv('OTEL_GENAI_CAPTURE_EMBEDDING_VECTORS')), + content_max_length=max_length, + ) diff --git a/backend/open_webui/utils/telemetry/genai/context.py b/backend/open_webui/utils/telemetry/genai/context.py new file mode 100644 index 0000000000000000000000000000000000000000..1c38e54850e5a8be02b8b2c14da35fb0cecd10fa --- /dev/null +++ b/backend/open_webui/utils/telemetry/genai/context.py @@ -0,0 +1,36 @@ +import contextvars +import logging + +from open_webui.utils.telemetry.genai import semconv +from opentelemetry.trace import Span + +logger = logging.getLogger(__name__) +_context = contextvars.ContextVar('openwebui_genai_context', default={}) + + +def set_context(*, chat_id=None, message_id=None, conversation_id=None, purpose='primary'): + values = { + 'chat_id': chat_id, + 'message_id': message_id, + 'conversation_id': conversation_id, + 'purpose': purpose, + } + _context.set(values) + return values + + +def get_context(): + return dict(_context.get()) + + +def apply_context(span: Span, values=None): + values = values or get_context() + attributes = { + semconv.GEN_AI_CONVERSATION_ID: values.get('conversation_id') or values.get('chat_id'), + semconv.OPENWEBUI_MESSAGE_ID: values.get('message_id'), + semconv.OPENWEBUI_OPERATION_PURPOSE: values.get('purpose', 'primary'), + } + try: + span.set_attributes({key: value for key, value in attributes.items() if value is not None}) + except Exception: + logger.exception('Unable to apply GenAI span context') diff --git a/backend/open_webui/utils/telemetry/genai/extractors.py b/backend/open_webui/utils/telemetry/genai/extractors.py new file mode 100644 index 0000000000000000000000000000000000000000..331c66c96b528b30331342090209dbf8a715a44c --- /dev/null +++ b/backend/open_webui/utils/telemetry/genai/extractors.py @@ -0,0 +1,38 @@ +import json + +from open_webui.utils.telemetry.genai.config import GenAIConfig + + +def _truncate(value, limit: int): + if value is None: + return None + if not isinstance(value, str): + value = json.dumps(value, ensure_ascii=False, default=str) + return value[:limit] + + +def capture(value, *, enabled: bool, config: GenAIConfig): + if not enabled: + return None + return _truncate(value, config.content_max_length) + + +def capture_attributes(attributes: dict, config: GenAIConfig) -> dict: + flags = { + 'inputs': config.capture_inputs, + 'outputs': config.capture_outputs, + 'system_instructions': config.capture_system_instructions, + 'tool_arguments': config.capture_tool_arguments, + 'tool_result': config.capture_tool_results, + 'retrieval_query': config.capture_retrieval_query, + 'retrieval_documents': config.capture_retrieval_documents, + 'embedding_text': config.capture_embedding_text, + 'embedding_vectors': config.capture_embedding_vectors, + } + captured = {} + for name, value in attributes.items(): + if name in flags: + value = capture(value, enabled=flags[name], config=config) + if value is not None: + captured[name] = value + return captured diff --git a/backend/open_webui/utils/telemetry/genai/inference.py b/backend/open_webui/utils/telemetry/genai/inference.py new file mode 100644 index 0000000000000000000000000000000000000000..51704bfca6dbe3cc0ac9cbdbcd4b382827b4808e --- /dev/null +++ b/backend/open_webui/utils/telemetry/genai/inference.py @@ -0,0 +1,53 @@ +from collections.abc import Awaitable, Callable + +from open_webui.utils.telemetry.genai.purpose import purpose_for_task +from open_webui.utils.telemetry.genai.streaming import wrap_stream + + +async def traced_inference( + tracer, + *, + provider: str, + request_payload: dict, + call: Callable[[], Awaitable], +): + metadata = request_payload.get('metadata') or {} + with tracer.inference( + 'chat_completion', + provider=provider, + request_model=request_payload.get('model'), + inputs=request_payload.get('messages'), + temperature=request_payload.get('temperature'), + top_p=request_payload.get('top_p'), + max_tokens=request_payload.get('max_tokens', request_payload.get('max_completion_tokens')), + seed=request_payload.get('seed'), + purpose=purpose_for_task(metadata.get('task')), + _defer_end=True, + ) as span: + response = await call() + body_iterator = getattr(response, 'body_iterator', None) + if body_iterator is not None: + chunks = [] + captured_length = 0 + + def collect(item): + nonlocal captured_length + if tracer.config.capture_outputs and captured_length < tracer.config.content_max_length: + if isinstance(item, bytes): + item = item.decode('utf-8', errors='replace') + elif not isinstance(item, str): + item = str(item) + remaining = tracer.config.content_max_length - captured_length + chunk = item[:remaining] + chunks.append(chunk) + captured_length += len(chunk) + + response.body_iterator = wrap_stream( + span, + body_iterator, + on_finish=lambda: tracer.finish_stream(span, chunks), + on_item=collect, + ) + else: + tracer.finish_inference(span, response) + return response diff --git a/backend/open_webui/utils/telemetry/genai/openinference.py b/backend/open_webui/utils/telemetry/genai/openinference.py new file mode 100644 index 0000000000000000000000000000000000000000..fdf354d613e23a8228d39fb680539b2a29b25839 --- /dev/null +++ b/backend/open_webui/utils/telemetry/genai/openinference.py @@ -0,0 +1,15 @@ +from open_webui.utils.telemetry.genai import semconv + +_KINDS = { + 'rerank': 'RERANKER', + 'tool': 'TOOL', + 'embedding': 'EMBEDDING', + 'retrieval': 'RETRIEVER', + 'inference': 'LLM', + 'workflow': 'CHAIN', +} + + +def attributes_for(kind: str) -> dict: + value = _KINDS.get(kind) + return {semconv.OPENINFERENCE_SPAN_KIND: value} if value else {} diff --git a/backend/open_webui/utils/telemetry/genai/purpose.py b/backend/open_webui/utils/telemetry/genai/purpose.py new file mode 100644 index 0000000000000000000000000000000000000000..eb20addd9252458501b9b34f678fceda166ed882 --- /dev/null +++ b/backend/open_webui/utils/telemetry/genai/purpose.py @@ -0,0 +1,10 @@ +TASK_PURPOSES = { + 'title_generation': 'title_generation', + 'follow_up_generation': 'followup_generation', + 'tags_generation': 'tags_generation', + 'query_generation': 'retrieval_query', +} + + +def purpose_for_task(task) -> str: + return TASK_PURPOSES.get(str(task), 'primary') diff --git a/backend/open_webui/utils/telemetry/genai/retrieval.py b/backend/open_webui/utils/telemetry/genai/retrieval.py new file mode 100644 index 0000000000000000000000000000000000000000..88620ed6d04db4d7d0d0a40f8dc7638c6f48bdc6 --- /dev/null +++ b/backend/open_webui/utils/telemetry/genai/retrieval.py @@ -0,0 +1,88 @@ +import functools +import inspect +import json + +from open_webui.utils.telemetry.genai import semconv +from open_webui.utils.telemetry.genai.extractors import capture +from open_webui.utils.telemetry.genai.openinference import attributes_for +from open_webui.utils.telemetry.genai.tracer import GenAITracer + + +def instrument_embedding(tracer: GenAITracer, embedding_function): + if embedding_function is None: + return None + + async def wrapped(query, *args, **kwargs): + with tracer.embedding('embedding', embedding_text=query) as span: + result = await embedding_function(query, *args, **kwargs) + if span.is_recording() and tracer.config.capture_embedding_vectors and result is not None: + value = capture(result, enabled=True, config=tracer.config) + if value is not None: + span.set_attribute(semconv.GEN_AI_EMBEDDINGS, value) + return result + + return wrapped + + +def instrument_reranker(tracer: GenAITracer, reranking_function): + if reranking_function is None: + return None + + def wrapped(query, documents, *args, **kwargs): + with tracer.rerank('rerank', retrieval_query=query, **attributes_for('rerank')) as span: + result = reranking_function(query, documents, *args, **kwargs) + if span.is_recording(): + span.set_attribute(semconv.OPENWEBUI_RETRIEVAL_CANDIDATE_COUNT, len(documents)) + return result + + return wrapped + + +def traced_retrieval(tracer: GenAITracer): + def decorator(function): + @functools.wraps(function) + async def wrapped(*args, **kwargs): + bound = inspect.signature(function).bind_partial(*args, **kwargs) + values = bound.arguments + queries = values.get('queries') or [] + query = queries[0] if isinstance(queries, list) and queries else queries + with tracer.retrieval( + 'retrieval', + operation_name='retrieval', + retrieval_query=query, + top_k=values.get('k'), + candidate_count=len(values.get('items') or []), + retrieval_mode='hybrid' if values.get('hybrid_search') else 'vector', + data_source=[ + item.get('collection_name') for item in values.get('items') or [] if item.get('collection_name') + ], + ) as span: + if 'embedding_function' in values: + values['embedding_function'] = instrument_embedding(tracer, values['embedding_function']) + result = await function(*bound.args, **bound.kwargs) + if span.is_recording() and isinstance(result, list): + documents = [] + for rank, source in enumerate(result[: tracer.config.retrieval_max_documents], 1): + metadata = source.get('metadata') or {} + distances = source.get('distances') or [] + entry = { + 'document_id': metadata.get('document_id') or metadata.get('id'), + 'file_id': metadata.get('file_id'), + 'chunk_id': metadata.get('chunk_id'), + 'rank': rank, + 'score': distances[rank - 1] if len(distances) >= rank else metadata.get('score'), + } + if tracer.config.capture_retrieval_documents: + entry['content'] = capture( + (source.get('document') or [''])[0], + enabled=True, + config=tracer.config, + ) + documents.append(json.dumps(entry, ensure_ascii=False, default=str)) + span.set_attribute(semconv.GEN_AI_RETRIEVAL_DOCUMENTS, documents) + span.set_attribute(semconv.OPENWEBUI_RETRIEVAL_SELECTED_COUNT, len(result)) + return result + + return wrapped + + return decorator diff --git a/backend/open_webui/utils/telemetry/genai/semconv.py b/backend/open_webui/utils/telemetry/genai/semconv.py new file mode 100644 index 0000000000000000000000000000000000000000..eecb9c737dab53b1d4673c08b2c5212d6c55c73f --- /dev/null +++ b/backend/open_webui/utils/telemetry/genai/semconv.py @@ -0,0 +1,84 @@ +try: + from opentelemetry.semconv._incubating.attributes import gen_ai_attributes as _gen_ai +except ImportError: # pragma: no cover - supports older OTel installations + _gen_ai = None + + +def _official(name: str, fallback: str) -> str: + return getattr(_gen_ai, name, fallback) if _gen_ai is not None else fallback + + +GEN_AI_OPERATION_NAME = _official('GEN_AI_OPERATION_NAME', 'gen_ai.operation.name') +GEN_AI_PROVIDER_NAME = _official('GEN_AI_PROVIDER_NAME', 'gen_ai.provider.name') +GEN_AI_REQUEST_MODEL = _official('GEN_AI_REQUEST_MODEL', 'gen_ai.request.model') +GEN_AI_RESPONSE_MODEL = _official('GEN_AI_RESPONSE_MODEL', 'gen_ai.response.model') +GEN_AI_RESPONSE_ID = _official('GEN_AI_RESPONSE_ID', 'gen_ai.response.id') +GEN_AI_CONVERSATION_ID = _official('GEN_AI_CONVERSATION_ID', 'gen_ai.conversation.id') +GEN_AI_USAGE_INPUT_TOKENS = _official('GEN_AI_USAGE_INPUT_TOKENS', 'gen_ai.usage.input_tokens') +GEN_AI_USAGE_OUTPUT_TOKENS = _official('GEN_AI_USAGE_OUTPUT_TOKENS', 'gen_ai.usage.output_tokens') +GEN_AI_TOOL_NAME = _official('GEN_AI_TOOL_NAME', 'gen_ai.tool.name') +GEN_AI_TOOL_CALL_ID = _official('GEN_AI_TOOL_CALL_ID', 'gen_ai.tool.call.id') +GEN_AI_TOOL_CALL_ARGUMENTS = _official('GEN_AI_TOOL_CALL_ARGUMENTS', 'gen_ai.tool.call.arguments') +GEN_AI_TOOL_CALL_RESULT = _official('GEN_AI_TOOL_CALL_RESULT', 'gen_ai.tool.call.result') +GEN_AI_TOOL_TYPE = _official('GEN_AI_TOOL_TYPE', 'gen_ai.tool.type') +GEN_AI_RETRIEVAL_QUERY_TEXT = _official('GEN_AI_RETRIEVAL_QUERY_TEXT', 'gen_ai.retrieval.query.text') +GEN_AI_RETRIEVAL_DOCUMENTS = _official('GEN_AI_RETRIEVAL_DOCUMENTS', 'gen_ai.retrieval.documents') +GEN_AI_INPUT_MESSAGES = _official('GEN_AI_INPUT_MESSAGES', 'gen_ai.input.messages') +GEN_AI_OUTPUT_MESSAGES = _official('GEN_AI_OUTPUT_MESSAGES', 'gen_ai.output.messages') +GEN_AI_SYSTEM_INSTRUCTIONS = _official('GEN_AI_SYSTEM_INSTRUCTIONS', 'gen_ai.system.instructions') +GEN_AI_RESPONSE_FINISH_REASONS = _official('GEN_AI_RESPONSE_FINISH_REASONS', 'gen_ai.response.finish_reasons') +GEN_AI_REQUEST_TEMPERATURE = _official('GEN_AI_REQUEST_TEMPERATURE', 'gen_ai.request.temperature') +GEN_AI_REQUEST_TOP_P = _official('GEN_AI_REQUEST_TOP_P', 'gen_ai.request.top_p') +GEN_AI_REQUEST_MAX_TOKENS = _official('GEN_AI_REQUEST_MAX_TOKENS', 'gen_ai.request.max_tokens') +GEN_AI_REQUEST_SEED = _official('GEN_AI_REQUEST_SEED', 'gen_ai.request.seed') +GEN_AI_EMBEDDINGS = _official('GEN_AI_EMBEDDINGS', 'gen_ai.embeddings') + +OPENWEBUI_MESSAGE_ID = 'openwebui.message.id' +OPENWEBUI_OPERATION_PURPOSE = 'openwebui.operation.purpose' +OPENWEBUI_TOOL_TYPE = 'openwebui.tool.type' +OPENWEBUI_RETRIEVAL_MODE = 'openwebui.retrieval.mode' +OPENWEBUI_RETRIEVAL_TOP_K = 'openwebui.retrieval.top_k' +OPENWEBUI_RETRIEVAL_THRESHOLD = 'openwebui.retrieval.threshold' +OPENWEBUI_RETRIEVAL_CANDIDATE_COUNT = 'openwebui.retrieval.candidate_count' +OPENWEBUI_RETRIEVAL_SELECTED_COUNT = 'openwebui.retrieval.selected_count' +OPENWEBUI_RETRIEVAL_DATA_SOURCE = 'openwebui.retrieval.data_source' +OPENINFERENCE_SPAN_KIND = 'openinference.span.kind' + +ALIASES = { + 'operation_name': GEN_AI_OPERATION_NAME, + 'provider': GEN_AI_PROVIDER_NAME, + 'request_model': GEN_AI_REQUEST_MODEL, + 'response_model': GEN_AI_RESPONSE_MODEL, + 'response_id': GEN_AI_RESPONSE_ID, + 'conversation_id': GEN_AI_CONVERSATION_ID, + 'input_tokens': GEN_AI_USAGE_INPUT_TOKENS, + 'output_tokens': GEN_AI_USAGE_OUTPUT_TOKENS, + 'tool_name': GEN_AI_TOOL_NAME, + 'tool_call_id': GEN_AI_TOOL_CALL_ID, + 'tool_arguments': GEN_AI_TOOL_CALL_ARGUMENTS, + 'tool_result': GEN_AI_TOOL_CALL_RESULT, + 'retrieval_query': GEN_AI_RETRIEVAL_QUERY_TEXT, + 'retrieval_documents': GEN_AI_RETRIEVAL_DOCUMENTS, + 'inputs': GEN_AI_INPUT_MESSAGES, + 'outputs': GEN_AI_OUTPUT_MESSAGES, + 'system_instructions': GEN_AI_SYSTEM_INSTRUCTIONS, + 'finish_reasons': GEN_AI_RESPONSE_FINISH_REASONS, + 'temperature': GEN_AI_REQUEST_TEMPERATURE, + 'top_p': GEN_AI_REQUEST_TOP_P, + 'max_tokens': GEN_AI_REQUEST_MAX_TOKENS, + 'seed': GEN_AI_REQUEST_SEED, + 'embedding_vectors': GEN_AI_EMBEDDINGS, + 'message_id': OPENWEBUI_MESSAGE_ID, + 'purpose': OPENWEBUI_OPERATION_PURPOSE, + 'tool_type': GEN_AI_TOOL_TYPE, + 'retrieval_mode': OPENWEBUI_RETRIEVAL_MODE, + 'top_k': OPENWEBUI_RETRIEVAL_TOP_K, + 'threshold': OPENWEBUI_RETRIEVAL_THRESHOLD, + 'candidate_count': OPENWEBUI_RETRIEVAL_CANDIDATE_COUNT, + 'selected_count': OPENWEBUI_RETRIEVAL_SELECTED_COUNT, + 'data_source': OPENWEBUI_RETRIEVAL_DATA_SOURCE, +} + + +def attribute_key(name: str) -> str: + return ALIASES.get(name, name) diff --git a/backend/open_webui/utils/telemetry/genai/streaming.py b/backend/open_webui/utils/telemetry/genai/streaming.py new file mode 100644 index 0000000000000000000000000000000000000000..aa0d3bfbaa772333d4932e51a76681806ae90bb0 --- /dev/null +++ b/backend/open_webui/utils/telemetry/genai/streaming.py @@ -0,0 +1,46 @@ +import asyncio +import logging + +from open_webui.utils.telemetry.genai.config import GenAIConfig +from opentelemetry.trace import StatusCode + +logger = logging.getLogger(__name__) + + +async def wrap_stream(span, async_iterator, *, on_item=None, on_finish=None): # noqa: C901 + ended = False + try: + async for item in async_iterator: + if on_item is not None: + try: + on_item(item) + except Exception: + logger.exception('Unable to capture GenAI stream item') + yield item + span.set_status(StatusCode.OK) + except asyncio.CancelledError: + span.set_status(StatusCode.ERROR) + raise + except BaseException as exc: + if span.is_recording(): + span.add_event( + 'exception', + { + 'exception.type': type(exc).__name__, + 'exception.message': str(exc)[: GenAIConfig().exception_max_length], + }, + ) + span.set_status(StatusCode.ERROR) + raise + finally: + if on_finish is not None: + try: + on_finish() + except Exception: + logger.exception('Unable to capture GenAI stream output') + if not ended: + ended = True + try: + span.end() + except Exception: + logger.exception('Unable to end GenAI streaming span') diff --git a/backend/open_webui/utils/telemetry/genai/tools.py b/backend/open_webui/utils/telemetry/genai/tools.py new file mode 100644 index 0000000000000000000000000000000000000000..62572567c876585a95e359bd2df73dbb4e2c3044 --- /dev/null +++ b/backend/open_webui/utils/telemetry/genai/tools.py @@ -0,0 +1,39 @@ +from collections.abc import Awaitable, Callable + +from open_webui.utils.telemetry.genai import semconv +from open_webui.utils.telemetry.genai.tracer import GenAITracer + + +def classify_tool(tool_type: str | None, direct: bool = False) -> str: + normalized = (tool_type or '').lower() + if normalized in {'mcp', 'openapi', 'external', 'action', 'terminal', 'builtin', 'function'}: + return normalized + return 'builtin' if direct else 'function' + + +async def traced_tool( + tracer: GenAITracer, + *, + name: str, + call_id: str | None, + tool_type: str | None, + arguments, + direct: bool, + call: Callable[[], Awaitable], +): + with tracer.tool( + 'execute_tool', + operation_name='execute_tool', + tool_name=name, + tool_call_id=call_id, + tool_type=classify_tool(tool_type, direct), + tool_arguments=arguments, + ) as span: + result = await call() + if span.is_recording() and tracer.config.capture_tool_results and result is not None: + from open_webui.utils.telemetry.genai.extractors import capture + + value = capture(result, enabled=True, config=tracer.config) + if value is not None: + span.set_attribute(semconv.GEN_AI_TOOL_CALL_RESULT, value) + return result diff --git a/backend/open_webui/utils/telemetry/genai/tracer.py b/backend/open_webui/utils/telemetry/genai/tracer.py new file mode 100644 index 0000000000000000000000000000000000000000..7dd74a02da250d6e51ddf4d27679d1e0a7ea57f4 --- /dev/null +++ b/backend/open_webui/utils/telemetry/genai/tracer.py @@ -0,0 +1,156 @@ +import logging +from contextlib import contextmanager + +from open_webui.utils.telemetry.genai import context, extractors, semconv +from open_webui.utils.telemetry.genai import openinference as oi +from open_webui.utils.telemetry.genai.config import GenAIConfig +from opentelemetry import trace +from opentelemetry.trace import StatusCode + +logger = logging.getLogger(__name__) + + +def _record_exception(span, exc, limit): + if not span.is_recording(): + return + span.add_event( + 'exception', + { + 'exception.type': type(exc).__name__, + 'exception.message': str(exc)[:limit], + }, + ) + + +class GenAITracer: + def __init__(self, tracer=None, config: GenAIConfig | None = None): + self.tracer = tracer or trace.get_tracer(__name__) + self.config = config or GenAIConfig.from_env() + + def workflow(self, name='invoke_workflow', **attrs): + return self._span('workflow', name, **attrs) + + def inference(self, name='inference', **attrs): + return self._span('inference', name, **attrs) + + def embedding(self, name='embedding', **attrs): + return self._span('embedding', name, **attrs) + + def retrieval(self, name='retrieval', **attrs): + return self._span('retrieval', name, **attrs) + + def rerank(self, name='rerank', **attrs): + return self._span('rerank', name, **attrs) + + def tool(self, name='execute_tool', **attrs): + return self._span('tool', name, **attrs) + + def finish_inference(self, span, response): + try: + if span.is_recording() and isinstance(response, dict): + usage = response.get('usage') or {} + attributes = { + semconv.GEN_AI_RESPONSE_MODEL: response.get('model'), + semconv.GEN_AI_RESPONSE_ID: response.get('id'), + semconv.GEN_AI_USAGE_INPUT_TOKENS: usage.get('prompt_tokens', usage.get('input_tokens')), + semconv.GEN_AI_USAGE_OUTPUT_TOKENS: usage.get('completion_tokens', usage.get('output_tokens')), + semconv.GEN_AI_RESPONSE_FINISH_REASONS: [ + choice.get('finish_reason') + for choice in response.get('choices', []) + if isinstance(choice, dict) and choice.get('finish_reason') is not None + ], + } + span.set_attributes({key: value for key, value in attributes.items() if value is not None}) + if self.config.capture_outputs: + output = _response_output(response) + captured = extractors.capture(output, enabled=True, config=self.config) + if captured is not None: + span.set_attribute(semconv.GEN_AI_OUTPUT_MESSAGES, captured) + span.end() + except Exception: + logger.exception('Unable to finish GenAI inference span') + + def finish_stream(self, span, chunks): + try: + if span.is_recording() and self.config.capture_outputs and chunks: + captured = extractors.capture(''.join(chunks), enabled=True, config=self.config) + if captured is not None: + span.set_attribute(semconv.GEN_AI_OUTPUT_MESSAGES, captured) + except Exception: + logger.exception('Unable to finish GenAI streaming span') + + @contextmanager + # The lifecycle guard intentionally handles independent failure points. + def _span(self, kind, name, **attrs): # noqa: C901 + if not self.config.enabled: + yield trace.INVALID_SPAN + return + defer_end = attrs.pop('_defer_end', False) + span = None + token = None + failed = False + try: + try: + span = self.tracer.start_span(name) + except Exception: + logger.exception('Unable to start GenAI span') + yield trace.INVALID_SPAN + return + + try: + token = trace.use_span(span, end_on_exit=False) + token.__enter__() + if span.is_recording(): + safe_attrs = extractors.capture_attributes(attrs, self.config) + resolved = { + semconv.attribute_key(key): value for key, value in safe_attrs.items() if value is not None + } + resolved.update(oi.attributes_for(kind)) + if resolved: + span.set_attributes(resolved) + if kind == 'workflow': + context.apply_context(span, attrs) + context.set_context( + chat_id=attrs.get('chat_id'), + message_id=attrs.get('message_id'), + conversation_id=attrs.get('conversation_id'), + purpose=attrs.get('purpose', 'primary'), + ) + except Exception: + logger.exception('Unable to initialize GenAI span') + + try: + yield span + except BaseException as exc: + failed = True + try: + _record_exception(span, exc, self.config.exception_max_length) + span.set_status(StatusCode.ERROR) + except Exception: + logger.exception('Unable to record GenAI span exception') + raise + finally: + if token is not None: + try: + token.__exit__(None, None, None) + except Exception: + logger.exception('Unable to detach GenAI span context') + if span is not None and (not defer_end or failed): + try: + span.end() + except Exception: + logger.exception('Unable to end GenAI span') + + +def _response_output(response): + choices = response.get('choices') or [] + messages = [] + for choice in choices: + if not isinstance(choice, dict): + continue + message = choice.get('message') or choice.get('text') + if message is not None: + messages.append(message) + if messages: + return messages + return response.get('output') or response.get('response') diff --git a/backend/open_webui/utils/telemetry/setup.py b/backend/open_webui/utils/telemetry/setup.py index b280d4519afb464504e5dc1d7ab56a740beeabdd..c2a3b6528a031a0aab526bd3ca47ebaed27f3383 100644 --- a/backend/open_webui/utils/telemetry/setup.py +++ b/backend/open_webui/utils/telemetry/setup.py @@ -31,7 +31,7 @@ def setup(app: FastAPI, db_engine: Engine): trace.set_tracer_provider(TracerProvider(resource=resource)) # Add basic auth header only if both username and password are not empty - headers = [] + headers = None if OTEL_BASIC_AUTH_USERNAME and OTEL_BASIC_AUTH_PASSWORD: auth_string = f'{OTEL_BASIC_AUTH_USERNAME}:{OTEL_BASIC_AUTH_PASSWORD}' auth_header = b64encode(auth_string.encode()).decode() diff --git a/backend/tests/test_asgi_middleware.py b/backend/tests/test_asgi_middleware.py new file mode 100644 index 0000000000000000000000000000000000000000..233bcb3e4841abe5594b40b16a592c638954f8cc --- /dev/null +++ b/backend/tests/test_asgi_middleware.py @@ -0,0 +1,34 @@ +from starlette.applications import Starlette +from starlette.responses import PlainTextResponse +from starlette.routing import Route +from starlette.testclient import TestClient + +from open_webui.utils.asgi_middleware import AppHTTPMiddleware + + +async def ok(_request): + return PlainTextResponse('ok') + + +def build_client(*, root_path: str = '') -> TestClient: + app = Starlette(routes=[Route('/', ok), Route('/watch', ok)]) + app.add_middleware(AppHTTPMiddleware) + return TestClient(app, root_path=root_path) + + +def test_redirect_middleware_keeps_root_path_for_watch_redirect(): + client = build_client(root_path='/openwebui') + + response = client.get('/watch?v=abc123', follow_redirects=False) + + assert response.status_code in (301, 302, 307, 308) + assert response.headers['location'] == '/openwebui/?youtube=abc123' + + +def test_redirect_middleware_uses_root_when_no_root_path_is_set(): + client = build_client() + + response = client.get('/watch?v=abc123', follow_redirects=False) + + assert response.status_code in (301, 302, 307, 308) + assert response.headers['location'] == '/?youtube=abc123' diff --git a/backend/tests/test_env.py b/backend/tests/test_env.py new file mode 100644 index 0000000000000000000000000000000000000000..0920a357e1d28e4e36dbfa531e890f8cb3490d08 --- /dev/null +++ b/backend/tests/test_env.py @@ -0,0 +1,11 @@ +import importlib + + +def test_env_exports_webui_subpath(monkeypatch): + monkeypatch.setenv('WEBUI_SECRET_KEY', 'test-secret') + monkeypatch.setenv('WEBUI_SUBPATH', '/openwebui') + + env_module = importlib.import_module('open_webui.env') + env_module = importlib.reload(env_module) + + assert env_module.WEBUI_SUBPATH == '/openwebui' diff --git a/pyproject.toml b/pyproject.toml index 202bf0a35266a98dbdb206a00d3dcf6ab683b074..c82c96cde5fa37614b8e144d3c815706c2ad5962 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -141,6 +141,20 @@ mariadb = [ unstructured = [ "unstructured==0.22.31", ] +observability = [ + "opentelemetry-api==1.42.1", + "opentelemetry-sdk==1.42.1", + "opentelemetry-exporter-otlp==1.42.1", + "opentelemetry-instrumentation==0.63b1", + "opentelemetry-instrumentation-fastapi==0.63b1", + "opentelemetry-instrumentation-sqlalchemy==0.63b1", + "opentelemetry-instrumentation-redis==0.63b1", + "opentelemetry-instrumentation-requests==0.63b1", + "opentelemetry-instrumentation-logging==0.63b1", + "opentelemetry-instrumentation-httpx==0.63b1", + "opentelemetry-instrumentation-aiohttp-client==0.63b1", + "opentelemetry-instrumentation-system-metrics==0.63b1", +] all = [ "psycopg2-binary==2.9.12", diff --git a/src/app.html b/src/app.html index 667808e01140741d52647707a55ef5dd803075e6..b711d8aa89b11331973d3ecf0b60ed702ae424e2 100644 --- a/src/app.html +++ b/src/app.html @@ -6,37 +6,51 @@ visual, textual, symbolic identifiers, metadata, and surrounding UI. Do not alter, remove, obscure, or replace it except as LICENSE permits: https://docs.openwebui.com/license. --> - + + - - + + - - + + + +
+
+ {$i18n.t('Model Usage')} + {totalMessages} {$i18n.t('messages')} +
+
+ +
+ {#if loading} +
+ +
+ {/if} + + {#if !modelStats.length && !loading} +
{$i18n.t('No data found')}
+ {:else if modelStats.length} + + + + + + + + + + + {#each sortedModels as model, idx (model.model_id)} + + + + + + + {/each} + +
# toggleSort('name')} + > +
+ {$i18n.t('Model')} + {#if orderBy === 'name'} + {#if direction === 'asc'}{:else}{/if} + {:else} + + {/if} +
+ 		toggleSort('count')} + > +
+ {$i18n.t('Messages')} + {#if orderBy === 'count'} + {#if direction === 'asc'}{:else}{/if} + {:else} + + {/if} +
+ 		toggleSort('percentage')} + > +
+ {$i18n.t('Share')} + {#if orderBy === 'percentage'} + {#if direction === 'asc'}{:else}{/if} + {:else} + + {/if} +
+
+ {idx + 1} + +
+ {model.name} { + e.target.src = `${base}/static/favicon.png`; + }} + /> + {model.name} +
+ 		+ {model.count.toLocaleString()} + + {((model.count / totalMessages) * 100).toFixed(1)}% +
+ {/if} +
+ +
+
+ ⓘ {$i18n.t('Message counts are based on assistant responses.')} +
+
diff --git a/src/lib/components/admin/Evaluations.svelte b/src/lib/components/admin/Evaluations.svelte index 5a7bb7965eca4aed50835cdb62dd122e52cc5bdc..d2112a708da149dba153b3078907483e183cf995 100644 --- a/src/lib/components/admin/Evaluations.svelte +++ b/src/lib/components/admin/Evaluations.svelte @@ -1,5 +1,7 @@ diff --git a/src/lib/components/channel/Thread.svelte b/src/lib/components/channel/Thread.svelte index b5516ae67b486c564bee9ec3f067ee171f44ba73..75300f186a5562feef903d6de358296d848feb70 100644 --- a/src/lib/components/channel/Thread.svelte +++ b/src/lib/components/channel/Thread.svelte @@ -1,4 +1,5 @@ diff --git a/src/routes/(app)/admin/analytics/+page.svelte b/src/routes/(app)/admin/analytics/+page.svelte index 302f4c4261fa08ed53aafd197705736f81fb1492..2bda342d26c344865e5601d04c2e08229bd766c8 100644 --- a/src/routes/(app)/admin/analytics/+page.svelte +++ b/src/routes/(app)/admin/analytics/+page.svelte @@ -1,13 +1,14 @@ diff --git a/src/routes/(app)/admin/analytics/[tab]/+page.svelte b/src/routes/(app)/admin/analytics/[tab]/+page.svelte index 302f4c4261fa08ed53aafd197705736f81fb1492..2bda342d26c344865e5601d04c2e08229bd766c8 100644 --- a/src/routes/(app)/admin/analytics/[tab]/+page.svelte +++ b/src/routes/(app)/admin/analytics/[tab]/+page.svelte @@ -1,13 +1,14 @@ diff --git a/src/routes/(app)/admin/evaluations/+page.svelte b/src/routes/(app)/admin/evaluations/+page.svelte index 8913071b7f2019638c89e9bc6fdea55c511fa80c..bd205f5e13bc097ef544cdd1d4b54932b184607c 100644 --- a/src/routes/(app)/admin/evaluations/+page.svelte +++ b/src/routes/(app)/admin/evaluations/+page.svelte @@ -1,8 +1,9 @@ diff --git a/src/routes/(app)/admin/functions/create/+page.svelte b/src/routes/(app)/admin/functions/create/+page.svelte index 6284948bc416af582d6eadcc8002f33bc68c0387..06a2531e034dc4eae05587437bf897570728b357 100644 --- a/src/routes/(app)/admin/functions/create/+page.svelte +++ b/src/routes/(app)/admin/functions/create/+page.svelte @@ -1,4 +1,5 @@ diff --git a/src/routes/(app)/admin/users/+page.svelte b/src/routes/(app)/admin/users/+page.svelte index ebf29a7ab5caf158bcc0fb867b712440e5812e6b..bc56c4f47ea86aa5d701f606ede6963c7a41d535 100644 --- a/src/routes/(app)/admin/users/+page.svelte +++ b/src/routes/(app)/admin/users/+page.svelte @@ -1,8 +1,9 @@ diff --git a/src/routes/(app)/automations/+page.svelte b/src/routes/(app)/automations/+page.svelte index 4e8c747c0be57e050e30bfa132aed0990a00cc87..26b8651c1e338592ca46df9e2390c77a3f4b187b 100644 --- a/src/routes/(app)/automations/+page.svelte +++ b/src/routes/(app)/automations/+page.svelte @@ -1,4 +1,5 @@ diff --git a/src/routes/(app)/calendar/+page.svelte b/src/routes/(app)/calendar/+page.svelte index 5e8da41b1ad4e9c45410d6b659349be59658503a..2da2df9c668fd3d9d219ce27518711d063ddd9e9 100644 --- a/src/routes/(app)/calendar/+page.svelte +++ b/src/routes/(app)/calendar/+page.svelte @@ -2,6 +2,7 @@ import { onMount, getContext, tick } from 'svelte'; import { toast } from 'svelte-sonner'; import { goto } from '$app/navigation'; + import { base } from '$app/paths'; import { WEBUI_NAME, mobile, showSidebar, user } from '$lib/stores'; import { getCalendars, @@ -139,9 +140,9 @@ const evt = e.detail; if (evt.meta?.automation_id) { if (evt.meta?.chat_id) { - goto(`/c/${evt.meta.chat_id}`); + goto(`${base}/c/${evt.meta.chat_id}`); } else { - goto(`/automations/${evt.meta.automation_id}`); + goto(`${base}/automations/${evt.meta.automation_id}`); } return; } diff --git a/src/routes/(app)/home/+layout.svelte b/src/routes/(app)/home/+layout.svelte index 311a54900567aafe20f9c7dd2f4fc54a94fe83d0..6792dda19890a774b3bf004f775876164959474f 100644 --- a/src/routes/(app)/home/+layout.svelte +++ b/src/routes/(app)/home/+layout.svelte @@ -1,4 +1,5 @@ diff --git a/src/routes/(app)/workspace/knowledge/create/+page.svelte b/src/routes/(app)/workspace/knowledge/create/+page.svelte index b5e1b33ef0b262586d81c5602e8efc0c3a32129d..d93a9b8c5d014e0fb6fdf6e1aa665f39b3f1e08e 100644 --- a/src/routes/(app)/workspace/knowledge/create/+page.svelte +++ b/src/routes/(app)/workspace/knowledge/create/+page.svelte @@ -1,5 +1,6 @@ - + diff --git a/src/routes/(app)/workspace/models/create/+page.svelte b/src/routes/(app)/workspace/models/create/+page.svelte index 419007e361b6a168ded58bd791416ab5eeeb1caa..890cb41d936477f38c03a80e37702246381e842a 100644 --- a/src/routes/(app)/workspace/models/create/+page.svelte +++ b/src/routes/(app)/workspace/models/create/+page.svelte @@ -1,4 +1,5 @@ diff --git a/src/routes/(app)/workspace/prompts/create/+page.svelte b/src/routes/(app)/workspace/prompts/create/+page.svelte index c6c80aed76be790567e9e44659f286a9396d8f55..9a6c400baeae82ff6d08387a3343aff534017271 100644 --- a/src/routes/(app)/workspace/prompts/create/+page.svelte +++ b/src/routes/(app)/workspace/prompts/create/+page.svelte @@ -1,5 +1,6 @@ - + diff --git a/src/routes/(app)/workspace/skills/create/+page.svelte b/src/routes/(app)/workspace/skills/create/+page.svelte index 72d4e591e2c363ad8533de93de294f1d8d67dc1c..f9a91916eeb513ab9e6d4ff94ade813963ad132b 100644 --- a/src/routes/(app)/workspace/skills/create/+page.svelte +++ b/src/routes/(app)/workspace/skills/create/+page.svelte @@ -1,4 +1,5 @@ diff --git a/src/routes/(app)/workspace/tools/create/+page.svelte b/src/routes/(app)/workspace/tools/create/+page.svelte index a6f3e0344c0942025e83ea26e15b04a3e91e2e5c..ab4ab87a44e982386e9a5c406f7f0020eec365c9 100644 --- a/src/routes/(app)/workspace/tools/create/+page.svelte +++ b/src/routes/(app)/workspace/tools/create/+page.svelte @@ -1,4 +1,5 @@ diff --git a/svelte.config.js b/svelte.config.js index 80e37b4f59de094dbae5ca155c2622cecf078b45..5a1d4cccca45de2518b480f2eeef67d9c4363adc 100644 --- a/svelte.config.js +++ b/svelte.config.js @@ -3,6 +3,11 @@ import { resolveBuildInfo } from './scripts/build-info.js'; import { vitePreprocess } from '@sveltejs/vite-plugin-svelte'; import fs from 'node:fs'; +// Optional URL subpath prefix for reverse-proxy deployments (e.g. "/openwebui"). +// Must start with a leading slash and must not end with a trailing slash. +// Leave empty to serve at the root path. +const WEBUI_BASE_PATH = process.env.WEBUI_SUBPATH || ''; + /** @type {import('@sveltejs/kit').Config} */ const config = { // Consult https://kit.svelte.dev/docs/integrations#preprocessors @@ -17,6 +22,16 @@ const config = { assets: 'build', fallback: 'index.html' }), + paths: { + base: WEBUI_BASE_PATH, + // Bake absolute, base-prefixed asset URLs instead of runtime-relative ones. + // With the default (relative: true) SvelteKit derives `base` from the + // document URL and emits relative `%sveltekit.assets%` paths, so static + // refs in app.html (e.g. /static/favicon.png) and nested-route asset loads + // drop the subpath. A fixed reverse-proxy subpath is always served from + // `base`, so absolute paths are correct and route-independent. + relative: false + }, // poll for new version name every 60 seconds (to trigger reload mechanic in +layout.svelte) version: { name: (() => { diff --git a/vite.config.ts b/vite.config.ts index 3780dbe6a9b0561b3e8b8a1bf30ce70c0183d9f6..6fe77cf21da6ecf8acbc7d3d46944f27c35f6d8d 100644 --- a/vite.config.ts +++ b/vite.config.ts @@ -7,6 +7,11 @@ import { viteStaticCopy } from 'vite-plugin-static-copy'; const backendTarget = process.env.WEBUI_BACKEND_URL || 'http://localhost:8080'; const buildInfo = resolveBuildInfo(); +// Optional URL subpath prefix for reverse-proxy deployments (e.g. "/openwebui"). +// Must start with a leading slash and must not end with a trailing slash. +// Leave empty to serve at the root path. +const WEBUI_BASE_PATH = process.env.WEBUI_SUBPATH || ''; + export default defineConfig({ resolve: { conditions: ['onnxruntime-web-use-extern-wasm'] @@ -28,6 +33,7 @@ export default defineConfig({ APP_BUILD_HASH: JSON.stringify(buildInfo.hash), APP_BUILD_CHANNEL: JSON.stringify(buildInfo.channel) }, + base: WEBUI_BASE_PATH || '/', build: { sourcemap: true },