test: add Open WebUI docker smoke

2026-03-27 01:11:16 +07:00 · 2026-03-25 05:26:38 -07:00
parent 9e95125f06
commit 39ad51426c
6 changed files with 327 additions and 2 deletions
--- a/docs/gateway/openai-http-api.md
+++ b/docs/gateway/openai-http-api.md
@@ -165,6 +165,30 @@ Set `stream: true` to receive Server-Sent Events (SSE):
 - Each event line is `data: <json>`
 - Stream ends with `data: [DONE]`

+## Open WebUI quick setup
+
+For a basic Open WebUI connection:
+
+- Base URL: `http://127.0.0.1:18789/v1`
+- Docker on macOS base URL: `http://host.docker.internal:18789/v1`
+- API key: your Gateway bearer token
+- Model: `openclaw/default`
+
+Expected behavior:
+
+- `GET /v1/models` should list `openclaw/default`
+- Open WebUI should use `openclaw/default` as the chat model id
+- If you want a specific backend provider/model for that agent, set the agent's normal default model or send `x-openclaw-model`
+
+Quick smoke:
+
+```bash
+curl -sS http://127.0.0.1:18789/v1/models \
+  -H 'Authorization: Bearer YOUR_TOKEN'
+```
+
+If that returns `openclaw/default`, most Open WebUI setups can connect with the same base URL and token.
+
 ## Examples

 Non-streaming:
--- a/docs/help/testing.md
+++ b/docs/help/testing.md
@@ -424,10 +424,16 @@ If you want to rely on env keys (e.g. exported in your `~/.profile`), run local

 ## Docker runners (optional "works in Linux" checks)

-These run `pnpm test:live` inside the repo Docker image, mounting your local config dir and workspace (and sourcing `~/.profile` if mounted). They also bind-mount only the needed CLI auth homes (or all supported ones when the run is not narrowed), then copy them into the container home before the run so external-CLI OAuth can refresh tokens without mutating the host auth store:
+These Docker runners split into two buckets:
+
+- Live-model runners: `test:docker:live-models` and `test:docker:live-gateway` run `pnpm test:live` inside the repo Docker image, mounting your local config dir and workspace (and sourcing `~/.profile` if mounted).
+- Container smoke runners: `test:docker:openwebui`, `test:docker:onboard`, `test:docker:gateway-network`, and `test:docker:plugins` boot one or more real containers and verify higher-level integration paths.
+
+The live-model Docker runners also bind-mount only the needed CLI auth homes (or all supported ones when the run is not narrowed), then copy them into the container home before the run so external-CLI OAuth can refresh tokens without mutating the host auth store:

 - Direct models: `pnpm test:docker:live-models` (script: `scripts/test-live-models-docker.sh`)
 - Gateway + dev agent: `pnpm test:docker:live-gateway` (script: `scripts/test-live-gateway-models-docker.sh`)
+- Open WebUI live smoke: `pnpm test:docker:openwebui` (script: `scripts/e2e/openwebui-docker.sh`)
 - Onboarding wizard (TTY, full scaffolding): `pnpm test:docker:onboard` (script: `scripts/e2e/onboard-docker.sh`)
 - Gateway networking (two containers, WS auth + health): `pnpm test:docker:gateway-network` (script: `scripts/e2e/gateway-network-docker.sh`)
 - Plugins (install smoke + `/plugin` alias + Claude-bundle restart semantics): `pnpm test:docker:plugins` (script: `scripts/e2e/plugins-docker.sh`)
@@ -440,6 +446,17 @@ real Telegram/Discord/etc. channel workers inside the container.
 `test:docker:live-models` still runs `pnpm test:live`, so pass through
 `OPENCLAW_LIVE_GATEWAY_*` as well when you need to narrow or exclude gateway
 live coverage from that Docker lane.
+`test:docker:openwebui` is a higher-level compatibility smoke: it starts an
+OpenClaw gateway container with the OpenAI-compatible HTTP endpoints enabled,
+starts a pinned Open WebUI container against that gateway, signs in through
+Open WebUI, verifies `/api/models` exposes `openclaw/default`, then sends a
+real chat request through Open WebUI's `/api/chat/completions` proxy.
+The first run can be noticeably slower because Docker may need to pull the
+Open WebUI image and Open WebUI may need to finish its own cold-start setup.
+This lane expects a usable live model key, and `OPENCLAW_PROFILE_FILE`
+(`~/.profile` by default) is the primary way to provide it in Dockerized runs.
+Successful runs print a small JSON payload like `{ "ok": true, "model":
+"openclaw/default", ... }`.

 Manual ACP plain-language thread smoke (not CI):

@@ -458,6 +475,9 @@ Useful env vars:
 - `OPENCLAW_LIVE_GATEWAY_MODELS=...` / `OPENCLAW_LIVE_MODELS=...` to narrow the run
 - `OPENCLAW_LIVE_GATEWAY_PROVIDERS=...` / `OPENCLAW_LIVE_PROVIDERS=...` to filter providers in-container
 - `OPENCLAW_LIVE_REQUIRE_PROFILE_KEYS=1` to ensure creds come from the profile store (not env)
+- `OPENCLAW_OPENWEBUI_MODEL=...` to choose the model exposed by the gateway for the Open WebUI smoke
+- `OPENCLAW_OPENWEBUI_PROMPT=...` to override the nonce-check prompt used by the Open WebUI smoke
+- `OPENWEBUI_IMAGE=...` to override the pinned Open WebUI image tag

 ## Docs sanity

--- a/docs/reference/test.md
+++ b/docs/reference/test.md
@@ -26,6 +26,7 @@ title: "Tests"
 - Gateway integration: opt-in via `OPENCLAW_TEST_INCLUDE_GATEWAY=1 pnpm test` or `pnpm test:gateway`.
 - `pnpm test:e2e`: Runs gateway end-to-end smoke tests (multi-instance WS/HTTP/node pairing). Defaults to `forks` + adaptive workers in `vitest.e2e.config.ts`; tune with `OPENCLAW_E2E_WORKERS=<n>` and set `OPENCLAW_E2E_VERBOSE=1` for verbose logs.
 - `pnpm test:live`: Runs provider live tests (minimax/zai). Requires API keys and `LIVE=1` (or provider-specific `*_LIVE_TEST=1`) to unskip.
+- `pnpm test:docker:openwebui`: Starts Dockerized OpenClaw + Open WebUI, signs in through Open WebUI, checks `/api/models`, then runs a real proxied chat through `/api/chat/completions`. Requires a usable live model key (for example OpenAI in `~/.profile`), pulls an external Open WebUI image, and is not expected to be CI-stable like the normal unit/e2e suites.

 ## Local PR gate

--- a/package.json
+++ b/package.json
@@ -704,13 +704,14 @@
    "test:contracts:plugins": "OPENCLAW_TEST_PROFILE=low pnpm test -- src/plugins/contracts",
    "test:coverage": "vitest run --config vitest.unit.config.ts --coverage",
    "test:coverage:changed": "vitest run --config vitest.unit.config.ts --coverage --changed origin/main",
-    "test:docker:all": "pnpm test:docker:live-models && pnpm test:docker:live-gateway && pnpm test:docker:onboard && pnpm test:docker:gateway-network && pnpm test:docker:qr && pnpm test:docker:doctor-switch && pnpm test:docker:plugins && pnpm test:docker:cleanup",
+    "test:docker:all": "pnpm test:docker:live-models && pnpm test:docker:live-gateway && pnpm test:docker:openwebui && pnpm test:docker:onboard && pnpm test:docker:gateway-network && pnpm test:docker:qr && pnpm test:docker:doctor-switch && pnpm test:docker:plugins && pnpm test:docker:cleanup",
    "test:docker:cleanup": "bash scripts/test-cleanup-docker.sh",
    "test:docker:doctor-switch": "bash scripts/e2e/doctor-install-switch-docker.sh",
    "test:docker:gateway-network": "bash scripts/e2e/gateway-network-docker.sh",
    "test:docker:live-gateway": "bash scripts/test-live-gateway-models-docker.sh",
    "test:docker:live-models": "bash scripts/test-live-models-docker.sh",
    "test:docker:onboard": "bash scripts/e2e/onboard-docker.sh",
+    "test:docker:openwebui": "bash scripts/e2e/openwebui-docker.sh",
    "test:docker:plugins": "bash scripts/e2e/plugins-docker.sh",
    "test:docker:qr": "bash scripts/e2e/qr-import-docker.sh",
    "test:e2e": "vitest run --config vitest.e2e.config.ts",
--- a/scripts/e2e/openwebui-docker.sh
+++ b/scripts/e2e/openwebui-docker.sh
@@ -0,0 +1,184 @@
+#!/usr/bin/env bash
+set -euo pipefail
+
+ROOT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)"
+source "$ROOT_DIR/scripts/lib/live-docker-auth.sh"
+
+IMAGE_NAME="openclaw-openwebui-e2e"
+OPENWEBUI_IMAGE="${OPENWEBUI_IMAGE:-ghcr.io/open-webui/open-webui:v0.8.10}"
+PROFILE_FILE="${OPENCLAW_PROFILE_FILE:-$HOME/.profile}"
+MODEL="${OPENCLAW_OPENWEBUI_MODEL:-openai/gpt-5.4}"
+PROMPT_NONCE="OPENWEBUI_DOCKER_E2E_$(date +%s)_$$"
+PROMPT="${OPENCLAW_OPENWEBUI_PROMPT:-Reply with exactly this token and nothing else: ${PROMPT_NONCE}}"
+PORT="${OPENCLAW_OPENWEBUI_GATEWAY_PORT:-18789}"
+WEBUI_PORT="${OPENCLAW_OPENWEBUI_PORT:-8080}"
+TOKEN="openwebui-e2e-$(date +%s)-$$"
+ADMIN_EMAIL="${OPENCLAW_OPENWEBUI_ADMIN_EMAIL:-openwebui-e2e@example.com}"
+ADMIN_PASSWORD="${OPENCLAW_OPENWEBUI_ADMIN_PASSWORD:-OpenWebUI-E2E-Password-$(date +%s)-$$}"
+NET_NAME="openclaw-openwebui-e2e-$$"
+GW_NAME="openclaw-openwebui-gateway-$$"
+OW_NAME="openclaw-openwebui-$$"
+
+PROFILE_MOUNT=()
+if [[ -f "$PROFILE_FILE" ]]; then
+  PROFILE_MOUNT=(-v "$PROFILE_FILE":/home/appuser/.profile:ro)
+fi
+
+AUTH_DIRS=()
+if [[ -n "${OPENCLAW_DOCKER_AUTH_DIRS:-}" ]]; then
+  while IFS= read -r auth_dir; do
+    [[ -n "$auth_dir" ]] || continue
+    AUTH_DIRS+=("$auth_dir")
+  done < <(openclaw_live_collect_auth_dirs)
+fi
+AUTH_DIRS_CSV="$(openclaw_live_join_csv "${AUTH_DIRS[@]}")"
+
+EXTERNAL_AUTH_MOUNTS=()
+for auth_dir in "${AUTH_DIRS[@]}"; do
+  host_path="$HOME/$auth_dir"
+  if [[ -d "$host_path" ]]; then
+    EXTERNAL_AUTH_MOUNTS+=(-v "$host_path":/host-auth/"$auth_dir":ro)
+  fi
+done
+
+cleanup() {
+  docker rm -f "$OW_NAME" >/dev/null 2>&1 || true
+  docker rm -f "$GW_NAME" >/dev/null 2>&1 || true
+  docker network rm "$NET_NAME" >/dev/null 2>&1 || true
+}
+trap cleanup EXIT
+
+echo "Building Docker image..."
+docker build -t "$IMAGE_NAME" -f "$ROOT_DIR/scripts/e2e/Dockerfile" "$ROOT_DIR"
+
+echo "Pulling Open WebUI image: $OPENWEBUI_IMAGE"
+docker pull "$OPENWEBUI_IMAGE" >/dev/null
+
+echo "Creating Docker network..."
+docker network create "$NET_NAME" >/dev/null
+
+echo "Starting gateway container..."
+docker run -d \
+  --name "$GW_NAME" \
+  --network "$NET_NAME" \
+  -e "OPENCLAW_DOCKER_AUTH_DIRS_RESOLVED=$AUTH_DIRS_CSV" \
+  -e "OPENCLAW_GATEWAY_TOKEN=$TOKEN" \
+  -e "OPENCLAW_OPENWEBUI_MODEL=$MODEL" \
+  -e "OPENCLAW_SKIP_CHANNELS=1" \
+  -e "OPENCLAW_SKIP_GMAIL_WATCHER=1" \
+  -e "OPENCLAW_SKIP_CRON=1" \
+  -e "OPENCLAW_SKIP_CANVAS_HOST=1" \
+  "${EXTERNAL_AUTH_MOUNTS[@]}" \
+  "${PROFILE_MOUNT[@]}" \
+  "$IMAGE_NAME" \
+  bash -lc '
+    set -euo pipefail
+    [ -f "$HOME/.profile" ] && source "$HOME/.profile" || true
+    IFS="," read -r -a auth_dirs <<<"${OPENCLAW_DOCKER_AUTH_DIRS_RESOLVED:-}"
+    for auth_dir in "${auth_dirs[@]}"; do
+      [ -n "$auth_dir" ] || continue
+      if [ -d "/host-auth/$auth_dir" ]; then
+        mkdir -p "$HOME/$auth_dir"
+        cp -R "/host-auth/$auth_dir/." "$HOME/$auth_dir"
+        chmod -R u+rwX "$HOME/$auth_dir" || true
+      fi
+    done
+
+    entry=dist/index.mjs
+    [ -f "$entry" ] || entry=dist/index.js
+
+    node "$entry" config set gateway.controlUi.enabled false >/dev/null
+    node "$entry" config set gateway.mode local >/dev/null
+    node "$entry" config set gateway.bind lan >/dev/null
+    node "$entry" config set gateway.auth.mode token >/dev/null
+    node "$entry" config set gateway.auth.token "$OPENCLAW_GATEWAY_TOKEN" >/dev/null
+    node "$entry" config set gateway.http.endpoints.chatCompletions.enabled true --strict-json >/dev/null
+    node "$entry" config set agents.defaults.model.primary "$OPENCLAW_OPENWEBUI_MODEL" >/dev/null
+
+    exec node "$entry" gateway --port '"$PORT"' --bind lan --allow-unconfigured > /tmp/openwebui-gateway.log 2>&1
+  '
+
+echo "Waiting for gateway HTTP surface..."
+gateway_ready=0
+for _ in $(seq 1 60); do
+  if [ "$(docker inspect -f '{{.State.Running}}' "$GW_NAME" 2>/dev/null || echo false)" != "true" ]; then
+    break
+  fi
+  if docker exec "$GW_NAME" bash -lc "node --input-type=module -e '
+    const res = await fetch(\"http://127.0.0.1:$PORT/v1/models\", {
+      headers: { authorization: \"Bearer $TOKEN\" },
+    }).catch(() => null);
+    process.exit(res?.status === 200 ? 0 : 1);
+  ' >/dev/null 2>&1"; then
+    gateway_ready=1
+    break
+  fi
+  sleep 1
+done
+
+if [ "$gateway_ready" -ne 1 ]; then
+  echo "Gateway failed to start"
+  docker logs "$GW_NAME" 2>&1 | tail -n 200 || true
+  exit 1
+fi
+
+echo "Starting Open WebUI container..."
+docker run -d \
+  --name "$OW_NAME" \
+  --network "$NET_NAME" \
+  -e ENV=prod \
+  -e WEBUI_NAME="OpenClaw E2E" \
+  -e WEBUI_SECRET_KEY="openclaw-openwebui-e2e-secret" \
+  -e OFFLINE_MODE=True \
+  -e ENABLE_VERSION_UPDATE_CHECK=False \
+  -e ENABLE_PERSISTENT_CONFIG=False \
+  -e ENABLE_OLLAMA_API=False \
+  -e ENABLE_OPENAI_API=True \
+  -e OPENAI_API_BASE_URLS="http://$GW_NAME:$PORT/v1" \
+  -e OPENAI_API_KEY="$TOKEN" \
+  -e OPENAI_API_KEYS="$TOKEN" \
+  -e RAG_EMBEDDING_MODEL_AUTO_UPDATE=False \
+  -e RAG_RERANKING_MODEL_AUTO_UPDATE=False \
+  -e WEBUI_ADMIN_EMAIL="$ADMIN_EMAIL" \
+  -e WEBUI_ADMIN_PASSWORD="$ADMIN_PASSWORD" \
+  -e WEBUI_ADMIN_NAME="OpenClaw E2E" \
+  -e ENABLE_SIGNUP=False \
+  -e DEFAULT_MODELS="openclaw/default" \
+  "$OPENWEBUI_IMAGE" >/dev/null
+
+echo "Waiting for Open WebUI..."
+ow_ready=0
+for _ in $(seq 1 90); do
+  if [ "$(docker inspect -f '{{.State.Running}}' "$OW_NAME" 2>/dev/null || echo false)" != "true" ]; then
+    break
+  fi
+  if docker exec "$GW_NAME" bash -lc "node --input-type=module -e '
+    const res = await fetch(\"http://$OW_NAME:$WEBUI_PORT/\").catch(() => null);
+    process.exit(res && res.status < 500 ? 0 : 1);
+  ' >/dev/null 2>&1"; then
+    ow_ready=1
+    break
+  fi
+  sleep 1
+done
+
+if [ "$ow_ready" -ne 1 ]; then
+  echo "Open WebUI failed to start"
+  docker logs "$OW_NAME" 2>&1 | tail -n 200 || true
+  exit 1
+fi
+
+echo "Running Open WebUI -> OpenClaw smoke..."
+docker exec \
+  -e "OPENWEBUI_BASE_URL=http://$OW_NAME:$WEBUI_PORT" \
+  -e "OPENWEBUI_ADMIN_EMAIL=$ADMIN_EMAIL" \
+  -e "OPENWEBUI_ADMIN_PASSWORD=$ADMIN_PASSWORD" \
+  -e "OPENWEBUI_EXPECTED_NONCE=$PROMPT_NONCE" \
+  -e "OPENWEBUI_PROMPT=$PROMPT" \
+  "$GW_NAME" \
+  node /app/scripts/e2e/openwebui-probe.mjs
+
+echo "Open WebUI container logs:"
+docker logs "$OW_NAME" 2>&1 | tail -n 80 || true
+
+echo "OK"
--- a/scripts/e2e/openwebui-probe.mjs
+++ b/scripts/e2e/openwebui-probe.mjs
@@ -0,0 +1,95 @@
+const baseUrl = process.env.OPENWEBUI_BASE_URL ?? "";
+const email = process.env.OPENWEBUI_ADMIN_EMAIL ?? "";
+const password = process.env.OPENWEBUI_ADMIN_PASSWORD ?? "";
+const expectedNonce = process.env.OPENWEBUI_EXPECTED_NONCE ?? "";
+const prompt = process.env.OPENWEBUI_PROMPT ?? "";
+
+if (!baseUrl || !email || !password || !expectedNonce || !prompt) {
+  throw new Error("Missing required OPENWEBUI_* environment variables");
+}
+
+function getCookieHeader(res) {
+  const raw = res.headers.get("set-cookie");
+  if (!raw) {
+    return "";
+  }
+  return raw
+    .split(/,(?=[^;]+=[^;]+)/g)
+    .map((part) => part.split(";", 1)[0]?.trim())
+    .filter(Boolean)
+    .join("; ");
+}
+
+function buildAuthHeaders(token, cookie) {
+  const headers = {};
+  if (token) {
+    headers.authorization = `Bearer ${token}`;
+  }
+  if (cookie) {
+    headers.cookie = cookie;
+  }
+  return headers;
+}
+
+const signinRes = await fetch(`${baseUrl}/api/v1/auths/signin`, {
+  method: "POST",
+  headers: { "content-type": "application/json" },
+  body: JSON.stringify({ email, password }),
+});
+if (!signinRes.ok) {
+  const body = await signinRes.text();
+  throw new Error(`signin failed: HTTP ${signinRes.status} ${body}`);
+}
+
+const signinJson = await signinRes.json();
+const token =
+  signinJson?.token ?? signinJson?.access_token ?? signinJson?.jwt ?? signinJson?.data?.token ?? "";
+const cookie = getCookieHeader(signinRes);
+const authHeaders = {
+  ...buildAuthHeaders(token, cookie),
+  accept: "application/json",
+};
+
+const modelsRes = await fetch(`${baseUrl}/api/models`, { headers: authHeaders });
+if (!modelsRes.ok) {
+  throw new Error(`/api/models failed: HTTP ${modelsRes.status} ${await modelsRes.text()}`);
+}
+const modelsJson = await modelsRes.json();
+const models = Array.isArray(modelsJson)
+  ? modelsJson
+  : Array.isArray(modelsJson?.data)
+    ? modelsJson.data
+    : Array.isArray(modelsJson?.models)
+      ? modelsJson.models
+      : [];
+const modelIds = models
+  .map((entry) => entry?.id ?? entry?.model ?? entry?.name)
+  .filter((value) => typeof value === "string");
+const targetModel =
+  modelIds.find((id) => id === "openclaw/default") ?? modelIds.find((id) => id === "openclaw");
+if (!targetModel) {
+  throw new Error(`openclaw model missing from Open WebUI model list: ${JSON.stringify(modelIds)}`);
+}
+
+const chatRes = await fetch(`${baseUrl}/api/chat/completions`, {
+  method: "POST",
+  headers: {
+    ...authHeaders,
+    "content-type": "application/json",
+  },
+  body: JSON.stringify({
+    model: targetModel,
+    messages: [{ role: "user", content: prompt }],
+  }),
+});
+if (!chatRes.ok) {
+  throw new Error(`/api/chat/completions failed: HTTP ${chatRes.status} ${await chatRes.text()}`);
+}
+const chatJson = await chatRes.json();
+const reply =
+  chatJson?.choices?.[0]?.message?.content ?? chatJson?.message?.content ?? chatJson?.content ?? "";
+if (typeof reply !== "string" || !reply.includes(expectedNonce)) {
+  throw new Error(`chat reply missing nonce: ${JSON.stringify(reply)}`);
+}
+
+console.log(JSON.stringify({ ok: true, model: targetModel, reply }, null, 2));