diff --git a/docs/features/chat-conversations/chat-features/code-execution/index.md b/docs/features/chat-conversations/chat-features/code-execution/index.md index ce504682..dcb1e880 100644 --- a/docs/features/chat-conversations/chat-features/code-execution/index.md +++ b/docs/features/chat-conversations/chat-features/code-execution/index.md @@ -25,17 +25,33 @@ Open WebUI supports multiple code execution backends, each suited to different u ### Pyodide (Default) -Pyodide runs Python in the browser via WebAssembly. It is sandboxed and safe for multi-user environments, but comes with constraints: +Pyodide runs Python in the browser via WebAssembly. It is sandboxed and safe for multi-user environments, but comes with some constraints: -- **No persistent storage** — the filesystem resets between executions. +- **Persistent file storage** — the virtual filesystem at `/mnt/uploads/` is backed by IndexedDB (IDBFS). Files persist across code executions within the same session and survive page reloads. +- **Built-in file browser** — when Code Interpreter is enabled, a file browser panel appears in the chat controls sidebar. You can browse, preview, upload, download, and delete files in the Pyodide filesystem — no terminal needed. +- **User file access** — files attached to messages are automatically placed in `/mnt/uploads/` before code execution, so the model (and your code) can read them directly. - **Limited library support** — only a subset of Python packages are available. Libraries that rely on C extensions or system calls may not work. - **No shell access** — cannot run shell commands, install packages, or interact with the OS. :::tip -Pyodide works well for **text analysis, hash computation, chart generation**, and other self-contained tasks. Chart libraries like matplotlib produce base64-encoded images that Open WebUI automatically captures, uploads as files, and injects direct image links into the output — so models can display charts directly in chat without any extra setup. +Pyodide works well for **text analysis, hash computation, chart generation, file processing**, and other self-contained tasks. Chart libraries like matplotlib produce base64-encoded images that Open WebUI automatically captures, uploads as files, and injects direct image links into the output — so models can display charts directly in chat without any extra setup. ::: -### Jupyter +:::warning Best for basic analysis only +Pyodide runs Python via WebAssembly inside the browser. The AI **cannot install additional libraries** beyond the small fixed set listed below — any code that imports an unsupported package will fail. Execution is also **significantly slower** than native Python, and large datasets or CPU-intensive tasks may hit browser memory limits. Pyodide is best suited for **basic file analysis, simple calculations, text processing, and chart generation**. For anything more demanding, use **Open Terminal** instead, which provides full native performance and unrestricted package access inside a Docker container. + +Available libraries: micropip, requests, beautifulsoup4, numpy, pandas, matplotlib, seaborn, scikit-learn, scipy, regex, sympy, tiktoken, pytz, and the Python standard library. **Nothing else can be installed at runtime.** +::: + +:::note Mutually exclusive with Open Terminal +The Code Interpreter toggle and the Open Terminal toggle cannot be active at the same time. Activating one will deactivate the other — they serve similar purposes but use different execution backends. +::: + +### Jupyter (Legacy) + +:::caution Legacy Engine +Jupyter is now considered a **legacy** code execution engine. The Pyodide engine is recommended for most use cases, and Open Terminal is recommended when you need full server-side execution. Jupyter support may be deprecated in a future release. +::: Jupyter provides a full Python environment and can handle virtually any task — file creation, package installation, and complex library usage. However, it has significant drawbacks in shared deployments: @@ -58,15 +74,17 @@ If you are running a multi-user or organizational deployment, **Jupyter is not r ### Comparison -| Consideration | Pyodide | Jupyter | Open Terminal | +| Consideration | Pyodide | Jupyter (Legacy) | Open Terminal | | :--- | :--- | :--- | :--- | | **Runs in** | Browser (WebAssembly) | Server (Python kernel) | Server (Docker container) | | **Library support** | Limited subset | Full Python ecosystem | Full OS — any language, any tool | | **Shell access** | ❌ None | ⚠️ Limited | ✅ Full shell | -| **File persistence** | ❌ Resets each execution | ✅ Shared filesystem | ✅ Container filesystem (until removal) | +| **File persistence** | ✅ IDBFS (persists across executions & reloads) | ✅ Shared filesystem | ✅ Container filesystem (until removal) | +| **File browser** | ✅ Built-in sidebar panel | ❌ None | ✅ Built-in sidebar panel | +| **User file access** | ✅ Attached files placed in `/mnt/uploads/` | ❌ Manual | ✅ Attached files available | | **Isolation** | ✅ Browser sandbox | ❌ Shared environment | ✅ Container-level (when using Docker) | | **Multi-user safety** | ✅ Per-user by design | ⚠️ Not isolated | ⚠️ Single instance (per-user containers planned) | -| **File generation** | ❌ Very limited | ✅ Full support | ✅ Full support with upload/download | +| **File generation** | ✅ Write to `/mnt/uploads/`, download via file browser | ✅ Full support | ✅ Full support with upload/download | | **Setup** | None (built-in) | Admin configures globally | Native integration via Settings → Integrations, or as a Tool Server | | **Recommended for orgs** | ✅ Safe default | ❌ Not without isolation | ✅ Per-user by design | | **Enterprise scalability** | ✅ Client-side, no server load | ❌ Single shared instance | ⚠️ Manual per-user instances | diff --git a/docs/features/chat-conversations/chat-features/code-execution/python.md b/docs/features/chat-conversations/chat-features/code-execution/python.md index 04e8d51f..8c2dd087 100644 --- a/docs/features/chat-conversations/chat-features/code-execution/python.md +++ b/docs/features/chat-conversations/chat-features/code-execution/python.md @@ -12,7 +12,7 @@ Open WebUI provides two ways to execute Python code: 1. **Manual Code Execution**: Run Python code blocks generated by LLMs using a "Run" button in the browser (uses Pyodide/WebAssembly). 2. **Code Interpreter**: An AI capability that allows models to automatically write and execute Python code as part of their response (uses Pyodide or Jupyter). -Both methods support visual outputs like matplotlib charts that can be displayed inline in your chat. +Both methods support visual outputs like matplotlib charts that can be displayed inline in your chat. When using the Pyodide engine, a **persistent virtual filesystem** at `/mnt/uploads/` is available — files survive across code executions and page reloads, and files attached to messages are automatically placed there for your code to access. ## Code Interpreter Capability @@ -35,7 +35,7 @@ The Code Interpreter is a model capability that enables LLMs to write and execut These settings can be configured at **Admin Panel → Settings → Code Execution**: - Enable/disable code interpreter -- Select engine (Pyodide or Jupyter) +- Select engine: **Pyodide** (recommended) or **Jupyter (Legacy)** - Configure Jupyter connection settings - Set blocked modules @@ -44,12 +44,16 @@ These settings can be configured at **Admin Panel → Settings → Code Executio | Variable | Default | Description | |----------|---------|-------------| | `ENABLE_CODE_INTERPRETER` | `true` | Enable/disable code interpreter globally | -| `CODE_INTERPRETER_ENGINE` | `pyodide` | Engine to use: `pyodide` (browser) or `jupyter` (server) | +| `CODE_INTERPRETER_ENGINE` | `pyodide` | Engine to use: `pyodide` (browser, recommended) or `jupyter` (server, legacy) | | `CODE_INTERPRETER_PROMPT_TEMPLATE` | (built-in) | Custom prompt template for code interpreter | | `CODE_INTERPRETER_BLACKLISTED_MODULES` | `""` | Comma-separated list of blocked Python modules | For Jupyter configuration, see the [Jupyter Notebook Integration](/tutorials/integrations/dev-tools/jupyter) tutorial. +:::note Filesystem Prompt Injection +When the Pyodide engine is selected, Open WebUI automatically appends a filesystem-awareness prompt to the code interpreter instructions. This tells the model about `/mnt/uploads/` and how to discover user-uploaded files. When using Jupyter, this filesystem prompt is not appended (since Jupyter has its own filesystem). You do not need to include filesystem instructions in your custom `CODE_INTERPRETER_PROMPT_TEMPLATE` — they are added automatically. +::: + ### Native Function Calling (Native Mode) When using **Native function calling mode** with a capable model (e.g., GPT-5, Claude 4.5, MiniMax M2.5), the code interpreter is available as a builtin tool called `execute_code`. This provides a more integrated experience: @@ -126,6 +130,8 @@ If you see raw base64 text appearing in chat responses, the model is incorrectly Open WebUI includes a browser-based Python environment using [Pyodide](https://pyodide.org/) (WebAssembly). This allows running Python scripts directly in your browser with no server-side setup. +The Pyodide worker is **persistent** — it is created once and reused across code executions. This means variables, imported modules, and files written to the virtual filesystem are retained between executions within the same session. + ### Running Code Manually 1. Ask an LLM to write Python code @@ -135,21 +141,63 @@ Open WebUI includes a browser-based Python environment using [Pyodide](https://p ### Supported Libraries -Pyodide includes the following pre-configured packages: +Pyodide includes the following packages, which are auto-detected from import statements and loaded on demand: -- micropip -- packaging -- requests -- beautifulsoup4 -- numpy -- pandas -- matplotlib -- scikit-learn -- scipy -- regex +| Package | Use case | +|---------|----------| +| micropip | Package installer (internal use) | +| requests | HTTP requests | +| beautifulsoup4 | HTML/XML parsing | +| numpy | Numerical computing | +| pandas | Data analysis and manipulation | +| matplotlib | Chart and plot generation | +| seaborn | Statistical data visualization | +| scikit-learn | Machine learning | +| scipy | Scientific computing | +| regex | Advanced regular expressions | +| sympy | Symbolic mathematics | +| tiktoken | Token counting for LLMs | +| pytz | Timezone handling | -:::note -Packages not pre-compiled in Pyodide cannot be installed at runtime. For additional packages, consider using the Jupyter integration or forking Pyodide to add custom packages. +The Python standard library is also fully available (json, csv, math, datetime, os, io, etc.). + +:::warning No runtime installation +The AI **cannot install additional libraries** beyond the list above. Any code that imports an unsupported package will fail with an import error. Packages that require C extensions, system calls, or native binaries (e.g., torch, tensorflow, opencv, psycopg2) are **not available** and cannot be made available in Pyodide. Pyodide is best suited for **basic file analysis, simple calculations, text processing, and chart generation**. For full Python package access, use **[Open Terminal](/features/chat-conversations/chat-features/code-execution#open-terminal)** instead. +::: + +## Persistent File System + +When using the Pyodide engine, a persistent virtual filesystem is mounted at `/mnt/uploads/`. This filesystem is backed by the browser's IndexedDB via [IDBFS](https://emscripten.org/docs/api_reference/Filesystem-API.html#filesystem-api-idbfs) and provides: + +- **Cross-execution persistence** — files written by one code execution are accessible in subsequent executions. +- **Cross-reload persistence** — files survive page reloads (stored in IndexedDB). +- **Automatic upload mounting** — files attached to messages are fetched from the server and placed in `/mnt/uploads/` before code execution, so the model can read them directly. +- **File browser panel** — when Code Interpreter is enabled, a file browser appears in the chat controls sidebar. You can browse, preview, upload, download, and delete files — no terminal needed. + +### Working with Files in Code + +```python +import os + +# List uploaded files +print(os.listdir('/mnt/uploads')) + +# Read a user-uploaded CSV +import pandas as pd +df = pd.read_csv('/mnt/uploads/data.csv') +print(df.head()) + +# Write output to the persistent filesystem (downloadable via file browser) +df.to_csv('/mnt/uploads/result.csv', index=False) +print('Saved result.csv to /mnt/uploads/') +``` + +:::tip +The file browser panel lets you download any file the model creates. Ask the model to save its output to `/mnt/uploads/` and it will appear in the file browser for download. +::: + +:::note Jupyter Engine +The persistent filesystem prompt and `/mnt/uploads/` integration are **Pyodide-only**. When using the Jupyter engine, files are managed through Jupyter's own filesystem. The file browser panel is not available for Jupyter. ::: ## Example: Creating a Chart diff --git a/docs/features/chat-conversations/chat-features/conversation-organization.md b/docs/features/chat-conversations/chat-features/conversation-organization.md index ef19841f..586d310b 100644 --- a/docs/features/chat-conversations/chat-features/conversation-organization.md +++ b/docs/features/chat-conversations/chat-features/conversation-organization.md @@ -36,9 +36,11 @@ Organize existing chats by moving them into folders: Folders can be nested within other folders to create hierarchical organization: -- Drag a folder onto another folder to make it a subfolder. -- Use the right-click menu to move folders between parent folders. +- **Create subfolder from menu**: Right-click (or click the three-dot menu ⋯) on any folder and select **"Create Folder"** to create a new subfolder directly inside it. +- **Drag and drop**: Drag a folder onto another folder to make it a subfolder. +- **Move via context menu**: Right-click on a folder and use the move option to relocate it under a different parent. - Folders can be expanded or collapsed to show/hide their contents. +- Subfolder names must be unique within the same parent folder. If a duplicate name is entered, a number is automatically appended (e.g., "Notes 1"). ### Starting a Chat in a Folder diff --git a/docs/features/chat-conversations/chat-features/reasoning-models.mdx b/docs/features/chat-conversations/chat-features/reasoning-models.mdx index 8651cf28..59432c10 100644 --- a/docs/features/chat-conversations/chat-features/reasoning-models.mdx +++ b/docs/features/chat-conversations/chat-features/reasoning-models.mdx @@ -38,7 +38,7 @@ If your model uses different tags, you can provide a list of tag pairs in the `r ## Configuration & Behavior - **Stripping from Payload**: The `reasoning_tags` parameter itself is an Open WebUI-specific control and is **stripped** from the payload before being sent to the LLM backend (OpenAI, Ollama, etc.). This ensures compatibility with providers that do not recognize this parameter. -- **Chat History**: Thinking tags are **not** stripped from the chat history. If previous messages in a conversation contain thinking blocks, they are sent back to the model as part of the context, allowing the model to "remember" its previous reasoning steps. +- **Chat History**: Reasoning content is preserved in chat history and **sent back to the model** across turns. When building messages for subsequent requests, Open WebUI serializes the reasoning content with its original tags (e.g., `...`) and includes it in the assistant message's `content` field. This allows the model to "remember" its previous reasoning steps across the entire conversation. - **UI Rendering**: Internally, reasoning blocks are processed and rendered using a specialized UI component. When saved or exported, they may be represented as HTML `
` tags. --- @@ -153,8 +153,8 @@ Open WebUI follows the **OpenAI Chat Completions API standard**. Reasoning conte ### Important Notes -- **Within-turn preservation**: Reasoning is preserved and sent back to the API only within the same turn (while tool calls are being processed) -- **Cross-turn behavior**: Between separate user messages, reasoning is **not** sent back to the API. The thinking content is displayed in the UI but stripped from the message content that gets sent in subsequent requests. +- **Within-turn preservation**: Reasoning is preserved and sent back to the API within the same turn (while tool calls are being processed). +- **Cross-turn behavior**: Reasoning content **is** sent back to the API across turns. When building messages for subsequent requests, Open WebUI serializes the reasoning content with its original tags (e.g., `...`) and includes it in the assistant message's `content` field. This allows the model to maintain context of its previous reasoning throughout the conversation. - **Text-based serialization**: Reasoning is sent as text wrapped in tags (e.g., `thinking content`), not as structured content blocks. This works with most OpenAI-compatible APIs but may not align with provider-specific formats like Anthropic's extended thinking content blocks. --- @@ -373,11 +373,11 @@ If the model uses tags that are not in the default list and have not been config ### Does the model see its own thinking? -**It depends on the context:** +**Yes.** Reasoning content is preserved and sent back to the model in both scenarios: - **Within the same turn (during tool calls)**: **Yes**. When a model makes tool calls, Open WebUI preserves the reasoning content and sends it back to the API as part of the assistant message. This enables the model to maintain context about what it was thinking when it made the tool call. -- **Across different turns**: **No**. When a user message starts a fresh turn, the reasoning from previous turns is **not** sent back to the API. The thinking content is extracted and displayed in the UI but stripped from the message content before being sent in subsequent requests. This follows the design of reasoning models like OpenAI's `o1`, where the "chain of thought" is intended to be internal and ephemeral. +- **Across different turns**: **Yes**. When building messages for subsequent requests, Open WebUI serializes reasoning content from previous turns with its original tags (e.g., `...`) and includes it in the assistant message's `content` field. This allows the model to reference its previous reasoning throughout the conversation. ### How is reasoning sent during tool calls? diff --git a/docs/features/chat-conversations/data-controls/import-export.md b/docs/features/chat-conversations/data-controls/import-export.md index 30f4063c..984c433c 100644 --- a/docs/features/chat-conversations/data-controls/import-export.md +++ b/docs/features/chat-conversations/data-controls/import-export.md @@ -29,14 +29,174 @@ It's a good practice to periodically export your chats, especially before major Click the **Import Chats** button and select a JSON file to restore conversations. Open WebUI supports importing from: - **Open WebUI exports**: Native JSON format from previous exports -- **ChatGPT exports**: Conversations exported from OpenAI's ChatGPT -- **Other compatible formats**: JSON files following the expected structure +- **ChatGPT exports**: Conversations exported from OpenAI's ChatGPT (auto-detected and converted) +- **Custom JSON files**: Any JSON file that follows the expected structure documented below ### Import Behavior - Imported chats are added to your existing conversations (they don't replace them) -- Duplicate detection may vary based on chat IDs -- If using ChatGPT exports, the format is automatically converted +- Each imported chat receives a new unique ID, so re-importing the same file will create duplicates +- If using ChatGPT exports, the format is automatically detected and converted + +## Chat Import JSON Schema + +The import file must be a **JSON array** of chat objects. There are two accepted formats: the **standard format** (used by Open WebUI exports) and a **legacy format**. + +### Standard Format (Recommended) + +Each object in the array should have a `chat` key containing the conversation data: + +```json +[ + { + "chat": { + "title": "My Conversation", + "models": ["llama3.2"], + "history": { + "currentId": "message-id-2", + "messages": { + "message-id-1": { + "id": "message-id-1", + "parentId": null, + "childrenIds": ["message-id-2"], + "role": "user", + "content": "Hello, how are you?", + "timestamp": 1700000000 + }, + "message-id-2": { + "id": "message-id-2", + "parentId": "message-id-1", + "childrenIds": [], + "role": "assistant", + "content": "I'm doing well, thank you!", + "model": "llama3.2", + "done": true, + "timestamp": 1700000005 + } + } + } + }, + "meta": { + "tags": ["greeting"] + }, + "pinned": false, + "folder_id": null, + "created_at": 1700000000, + "updated_at": 1700000005 + } +] +``` + +### Legacy Format + +If the objects in the array do **not** have a `chat` key, the entire object is treated as the chat data itself (i.e. the object is wrapped in `{ "chat": }` automatically): + +```json +[ + { + "title": "My Conversation", + "models": ["llama3.2"], + "history": { + "currentId": "message-id-2", + "messages": { + "message-id-1": { + "id": "message-id-1", + "parentId": null, + "childrenIds": ["message-id-2"], + "role": "user", + "content": "Hello!" + }, + "message-id-2": { + "id": "message-id-2", + "parentId": "message-id-1", + "childrenIds": [], + "role": "assistant", + "content": "Hi there!", + "model": "llama3.2", + "done": true + } + } + } + } +] +``` + +### Field Reference + +#### Top-Level Chat Object (Standard Format) + +| Field | Type | Required | Description | +|---|---|---|---| +| `chat` | object | ✅ | The conversation data (see Chat Data below) | +| `meta` | object | ❌ | Metadata such as `tags` (array of strings). Defaults to `{}` | +| `pinned` | boolean | ❌ | Whether the chat is pinned. Defaults to `false` | +| `folder_id` | string \| null | ❌ | ID of the folder to place the chat in. Defaults to `null` | +| `created_at` | integer \| null | ❌ | Unix timestamp (seconds) for when the chat was created | +| `updated_at` | integer \| null | ❌ | Unix timestamp (seconds) for when the chat was last updated | + +#### Chat Data Object + +| Field | Type | Required | Description | +|---|---|---|---| +| `title` | string | ❌ | The conversation title. Defaults to `"New Chat"` | +| `models` | string[] | ❌ | List of model identifiers used in the conversation | +| `history` | object | ✅ | Contains the message tree (see History below) | +| `options` | object | ❌ | Chat-level options/parameters | + +#### History Object + +| Field | Type | Required | Description | +|---|---|---|---| +| `currentId` | string | ✅ | ID of the last message in the active conversation branch | +| `messages` | object | ✅ | A map of message ID → message object (see Message below) | + +#### Message Object + +| Field | Type | Required | Description | +|---|---|---|---| +| `id` | string | ✅ | Unique identifier for the message | +| `parentId` | string \| null | ✅ | ID of the parent message, or `null` for the first message | +| `childrenIds` | string[] | ✅ | Array of child message IDs. Empty array `[]` for the last message | +| `role` | string | ✅ | Either `"user"` or `"assistant"` | +| `content` | string | ✅ | The message text (supports Markdown) | +| `model` | string | ❌ | Model identifier (relevant for assistant messages) | +| `done` | boolean | ❌ | Whether the response is complete | +| `timestamp` | integer | ❌ | Unix timestamp (seconds) for the message | +| `context` | string \| null | ❌ | Additional context for the message | + +:::info Message Tree Structure +Messages use a **tree structure** rather than a flat list. Each message references its parent via `parentId` and its children via `childrenIds`. This allows Open WebUI to support branching conversations (e.g. editing a message and getting a different response). The `history.currentId` field points to the last message in the currently active branch. +::: + +### ChatGPT Export Format + +ChatGPT exports are automatically detected when the first object in the array contains a `mapping` key. You don't need to manually convert ChatGPT exports—just import the file directly and Open WebUI will handle the conversion. + +### Minimal Working Example + +The smallest valid import file looks like this: + +```json +[ + { + "title": "Quick Chat", + "history": { + "currentId": "msg-1", + "messages": { + "msg-1": { + "id": "msg-1", + "parentId": null, + "childrenIds": [], + "role": "user", + "content": "Hello!" + } + } + } + } +] +``` + +This uses the legacy format (no `chat` wrapper) with a single user message. ## FAQ @@ -44,7 +204,13 @@ Click the **Import Chats** button and select a JSON file to restore conversation **A:** No. Imported chats are added alongside your existing conversations. **Q: Can I import chats from Claude, Gemini, or other platforms?** -**A:** Currently, native import support is available for Open WebUI and ChatGPT formats. Other platforms may require manual conversion to the expected JSON structure. +**A:** There is no built-in converter for these platforms. You would need to transform your export into the JSON structure documented above. The key requirement is building the message tree with correct `parentId` / `childrenIds` relationships. **Q: Is there a size limit for imports?** **A:** There's no hard-coded limit, but very large files may take longer to process. The practical limit depends on your server configuration and available memory. + +**Q: What happens if I import the same file twice?** +**A:** Each import creates new chats with fresh IDs, so you will end up with duplicate conversations. + +**Q: What message roles are supported?** +**A:** The import supports `"user"` and `"assistant"` roles. System messages are typically set via model configurations rather than stored in the chat history. diff --git a/docs/features/extensibility/open-terminal/index.md b/docs/features/extensibility/open-terminal/index.md index 33b6c973..77141a4f 100644 --- a/docs/features/extensibility/open-terminal/index.md +++ b/docs/features/extensibility/open-terminal/index.md @@ -69,12 +69,50 @@ If you don't set an API key, one is generated automatically. Grab it with `docke | **Data processing** | jq, xmlstarlet, sqlite3 | | **Compression** | zip, unzip, tar, gzip, bzip2, xz, zstd, p7zip | | **System** | procps, htop, lsof, strace, sysstat, sudo, tmux, screen | +| **Container tools** | Docker CLI, Docker Compose, Docker Buildx | | **Python libraries** | numpy, pandas, scipy, scikit-learn, matplotlib, seaborn, plotly, jupyter, ipython, requests, beautifulsoup4, lxml, sqlalchemy, pyyaml, rich, and more | You can customize the image by forking the repo and editing the [Dockerfile](https://github.com/open-webui/open-terminal/blob/main/Dockerfile). +#### Customizing the Docker Environment + +The easiest way to add extra packages is with environment variables — no fork needed: + +```bash +docker run -d --name open-terminal -p 8000:8000 \ + -e OPEN_TERMINAL_PACKAGES="cowsay figlet" \ + -e OPEN_TERMINAL_PIP_PACKAGES="httpx polars" \ + ghcr.io/open-webui/open-terminal +``` + +| Variable | Description | +| :--- | :--- | +| `OPEN_TERMINAL_PACKAGES` | Space-separated list of **apt** packages to install at startup | +| `OPEN_TERMINAL_PIP_PACKAGES` | Space-separated list of **pip** packages to install at startup | + +:::note +Packages are installed each time the container starts, so startup will take longer with large package lists. For heavy customization, build a custom image instead. +::: + +#### Docker Access + +The container image includes the Docker CLI, Compose, and Buildx. To let agents build images, run containers, and manage Docker resources, mount the host's Docker socket: + +```bash +docker run -d --name open-terminal -p 8000:8000 \ + -v /var/run/docker.sock:/var/run/docker.sock \ + -v open-terminal:/home/user \ + ghcr.io/open-webui/open-terminal +``` + +The entrypoint automatically fixes socket group permissions so `docker` commands work without `sudo`. + +:::warning +Mounting the Docker socket gives the container full access to the host's Docker daemon. Only do this in trusted environments. +::: + ### Bare Metal Want your AI to work directly on **your machine**, with **your files**, **your tools**, and **your environment**? Bare metal mode is for you. No container boundary. The AI gets the same access you do. @@ -146,7 +184,11 @@ When both services run in the same Docker Compose stack, use the **service name* | | `~/.local/state/open-terminal/logs` | `OPEN_TERMINAL_LOG_DIR` | Directory for process log files | | | `16` | `OPEN_TERMINAL_MAX_SESSIONS` | Maximum concurrent interactive terminal sessions | | | `true` | `OPEN_TERMINAL_ENABLE_TERMINAL` | Enable or disable the interactive terminal feature | +| | `true` | `OPEN_TERMINAL_ENABLE_NOTEBOOKS` | Enable or disable the notebook execution endpoints | | | | `OPEN_TERMINAL_API_KEY_FILE` | Load the API key from a file instead of an env var (for Docker secrets) | +| | `xterm-256color` | `OPEN_TERMINAL_TERM` | TERM environment variable set in terminal sessions (controls color support) | +| | Unset | `OPEN_TERMINAL_EXECUTE_TIMEOUT` | Default wait time (seconds) for command execution when the caller omits the `wait` parameter. Helps smaller models get output inline. | +| | Unset | `OPEN_TERMINAL_EXECUTE_DESCRIPTION` | Custom text appended to the execute endpoint's OpenAPI description, letting you tell AI models about installed tools, capabilities, or conventions. | When no API key is provided, Open Terminal generates a random key and prints it to the console on startup. @@ -167,6 +209,10 @@ api_key = "your-secret-key" log_dir = "/var/log/open-terminal" max_terminal_sessions = 16 enable_terminal = true +enable_notebooks = true +term = "xterm-256color" +execute_timeout = 5 +execute_description = "This terminal has ffmpeg and ImageMagick pre-installed." ``` Using a config file keeps the API key out of `ps` / `htop` output. @@ -295,6 +341,28 @@ You can control this feature with: - **`OPEN_TERMINAL_ENABLE_TERMINAL=false`** — disables the interactive terminal entirely. Useful if you only want command execution and file operations. - **`OPEN_TERMINAL_MAX_SESSIONS=16`** — limits the number of concurrent terminal sessions (default: 16). Prevents resource exhaustion from too many open terminals. +## Port Detection and Proxy + +Open Terminal can discover TCP ports that servers (started via the terminal or `/execute`) are listening on and proxy HTTP traffic to them. + +- **`GET /ports`** — returns a list of TCP ports listening on localhost, scoped to descendant processes of Open Terminal. Cross-platform: parses `/proc/net/tcp` on Linux, `lsof` on macOS, and `netstat` on Windows. No extra dependencies. +- **`/proxy/{port}/{path}`** — reverse-proxies HTTP requests to `localhost:{port}`. Supports all HTTP methods, forwards headers and body, and returns 502 on connection failure. + +This is useful when the AI starts a development server (e.g. a Flask app, a Node.js server, or a static file server) inside the terminal and you want to interact with it from Open WebUI or your browser. + +## Notebook Execution + +Open Terminal includes built-in Jupyter notebook execution via REST endpoints, powered by `nbclient`. Each session gets its own kernel, and you can execute cells one at a time with full rich output support (images, HTML, LaTeX). + +- **`POST /notebooks`** — create a new notebook session (starts a kernel) +- **`POST /notebooks/{id}/execute`** — execute a cell in the session +- **`GET /notebooks/{id}`** — get session status +- **`DELETE /notebooks/{id}`** — stop the kernel and clean up + +Notebook execution is **enabled by default**. Disable it with `OPEN_TERMINAL_ENABLE_NOTEBOOKS=false` (or `enable_notebooks = false` in config.toml). + +`nbclient` and `ipykernel` are core dependencies and included in the Docker image. For bare metal installs, they are included by default with `pip install open-terminal`. + ## Security - **Always use Docker in production.** Bare metal exposes your host to any command the model generates. @@ -304,6 +372,7 @@ You can control this feature with: - **Network isolation.** Place the terminal container on an internal Docker network that only Open WebUI can reach. - **Use named volumes.** Files inside the container are lost when removed. The default `docker run` command mounts a volume at `/home/user` for persistence. - **Disable the interactive terminal** if you don't need it, with `OPEN_TERMINAL_ENABLE_TERMINAL=false`. +- **Docker socket access.** Only mount the Docker socket (`/var/run/docker.sock`) in trusted environments. It gives the container full control over the host's Docker daemon. ## Further Reading diff --git a/docs/features/extensibility/plugin/development/events.mdx b/docs/features/extensibility/plugin/development/events.mdx index 8ada34dc..395e136e 100644 --- a/docs/features/extensibility/plugin/development/events.mdx +++ b/docs/features/extensibility/plugin/development/events.mdx @@ -87,6 +87,10 @@ result = await __event_call__( # result will contain the user's input value ``` +:::tip Configurable Timeout +By default, `__event_call__` waits up to **300 seconds** (5 minutes) for a user response before timing out with an exception. This timeout is configurable via the [`WEBSOCKET_EVENT_CALLER_TIMEOUT`](/reference/env-configuration#websocket_event_caller_timeout) environment variable. Increase this value if your users need more time to fill out forms, make decisions, or complete complex interactions. +::: + --- ## 📜 Event Payload Structure @@ -582,7 +586,30 @@ Because `execute` runs unsandboxed JavaScript in the user's browser session, it :::warning Pipes: Return Value vs Events For Pipes, be careful about mixing content delivery methods. If your `pipe()` method **returns a string**, that string becomes the final message content. If it **yields** (generator), the yielded chunks are streamed. If you also emit `chat:message:delta` events during execution, both the return/yield content and the event-based content are processed and can conflict. -**Recommendation**: Either use return/yield for content delivery, **or** use `chat:message:delta`/`chat:message` events, but avoid using both simultaneously. +**Recommendation**: Use return/yield as your primary content delivery mechanism. Events like `status`, `source`, `files`, and `notification` work well alongside return/yield, but avoid using `chat:message:delta` or `chat:message` events as your **sole** way to deliver message content from a pipe. + +**Why event-only content delivery is fragile for pipes**: When a pipe completes, the frontend saves the entire chat history (including all message content from its local state) to the database. This full-history save can **overwrite** content that was previously persisted by the backend event emitter. If the pipe returns `None` or an empty string and relies solely on `type: "message"` events for content, the final save may write empty content to the database — erasing what the event emitter had written. + +```python +# ❌ Fragile: relies only on events for content — can be overwritten on save +async def pipe(self, body: dict, __event_emitter__=None): + await __event_emitter__({"type": "message", "data": {"content": "Hello!"}}) + # Returns None — frontend may save empty content, overwriting the emitted content + +# ✅ Correct: return content directly, use events for supplementary data +async def pipe(self, body: dict, __event_emitter__=None): + await __event_emitter__({"type": "status", "data": {"description": "Working...", "done": False}}) + result = "Hello!" + await __event_emitter__({"type": "status", "data": {"description": "Done", "done": True}}) + return result + +# ✅ Also correct: yield for streaming, use events for supplementary data +async def pipe(self, body: dict, __event_emitter__=None): + await __event_emitter__({"type": "status", "data": {"description": "Streaming...", "done": False}}) + for chunk in ["Hello", ", ", "world", "!"]: + yield chunk + await __event_emitter__({"type": "status", "data": {"description": "Done", "done": True}}) +``` ::: --- @@ -801,6 +828,14 @@ These 6 types always write to the database inside the event emitter function its The backend event emitter only recognizes the short names above for DB writes. If you emit `"chat:message:embeds"` instead of `"embeds"`, the frontend handles it identically, but the **backend won't persist it**. Always use the short names (`"status"`, `"message"`, `"replace"`, `"embeds"`, `"files"`, `"source"`) if you need persistence. ::: +:::caution Pipes: Backend Persistence Can Be Overwritten +For **Pipes specifically**, the backend-persisted content from `"message"` and `"replace"` events can be **overwritten** by the frontend after the pipe completes. When a pipe's `pipe()` method returns, the frontend saves the entire local chat history to the database. If the pipe returned `None` or empty content and relied solely on `"message"` events, the frontend's local state may still have empty content for the assistant message — causing it to overwrite the event-emitter-written content with an empty string. + +This does **not** affect Tools, Actions, or Filters, where events supplement the return value rather than replace it. It also does not affect `"status"`, `"files"`, `"source"`, or `"embeds"` events, which update separate fields that aren't overwritten by the content save. + +**Bottom line for Pipes**: Use return/yield for message content. Use events for status updates, sources, files, embeds, and notifications. +::: + #### ❌ Not persisted (lost on tab close) | Type | Why it's lost | @@ -830,7 +865,7 @@ If your pipe or tool needs to call an LLM and have the result persist even when | `execute` via `__event_call__` | Uses `sio.call()` — waits for client response, will timeout | | `execute` via `__event_emitter__` | Fires and forgets — **will not error**, but JS may not run if no browser is connected | -`confirmation` and `input` fundamentally require a live browser connection via `__event_call__`. If the tab is closed, `sio.call()` will timeout and raise an exception in your function code. +`confirmation` and `input` fundamentally require a live browser connection via `__event_call__`. If the tab is closed, `sio.call()` will timeout and raise an exception in your function code. The timeout is configurable via the [`WEBSOCKET_EVENT_CALLER_TIMEOUT`](/reference/env-configuration#websocket_event_caller_timeout) environment variable (default: 300 seconds). `execute` is more flexible: when used via `__event_emitter__`, it fires without waiting for a response, so it won't error on tab close (though the JS won't execute if no browser is listening). This makes `__event_emitter__` the safer choice for `execute` calls where you don't need the return value — particularly for file downloads on iOS PWA, where the two-way channel can fail with `"TypeError: Load failed"`. @@ -846,6 +881,10 @@ When a pipe's `pipe()` method returns (or its generator finishes yielding), the Either way, the final assistant message is always persisted. When you reopen the chat, it will be there. +:::caution +The return value takes precedence over event-emitted content. If your pipe emits `"message"` events but returns `None`, the saved content will be empty — the frontend's final save overwrites whatever the event emitter wrote to the database. Always return or yield your content directly from the `pipe()` method. +::: + #### Tools | Return type | What happens | Persisted? | diff --git a/docs/features/extensibility/plugin/functions/action.mdx b/docs/features/extensibility/plugin/functions/action.mdx index 1121c308..66199402 100644 --- a/docs/features/extensibility/plugin/functions/action.mdx +++ b/docs/features/extensibility/plugin/functions/action.mdx @@ -128,12 +128,12 @@ actions = [ { "id": "summarize", "name": "Summarize", - "icon_url": "data:image/svg+xml;base64,..." + "icon_url": "https://example.com/icons/summarize.svg" }, { "id": "translate", "name": "Translate", - "icon_url": "data:image/svg+xml;base64,..." + "icon_url": "https://example.com/icons/translate.svg" } ] @@ -231,9 +231,20 @@ Example of supported frontmatter fields: - `author`: Name of the creator. - `version`: Version number of the Action. - `required_open_webui_version`: Minimum compatible version of Open WebUI. -- `icon_url (optional)`: URL or Base64 string for a custom icon. +- `icon_url (optional)`: A URL pointing to an icon image (PNG, SVG, JPEG, etc.). While base64 data URIs are technically supported, **using a hosted URL is strongly recommended** — see the warning below. -**Base64-Encoded Example:** +:::danger Avoid Base64 Icons — Use URLs Instead +Do **not** embed base64-encoded images as your `icon_url`. The icon data for every action is included in the `/api/models` API response, which is sent to the frontend on every page load for **every model** that has the action enabled. + +**Example of the impact:** If you use a 500 KB base64 icon for an action, and that action is enabled on 20 models, the API response grows by **20 × 500 KB = ~10 MB** — just for that one action. If you have three such actions, that becomes **~30 MB of unnecessary payload**. This will: +- **Significantly slow down frontend load times** for all users +- **Increase backend memory usage and network bandwidth** on every request +- **Degrade the overall user experience**, especially on slower connections + +Instead, host your icon as a static file (e.g., on your web server, a CDN, or a public URL) and reference it by URL. This keeps the API payload minimal. +::: + +**Example (Recommended — URL icon):**
Example @@ -244,7 +255,7 @@ title: Enhanced Message Processor author: @admin version: 1.2.0 required_open_webui_version: 0.5.0 -icon_url: data:image/svg+xml;base64,PHN2ZyB3aWR0aD0iMjQiIGhlaWdodD0iMjQiIHZpZXdCb3g9IjAgMCAyNCAyNCIgZmlsbD0ibm9uZSIgeG1sbnM9Imh0dHA6Ly93d3cudzMub3JnLzIwMDAvc3ZnIj4KPHBhdGggZD0iTTEyIDJMMTMuMDkgOC4yNkwyMCA5TDEzLjA5IDE1Ljc0TDEyIDIyTDEwLjkxIDE1Ljc0TDQgOUwxMC45MSA4LjI2TDEyIDJaIiBzdHJva2U9ImN1cnJlbnRDb2xvciIgc3Ryb2tlLXdpZHRoPSIyIiBzdHJva2UtbGluZWNhcD0icm91bmQiIHN0cm9rZS1saW5lam9pbj0icm91bmQiLz4KPHN2Zz4K +icon_url: https://example.com/icons/message-processor.svg requirements: requests,beautifulsoup4 """ diff --git a/docs/features/extensibility/plugin/functions/filter.mdx b/docs/features/extensibility/plugin/functions/filter.mdx index 6ad4540e..7284e625 100644 --- a/docs/features/extensibility/plugin/functions/filter.mdx +++ b/docs/features/extensibility/plugin/functions/filter.mdx @@ -85,8 +85,9 @@ class Filter: def __init__(self): self.valves = self.Valves() self.toggle = True # IMPORTANT: This creates a switch UI in Open WebUI - # TIP: Use SVG Data URI! - self.icon = """data:image/svg+xml;base64,PHN2ZyB4bWxucz0iaHR0cDovL3d3dy53My5vcmcvMjAwMC9zdmciIGZpbGw9Im5vbmUiIHZpZXdCb3g9IjAgMCAyNCAyNCIgc3Ryb2tlLXdpZHRoPSIxLjUiIHN0cm9rZT0iY3VycmVudENvbG9yIiBjbGFzcz0ic2l6ZS02Ij4KICA8cGF0aCBzdHJva2UtbGluZWNhcD0icm91bmQiIHN0cm9rZS1saW5lam9pbj0icm91bmQiIGQ9Ik0xMiAxOHYtNS4yNW0wIDBhNi4wMSA2LjAxIDAgMCAwIDEuNS0uMTg5bS0xLjUuMTg5YTYuMDEgNi4wMSAwIDAgMS0xLjUtLjE4OW0zLjc1IDcuNDc4YTEyLjA2IDEyLjA2IDAgMCAxLTQuNSAwbTMuNzUgMi4zODNhMTQuNDA2IDE0LjQwNiAwIDAgMS0zIDBNMTQuMjUgMTh2LS4xOTJjMC0uOTgzLjY1OC0xLjgyMyAxLjUwOC0yLjMxNmE3LjUgNy41IDAgMSAwLTcuNTE3IDBjLjg1LjQ5MyAxLjUwOSAxLjMzMyAxLjUwOSAyLjMxNlYxOCIgLz4KPC9zdmc+Cg==""" + # TIP: Use a hosted URL for your icon instead of base64 to avoid API payload bloat. + # See the Action Function docs for details on why base64 icons are not recommended. + self.icon = "https://example.com/icons/lightbulb.svg" pass async def inlet( @@ -107,7 +108,7 @@ class Filter: #### 🖼️ What’s happening? - **toggle = True** creates a switch UI in Open WebUI—users can manually enable or disable the filter in real time. -- **icon** (with a Data URI) will show up as a little image next to the filter’s name. You can use any SVG as long as it’s Data URI encoded! +- **icon** will show up as a little image next to the filter’s name. You can use a URL pointing to any image (SVG, PNG, JPEG, etc.). While base64 data URIs are technically supported, **using a hosted URL is strongly recommended** to avoid bloating the `/api/models` payload — see the [Action Function icon_url warning](/features/extensibility/plugin/functions/action#example---specifying-action-frontmatter) for details. - **The `inlet` function** uses the `__event_emitter__` special argument to broadcast feedback/status to the UI, such as a little toast/notification that reads "Toggled!" ![Toggle Filter](/images/features/plugin/functions/toggle-filter.png) @@ -292,7 +293,7 @@ class ContentModerationFilter: class WebSearchFilter: def __init__(self): self.toggle = True # User can turn on/off - self.icon = "data:image/svg+xml;base64,..." # Shows in UI + self.icon = "https://example.com/icons/web-search.svg" # Shows in UI async def inlet(self, body: dict, __event_emitter__) -> dict: # Only runs when user has enabled this filter diff --git a/docs/features/extensibility/plugin/tools/development.mdx b/docs/features/extensibility/plugin/tools/development.mdx index de5d625d..b3b90ea8 100644 --- a/docs/features/extensibility/plugin/tools/development.mdx +++ b/docs/features/extensibility/plugin/tools/development.mdx @@ -61,7 +61,7 @@ Valves and UserValves are used for specifying customizable settings of the Tool, ### Optional Arguments Below is a list of optional arguments your tools can depend on: - `__event_emitter__`: Emit events (see following section) -- `__event_call__`: Same as event emitter but can be used for user interactions +- `__event_call__`: Same as event emitter but can be used for user interactions. The server-side timeout for event calls is configurable via [`WEBSOCKET_EVENT_CALLER_TIMEOUT`](/reference/env-configuration#websocket_event_caller_timeout) (default: 300s). - `__user__`: A dictionary with user information. It also contains the `UserValves` object in `__user__["valves"]`. - `__metadata__`: Dictionary with chat metadata - `__messages__`: List of previous messages diff --git a/docs/reference/env-configuration.mdx b/docs/reference/env-configuration.mdx index 886b738b..3d5441e7 100644 --- a/docs/reference/env-configuration.mdx +++ b/docs/reference/env-configuration.mdx @@ -4802,7 +4802,7 @@ You must also set `OPENID_PROVIDER_URL` or otherwise logout may not work. - Type: `str` - Default: `openid email profile` -- Description: Sets the scope for Microsoft OAuth authentication. +- Description: Sets the scope for Microsoft OAuth authentication. This scope is also included in refresh token requests when `OAUTH_REFRESH_TOKEN_INCLUDE_SCOPE` is enabled, which is required by Azure AD to avoid `AADSTS90009` errors. If you use custom API scopes, include them here (e.g., `openid email profile offline_access api:///`). - Persistence: This environment variable is a `PersistentConfig` variable. #### `MICROSOFT_REDIRECT_URI` @@ -5073,6 +5073,19 @@ This is useful when you need a JWT access token for downstream validation or whe ::: +#### `OAUTH_REFRESH_TOKEN_INCLUDE_SCOPE` + +- Type: `bool` +- Default: `False` +- Description: When enabled, includes the configured OAuth scope in refresh token requests. Some OAuth providers, notably Microsoft Azure AD, require the scope to be explicitly provided when refreshing a token. Without it, Azure AD treats the request as the application requesting a token for itself, resulting in `AADSTS90009` errors. Enable this if you encounter token refresh failures with your OAuth provider. +- Persistence: This environment variable is a `PersistentConfig` variable. + +:::info + +This setting is compliant with [RFC 6749 Section 6](https://datatracker.ietf.org/doc/html/rfc6749#section-6), where the `scope` parameter is optional during refresh token requests. Only enable this for providers that require it, such as certain Azure AD configurations. + +::: + ## LDAP #### `ENABLE_LDAP` @@ -6376,6 +6389,12 @@ When this variable is left empty (default), `REDIS_SOCKET_CONNECT_TIMEOUT` is au - Default: `25` - Description: The frequency for a ping to Redis in seconds. +#### `WEBSOCKET_EVENT_CALLER_TIMEOUT` + +- Type: `int` +- Default: `300` +- Description: Sets the timeout in seconds for the `sio.call()` used in `event_call` events. This controls how long the server waits for a user to respond to interactive prompts, forms, or dropdowns generated by tools and actions using `event_call` (e.g., `execute()` type events). Increase this value if users need more time to fill out forms or make decisions presented by event calls. + #### `ENABLE_STAR_SESSIONS_MIDDLEWARE` - Type: `bool` diff --git a/docs/troubleshooting/performance.md b/docs/troubleshooting/performance.md index 5b7cc77b..cc5ecb31 100644 --- a/docs/troubleshooting/performance.md +++ b/docs/troubleshooting/performance.md @@ -430,6 +430,7 @@ These are real-world mistakes that cause organizations to massively over-provisi | **Scaling replicas to mask memory leaks** | Leaky processes → OOM kills → auto-scaler adds more pods → more Redis connections → Redis overwhelmed | Fix the leaks first (content extraction, embedding engine), then right-size | | **Using Default (prompt-based) tool calling** | Injected prompts may break KV cache → higher latency → more resources needed per request | Switch to Native Mode for all capable models | | **Not configuring Redis stale connection timeout** | Connections accumulate forever → Redis OOM → you deploy Redis Cluster | Add `timeout 1800` to redis.conf | +| **Using base64-encoded icons in Actions/Filters** | Icon data is embedded in `/api/models` responses sent to the frontend on every page load for every model. A 500 KB base64 icon on 3 actions across 20 models = **30 MB of payload bloat** per request → slow frontend loads, high bandwidth usage, unnecessary backend memory pressure | Host icons as static files and reference them by URL in `icon_url` / `self.icon`. See [Action Function icon_url warning](/features/extensibility/plugin/functions/action#example---specifying-action-frontmatter) | --- diff --git a/docs/tutorials/integrations/auth-identity/dual-oauth-configuration.mdx b/docs/tutorials/integrations/auth-identity/dual-oauth-configuration.mdx index e7ce2eea..6727d2b6 100644 --- a/docs/tutorials/integrations/auth-identity/dual-oauth-configuration.mdx +++ b/docs/tutorials/integrations/auth-identity/dual-oauth-configuration.mdx @@ -44,6 +44,12 @@ MICROSOFT_CLIENT_TENANT_ID=your_tenant_id MICROSOFT_REDIRECT_URI=https://your-webui.com/oauth/microsoft/callback OPENID_PROVIDER_URL=https://login.microsoftonline.com/your_tenant_id/v2.0/.well-known/openid-configuration +# Optional: Custom scope for Microsoft OAuth (required if using custom API scopes) +# MICROSOFT_OAUTH_SCOPE=openid email profile offline_access api:/// + +# Optional: Include scope in refresh token requests (required for some Azure AD configurations) +# OAUTH_REFRESH_TOKEN_INCLUDE_SCOPE=true + # 2. Google as a secondary OAuth provider # Note: Do NOT provide an OPENID_PROVIDER_URL for Google. # The system will use its internal Google OAuth implementation. @@ -62,3 +68,4 @@ GOOGLE_CLIENT_SECRET=your_google_client_secret - **Redirect Mismatch**: Ensure your Redirect URIs in both consoles match your `WEBUI_URL`. - **Merge Failures**: Double-check that `OAUTH_MERGE_ACCOUNTS_BY_EMAIL` is set to `true`. - **Microsoft Logout**: Microsoft often requires the `OPENID_PROVIDER_URL` to handle the logout redirect correctly. If logout fails, ensure this URL is correct for your tenant. +- **Azure AD Refresh Token Failures (`AADSTS90009`)**: If token refresh fails with the error "Application is requesting a token for itself", set `OAUTH_REFRESH_TOKEN_INCLUDE_SCOPE=true`. Azure AD requires the scope to be explicitly included in refresh token requests. You may also need to set `MICROSOFT_OAUTH_SCOPE` to include `offline_access` and any custom API scopes.