📘 feat: Update Agents, OCR, and MCP Docs with new Features (#336)
* 📚 docs: Update OCR documentation for clarity and structure * 📚 docs: Add Model Context Protocol (MCP) documentation and update features * 📚 docs: Add Model Context Protocol (MCP) section to features documentation * 📚 docs: Enhance MCP documentation with OAuth authentication details and configuration examples * 📚 docs: Update MCP documentation with improved configuration examples and OAuth authentication details * 📚 docs: Update dotenv documentation to reflect new index header constants * 📚 docs: Revise MCP integration documentation for Agents with clearer instructions and new images - Updated the steps for adding MCP tools to agents for better clarity and organization. - Introduced new images to illustrate the MCP tools interface. - Removed outdated image and improved the overall structure of the documentation. * 📚 docs: Enhance MCP documentation with custom user variables support --------- Co-authored-by: Aleš Kůtek <kutekales@gmail.com>
@@ -10,4 +10,9 @@
|
||||
- Added additional user field placeholders to [MCP Servers Object Structure](/docs/configuration/librechat_yaml/object_structure/mcp_servers):
|
||||
- `{{LIBRECHAT_USER_ID}}`: Will be replaced with the current user's ID, enabling multi-user support.
|
||||
- `{{LIBRECHAT_USER_*}}`: Dynamic user field placeholders. Replace `*` with the UPPERCASE version of any allowed field.
|
||||
- See: **[`headers`](/docs/configuration/librechat_yaml/object_structure/mcp_servers#headers)** and **[`url`](/docs/configuration/librechat_yaml/object_structure/mcp_servers#url)** fields, which both support these placeholders.
|
||||
- See: **[`headers`](/docs/configuration/librechat_yaml/object_structure/mcp_servers#headers)** and **[`url`](/docs/configuration/librechat_yaml/object_structure/mcp_servers#url)** fields, which both support these placeholders.
|
||||
- Added `customUserVars` to MCP Servers for per-user credentials:
|
||||
- Allows users to securely provide their own API keys and credentials for MCP servers
|
||||
- Configurable through UI via settings icon in tool selection or MCP Settings panel
|
||||
- Ensures authentication details remain private in multi-user environments
|
||||
- See: [MCP Servers Object Structure - customUserVars](/docs/configuration/librechat_yaml/object_structure/mcp_servers#customuservars)
|
||||
@@ -98,9 +98,9 @@ Sets the [Cache-Control](https://developer.mozilla.org/en-US/docs/Web/HTTP/Heade
|
||||
|
||||
<OptionTable
|
||||
options={[
|
||||
['INDEX_HTML_CACHE_CONTROL', 'string', 'Cache-Control header for index.html','INDEX_HTML_CACHE_CONTROL=no-cache, no-store, must-revalidate'],
|
||||
['INDEX_HTML_PRAGMA', 'string', 'Pragma header for index.html','INDEX_HTML_PRAGMA=no-cache'],
|
||||
['INDEX_HTML_EXPIRES', 'string', 'Expires header for index.html','INDEX_HTML_EXPIRES=0'],
|
||||
['INDEX_CACHE_CONTROL', 'string', 'Cache-Control header for index.html','INDEX_CACHE_CONTROL=no-cache, no-store, must-revalidate'],
|
||||
['INDEX_PRAGMA', 'string', 'Pragma header for index.html','INDEX_PRAGMA=no-cache'],
|
||||
['INDEX_EXPIRES', 'string', 'Expires header for index.html','INDEX_EXPIRES=0'],
|
||||
]}
|
||||
/>
|
||||
|
||||
|
||||
@@ -401,6 +401,7 @@ see: [Model Specs Object Structure](/docs/configuration/librechat_yaml/object_st
|
||||
- `timeout`: Timeout in milliseconds for MCP server requests. Determines how long to wait for a response for tool requests.
|
||||
- `initTimeout`: Timeout in milliseconds for MCP server initialization. Determines how long to wait for the server to initialize.
|
||||
- `serverInstructions`: Controls whether server instructions are included in agent context. Can be `true` (use server-provided), `false` (disabled), or a custom string (overrides server-provided).
|
||||
- `customUserVars`: (Optional) Defines custom variables (e.g., API keys, URLs) that individual users can set for an MCP server. These per-user values, provided through the UI, can then be referenced in the server's `headers` or `env` configurations using `{{VARIABLE_NAME}}` syntax. This allows for per-user authentication or customization for MCP tools.
|
||||
- see: [MCP Servers Object Structure](/docs/configuration/librechat_yaml/object_structure/mcp_servers)
|
||||
|
||||
**Example:**
|
||||
@@ -442,6 +443,16 @@ mcpServers:
|
||||
headers:
|
||||
Authorization: "Bearer ${API_TOKEN}"
|
||||
timeout: 30000
|
||||
per-user-crendentials-example:
|
||||
type: sse
|
||||
url: "https//some.mcp/sse"
|
||||
headers:
|
||||
X-Custom-Auth-Token: "{{USER_API_KEY}}" # Placeholder for the user-provided API key, defined in `customUserVars` below.
|
||||
customUserVars:
|
||||
USER_API_KEY:
|
||||
title: "Service API Key"
|
||||
description: "Your personal API key for this service. You can get it <a href='https://example.com/api-keys' target='_blank'>here</a>."
|
||||
serverInstructions: true
|
||||
```
|
||||
|
||||
see: [MCP Servers Object Structure](/docs/configuration/librechat_yaml/object_structure/mcp_servers)
|
||||
|
||||
@@ -39,6 +39,15 @@ mcpServers:
|
||||
streamable-http-server:
|
||||
type: streamable-http
|
||||
url: https://example.com/api/
|
||||
per-user-credentials-example:
|
||||
type: streamable-http
|
||||
url: "https://example.com/api/"
|
||||
headers:
|
||||
X-Auth-Token: "{{MY_SERVICE_API_KEY}}"
|
||||
customUserVars:
|
||||
MY_SERVICE_API_KEY:
|
||||
title: "My Service API Key"
|
||||
description: "Enter your personal API key for the service. You can generate one at <a href='https://myservice.example.com/developer/keys' target='_blank'>Service Developer Portal</a>."
|
||||
```
|
||||
|
||||
## `<serverName>`
|
||||
@@ -66,6 +75,7 @@ mcpServers:
|
||||
['initTimeout', 'Number', '(Optional) Timeout in milliseconds for MCP server initialization. Determines how long to wait for the server to initialize.', 'initTimeout: 10000'],
|
||||
['env', 'Object', '(Optional, `stdio` type only) Environment variables to use when spawning the process.', 'env:\n NODE_ENV: "production"'],
|
||||
['stderr', 'String or Stream or Number', '(Optional, `stdio` type only) How to handle `stderr` of the child process. Defaults to `"inherit"`.', 'stderr: "inherit"'],
|
||||
['customUserVars', 'Object', '(Optional) Defines custom variables that users can set for this MCP server, allowing for per-user credentials or configurations (e.g., API keys). These variables can then be referenced in `headers` or `env` fields.', 'customUserVars:\n API_KEY:\n title: "API Key"\n description: "Your personal API key."'],
|
||||
]}
|
||||
/>
|
||||
|
||||
@@ -97,11 +107,12 @@ mcpServers:
|
||||
#### `headers`
|
||||
|
||||
- **Type:** Object (Optional, for `sse` and `streamable-http` types)
|
||||
- **Description:** Custom headers to send with the request.
|
||||
- **Special Values:**
|
||||
- **Description:** Custom headers to send with the request. Supports various placeholder types for dynamic value substitution.
|
||||
- **Placeholder Support:**
|
||||
- `{{LIBRECHAT_USER_ID}}`: Will be replaced with the current user's ID, enabling multi-user support.
|
||||
- `{{LIBRECHAT_USER_*}}`: Dynamic user field placeholders. Replace `*` with the UPPERCASE version of any allowed field.
|
||||
- `${ENV_VAR}`: Will be replaced with the value of the environment variable `ENV_VAR`.
|
||||
- `{{CUSTOM_VARIABLE_NAME}}`: Replaced with the value provided by the user for a variable defined in `customUserVars` (e.g., `{{MY_API_KEY}}`).
|
||||
- `${ENV_VAR}`: Will be replaced with the value of the environment variable `{{ENV_VAR}}`.
|
||||
|
||||
**Available User Field Placeholders:**
|
||||
|
||||
@@ -178,7 +189,12 @@ mcpServers:
|
||||
#### `env`
|
||||
|
||||
- **Type:** Object (Optional, `stdio` type only)
|
||||
- **Description:** Environment variables to use when spawning the process. Supports the same dynamic user field placeholders as headers (`{{LIBRECHAT_USER_*}}`) and environment variable substitution (`${ENV_VAR}`).
|
||||
- **Description:** Environment variables to use when spawning the process.
|
||||
- **Placeholder Support:**
|
||||
- `{{LIBRECHAT_USER_ID}}`: Replaced with the current user's ID.
|
||||
- `{{LIBRECHAT_USER_*}}`: Dynamic user field placeholders (e.g., `{{LIBRECHAT_USER_EMAIL}}`).
|
||||
- `{{CUSTOM_VARIABLE_NAME}}`: Replaced with the value provided by the user for a variable defined in `customUserVars` (e.g., `{{MY_API_KEY}}`).
|
||||
- `${ENV_VAR}`: Replaced with the value of the server-side environment variable `{{ENV_VAR}}`.
|
||||
|
||||
#### `timeout`
|
||||
|
||||
@@ -198,6 +214,41 @@ mcpServers:
|
||||
- **Description:** How to handle `stderr` of the child process. This matches the semantics of Node's [`child_process.spawn`](https://nodejs.org/api/child_process.html#child_processspawncommand-args-options).
|
||||
- **Default Value:** `"inherit"` (messages to `stderr` will be printed to the parent process's `stderr`).
|
||||
|
||||
#### `customUserVars`
|
||||
|
||||
- **Type:** Object (Optional)
|
||||
- **Description:** Defines custom variables that users can set for this MCP server. This allows administrators to specify variables (e.g., API keys, URLs) that each user must configure individually. These user-provided values can then be used in `headers` configuration.
|
||||
- **Structure:**
|
||||
- The `customUserVars` object contains keys, where each key represents a variable name (e.g., `MY_API_KEY`). This name will be used in placeholders like `{{MY_API_KEY}}`.
|
||||
- Each variable name is an object with the following subkeys:
|
||||
- `title`: String (Required) - A user-friendly title for the variable, displayed in the configuration UI.
|
||||
- `description`: String (Optional) - A description or instructions for the variable, also displayed in the UI to guide the user. HTML can be used in this field (e.g., to create a link: `<a href="https://example.com" target="_blank">More info</a>`).
|
||||
- **Usage in `headers` and `env`:**
|
||||
- Once defined under `customUserVars`, these variables can be referenced in the `headers` (for `sse` and `streamable-http` types) or `env` (for `stdio` type) sections using the `{{VARIABLE_NAME}}` syntax.
|
||||
- Users provide these values through the UI. These settings can be accessed in two ways:
|
||||
- **From Assistant Chat Input**: When selecting MCP tools for an assistant, a settings icon will appear next to configurable MCP servers in the tool selection dropdown. Clicking this icon opens a dialog to manage credentials for that server.
|
||||
<img src="/images/mcp/mcp_per_user_vars_assistant.png" alt="MCP Per-User Variables Configuration - Assistant Access" width={600} />
|
||||
<img src="/images/mcp/mcp_per_user_vars_assistant_dialog.png" alt="MCP Per-User Variables Configuration - Assistant Access Dialog" width={800} />
|
||||
- **From Settings Panel**: A dedicated "MCP Settings" section in the right panel lists all MCP servers with definable custom variables. Users can click on a server to open the configuration dialog to set or update their credentials for that specific MCP server.
|
||||
<img src="/images/mcp/mcp_per_user_vars_side_panel.png" alt="MCP Per-User Variables Configuration - Settings Panel Access" width={300} />
|
||||
- These user-provided values are stored securely, associated with the individual user and the specific MCP server, and substituted at runtime.
|
||||
- **Example:**
|
||||
```yaml
|
||||
customUserVars:
|
||||
MY_SERVICE_API_KEY:
|
||||
title: "My Service API Key"
|
||||
description: "Your personal API access key for My Service. Find it at <a href='https://myservice.example.com/settings/api' target='_blank'>My Service API Settings</a>."
|
||||
SOME_OTHER_VAR:
|
||||
title: "Some Other Variable"
|
||||
description: "The specific value for some other configuration (e.g., a specific path or identifier)."
|
||||
```
|
||||
Usage in `headers`:
|
||||
```yaml
|
||||
headers:
|
||||
X-Auth-Token: "{{MY_SERVICE_API_KEY}}"
|
||||
X-Some-Other-Config: "{{SOME_OTHER_VAR}}"
|
||||
```
|
||||
|
||||
### Notes
|
||||
|
||||
- **Type Inference:**
|
||||
@@ -273,6 +324,20 @@ user-aware-server:
|
||||
Authorization: "Bearer ${API_TOKEN}"
|
||||
```
|
||||
|
||||
### MCP Server with Per-User Credentials via `customUserVars`
|
||||
|
||||
```yaml filename="MCP Server with customUserVars"
|
||||
my-mcp-server:
|
||||
type: streamable-http
|
||||
url: "https://api.example-service.com/api/" # Example URL
|
||||
headers:
|
||||
X-Auth-Token: "{{API_KEY}}" # Uses the API_KEY defined below
|
||||
customUserVars:
|
||||
API_KEY: # This key will be used as {{API_KEY}} in headers/url
|
||||
title: "API Key" # This is the label shown above the input field
|
||||
description: "Get your API key <a href='https://example.com/api-keys' target='_blank'>here</a>." # This description appears below the input
|
||||
```
|
||||
|
||||
### MCP Server with Custom Icon
|
||||
|
||||
```yaml filename="MCP Server with Icon"
|
||||
|
||||
@@ -112,6 +112,6 @@ ocr:
|
||||
|
||||
**Available Strategies:**
|
||||
|
||||
- `mistral_ocr`: Uses Mistral's OCRcapabilities via the standard [Mistral API](/docs/features/ocr#mistral-ocr) .
|
||||
- `azure_mistral_ocr`: Uses Mistral OCR models deployed on [Azure AI Foundry](/docs/features/ocr#2-azure-deployed-mistral-ocr).
|
||||
- `mistral_ocr`: Uses Mistral's OCRcapabilities via the standard [Mistral API](/docs/features/ocr#1-mistral-ocr-default) .
|
||||
- `azure_mistral_ocr`: Uses Mistral OCR models deployed on [Azure AI Foundry](/docs/features/ocr#2-azure-mistral-ocr).
|
||||
- `custom_ocr`: Uses a custom OCR service specified by the `baseURL` [(not yet implemented)](/docs/features/ocr#future-enhancements).
|
||||
@@ -1,5 +1,6 @@
|
||||
export default {
|
||||
index: 'Overview',
|
||||
mcp: 'MCP',
|
||||
agents: 'Agents',
|
||||
code_interpreter: 'Code Interpreter API',
|
||||
artifacts: 'Artifacts - Generative UI',
|
||||
|
||||
@@ -71,6 +71,41 @@ The File Context (OCR) capability allows your agent to extract and process text
|
||||
- [More info about OCR](/docs/features/ocr)
|
||||
- **Currently uses Mistral OCR API which may incur costs**
|
||||
|
||||
### Model Context Protocol (MCP)
|
||||
|
||||
MCP is an open protocol that standardizes how applications provide context to Large Language Models (LLMs), acting like a universal adapter for AI tools and data sources.
|
||||
|
||||
For more information, see documentation on [MCP](/docs/features/mcp).
|
||||
|
||||
#### Agents with MCP Tools
|
||||
|
||||
|
||||
1. Configure MCP servers in your `librechat.yaml` file
|
||||
2. Restart LibreChat to initialize the connections
|
||||
3. Create or edit an agent
|
||||
4. Click the "Add Tools" button in the Agent Builder panel to open the Tools Dialog shown below
|
||||
5. Select the MCP server(s) you want to add - each server appears as a single entry
|
||||
6. Save your changes to the agent
|
||||
|
||||
In this example, we've added the **[Spotify MCP server](https://github.com/LibreChat-AI/spotify-mcp)** to an agent.
|
||||
|
||||

|
||||
|
||||
#### Managing MCP Tools
|
||||
|
||||
Once an MCP server is added to an agent, you can fine-tune which specific tools are available:
|
||||
|
||||
- After adding an MCP server, expand it to see all available tools
|
||||
- Check/uncheck individual tools as needed
|
||||
- For example, the Spotify MCP server provides ~20 tools (search, playback control, playlist management, etc.)
|
||||
- This granular control lets you limit agents to only the tools they need
|
||||
|
||||

|
||||
|
||||
Learn More:
|
||||
- [Configuring MCP Servers](/docs/features/mcp)
|
||||
- [Model Context Protocol Introduction](https://modelcontextprotocol.io/introduction)
|
||||
|
||||
### Artifacts
|
||||
|
||||
The Artifacts capability enables your agent to generate and display interactive content:
|
||||
@@ -174,94 +209,6 @@ A single, non-tool response is 1 step. A singular round of tool usage is usually
|
||||
|
||||
1. API Request -> 2. Tool Usage (1 or many tools) -> 3. Follow-up API Request
|
||||
|
||||
### Model Context Protocol (MCP)
|
||||
|
||||
MCP is an open protocol that standardizes how applications provide context to Large Language Models (LLMs), acting like a universal adapter for AI tools and data sources.
|
||||
|
||||
Imagine MCP as the "USB-C of AI" - just as USB-C provides a universal connection standard for electronic devices, MCP offers a standardized way to connect AI models to diverse tools, data sources, and services.
|
||||
|
||||
By configuring `mcpServers` in the `librechat.yaml` file, you can:
|
||||
- Add custom tools from various sources
|
||||
- Integrate specialized data access tools
|
||||
- Extend AI capabilities beyond default offerings
|
||||
|
||||
Here are some tools added by configuring the ["Filesystem" MCP server](https://github.com/modelcontextprotocol/servers/tree/main/src/filesystem):
|
||||

|
||||
|
||||
Key Features:
|
||||
- Upon app restart, tools are available in the "Add Tools" button within the Agent Builder panel
|
||||
- Seamlessly connect custom tools and data sources
|
||||
- Leverage a growing ecosystem of MCP-compatible servers and integrations
|
||||
|
||||
Learn More:
|
||||
- [Configuring MCP Servers](/docs/configuration/librechat_yaml/object_structure/mcp_servers)
|
||||
- [Model Context Protocol Introduction](https://modelcontextprotocol.io/introduction)
|
||||
|
||||
#### MCP Server Types and Multi-User Support
|
||||
|
||||
MCP servers can be configured to use different transport mechanisms:
|
||||
|
||||
**STDIO Servers**
|
||||
- Work well for local, single-user environments
|
||||
- Not scalable for remote or cloud deployments
|
||||
- Limited multi-user support
|
||||
|
||||
**Server-Sent Events (SSE) Servers**
|
||||
- More scalable transport mechanism for multi-user environments
|
||||
- Enhanced multi-user support with user-specific connections
|
||||
|
||||
**Streamable HTTP Servers**
|
||||
- Uses HTTP POST for sending messages and supports streaming responses
|
||||
- Operates as an independent process that can handle multiple client connections
|
||||
- Supports both basic requests and streaming via Server-Sent Events (SSE)
|
||||
- More performant alternative to the legacy HTTP+SSE transport
|
||||
- Supports proper multi-user server configurations
|
||||
|
||||
**Multi-User Features**
|
||||
- User-level and app-level connections with proper establishment and cleanup
|
||||
- MCP Servers can distinguish connections by session ID
|
||||
- Dynamic user field placeholders in headers, URLs, and environment variables:
|
||||
- `{{LIBRECHAT_USER_ID}}` - User's unique identifier
|
||||
- Basic fields: `{{LIBRECHAT_USER_EMAIL}}`, `{{LIBRECHAT_USER_USERNAME}}`, `{{LIBRECHAT_USER_NAME}}`, `{{LIBRECHAT_USER_ROLE}}`, `{{LIBRECHAT_USER_PROVIDER}}`
|
||||
- Provider IDs: `{{LIBRECHAT_USER_GOOGLEID}}`, `{{LIBRECHAT_USER_OPENIDID}}`, `{{LIBRECHAT_USER_SAMLID}}`, `{{LIBRECHAT_USER_LDAPID}}`, etc.
|
||||
- Boolean fields: `{{LIBRECHAT_USER_EMAILVERIFIED}}`, `{{LIBRECHAT_USER_TWOFACTORENABLED}}`, `{{LIBRECHAT_USER_TERMSACCEPTED}}`
|
||||
- See [MCP Servers documentation](/docs/configuration/librechat_yaml/object_structure/mcp_servers#headers) for complete field list
|
||||
- Environment variable support: `${SOME_API_KEY}`
|
||||
- User idle timeout management for efficient resource usage
|
||||
|
||||
**Example Configuration:**
|
||||
```yaml
|
||||
mcpServers:
|
||||
googlesheets:
|
||||
type: sse
|
||||
url: https://mcp.composio.dev/googlesheets/some-endpoint
|
||||
headers:
|
||||
X-User-ID: "{{LIBRECHAT_USER_ID}}"
|
||||
X-User-Email: "{{LIBRECHAT_USER_EMAIL}}"
|
||||
X-API-Key: "${SOME_API_KEY}"
|
||||
|
||||
streamable-http-example:
|
||||
type: streamable-http
|
||||
url: https://example.com/mcp/users/{{LIBRECHAT_USER_USERNAME}}
|
||||
headers:
|
||||
X-User-ID: "{{LIBRECHAT_USER_ID}}"
|
||||
X-User-Role: "{{LIBRECHAT_USER_ROLE}}"
|
||||
Authorization: "Bearer ${API_TOKEN}"
|
||||
|
||||
user-context-server:
|
||||
type: stdio
|
||||
command: node
|
||||
args: ["server.js"]
|
||||
env:
|
||||
USER_EMAIL: "{{LIBRECHAT_USER_EMAIL}}"
|
||||
USER_PROVIDER: "{{LIBRECHAT_USER_PROVIDER}}"
|
||||
API_KEY: "${MCP_API_KEY}"
|
||||
```
|
||||
|
||||
LibreChat is at the forefront of implementing flexible, scalable MCP server integrations to support diverse usage scenarios.
|
||||
|
||||
For detailed configuration options, see [Configuring MCP Servers](/docs/configuration/librechat_yaml/object_structure/mcp_servers).
|
||||
|
||||
## File Management
|
||||
|
||||
Agents support four distinct file upload categories:
|
||||
|
||||
@@ -45,6 +45,14 @@ import Image from 'next/image'
|
||||
- **Configuration Required**: Must be explicitly configured in `librechat.yaml` to work
|
||||
- **[Learn More →](/docs/features/memory)**
|
||||
|
||||
### 🔌 [Model Context Protocol (MCP)](/docs/features/mcp)
|
||||
|
||||
- **Universal AI Tool Integration**: Connect AI models to any tool or service through a standardized protocol
|
||||
- **Production-Ready Transports**: Support for scalable HTTP-based servers ideal for multi-user deployments
|
||||
- **Dynamic User Context**: Pass user-specific information to MCP servers for personalized experiences
|
||||
- **Flexible Usage**: Use MCP tools directly in chat or integrate them with custom agents
|
||||
- **[Learn More →](/docs/features/mcp)**
|
||||
|
||||
### 🪄 **[Artifacts](/docs/features/artifacts)**
|
||||
|
||||
- **Generative UI:** React, HTML, Mermaid diagrams
|
||||
|
||||
297
pages/docs/features/mcp.mdx
Normal file
@@ -0,0 +1,297 @@
|
||||
---
|
||||
title: Model Context Protocol (MCP) - Extend LibreChat with Universal AI Tools
|
||||
description: Learn how LibreChat leverages Model Context Protocol (MCP) to provide seamless integration with external tools, data sources, and specialized services.
|
||||
---
|
||||
|
||||
# Model Context Protocol (MCP)
|
||||
|
||||
Model Context Protocol (MCP) is an open protocol that standardizes how applications provide context to Large Language Models (LLMs). Think of MCP as the **"USB-C of AI"** - just as USB-C provides a universal connection standard for electronic devices, MCP offers a standardized way to connect AI models to diverse tools, data sources, and services.
|
||||
|
||||
LibreChat leverages MCP to dramatically expand what your AI agents can do, allowing you to integrate everything from file system access, web browsers, specialized APIs, to custom business tools.
|
||||
|
||||
## Why MCP Matters
|
||||
|
||||
LLMs are limited to their built-in capabilities. With MCP, LibreChat breaks down these walls by:
|
||||
|
||||
- **Connecting to any tool or service** that provides an MCP server
|
||||
- **Standardizing integrations** so you don't need to edit LibreChat's code for each tool
|
||||
- **Supporting multi-user environments** with proper authentication and isolation
|
||||
- **Providing a growing ecosystem** of dynamic, ready-to-use integrations
|
||||
|
||||
## How MCP Works in LibreChat
|
||||
|
||||
LibreChat provides two ways to use MCP servers.
|
||||
|
||||
1. First, configure MCP servers in your `librechat.yaml` file ([see example below](#basic-configuration)).
|
||||
2. Restart LibreChat to initialize the connections.
|
||||
|
||||
### In Chat Area
|
||||
|
||||

|
||||
|
||||
LibreChat displays configured MCP servers directly in the chat area when using traditional endpoints (OpenAI, Anthropic, Google, Bedrock, etc.):
|
||||
|
||||
- Select any non-agent endpoint first, and a tool-compatible model
|
||||
- MCP servers appear in a dropdown in the chat interface below your text input
|
||||
- When selected, all tools from that server become available to your current model
|
||||
- Quick access to MCP tools without creating an agent, allowing multiple servers to be used at once
|
||||
|
||||
To disable MCP servers from appearing in the chat dropdown (keeping them agent-only), set [`chatMenu: false`](/docs/configuration/librechat_yaml/object_structure/mcp_servers#chatmenu) in your configuration:
|
||||
|
||||
```yaml
|
||||
mcpServers:
|
||||
internal-tools:
|
||||
command: npx
|
||||
args: ["-y", "internal-mcp-server"]
|
||||
chatMenu: false # Only available in agent builder
|
||||
```
|
||||
|
||||
### With Agents
|
||||
|
||||
MCP servers integrate seamlessly with LibreChat Agents:
|
||||
|
||||
1. Create or edit an agent
|
||||
2. Click "Add Tools" to open the Tools Dialog from the Agent Builder panel
|
||||
3. Select MCP servers once added, each appears as a single entry
|
||||
4. Fine-tune by enabling/disabling individual tools after adding
|
||||
5. Save your agent
|
||||
|
||||

|
||||
|
||||
This higher-level organization keeps the interface manageable - even servers with 20+ tools (like Spotify) appear as single entries that can be expanded for granular control.
|
||||
|
||||
### Basic Configuration
|
||||
|
||||
Add MCP servers to your `librechat.yaml` file:
|
||||
|
||||
```yaml
|
||||
mcpServers:
|
||||
# File system access
|
||||
filesystem:
|
||||
command: npx
|
||||
args:
|
||||
- -y
|
||||
- "@modelcontextprotocol/server-filesystem"
|
||||
- /path/to/your/documents
|
||||
|
||||
# Web browser automation
|
||||
puppeteer:
|
||||
command: npx
|
||||
args:
|
||||
- -y
|
||||
- "@modelcontextprotocol/server-puppeteer"
|
||||
|
||||
# Production-ready cloud service
|
||||
business-api:
|
||||
type: streamable-http
|
||||
url: https://api.yourbusiness.com/mcp
|
||||
headers:
|
||||
X-User-ID: "{{LIBRECHAT_USER_ID}}"
|
||||
Authorization: "Bearer ${API_TOKEN}"
|
||||
timeout: 30000
|
||||
serverInstructions: true
|
||||
```
|
||||
|
||||
For detailed configuration options and examples, see:
|
||||
- [MCP Servers Configuration](/docs/configuration/librechat_yaml/object_structure/mcp_servers)
|
||||
- [Agent Configuration](/docs/configuration/librechat_yaml/object_structure/agents)
|
||||
- [Advanced Agent Features](/docs/features/agents#model-context-protocol-mcp)
|
||||
|
||||
## LibreChat-Specific Features
|
||||
|
||||
LibreChat's MCP implementation is designed for highly configurable, real-world, multi-user environments.
|
||||
|
||||
### User-Specific Connections
|
||||
- Each user gets their own isolated connection to MCP servers
|
||||
- User authentication and permissions are respected
|
||||
- Personal data and context remain private
|
||||
|
||||
### Dynamic User Context
|
||||
MCP servers can access user information through placeholders in **URLs and headers** (for SSE and Streamable HTTP transports):
|
||||
|
||||
```yaml
|
||||
mcpServers:
|
||||
user-api:
|
||||
type: streamable-http
|
||||
url: https://api.example.com/users/{{LIBRECHAT_USER_USERNAME}}/mcp
|
||||
headers:
|
||||
X-User-ID: "{{LIBRECHAT_USER_ID}}"
|
||||
X-User-Email: "{{LIBRECHAT_USER_EMAIL}}"
|
||||
X-User-Role: "{{LIBRECHAT_USER_ROLE}}"
|
||||
Authorization: "Bearer ${API_TOKEN}"
|
||||
```
|
||||
|
||||
Available placeholders include:
|
||||
- `{{LIBRECHAT_USER_ID}}` - Unique user identifier
|
||||
- `{{LIBRECHAT_USER_EMAIL}}` - User's email address
|
||||
- `{{LIBRECHAT_USER_ROLE}}` - User role (admin, user, etc.)
|
||||
- `{{LIBRECHAT_USER_USERNAME}}` - Username
|
||||
- And many more (see [MCP Servers Configuration](/docs/configuration/librechat_yaml/object_structure/mcp_servers#headers) for complete list)
|
||||
|
||||
### Server Instructions
|
||||
|
||||
`serverInstructions` is a LibreChat feature that dynamically adds configured instructions when any tool from that MCP server is selected:
|
||||
|
||||
```yaml
|
||||
mcpServers:
|
||||
filesystem:
|
||||
command: npx
|
||||
args: ["-y", "@modelcontextprotocol/server-filesystem", "/docs"]
|
||||
serverInstructions: |
|
||||
When accessing files:
|
||||
- Always check file permissions first
|
||||
- Use absolute paths for reliability
|
||||
- Handle errors gracefully
|
||||
```
|
||||
|
||||
Options:
|
||||
- `true`: Use server-provided instructions
|
||||
- `false`: Disable instructions
|
||||
- `string`: Custom instructions (shown above)
|
||||
|
||||
### Timeout Configuration
|
||||
|
||||
For long-running MCP operations, configure appropriate timeouts for both initialization and tool operations.
|
||||
|
||||
```yaml
|
||||
mcpServers:
|
||||
data-processor:
|
||||
type: streamable-http
|
||||
url: https://api.example.com/mcp
|
||||
initTimeout: 15000 # 15 seconds for server initialization
|
||||
timeout: 60000 # 60 seconds for tool operations
|
||||
```
|
||||
|
||||
**Note**: If operations are still being cut short, check your proxy configuration (e.g., nginx, traefik, etc.) which may be severing connections prematurely due to default timeouts.
|
||||
|
||||
### User Provided Credentials
|
||||
|
||||
You can allow users to provide their own credentials for MCP servers through `customUserVars`. This enables secure, user-specific authentication without storing credentials in configuration files.
|
||||
|
||||
```yaml
|
||||
mcpServers:
|
||||
my-api-server:
|
||||
type: streamable-http
|
||||
url: "https://api.example.com/mcp"
|
||||
headers:
|
||||
X-Auth-Token: "{{MY_API_KEY}}" # Uses the user-provided value
|
||||
customUserVars:
|
||||
MY_API_KEY:
|
||||
title: "API Key"
|
||||
description: "Enter your personal API key from <a href='https://example.com/keys' target='_blank'>your account settings</a>"
|
||||
```
|
||||
|
||||
Users can configure these credentials:
|
||||
- **From Chat Area**: Click the settings icon next to configurable MCP servers in the tool selection dropdown
|
||||
- **From MCP Settings Panel**: Access "MCP Settings" in the right panel to manage credentials for all configured servers
|
||||
|
||||
### OAuth Authentication
|
||||
|
||||
LibreChat supports OAuth authentication for MCP servers, following Anthropic's recommendation for secure MCP connections. OAuth provides a standardized, secure way to authenticate without storing long-lived credentials.
|
||||
|
||||
#### Supported OAuth Flows
|
||||
|
||||
LibreChat MCP servers support OAuth 2.0 with:
|
||||
- **Authorization Code Flow with PKCE**: Recommended for maximum security
|
||||
- **Client Discovery**: Automatic client registration when supported by the OAuth provider
|
||||
- **Refresh Tokens**: Automatic token renewal when available
|
||||
|
||||
#### Configuration Examples
|
||||
|
||||
```yaml
|
||||
mcpServers:
|
||||
# Public remote MCP server for PayPal, uses OAuth Client Discovery
|
||||
# ❌ Refresh Tokens: you may need to re-authenticate periodically
|
||||
# More info: https://developer.paypal.com/tools/mcp-server/
|
||||
paypal:
|
||||
type: "sse"
|
||||
initTimeout: 150000 # higher timeout to allow for initial authentication
|
||||
url: "https://mcp.paypal.com/sse"
|
||||
|
||||
# Example self-hosted remote MCP server for Spotify, uses OAuth Client Discovery
|
||||
# ✅ Refresh Tokens: refreshes token for authentication automatically
|
||||
# Hosted on Cloudflare Workers, more info: https://github.com/LibreChat-AI/spotify-mcp
|
||||
spotify:
|
||||
type: "streamable-http"
|
||||
initTimeout: 150000
|
||||
url: "https://mcp-spotify-oauth-example.account.workers.dev/mcp"
|
||||
```
|
||||
|
||||
#### OAuth Authentication Flow
|
||||
|
||||
When you first configure an OAuth-enabled MCP server:
|
||||
|
||||
1. **Initial Connection**: LibreChat attempts to connect to the MCP server
|
||||
2. **Authentication Required**: If no valid token exists, you'll see an authentication URL in the console
|
||||
3. **Browser Authentication**: Copy the URL and authenticate in your browser
|
||||
4. **Token Storage**: LibreChat securely stores the tokens for future use
|
||||
5. **Automatic Refresh**: If refresh tokens are supported, LibreChat handles renewal automatically
|
||||
|
||||
Example authentication prompt:
|
||||
```bash
|
||||
═══════════════════════════════════════════════════════════════════════
|
||||
Please visit the following URL to authenticate:
|
||||
|
||||
https://mcp-server.example.com/authorize?response_type=code&client_id=...
|
||||
|
||||
═══════════════════════════════════════════════════════════════════════
|
||||
```
|
||||
|
||||
Soon, you will be able to add MCP Servers and configure OAuth authentication directly from the LibreChat interface.
|
||||
|
||||
#### Token Management
|
||||
|
||||
LibreChat handles OAuth tokens intelligently:
|
||||
|
||||
- **Secure Storage**: Tokens are encrypted and stored securely
|
||||
- **Automatic Refresh**: When refresh tokens are available, LibreChat automatically renews expired access tokens
|
||||
- **Session Management**: Each user maintains their own OAuth sessions for multi-user environments
|
||||
|
||||
Each user will be prompted to authenticate with their own OAuth login when they first use an OAuth-enabled MCP server. This ensures that connection and authentication details are unique to each user, maintaining security and privacy in multi-user environments.
|
||||
|
||||

|
||||
|
||||
> Note: The tokens shown during app startup are for app-level initialization only and are not used for individual user connections.
|
||||
|
||||
Example of automatic token refresh:
|
||||
```bash
|
||||
[MCP][spotify] Access token missing
|
||||
[MCP][spotify] Attempting to refresh token
|
||||
[MCP][spotify] Successfully refreshed and stored OAuth tokens
|
||||
[MCP][spotify] ✓ Initialized
|
||||
```
|
||||
|
||||
#### Best Practices
|
||||
|
||||
1. **Use OAuth when available**: Prefer OAuth over API keys for better security
|
||||
2. **Configure appropriate timeouts**: Set higher `initTimeout` for OAuth flows (e.g., 150000ms)
|
||||
3. **Monitor token expiration**: Check logs for authentication issues
|
||||
4. **Plan for re-authentication**: Some providers don't support refresh tokens
|
||||
|
||||
**Note**: UI-based OAuth configuration is coming soon, which will streamline the authentication process directly from the LibreChat interface.
|
||||
|
||||
## Server Transports
|
||||
|
||||
MCP servers can be configured to use different transport mechanisms:
|
||||
|
||||
**STDIO Servers**
|
||||
- Work wells for local, single-user environments
|
||||
- Not scalable for remote or cloud deployments
|
||||
|
||||
**Server-Sent Events (SSE) Servers**
|
||||
- Remote transport mechanism but not recommended for production environment
|
||||
|
||||
**Streamable HTTP Servers**
|
||||
- Uses HTTP POST for sending messages and supports streaming responses
|
||||
- Operates as an independent process that can handle multiple client connections
|
||||
- Supports both basic requests and streaming via Server-Sent Events (SSE)
|
||||
- More performant alternative to the legacy HTTP+SSE transport
|
||||
- Supports proper multi-user server configurations
|
||||
|
||||
**For production environments**, only MCP servers with ["Streamable HTTP" transports](https://modelcontextprotocol.io/specification/draft/basic/transports#streamable-http) are recommended. Unlike SSE which maintains long-running connections, Streamable HTTP offers stateless options that are better suited for scalable, multi-user deployments.
|
||||
|
||||
LibreChat is at the forefront of implementing flexible, scalable MCP server integrations to support diverse usage scenarios and help you build the AI workflows of tomorrow.
|
||||
|
||||
---
|
||||
|
||||
**Ready to extend your AI capabilities?** Start by configuring your first MCP server and discover how LibreChat can connect to virtually any tool or service your organization needs.
|
||||
@@ -21,91 +21,77 @@ OCR functionality in LibreChat allows agents to:
|
||||
|
||||
Currently, OCR is **only available as an agent capability**. This means you must use an agent via the Agents endpoint to leverage OCR functionality.
|
||||
|
||||
## Configuration
|
||||
## OCR Strategies
|
||||
|
||||
OCR can be enabled in the LibreChat configuration file (`librechat.yaml`). The OCR configuration supports three strategies:
|
||||
LibreChat supports multiple OCR strategies to meet different deployment needs and requirements. Choose the strategy that best fits your infrastructure and compliance requirements.
|
||||
|
||||
1. **Mistral OCR** (Default)
|
||||
2. **Azure Mistral OCR** (For Azure-deployed models)
|
||||
3. **Custom OCR** (Planned for future releases)
|
||||
### 1. Mistral OCR (Default)
|
||||
|
||||
### Basic Configuration Example
|
||||
|
||||
If using the Mistral OCR API, you only need the following environment variables to get started:
|
||||
The default strategy uses Mistral's cloud API service directly. This is the simplest setup and requires only an API key from Mistral.
|
||||
|
||||
**Environment Variables:**
|
||||
```.env
|
||||
# `.env`
|
||||
OCR_API_KEY=your-mistral-api-key
|
||||
# OCR_BASEURL=https://api.mistral.ai/v1 # this is the default value
|
||||
```
|
||||
|
||||
For additional, detailed configuration options, see the [OCR Config Object Structure](/docs/configuration/librechat_yaml/object_structure/ocr).
|
||||
|
||||
**Configuration:**
|
||||
```yaml
|
||||
# `librechat.yaml`
|
||||
ocr:
|
||||
mistralModel: "mistral-ocr-latest" # Optional: Specify Mistral model, defaults to "mistral-ocr-latest"
|
||||
apiKey: "your-mistral-api-key" # Optional: Defaults to OCR_API_KEY env variable
|
||||
baseURL: "https://api.mistral.ai/v1" # Optional: Defaults to OCR_BASEURL env variable, or Mistral's API if no variable set
|
||||
strategy: "mistral_ocr" # Optional: Defaults to "mistral_ocr"
|
||||
mistralModel: "mistral-ocr-latest" # Optional: Specify Mistral model, defaults to "mistral-ocr-latest"
|
||||
apiKey: "your-mistral-api-key" # Optional: Defaults to OCR_API_KEY env variable
|
||||
baseURL: "https://api.mistral.ai/v1" # Optional: Defaults to OCR_BASEURL env variable, or Mistral's API if no variable set
|
||||
strategy: "mistral_ocr" # Optional: Defaults to "mistral_ocr"
|
||||
```
|
||||
|
||||
### Azure Mistral OCR Configuration
|
||||
|
||||
For organizations using Azure AI Foundry, you can deploy Mistral OCR models to your Azure infrastructure:
|
||||
|
||||
```yaml
|
||||
# `librechat.yaml`
|
||||
ocr:
|
||||
mistralModel: "deployed-mistral-ocr-2503" # Should match your Azure deployment name
|
||||
apiKey: "${AZURE_MISTRAL_OCR_API_KEY}" # Reference to your Azure API key in .env
|
||||
baseURL: "https://your-deployed-endpoint.models.ai.azure.com/v1" # Your Azure endpoint
|
||||
strategy: "azure_mistral_ocr" # Use Azure strategy
|
||||
```
|
||||
|
||||
## Mistral OCR
|
||||
|
||||
LibreChat supports Mistral's OCR capabilities through two deployment options:
|
||||
|
||||
### 1. Standard Mistral OCR API
|
||||
|
||||
The default option uses Mistral's cloud API service directly. This is the simplest setup and requires only an API key from Mistral.
|
||||
|
||||
### 2. Azure-Deployed Mistral OCR
|
||||
|
||||
For organizations with specific data residency or compliance requirements, Mistral OCR models can be deployed on Azure AI Foundry. Currently, the **Mistral OCR 2503** model is available for Azure deployment.
|
||||
|
||||
Benefits of Azure deployment:
|
||||
- **Data Residency**: Keep data processing within your Azure region
|
||||
- **Compliance**: Meet organizational security and compliance requirements
|
||||
- **Integration**: Leverage existing Azure infrastructure and security policies
|
||||
- **Control**: Manage your own deployment and scaling
|
||||
|
||||
You can explore the latest Mistral OCR model available on Azure AI Foundry here (requires Azure subscription):
|
||||
|
||||
https://ai.azure.com/explore/models/mistral-ocr-2503
|
||||
|
||||
### Key Features of Mistral OCR
|
||||
|
||||
**Key Features:**
|
||||
- **Document Structure Preservation**: Maintains formatting like headers, paragraphs, lists, and tables
|
||||
- **Multilingual Support**: Processes text in multiple languages and scripts
|
||||
- **Complex Layout Handling**: Handles multi-column text and mixed content
|
||||
- **Mathematical Expression Recognition**: Accurately processes equations and formulas
|
||||
- **High-Speed Processing**: Processes up to 2000 pages per minute
|
||||
|
||||
### Important Considerations
|
||||
|
||||
**Considerations:**
|
||||
- **Cost**: Using Mistral OCR may incur costs as it's a paid API service (though free trials may be available)
|
||||
- **Data Privacy**: Data processed through Mistral OCR is subject to Mistral's cloud environment and their terms of service
|
||||
- **Document Limitations**:
|
||||
- Maximum file size: 50 MB
|
||||
- Maximum document length: 1,000 pages
|
||||
|
||||
### Future Plans
|
||||
### 2. Azure Mistral OCR
|
||||
|
||||
- Mistral plans to make their OCR API available through their cloud partners, such as GCP and AWS, and enterprise self-hosting for organizations with stringent data privacy requirements ([source](https://mistral.ai/fr/news/mistral-ocr)).
|
||||
- LibreChat will continue to support Mistral OCR and explore additional OCR providers, including open-source solutions, for enhanced functionality.
|
||||
- LibreChat currently does not include the parsed image content from the OCR process in its responses, even though services like [Mistral's OCR API may provide](https://docs.mistral.ai/api/#tag/ocr) these in the result. This feature may be supported in future updates.
|
||||
For organizations using Azure AI Foundry, you can deploy Mistral OCR models to your Azure infrastructure. Currently, the **Mistral OCR 2503** model is available for Azure deployment.
|
||||
|
||||
**Configuration:**
|
||||
```yaml
|
||||
# `librechat.yaml`
|
||||
ocr:
|
||||
mistralModel: "deployed-mistral-ocr-2503" # Should match your Azure deployment name
|
||||
apiKey: "${AZURE_MISTRAL_OCR_API_KEY}" # Reference to your Azure API key in .env
|
||||
baseURL: "https://your-deployed-endpoint.models.ai.azure.com/v1" # Your Azure endpoint
|
||||
strategy: "azure_mistral_ocr" # Use Azure strategy
|
||||
```
|
||||
|
||||
**Benefits:**
|
||||
- **Data Residency**: Keep data processing within your Azure region
|
||||
- **Compliance**: Meet organizational security and compliance requirements
|
||||
- **Integration**: Leverage existing Azure infrastructure and security policies
|
||||
- **Control**: Manage your own deployment and scaling
|
||||
|
||||
**Azure Model Information:**
|
||||
You can explore the latest Mistral OCR model available on Azure AI Foundry here (requires Azure subscription):
|
||||
|
||||
https://ai.azure.com/explore/models/mistral-ocr-2503
|
||||
|
||||
### 3. Custom OCR (Planned)
|
||||
|
||||
Support for custom OCR providers and user-defined strategies is planned for future releases.
|
||||
|
||||
## Detailed Configuration
|
||||
|
||||
For additional, detailed configuration options, see the [OCR Config Object Structure](/docs/configuration/librechat_yaml/object_structure/ocr).
|
||||
|
||||
## Using File Context (OCR) in LibreChat
|
||||
|
||||
@@ -160,6 +146,8 @@ LibreChat plans to expand OCR capabilities in future releases:
|
||||
- Integration with open-source OCR solutions
|
||||
- Enhanced document processing options
|
||||
- More granular control over OCR settings
|
||||
- Mistral plans to make their OCR API available through their cloud partners, such as GCP and AWS, and enterprise self-hosting for organizations with stringent data privacy requirements ([source](https://mistral.ai/fr/news/mistral-ocr))
|
||||
- LibreChat currently does not include the parsed image content from the OCR process in its responses, even though services like [Mistral's OCR API may provide](https://docs.mistral.ai/api/#tag/ocr) these in the result. This feature may be supported in future updates.
|
||||
|
||||
---
|
||||
|
||||
|
||||
BIN
public/images/agents/mcp_agent_tools.png
Normal file
|
After Width: | Height: | Size: 23 KiB |
BIN
public/images/agents/mcp_chat.png
Normal file
|
After Width: | Height: | Size: 21 KiB |
BIN
public/images/agents/mcp_oauth_flow.png
Normal file
|
After Width: | Height: | Size: 22 KiB |
BIN
public/images/agents/mcp_oauth_success.png
Normal file
|
After Width: | Height: | Size: 20 KiB |
|
Before Width: | Height: | Size: 113 KiB |
BIN
public/images/agents/mcp_tools_v2.png
Normal file
|
After Width: | Height: | Size: 143 KiB |
BIN
public/images/mcp/mcp_per_user_vars_assistant.png
Normal file
|
After Width: | Height: | Size: 71 KiB |
BIN
public/images/mcp/mcp_per_user_vars_assistant_dialog.png
Normal file
|
After Width: | Height: | Size: 245 KiB |
BIN
public/images/mcp/mcp_per_user_vars_side_panel.png
Normal file
|
After Width: | Height: | Size: 90 KiB |