📘 feat: Update Agents, OCR, and MCP Docs with new Features (#336)

* 📚 docs: Update OCR documentation for clarity and structure

* 📚 docs: Add Model Context Protocol (MCP) documentation and update features

* 📚 docs: Add Model Context Protocol (MCP) section to features documentation

* 📚 docs: Enhance MCP documentation with OAuth authentication details and configuration examples

* 📚 docs: Update MCP documentation with improved configuration examples and OAuth authentication details

* 📚 docs: Update dotenv documentation to reflect new index header constants

* 📚 docs: Revise MCP integration documentation for Agents with clearer instructions and new images

- Updated the steps for adding MCP tools to agents for better clarity and organization.
- Introduced new images to illustrate the MCP tools interface.
- Removed outdated image and improved the overall structure of the documentation.

* 📚 docs: Enhance MCP documentation with custom user variables support

---------

Co-authored-by: Aleš Kůtek <kutekales@gmail.com>
This commit is contained in:
Danny Avila
2025-06-19 20:36:29 -04:00
committed by GitHub
parent 223d7513fe
commit 5bec56610f
19 changed files with 477 additions and 155 deletions

View File

@@ -10,4 +10,9 @@
- Added additional user field placeholders to [MCP Servers Object Structure](/docs/configuration/librechat_yaml/object_structure/mcp_servers): - Added additional user field placeholders to [MCP Servers Object Structure](/docs/configuration/librechat_yaml/object_structure/mcp_servers):
- `{{LIBRECHAT_USER_ID}}`: Will be replaced with the current user's ID, enabling multi-user support. - `{{LIBRECHAT_USER_ID}}`: Will be replaced with the current user's ID, enabling multi-user support.
- `{{LIBRECHAT_USER_*}}`: Dynamic user field placeholders. Replace `*` with the UPPERCASE version of any allowed field. - `{{LIBRECHAT_USER_*}}`: Dynamic user field placeholders. Replace `*` with the UPPERCASE version of any allowed field.
- See: **[`headers`](/docs/configuration/librechat_yaml/object_structure/mcp_servers#headers)** and **[`url`](/docs/configuration/librechat_yaml/object_structure/mcp_servers#url)** fields, which both support these placeholders. - See: **[`headers`](/docs/configuration/librechat_yaml/object_structure/mcp_servers#headers)** and **[`url`](/docs/configuration/librechat_yaml/object_structure/mcp_servers#url)** fields, which both support these placeholders.
- Added `customUserVars` to MCP Servers for per-user credentials:
- Allows users to securely provide their own API keys and credentials for MCP servers
- Configurable through UI via settings icon in tool selection or MCP Settings panel
- Ensures authentication details remain private in multi-user environments
- See: [MCP Servers Object Structure - customUserVars](/docs/configuration/librechat_yaml/object_structure/mcp_servers#customuservars)

View File

@@ -98,9 +98,9 @@ Sets the [Cache-Control](https://developer.mozilla.org/en-US/docs/Web/HTTP/Heade
<OptionTable <OptionTable
options={[ options={[
['INDEX_HTML_CACHE_CONTROL', 'string', 'Cache-Control header for index.html','INDEX_HTML_CACHE_CONTROL=no-cache, no-store, must-revalidate'], ['INDEX_CACHE_CONTROL', 'string', 'Cache-Control header for index.html','INDEX_CACHE_CONTROL=no-cache, no-store, must-revalidate'],
['INDEX_HTML_PRAGMA', 'string', 'Pragma header for index.html','INDEX_HTML_PRAGMA=no-cache'], ['INDEX_PRAGMA', 'string', 'Pragma header for index.html','INDEX_PRAGMA=no-cache'],
['INDEX_HTML_EXPIRES', 'string', 'Expires header for index.html','INDEX_HTML_EXPIRES=0'], ['INDEX_EXPIRES', 'string', 'Expires header for index.html','INDEX_EXPIRES=0'],
]} ]}
/> />

View File

@@ -401,6 +401,7 @@ see: [Model Specs Object Structure](/docs/configuration/librechat_yaml/object_st
- `timeout`: Timeout in milliseconds for MCP server requests. Determines how long to wait for a response for tool requests. - `timeout`: Timeout in milliseconds for MCP server requests. Determines how long to wait for a response for tool requests.
- `initTimeout`: Timeout in milliseconds for MCP server initialization. Determines how long to wait for the server to initialize. - `initTimeout`: Timeout in milliseconds for MCP server initialization. Determines how long to wait for the server to initialize.
- `serverInstructions`: Controls whether server instructions are included in agent context. Can be `true` (use server-provided), `false` (disabled), or a custom string (overrides server-provided). - `serverInstructions`: Controls whether server instructions are included in agent context. Can be `true` (use server-provided), `false` (disabled), or a custom string (overrides server-provided).
- `customUserVars`: (Optional) Defines custom variables (e.g., API keys, URLs) that individual users can set for an MCP server. These per-user values, provided through the UI, can then be referenced in the server's `headers` or `env` configurations using `{{VARIABLE_NAME}}` syntax. This allows for per-user authentication or customization for MCP tools.
- see: [MCP Servers Object Structure](/docs/configuration/librechat_yaml/object_structure/mcp_servers) - see: [MCP Servers Object Structure](/docs/configuration/librechat_yaml/object_structure/mcp_servers)
**Example:** **Example:**
@@ -442,6 +443,16 @@ mcpServers:
headers: headers:
Authorization: "Bearer ${API_TOKEN}" Authorization: "Bearer ${API_TOKEN}"
timeout: 30000 timeout: 30000
per-user-crendentials-example:
type: sse
url: "https//some.mcp/sse"
headers:
X-Custom-Auth-Token: "{{USER_API_KEY}}" # Placeholder for the user-provided API key, defined in `customUserVars` below.
customUserVars:
USER_API_KEY:
title: "Service API Key"
description: "Your personal API key for this service. You can get it <a href='https://example.com/api-keys' target='_blank'>here</a>."
serverInstructions: true
``` ```
see: [MCP Servers Object Structure](/docs/configuration/librechat_yaml/object_structure/mcp_servers) see: [MCP Servers Object Structure](/docs/configuration/librechat_yaml/object_structure/mcp_servers)

View File

@@ -39,6 +39,15 @@ mcpServers:
streamable-http-server: streamable-http-server:
type: streamable-http type: streamable-http
url: https://example.com/api/ url: https://example.com/api/
per-user-credentials-example:
type: streamable-http
url: "https://example.com/api/"
headers:
X-Auth-Token: "{{MY_SERVICE_API_KEY}}"
customUserVars:
MY_SERVICE_API_KEY:
title: "My Service API Key"
description: "Enter your personal API key for the service. You can generate one at <a href='https://myservice.example.com/developer/keys' target='_blank'>Service Developer Portal</a>."
``` ```
## `<serverName>` ## `<serverName>`
@@ -66,6 +75,7 @@ mcpServers:
['initTimeout', 'Number', '(Optional) Timeout in milliseconds for MCP server initialization. Determines how long to wait for the server to initialize.', 'initTimeout: 10000'], ['initTimeout', 'Number', '(Optional) Timeout in milliseconds for MCP server initialization. Determines how long to wait for the server to initialize.', 'initTimeout: 10000'],
['env', 'Object', '(Optional, `stdio` type only) Environment variables to use when spawning the process.', 'env:\n NODE_ENV: "production"'], ['env', 'Object', '(Optional, `stdio` type only) Environment variables to use when spawning the process.', 'env:\n NODE_ENV: "production"'],
['stderr', 'String or Stream or Number', '(Optional, `stdio` type only) How to handle `stderr` of the child process. Defaults to `"inherit"`.', 'stderr: "inherit"'], ['stderr', 'String or Stream or Number', '(Optional, `stdio` type only) How to handle `stderr` of the child process. Defaults to `"inherit"`.', 'stderr: "inherit"'],
['customUserVars', 'Object', '(Optional) Defines custom variables that users can set for this MCP server, allowing for per-user credentials or configurations (e.g., API keys). These variables can then be referenced in `headers` or `env` fields.', 'customUserVars:\n API_KEY:\n title: "API Key"\n description: "Your personal API key."'],
]} ]}
/> />
@@ -97,11 +107,12 @@ mcpServers:
#### `headers` #### `headers`
- **Type:** Object (Optional, for `sse` and `streamable-http` types) - **Type:** Object (Optional, for `sse` and `streamable-http` types)
- **Description:** Custom headers to send with the request. - **Description:** Custom headers to send with the request. Supports various placeholder types for dynamic value substitution.
- **Special Values:** - **Placeholder Support:**
- `{{LIBRECHAT_USER_ID}}`: Will be replaced with the current user's ID, enabling multi-user support. - `{{LIBRECHAT_USER_ID}}`: Will be replaced with the current user's ID, enabling multi-user support.
- `{{LIBRECHAT_USER_*}}`: Dynamic user field placeholders. Replace `*` with the UPPERCASE version of any allowed field. - `{{LIBRECHAT_USER_*}}`: Dynamic user field placeholders. Replace `*` with the UPPERCASE version of any allowed field.
- `${ENV_VAR}`: Will be replaced with the value of the environment variable `ENV_VAR`. - `{{CUSTOM_VARIABLE_NAME}}`: Replaced with the value provided by the user for a variable defined in `customUserVars` (e.g., `{{MY_API_KEY}}`).
- `${ENV_VAR}`: Will be replaced with the value of the environment variable `{{ENV_VAR}}`.
**Available User Field Placeholders:** **Available User Field Placeholders:**
@@ -178,7 +189,12 @@ mcpServers:
#### `env` #### `env`
- **Type:** Object (Optional, `stdio` type only) - **Type:** Object (Optional, `stdio` type only)
- **Description:** Environment variables to use when spawning the process. Supports the same dynamic user field placeholders as headers (`{{LIBRECHAT_USER_*}}`) and environment variable substitution (`${ENV_VAR}`). - **Description:** Environment variables to use when spawning the process.
- **Placeholder Support:**
- `{{LIBRECHAT_USER_ID}}`: Replaced with the current user's ID.
- `{{LIBRECHAT_USER_*}}`: Dynamic user field placeholders (e.g., `{{LIBRECHAT_USER_EMAIL}}`).
- `{{CUSTOM_VARIABLE_NAME}}`: Replaced with the value provided by the user for a variable defined in `customUserVars` (e.g., `{{MY_API_KEY}}`).
- `${ENV_VAR}`: Replaced with the value of the server-side environment variable `{{ENV_VAR}}`.
#### `timeout` #### `timeout`
@@ -198,6 +214,41 @@ mcpServers:
- **Description:** How to handle `stderr` of the child process. This matches the semantics of Node's [`child_process.spawn`](https://nodejs.org/api/child_process.html#child_processspawncommand-args-options). - **Description:** How to handle `stderr` of the child process. This matches the semantics of Node's [`child_process.spawn`](https://nodejs.org/api/child_process.html#child_processspawncommand-args-options).
- **Default Value:** `"inherit"` (messages to `stderr` will be printed to the parent process's `stderr`). - **Default Value:** `"inherit"` (messages to `stderr` will be printed to the parent process's `stderr`).
#### `customUserVars`
- **Type:** Object (Optional)
- **Description:** Defines custom variables that users can set for this MCP server. This allows administrators to specify variables (e.g., API keys, URLs) that each user must configure individually. These user-provided values can then be used in `headers` configuration.
- **Structure:**
- The `customUserVars` object contains keys, where each key represents a variable name (e.g., `MY_API_KEY`). This name will be used in placeholders like `{{MY_API_KEY}}`.
- Each variable name is an object with the following subkeys:
- `title`: String (Required) - A user-friendly title for the variable, displayed in the configuration UI.
- `description`: String (Optional) - A description or instructions for the variable, also displayed in the UI to guide the user. HTML can be used in this field (e.g., to create a link: `<a href="https://example.com" target="_blank">More info</a>`).
- **Usage in `headers` and `env`:**
- Once defined under `customUserVars`, these variables can be referenced in the `headers` (for `sse` and `streamable-http` types) or `env` (for `stdio` type) sections using the `{{VARIABLE_NAME}}` syntax.
- Users provide these values through the UI. These settings can be accessed in two ways:
- **From Assistant Chat Input**: When selecting MCP tools for an assistant, a settings icon will appear next to configurable MCP servers in the tool selection dropdown. Clicking this icon opens a dialog to manage credentials for that server.
<img src="/images/mcp/mcp_per_user_vars_assistant.png" alt="MCP Per-User Variables Configuration - Assistant Access" width={600} />
<img src="/images/mcp/mcp_per_user_vars_assistant_dialog.png" alt="MCP Per-User Variables Configuration - Assistant Access Dialog" width={800} />
- **From Settings Panel**: A dedicated "MCP Settings" section in the right panel lists all MCP servers with definable custom variables. Users can click on a server to open the configuration dialog to set or update their credentials for that specific MCP server.
<img src="/images/mcp/mcp_per_user_vars_side_panel.png" alt="MCP Per-User Variables Configuration - Settings Panel Access" width={300} />
- These user-provided values are stored securely, associated with the individual user and the specific MCP server, and substituted at runtime.
- **Example:**
```yaml
customUserVars:
MY_SERVICE_API_KEY:
title: "My Service API Key"
description: "Your personal API access key for My Service. Find it at <a href='https://myservice.example.com/settings/api' target='_blank'>My Service API Settings</a>."
SOME_OTHER_VAR:
title: "Some Other Variable"
description: "The specific value for some other configuration (e.g., a specific path or identifier)."
```
Usage in `headers`:
```yaml
headers:
X-Auth-Token: "{{MY_SERVICE_API_KEY}}"
X-Some-Other-Config: "{{SOME_OTHER_VAR}}"
```
### Notes ### Notes
- **Type Inference:** - **Type Inference:**
@@ -273,6 +324,20 @@ user-aware-server:
Authorization: "Bearer ${API_TOKEN}" Authorization: "Bearer ${API_TOKEN}"
``` ```
### MCP Server with Per-User Credentials via `customUserVars`
```yaml filename="MCP Server with customUserVars"
my-mcp-server:
type: streamable-http
url: "https://api.example-service.com/api/" # Example URL
headers:
X-Auth-Token: "{{API_KEY}}" # Uses the API_KEY defined below
customUserVars:
API_KEY: # This key will be used as {{API_KEY}} in headers/url
title: "API Key" # This is the label shown above the input field
description: "Get your API key <a href='https://example.com/api-keys' target='_blank'>here</a>." # This description appears below the input
```
### MCP Server with Custom Icon ### MCP Server with Custom Icon
```yaml filename="MCP Server with Icon" ```yaml filename="MCP Server with Icon"

View File

@@ -112,6 +112,6 @@ ocr:
**Available Strategies:** **Available Strategies:**
- `mistral_ocr`: Uses Mistral's OCRcapabilities via the standard [Mistral API](/docs/features/ocr#mistral-ocr) . - `mistral_ocr`: Uses Mistral's OCRcapabilities via the standard [Mistral API](/docs/features/ocr#1-mistral-ocr-default) .
- `azure_mistral_ocr`: Uses Mistral OCR models deployed on [Azure AI Foundry](/docs/features/ocr#2-azure-deployed-mistral-ocr). - `azure_mistral_ocr`: Uses Mistral OCR models deployed on [Azure AI Foundry](/docs/features/ocr#2-azure-mistral-ocr).
- `custom_ocr`: Uses a custom OCR service specified by the `baseURL` [(not yet implemented)](/docs/features/ocr#future-enhancements). - `custom_ocr`: Uses a custom OCR service specified by the `baseURL` [(not yet implemented)](/docs/features/ocr#future-enhancements).

View File

@@ -1,5 +1,6 @@
export default { export default {
index: 'Overview', index: 'Overview',
mcp: 'MCP',
agents: 'Agents', agents: 'Agents',
code_interpreter: 'Code Interpreter API', code_interpreter: 'Code Interpreter API',
artifacts: 'Artifacts - Generative UI', artifacts: 'Artifacts - Generative UI',

View File

@@ -71,6 +71,41 @@ The File Context (OCR) capability allows your agent to extract and process text
- [More info about OCR](/docs/features/ocr) - [More info about OCR](/docs/features/ocr)
- **Currently uses Mistral OCR API which may incur costs** - **Currently uses Mistral OCR API which may incur costs**
### Model Context Protocol (MCP)
MCP is an open protocol that standardizes how applications provide context to Large Language Models (LLMs), acting like a universal adapter for AI tools and data sources.
For more information, see documentation on [MCP](/docs/features/mcp).
#### Agents with MCP Tools
1. Configure MCP servers in your `librechat.yaml` file
2. Restart LibreChat to initialize the connections
3. Create or edit an agent
4. Click the "Add Tools" button in the Agent Builder panel to open the Tools Dialog shown below
5. Select the MCP server(s) you want to add - each server appears as a single entry
6. Save your changes to the agent
In this example, we've added the **[Spotify MCP server](https://github.com/LibreChat-AI/spotify-mcp)** to an agent.
![Agents - MCP](/images/agents/mcp_tools_v2.png)
#### Managing MCP Tools
Once an MCP server is added to an agent, you can fine-tune which specific tools are available:
- After adding an MCP server, expand it to see all available tools
- Check/uncheck individual tools as needed
- For example, the Spotify MCP server provides ~20 tools (search, playback control, playlist management, etc.)
- This granular control lets you limit agents to only the tools they need
![Agents - MCP Tools](/images/agents/mcp_agent_tools.png)
Learn More:
- [Configuring MCP Servers](/docs/features/mcp)
- [Model Context Protocol Introduction](https://modelcontextprotocol.io/introduction)
### Artifacts ### Artifacts
The Artifacts capability enables your agent to generate and display interactive content: The Artifacts capability enables your agent to generate and display interactive content:
@@ -174,94 +209,6 @@ A single, non-tool response is 1 step. A singular round of tool usage is usually
1. API Request -> 2. Tool Usage (1 or many tools) -> 3. Follow-up API Request 1. API Request -> 2. Tool Usage (1 or many tools) -> 3. Follow-up API Request
### Model Context Protocol (MCP)
MCP is an open protocol that standardizes how applications provide context to Large Language Models (LLMs), acting like a universal adapter for AI tools and data sources.
Imagine MCP as the "USB-C of AI" - just as USB-C provides a universal connection standard for electronic devices, MCP offers a standardized way to connect AI models to diverse tools, data sources, and services.
By configuring `mcpServers` in the `librechat.yaml` file, you can:
- Add custom tools from various sources
- Integrate specialized data access tools
- Extend AI capabilities beyond default offerings
Here are some tools added by configuring the ["Filesystem" MCP server](https://github.com/modelcontextprotocol/servers/tree/main/src/filesystem):
![Agents - MCP](/images/agents/mcp_tools.png)
Key Features:
- Upon app restart, tools are available in the "Add Tools" button within the Agent Builder panel
- Seamlessly connect custom tools and data sources
- Leverage a growing ecosystem of MCP-compatible servers and integrations
Learn More:
- [Configuring MCP Servers](/docs/configuration/librechat_yaml/object_structure/mcp_servers)
- [Model Context Protocol Introduction](https://modelcontextprotocol.io/introduction)
#### MCP Server Types and Multi-User Support
MCP servers can be configured to use different transport mechanisms:
**STDIO Servers**
- Work well for local, single-user environments
- Not scalable for remote or cloud deployments
- Limited multi-user support
**Server-Sent Events (SSE) Servers**
- More scalable transport mechanism for multi-user environments
- Enhanced multi-user support with user-specific connections
**Streamable HTTP Servers**
- Uses HTTP POST for sending messages and supports streaming responses
- Operates as an independent process that can handle multiple client connections
- Supports both basic requests and streaming via Server-Sent Events (SSE)
- More performant alternative to the legacy HTTP+SSE transport
- Supports proper multi-user server configurations
**Multi-User Features**
- User-level and app-level connections with proper establishment and cleanup
- MCP Servers can distinguish connections by session ID
- Dynamic user field placeholders in headers, URLs, and environment variables:
- `{{LIBRECHAT_USER_ID}}` - User's unique identifier
- Basic fields: `{{LIBRECHAT_USER_EMAIL}}`, `{{LIBRECHAT_USER_USERNAME}}`, `{{LIBRECHAT_USER_NAME}}`, `{{LIBRECHAT_USER_ROLE}}`, `{{LIBRECHAT_USER_PROVIDER}}`
- Provider IDs: `{{LIBRECHAT_USER_GOOGLEID}}`, `{{LIBRECHAT_USER_OPENIDID}}`, `{{LIBRECHAT_USER_SAMLID}}`, `{{LIBRECHAT_USER_LDAPID}}`, etc.
- Boolean fields: `{{LIBRECHAT_USER_EMAILVERIFIED}}`, `{{LIBRECHAT_USER_TWOFACTORENABLED}}`, `{{LIBRECHAT_USER_TERMSACCEPTED}}`
- See [MCP Servers documentation](/docs/configuration/librechat_yaml/object_structure/mcp_servers#headers) for complete field list
- Environment variable support: `${SOME_API_KEY}`
- User idle timeout management for efficient resource usage
**Example Configuration:**
```yaml
mcpServers:
googlesheets:
type: sse
url: https://mcp.composio.dev/googlesheets/some-endpoint
headers:
X-User-ID: "{{LIBRECHAT_USER_ID}}"
X-User-Email: "{{LIBRECHAT_USER_EMAIL}}"
X-API-Key: "${SOME_API_KEY}"
streamable-http-example:
type: streamable-http
url: https://example.com/mcp/users/{{LIBRECHAT_USER_USERNAME}}
headers:
X-User-ID: "{{LIBRECHAT_USER_ID}}"
X-User-Role: "{{LIBRECHAT_USER_ROLE}}"
Authorization: "Bearer ${API_TOKEN}"
user-context-server:
type: stdio
command: node
args: ["server.js"]
env:
USER_EMAIL: "{{LIBRECHAT_USER_EMAIL}}"
USER_PROVIDER: "{{LIBRECHAT_USER_PROVIDER}}"
API_KEY: "${MCP_API_KEY}"
```
LibreChat is at the forefront of implementing flexible, scalable MCP server integrations to support diverse usage scenarios.
For detailed configuration options, see [Configuring MCP Servers](/docs/configuration/librechat_yaml/object_structure/mcp_servers).
## File Management ## File Management
Agents support four distinct file upload categories: Agents support four distinct file upload categories:

View File

@@ -45,6 +45,14 @@ import Image from 'next/image'
- **Configuration Required**: Must be explicitly configured in `librechat.yaml` to work - **Configuration Required**: Must be explicitly configured in `librechat.yaml` to work
- **[Learn More →](/docs/features/memory)** - **[Learn More →](/docs/features/memory)**
### 🔌 [Model Context Protocol (MCP)](/docs/features/mcp)
- **Universal AI Tool Integration**: Connect AI models to any tool or service through a standardized protocol
- **Production-Ready Transports**: Support for scalable HTTP-based servers ideal for multi-user deployments
- **Dynamic User Context**: Pass user-specific information to MCP servers for personalized experiences
- **Flexible Usage**: Use MCP tools directly in chat or integrate them with custom agents
- **[Learn More →](/docs/features/mcp)**
### 🪄 **[Artifacts](/docs/features/artifacts)** ### 🪄 **[Artifacts](/docs/features/artifacts)**
- **Generative UI:** React, HTML, Mermaid diagrams - **Generative UI:** React, HTML, Mermaid diagrams

297
pages/docs/features/mcp.mdx Normal file
View File

@@ -0,0 +1,297 @@
---
title: Model Context Protocol (MCP) - Extend LibreChat with Universal AI Tools
description: Learn how LibreChat leverages Model Context Protocol (MCP) to provide seamless integration with external tools, data sources, and specialized services.
---
# Model Context Protocol (MCP)
Model Context Protocol (MCP) is an open protocol that standardizes how applications provide context to Large Language Models (LLMs). Think of MCP as the **"USB-C of AI"** - just as USB-C provides a universal connection standard for electronic devices, MCP offers a standardized way to connect AI models to diverse tools, data sources, and services.
LibreChat leverages MCP to dramatically expand what your AI agents can do, allowing you to integrate everything from file system access, web browsers, specialized APIs, to custom business tools.
## Why MCP Matters
LLMs are limited to their built-in capabilities. With MCP, LibreChat breaks down these walls by:
- **Connecting to any tool or service** that provides an MCP server
- **Standardizing integrations** so you don't need to edit LibreChat's code for each tool
- **Supporting multi-user environments** with proper authentication and isolation
- **Providing a growing ecosystem** of dynamic, ready-to-use integrations
## How MCP Works in LibreChat
LibreChat provides two ways to use MCP servers.
1. First, configure MCP servers in your `librechat.yaml` file ([see example below](#basic-configuration)).
2. Restart LibreChat to initialize the connections.
### In Chat Area
![MCP Tools in Chat Area](/images/agents/mcp_chat.png)
LibreChat displays configured MCP servers directly in the chat area when using traditional endpoints (OpenAI, Anthropic, Google, Bedrock, etc.):
- Select any non-agent endpoint first, and a tool-compatible model
- MCP servers appear in a dropdown in the chat interface below your text input
- When selected, all tools from that server become available to your current model
- Quick access to MCP tools without creating an agent, allowing multiple servers to be used at once
To disable MCP servers from appearing in the chat dropdown (keeping them agent-only), set [`chatMenu: false`](/docs/configuration/librechat_yaml/object_structure/mcp_servers#chatmenu) in your configuration:
```yaml
mcpServers:
internal-tools:
command: npx
args: ["-y", "internal-mcp-server"]
chatMenu: false # Only available in agent builder
```
### With Agents
MCP servers integrate seamlessly with LibreChat Agents:
1. Create or edit an agent
2. Click "Add Tools" to open the Tools Dialog from the Agent Builder panel
3. Select MCP servers once added, each appears as a single entry
4. Fine-tune by enabling/disabling individual tools after adding
5. Save your agent
![MCP Tools in Agent Builder](/images/agents/mcp_tools_v2.png)
This higher-level organization keeps the interface manageable - even servers with 20+ tools (like Spotify) appear as single entries that can be expanded for granular control.
### Basic Configuration
Add MCP servers to your `librechat.yaml` file:
```yaml
mcpServers:
# File system access
filesystem:
command: npx
args:
- -y
- "@modelcontextprotocol/server-filesystem"
- /path/to/your/documents
# Web browser automation
puppeteer:
command: npx
args:
- -y
- "@modelcontextprotocol/server-puppeteer"
# Production-ready cloud service
business-api:
type: streamable-http
url: https://api.yourbusiness.com/mcp
headers:
X-User-ID: "{{LIBRECHAT_USER_ID}}"
Authorization: "Bearer ${API_TOKEN}"
timeout: 30000
serverInstructions: true
```
For detailed configuration options and examples, see:
- [MCP Servers Configuration](/docs/configuration/librechat_yaml/object_structure/mcp_servers)
- [Agent Configuration](/docs/configuration/librechat_yaml/object_structure/agents)
- [Advanced Agent Features](/docs/features/agents#model-context-protocol-mcp)
## LibreChat-Specific Features
LibreChat's MCP implementation is designed for highly configurable, real-world, multi-user environments.
### User-Specific Connections
- Each user gets their own isolated connection to MCP servers
- User authentication and permissions are respected
- Personal data and context remain private
### Dynamic User Context
MCP servers can access user information through placeholders in **URLs and headers** (for SSE and Streamable HTTP transports):
```yaml
mcpServers:
user-api:
type: streamable-http
url: https://api.example.com/users/{{LIBRECHAT_USER_USERNAME}}/mcp
headers:
X-User-ID: "{{LIBRECHAT_USER_ID}}"
X-User-Email: "{{LIBRECHAT_USER_EMAIL}}"
X-User-Role: "{{LIBRECHAT_USER_ROLE}}"
Authorization: "Bearer ${API_TOKEN}"
```
Available placeholders include:
- `{{LIBRECHAT_USER_ID}}` - Unique user identifier
- `{{LIBRECHAT_USER_EMAIL}}` - User's email address
- `{{LIBRECHAT_USER_ROLE}}` - User role (admin, user, etc.)
- `{{LIBRECHAT_USER_USERNAME}}` - Username
- And many more (see [MCP Servers Configuration](/docs/configuration/librechat_yaml/object_structure/mcp_servers#headers) for complete list)
### Server Instructions
`serverInstructions` is a LibreChat feature that dynamically adds configured instructions when any tool from that MCP server is selected:
```yaml
mcpServers:
filesystem:
command: npx
args: ["-y", "@modelcontextprotocol/server-filesystem", "/docs"]
serverInstructions: |
When accessing files:
- Always check file permissions first
- Use absolute paths for reliability
- Handle errors gracefully
```
Options:
- `true`: Use server-provided instructions
- `false`: Disable instructions
- `string`: Custom instructions (shown above)
### Timeout Configuration
For long-running MCP operations, configure appropriate timeouts for both initialization and tool operations.
```yaml
mcpServers:
data-processor:
type: streamable-http
url: https://api.example.com/mcp
initTimeout: 15000 # 15 seconds for server initialization
timeout: 60000 # 60 seconds for tool operations
```
**Note**: If operations are still being cut short, check your proxy configuration (e.g., nginx, traefik, etc.) which may be severing connections prematurely due to default timeouts.
### User Provided Credentials
You can allow users to provide their own credentials for MCP servers through `customUserVars`. This enables secure, user-specific authentication without storing credentials in configuration files.
```yaml
mcpServers:
my-api-server:
type: streamable-http
url: "https://api.example.com/mcp"
headers:
X-Auth-Token: "{{MY_API_KEY}}" # Uses the user-provided value
customUserVars:
MY_API_KEY:
title: "API Key"
description: "Enter your personal API key from <a href='https://example.com/keys' target='_blank'>your account settings</a>"
```
Users can configure these credentials:
- **From Chat Area**: Click the settings icon next to configurable MCP servers in the tool selection dropdown
- **From MCP Settings Panel**: Access "MCP Settings" in the right panel to manage credentials for all configured servers
### OAuth Authentication
LibreChat supports OAuth authentication for MCP servers, following Anthropic's recommendation for secure MCP connections. OAuth provides a standardized, secure way to authenticate without storing long-lived credentials.
#### Supported OAuth Flows
LibreChat MCP servers support OAuth 2.0 with:
- **Authorization Code Flow with PKCE**: Recommended for maximum security
- **Client Discovery**: Automatic client registration when supported by the OAuth provider
- **Refresh Tokens**: Automatic token renewal when available
#### Configuration Examples
```yaml
mcpServers:
# Public remote MCP server for PayPal, uses OAuth Client Discovery
# ❌ Refresh Tokens: you may need to re-authenticate periodically
# More info: https://developer.paypal.com/tools/mcp-server/
paypal:
type: "sse"
initTimeout: 150000 # higher timeout to allow for initial authentication
url: "https://mcp.paypal.com/sse"
# Example self-hosted remote MCP server for Spotify, uses OAuth Client Discovery
# ✅ Refresh Tokens: refreshes token for authentication automatically
# Hosted on Cloudflare Workers, more info: https://github.com/LibreChat-AI/spotify-mcp
spotify:
type: "streamable-http"
initTimeout: 150000
url: "https://mcp-spotify-oauth-example.account.workers.dev/mcp"
```
#### OAuth Authentication Flow
When you first configure an OAuth-enabled MCP server:
1. **Initial Connection**: LibreChat attempts to connect to the MCP server
2. **Authentication Required**: If no valid token exists, you'll see an authentication URL in the console
3. **Browser Authentication**: Copy the URL and authenticate in your browser
4. **Token Storage**: LibreChat securely stores the tokens for future use
5. **Automatic Refresh**: If refresh tokens are supported, LibreChat handles renewal automatically
Example authentication prompt:
```bash
═══════════════════════════════════════════════════════════════════════
Please visit the following URL to authenticate:
https://mcp-server.example.com/authorize?response_type=code&client_id=...
═══════════════════════════════════════════════════════════════════════
```
Soon, you will be able to add MCP Servers and configure OAuth authentication directly from the LibreChat interface.
#### Token Management
LibreChat handles OAuth tokens intelligently:
- **Secure Storage**: Tokens are encrypted and stored securely
- **Automatic Refresh**: When refresh tokens are available, LibreChat automatically renews expired access tokens
- **Session Management**: Each user maintains their own OAuth sessions for multi-user environments
Each user will be prompted to authenticate with their own OAuth login when they first use an OAuth-enabled MCP server. This ensures that connection and authentication details are unique to each user, maintaining security and privacy in multi-user environments.
![User-specific OAuth authentication flow](/images/agents/mcp_oauth_flow.png)
> Note: The tokens shown during app startup are for app-level initialization only and are not used for individual user connections.
Example of automatic token refresh:
```bash
[MCP][spotify] Access token missing
[MCP][spotify] Attempting to refresh token
[MCP][spotify] Successfully refreshed and stored OAuth tokens
[MCP][spotify] ✓ Initialized
```
#### Best Practices
1. **Use OAuth when available**: Prefer OAuth over API keys for better security
2. **Configure appropriate timeouts**: Set higher `initTimeout` for OAuth flows (e.g., 150000ms)
3. **Monitor token expiration**: Check logs for authentication issues
4. **Plan for re-authentication**: Some providers don't support refresh tokens
**Note**: UI-based OAuth configuration is coming soon, which will streamline the authentication process directly from the LibreChat interface.
## Server Transports
MCP servers can be configured to use different transport mechanisms:
**STDIO Servers**
- Work wells for local, single-user environments
- Not scalable for remote or cloud deployments
**Server-Sent Events (SSE) Servers**
- Remote transport mechanism but not recommended for production environment
**Streamable HTTP Servers**
- Uses HTTP POST for sending messages and supports streaming responses
- Operates as an independent process that can handle multiple client connections
- Supports both basic requests and streaming via Server-Sent Events (SSE)
- More performant alternative to the legacy HTTP+SSE transport
- Supports proper multi-user server configurations
**For production environments**, only MCP servers with ["Streamable HTTP" transports](https://modelcontextprotocol.io/specification/draft/basic/transports#streamable-http) are recommended. Unlike SSE which maintains long-running connections, Streamable HTTP offers stateless options that are better suited for scalable, multi-user deployments.
LibreChat is at the forefront of implementing flexible, scalable MCP server integrations to support diverse usage scenarios and help you build the AI workflows of tomorrow.
---
**Ready to extend your AI capabilities?** Start by configuring your first MCP server and discover how LibreChat can connect to virtually any tool or service your organization needs.

View File

@@ -21,91 +21,77 @@ OCR functionality in LibreChat allows agents to:
Currently, OCR is **only available as an agent capability**. This means you must use an agent via the Agents endpoint to leverage OCR functionality. Currently, OCR is **only available as an agent capability**. This means you must use an agent via the Agents endpoint to leverage OCR functionality.
## Configuration ## OCR Strategies
OCR can be enabled in the LibreChat configuration file (`librechat.yaml`). The OCR configuration supports three strategies: LibreChat supports multiple OCR strategies to meet different deployment needs and requirements. Choose the strategy that best fits your infrastructure and compliance requirements.
1. **Mistral OCR** (Default) ### 1. Mistral OCR (Default)
2. **Azure Mistral OCR** (For Azure-deployed models)
3. **Custom OCR** (Planned for future releases)
### Basic Configuration Example The default strategy uses Mistral's cloud API service directly. This is the simplest setup and requires only an API key from Mistral.
If using the Mistral OCR API, you only need the following environment variables to get started:
**Environment Variables:**
```.env ```.env
# `.env` # `.env`
OCR_API_KEY=your-mistral-api-key OCR_API_KEY=your-mistral-api-key
# OCR_BASEURL=https://api.mistral.ai/v1 # this is the default value # OCR_BASEURL=https://api.mistral.ai/v1 # this is the default value
``` ```
For additional, detailed configuration options, see the [OCR Config Object Structure](/docs/configuration/librechat_yaml/object_structure/ocr). **Configuration:**
```yaml ```yaml
# `librechat.yaml` # `librechat.yaml`
ocr: ocr:
mistralModel: "mistral-ocr-latest" # Optional: Specify Mistral model, defaults to "mistral-ocr-latest" mistralModel: "mistral-ocr-latest" # Optional: Specify Mistral model, defaults to "mistral-ocr-latest"
apiKey: "your-mistral-api-key" # Optional: Defaults to OCR_API_KEY env variable apiKey: "your-mistral-api-key" # Optional: Defaults to OCR_API_KEY env variable
baseURL: "https://api.mistral.ai/v1" # Optional: Defaults to OCR_BASEURL env variable, or Mistral's API if no variable set baseURL: "https://api.mistral.ai/v1" # Optional: Defaults to OCR_BASEURL env variable, or Mistral's API if no variable set
strategy: "mistral_ocr" # Optional: Defaults to "mistral_ocr" strategy: "mistral_ocr" # Optional: Defaults to "mistral_ocr"
``` ```
### Azure Mistral OCR Configuration **Key Features:**
For organizations using Azure AI Foundry, you can deploy Mistral OCR models to your Azure infrastructure:
```yaml
# `librechat.yaml`
ocr:
mistralModel: "deployed-mistral-ocr-2503" # Should match your Azure deployment name
apiKey: "${AZURE_MISTRAL_OCR_API_KEY}" # Reference to your Azure API key in .env
baseURL: "https://your-deployed-endpoint.models.ai.azure.com/v1" # Your Azure endpoint
strategy: "azure_mistral_ocr" # Use Azure strategy
```
## Mistral OCR
LibreChat supports Mistral's OCR capabilities through two deployment options:
### 1. Standard Mistral OCR API
The default option uses Mistral's cloud API service directly. This is the simplest setup and requires only an API key from Mistral.
### 2. Azure-Deployed Mistral OCR
For organizations with specific data residency or compliance requirements, Mistral OCR models can be deployed on Azure AI Foundry. Currently, the **Mistral OCR 2503** model is available for Azure deployment.
Benefits of Azure deployment:
- **Data Residency**: Keep data processing within your Azure region
- **Compliance**: Meet organizational security and compliance requirements
- **Integration**: Leverage existing Azure infrastructure and security policies
- **Control**: Manage your own deployment and scaling
You can explore the latest Mistral OCR model available on Azure AI Foundry here (requires Azure subscription):
https://ai.azure.com/explore/models/mistral-ocr-2503
### Key Features of Mistral OCR
- **Document Structure Preservation**: Maintains formatting like headers, paragraphs, lists, and tables - **Document Structure Preservation**: Maintains formatting like headers, paragraphs, lists, and tables
- **Multilingual Support**: Processes text in multiple languages and scripts - **Multilingual Support**: Processes text in multiple languages and scripts
- **Complex Layout Handling**: Handles multi-column text and mixed content - **Complex Layout Handling**: Handles multi-column text and mixed content
- **Mathematical Expression Recognition**: Accurately processes equations and formulas - **Mathematical Expression Recognition**: Accurately processes equations and formulas
- **High-Speed Processing**: Processes up to 2000 pages per minute - **High-Speed Processing**: Processes up to 2000 pages per minute
### Important Considerations **Considerations:**
- **Cost**: Using Mistral OCR may incur costs as it's a paid API service (though free trials may be available) - **Cost**: Using Mistral OCR may incur costs as it's a paid API service (though free trials may be available)
- **Data Privacy**: Data processed through Mistral OCR is subject to Mistral's cloud environment and their terms of service - **Data Privacy**: Data processed through Mistral OCR is subject to Mistral's cloud environment and their terms of service
- **Document Limitations**: - **Document Limitations**:
- Maximum file size: 50 MB - Maximum file size: 50 MB
- Maximum document length: 1,000 pages - Maximum document length: 1,000 pages
### Future Plans ### 2. Azure Mistral OCR
- Mistral plans to make their OCR API available through their cloud partners, such as GCP and AWS, and enterprise self-hosting for organizations with stringent data privacy requirements ([source](https://mistral.ai/fr/news/mistral-ocr)). For organizations using Azure AI Foundry, you can deploy Mistral OCR models to your Azure infrastructure. Currently, the **Mistral OCR 2503** model is available for Azure deployment.
- LibreChat will continue to support Mistral OCR and explore additional OCR providers, including open-source solutions, for enhanced functionality.
- LibreChat currently does not include the parsed image content from the OCR process in its responses, even though services like [Mistral's OCR API may provide](https://docs.mistral.ai/api/#tag/ocr) these in the result. This feature may be supported in future updates. **Configuration:**
```yaml
# `librechat.yaml`
ocr:
mistralModel: "deployed-mistral-ocr-2503" # Should match your Azure deployment name
apiKey: "${AZURE_MISTRAL_OCR_API_KEY}" # Reference to your Azure API key in .env
baseURL: "https://your-deployed-endpoint.models.ai.azure.com/v1" # Your Azure endpoint
strategy: "azure_mistral_ocr" # Use Azure strategy
```
**Benefits:**
- **Data Residency**: Keep data processing within your Azure region
- **Compliance**: Meet organizational security and compliance requirements
- **Integration**: Leverage existing Azure infrastructure and security policies
- **Control**: Manage your own deployment and scaling
**Azure Model Information:**
You can explore the latest Mistral OCR model available on Azure AI Foundry here (requires Azure subscription):
https://ai.azure.com/explore/models/mistral-ocr-2503
### 3. Custom OCR (Planned)
Support for custom OCR providers and user-defined strategies is planned for future releases.
## Detailed Configuration
For additional, detailed configuration options, see the [OCR Config Object Structure](/docs/configuration/librechat_yaml/object_structure/ocr).
## Using File Context (OCR) in LibreChat ## Using File Context (OCR) in LibreChat
@@ -160,6 +146,8 @@ LibreChat plans to expand OCR capabilities in future releases:
- Integration with open-source OCR solutions - Integration with open-source OCR solutions
- Enhanced document processing options - Enhanced document processing options
- More granular control over OCR settings - More granular control over OCR settings
- Mistral plans to make their OCR API available through their cloud partners, such as GCP and AWS, and enterprise self-hosting for organizations with stringent data privacy requirements ([source](https://mistral.ai/fr/news/mistral-ocr))
- LibreChat currently does not include the parsed image content from the OCR process in its responses, even though services like [Mistral's OCR API may provide](https://docs.mistral.ai/api/#tag/ocr) these in the result. This feature may be supported in future updates.
--- ---

Binary file not shown.

After

Width:  |  Height:  |  Size: 23 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 21 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 22 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 20 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 113 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 143 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 71 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 245 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 90 KiB