centersl/docker-docs
1
0
Fork 0
mirror of https://github.com/docker/docs.git synced 2026-03-27 06:18:55 +07:00
Code Issues Packages Projects Releases Wiki Activity

model-runner: document inference engine support and GitHub issue reporting

Browse Source 
Signed-off-by: Dorin Geman <dorin.geman@docker.com>
This commit is contained in:
Dorin Geman
2025-12-04 17:44:37 +02:00
parent a55bf1ba7b
commit e9f52993bc
2 changed files with 10 additions and 10 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
_vale/config/vocabularies/Docker/accept.txt
View File

@@ -170,6 +170,7 @@ Quickview
rebalance
reimplement
Rekor
ROCm
rollback
rootful
runc
@@ -213,6 +214,7 @@ Visual Studio Code
VMware
vpnkit
vSphere
Vulkan
Vue
Wasm
Wasmtime

18
content/manuals/ai/model-runner/_index.md
View File

@@ -6,7 +6,7 @@ params:
group: AI
weight: 30
description: Learn how to use Docker Model Runner to manage and run AI models.
keywords: Docker, ai, model runner, docker desktop, docker engine, llm
keywords: Docker, ai, model runner, docker desktop, docker engine, llm, openai, llama.cpp, vllm, cpu, nvidia, cuda, amd, rocm, vulkan
aliases:
- /desktop/features/model-runner/
- /model-runner/
@@ -34,7 +34,8 @@ with AI models locally.
- [Pull and push models to and from Docker Hub](https://hub.docker.com/u/ai)
- Serve models on OpenAI-compatible APIs for easy integration with existing apps
- Package GGUF files as OCI Artifacts and publish them to any Container Registry
- Support for both llama.cpp and vLLM inference engines (vLLM currently supported on Linux x86_64/amd64 with NVIDIA GPUs only)
- Package GGUF and Safetensors files as OCI Artifacts and publish them to any Container Registry
- Run and interact with AI models directly from the command line or from the Docker Desktop GUI
- Manage local models and display logs
- Display prompt and response details
@@ -68,8 +69,8 @@ Windows(arm64):
Docker Engine only:
- Linux CPU & Linux NVIDIA
- NVIDIA drivers 575.57.08+
- Supports CPU, NVIDIA (CUDA), AMD (ROCm), and Vulkan backends
- Requires NVIDIA driver 575.57.08+ when using NVIDIA GPUs
{{< /tab >}}
{{</tabs >}}
@@ -83,6 +84,8 @@ initial pull may take some time. After that, they're cached locally for faster
access. You can interact with the model using
[OpenAI-compatible APIs](api-reference.md).
Docker Model Runner supports both [llama.cpp](https://github.com/ggerganov/llama.cpp) and [vLLM](https://github.com/vllm-project/vllm) as inference engines, providing flexibility for different model formats and performance requirements. For more details, see the [Docker Model Runner repository](https://github.com/docker/model-runner).
> [!TIP]
>
> Using Testcontainers or Docker Compose?
@@ -111,14 +114,9 @@ $ ln -s /Applications/Docker.app/Contents/Resources/cli-plugins/docker-model ~/.
Once linked, rerun the command.
### No consistent digest support in Model CLI
The Docker Model CLI currently lacks consistent support for specifying models by image digest. As a temporary workaround, you should refer to models by name instead of digest.
## Share feedback
Thanks for trying out Docker Model Runner. Give feedback or report any bugs
you may find through the **Give feedback** link next to the **Enable Docker Model Runner** setting.
Thanks for trying out Docker Model Runner. To report bugs or request features, [open an issue on GitHub](https://github.com/docker/model-runner/issues). You can also give feedback through the **Give feedback** link next to the **Enable Docker Model Runner** setting.
## Next steps