---
title: Integrate Local Models Deployed by GPUStack
---

[GPUStack](https://github.com/gpustack/gpustack) is an open-source GPU cluster manager for running AI models.

Dify allows integration with GPUStack for local deployment of large language model inference, embedding, reranking, speech to text and text to speech to capabilities.

## Deploying GPUStack

You can refer to the official [Documentation](https://docs.gpustack.ai) for deployment, or quickly integrate following the steps below:

### Linux or MacOS

GPUStack provides a script to install it as a service on systemd or launchd based systems. To install GPUStack using this method, just run:

```bash
curl -sfL https://get.gpustack.ai | sh -s -
```

### Windows

Run PowerShell as administrator (**avoid** using PowerShell ISE), then run the following command to install GPUStack:

```powershell
Invoke-Expression (Invoke-WebRequest -Uri "https://get.gpustack.ai" -UseBasicParsing).Content
```

Then you can follow the printed instructions to access the GPUStack UI.

## Deploying LLM

Using a LLM hosted on GPUStack as an example:

1. In GPUStack UI, navigate to the "Models" page and click on "Deploy Model", choose `Hugging Face` from the dropdown.

2. Use the search bar in the top left to search for the model name `Qwen/Qwen2.5-0.5B-Instruct-GGUF`.

3. Click `Save` to deploy the model.

![gpustack-deploy-llm](https://assets-docs.dify.ai/2025/04/d19b69596b95ae94071f9430a1b983e6.png)

## Create an API Key

1. Navigate to the "API Keys" page and click on "New API Key".

2. Fill in the name, then click `Save`.

3. Copy the API key and save it for later use.

## Integrating GPUStack into Dify

5. Go to `Settings > Model Providers > GPUStack` and fill in:

   - Model Type: `LLM`

   - Model Name: `qwen2.5-0.5b-instruct`

   - Server URL: `http://your-gpustack-server-ip`

   - API Key: `Input the API key you copied from previous steps`

   Click "Save" to use the model in the application.

![add-gpustack-llm](https://assets-docs.dify.ai/2025/04/8178418f93ccea1668564d8bd9777303.png)

For more information about GPUStack, please refer to [GitHub Repo](https://github.com/gpustack/gpustack).

{/*
Contributing Section
DO NOT edit this section!
It will be automatically generated by the script.
*/}

---

[Edit this page](https://github.com/langgenius/dify-docs/edit/main/en/development/models-integration/gpustack.mdx) | [Report an issue](https://github.com/langgenius/dify-docs/issues/new?template=docs.yml)

