From d340667a55a382fdfc568132a8f16dadb1705730 Mon Sep 17 00:00:00 2001 From: Marcel Klehr Date: Mon, 3 Jun 2024 15:58:13 +0200 Subject: [PATCH] fix(admin/AI): llm2 & stt_whisper2 still don't support GPU yet Signed-off-by: Marcel Klehr --- admin_manual/ai/app_llm2.rst | 11 +---------- admin_manual/ai/app_stt_whisper2.rst | 12 +----------- 2 files changed, 2 insertions(+), 21 deletions(-) diff --git a/admin_manual/ai/app_llm2.rst b/admin_manual/ai/app_llm2.rst index 2c78c11e4..bcc71aeb7 100644 --- a/admin_manual/ai/app_llm2.rst +++ b/admin_manual/ai/app_llm2.rst @@ -16,19 +16,10 @@ Requirements * This app is built as an External App and thus depends on AppAPI v2.3.0 * Nextcloud AIO is supported -* Using GPU processing is supported, but not required; be prepared for slow performance unless you are using GPU -* We currently only support NVIDIA GPUs -* GPU Sizing - - * You will need a GPU with enough VRAM to hold the model you choose - - * for 7B parameter models, 5bit-quantized variants and lower should fit on a 8GB VRAM, but of course have lower quality - * for 7B parameter models, 6bit-quantized variants and up will need 12GB VRAM - * If you want better reasoning capabilities, you will need to look for models with more parameters, like 14B and higher, which of course also need more VRAM +* Using GPU is currently not supported * CPU Sizing - * If you don't have a GPU, this app will utilize your CPU cores * The more cores you have and the more powerful the CPU the better, we recommend 10-20 cores * The app will hog all cores by default, so it is usually better to run it on a separate machine diff --git a/admin_manual/ai/app_stt_whisper2.rst b/admin_manual/ai/app_stt_whisper2.rst index 4ef327dd0..e49a7a88b 100644 --- a/admin_manual/ai/app_stt_whisper2.rst +++ b/admin_manual/ai/app_stt_whisper2.rst @@ -17,20 +17,10 @@ Requirements * Minimal Nextcloud version: 28 * This app is built as an External App and thus depends on AppAPI v2.3.0 * Nextcloud AIO is supported -* Using GPU processing is supported, but not required; be prepared for slow performance unless you are using GPU -* We currently only support NVIDIA GPUs -* GPU Sizing - - * You will need a GPU with enough VRAM to hold the model you choose - - * the small model should fit on 2GB VRAM - * the large-v2 (the best and largest) will need 6GB VRAM - - * The distil-whisper variants have half the parameters of the original models while supposedly staying within 1% of the original error rate (your mileage may vary) +* Using GPU is currently not supported * CPU Sizing - * If you don't have a GPU, this app will utilize your CPU cores * The more cores you have and the more powerful the CPU the better, we recommend 10-20 cores * The app will hog all cores by default, so it is usually better to run it on a separate machine