From 5c4ac5210db8df39c9ba0a2852e6f5f9167006f2 Mon Sep 17 00:00:00 2001 From: Marcel Klehr Date: Fri, 9 May 2025 12:03:48 +0200 Subject: [PATCH 1/3] enh(context_agent): Correct model requirements llm2 now supports tool calling with llama 3.1 Signed-off-by: Marcel Klehr --- admin_manual/ai/app_context_agent.rst | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/admin_manual/ai/app_context_agent.rst b/admin_manual/ai/app_context_agent.rst index 16d722dac..e95fed744 100644 --- a/admin_manual/ai/app_context_agent.rst +++ b/admin_manual/ai/app_context_agent.rst @@ -108,8 +108,10 @@ Installation Model requirements ~~~~~~~~~~~~~~~~~~ -This app requires underlying Large language models to support tool calling. The default model in *llm2* does *not* support tool calling. Instead we recommend: +This app requires underlying Large language models to support tool calling. The default model in *llm2* does support tool calling since version 2.4.0. +Other models that may give good results are: +* Google Gemma 3 12B or higher * Mistral 3 small 24B * Qwen 2.5 8B or higher (May not work well with languages other than English) * Watt Tool 8B or higher From ad2bedce7c072c53ddbf517b1b41d4515d5596ce Mon Sep 17 00:00:00 2001 From: Marcel Klehr Date: Fri, 9 May 2025 12:04:22 +0200 Subject: [PATCH 2/3] enh(llm2): Update required CUDA version Signed-off-by: Marcel Klehr --- admin_manual/ai/app_llm2.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/admin_manual/ai/app_llm2.rst b/admin_manual/ai/app_llm2.rst index f01379be7..1769acb23 100644 --- a/admin_manual/ai/app_llm2.rst +++ b/admin_manual/ai/app_llm2.rst @@ -39,7 +39,7 @@ Requirements * Nextcloud AIO is supported * We currently support NVIDIA GPUs and x86_64 CPUs * CPU that supports AVX and AVX2 instruction -* CUDA >= v12.2 on your host system +* CUDA >= v12.8 on your host system * GPU Sizing * A NVIDIA GPU with at least 8GB VRAM From 6ac91acbdab3086bad302c81590b624b2b41d977 Mon Sep 17 00:00:00 2001 From: Marcel Klehr Date: Fri, 9 May 2025 12:04:35 +0200 Subject: [PATCH 3/3] enh(stt_whisper2): Runs on GPU as well now Signed-off-by: Marcel Klehr --- admin_manual/ai/app_stt_whisper2.rst | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/admin_manual/ai/app_stt_whisper2.rst b/admin_manual/ai/app_stt_whisper2.rst index 3dc259b38..fb4d1418c 100644 --- a/admin_manual/ai/app_stt_whisper2.rst +++ b/admin_manual/ai/app_stt_whisper2.rst @@ -21,7 +21,11 @@ Requirements * Minimal Nextcloud version: 28 * This app is built as an External App and thus depends on AppAPI v2.3.0 * Nextcloud AIO is supported -* Using GPU is currently not supported +* We currently support NVIDIA GPUs and x86_64 CPUs +* CUDA >= v12.2 on your host system +* GPU Sizing + + * A NVIDIA GPU with at least 4GB VRAM * CPU Sizing