centersl/nextcloud-docs
1
0
Fork 0
mirror of https://github.com/nextcloud/documentation.git synced 2026-03-26 13:28:45 +07:00
Code Issues Packages Projects Releases Wiki Activity

feat(AI/LiveTranscription): add live translation docs (#14045)

Browse Source 
Signed-off-by: Anupam Kumar <kyteinsky@gmail.com>
This commit is contained in:
Anupam Kumar
2026-02-03 15:20:17 +05:30
committed by GitHub
parent 1fd0759b93
commit 99b2c30ff2
3 changed files with 45 additions and 19 deletions
Download Patch File Download Diff File Expand all files Collapse all files

59
admin_manual/ai/app_live_transcription.rst
View File

@@ -1,13 +1,14 @@
============================================================== ==============================================================================
App: Live Transcription in Nextcloud Talk (live_transcription) App: Live Transcription and Translation in Nextcloud Talk (live_transcription)
============================================================== ==============================================================================
.. _ai-live-transcription: .. _ai-live-transcription:
This app provides live transcription of speech in Nextcloud Talk calls using open source AI models provided by `Vosk <https://alphacephei.com/vosk/>`_. | This app provides live transcription and translation of speech in Nextcloud Talk calls using open source AI models provided by `Vosk <https://alphacephei.com/vosk/>`_.
The transcription is done on your own server, preserving your privacy and data sovereignty. | The transcription is done on your own server, preserving your privacy and data sovereignty, while the translation is done using a translation task processing provider like the :ref:`translate2 app <ai-app-translate2>`. `OpenAI and LocalAI integration <https://apps.nextcloud.com/apps/integration_openai>`_ and `DeepL integration <http://apps.nextcloud.com/apps/integration_deepl>`_ apps will soon also be supported for translation.
A good set of language models are auto-downloaded. They include Arabic, Arabic (Tunisian), Breton, Catalan, Czech, German, English, Esperanto, Spanish, Persian (Farsi), French, Hindi, Italian, Japanese, Kazakh, Korean, Dutch, Polish, Portuguese (Brazilian), Russian, Telegu, Tajik, Turkish, Ukrainian, Uzbek, Vietnamese and Chinese. | A good set of language models for transcription are auto-downloaded. They include Arabic, Arabic (Tunisian), Breton, Catalan, Czech, German, English, Esperanto, Spanish, Persian (Farsi), French, Hindi, Italian, Japanese, Kazakh, Korean, Dutch, Polish, Portuguese (Brazilian), Russian, Telegu, Tajik, Turkish, Ukrainian, Uzbek, Vietnamese and Chinese.
| The translation capabilities depend on the installed translation task processing provider app. A list of translation-capable apps can be found :ref:`here <mt-consumer-apps>` in the "Backend apps" section.
Installation Installation
------------ ------------
@@ -24,21 +25,42 @@ Installation
--env LT_INTERNAL_SECRET=1234 \ --env LT_INTERNAL_SECRET=1234 \
--wait-finish --wait-finish
.. important::
The environment variables ``LT_HPB_URL`` and ``LT_INTERNAL_SECRET`` must be set in the :ref:`Deploy Options <ai-app_api_deploy_options>` during installation,
and the High-Performance Backend must be functionally configured in Nextcloud Talk settings for the app to work.
Changing these environment variables after installation is possible through a re-installation of the app after uninstalling it first.
5. Install a Text-to-text task processing provider app for translation capabilities from the "Backend apps" section :ref:`here <mt-consumer-apps>`.
Requirements
------------
* Minimal Nextcloud version: 33
* Nextcloud AIO is supported
* We currently support NVIDIA GPUs and x86_64 CPUs. Only CPU-based transcription is also supported and works well on modern x86 CPUs.
* CUDA >= v12.4.1 on your host system for GPU-based transcription
* GPU Sizing
* A NVIDIA GPU with at least 10 GB VRAM
* 16 GB of system RAM should be enough for one or two concurrent calls
* CPU Sizing
* x86 CPU with 4 threads. Additional 2 threads per concurrent call.
* 16 GB of RAM should be enough for one or two concurrent calls
* Space usage
* ~ 2.8 GB for the docker container
* ~ 6.0 GB for the default models
.. note:: .. note::
Environment variables and mounts can be set during the app installation from the "Deploy Options" button. We currently have very little real-world experience running this software on production instances.
The models are stored in a persistent volume at ``/nc_app_live_transcription_data``. The above sizing recommendations come from our estimates and are not real-world benchmarks.
This volume is created automatically during the installation but you can also mount your own volume there. Actual requirements will vary based on factors such as the number of concurrent calls, audio quality, and selected languages.
As the name suggests, this volume is persistent and will not be deleted when the app is updated or uninstalled Please do thorough testing to confirm your hardware meets your needs.
(without removing data).
.. important::
The environment variables ``LT_HPB_URL`` and ``LT_INTERNAL_SECRET`` must be set in the Deploy Options,
and the High-Performance Backend must be functionally configured in Nextcloud Talk settings for the app to work.
App store App store
--------- ---------
@@ -59,3 +81,4 @@ Limitations
* The app currently supports only a limited number of languages. More languages may be added in the future. * The app currently supports only a limited number of languages. More languages may be added in the future.
* The languages other than English may have lower accuracy mainly due to the shipped models being smaller. * The languages other than English may have lower accuracy mainly due to the shipped models being smaller.
* The app currently does not support punctuation in the transcription. * The app currently does not support punctuation in the transcription.
* `OpenAI and LocalAI integration <https://apps.nextcloud.com/apps/integration_openai>`_ and `DeepL integration <http://apps.nextcloud.com/apps/integration_deepl>`_ apps are not yet supported for translation.

3
admin_manual/ai/overview.rst
View File

@@ -137,13 +137,14 @@ Frontend apps
* *Text* for offering the translation menu * *Text* for offering the translation menu
* `Assistant <https://apps.nextcloud.com/apps/assistant>`_ offering a graphical translation UI * `Assistant <https://apps.nextcloud.com/apps/assistant>`_ offering a graphical translation UI
* `Analytics <https://apps.nextcloud.com/apps/analytics>`_ for translating graph labels * `Analytics <https://apps.nextcloud.com/apps/analytics>`_ for translating graph labels
* `Talk <https://apps.nextcloud.com/apps/spreed>`_ for translating messages and live translations in calls in conjunction with the :ref:`Live Transcription app <ai-live-transcription>`
Backend apps Backend apps
~~~~~~~~~~~~ ~~~~~~~~~~~~
* :ref:`translate2 (ExApp)<ai-app-translate2>` - Runs open source AI translation models locally on your own server hardware (Customer support available upon request) * :ref:`translate2 (ExApp)<ai-app-translate2>` - Runs open source AI translation models locally on your own server hardware (Customer support available upon request)
* `OpenAI and LocalAI integration (via OpenAI API) <https://apps.nextcloud.com/apps/integration_openai>`_ - Integrates with the OpenAI API to provide AI functionality from OpenAI servers (Customer support available upon request; see :ref:`AI as a Service<ai-ai_as_a_service>`) * `OpenAI and LocalAI integration (via OpenAI API) <https://apps.nextcloud.com/apps/integration_openai>`_ - Integrates with the OpenAI API to provide AI functionality from OpenAI servers (Customer support available upon request; see :ref:`AI as a Service<ai-ai_as_a_service>`)
* *integration_deepl* - Integrates with the deepl API to provide translation functionality from Deepl.com servers (Only community supported) * `DeepL integration <http://apps.nextcloud.com/apps/integration_deepl>`__ - Integrates with the deepl API to provide translation functionality from Deepl.com servers (Only community supported)
Speech-To-Text Speech-To-Text
^^^^^^^^^^^^^^ ^^^^^^^^^^^^^^

2
admin_manual/exapps_management/AdvancedDeployOptions.rst
View File

@@ -2,6 +2,8 @@
Advanced Deploy Options Advanced Deploy Options
======================= =======================
.. _ai-app_api_deploy_options:
AppAPI allows optionally to configure environment variables and mounts for the ExApp container. AppAPI allows optionally to configure environment variables and mounts for the ExApp container.
It is available via "Deploy options" modal next to "Deploy and Enable" button in the sidebar of the ExApp page on the Apps management page: It is available via "Deploy options" modal next to "Deploy and Enable" button in the sidebar of the ExApp page on the Apps management page: