diff --git a/admin_manual/configuration_files/index.rst b/admin_manual/configuration_files/index.rst index 87e6b689b..82cb5b9e9 100644 --- a/admin_manual/configuration_files/index.rst +++ b/admin_manual/configuration_files/index.rst @@ -5,15 +5,14 @@ File sharing and management .. toctree:: :maxdepth: 2 - + file_sharing_configuration federated_cloud_sharing_configuration big_file_upload_configuration - default_files_configuration + default_files_configuration external_storage_configuration_gui external_storage_configuration external_storage/auth_mechanisms - primary_storage encryption_configuration files_locking_transactional previews_configuration diff --git a/admin_manual/configuration_files/primary_storage.rst b/admin_manual/configuration_files/primary_storage.rst deleted file mode 100644 index 6035135a0..000000000 --- a/admin_manual/configuration_files/primary_storage.rst +++ /dev/null @@ -1,131 +0,0 @@ -=============== -Primary Storage -=============== - -It's possible to use an object store as primary storage, this replaces the default -way of storing files in :code:`nextcloud/data` (note that the data directory might still be used -for other reasons) - -Implications ------------- - -When using an object store as primary storage, Nextcloud assumes exclusive access -over the bucket being used. - -Contrary to using an object store as external storage, when an object store is used -as primary storage, no metadata (names, directory structures, etc) is stored in the -object store. The metadata is only stored in the database and the object store only -holds the file content by unique identifier. - -Because of this primary object stores usually perform better than when using the same -object store as external storage but it restricts being able to access the files from -outside of Nextcloud. - -Configuring ------------ - -Primary object stores need to be configured in :code:`config.php` by specifying the objectstore -backend and any backend specific configuration. - -.. note:: Configuring a primary object store on an existing Nextcloud instance will - make all existing files on the instance inaccessible. - -The configuration has the following structure. - -:: - - 'objectstore' => array( - 'class' => 'Object\\Storage\\Backend\\Class', - 'arguments' => array( - ... - ), - ), - -Openstack Swift -^^^^^^^^^^^^^^^ - -The Swift backend mounts a container on an OpenStack Object Storage server into the virtual filesystem. The class to be used is :code:`\\OC\\Files\\ObjectStore\\Swift` - -:: - - 'objectstore' => array( - 'class' => 'OC\\Files\\ObjectStore\\Swift', - 'arguments' => array( - 'username' => 'username', - 'password' => 'Secr3tPaSSWoRdt7', - // the container to store the data in - 'bucket' => 'nextcloud', - 'autocreate' => true, - 'region' => 'RegionOne', - // The Identity / Keystone endpoint - 'url' => 'http://example.com/v2.0', - // optional on some swift implementations - 'tenantName' => 'username', - 'serviceName' => 'swift', - // The Interface / url Type, optional - 'urlType' => 'internal' - ), - ), - -Amazon S3 ---------- - -The S3 backend mounts a bucket on an Amazon S3 Storage or compatible server into the virtual filesystem. The class to be used is :code:`\\OC\\Files\\ObjectStore\\S3` - -:: - - 'objectstore' => array( - 'class' => 'OC\\Files\\ObjectStore\\S3', - 'arguments' => array( - 'bucket' => 'nextcloud', - 'autocreate' => true, - 'key' => 'EJ39ITYZEUH5BGWDRUFY', - 'secret' => 'M5MrXTRjkyMaxXPe2FRXMTfTfbKEnZCu+7uRTVSj', - 'hostname' => 'example.com', - 'port' => 1234, - 'use_ssl' => true, - 'region' => 'optional', - // required for some non amazon s3 implementations - 'use_path_style' => true, - 'legacy_auth' => false - ), - ), - -Not all configuration options are required for all S3 servers. -Overriding the hostname, port and region of your S3 server is only -required for non-Amazon servers such as Ceph Object Gateway, which in turn -usually don't require the region to be set. - -:code:`use_path_style` is usually not required (and is, in fact, incompatible with newer Amazon datacenters), -but can be used with non-Amazon servers where the DNS infrastructure cannot be controlled. Ordinarily, -requests will be made with http://bucket.hostname.domain/, but with path style enabled, -requests are made with http://hostname.domain/bucket instead. - -:code:`legacy_auth` is only required for S3 servers that only implement version 2 authentication, -on default version 4 authentication will be used. - -Multibucket Object Store ---------------------------- - -It's possible to configure Nextcloud to distribute it's data over multiple buckets for scalability purpose. - -To setup multiple buckets, :code:`config.php` needs to be configured using :code:`'objectstore_multibucket'` - -:: - - 'objectstore_multibucket' => array( - 'class' => 'Object\\Storage\\Backend\\Class', - 'arguments' => array( - // optional, defaults to 64 - 'num_buckets' => 64, - // will be prefixed by an integer in the range from 0 to (num_nuckets-1) - 'bucket' => 'nextcloud_', - ... - ), - ), - -Nextcloud will map every user to a range of buckets and save all files for that user in it's respective bucket. - -.. note:: Changing the number of buckets for an existing Nextcloud instance is supported but the - mapping from users to buckets is persistent so only newly created users will be mapped to the - updated range of buckets. diff --git a/admin_manual/configuration_server/custom_client_repos.rst b/admin_manual/configuration_server/custom_client_repos.rst deleted file mode 100644 index 479d9e079..000000000 --- a/admin_manual/configuration_server/custom_client_repos.rst +++ /dev/null @@ -1,27 +0,0 @@ -=================================== -Custom client download repositories -=================================== - -You may configure the URLs to your own download repositories for your Nextcloud -desktop clients and mobile apps in :file:`config/config.php`. This example shows -the default download locations: - -:: - - "https://nextcloud.com/install/", - "customclient_android" => "https://play.google.com/store/apps/details?id=com.nextcloud.client", - "customclient_ios" => "https://itunes.apple.com/us/app/nextcloud/id1125420102?mt=8", - -Simply replace the URLs with the links to your own preferred download repos. - -You may test alternate URLs without editing :file:`config/config.php` by setting a test URL as an environment variable:: - - export OCC_UPDATE_URL=https://test.example.com - -When you're finished testing you can disable the environment variable:: - - unset OCC_UPDATE_URL - - diff --git a/admin_manual/configuration_server/index.rst b/admin_manual/configuration_server/index.rst index 3004c2eb0..c249b02a3 100644 --- a/admin_manual/configuration_server/index.rst +++ b/admin_manual/configuration_server/index.rst @@ -8,18 +8,15 @@ Server configuration security_setup_warnings occ_command activity_configuration - sso_configuration caching_configuration - background_jobs_configuration + background_jobs_configuration config_sample_php_parameters email_configuration external_sites - custom_client_repos - knowledgebase_configuration language_configuration logging_configuration harden_server - reverse_proxy_configuration + reverse_proxy_configuration thirdparty_php_configuration automatic_configuration server_tuning diff --git a/admin_manual/configuration_server/knowledgebase_configuration.rst b/admin_manual/configuration_server/knowledgebase_configuration.rst deleted file mode 100644 index 002079dd3..000000000 --- a/admin_manual/configuration_server/knowledgebase_configuration.rst +++ /dev/null @@ -1,20 +0,0 @@ -============================ -Knowledge base configuration -============================ -The usage of Nextcloud is more or less self explaining but nevertheless a user -might run into a problem where he needs to consult the documentation or knowledge base. To ease access to the Nextcloud -documentation and knowledge base, a help menu item is shown in the settings menu by default. - -Parameters ----------- - -If you want to disable the Nextcloud help menu item you can use the **knowledgebaseenabled** parameter inside the -:file:`config/config.php`. - -:: - - true, - -.. note:: Disabling the help menu item might increase the number of support requests you have to answer in the future diff --git a/admin_manual/configuration_server/sso_configuration.rst b/admin_manual/configuration_server/sso_configuration.rst deleted file mode 100644 index 42964e724..000000000 --- a/admin_manual/configuration_server/sso_configuration.rst +++ /dev/null @@ -1,69 +0,0 @@ -========================== -Configuring single-sign-on -========================== - -Using the SSO & SAML app of your Nextcloud you can make it easily possible to integrate your existing Single-Sign-On -solution with Nextcloud. In addition, you can use the Nextcloud LDAP user provider to keep the convenience for users. (e.g. -when sharing) - -The following providers are supported and tested at the moment: - -- SAML 2.0 - - OneLogin - - Shibboleth - - Active Directory Federation Services (ADFS) -- Authentication via Environment Variable - - Kerberos (mod_auth_kerb) - - Any other provider that authenticates using the environment variable - -While theoretically any other authentication provider implementing either one of those standards is compatible, we like -to note that they are not part of any internal test matrix. - -Enabling the SSO & SAML app ---------------------------- - -.. warning:: Make sure to configure an administrative user that can access the instance via SSO. Logging-in with your - regular Nextcloud account won't be possible anymore. - - -The "SSO & SAML" App is shipped and disabled by default. To enable the app enabled simply go to your Nextcloud Apps page -to enable it. It can then be found in the "SSO & SAML authentication" section of your Nextcloud. - -Configuring SAML 2.0 --------------------- - -To configure using SAML choose the "SAML authentication" in the setup wizard of the application. Then configure the application -as required by your Service Provider. - - .. figure:: ./images/saml_app_overview.png - - -Configuring environment based authentication --------------------------------------------- -It is possible to authenticate against Nextcloud using an environment variable. This is for example relevant in case you -use an service provider incompatible with SAML such as Kerberos or don't want to configure SAML in the software yourself. - -To enable that choose the "Environment variable" authentication provider in the application and then specify the environment -variable. (e.g. `REMOTE_USER` for Kerberos) - -Once done you also need to protect the login route properly. On an Apache server with mod_auth_kerb the following configuration -would protect the login route: - -.. code-block:: apache - - - AuthType Kerberos - AuthName "Kerberos Login" - KrbServiceName HTTP - KrbMethodNegotiate On - KrbMethodK5Passwd Off - KrbSaveCredentials Off - KrbVerifyKDC On - KrbAuthRealms NEXTCLOUD-AD.LOCAL - Krb5KeyTab /etc/apache2/webpage.HTTP.keytab - Require valid-user - - - -.. warning:: If this authentication approach is used clients do require an application specific password for authentication. - A better integration into our desktop and mobile clients is considered for the future though. diff --git a/admin_manual/file_workflows/access_control.rst b/admin_manual/file_workflows/access_control.rst deleted file mode 100644 index 26f38db85..000000000 --- a/admin_manual/file_workflows/access_control.rst +++ /dev/null @@ -1,117 +0,0 @@ -==================== -Files access control -==================== - -Nextcloud's File Access Control app enables administrators to create and -manage a set of rule groups. Each of the rule groups consists of one or more -rules. If all rules of a group hold true, the group matches the request and -access is being denied. The rules criteria range from IP address, to user -groups, collaborative tags and :ref:`some more `. - -Denied access -------------- - -If access to a file has been denied for a user, the user can not: - -* Create/upload the file -* Modify the files -* Delete the file -* Download the file -* Syncronise the file with clients, such as the Nextcloud desktop and mobile clients - -Examples --------- - - .. figure:: images/files_access_control_sample_rules.png - :alt: Sample rules to block on user group, time and IP base. - -The first rule group ``Support only 9-5`` denies any access to files for users -of the Support user group, between 5pm and 9am. - -The second rule group ``Internal testing`` prevents users of the Internal -testers group to access files from outside of the local network. - -Denying access to folders -------------------------- - -The easiest way to block access to a folder, is to use a collaborative tag. As -mentioned in the :ref:`Available rules ` section below, -either the file itself or one of the parents needs to have the given tag -assigned. - -So you just need to assign the tag to the folder or file, and then block the -tag with a rule group. The check is independent of the user's permissions for -the tag. Therefor restricted and invisible tags are recommended, otherwise a -user could remove and reassign the tag. - -This example blocks access to any folder with the tag ``Confidential``. - - .. figure:: images/files_access_control_collaborative_tags.png - :alt: Deny access based on collaborative tag - -Prevent uploading of specific files ------------------------------------ - -It's possible to prevent specific files from being uploaded to Nextcloud. You -simply need to define a rule based on the mimetype and our powerful access control -engine will block any attempt to upload the file. The safest way to define the rule -is to use a regular expression, as it will help you cover all the known media types -used for the type of file you're trying to block. - -The following example prevents zip files from being uploaded by using the regular -expression: ``/^application\/(zip|x-zip-compressed)$/i`` - - .. figure:: images/files_access_control_block_mimetype.png - :alt: Prevent upload based on mimetype - -Common misconfigurations ------------------------- - -Blocking user groups -^^^^^^^^^^^^^^^^^^^^ - -When trying to deny access to a group of users, make sure that sharing does not -allow them to create a way back in. When users are able to create a public link, -the users can log themselves out and visit their own public link to access the -files. Since at this point they are no user and therefor no member of the -blocked group, they will be able to read and change the file. - -The recommended work around is to create the same rule again, and deny access -for all users that are ``not member of`` a group, that contains all users of -your installation. - -External storage -^^^^^^^^^^^^^^^^ - -While access to files in external storages is not possible via Nextcloud, users -that have direct access to the external storage, can of course change files -there directly. Therefor it is recommended to disable the ``Allow users to mount -external storage`` option, when trying to to completely lock out users. - -.. _available-rules-label: - -Available rules ---------------- - -All rules can also be inverted (from ``is`` to ``is not``) using the operator -option. - -* **File collaborative tag:** Either the file itself, or any of the file - owner's parent folders needs to be tagged with the tag. - - .. note:: Tags used in access control rules should be restricted tags, - otherwise any user can remove the tag to access the file again. - The best way to do this is with the :doc:`automated_tagging`. - -* **File mimetype:** The mimetype of the file, e.g. ``text/plain`` -* **File size:** The size of the file (*Only available on upload*) - -* **Request remote address:** An IP range (either v4 or v6) for the accessing user -* **Request time:** Time span and timezone when the request happens -* **Request URL:** The URL which requests the file. (*This is the URL the file - is served from, not the URL the user is currently looking at.*) -* **Request user agent:** The user agent of the users browser or client. - Nextcloud desktop, Android and iOS clients are available as preconfigured - options. - -* **User group membership:** Whether the user is a member of the given group. diff --git a/admin_manual/file_workflows/automated_tagging.rst b/admin_manual/file_workflows/automated_tagging.rst deleted file mode 100644 index 295444b53..000000000 --- a/admin_manual/file_workflows/automated_tagging.rst +++ /dev/null @@ -1,31 +0,0 @@ -========================== -Automated tagging of files -========================== - -Nextcloud's Files Automated Tagging app allows to assign collaborative tags -to files and folders based on rules, similar to :doc:`access_control`. - -Assigning restricted and invisible tags ---------------------------------------- - -The main functionality of this app is to allow users to indirectly assign -restricted and invisible tags to files they upload. - -This is especially useful for retention and :doc:`access_control`, so people -that got the files shared can not remove the tag to stop the retention or -allow access against the owners will. - - - .. figure:: images/automated_tagging_sample_rule.png - :alt: Sample rule to assign a restricted tag. - -In the sample you can see a simple rule with only one condition. -It will tag all files with the restricted tag ``Protected file`` that are -uploaded into a folder that is tagged with ``Protect content``. No user can -remove the tag ``Protected file`` and therefor access control and retention -both work fine without users being able to work around them. - -Available rules ---------------- - -The available rules can be seen in the access control section: :ref:`available-rules-label`. diff --git a/admin_manual/file_workflows/images/automated_tagging_sample_rule.png b/admin_manual/file_workflows/images/automated_tagging_sample_rule.png deleted file mode 100644 index ca456eac4..000000000 Binary files a/admin_manual/file_workflows/images/automated_tagging_sample_rule.png and /dev/null differ diff --git a/admin_manual/file_workflows/images/files_access_control_block_mimetype.png b/admin_manual/file_workflows/images/files_access_control_block_mimetype.png deleted file mode 100644 index 1dc9b7b1c..000000000 Binary files a/admin_manual/file_workflows/images/files_access_control_block_mimetype.png and /dev/null differ diff --git a/admin_manual/file_workflows/images/files_access_control_collaborative_tags.png b/admin_manual/file_workflows/images/files_access_control_collaborative_tags.png deleted file mode 100644 index 4cded6155..000000000 Binary files a/admin_manual/file_workflows/images/files_access_control_collaborative_tags.png and /dev/null differ diff --git a/admin_manual/file_workflows/images/files_access_control_sample_rules.png b/admin_manual/file_workflows/images/files_access_control_sample_rules.png deleted file mode 100644 index 780570eb0..000000000 Binary files a/admin_manual/file_workflows/images/files_access_control_sample_rules.png and /dev/null differ diff --git a/admin_manual/file_workflows/images/retention_sample.png b/admin_manual/file_workflows/images/retention_sample.png deleted file mode 100644 index 86087cdd1..000000000 Binary files a/admin_manual/file_workflows/images/retention_sample.png and /dev/null differ diff --git a/admin_manual/file_workflows/index.rst b/admin_manual/file_workflows/index.rst deleted file mode 100644 index 3056b0a3c..000000000 --- a/admin_manual/file_workflows/index.rst +++ /dev/null @@ -1,11 +0,0 @@ -============== -File workflows -============== - - -.. toctree:: - :maxdepth: 2 - - access_control - automated_tagging - retention diff --git a/admin_manual/file_workflows/retention.rst b/admin_manual/file_workflows/retention.rst deleted file mode 100644 index 2ee332c53..000000000 --- a/admin_manual/file_workflows/retention.rst +++ /dev/null @@ -1,33 +0,0 @@ -================== -Retention of files -================== - -Nextcloud's Files Retention app allows to automatically delete files that -are tagged with a collaborative tag and have a certain age. - -Sample ------- - - .. figure:: images/retention_sample.png - :alt: Sample rule to delete files after 14 days. - -The rule from the sample will delete all files tagged with ``Temporary file`` after 14 days. - -Common misconfigurations ------------------------- - -Public collaborative tag -^^^^^^^^^^^^^^^^^^^^^^^^ - -Similar to :doc:`access_control` retention should use ``restricted`` or ``invisible`` -tags. Otherwise any user can remove the tag and the file is not removed after the given -period. Use :doc:`automated_tagging` to assign such tags to newly uploaded files. - -File age -^^^^^^^^ - -Currently retention is based on the creation date of the file. The sync client sends -the **original** creation date to the server, while uploading through the web interface -will create a new file with a **new** creation date. -We hope to be able to add a ``upload date`` to the filesystem soon, which would make more -sense. Until then this potentially unexpected behavior has to be taken into account. diff --git a/admin_manual/operations/considerations_on_monitoring.rst b/admin_manual/operations/considerations_on_monitoring.rst deleted file mode 100644 index 1c548f54f..000000000 --- a/admin_manual/operations/considerations_on_monitoring.rst +++ /dev/null @@ -1,92 +0,0 @@ -============================ -Considerations on monitoring -============================ - -.. toctree:: - :maxdepth: 2 - :hidden: - -Large scale Nextcloud deployments are typically installed as load balanced -n-tier web applications. Successfully managing such an installation requires -active monitoring of the application and supporting infrastructure components. -The purpose of this section is to outline the components of Nextcloud that need -to be monitored, and provide guidance on what to look for in Nextcloud in an -enterprise installation. - -Nextcloud deployment architecture ---------------------------------- - -Before discussing how to monitor Nextcloud, it is important to understand the architecture of a -typical Nextcloud deployment. These monitoring best practices are developed based on the use of load -balanced Web servers, a clustered database running a distributed database storage engine, such as -MySQL NDB, and a clustered filesystem, such as Red Hat Storage. - -It is assumed that specific enterprise tools (monitoring, log management, etc) to monitor -operations are available, and that Nextcloud is simply a new target for these tools. - - -The important components of Nextcloud -------------------------------------- - -Nextcloud is a PHP application that depends on a filesystem for file storage, and a database for storing -user and file meta data, as well as some application specific information. -While the loss of an app server or a node in the database or storage clusters should not bring the -system down, knowing that this happened and resolving it is essential to keeping the service running -efficiently. Therefore it is important to monitor the Nextcloud servers, the Load Balancer, the Storage -Cluster and the Database. This documentation starts with the Nextcloud application and works out from -there through the layers of infrastructure. - - -Status.php -^^^^^^^^^^ - -Nextcloud provides a very simple mechanism for determining if an application server is up and functioning – -call the status.php file on each Nextcloud server. This file can be found in the root Nextcloud directory on -the server, which by default is /status.php. If the server is functioning normally, the response -looks something like this: - -:: - - {"installed":"true","version":"6.0.0.16","versionstring":"6.0.1","edition":""} - - -We recommend monitoring this file on each Nextcloud application server to provide a basic check that the -server is operating properly. - - -Nextcloud.log -^^^^^^^^^^^^^ - -Nextcloud also provides a built in logging function. If the Nextcloud logging application -is enabled, this file will track user logins and shared file activity. If these logging applications are -not enabled, this log file still tracks basic Nextcloud health. Given the potential for this file to get -quite large, the log file should be rotated on a daily basis, and given the importance of the error information -in the log file, this should be integrated with an enterprise log manager. - - -Logfile entries that start with the keyword “Error” should be logged and reported to Nextcloud support. - -**Apache** - -The apache error and access log should also be monitored. Significant spontaneous changes of the number -of requests per second should also be monitored and looked into. - - -**Database server** - -The load and general health of the database server or cluster has to be monitored also. -All mysql vendors provide tools to monitor this. - - -Clustered filesystem -^^^^^^^^^^^^^^^^^^^^ - -The available space of the filesystem should be monitored to prevent a full Nextcloud. This functionality is -provided by the operating-system and/or the cluster filesystem vendor. - -Load balancer -^^^^^^^^^^^^^ - -The load balancer is monitoring the health of the application servers and is distributing the traffic in -the optimal way. The application-servers should also be monitored to detect long lasting OS or -hardware problems. Monitoring solutions like Nagios provide built in functionality to do this. diff --git a/admin_manual/operations/index.rst b/admin_manual/operations/index.rst deleted file mode 100644 index ff662469e..000000000 --- a/admin_manual/operations/index.rst +++ /dev/null @@ -1,13 +0,0 @@ -========== -Operations -========== - -Advanced operation including monitoring, scaling across multiple machines, and -creating a custom theme for your Nextcloud server. - -.. toctree:: - :maxdepth: 2 - - considerations_on_monitoring - scaling_multiple_machines.rst - diff --git a/admin_manual/operations/scaling_multiple_machines.rst b/admin_manual/operations/scaling_multiple_machines.rst deleted file mode 100644 index 7d080a029..000000000 --- a/admin_manual/operations/scaling_multiple_machines.rst +++ /dev/null @@ -1,141 +0,0 @@ -================================ -Scaling across multiple machines -================================ - -.. toctree:: - :maxdepth: 2 - :hidden: - -This document will cover the reference architecture for the Nextcloud Scale Out -model for a single datacenter implementation. The document will focus on the -three main elements of an Nextcloud deployment: - -* Application layer -* Database Layer -* Storage Layer - -At each layer the goal is to provide the ability to scale, and providing a high -availability while maintaining the needed level of performance. - -Application layer ------------------ - -For the application layer of this reference architecture we used Oracle -Enterprise Linux as the front end servers to host the Nextcloud code. In this -instance we made ``httpd`` a permissive domain, allowing it to operate within the -SELinux environment. In this example we also used the standard directory -structure placing the Nextcloud code in the Apache root directory. The -following components were installed on each application server: - -* Apache -* PHP 5.6.x -* PHP-GD -* PHP-XML -* PHP-MYSQL -* PHP-CURL - -Also required is the PHP smbclient module or smbclient (see -:doc:`../configuration_files/external_storage/smb`). - -It is also worth mentioning that the appropriate exceptions where made in the -firewall to allow the Nextcloud traffic (for the purpose of testing we -enable both encrypted SSL via port 443 and unencrypted via port 80). - -The next step was to generate and import the needed SSL certificates following -the standard process in the OEL documentation. - -The next step is to create a scalable environment at the application layer, -which introduces the load balancer. Because the application servers here are -stateless, simply taking the configuration above and replicating (once -configured with storage and database connections) and placing behind a load -balancer will provide a scalable and highly available environment. -For this purpose we chose HAProxy and configured it for HTTPS traffic following -the documentation found here `http://haproxy.1wt.eu/#doc1.5 -`_ - -It is worth noting that this particular load balancer is not required, and the use of -any commercial load balancer (i.e. F5) will work here. It is also worth noting -that the HAProxy servers were setup with a heartbeat and IP Shift to failover -the IP address should one fail. - -.. image:: ../images/scaling-1.png - :width: 3.4937in - :height: 4.5134in - -.. Session Management (this section is a WIP pending testing based on customer feedback). - -.. The load balancer is to be configured to spread the workload across the various -.. Nextcloud application servers, with details to be filled in around session -.. management upon further testing. - -Database layer --------------- - -For the purposes of this example, we have chosen a MySQL cluster using the NDB -Storage engine. The cluster was configured based on the documentation found -here `http://dev.mysql.com/doc/refman/5.1/en/mysql-cluster.html -`_ with a sample -looking like this: - -.. image:: ../images/scaling-2.png - :width: 6in - :height: 3.0673in - -Taking a closer look at the database architecture, we have created redundant -MySQL NDB Management nodes for redundancy and we have configured 3 NDB -SQL/Storage nodes across which we are able to spread the database traffic. All -of the clients (Nextcloud Application Servers) will connect to the database via -the My SQL Proxy. It is worth noting that MySQL proxy is still in beta and that using -another load balancing method like HAProxy or F5 is supported, in that you will -be distributing traffic among the various SQL/Storage nodes. Here, we simply -swap out the MySQL Proxy for a properly configured HAProxy giving us the -following: - -.. image:: ../images/scaling-3.png - :width: 6in - :height: 0.7311in - -In this example we have also added a second HAProxy server with Heartbeat to prevent any single point of failure. -We have also implemented NIC bonding to load balance the traffic across multiple physical NICs. - -Storage layer -------------- - -Storage was deployed using the Red Hat Storage server with the GlusterFS -(pre-configured as part of the Red Hat Storage Server offering). - -The Red Hat Storage Servers where configured based on documentation found here -`https://access.redhat.com/site/documentation/en-US/Red_Hat_Storage/2.0/html/Administration_Guide/Admin_Guide_Part_1.html -`_ - -For the purposes of scale and high availability we configured a distributed -replicated volume with IP Failover. The storage was configured on a separate -subnet with bonded NICs at the application server level. We have chosen to -address the storage using NFS, and for high availability we have chosen to -implement IP Failover of the storage as documented here -`https://access.redhat.com/site/documentation/en-US/Red_Hat_Storage/2.0/html/Administration_Guide/ch09s04.html -`_ - -.. image:: ../images/scaling-4.png - :width: 3.8693in - :height: 3.6272in - -We chose to deploy the storage in this fashion to address both HA and -extensibility of the storage, along with managing performance by simply adding -additional bricks to the storage volume, back-ended by additional physical -disk. - -It is worth noting that several options are available for storage configuration -(such as striped replicated volumes). A discussion around the type of IO -performance required and the needed HA configuration needs to take place to -understand the best possible option here. - -If we add up the parts, we have the following: - -* An application layer that supports dynamic expansion through the addition of additional servers and provides HA behind a load balancer -* A database layer that can also be scaled through the addition of additional SQL/Storage nodes and will provide HA behind a load balancer -* A storage layer that can dynamically expand to meet storage needs, will scale based on backend hardware, and provides HA via IP Failover - -.. image:: ../images/scaling-5.png - :width: 4.4937in - :height: 5.2134in