Merge branch 'main' into manager_install_and_bootstrap

.github/linkspector.yml

@@ -0,0 +1,4 @@
+---
+ignorePatterns:
+  - pattern: "^(?!http(s)?:\/\/.*)|^(http(s)?:\/\/.*((osism.xyz)|(in-a-box.cloud)))(:?[0-9]+)?(\/.*)?$"
+  - pattern: "^https:\/\/www.wireguard.com$"

.github/workflows/check-markdown-links.yml

@@ -0,0 +1,17 @@
+name: Check Markdown links
+on:
+  push:
+
+jobs:
+  check-markdown-links:
+    name: Run linkspector
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+      - name: Run linkspector
+        uses: umbrelladocs/action-linkspector@v1
+        with:
+          config-file: .github/linkspector.yml
+          github_token: ${{ secrets.github_token }}
+          reporter: github-pr-review
+          fail_on_error: true

docs/guides/concept-guide/design.md

@@ -15,10 +15,10 @@ and have been published under the Apache Licence 2.0.
 
 ## Compute architecture
 
+## Control plane architecture
+
 ## Storage architecture
 
 ## Network architecture
 
 ## Identity architecture
-
-## Control plane architecture

docs/guides/concept-guide/nodes.md

@@ -0,0 +1,18 @@
+---
+sidebar_label: Nodes in a cluster
+sidebar_position: 25
+---
+
+# Nodes in a cluster
+
+## Compute Node
+
+## Control Node
+
+## Data Node
+
+## Management Node
+
+## Monitoring Node
+
+## Network Node

docs/guides/configuration-guide/ceph.md

@@ -32,9 +32,11 @@ The `client.admin` keyring is placed in the file `environments/infrastructure/fi
 The swappiness is set via the `os_tuning_params` dictionary. The dictionary can
 only be completely overwritten via an entry in the file `environments/ceph/configuration.yml`.
 
-By default, the dictionary looks like this:
+By default, the dictionary looks like this. If the swappiness of `10` is to be used, it is not
+necessary to add the `os_tuning_params` dictionary to the configuration repository. This is only
+necessary if the swappiness is to be customised.
 
-```
+```yaml title="environments/ceph/configuration.yml"
 os_tuning_params:
   - { name: fs.file-max, value: 26234859 }
   - { name: vm.zone_reclaim_mode, value: 0 }
@@ -152,14 +154,7 @@ pools are to be created is `ceph.rbd`, then the parameters would be stored in
 | `openstack_pool_default_pg_num`   | 64            |
 | `openstack_pool_default_min_size` | 0             |
 
-## LVM devices
-
-For more advanced OSD layout requirements leave out the `devices` key
-and instead use `lvm_volumes`. Details for this can be found on the
-[OSD Scenario](https://docs.ceph.com/projects/ceph-ansible/en/latest/osds/scenarios.html) documentation.
-
-In order to aid in creating the `lvm_volumes` config entries and provision the LVM devices for them,
-OSISM has the two playbooks `ceph-configure-lvm-volumes` and `ceph-create-lvm-devices` available.
+## OSD devices
 
 1. For each Ceph storage node edit the file `inventory/host_vars/<nodename>.yml`
    add a configuration like the following to it. Ensure that no `devices` parameter

docs/guides/configuration-guide/commons/certificates.md

@@ -20,5 +20,5 @@ The role is part of the bootstrap of a node. CA certificates can be added at a l
 point in time via `osism apply certificates` on a node.
 
 Further details on the use of self-signed certificates can be found in chapter
-[Self-signed certificates](../self-signed-certificates)
+[Self-signed certificates](../loadbalancer#self-signed-certificates)
 of the configuration guide.

docs/guides/configuration-guide/configuration-repository.md

@@ -19,7 +19,7 @@ you for the basic details of the new cluster.
 The configuration repository is not created on the future Manager node. It is created on a
 local workstation. If the local workstation cannot be used for this purpose, a dedicated
 virtual system can be used. For more information on this topic, refer to the
-[Seed Deploy Guide](../deploy-guide/seed.md)..
+[Seed Deploy Guide](../deploy-guide/seed.md).
 
 ### Step 1: Preparation
 
@@ -66,7 +66,7 @@ listed there will be queried during the execution of Cookiecutter.
      --rm -it quay.io/osism/cookiecutter
    ```
 
-3. A few parameters are requested. The parameters are documented in detail in the [Parameters reference](#parameters-reference).
+3. A few parameters are requested. The parameters are documented in detail in the [parameter reference](#parameter-reference).
 
    If you want to use the `latest` version, this is done using the `manager_version` parameter. By default,
    this is always set to the latest stable version.
@@ -141,7 +141,7 @@ The following 6 points must be changed after the initial creation of the configu
 3. [Global inventory](#global-inventory)
 4. [DNS servers](#dns-servers)
 5. [NTP servers](#ntp-servers)
-6. [SSL certificates](#ssl-certificates)
+6. [Certificates](#certificates)
 
 #### Secrets
 
@@ -389,8 +389,18 @@ chrony_servers:
   - 4.de.pool.ntp.org
 ```
 
-#### SSL certificates
+#### Certificates
 
+The certificates must be created and added in the configuration repository in the files
+`environments/kolla/certificates/haproxy.pem` and `environments/kolla/certificates/haproxy-internal.pem`. Further information in the [Loadbalancer Configuration Guide](./loadbalancer.md).
+
+If no certificates are to be used, the encryption must be deactivated. This is not
+recommended.
+
+```yaml title="environments/kolla/configuration.yml"
+kolla_enable_tls_external: "yes"
+kolla_enable_tls_internal: "yes"
+```
 
 ## Using latest
 
@@ -482,3 +492,31 @@ $ osism apply facts
 2024-06-02 10:53:08 | INFO     | It takes a moment until task 6ac9a526-f88d-4756-bf46-2179636dfb42 (facts) has been started and output is visible here.
 ERROR: The configuration repository is locked.
 ```
+
+## Working with encrypted files
+
+To make it easier to work with encrypted files, the configuration repository has several make
+targets that can be used to view encrypted files and to edit encrypted files.
+
+* Show secrets in all encrypted files.
+
+  This opens a pager, e.g. less, and you can search with `/` for specific files, keys and passwords.
+
+  ```
+  make ansible_vault_show
+  ```
+
+* Change or add secrets in an encrypted file with the editor set in ` $EDITOR`.
+
+  ```
+  make ansible_vault_edit FILE=environments/secrets.yml EDITOR=nano
+  ```
+
+* Re-encrypt all encrypted files with a new key.
+
+  This creates a new `secrets/vaultpass` and creates backups of the old to
+  `secrets/vaultpass_backup_<timestamp>`.
+
+  ```
+  make ansible_vault_rekey
+  ```

docs/guides/configuration-guide/inventory.md

@@ -7,6 +7,9 @@ sidebar_position: 10
 
 The inventory used for the environment is located in the `inventory` directory.
 
+How an inventory works is described in detail in the [Ansible documentation](https://docs.ansible.com/ansible/latest/inventory_guide/intro_inventory.html).
+In this chapter, we only deal with special features in the context of OSISM.
+
 ## Manager
 
 The manager has his own inventory which is used exclusively for the seed phase of the manager.

docs/guides/configuration-guide/loadbalancer.md

@@ -5,6 +5,95 @@ sidebar_position: 20
 
 # Loadbalancer
 
+## IP addresses & FQDNs
+
+```yaml title="environments/kolla/configuration.yml"
+kolla_internal_vip_address: 192.168.16.9
+kolla_external_vip_address: 192.168.16.254
+```
+
+```yaml title="environments/kolla/configuration.yml"
+kolla_internal_fqdn: api-int.testbed.osism.xyz
+kolla_external_fqdn: api.testbed.osism.xyz
+```
+
+```yaml title="environments/configuration.yml"
+hosts_additional_entries:
+  api-int.testbed.osism.xyz: 192.168.16.9
+  api.testbed.osism.xyz: 192.168.16.254
+```
+
+## TLS certificates
+
+To enable external TLS encryption:
+
+```yaml title="environments/kolla/configuration.yml"
+kolla_enable_tls_external: "yes"
+```
+
+To enable internal TLS encryption:
+
+```yaml title="environments/kolla/configuration.yml"
+kolla_enable_tls_internal: "yes"
+```
+
+Two certificate files are required to use TLS securely with authentication,
+which will be provided by your Certificate Authority:
+
+* the server certificate with private key
+* the CA certificate with any intermediate certificates
+
+The combined server certificate and private key needs to be provided at
+the following locations in the configuration repository:
+
+* private key & certificates for `kolla_external_fqdn`: `environments/kolla/certificates/haproxy.pem`
+* private key & certificates for `kolla_internal_fqdn`: `environments/kolla/certificates/haproxy-internal.pem`
+
+## Generating TLS certificates with Let’s Encrypt
+
+## Self-signed certificates
+
+The use of self-signed certificates with a custom CA is possible. However, a few
+additional parameters are then required in the configuration so that the custom CA
+is known everywhere and the self-signed certificates are accepted as valid.
+
+1. Import custom CA
+
+   Any custom CA can be added via the `certificates_ca` parameter.
+   The import on the nodes is done via `osism apply certificates`.
+   This is already done in the bootstrap of the nodes.
+
+   ```yaml title="environments/configuration.yml"
+   certificates_ca:
+     - name: custom.crt
+       certificate: |
+         -----BEGIN CERTIFICATE-----
+         [...]
+         -----END CERTIFICATE-----
+   ```
+
+2. Manager service
+
+   The local environment variable `REQUESTS_CA_BUNDLE` must be set explicitly so that
+   the manager service knows the custom CA in all necessary places.
+
+   ```yaml title="environments/manager/configuration.yml"
+   manager_environment_extra:
+     REQUESTS_CA_BUNDLE: /etc/ssl/certs/ca-certificates.crt
+   ```
+
+3. Use in OpenStack
+
+   The custom CA must also be copied into the OpenStack containers. To do this, the custom
+   CA is first added in a file in the `environments/kolla/certificates/ca` of the configuration
+   repository.  It makes sense to use the same filename like in step 1.
+
+   The import of the custom CA must then be explicitly enabled.
+
+   ```yaml title="environments/kolla/configuration.yml"
+   kolla_copy_ca_into_containers: "yes"
+   openstack_cacert: /etc/ssl/certs/ca-certificates.crt
+
 ## Second Loadbalancer
 
 :::info
@@ -97,3 +186,9 @@ loadbalancer. This will be possible in the future.
 ```
 osism apply --sub external loadbalancer-without-service-config
 ```
+
+## ProxySQL
+
+```yaml title="environments/kolla/configuration.yml"
+enable_proxysql: "yes"
+```

docs/guides/configuration-guide/openstack/index.md

@@ -494,3 +494,5 @@ These parameters are all set in `environments/kolla/configuration.yml`.
 | octavia_healthmanager_stats_workers    |
 | placement_api_workers                  |
 | skyline_gunicorn_workers               |
+
+## Back-end TLS configuration

docs/guides/configuration-guide/proxy.md

@@ -7,7 +7,8 @@ sidebar_position: 15
 
 In the following examples, it is assumed that the Squid proxy integrated by OSISM
 is used on the first manager node. Any other proxy accessible from the nodes can
-also be used here.
+also be used here. `http://{{ groups['manager'][0] }}:3128` which is used here as an
+example is then replaced accordingly with the address of the proxy.
 
 The Squid service can be deployed on the first manager. This is useful if no proxy
 can be used in the environment. The first manager node is then used by all other nodes
@@ -44,9 +45,9 @@ proxy_proxies:
   https: "http://{{ groups['manager'][0] }}:3128"
 ```
 
-## Kolla
+## OpenStack
 
-Proxy settings for containers such as magnum that need internet access.
+Proxy settings for containers such as Magnum that need internet access.
 
 ```yaml title="environments/kolla/configuration.yml"
 ##########################################################