Skip to content

Commit

Permalink
Support for disconnected cluster installation using ABI on kvm (#345)
Browse files Browse the repository at this point in the history
Signed-off-by: Sumit Solanki <[email protected]>
Co-authored-by: Sumit Solanki <[email protected]>
  • Loading branch information
isumitsolanki and Sumit Solanki authored Nov 20, 2024
1 parent 03770dc commit 25e054f
Show file tree
Hide file tree
Showing 9 changed files with 141 additions and 7 deletions.
12 changes: 9 additions & 3 deletions docs/run-the-playbooks-for-abi.md
Original file line number Diff line number Diff line change
@@ -1,8 +1,11 @@
# Run the Agent Based Installer (ABI) Playbooks
## Prerequisites
* KVM host with root user access or user with sudo privileges. ( applicable only in case of KVM)
* z/VM bastion with root user access or user with sudo privileges. ( applicable only in case of z/VM)
* z/VM Host with desired network cards enabled and storage details. ( applicable only in case of z/VM)
* KVM
* Host with root user access or user with sudo privileges.
* z/VM
* Bastion with root user access or user with sudo privileges.
* Host with desired network cards enabled and storage details.
* Checklist for Disconnected Cluster Installation ( [Disconnected Pre-requisites](https://github.com/IBM/Ansible-OpenShift-Provisioning/blob/main/docs/run-the-playbooks-for-disconnected.md#pre-requisites) )

## Note:
* This playbook support SNO, Compact and HA type of OCP cluster installation on KVM using ABI.
Expand All @@ -21,6 +24,7 @@
* In case of SNO Section 9 ( `Compute Nodes` ) need to be comment or remove
* First playbook to be run is `0_setup.yaml` which will create inventory file for ABI and will add ssh key to the kvm host.
* In case of z/VM update variables in [zvm.yaml](https://github.com/IBM/Ansible-OpenShift-Provisioning/blob/main/inventories/default/group_vars/zvm.yaml).
* In case of disconnected cluster installation update variables in [disconnected.yaml](https://github.com/IBM/Ansible-OpenShift-Provisioning/blob/main/inventories/default/group_vars/disconnected.yaml).

* Run this shell command:
```
Expand All @@ -33,7 +37,9 @@ ansible-playbook playbooks/0_setup.yaml
* 3_setup_kvm_host.yaml ([code](https://github.com/IBM/Ansible-OpenShift-Provisioning/blob/main/playbooks/3_setup_kvm_host.yaml))
* 4_create_bastion.yaml ([code](https://github.com/IBM/Ansible-OpenShift-Provisioning/blob/main/playbooks/4_create_bastion.yaml))
* 5_setup_bastion.yaml ([code](https://github.com/IBM/Ansible-OpenShift-Provisioning/blob/main/playbooks/5_setup_bastion.yaml))
* disconnected_mirror_artifacts.yaml (when disconnected is True) [code](https://github.com/IBM/Ansible-OpenShift-Provisioning/blob/main/playbooks/disconnected_mirror_artifacts.yaml)
* create_abi_cluster.yaml ([code](https://github.com/IBM/Ansible-OpenShift-Provisioning/blob/main/playbooks/create_abi_cluster.yaml))
* monitor_create_abi_cluster.yaml ([code](https://github.com/IBM/Ansible-OpenShift-Provisioning/blob/main/playbooks/monitor_create_abi_cluster.yaml))

* Watch Ansible as it completes the installation, correcting errors if they arise.
* To look at what tasks are running in detail, open the playbook or roles/role-name/tasks/main.yaml
Expand Down
3 changes: 2 additions & 1 deletion playbooks/5_setup_bastion.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -92,7 +92,7 @@
ansible.builtin.include_tasks:
file: ../roles/bastion_setup_hipersocket_LPAR/tasks/main.yml
when: vars_file.stat.exists and networking.mode is defined and networking.mode | lower =='hipersocket'
when: installation_type | lower =="lpar"
when: installation_type | lower == "lpar"

- name: 5 setup bastion - configure bastion node with essential services
hosts: bastion
Expand All @@ -103,6 +103,7 @@
control_node_count: "{{ env.cluster.nodes.control.ip | length }}"
vars_files:
- "{{ inventory_dir }}/group_vars/all.yaml"
- "{{ inventory_dir }}/group_vars/disconnected.yaml"
pre_tasks:
- name: Import initial-resolv.yaml
ansible.builtin.import_role:
Expand Down
4 changes: 3 additions & 1 deletion playbooks/create_abi_cluster.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -7,9 +7,11 @@
vars_files:
- "{{ inventory_dir }}/group_vars/all.yaml"
- "{{ inventory_dir }}/group_vars/zvm.yaml"
- "{{ inventory_dir }}/group_vars/disconnected.yaml"
roles:
- common # Common Variable the will be used by all the inwalked roles.
- download_ocp_installer # Download Openshift Installer.
- { role: offline_artifacts, when: disconnected.enabled } # Update CA Certificates & Download OCP Packages
- { role: download_ocp_installer, when: not disconnected.enabled } # Download Openshift Installer.
- prepare_configs # Prepare AgentConfig & InstallConfig.
- create_agent # Create Agents || Build initrd.img, rootfs.img & kernelfs.img.

Expand Down
2 changes: 2 additions & 0 deletions playbooks/master_playbook_for_abi.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -6,5 +6,7 @@
when: ( installation_type | lower == "kvm" )
- import_playbook: 4_create_bastion.yaml # Import Playbook To Create Bastion.
- import_playbook: 5_setup_bastion.yaml # Import Playbook To Configure Bastion.
- import_playbook: disconnected_mirror_artifacts.yaml
when: disconnected.enabled
- import_playbook: create_abi_cluster.yaml # Import Playbook To Create ABI Cluster.
- import_playbook: monitor_create_abi_cluster.yaml # Import Playbook To Monitor ABI Cluster Installation.
5 changes: 5 additions & 0 deletions roles/dns/templates/dns.db.j2
Original file line number Diff line number Diff line change
Expand Up @@ -30,4 +30,9 @@ api-int.{{ env.cluster.networking.metadata_name }} IN CNAME {{ env.bastion.ne
apps.{{ env.cluster.networking.metadata_name }} IN CNAME {{ env.bastion.networking.hostname }}.{{ env.cluster.networking.base_domain }}.
*.apps.{{ env.cluster.networking.metadata_name }} IN CNAME {{ env.bastion.networking.hostname }}.{{ env.cluster.networking.base_domain }}.

{% if disconnected.enabled %}
;entry for mirror host.
{{ env.cluster.networking.metadata_name }}.{{ env.cluster.networking.metadata_name }}.{{ env.cluster.networking.base_domain }}. IN A {{ env.file_server.ip }}
{% endif %}

;EOF
105 changes: 105 additions & 0 deletions roles/offline_artifacts/tasks/main.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,105 @@
---
- name: Offline Artifacts
block:
- name: Get user home directory
shell: >
getent passwd {{ ansible_user }} | awk -F: '{ print $6 }'
changed_when: false
register: user_home

- name: Delete OCP download directory for idempotency.
become: true
file:
path: "{{ user_home.stdout }}/ocp_pkgs"
state: absent

- name: Create OCP download directory
file:
path: "{{ user_home.stdout }}/ocp_pkgs"
state: directory

- name: Unzip OCP client and oc-mirror
ansible.builtin.unarchive:
src: "{{ item }}"
dest: "{{ user_home.stdout }}/ocp_pkgs/"
remote_src: yes
loop:
- "{{ env.file_server.protocol }}://{{ env.file_server.user + ':' + env.file_server.pass + '@' if env.file_server.protocol == 'ftp' else '' }}{{ env.file_server.ip }}{{ ':' + env.file_server.port if env.file_server.port | default('') | length > 0 else '' }}/{{ disconnected.mirroring.file_server.clients_dir }}/{{ disconnected.mirroring.file_server.oc_mirror_tgz }}"
- "{{ env.file_server.protocol }}://{{ env.file_server.user + ':' + env.file_server.pass + '@' if env.file_server.protocol == 'ftp' else '' }}{{ env.file_server.ip }}{{ ':' + env.file_server.port if env.file_server.port | default('') | length > 0 else '' }}/{{ disconnected.mirroring.file_server.clients_dir }}/{{ disconnected.mirroring.client_download.ocp_client_tgz }}"

- name: Copy kubectl, oc, and oc-mirror binaries to /usr/local/sbin
become: true
ansible.builtin.copy:
src: "{{ user_home.stdout }}/ocp_pkgs/{{ item }}"
dest: /usr/sbin/{{ item }}
owner: root
group: root
mode: "755"
remote_src: yes
loop:
- kubectl
- oc
- oc-mirror

- name: Check if directory {{ user_home.stdout }}/.docker exists
ansible.builtin.stat:
path: "{{ user_home.stdout }}/.docker"
register: home_docker

- name: Create directory {{ user_home.stdout }}/.docker
file:
path: "{{ user_home.stdout }}/.docker"
state: directory
when: not home_docker.stat.exists

- name: create pull secret file for mirroring
ansible.builtin.template:
src: mirror-secret.json.j2
dest: "{{ user_home.stdout }}/.docker/config.json"
backup: yes
force: yes

- name: create ca cert file for adding to ca trust when ca is not trusted and updating ca trust
become: true
block:
- name: create ca cert file when ca is untrusted
ansible.builtin.template:
src: ca.crt.j2
dest: /etc/pki/ca-trust/source/anchors/registry.crt
force: yes
- name: update ca trust with the cert file
ansible.builtin.shell: |
set -o pipefail
update-ca-trust
when: not disconnected.registry.ca_trusted

- name: run the oc adm release extract command
block:
- name: run the oc adm release extract command
ansible.builtin.shell: |
set -o pipefail
oc adm -a {{ user_home.stdout }}/.docker/config.json release extract \
--command=openshift-install "{{ disconnected.registry.url }}/openshift/release-images:{{ disconnected.mirroring.oc_mirror.release_image_tag }}"
register: cmd_oc_extract
args:
chdir: "{{ user_home.stdout }}/ocp_pkgs"

- name: print output of cmd_oc_extract
ansible.builtin.debug:
var: cmd_oc_extract

- name: Copy openshift-install binaries to /usr/local/sbin
become: true
ansible.builtin.copy:
src: "{{ user_home.stdout }}/ocp_pkgs/openshift-install"
dest: /usr/sbin/openshift-install
owner: root
group: root
mode: "755"
remote_src: yes

- name: Install NMState package
ansible.builtin.yum:
name: nmstate
state: latest
skip_broken: yes
1 change: 1 addition & 0 deletions roles/offline_artifacts/templates/ca.crt.j2
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
{{ disconnected.registry.ca_cert }}
1 change: 1 addition & 0 deletions roles/offline_artifacts/templates/mirror-secret.json.j2
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
{{ disconnected.registry.mirror_pull_secret }}
15 changes: 13 additions & 2 deletions roles/prepare_configs/templates/install-config.yaml.j2
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,18 @@ networking:
- {{ install_config_vars.service_network }}
platform:
none: {}
pullSecret: >
{{ env.redhat.pull_secret }}
sshKey: >
{{ ssh_key.stdout }}
pullSecret: '{{ env.redhat.pull_secret if not disconnected.enabled else disconnected.registry.pull_secret }}'
{% if disconnected.enabled %}
{{ 'imageContentSources: ' }}
{{ '- mirrors:'}}
{{ ' - ' + disconnected.registry.url + '/' }}{{ disconnected.mirroring.legacy.ocp_org if disconnected.mirroring.legacy.platform else 'openshift' }}{{ '/' }}{{ disconnected.mirroring.legacy.ocp_repo if disconnected.mirroring.legacy.platform else 'release-images' }}
{{ ' source: quay.io/openshift-release-dev/ocp-release' }}
{{ '- mirrors:'}}
{{ ' - ' + disconnected.registry.url + '/' }}{{ disconnected.mirroring.legacy.ocp_org if disconnected.mirroring.legacy.platform else 'openshift' }}{{ '/' }}{{ disconnected.mirroring.legacy.ocp_repo if disconnected.mirroring.legacy.platform else 'release' }}
{{ ' source: quay.io/openshift-release-dev/ocp-v4.0-art-dev' }}
{% endif %}
{% if disconnected.enabled and not disconnected.registry.ca_trusted %}
{{ 'additionalTrustBundle: |' }}{% for line in disconnected.registry.ca_cert.split('\n') %}{{ '\n ' + line }}{% endfor %}
{% endif %}

0 comments on commit 25e054f

Please sign in to comment.