Skip to content

Latest commit

 

History

History
404 lines (264 loc) · 16.2 KB

amazon-eks-jx.md

File metadata and controls

404 lines (264 loc) · 16.2 KB
Raw

Amazon EKS

Setup Overview

The diagram below shows an EKS cluster deployed in AWS and exposed to the outside world using Network Load Balancer NLB and Route 53 domain mapping.

Jenkins-X platform is then deployed and configured with Git provider to trigger Kubernetes-based pipelines which produce Docker images pushed into private Elastic Container Registry (ECR). Jenkins-X will also deploy our Activiti Cloud Helm chart into an EKS cluster and expose it via ingress to the outside world.

The diagram above uses Kubernetes Deployment Language: https://github.com/raffaelespazzoli/kdl

All of the persistence volume claims of Activiti Cloud infrastructure and applications will be handled by Amazon services using Amazon Elastic Block Store (Amazon EBS), which will be used to provision Kubernetes persistent volumes for Audit, Query, and Runtime Bundle using PostgreSQL database service.

The Short Version

In case, you know what you are doing, get Jx from https://jenkins-x.io/getting-started/install/ and then follow the short version:

export AWS_ACCESS_KEY_ID=1234567890
export AWS_SECRET_ACCESS_KEY=123456789
export CLUSTER_NAME=activiti-cloud

jx upgrade cli
jx create cluster eks \
    --cluster-name=$CLUSTER_NAME \
    --skip-installation=true \
    --node-type='m5.xlarge' \
    --nodes=2 \
    --advanced-mode
jx install \
    --provider=eks \
    --no-default-environments \
    --static-jenkins \
    --no-tiller=false \
    --advanced-mode

Then, skip to Let’s Deploy Activiti Cloud in EKS section at the end of this document.

The Long version

Configuring Domain Name using Nip.io

In case, we don’t need the real domain name, Jenkins-X provides support to use wildcard DNS nip.io service. The nip.io allows to resolve any IP Address in the following DNS wildcard entries:

10.0.0.1.nip.io --> 10.0.0.1 app.10.0.0.1.nip.io --> 10.0.0.1 customer1.app.10.0.0.1.nip.io --> 10.0.0.1 customer2.app.10.0.0.1.nip.io --> 10.0.0.1 otherapp.10.0.0.1.nip.io --> 10.0.0.1

Simply put NIP.IO maps ..nip.io to the corresponding , simply by resolving 127.0.0.1.nip.io -> 127.0.0.1

To use nip.io DNS name, Jenkins-X will install and resolve the IP address of the Network Load Balancer during setup. Then the jenkins.1.2.3.4.nip,io DNS name would resolve to IP address, 1.2.3.4 giving wildcard DNS without need to configure DNS in Route 53.

Configuring Domain Name in Route 53

To access our EKS cluster by domain name, we need to configure a real domain that will be used by users to access Activiti Cloud application and by DevOps team to access our Jenkins server.

In order to add a domain to AWS, choose Route 53 Service from the AWS console and click the blue Create Hosted Zone button. In the dialog box, type the name of your domain and click the blue Create button.

After creating the hosted zone, you should see it in your Route 53 console below. You should configure your domain to delegate to the AWS name servers defined in NS record of your hosted zone as shown here:

Installing the Jenkins-X Cli

The Jenkins X provides Cli client called ‘jx’ (https://github.com/jenkins-x/jx). It is used to manage the Jenkins X infrastructure and Continuous Delivery pipelines. Detailed instructions on how to install jx, can be found https://jenkins-x.io/getting-started/install/

After successful installation of jx client you should now be able to display the jx client version by executing the following command:

$ jx version
NAME               VERSION
Jx                 2.0.323

Upgrade Warning for jx

jx version is aggressive about upgrading, so much so that the version command prompts to upgrade to the latest (if a newer is available). The default is Y , yes. To remove variables when using this GitOps approach, this author recommends saying n , no. Leave the upgrading of Jenkin-x to a purposeful plan both on the client and deployed infrastructure, ensuring both are always a match.

~ $ jx version

NAME VERSION
jx   2.0.232
git  git version 2.17.2 (Apple Git-113)

A new jx version is available: 2.0.333
? Would you like to upgrade to the new jx version? No

Installing the Git Cli

In order to setup Jenkins-X in Kubernetes with Git provider for CI/CD GitOps, we need to create a user account on Github to host Jenkins-X environment repositories. We will also need to install and configure Git Cli your local terminal console environment from https://git-scm.com/downloads

Creating the EKS Kubernetes Cluster

JX will attempt to locate any required dependencies. If JX discovers that some dependencies are missing it will prompt to install them for you in the ~/.jx/bin folder. It is recommended that you allow jx to install any missing dependencies. You may also choose to install required dependencies for your OS manually:

You can also pre-install any dependencies manually before we create the cluster.

Setting Up AWS Credentials

Ensure that you have your AWS credentials set up. The Jenkins X client is able to retrieve AWS credentials from standard AWS cli locations, including environment variables or ~/.aws config files. For the purposes of this example, we can just use environment variables:

$ export AWS_ACCESS_KEY_ID=1234567890
$ export AWS_SECRET_ACCESS_KEY=123456789
$ export CLUSTER_NAME=activiti-cloud

Creating EKS Cluster

Let’s create the new Kubernetes cluster with EKS using the following command:

Note: Jx cli simply wraps eksctl cli to create cluster using Aws api. You may choose to create cluster manually based on your requirements using eksctl command, if you know what you are doing.

jx create cluster eks \
    --cluster-name=$CLUSTER_NAME \
    --skip-installation=true \
    --node-type='m5.xlarge' \
    --nodes=2 \
    --advanced-mode

This will provision activiti-cloud EKS cluster with 2 nodes of type m5.xlarge (4CPU’s, 16GB RAM)

You can watch progress in the CloudFormation console: https://console.aws.amazon.com/cloudformation/

Execution of “jx create cluster” can take a while (~-20-25 min) while eksctl waits until the EKS cluster is fully initialized. As soon as the command finishes, you can verify installation using the following command:

$ jx get eks

NAME
activiti-cloud

Installing the Jenkins X platform Into Your Kubernetes Cluster

Now that we have the EKS cluster up and running, we can install the Jenkins X platform into it. The Jenkins X platform consists of the Jenkins server itself, a Nexus server, configuration stored in Kubernetes, and a couple of other components, all packaged into single Helm chart.

The Jenkins X platform can be installed using the following command without installing default environments repos. We will create Activiti Cloud Environments GitOps repos later.

jx install \
    --provider=eks \
    --no-default-environments \
    --static-jenkins \
    --no-tiller=false \
    --advanced-mode

Notes: To setup and use other than Github Git provider see: https://jenkins-x.io/developing/git/

During the installation process, jx may ask you to confirm some settings, mostly related to Git configuration and domain name.

If you don’t have domain name configured in Route 53, choose not to register wildcard DNS name in Route 53 for you:

Jx requires to create a Git API token, which it will use to create and access Git repositories on your behalf. Follow the instructions from jx console to generate the API token (by clicking the link generated for you by jx).

In the GitHub dialog window, specify the unique name i.e. activiti-cloud, of the token as shown below:

Then click the green Generate Token button, copy the value of the token, and paste it to the jx cli input.

During the installation process you might notice that jx automatically opens the Jenkins server running on “http://jenkins.jx.35.164.211.214.nip.io/me/configure” address logs into it, and saves some settings. Jx uses headless browser automation to generate Jenkins API token on your behalf. This token will be later used internally by Jenkins X to automatically create Jenkins jobs on your behalf.

After a few minutes, the jx install command should finish creating the Jenkins X platform. It should also tell you the URL that you can use to access your Jenkins server, and a random admin password for this server.

The URL to access Jenkins will printed in the output of the console together with random admin password auto-generated by jx We now can use Jenkins credentials to log into our Jenkins server and see that Jenkins has successfully installed.

Note: after you have installed Jenkins-X, you may want to configure Jenkins-X to make all new repositories private by default with the following command:

jx edit gitprivate

To open Jenkins UI in the browser, run the following command::

jx console

In the Jenkins UI, you can confirm that Jenkins X has successfully installed Jenkins instance:

Close the Jenkins UI tab and go back to your shell.

Let’s Now Deploy Activiti Cloud in EKS Cluster using Jenkins X CI/CD with GitOps Promotion

Create Activiti Cloud Environments for GitOps

We will now proceed to provision two GitOps environments for deploying Activiti Cloud Platform: a staging environment and a production environment.

Get Jx Cluster Domain

Set the following environment variables:

export CLUSTER_DOMAIN=$(kubectl get cm ingress-config -o=go-template --template='{{.data.domain}}' -n jx) && echo $CLUSTER_DOMAIN
export ENV_PREFIX=$CLUSTER_NAME && echo $ENV_PREFIX

Create Activiti Cloud DevOps Environments

Run this command to configure environments using domain name of the cluster with the Activiti Cloud Environment Git repository used as the fork when creating new GitOps environment Git repo and custom environment repo prefix. Your Environment Git repo will be of the form 'environment-$prefix-$envName'

export ENV_REPO=https://github.com/activiti/activiti-cloud-environments.git

Select default options when running following commands to create Activiti Cloud Platform GitOps environments;

jx create env --domain $CLUSTER_DOMAIN \
    --fork-git-repo=$ENV_REPO \
    --prefix=$ENV_PREFIX \
    --name=staging \
    --namespace=staging \
    --git-private=true \
    --label=Staging \
    --promotion=Auto \
    --batch-mode
jx create env --domain $CLUSTER_DOMAIN \
    --fork-git-repo=$ENV_REPO \
    --prefix=$ENV_PREFIX \
    --name=production \
    --namespace=production \
    --git-private=true \
    --label=Production \
    --promotion=Manual \
    --batch-mode

After Jx created new environments, run jx console command to open Jenkins UI, you can confirm that Jenkins X has successfully provisioned two environments for us: a staging environment and a production environment:

jx console

You can also navigate to GitHub and see that Jenkins X provisioned the projects representing the environments’ Helm chart definitions as just built by Jenkins.

Configure Activiti Environmnents Domain Name

After Jx creates the environment Helm chart in your Git repository, open staging and production repos, then edit env/values.yaml file to set the value of global.gateway.domain key using expose.config.domain value.

expose:
  config:
    domain: X.X.X.X.nip.io # <==== Use this value to set global.gateway.domain

global:
  gateway:
    # Set value from expose.config.domain key, i.e. 1.2.3.4.nip.io
    domain: REPLACEME

Then, click to Commit the change. This should trigger Jenkins pipeline to rebuild and update the environments Helm chart configurations. We are now ready to deploy Activiti Cloud Platform Helm chart into GitOps environments.

Create Activiti Cloud Platform CI/CD Quickstarts

Create Activiti Cloud Quickstart Location in Jx Team Environment

We will need to create Activiti Jx Quickstart Location to use Activiti Quickstart CI/CD Templates published in Activiti Github https://github.com/Activiti organization repository:

Run this command to add Activiti Quickstart location for your team:

jx create quickstartlocation --owner activiti

To get the list of registered quickstart locations run command:

jx get quickstartlocations
GIT SERVER         KIND   OWNER                 INCLUDES EXCLUDES
https://github.com github jenkins-x-quickstarts *        WIP-*
https://github.com github activiti

Setup Activiti Cloud Platform in your Git repository

Simply run the following command to create your first Activiti Cloud platform Git repository:

jx create quickstart --owner activiti \
    --filter activiti-cloud-platform-quickstart \
    --project-name=activiti-cloud-application \
    --batch-mode

In terminal run Jx commands to monitor the deployment progress:

jx get activity -w
jx get pipelines
jx console

Then, after ~4-5 minutes you should see your Activiti Cloud Platform deployed into staging environment:

watch kubectl get pods -n staging

Setup Activiti Cloud Connector in your Git repository

Run the following command to create your first connector project Git repository:

jx create quickstart --owner activiti \
    --filter activiti-cloud-connector-quickstart \
    --project-name=activiti-my-connector \
    --batch-mode

Setup Activiti Cloud Runtime Bundle in your Git repository

Run the following command to create your first runtime bundle project Git repository:

jx create quickstart --owner activiti \
    --filter activiti-cloud-runtime-bundle-quickstart \
    --project-name=activiti-my-rb \
    --batch-mode

Then, after ~4-5 minutes you should see your Connector and Runtime Bundle deployed into staging environment:

watch kubectl get pods -n staging

Open Activiti Cloud Modeler

http://gateway.staging.1.2.3.4.nip.io/modeling

Here instead of 1.2.3.4.nip.io use the earlier $CLUSTER_DOMAIN environment variable, i.e.

echo gateway.$CLUSTER_DOMAIN/modeling

Login into using credentials: modeler/password