Website updates (#22)

* jenkins projct + website updates

projects/README.md

@@ -1,5 +1,15 @@
 # Projects
 
+## Jenkins Cluster
+[Jenkins](https://www.jenkins.io/doc/) is an open source orchestration tool for automation pipelines. It is traditionally run on servers with static IPs, however it can be run in a serverless environment. This implementation leverages ECS Fargate tasks and VPC endpoints making it a cost-effective, scalable, and secure."
+
+Component Details:
+ - [Overview](https://github.com/kspeer825/portfolio/tree/main/projects/jenkins/)
+ - [Application Load Balancer](https://github.com/kspeer825/portfolio/tree/main/projects/jenkins/terraform/alb.tf) accesible via white-listed IPs
+ - [ECS Cluster](https://github.com/kspeer825/portfolio/tree/main/projects/jenkins/terraform/ecs.tf) running Fargate tasks in a private subnet
+ - [VPC](https://github.com/kspeer825/portfolio/tree/main/projects/jenkins/terraform/vpc.tf) using AWS PrivateLink for service-to-service communication
+ - [ECR](https://github.com/kspeer825/portfolio/tree/main/projects/jenkins/terraform/ecr.tf) private repo for storing the [Jenkins Docker image](https://github.com/kspeer825/portfolio/tree/main/projects/jenkins/jenkins/)
+
 ## Personal Website
 My personal website, currently available at [speerportfolio.com](https://speerportfolio.com/), is meant to be an extension of my resume with details of personal projects and professionl experiences.
 

projects/jenkins/.gitignore

@@ -0,0 +1,5 @@
+jenkins_home/
+.*
+*.tfplan
+*.tfstate
+!.gitignore

projects/jenkins/Jenkinsfile

@@ -0,0 +1,47 @@
+pipeline {
+
+    agent any
+
+    environment {
+        SLEEP_TIME = 2
+        SECRET_TEXT = credentials('TEST_SECRET_TEXT')
+    }
+
+    stages {
+
+        stage('Build') {
+            steps {
+                sh 'echo "Building..."'
+                sh '''
+                    echo "Can execute multiple commands within a single step"
+                    ls -alh
+                '''
+            }
+        }
+
+        stage('Test') {
+            steps {
+                sh 'echo "Testing..."'
+                retry(3) {
+                    sh 'echo "I remembered to put echo before my string."'
+                }
+            }
+        }
+
+        stage('Deploy') {
+            steps {
+                sh 'echo "Deploying..."'
+                timeout(time: 3, unit: 'SECONDS') {
+                    sh 'sleep $SLEEP_TIME'
+                }
+                sh 'echo "($SECRET_TEXT)"'
+            }
+        }
+    }
+    post {
+        always {echo 'I always execute.'}
+        success {echo 'I execute on successful builds.'}
+        failure {echo 'I execute on failed builds.'}
+        unstable {echo 'I execute when build is unstable.'}
+    }
+}

projects/jenkins/Makefile

@@ -0,0 +1,92 @@
+SHELL := /bin/bash
+
+# Setup
+
+.PHONY: check-bucket check-env-vars debug-env setup-env
+
+check-bucket:
+ifndef AWS_BUCKET_NAME
+	$(error "AWS_BUCKET_NAME is not set.")
+endif
+
+check-env-vars: check-bucket
+
+debug-env:
+	env
+
+setup-env : check-env-vars
+	$(shell aws configure export-credentials > tmp_creds.json)
+	$(eval export AWS_ACCESS_KEY_ID=$(shell cat tmp_creds.json | jq -r .AccessKeyId))
+	$(eval export AWS_SECRET_ACCESS_KEY=$(shell cat tmp_creds.json | jq -r .SecretAccessKey))
+	@rm -f tmp_creds.json
+
+# Docker
+
+.PHONY: dc-build dc-build-clean dc-up dc-down dc-logs
+
+dir ?= jenkins
+f ?= ./docker-compose.yml
+dc = cd $(dir) && docker compose -f $(f)
+
+dc-build:
+	$(dc) build
+
+dc-build-clean:
+	$(dc) build --no-cache
+
+dc-up:
+	$(dc) up -d
+
+dc-down:
+	$(dc) down -v
+
+dc-logs:
+	$(dc) logs -f
+
+dc-run:
+	$(dc) run $(svc)
+
+# AWS
+
+.PHONY: ecr-login ecr-push
+
+image ?= jenkins
+version ?= $(shell cat ./jenkins/version.txt)
+region ?= us-east-1
+
+ecr-login: setup-env
+	aws ecr get-login-password --region $(region) | docker login --username AWS --password-stdin ${AWS_ACCOUNT_ID}.dkr.ecr.$(region).amazonaws.com
+
+ecr-push: ecr-login
+	docker tag $(image) $(image):$(version)
+	docker tag $(image):$(version) ${AWS_ACCOUNT_ID}.dkr.ecr.$(region).amazonaws.com/$(image)
+	docker push ${AWS_ACCOUNT_ID}.dkr.ecr.$(region).amazonaws.com/$(image)
+
+# Terraform
+
+.PHONY: tf-up tf-down tf-run
+
+tf-up:
+	cd terraform && docker compose up
+tf-down:
+	cd terraform && docker compose down
+tf-run:
+	cd terraform && docker compose run infra
+
+# Convenience rules
+
+.PHONY: nuke deploy-image
+
+nuke: dc-down dc-build-clean dc-up
+
+deploy-image: dc-build-clean ecr-push
+
+# Initial setup
+
+.PHONY: setup-home bootstrap
+
+setup-home:
+	mkdir -p jenkins_home
+
+bootstrap: setup-home dc-build-clean dc-up dc-logs
+	@echo "Bootstrap Complete."

projects/jenkins/README.md

@@ -0,0 +1,79 @@
+# A Jenkins cluster in ECS
+[Jenkins](https://www.jenkins.io/doc/) is an open source orchestration tool for automation pipelines. It is traditionally run on servers with static IPs, however it can be run in a serverless environment such as ECS Fargate. This implementation is meant for my own edification and not meant as a public module - although it is set up to run like one.
+
+The cluster runs in private subnets sitting behind an ALB which is publicly accessible only to white-listed IPs. The control node serves up the Jenkins UI and uses [amazon-ecs-fargate](https://github.com/jenkinsci/amazon-ecs-plugin) to dispatch builds to worker agents.
+
+## Prerequisites
+ - Docker
+ - AWS account
+ - AWS CLI and a configured profile w/ proper permissions
+ - ACM public certificate
+ - R53 Hosted Zone
+
+## Bootstrapping the Jenkins cluster
+
+Build the image and run the docker container locally.
+
+	make bootstrap
+
+The server is available at http://localhost:8080/. Use the temporary admin creds to login.
+
+![bootstrap-logs](/projects/jenkins/screenshots/bootstrap_logs.png?raw=true)
+
+The default plugins, and the ECS Fargate plugin for Jenkins will be installed [here](/projects/jenkins/jenkins/jenkins/plugins.yml).
+
+Additional plugins can be appended to the list, or installed manually under `Manage Jenkins > Plugins > Available Plugins`.
+
+![initial-plugins](/projects/jenkins/screenshots/initial_plugins.png?raw=true)
+
+Complete the initial setup by creating an admin user and log back in. You should see a page like this.
+
+![welcome-to-jenkins](/projects/jenkins/screenshots/welcome_to_jenkins.png?raw=true)
+
+Stand up a private ECR repo to store the image.
+
+	# Enter Terraform container
+	make tf-run
+
+	# Initialize the TF project
+	make tf-init
+
+	# Execute a targeted apply
+	make tf-plan options="-target=aws_ecr_repository.jenkins-repo"
+	make tf-apply
+
+Exit the Terraform container, then push the image up to ECR.
+
+	make ecr-push version=latest
+
+To launch the Jenkins server in ECS, you will need to set the following environment variables:
+
+	export AWS_BUCKET_NAME=<bucket-name>
+	export AWS_PROFILE=<profile>
+	export AWS_ACCOUNT_ID=<account
+
+	export TF_VAR_ip_allow_list=[\"<white-listed>\",\"<cidr-blocks>\"]
+	export TF_VAR_aws_r53_zone_id=<hosted-zone-id>
+	export TF_VAR_aws_r53_record_name=<A-record-name>
+
+
+The Terraform module will take care of spinning up the ALB, ECS service and task definition, Cloudwatch config, IAM roles and policies, additional security groups, and EFS mount points. It will launch these components in a new VPC with 2 private and 2 public subnets along with the necessary VPC endpoints.
+
+	# Enter Terraform container
+	make tf-run
+
+	# Initialize the TF project
+	make tf-init
+
+	# Deploy the stack
+	make tf-plan tf-apply
+
+NOTE: I am using VPC endpoints to enable connections from Fargate tasks in the private subnet to public endpoints in S3, ECR, and Cloudwatch. This is cheaper than running a NAT Gateway or assigning a public IP4 address to the control agent. But using AWS Privatelink will still incur costs (although much less) for the Interface endpoints. The main drawback with this approach is that each service requires it's own endpoint, and in some cases like ECR it requires multiple endpoints. With that said the cost savings more than justify it, you just need to keep in mind that any service that requires external traffic will need an endpoint.
+
+## ToDo's
+This is a work in progress, the following are outstanding tasks:
+  - [ ] Enable EFS backups
+  - [ ] Instructions for enabling ECS Jenkins cloud (worker agents) from control node
+  - [ ] Include example pipelines to run on Jenkins cluster
+  - [ ] Configure admin users at deploy time so you don't need to manually post-deploy
+  - [ ] Disable builds on the controller node (they should run on workers in the private subnet)

projects/jenkins/jenkins/Dockerfile

@@ -0,0 +1,10 @@
+FROM jenkins/jenkins:latest
+
+USER root
+
+RUN apt-get update && apt-get install -y make apt-transport-https ca-certificates gnupg curl unzip jq
+
+USER jenkins
+
+COPY --chown=jenkins:jenkins plugins.yml /usr/share/jenkins/ref/plugins.yml
+RUN jenkins-plugin-cli -f /usr/share/jenkins/ref/plugins.yml

projects/jenkins/jenkins/docker-compose.yml

@@ -0,0 +1,13 @@
+services:
+  jenkins:
+    image: jenkins
+    build:
+      context: .
+    ports:
+      - "8080:8080"
+    volumes:
+      - jenkins_home:/var/jenkins_home
+  ssh-agent:
+    image: jenkins/ssh-agent
+volumes:
+  jenkins_home: