Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Converted to Chef provisioning #174

Open
wants to merge 84 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
84 commits
Select commit Hold shift + click to select a range
31c1140
Initial chef commit
Chan9390 Jul 28, 2017
d10e6a6
Disable auto logon
Chan9390 Jul 28, 2017
25f699d
Disabled firewall
Chan9390 Jul 28, 2017
48c17d8
Changed name of batch resource
Chan9390 Jul 28, 2017
5d350cf
Install Boxstarter
Chan9390 Jul 28, 2017
58ffb5b
Added 7zip
Chan9390 Jul 28, 2017
00c6ea3
Apply password settings
Chan9390 Jul 28, 2017
f116738
Setup FTP
Chan9390 Jul 28, 2017
109fac0
Install Java
Chan9390 Jul 28, 2017
44dc582
Added Tomcat
Chan9390 Jul 28, 2017
e0872d2
Added vcredist2008
Chan9390 Jul 28, 2017
9ec163c
Added Ruby
Chan9390 Jul 28, 2017
43be6e4
Squashed commit of the following:
Chan9390 Jul 28, 2017
c07e1b6
Committed the following:
Chan9390 Jul 28, 2017
d1dfa35
Disabled Auto-logon
Chan9390 Jul 28, 2017
73589c0
Squashed commit of the following:
Chan9390 Jul 28, 2017
613733a
Disabled firewall
Chan9390 Jul 28, 2017
a35dfd2
Install BoxStarter
Chan9390 Jul 28, 2017
cadd5da
Added 7zip
Chan9390 Jul 28, 2017
b7e0295
Configure password settings
Chan9390 Jul 28, 2017
68c47da
Squashed commit of the following:
Chan9390 Jul 29, 2017
728c50e
Squashed commit of the following:
Chan9390 Jul 29, 2017
601c682
Setup FTP
Chan9390 Jul 29, 2017
818315f
Added Java
Chan9390 Jul 29, 2017
ffd101b
Added Tomcat
Chan9390 Jul 29, 2017
96d33bd
Squashed commit of the following:
Chan9390 Jul 29, 2017
bd7ee0e
Squashed commit of the following:
Chan9390 Jul 29, 2017
6a6ed95
Squashed commit of the following:
Chan9390 Jul 29, 2017
47c2416
Added vcredist2008
Chan9390 Jul 29, 2017
589d740
Squashed commit of the following:
Chan9390 Jul 29, 2017
b0c993b
Squashed commit of the following:
Chan9390 Jul 29, 2017
7ceab05
Squashed commit of the following:
Chan9390 Jul 29, 2017
32a562b
Squashed commit of the following:
Chan9390 Jul 29, 2017
8145179
Added Ruby
Chan9390 Jul 29, 2017
22789a6
Install DevKit
Chan9390 Jul 29, 2017
892cfc9
Squashed commit of the following:
Chan9390 Jul 30, 2017
7f33ee1
Squashed commit of the following:
Chan9390 Jul 31, 2017
bae4899
Squashed commit of the following:
Chan9390 Jul 31, 2017
2febec1
Squashed commit of the following:
Chan9390 Aug 1, 2017
e6fe586
Added backdoors
Chan9390 Aug 1, 2017
f9be0e2
Added SNMP
Chan9390 Aug 1, 2017
3c1b46d
- Disabled firewall
Chan9390 Aug 1, 2017
1ff52bb
Squashed commit of the following:
Chan9390 Aug 1, 2017
d91f8b3
Updated metadata.rb
Chan9390 Aug 1, 2017
282bf63
Squashed commit of the following:
Chan9390 Aug 3, 2017
7b5f0b0
Squashed commit of the following:
Chan9390 Aug 3, 2017
424b35c
Added final packer cleaning
Chan9390 Aug 3, 2017
c8289b5
Squashed commit of the following:
Chan9390 Aug 3, 2017
e4e13d4
Changed service to elasticsearch x64
Chan9390 Aug 3, 2017
fb1c9b5
Disabled firewall in autounattend
Chan9390 Aug 4, 2017
429cdd5
Linked chef recipes
Chan9390 Aug 4, 2017
a588e8a
Corrected mistake in ftp recipe
Chan9390 Aug 11, 2017
e2122bd
Added jenkins.exe
Chan9390 Aug 12, 2017
31c2340
Changed batch resource to execute in 7zip
Chan9390 Aug 12, 2017
e82bc65
Added JMX.exe
Chan9390 Aug 12, 2017
9f10e0b
Changed batch resource to execute in ruby
Chan9390 Aug 12, 2017
9006d4e
Added JDK 8
Chan9390 Aug 13, 2017
b4e2607
Corrected mistake in elasticsearch recipe
Chan9390 Aug 13, 2017
c11a5f1
Corrected mistake in packer file
Chan9390 Aug 13, 2017
62ced68
Removed file provisioners
Chan9390 Aug 13, 2017
16d7352
Updated flags
Chan9390 Aug 13, 2017
e1a2006
Updated files
Chan9390 Aug 13, 2017
0cf5abe
Converted batch to execute resource
Chan9390 Aug 13, 2017
3ea874a
Updated rails
Chan9390 Aug 13, 2017
bb1c6fd
Added 30s pause to get powershell completely installed
Chan9390 Aug 14, 2017
749d9c7
Updated metadata.rb
Chan9390 Aug 16, 2017
448db9c
Added -y tag to vcredist install
Chan9390 Aug 16, 2017
ded8b00
Moved execute resource to chocolatey_package
Chan9390 Aug 16, 2017
12f36ee
Linked all recipes with their dependencies
Chan9390 Aug 17, 2017
5c422bd
Added IIS and SNMP
Chan9390 Aug 19, 2017
45689b9
Removed BoxStarter from packer provisioner
Chan9390 Aug 28, 2017
28e6e76
Replaced batch resource to choco in BoxStarter recipe
Chan9390 Aug 28, 2017
bd93c20
Added file
Chan9390 Aug 28, 2017
020ac27
Linked b64 file
Chan9390 Aug 28, 2017
a01119d
Added vagrantfile to versions folder
Chan9390 Aug 29, 2017
f9ce6cc
Removed executable file permission
Chan9390 Aug 29, 2017
e9e4e9b
Updated elasticsearch.rb
Chan9390 Aug 30, 2017
a9b9f19
Added rails app
Chan9390 Sep 1, 2017
12c5eb2
Updated elasticsearch.rb
Chan9390 Sep 1, 2017
4b71961
Added check condition in elasticsearch.rb
Chan9390 Sep 1, 2017
eba06a6
Converted sleep to ruby_block
Chan9390 Sep 1, 2017
9751b97
Squashed commit of the following:
Chan9390 Sep 1, 2017
1f549fd
Minor changes
Chan9390 Sep 2, 2017
bf97e32
Added firewall rules
Chan9390 Sep 4, 2017
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions answer_files/2008_r2/Autounattend.xml
Original file line number Diff line number Diff line change
Expand Up @@ -255,6 +255,11 @@
<Description>Create directory for vagrant files to avoid provisioner bug with packer.</Description>
<Order>26</Order>
</SynchronousCommand>
<SynchronousCommand wcm:action="add">
<CommandLine>cmd.exe /c netsh advfirewall set allprofiles state off</CommandLine>
<Description>Disable firewall for vagrant to connect.</Description>
<Order>27</Order>
</SynchronousCommand>
<SynchronousCommand wcm:action="add">
<CommandLine>cmd.exe /c C:\Windows\System32\WindowsPowerShell\v1.0\powershell.exe -File a:\install_dotnet45.ps1 -AutoStart</CommandLine>
<Order>97</Order>
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
---
driver:
name: vagrant
synced_folders:
- [<%= File.join(ENV['PWD'], '..', '..')%>, '/tmp/repo-data']

provisioner:
name: chef_zero
encrypted_data_bag_secret_key_path: 'secrets/fakey-mcfakerton'
data_bags_path: './data_bags'
product_name: chefdk

platforms:
- name: ubuntu-16.04
- name: centos-7.3

suites:
- name: default
run_list:
- recipe[test]
attributes:
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
source 'https://supermarket.chef.io'

metadata

group :delivery do
cookbook 'test', path: './test/fixtures/cookbooks/test'
end
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
Copyright 2017 The Authors

All rights reserved, do not redistribute.
146 changes: 146 additions & 0 deletions chef/cookbooks/metasploitable/.delivery/build_cookbook/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,146 @@
# build_cookbook

A build cookbook for running the parent project through Chef Delivery

This build cookbook should be customized to suit the needs of the parent project. Using this cookbook can be done outside of Chef Delivery, too. If the parent project is a Chef cookbook, we've detected that and "wrapped" [delivery-truck](https://github.com/chef-cookbooks/delivery-truck). That means it is a dependency, and each of its pipeline phase recipes is included in the appropriate phase recipes in this cookbook. If the parent project is not a cookbook, it's left as an exercise to the reader to customize the recipes as needed for each phase in the pipeline.

## .delivery/config.json

In the parent directory to this build_cookbook, the `config.json` can be modified as necessary. For example, phases can be skipped, publishing information can be added, and so on. Refer to customer support or the Chef Delivery documentation for assistance on what options are available for this configuration.

## Test Kitchen - Local Verify Testing

This cookbook also has a `.kitchen.yml` which can be used to create local build nodes with Test Kitchen to perform the verification phases, `unit`, `syntax`, and `lint`. When running `kitchen converge`, the instances will be set up like Chef Delivery "build nodes" with the [delivery_build cookbook](https://github.com/chef-cookbooks/delivery_build). The reason for this is to make sure that the same exact kind of nodes are used by this build cookbook are run on the local workstation as would run Delivery. It will run `delivery job verify PHASE` for the parent project.

Modify the `.kitchen.yml` if necessary to change the platforms or other configuration to run the verify phases. After making changes in the parent project, `cd` into this directory (`.delivery/build_cookbook`), and run:

```
kitchen test
```

## Recipes

Each of the recipes in this build_cookbook are run in the named phase during the Chef Delivery pipeline. The `unit`, `syntax`, and `lint` recipes are additionally run when using Test Kitchen for local testing as noted in the above section.

## Making Changes - Cookbook Example

When making changes in the parent project (that which lives in `../..` from this directory), or in the recipes in this build cookbook, there is a bespoke workflow for Chef Delivery. As an example, we'll discuss a Chef Cookbook as the parent.

First, create a new branch for the changes.

```
git checkout -b testing-build-cookbook
```

Next, increment the version in the metadata.rb. This should be in the _parent_, not in this, the build_cookbook. If this is not done, the verify phase will fail.

```
% git diff
<SNIP>
-version '0.1.0'
+version '0.1.1'
```

The change we'll use for an example is to install the `zsh` package. Write a failing ChefSpec in the cookbook project's `spec/unit/recipes/default_spec.rb`.

```ruby
require 'spec_helper'

describe 'godzilla::default' do
context 'When all attributes are default, on Ubuntu 16.04' do
let(:chef_run) do
runner = ChefSpec::ServerRunner.new(platform: 'ubuntu', version: '16.04')
runner.converge(described_recipe)
end

it 'installs zsh' do
expect(chef_run).to install_package('zsh')
end
end
end
```

Commit the local changes as work in progress. The `delivery job` expects to use a clean git repository.

```
git add ../..
git commit -m 'WIP: Testing changes'
```

From _this_ directory (`.delivery/build_cookbook`, relative to the parent cookbook project), run

```
cd .delivery/build_cookbook
kitchen converge
```

This will take some time at first, because the VMs need to be created, Chef installed, the Delivery CLI installed, etc. Later runs will be faster until they are destroyed. It will also fail on the first VM, as expected, because we wrote the test first. Now edit the parent cookbook project's default recipe to install `zsh`.

```
cd ../../
$EDITOR/recipes/default.rb
```

It should look like this:

```
package 'zsh'
```

Create another commit.

```
git add .
git commit -m 'WIP: Install zsh in default recipe'
```

Now rerun kitchen from the build_cookbook.

```
cd .delivery/build_cookbook
kitchen converge
```

This will take awhile because it will now pass on the first VM, and then create the second VM. We should have warned you this was a good time for a coffee break.

```
Recipe: test::default
- execute HOME=/home/vagrant delivery job verify unit --server localhost --ent test --org kitchen
* execute[HOME=/home/vagrant delivery job verify lint --server localhost --ent test --org kitchen] action run
- execute HOME=/home/vagrant delivery job verify lint --server localhost --ent test --org kitchen
- execute HOME=/home/vagrant delivery job verify syntax --server localhost --ent test --org kitchen
Running handlers:
Running handlers complete
Chef Client finished, 3/32 resources updated in 54.665445968 seconds
Finished converging <default-centos-71> (1m26.83s).
```

Victory is ours! Our verify phase passed on the build nodes.

We are ready to run this through our Delivery pipeline. Simply run `delivery review` on the local system from the parent project, and it will open a browser window up to the change we just added.

```
cd ../..
delivery review
```

## FAQ

### Why don't I just run rspec and foodcritic/rubocop on my local system?

An objection to the Test Kitchen approach is that it is much faster to run the unit, lint, and syntax commands for the project on the local system. That is totally true, and also totally valid. Do that for the really fast feedback loop. However, the dance we do with Test Kitchen brings a much higher degree of confidence in the changes we're making, that everything will run on the build nodes in Chef Delivery. We strongly encourage this approach before actually pushing the changes to Delivery.

### Why do I have to make a commit every time?

When running `delivery job`, it expects to merge the commit for the changeset against the clean master branch. If we don't save our progress by making a commit, our local changes aren't run through `delivery job` in the Test Kitchen build instances. We can always perform an interactive rebase, and modify the original changeset message in Delivery with `delivery review --edit`. The latter won't modify the git commits, only the changeset in Delivery.

### What do I do next?

Make changes in the cookbook project as required for organizational goals and needs. Modify the `build_cookbook` as necessary for the pipeline phases that the cookbook should go through.

### What if I get stuck?

Contact Chef Support, or your Chef Customer Success team and they will help you get unstuck.
107 changes: 107 additions & 0 deletions chef/cookbooks/metasploitable/.delivery/build_cookbook/chefignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,107 @@
# Put files/directories that should be ignored in this file when uploading
# to a chef-server or supermarket.
# Lines that start with '# ' are comments.

# OS generated files #
######################
.DS_Store
Icon?
nohup.out
ehthumbs.db
Thumbs.db

# SASS #
########
.sass-cache

# EDITORS #
###########
\#*
.#*
*~
*.sw[a-z]
*.bak
REVISION
TAGS*
tmtags
*_flymake.*
*_flymake
*.tmproj
.project
.settings
mkmf.log

## COMPILED ##
##############
a.out
*.o
*.pyc
*.so
*.com
*.class
*.dll
*.exe
*/rdoc/

# Testing #
###########
.watchr
.rspec
spec/*
spec/fixtures/*
test/*
features/*
examples/*
Guardfile
Procfile
.kitchen*
.rubocop.yml
spec/*
Rakefile
.travis.yml
.foodcritic
.codeclimate.yml

# SCM #
#######
.git
*/.git
.gitignore
.gitmodules
.gitconfig
.gitattributes
.svn
*/.bzr/*
*/.hg/*
*/.svn/*

# Berkshelf #
#############
Berksfile
Berksfile.lock
cookbooks/*
tmp

# Policyfile #
##############
Policyfile.rb
Policyfile.lock.json

# Cookbooks #
#############
CONTRIBUTING*
CHANGELOG*
TESTING*
MAINTAINERS.toml

# Strainer #
############
Colanderfile
Strainerfile
.colander
.strainer

# Vagrant #
###########
.vagrant
Vagrantfile
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
{"id": "delivery_builder_keys"}
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
name 'build_cookbook'
maintainer 'The Authors'
maintainer_email '[email protected]'
license 'all_rights'
version '0.1.0'
chef_version '>= 12.1' if respond_to?(:chef_version)

depends 'delivery-truck'
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
#
# Cookbook:: build_cookbook
# Recipe:: default
#
# Copyright:: 2017, The Authors, All Rights Reserved.
include_recipe 'delivery-truck::default'
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
#
# Cookbook:: build_cookbook
# Recipe:: deploy
#
# Copyright:: 2017, The Authors, All Rights Reserved.
include_recipe 'delivery-truck::deploy'
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
#
# Cookbook:: build_cookbook
# Recipe:: functional
#
# Copyright:: 2017, The Authors, All Rights Reserved.
include_recipe 'delivery-truck::functional'
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
#
# Cookbook:: build_cookbook
# Recipe:: lint
#
# Copyright:: 2017, The Authors, All Rights Reserved.
include_recipe 'delivery-truck::lint'
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
#
# Cookbook:: build_cookbook
# Recipe:: provision
#
# Copyright:: 2017, The Authors, All Rights Reserved.
include_recipe 'delivery-truck::provision'
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
#
# Cookbook:: build_cookbook
# Recipe:: publish
#
# Copyright:: 2017, The Authors, All Rights Reserved.
include_recipe 'delivery-truck::publish'
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
#
# Cookbook:: build_cookbook
# Recipe:: quality
#
# Copyright:: 2017, The Authors, All Rights Reserved.
include_recipe 'delivery-truck::quality'
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
#
# Cookbook:: build_cookbook
# Recipe:: security
#
# Copyright:: 2017, The Authors, All Rights Reserved.
include_recipe 'delivery-truck::security'
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
#
# Cookbook:: build_cookbook
# Recipe:: smoke
#
# Copyright:: 2017, The Authors, All Rights Reserved.
include_recipe 'delivery-truck::smoke'
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
#
# Cookbook:: build_cookbook
# Recipe:: syntax
#
# Copyright:: 2017, The Authors, All Rights Reserved.
include_recipe 'delivery-truck::syntax'
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
#
# Cookbook:: build_cookbook
# Recipe:: unit
#
# Copyright:: 2017, The Authors, All Rights Reserved.
include_recipe 'delivery-truck::unit'
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
name 'test'
version '0.1.0'
Loading