diff --git a/CODE_OF_CONDUCT.md b/CODE_OF_CONDUCT.md index 32a47d5..be3c59c 100644 --- a/CODE_OF_CONDUCT.md +++ b/CODE_OF_CONDUCT.md @@ -55,7 +55,7 @@ further defined and clarified by project maintainers. ## Enforcement Instances of abusive, harassing, or otherwise unacceptable behavior may be -reported by contacting the project team at [INSERT EMAIL ADDRESS]. All +reported by contacting the project team at `info@labzero.com`. All complaints will be reviewed and investigated and will result in a response that is deemed necessary and appropriate to the circumstances. The project team is obligated to maintain confidentiality with regard to the reporter of an incident. diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md new file mode 100644 index 0000000..77dec94 --- /dev/null +++ b/CONTRIBUTING.md @@ -0,0 +1,68 @@ +# Contributing + +You want to contribute? Awesome! We'd love the help. If you have an idea already, great. If not, +take a look at our [issue tracker][issues] and see if anything appeals. More tests and +documentation are always appreciated too. + +## Getting Started + +1. Fork the repository +2. Make sure the tests pass locally _before_ you start developing. +3. Write a test or two that cover your feature/bug/refactor (not needed for documentation-only changes) +4. Make your test pass by adding that slick new code. +5. Add documentation for your change (if appropriate) +6. Run `mix credo --strict` and `mix dialyzer` to ensure you haven't missed any coding standards. +7. Commit your changes and open a [pull request][pulls]. +8. Wait for a speedy review of your change! Thank you! + +## Development Dependencies + +In order to run the functional tests, you need to have [Docker][docker] installed locally. The +community edition is fine, but you'll want to avoid the old versions that require a VM. + +## Code Standards + +Most of the code standards can be found in `.credo.exs`, and will be checked automatically by the +CI process. When in doubt, follow the standards in the file you are changing. Terse but descriptive +variable and function names make us happy. The standard Elixir guide on [writing documentation][writing-docs] +has some good tips on names. Documentation for new public functions is expected, as are tests for +any code change. + +Good commit messages and PR descriptions are also important. See our guide on +[commit messages](https://github.com/labzero/guides/blob/master/process/commit_guide.md) for more details. + +## Testing + +Good tests are arguably more important than good code, so please take a moment to make sure +you have a few with your PR. Try to avoid mock-only tests, as they can get out of sync with reality +fairly easily. They are great for doing basic unit testing though! You'll see we use +[mock](https://github.com/jjh42/mock) as our mocking framework of choice. + +Functional tests are much more reliable with a tool like Bootleg, and there are plenty of examples +in the project. `Bootleg.FunctionalCase` provides a simple interface for writing [Docker][docker] +based functional tests. By default each test case will get a single docker container provisioned, +and the details will be passed to `setup` under the key `hosts`. You can request more containers +using `@tag boot: 2` where `2` is the number of containers you'd like. During test development it's +often helpful to have the containers left running after the tests finish, and you can request that +by setting the `ENV` variable `TEST_LEAVE_CONTAINER` when running your tests. It's best to limit how +many tests are run in that case, or you may kill your machine with too many docker containers at once. + +If you need a project to test against (this a deployment tool after all), take a look at +`Bootleg.Fixtures.inflate_project/1`. It will take any of the fixture projects and create a new +instance for use during testing. The `test/fixtures` directory contains all the currently available +fixture projects. Instances of projects created via `inflate_project/1` will be cleaned up when the +test suite exits, but you can suppress that by setting `TEST_LEAVE_TEMP` in the `ENV`. Fixtures are +always inflated to your OS temporary directory. + +## Contact + +You can reach the core Bootleg team in [#deployment](https://elixir-lang.slack.com/messages/C0LH49EPQ) +or [#bootleg](https://elixir-lang.slack.com/messages/C6D2BQY4R) on Elixir Slack. We are also reachable +via email at `bootleg@labzero.com`. Don't hesitate to get in touch, we'd love to hear from you. + +Use the [issue tracker][issues] for bug reports or feature requests. + + [issues]: https://github.com/labzero/bootleg/issues + [pulls]: https://github.com/labzero/bootleg/pulls + [writing-docs]: http://elixir-lang.org/docs/stable/elixir/writing-documentation.html + [docker]: https://www.docker.com/ diff --git a/README.md b/README.md index 7725074..13a0bc3 100644 --- a/README.md +++ b/README.md @@ -15,7 +15,7 @@ add additional support. ```elixir def deps do [{:distillery, "~> 1.3", - {:bootleg, "~> 0.1.0"}] + {:bootleg, "~> 0.3"}] end ``` @@ -151,6 +151,9 @@ Alternatively the above commands can be rolled into one with: mix bootleg.update production ``` +Note that `bootleg.update` will stop any running nodes and then perform a cold start. The stop is performed with +the task `stop_silent`, which differs from `stop` in that it does not fail if the node is already stopped. + ## Admin Commands Bootleg has a set of commands to check up on your running nodes: @@ -162,6 +165,15 @@ mix bootleg.stop production # Stops a deployed release. mix bootleg.ping production # Check status of running nodes ``` +## Other Comamnds + +Bootleg has a few utility commands to help streamline its usage: + +```console +mix bootleg.init # Initializes a project for use with Bootleg +mix bootleg.invoke # Calls an arbitrary Bootleg task +``` + ## Hooks Hooks may be defined by the user in order to perform additional (or exceptional) @@ -287,7 +299,8 @@ end The workhorse of the Bootleg DSL is `remote`: it executes shell commands on remote servers and returns the results. It takes a role and a block of commands to execute. The commands are executed on all servers -belonging to the role, and raises an `SSHError` if an error is encountered. +belonging to the role, and raises an `SSHError` if an error is encountered. Optionally, a list of options +can be provided to filter the hosts where the commands are run. ```elixir use Bootleg.Config @@ -312,6 +325,11 @@ end remote :app do "false" end + +# filtering - only runs on app hosts with an option of primary set to true +remote :app, primary: true do + "mix ecto.migrate" +end ``` ## Phoenix Support @@ -326,8 +344,8 @@ for building phoenix releases. # mix.exs def deps do [{:distillery, "~> 1.3"}, - {:bootleg, "~> 0.2.0"}, - {:bootleg_phoenix, "~> 0.1.0"}] + {:bootleg, "~> 0.3"}, + {:bootleg_phoenix, "~> 0.1"}] end ``` diff --git a/lib/bootleg/config.ex b/lib/bootleg/config.ex index 80abf2e..9415f6f 100644 --- a/lib/bootleg/config.ex +++ b/lib/bootleg/config.ex @@ -9,7 +9,7 @@ defmodule Bootleg.Config do defmacro __using__(_) do quote do import Bootleg.Config, only: [role: 2, role: 3, config: 2, config: 0, before_task: 2, - after_task: 2, invoke: 1, task: 2, remote: 1, remote: 2, load: 1] + after_task: 2, invoke: 1, task: 2, remote: 1, remote: 2, remote: 3, load: 1, upload: 3] end end @@ -22,7 +22,7 @@ defmodule Bootleg.Config do `name` is the name of the role, and is globally unique. Calling `role/3` multiple times with the same name will result in the host lists being merged. If the same host shows up mutliple - times, it will have its `options` merged. + times, it will have its `options` merged. The name `:all` is reserved and cannot be used here. `hosts` can be a single hostname, or a `List` of hostnames. @@ -41,6 +41,9 @@ defmodule Bootleg.Config do """ defmacro role(name, hosts, options \\ []) do # user is in the role options for scm + if name == :all do + raise ArgumentError, ":all is reserved by bootleg and refers to all defined roles." + end user = Keyword.get(options, :user, System.get_env("USER")) ssh_options = Enum.filter(options, &Enum.member?(SSH.supported_options, elem(&1, 0)) == true) role_options = @@ -325,11 +328,28 @@ defmodule Bootleg.Config do end defmacro remote(role, do: {:__block__, _, lines}) do - quote do: remote(unquote(role), unquote(lines)) + quote do: remote(unquote(role), [], unquote(lines)) end defmacro remote(role, do: lines) do - quote do: remote(unquote(role), unquote(lines)) + quote do: remote(unquote(role), [], unquote(lines)) + end + + @doc """ + Executes commands on all remote hosts within a role. + + This is equivalent to calling `remote/3` with a `filter` of `[]`. + """ + defmacro remote(role, lines) do + quote do: remote(unquote(role), [], unquote(lines)) + end + + defmacro remote(role, filter, do: {:__block__, _, lines}) do + quote do: remote(unquote(role), unquote(filter), unquote(lines)) + end + + defmacro remote(role, filter, do: lines) do + quote do: remote(unquote(role), unquote(filter), unquote(lines)) end @doc """ @@ -343,6 +363,10 @@ defmodule Bootleg.Config do used as a command. Each command will be simulataneously executed on all hosts in the role. Once all hosts have finished executing the command, the next command in the list will be sent. + `filter` is an optional `Keyword` list of host options to filter with. Any host whose options match + the filter will be included in the remote execution. A host matches if it has all of the filtering + options defined and the values match (via `==/2`) the filter. + `role` can be a single role, a list of roles, or the special role `:all` (all roles). If the same host exists in multiple roles, the commands will be run once for each role where the host shows up. In the case of multiple roles, each role is processed sequentially. @@ -367,18 +391,22 @@ defmodule Bootleg.Config do # runs for hosts found in :build first, then for hosts in :app remote [:build, :app], do: "hostname" + + # only runs on `host1.example.com` + role :build, "host2.example.com" + role :build, "host1.example.com", primary: true, another_attr: :cat + + remote :build, primary: true do + "hostname" + end ``` """ - defmacro remote(role, lines) do - roles = if role == :all do - quote do: Keyword.keys(Bootleg.Config.Agent.get(:roles)) - else - quote do: List.wrap(unquote(role)) - end + defmacro remote(role, filter, lines) do + roles = unpack_role(role) quote bind_quoted: binding() do Enum.reduce(roles, [], fn role, outputs -> role - |> SSH.init + |> SSH.init([], filter) |> SSH.run!(lines) |> SSH.merge_run_results(outputs) end) @@ -401,6 +429,56 @@ defmodule Bootleg.Config do end end + @doc """ + Uploads a local file to remote hosts. + + Uploading works much like `remote/3`, but instead of transferring shell commands over SSH, + it transfers files via SCP. The remote host does need to support SCP, which should be provided + by most SSH implementations automatically. + + `role` can either be a single role name, a list of roles, or a list of roles and filter + attributes. The special `:all` role is also supported. See `remote/3` for details. + + `local_path` can either be a file or directory found on the local machine. If its a directory, + the entire directory will be recursively copied to the remote hosts. Relative paths are resolved + relative to the root of the local project. + + `remote_path` is the file or directory where the transfered files should be placed. The semantics + of how `remote_path` is treated vary depending on what `local_path` refers to. If `local_path` points + to a file, `remote_path` is treated as a file unless it's `.` or ends in `/`, in which case it's + treated as a directory and the filename of the local file will be used. If `local_path` is a directory, + `remote_path` is treated as a directory as well. Relative paths are resolved relative to the projects + remote `workspace`. Missing directories are not implicilty created. + + The files on the remote server are created using the authenticating user's `uid`/`gid` and `umask`. + + ``` + use Bootleg.Config + + # copies ./my_file to ./new_name on the remote host + upload :app, "my_file", "new_name" + + # copies ./my_file to ./a_dir/my_file on the remote host. ./a_dir must already exist + upload :app, "my_file", "a_dir/" + + # recursively copies ./some_dir to ./new_dir on the remote host. ./new_dir will be created if missing + upload :app, "some_dir", "new_dir" + + # copies ./my_file to /tmp/foo on the remote host + upload :app, "my_file", "/tmp/foo" + """ + defmacro upload(role, local_path, remote_path) do + {roles, filters} = split_roles_and_filters(role) + roles = unpack_role(roles) + quote bind_quoted: binding() do + Enum.each(roles, fn role -> + role + |> SSH.init([], filters) + |> SSH.upload(local_path, remote_path) + end) + end + end + @doc false @spec get_config(atom, any) :: any def get_config(key, default \\ nil) do @@ -418,4 +496,23 @@ defmodule Bootleg.Config do def version do get_config(:version, Project.config[:version]) end + + @doc false + @spec split_roles_and_filters(atom | keyword) :: {[atom], keyword} + defp split_roles_and_filters(role) do + role + |> List.wrap + |> Enum.split_while(fn term -> !is_tuple(term) end) + end + + @doc false + @spec unpack_role(atom | keyword) :: tuple + defp unpack_role(role) do + wrapped_role = List.wrap(role) + if Enum.any?(wrapped_role, fn role -> role == :all end) do + quote do: Keyword.keys(Bootleg.Config.Agent.get(:roles)) + else + quote do: unquote(wrapped_role) + end + end end diff --git a/lib/bootleg/ssh.ex b/lib/bootleg/ssh.ex index acf5964..cee152b 100644 --- a/lib/bootleg/ssh.ex +++ b/lib/bootleg/ssh.ex @@ -6,32 +6,46 @@ defmodule Bootleg.SSH do alias SSHKit.SSH, as: SSHKitSSH alias Bootleg.{UI, Host, Role, Config} - def init(role, options \\ []) - def init(%Role{} = role, options) do + def init(%Role{} = role, options, filter) do role_options = Keyword.merge(role.options, [user: role.user]) - init(role.hosts, Keyword.merge(role_options, options)) + role.hosts + |> Enum.reduce([], fn host, acc -> + if filter_match?(host.options, filter) do + acc ++ [host] + else + acc + end + end) + |> init(Keyword.merge(role_options, options)) end - def init(nil, _options) do + def init(nil, _options, _filter) do raise ArgumentError, "You must supply a %Host{}, a %Role{} or a defined role_name." end - def init(role_name, options) when is_atom(role_name) do - init(Config.get_role(role_name), options) + def init(role_name, options, filter) when is_atom(role_name) do + init(Config.get_role(role_name), options, filter) end + def init(role, options \\ []) + def init(role, options) when is_atom(role) do + init(role, options, []) + end + def init(%Role{} = role, options) do + init(role, options, []) + end def init(hosts, options) do - workspace = Keyword.get(options, :workspace, ".") - create_workspace = Keyword.get(options, :create_workspace, true) - UI.puts "Creating remote context at '#{workspace}'" + workspace = Keyword.get(options, :workspace, ".") + create_workspace = Keyword.get(options, :create_workspace, true) + UI.puts "Creating remote context at '#{workspace}'" - :ssh.start() + :ssh.start() - hosts - |> List.wrap - |> Enum.map(&ssh_host_options/1) - |> SSHKit.context - |> validate_workspace(workspace, create_workspace) + hosts + |> List.wrap + |> Enum.map(&ssh_host_options/1) + |> SSHKit.context + |> validate_workspace(workspace, create_workspace) end def ssh_host_options(%Host{} = host) do @@ -111,8 +125,9 @@ defmodule Bootleg.SSH do def upload(conn, local_path, remote_path) do UI.puts_upload conn, local_path, remote_path - case SSHKit.upload(conn, local_path, as: remote_path) do + case SSHKit.upload(conn, local_path, as: remote_path, recursive: true) do [:ok|_] -> :ok + [] -> :ok [{_, msg}|_] -> raise "SCP upload error #{msg}" end end @@ -142,6 +157,9 @@ defmodule Bootleg.SSH do def merge_run_results(new, []) do new end + def merge_run_results([], orig) do + orig + end def merge_run_results(new, orig) when is_list(orig) do new |> Enum.zip(orig) @@ -149,4 +167,14 @@ defmodule Bootleg.SSH do List.wrap(o) ++ List.wrap(n) end) end + + defp filter_match?(list, filter) do + Enum.reduce(filter, true, fn {key, value}, match -> + if match do + list[key] == value + else + match + end + end) + end end diff --git a/lib/bootleg/tasks/update.exs b/lib/bootleg/tasks/update.exs index 39dd6d7..1132eb1 100644 --- a/lib/bootleg/tasks/update.exs +++ b/lib/bootleg/tasks/update.exs @@ -3,5 +3,13 @@ use Bootleg.Config task :update do invoke :build invoke :deploy + invoke :stop_silent invoke :start end + +task :stop_silent do + nodetool = "bin/#{Config.app}" + remote :app do + "#{nodetool} describe && (#{nodetool} stop || true)" + end +end diff --git a/lib/mix/tasks/invoke.ex b/lib/mix/tasks/invoke.ex new file mode 100644 index 0000000..639fca0 --- /dev/null +++ b/lib/mix/tasks/invoke.ex @@ -0,0 +1,27 @@ +defmodule Mix.Tasks.Bootleg.Invoke do + use Bootleg.MixTask + alias Bootleg.{UI, Config} + + @shortdoc "Calls an arbitrary Bootleg task" + + @moduledoc """ + #{@shortdoc} + + # Usage: + + * mix bootleg.invoke <:task> + + """ + + def run([]) do + UI.error "You must supply a task identifier as the first argument." + System.halt(1) + end + + def run([task | _]) do + use Config + + invoke String.to_atom(task) + end + +end diff --git a/lib/mix/tasks/update.ex b/lib/mix/tasks/update.ex index a228528..d9ff620 100644 --- a/lib/mix/tasks/update.ex +++ b/lib/mix/tasks/update.ex @@ -4,7 +4,11 @@ defmodule Mix.Tasks.Bootleg.Update do @shortdoc "Build, deploy, and start a release all in one command." @moduledoc """ - Update a release + Update a release. + + Note that this will stop any running nodes and then perform a cold start. The stop is performed with + the task `stop_silent`, which differs from `stop` in that it does not require a node to already be + running. # Usage: diff --git a/lib/ui.ex b/lib/ui.ex index 8e04236..cc004db 100644 --- a/lib/ui.ex +++ b/lib/ui.ex @@ -37,6 +37,10 @@ defmodule Bootleg.UI do puts(:info, output, setting) end + def error(output, setting \\ nil) do + puts(:error, output, setting) + end + @doc """ Get configured output verbosity and sanitize it for our uses. Defaults to :info @@ -53,10 +57,14 @@ defmodule Bootleg.UI do defp verbosity_includes(setting, level) defp verbosity_includes(:info, :info), do: true defp verbosity_includes(:info, :warning), do: true + defp verbosity_includes(:info, :error), do: true defp verbosity_includes(:warning, :warning), do: true + defp verbosity_includes(:warning, :error), do: true + defp verbosity_includes(:error, :error), do: true defp verbosity_includes(:debug, :info), do: true defp verbosity_includes(:debug, :warning), do: true defp verbosity_includes(:debug, :debug), do: true + defp verbosity_includes(:debug, :error), do: true defp verbosity_includes(_, _), do: false ### SSH formatting functions diff --git a/mix.exs b/mix.exs index 0a35147..ce99a0c 100644 --- a/mix.exs +++ b/mix.exs @@ -1,7 +1,7 @@ defmodule Bootleg.Mixfile do use Mix.Project - @version "0.2.0" + @version "0.3.0" @source "https://github.com/labzero/bootleg" def project do diff --git a/test/bootleg/config_functional_test.exs b/test/bootleg/config_functional_test.exs index 673e057..d153629 100644 --- a/test/bootleg/config_functional_test.exs +++ b/test/bootleg/config_functional_test.exs @@ -9,11 +9,13 @@ defmodule Bootleg.ConfigFunctionalTest do build_hosts = tl(hosts) role :app, app_host.ip, port: app_host.port, user: app_host.user, - password: app_host.password, silently_accept_hosts: true, workspace: "workspace" + password: app_host.password, silently_accept_hosts: true, workspace: "workspace", foo: :bar - Enum.each(build_hosts, fn build_host -> + build_hosts + |> Enum.with_index + |> Enum.each(fn {build_host, index} -> role :build, build_host.ip, port: build_host.port, user: build_host.user, - password: build_host.password, silently_accept_hosts: true, workspace: "workspace" + password: build_host.password, silently_accept_hosts: true, workspace: "workspace", foo: index end) end @@ -120,7 +122,6 @@ defmodule Bootleg.ConfigFunctionalTest do end test "remote/2 fails remotely" do - # credo:disable-for-next-line Credo.Check.Consistency.MultiAliasImportRequireUse use Bootleg.Config task :remote_functional_negative_test do @@ -133,4 +134,104 @@ defmodule Bootleg.ConfigFunctionalTest do assert_raise SSHError, fn -> invoke :remote_functional_negative_test end end) end + + @tag boot: 3 + test "remote/3 filtering" do + capture_io(fn -> + use Bootleg.Config + + assert [{:ok, out_0, 0, _}] = remote :build, [foo: 0], "hostname" + assert [{:ok, out_1, 0, _}] = remote :build, [foo: 1], do: "hostname" + assert out_1 != out_0 + + assert [] = remote :build, [foo: :bar], "hostname" + assert [{:ok, out_all, 0, _}] = remote :all, [foo: :bar], "hostname" + assert out_1 != out_0 != out_all + + remote :all, [foo: :bar] do "hostname" end + end) + end + + @tag boot: 3 + test "upload/3" do + capture_io(fn -> + use Bootleg.Config + + task :upload_single_role_single_host do + path = Temp.open!("upload", &IO.write(&1, "upload_single_role")) + upload :app, path, "single_role" + end + + task :upload_single_role_multi_host do + path = Temp.open!("upload", &IO.write(&1, "upload_single_role_multi")) + upload :build, path, "single_role_multi" + end + + task :upload_multi_role do + path = Temp.open!("upload", &IO.write(&1, "upload_multi_role")) + upload [:app, :build], path, "multi_role" + end + + task :upload_all_role do + path = Temp.open!("upload", &IO.write(&1, "upload_all_role")) + upload :all, path, "all_role" + end + + task :upload_role_filtered do + path = Temp.open!("upload", &IO.write(&1, "upload_role_filtered")) + upload [:all, primary: true], path, "role_filtered" + end + + task :upload_directory do + path = Temp.mkdir!("upload") + File.write!(Path.join(path, "foo"), "some data") + File.write!(Path.join(path, "bar"), "more data") + File.mkdir!(Path.join(path, "some_dir")) + File.write!(Path.join([path, "some_dir", "war"]), "different data") + upload :app, path, "should_be_dir" + end + + task :upload_absolute do + path = Temp.open!("upload", &IO.write(&1, "absolute")) + upload :app, path, "/tmp/absolute" + end + + task :upload_preserve_name do + path = Temp.open!("upload", &IO.write(&1, "same name")) + upload :app, path, "." + remote :app, do: "grep '^same name$' #{Path.basename(path)}" + end + + invoke :upload_single_role_single_host + remote :app, do: "grep '^upload_single_role$' single_role" + + invoke :upload_single_role_multi_host + remote :build, do: "grep '^upload_single_role_multi$' single_role_multi" + + invoke :upload_multi_role + remote [:app, :build], do: "grep '^upload_multi_role$' multi_role" + + invoke :upload_all_role + remote :all, do: "grep '^upload_all_role$' all_role" + + invoke :upload_role_filtered + assert_raise SSHError, fn -> + # credo:disable-for-next-line Credo.Check.Consistency.MultiAliasImportRequireUse + use Bootleg.Config + remote :all, do: "grep '^upload_role_filtered$' role_filtered" + end + + invoke :upload_directory + remote :app, do: "[ -d should_be_dir ]" + remote :app, do: "[ -d should_be_dir/some_dir ]" + remote :app, do: "grep '^some data$' should_be_dir/foo" + remote :app, do: "grep '^more data$' should_be_dir/bar" + remote :app, do: "grep '^different data$' should_be_dir/some_dir/war" + + invoke :upload_absolute + remote :app, do: "grep '^absolute$' /tmp/absolute" + + invoke :upload_preserve_name + end) + end end diff --git a/test/bootleg/config_test.exs b/test/bootleg/config_test.exs index 9b9d713..b9f710f 100644 --- a/test/bootleg/config_test.exs +++ b/test/bootleg/config_test.exs @@ -25,6 +25,19 @@ defmodule Bootleg.ConfigTest do end end + # credit: https://gist.github.com/henrik/1054546364ac68da4102 + defmacro assert_compile_time_raise(expected_exception, fun) do + # At compile-time, the fun is in AST form and thus cannot raise. + # At run-time, we will evaluate this AST, and it may raise. + fun_quoted_at_runtime = Macro.escape(fun) + + quote do + assert_raise unquote(expected_exception), fn -> + Code.eval_quoted(unquote(fun_quoted_at_runtime)) + end + end + end + setup do %{ local_user: System.get_env("USER") @@ -97,6 +110,18 @@ defmodule Bootleg.ConfigTest do assert_next_received :next end + test "role/2,3 do not allow a name of :all" do + assert_compile_time_raise ArgumentError, fn -> + use Bootleg.Config + role :all, "build1.example.com" + end + + assert_compile_time_raise ArgumentError, fn -> + use Bootleg.Config + role :all, "build2.example.com", an_option: true + end + end + test "get_role/1", %{local_user: local_user} do use Bootleg.Config role :build, "build.labzero.com" @@ -290,10 +315,9 @@ defmodule Bootleg.ConfigTest do end test_with_mock "remote/2", SSH, [:passthrough], [ - init: fn(role) -> {role} end, + init: fn(role, _options, _filter) -> {role} end, run!: fn(_, _cmd) -> [:ok] end ] do - # credo:disable-for-next-line Credo.Check.Consistency.MultiAliasImportRequireUse use Bootleg.Config task :remote_test_1 do @@ -342,14 +366,14 @@ defmodule Bootleg.ConfigTest do invoke :remote_test_1 - assert called SSH.init(:test_1) + assert called SSH.init(:test_1, :_, :_) assert called SSH.run!({:test_1}, "echo Hello World!") invoke :remote_test_2 - assert called SSH.init(:foo) + assert called SSH.init(:foo, :_, :_) assert called SSH.run!({:foo}, "echo Hello World2!") - assert called SSH.init(:bar) + assert called SSH.init(:bar, :_, :_) assert called SSH.run!({:bar}, "echo Hello World2!") invoke :remote_test_3 @@ -359,7 +383,7 @@ defmodule Bootleg.ConfigTest do invoke :remote_test_4 - assert called SSH.init(:test_4) + assert called SSH.init(:test_4, :_, :_) assert called SSH.run!({:test_4}, ["echo Hello", "echo World"]) with_mock Time, [], [utc_now: fn -> :now end] do @@ -377,9 +401,9 @@ defmodule Bootleg.ConfigTest do invoke :remote_test_all - assert called SSH.init(:foo) + assert called SSH.init(:foo, :_, :_) assert called SSH.run!({:foo}, "echo Hello World All!") - assert called SSH.init(:bar) + assert called SSH.init(:bar, :_, :_) assert called SSH.run!({:bar}, "echo Hello World All!") invoke :remote_test_all_multi @@ -391,16 +415,144 @@ defmodule Bootleg.ConfigTest do invoke :remote_test_roles - refute called SSH.init(:car) - assert called SSH.init(:foo) + refute called SSH.init(:car, :_, :_) + assert called SSH.init(:foo, :_, :_) assert called SSH.run!({:foo}, "echo Hello World Multi!") - assert called SSH.init(:bar) + assert called SSH.init(:bar, :_, :_) assert called SSH.run!({:bar}, "echo Hello World Multi!") invoke :remote_test_roles_multi - refute called SSH.init(:car) + refute called SSH.init(:car, :_, :_) assert called SSH.run!({:foo}, ["echo Multi Hello", "echo Multi World!"]) assert called SSH.run!({:bar}, ["echo Multi Hello", "echo Multi World!"]) end + + test_with_mock "remote/3 filtering", SSH, [:passthrough], [ + init: fn(role, options, filter) -> {role, options, filter} end, + run!: fn(_, _cmd) -> [:ok] end + ] do + use Bootleg.Config + + task :remote_test_role_one_line_filtered do + remote :one_line, [a_filter: true], "echo Multi Hello" + end + + task :remote_test_role_inline_filtered do + remote :inline, [b_filter: true], do: "echo Multi Hello" + end + + task :remote_test_role_filtered do + remote :car, passenger: true do + "echo Multi Hello" + end + end + + task :remote_test_roles_filtered do + remote [:foo, :bar], primary: true do + "echo Multi Hello" + end + end + + invoke :remote_test_role_one_line_filtered + + assert called SSH.init(:one_line, [], a_filter: true) + + invoke :remote_test_role_inline_filtered + + assert called SSH.init(:inline, [], b_filter: true) + + invoke :remote_test_role_filtered + + assert called SSH.init(:car, [], passenger: true) + + invoke :remote_test_roles_filtered + + assert called SSH.init(:foo, [], [primary: true]) + assert called SSH.init(:bar, [], [primary: true]) + end + + test_with_mock "upload/3", SSH, [:passthrough], [ + init: fn(role, options, filter) -> {role, options, filter} end, + upload: fn(_conn, _local, _remote) -> :ok end, + ] do + # credo:disable-for-next-line Credo.Check.Consistency.MultiAliasImportRequireUse + use Bootleg.Config + + role :foo, "never-used-foo.example.com" + role :car, "never-used-bar.example.com" + + task :upload_single_role do + upload :foo, "the/local/path", "some/remote/path" + end + + task :upload_multi_role do + upload [:foo, :bar], "the/local/path", "some/remote/path" + end + + task :upload_all_role do + upload :all, "the/local/path", "some/remote/path" + end + + task :upload_single_role_filter do + upload [:foo, primary: true], "the/local/path", "some/remote/path" + end + + task :upload_multi_role_filter do + upload [:foo, :bar, primary: true], "the/local/path", "some/remote/path" + end + + task :upload_multi_role_complex_filter do + upload [:foo, :bar, primary: true, db: :mysql], "the/local/path", "some/remote/path" + end + + task :upload_all_role_filter do + upload [:all, db: :mysql], "the/local/path", "some/remote/path" + end + + invoke :upload_single_role + + assert called SSH.init(:foo, [], []) + assert called SSH.upload({:foo, [], []}, "the/local/path", "some/remote/path") + + invoke :upload_multi_role + + assert called SSH.init(:foo, [], []) + assert called SSH.init(:bar, [], []) + assert called SSH.upload({:foo, [], []}, "the/local/path", "some/remote/path") + assert called SSH.upload({:bar, [], []}, "the/local/path", "some/remote/path") + + invoke :upload_all_role + + assert called SSH.init(:foo, [], []) + assert called SSH.init(:car, [], []) + assert called SSH.upload({:foo, [], []}, "the/local/path", "some/remote/path") + assert called SSH.upload({:car, [], []}, "the/local/path", "some/remote/path") + + invoke :upload_single_role_filter + + assert called SSH.init(:foo, [], [primary: true]) + assert called SSH.upload({:foo, [], [primary: true]}, "the/local/path", "some/remote/path") + + invoke :upload_multi_role_filter + + assert called SSH.init(:foo, [], [primary: true]) + assert called SSH.init(:bar, [], [primary: true]) + assert called SSH.upload({:foo, [], [primary: true]}, "the/local/path", "some/remote/path") + assert called SSH.upload({:bar, [], [primary: true]}, "the/local/path", "some/remote/path") + + invoke :upload_multi_role_complex_filter + + assert called SSH.init(:foo, [], [primary: true, db: :mysql]) + assert called SSH.init(:bar, [], [primary: true, db: :mysql]) + assert called SSH.upload({:foo, [], [primary: true, db: :mysql]}, "the/local/path", "some/remote/path") + assert called SSH.upload({:bar, [], [primary: true, db: :mysql]}, "the/local/path", "some/remote/path") + + invoke :upload_all_role_filter + + assert called SSH.init(:foo, [], [db: :mysql]) + assert called SSH.init(:car, [], [db: :mysql]) + assert called SSH.upload({:foo, [], [db: :mysql]}, "the/local/path", "some/remote/path") + assert called SSH.upload({:car, [], [db: :mysql]}, "the/local/path", "some/remote/path") + end end diff --git a/test/bootleg/ssh_functional_test.exs b/test/bootleg/ssh_functional_test.exs index 1ef546e..ec79061 100644 --- a/test/bootleg/ssh_functional_test.exs +++ b/test/bootleg/ssh_functional_test.exs @@ -68,7 +68,6 @@ defmodule Bootleg.SSHFunctionalTest do end test "init/2 with Role name atom and identity", %{hosts: [host]} do - # credo:disable-for-next-line Credo.Check.Consistency.MultiAliasImportRequireUse use Bootleg.Config ip = host.ip role :build, ip, port: host.port, user: host.user, @@ -84,6 +83,23 @@ defmodule Bootleg.SSHFunctionalTest do end) end + @tag boot: 2 + test "init/3 host filtering for roles", %{hosts: [host_1, host_2]} do + # credo:disable-for-next-line Credo.Check.Consistency.MultiAliasImportRequireUse + use Bootleg.Config + + ip_1 = host_1.ip + ip_2 = host_2.ip + role :build, ip_1, port: host_1.port, user: host_1.user, foo: :car, + workspace: "/", silently_accept_hosts: true, identity: host_1.private_key_path + role :build, ip_2, port: host_2.port, user: host_2.user, foo: :bar, + workspace: "/", silently_accept_hosts: true, identity: host_2.private_key_path + capture_io(fn -> + assert %SSHKitContext{hosts: [%SSHKitHost{name: ^ip_2}]} = SSH.init(:build, [], foo: :bar) + assert %SSHKitContext{hosts: [%SSHKitHost{name: ^ip_1}]} = SSH.init(:build, [], foo: :car) + end) + end + test "run!/2 raises an error if the host refuses the connection", %{hosts: [host]} do capture_io(fn -> conn = SSHKit.context(SSHKit.host(host.ip)) diff --git a/test/bootleg/ssh_test.exs b/test/bootleg/ssh_test.exs index 4bc1bdd..5f0bf2d 100644 --- a/test/bootleg/ssh_test.exs +++ b/test/bootleg/ssh_test.exs @@ -52,5 +52,6 @@ defmodule Bootleg.SSHTest do assert [[2, 4, 1, 2], [5, 6, 3, 4]] = SSH.merge_run_results([[1, 2], [3, 4]], [[2, 4], [5, 6]]) assert [1, 2] = SSH.merge_run_results([1, 2], []) assert [[1, 2]] = SSH.merge_run_results([[1, 2]], []) + assert [[1, 2]] = SSH.merge_run_results([], [[1, 2]]) end end diff --git a/test/bootleg/tasks/invoke_task_test.exs b/test/bootleg/tasks/invoke_task_test.exs new file mode 100644 index 0000000..f2ee089 --- /dev/null +++ b/test/bootleg/tasks/invoke_task_test.exs @@ -0,0 +1,33 @@ +defmodule Bootleg.Tasks.InvokeTaskTest do + use Bootleg.TestCase + alias Bootleg.Fixtures + + setup do + location = Fixtures.inflate_project(:bootstraps) + + location + |> List.wrap + |> Kernel.++(["config", "deploy.exs"]) + |> Path.join() + |> File.write(""" + use Bootleg.Config + task :hello, do: IO.puts "HELLO WORLD!" + """, [:write]) + + %{project_location: location} + end + + test "mix bootleg.invoke", %{project_location: location} do + shell_env = [{"BOOTLEG_PATH", File.cwd!}] + cmd_options = [env: shell_env, cd: location] + + assert {_, 0} = System.cmd("mix", ["deps.get"], cmd_options) + assert {out, 0} = System.cmd("mix", ["bootleg.invoke", "hello"], cmd_options) + assert String.match?(out, ~r/HELLO WORLD!/) + + assert {_, 0} = System.cmd("mix", ["bootleg.invoke", "unknown"], cmd_options) + + assert {out, 1} = System.cmd("mix", ["bootleg.invoke"], cmd_options) + assert String.match?(out, ~r/You must supply a task identifier as the first argument\./) + end +end diff --git a/test/bootleg_functional_test.exs b/test/bootleg_functional_test.exs index 670b596..332aab9 100644 --- a/test/bootleg_functional_test.exs +++ b/test/bootleg_functional_test.exs @@ -60,6 +60,32 @@ defmodule Bootleg.FunctionalTest do invoke :update end), ~r/build_me started/) + + capture_io(fn -> + # credo:disable-for-next-line Credo.Check.Consistency.MultiAliasImportRequireUse + use Bootleg.Config + + remote :app do + "wait-for-app build_me" + end + + [{:ok, [stdout: pid_1], 0, _}, {:ok, [stdout: pid_2], 0, _}] = remote :app do + "bin/build_me pid" + end + + invoke :update + + remote :app do + "wait-for-app build_me" + end + + [{:ok, [stdout: new_pid_1], 0, _}, {:ok, [stdout: new_pid_2], 0, _}] = remote :app do + "bin/build_me pid" + end + + assert pid_1 != new_pid_1 + assert pid_2 != new_pid_2 + end) end) end diff --git a/test/strategies/build/distillery_test.exs b/test/strategies/build/distillery_test.exs index 88c835c..a2880de 100644 --- a/test/strategies/build/distillery_test.exs +++ b/test/strategies/build/distillery_test.exs @@ -16,7 +16,9 @@ defmodule Bootleg.Strategies.Build.DistilleryTest do with_mocks([ { SSH, [:passthrough], [ - init: fn _ -> %SSHKit.Context{} end, + init: fn role -> SSH.init(role, []) end, + init: fn role, _, _ -> SSH.init(role, []) end, + init: fn _, _ -> %SSHKit.Context{} end, run!: fn _, _ -> [{:ok, [stdout: ""], 0, ssh_host}] end, ssh_host_options: fn _ -> ssh_host end, download: fn _, _, _ -> :ok end diff --git a/test/support/docker/fixtures/bin/launch-app b/test/support/docker/fixtures/bin/launch-app index 10500f4..d04b8bd 100644 --- a/test/support/docker/fixtures/bin/launch-app +++ b/test/support/docker/fixtures/bin/launch-app @@ -3,11 +3,4 @@ set -e bin/$1 start -TRIES=0 -while [[ "`bin/$1 ping`" != "pong" ]]; do - sleep 0.5 - TRIES=$(($TRIES + 1)) - if [[ $TRIES -gt 9 ]]; then - exit 1 - fi -done +wait-for-app $1 diff --git a/test/support/docker/fixtures/bin/wait-for-app b/test/support/docker/fixtures/bin/wait-for-app new file mode 100644 index 0000000..4bcee16 --- /dev/null +++ b/test/support/docker/fixtures/bin/wait-for-app @@ -0,0 +1,12 @@ +#!/bin/sh + +set -e + +TRIES=0 +while [[ "`bin/$1 ping`" != "pong" ]]; do + sleep 0.5 + TRIES=$(($TRIES + 1)) + if [[ $TRIES -gt 9 ]]; then + exit 1 + fi +done diff --git a/test/ui_test.exs b/test/ui_test.exs index ae3e7d6..831270d 100644 --- a/test/ui_test.exs +++ b/test/ui_test.exs @@ -26,16 +26,25 @@ defmodule Bootleg.UITest do assert :ok == UI.puts(:info, "", :info) assert :ok == UI.puts(:warning, "", :info) assert nil == UI.puts(:debug, "", :info) + assert :ok == UI.puts(:error, "", :info) # :warning is set now assert nil == UI.puts(:info, "", :warning) assert :ok == UI.puts(:warning, "", :warning) assert nil == UI.puts(:debug, "", :warning) + assert :ok == UI.puts(:error, "", :warning) + + # :error is set now + assert nil == UI.puts(:info, "", :error) + assert nil == UI.puts(:warning, "", :error) + assert nil == UI.puts(:debug, "", :error) + assert :ok == UI.puts(:error, "", :error) # :debug is set now and should unrestrict output assert :ok == UI.puts(:info, "", :debug) assert :ok == UI.puts(:warning, "", :debug) assert :ok == UI.puts(:debug, "", :debug) + assert :ok == UI.puts(:error, "", :debug) end test "verbosity is validated and defaults to :info" do @@ -54,6 +63,10 @@ defmodule Bootleg.UITest do assert capture_io(fn -> UI.debug("baz", :debug) end) == "baz\n" + + assert capture_io(fn -> + UI.error("caz", :error) + end) == "caz\n" end # SSH-specific output tests