v2.8.4

Bug Fixes

• Manifests that contained values with percent signs were causing Genesis
  failures. This has been resolved.

v2.8.3

Improvements

• Command line can now take the path/env before the subcommand, without
  using -C.

  Command line can now take the enviornment before or after the command,
  and no longer needs the -C. Moving forward in 3.0, the preference will
  be prefix instead of postfix, and the -C option will be dropped.

  Unlike the postfix method, the prefix method can take a path component.
  Unlike the -C option, the .yml doesn't need to be included (unless it
  is ambiguous, such as your file is named the same as a genesis command).

  Before:

  genesis -C ../alt/path deploy my-env
  

  Now:

  genesis ../alt/path/my-env deploy
  

  Before:

  genesis -C deploy-dir/env.yml deploy
  

  Now:

  genesis deploy-dir/env deploy
  

• User is now notified if the local kit version has changed from what
  was deployed when running genesis info

Bug Fixes

• With the changes in 2.8.2, a quazi-regression was introduced regarding
  BOSH targeting. If the BOSH environment variables were set in an
  environment, Genesis would use those to target the BOSH director
  instead of what was specified in the environment file.

  The target BOSH and the manner in which it is targeted has been in
  flux for a number of years. Genesis 2.8.x was meant to unify this,
  but there are multiple edge cases that we've worked through, and a lot
  of habits that need to be unlearned. The following is the intended
  best practices, that if followed, will keep you out of trouble.

  • Whenever possible, use genesis bosh <env> <bosh opts and args> to
    interact with the BOSH deployment associated with . There
    should be very little technical reasons to not use this method.

    UNLEARN:

    Genesis is 'deployment-focused', while the BOSH cli is
    'director-focused'. When you target a Genesis environment file, you
    need to stop thinking about which director you are using, but which
    deployment you care about; Genesis will figure out the associated
    BOSH director to talk to. One of the reasons for this paradigm
    change is so users that have access to a deployment's env file, but
    not the corresponding bosh deployment's repo can still access the
    BOSH director needed to manage their deployment.

  • If you need to access a Genesis deployment AS a BOSH director, use
    the -A option. This will tell Genesis that the target environment
    contains the connection details for the BOSH director, not its
    parent. It will also not set the -d|--deployment option.

    Example: genesis bosh -A <env> <bosh opts and args>

  • If you need to use the BOSH cli directly, you have two choices, each
    somes with side effects:

    • Set the environment variables to connect to bosh. This is what
      Genesis does internally, but if you do this, be aware that it
      lasts for the lifetime of the shell you are in, which can lead to
      unexpected side-effects. You can do this by running

      eval "$(genesis bosh --connnect [-A] <env>)"

      where the optional -A specifies use as the director instead
      of the deployment.

      You can also use the bosh-genesis-kit's printenv addon, if you
      have access to that deployment environment file. While it is more
      powerful, including support for socks5 proxy, it also impacts the
      shell more. See the bosh kit for more details.

    • Use BOSH alias configuration and log in (daily) using the bosh
      deployment configuration. This is the method Genesis used prior
      to v2.8.0, and still works if you're using bosh-genesis-kit v2.2.0
      or greater, and bosh-cli 6.4.4 or greater. This has the advantage
      of needing to be explicitly stated with the -e option on the bosh
      call, and allowing multiple bosh directors to be authenticated
      against at the same time. The down side is you'll need to
      re-authenticate periodically.

  NOTE: If you have the BOSH environment variables set, Genesis will
  now ignore them when operating on a genesis deployment environment,
  but will issue a warning that it is doing so. The environment's
  explicitly or implicitly specified bosh environment will be used.

• Fixes outdated references to pre-2.8 BOSH library calls.

• Resolves runtime error message when not specifying kit on genesis init

• Resolves runtime error message for unspecified use_create_env property (2.8.0 feature)

• Correctly allows recreation of existing kits with --force option in compile-kit

• Fixes some internal kit hook bash wrappers around genesis callbacks

• Fixes race condition when creating a new bosh enviornment where it
  doesn't know the bosh director or if it is a proto bosh or not before
  it asks for those values.

• Reverts a change in v2.8.2, which would set the enviornment variable
  used internally and by the hooks to use the BOSH direcctor identified
  by BOSH_ENVIRONMENT. The normal environment variable to do this is
  GENESIS_BOSH_ENVIRONMENT, but the previous change erroneously also
  included BOSH_ENVIRONMENT as a source for setting the BOSH director.

• Fixed bug Can't locate object method "base_url" via package "Genesis::Kit::Provider::GenesisCommunity"
  that occurred when genesis repipe was run

v2.8.3-rc.6

This is a prerelease - please see commit messages for changes

v2.8.3-rc.5

This is a prerelease - please see commit messages for changes

v2.8.3-rc.4

This is a prerelease - please see commit messages for changes

v2.8.2

Bug Fixes

• Replaces error Can't call method "execute" on an undefined value
  with more useful information on why BOSH director cannot be accessed.

• Suppress repetitive deprecation messages for the environment when
  processing various genesis commands.

• The existing bosh wrapper function in Genesis::Helpers was not
  compatible with the Genesis v2.8 internals regarding BOSH connectivity.
  If the environment file exists, helpers will simply use genesis bosh ... calls to make it Just Work™. For the rare situation where the
  enviroment file doesn't exist, a much more complicated validation and
  connectivity routine is employed.

• When generating the BOSH director for an environment, and is using the
  environment variables to determine the BOSH director configuration, make
  sure to set the deployment name.

Kit Authoring Improvements

• Added version_check bash function to check that the version of Genesis
  meets or exceeds your hook's required version. If you use this, make sure
  your kit specifies genesis_version_minimum of 2.8.2.

• Improvements to genesis compile-kit interface and functionality:

  • Now creates a local rc release when no version is specified, for the
    next patch level above the highest remote or local version found.

  • Can specify --target|-t to have the compiled kit placed in the
    desired directory. If the directory is a genesis repo, it will
    place it under the directory's .genesis/kits path.

  • When using -v , if that version already exists locally or
    remotely, it will error out to prevent reissuing different contents
    under the same version (--force can be used to overcome this
    limitation).

  • When not specifying the explicit version, you can bump the major or
    minor version instead of the path by using --major|-M or --minor|-m
    options. The --final option will create the next version without a
    -rc# component.

  • Improved feedback while processing.

v2.8.1

Bug Fixes

• Genesis v2.8.0 reworked how Genesis interacted with BOSH, getting the
  login information from exodus data. However, when the exodus data is
  not available, it lost its ability to specify the BOSH connectivity
  via environment variables. This has now been resolved.

• Fixes error Can't locate object method "from_envvars" via package "Genesis::BOSH::Director" when generating new genesis environment.

• Fixes error when running genesis bosh -h, documents how to get help
  for bosh commands instead of help for genesis bosh

v2.8.0

New Features

• Auto-connect to BOSH director.

  Genesis can now automatically connect to the correct BOSH director for the
  specified environment, without need to set up a bosh configuration or
  manually logging into the director.

  Caveat: The BOSH director that deployed your environment MUST have been
  deployed by Genesis. If not, you will still have to target and log into the
  BOSH director manually.

• Added genesis bosh command.

  Instead of worrying about logging into the right BOSH director, and
  translating genesis environment into BOSH environment and deployment, let
  Genesis do the heavy lifting.

  If you want to know something about your concourse vms, for example, you
  just run genesis bosh my-concourse-env vms --vitals and you'll get the
  right output. Genesis will determine which bosh director deployed
  my-concourse-env, how to connect to it, what the BOSH deployment is named,
  and return the results of the command you specified.

  By default, the genesis bosh command is deployment centric, but sometimes
  you need to talk to the BOSH director itself. In that case, you can specify
  the BOSH deployment itself, and use the -as-director (or -A) option to
  indicate that the environment is the director, not a deployment on a
  different director. For proto-BOSH environments, this is required.

  See genesis bosh -h for more usage details.

• Connect to BOSH director without using the BOSH deployment kit.

  If you still need to connect to a BOSH director directly, you can configure
  your shell by using eval "$(genesis bosh my-target-env --connect)".
  Please note that this connects to the BOSH director that deployed the
  my-target-env. If my-target-env is a bosh deployment, the command will
  authenticate to the BOSH director that deployed it, not itself. In this
  case, you need to use the --as-director option to explicitly state that
  this is the BOSH environment you wish to connect to.

Breaking Changes

• Removes legacy support for kits without new or blueprint.

  As of v2.8.0, kits must use new and blueprint hooks. All modern kits
  support this model, so anyone actively keeping current with genesis would
  not be impacted.

• Remove subkits, error if used in env file.

  Subkits have been deprecated since at least 2.6.13, so its time to remove
  them.

• Remove --environment global option.

  The environment option has been replaced with --bosh-env option to remove
  confusion regarding the overloaded 'environment' name. Henceforth,
  environment refers to just the deployment environment. The -e shortform
  is still usable.

Improvements

Better Support for using BOSH create-env

• Starting in v2.8.0, kits that indicate that they are compatible with
  v2.8.0 can support use_create_env as their method of indicating
  their ability to use create-env instead of deploy for deployments.

  • Valid values are yes for this kit always uses create-env, no for
    kits that cannot use create-env, and allow for kits that permit
    either.

  • The kit will have to detect genesis.use_create_env in the
    environment to determine blueprint and other create-env kit
    behaviour. Genesis will manage the actual deployment and state file
    associated with it.

• Genesis properly handles both legacy features (create-env,bosh-init,
  proto) and v2.8.0's genesis.use_create_env attribute to determine is
  create-env is to be used. It will also detect when an incompatible
  mixture of using create-env and specifying a genesis.bosh-env
  attribute, and fail with an appropriate error message. A quick update
  of the environment file by the user will resolve this issue when
  encountered.

• Provide create-env (i.e.: bosh proto) deployments with diff/confirm

  Since BOSH does not provide a confirmation after a diff on the
  environment to be deployed, Genesis provides this based on a previous
  deployment, assuming it was added to the repo.

  If the exodus data from the previous deployment doesn't match the local
  archived manifest, a warning will be stated, or in the case of running
  under -y, the deploy will be aborted.

  This does require that you ensure you commit your repo after doing
  deployments, which you should be doing regardless of this feature, as
  a method to record your deployment history.

Kit Overrides by Environment

• Environments can now customize kit behaviour directly, by specifying a
  overrides parameter under the top-level kit attribute. This
  replaces the repo-wide kit-overrides.yml file (though it is still
  supported.

  This improves the previous behaviour because environments can target
  different versions of a kit, meaning the one-file-fits-all can be
  sub-optimal or even broken. It also means the overrides can propagate
  through the repo using hierarchical inheritance.

  It comes in three forms:

  • Simple string: By placing the overrides (in yml form) inside a
    string block, it is applied verbatim. Any spruce operators will be
    applied to the kits kit.yml only.

    kit:
      overrides: |
        docs: "https://internal-docs-for-kit.mycorp.com"
        genesis_version_min: 2.8.0
        use_create_env: yes
    

  • Simple YAML hashmap: By using a hash-map, you can use spruce
    operators to take values from the environment yaml hierarchy (though
    not the full manifest, as that would be a chicken/egg issue with the
    kit itself). If you want to use spruce operators against values in
    the kits kit.yml file, use the defer operator in front:

    kit:
      overrides:
        credentials:
          base:
            my-secret: (( grab params.my-secret )) # from env.yml
            old-secret: (( defer concat credentials.base.secret "-disabled" )) # from kit.yml
    

  • Array of the above types: By permitting an array, you can apply the
    changes in layers, or build from hierarchical ancestors

    kit:
      overrides:
      - |
        first_stuff: 1
    
      - second:
          stuff: also
    
    

    And in an inheritting file:

    kit:
      overrides:
      - (( append ))
      - second:
          stuff: overwritten
      - |
        even_more: true
    

Improved Kit Behaviour

• Improve params handling for use in new hook.

  • Support multiline values in keys and arrays in param_entry.

  • Echo param_comment to screen so the same information that prefaces the
    value in the env.yml file can be used as an explanatory blurb prior to the
    prompt.

• Improve decompile-kit

  • Can decompile kit to directories other than ./dev (--directory <dir>)

  • Can specify an environment YAML file to identify which kit to decompile,
    instead of explicitly stating the kit/version.

Better BOSH Config Management

• Support downloading multiple configs.

  Before, if you specified to download a config type (ie cloud or runtime)
  without a name, it would download only the unnamed default config of that
  type. Experience has shown that the more correct interpretation should be
  to download all configs of that type, and merge them (in the order they are
  specified in the BOSH director). The old behaviour can still be used by
  explicitly stating default as the name.

Messaging / Error Handling / Debugging

• Added stderr output capture for Genesis::Vault#get.

  Rather than have the raw safe stderr leak out to the user, we now capture it
  and write it to the DEBUG stream.

• Improve deploy help output for create-env environments

  There are difference behaviours for create-env environment than from
  environments deployed via a BOSH director. This update calls them out
  and gives instructions on their use.

• Improved error message when kit.features is incorrectly formed in environment YAML files.

• Better diagnosis messages when running PING in debug mode.

Bug Fixes

• Kits can now specify required prerequisites via hook.

  Although the prereqs hook was documented, it was only partially
  implemented. This commit completes that implementation, so that if
  hooks/prereqs exists in a kit, it will be run prior to usage.

• Fix bug where default config type was lost when fetching configs from BOSH
  director, resulting in bad messages and error reports.

• Export GENESIS_EXODUS_MOUNT to hooks environment. This was missing from the
  set of vault path environment variables available to hook scripts.

Internal Changes/Improvements

BOSH Refactor

• Add Genesis::BOSH for controlling BOSH director.

  This is a major refactor to facilitate better operation and connections
  to BOSH directors. Connections to Bosh will use exodus data to
  determine the corresponding BOSH director address and connection
  details.

• Set GENESIS_BOSH_COMMAND when checking BOSH cmd

  Also removes check_bosh_version subroutine that was replaced by
  Genesis::BOSH#command

• Change needs_bosh_create_env to use_create_env

Functionality Improvement

• Allow Genesis::Kit#metadata to take args.

  If Genesis::Kit#metadata receives no arguments, it will return a hash
  reference for all metadata for the kit. If it gets a single value, it
  returns the metadata corresponding to the key of that value. Finally,
  if it gets a list of values, it will return a list of the metadata
  corresponding to the keys of the given values.

Efficiencies

• Lazy-load kit provider types as needed.

  By using require instead of use for loading the
  Genesis::Kit::Provider::* classes, they are loaded as needed during
  runtime instead of fully loaded at compile-time.

• Memoize Genesis::Env params and actual_environment_files.

Structure / Maintainability

• Reordered lib/Genesis/Env.pm, with folding and grouping.

  This is done to...

Genesis v2.8.0-rc4

New Features

• Auto-connect to BOSH director.

  Genesis can now automatically connect to the correct BOSH director for the
  specified environment, without need to set up a bosh configuration or
  manually logging into the director.

  Caveat: The BOSH director that deployed your environment MUST have been
  deployed by Genesis. If not, you will still have to target and log into the
  BOSH director manually.

• Added genesis bosh command.

  Instead of worrying about logging into the right BOSH director, and
  translating genesis environment into BOSH environment and deployment, let
  Genesis do the heavy lifting.

  If you want to know something about your concourse vms, for example, you
  just run genesis bosh my-concourse-env vms --vitals and you'll get the
  right output. Genesis will determine which bosh director deployed
  my-concourse-env, how to connect to it, what the BOSH deployment is named,
  and return the results of the command you specified.

  By default, the genesis bosh command is deployment centric, but sometimes
  you need to talk to the BOSH director itself. In that case, you can specify
  the BOSH deployment itself, and use the -as-director (or -A) option to
  indicate that the environment is the director, not a deployment on a
  different director. For proto-BOSH environments, this is required.

  See genesis bosh -h for more usage details.

• Connect to BOSH director without using the BOSH deployment kit.

  If you still need to connect to a BOSH director directly, you can configure
  your shell by using eval "$(genesis bosh my-target-env --connect)".
  Please note that this connects to the BOSH director that deployed the
  my-target-env. If my-target-env is a bosh deployment, the command will
  authenticate to the BOSH director that deployed it, not itself. In this
  case, you need to use the --as-director option to explicitly state that
  this is the BOSH environment you wish to connect to.

Breaking Changes

• Removes legacy support for kits without new or blueprint.

  As of v2.8.0, kits must use new and blueprint hooks. All modern kits
  support this model, so anyone actively keeping current with genesis would
  not be impacted.

• Remove subkits, error if used in env file.

  Subkits have been deprecated since at least 2.6.13, so its time to remove
  them.

• Remove --environment global option.

  The environment option has been replaced with --bosh-env option to remove
  confusion regarding the overloaded 'environment' name. Henceforth,
  environment refers to just the deployment environment. The -e shortform
  is still usable.

Improvements

Better Support for using BOSH create-env

• Starting in v2.8.0, kits that indicate that they are compatible with
  v2.8.0 can support use_create_env as their method of indicating
  their ability to use create-env instead of deploy for deployments.

  • Valid values are yes for this kit always uses create-env, no for
    kits that cannot use create-env, and allow for kits that permit
    either.

  • The kit will have to detect genesis.use_create_env in the
    environment to determine blueprint and other create-env kit
    behaviour. Genesis will manage the actual deployment and state file
    associated with it.

• Genesis properly handles both legacy features (create-env,bosh-init,
  proto) and v2.8.0's genesis.use_create_env attribute to determine is
  create-env is to be used. It will also detect when an incompatible
  mixture of using create-env and specifying a genesis.bosh-env
  attribute, and fail with an appropriate error message. A quick update
  of the environment file by the user will resolve this issue when
  encountered.

• Provide create-env (i.e.: bosh proto) deployments with diff/confirm

  Since BOSH does not provide a confirmation after a diff on the
  environment to be deployed, Genesis provides this based on a previous
  deployment, assuming it was added to the repo.

  If the exodus data from the previous deployment doesn't match the local
  archived manifest, a warning will be stated, or in the case of running
  under -y, the deploy will be aborted.

  This does require that you ensure you commit your repo after doing
  deployments, which you should be doing regardless of this feature, as
  a method to record your deployment history.

Kit Overrides by Environment

• Environments can now customize kit behaviour directly, by specifying a
  overrides parameter under the top-level kit attribute. This
  replaces the repo-wide kit-overrides.yml file (though it is still
  supported.

  This improves the previous behaviour because environments can target
  different versions of a kit, meaning the one-file-fits-all can be
  sub-optimal or even broken. It also means the overrides can propagate
  through the repo using hierarchical inheritance.

  It comes in three forms:

  • Simple string: By placing the overrides (in yml form) inside a
    string block, it is applied verbatim. Any spruce operators will be
    applied to the kits kit.yml only.

    kit:
      overrides: |
        docs: "https://internal-docs-for-kit.mycorp.com"
        genesis_version_min: 2.8.0
        use_create_env: yes
    

  • Simple YAML hashmap: By using a hash-map, you can use spruce
    operators to take values from the environment yaml hierarchy (though
    not the full manifest, as that would be a chicken/egg issue with the
    kit itself). If you want to use spruce operators against values in
    the kits kit.yml file, use the defer operator in front:

    kit:
      overrides:
        credentials:
          base:
            my-secret: (( grab params.my-secret )) # from env.yml
            old-secret: (( defer concat credentials.base.secret "-disabled" )) # from kit.yml
    

  • Array of the above types: By permitting an array, you can apply the
    changes in layers, or build from hierarchical ancestors

    kit:
      overrides:
      - |
        first_stuff: 1
    
      - second:
          stuff: also
    
    

    And in an inheritting file:

    kit:
      overrides:
      - (( append ))
      - second:
          stuff: overwritten
      - |
        even_more: true
    

Improved Kit Behaviour

• Improve params handling for use in new hook.

  • Support multiline values in keys and arrays in param_entry.

  • Echo param_comment to screen so the same information that prefaces the
    value in the env.yml file can be used as an explanatory blurb prior to the
    prompt.

• Improve decompile-kit

  • Can decompile kit to directories other than ./dev (--directory <dir>)

  • Can specify an environment YAML file to identify which kit to decompile,
    instead of explicitly stating the kit/version.

Better BOSH Config Management

• Support downloading multiple configs.

  Before, if you specified to download a config type (ie cloud or runtime)
  without a name, it would download only the unnamed default config of that
  type. Experience has shown that the more correct interpretation should be
  to download all configs of that type, and merge them (in the order they are
  specified in the BOSH director). The old behaviour can still be used by
  explicitly stating default as the name.

Messaging / Error Handling / Debugging

• Added stderr output capture for Genesis::Vault#get.

  Rather than have the raw safe stderr leak out to the user, we now capture it
  and write it to the DEBUG stream.

• Improve deploy help output for create-env environments

  There are difference behaviours for create-env environment than from
  environments deployed via a BOSH director. This update calls them out
  and gives instructions on their use.

Bug Fixes

• Kits can now specify required prerequisites via hook.

  Although the prereqs hook was documented, it was only partially
  implemented. This commit completes that implementation, so that if
  hooks/prereqs exists in a kit, it will be run prior to usage.

• Fix bug where default config type was lost when fetching configs from BOSH
  director, resulting in bad messages and error reports.

• Export GENESIS_EXODUS_MOUNT to hooks environment. This was missing from the
  set of vault path environment variables available to hook scripts.

Internal Changes/Improvements

BOSH Refactor

• Add Genesis::BOSH for controlling BOSH director.

  This is a major refactor to facilitate better operation and connections
  to BOSH directors. Connections to Bosh will use exodus data to
  determine the corresponding BOSH director address and connection
  details.

• Set GENESIS_BOSH_COMMAND when checking BOSH cmd

  Also removes check_bosh_version subroutine that was replaced by
  Genesis::BOSH#command

• Change needs_bosh_create_env to use_create_env

Functionality Improvement

• Allow Genesis::Kit#metadata to take args.

  If Genesis::Kit#metadata receives no arguments, it will return a hash
  reference for all metadata for the kit. If it gets a single value, it
  returns the metadata corresponding to the key of that value. Finally,
  if it gets a list of values, it will return a list of the metadata
  corresponding to the keys of the given values.

Efficiencies

• Lazy-load kit provider types as needed.

  By using require instead of use for loading the
  Genesis::Kit::Provider::* classes, they are loaded as needed during
  runtime instead of fully loaded at compile-time.

• Memoize Genesis::Env params and actual_environment_files.

Stru...

Genesis v2.7.35

Bug Fix

• Update Genesis bosh activities to work with BOSH cli v6.4.4

  As of BOSH cli 6.4.4, BOSH_* variables set to "" cause errors, so now they are simply unset.