implement switching proxy / caps without destroying the host in Forem…

…an (#227)

Changes in README.md to explain, and lots of code, obviously. Quite
a few changes were required in functions that other parts of the code
use as well, like get_bootstrap_rpm and put_json.

README.md

@@ -7,6 +7,9 @@ bootstrap Script for migrating existing running systems to Foreman with the Kate
 This script can take a system that is registered to Spacewalk, Satellite 5, Red Hat
 Network Classic and get it registered to Foreman & Katello.
 
+Optionally, you can also move systems between Capsules (both internal and
+external) of one Katello installation by using the `--new-capsule` option.
+
 # What does the Script do?
 
 * Identify which systems management platform is the system registered to (Classic/Sat5 or None)  then perform the following
@@ -21,6 +24,18 @@ Network Classic and get it registered to Foreman & Katello.
 * Configuring the system with a proper Puppet configuration pointing at Foreman
 * Removing/disabling old RHN Classic packages/daemons (rhnsd, osad, etc)
 
+## System already registered to a Foreman + Katello server / Capsule (--new-capsule)
+* Clean the existing Katello agent installation
+* Install the Katello consumer RPM for the target Foreman + Katello server / Capsule
+* Install the Katello agent software again, using the configuration for the
+  target Foreman + Katello / Capsule server
+* Make API calls to switch the system to a different hostgroup (optional)
+* Make API calls to update the Puppet master, Puppet CA, content source and
+  OpenSCAP proxy IDs (optional, except for content source)
+* Re-enable rhsmcertd
+* Update the Puppet configuration for the system to point to the right capsule (optional)
+* Restart Puppet and call for the user to go sign the CSR
+
 ## System not registered to any Red Hat Systems Management Platform:
 
 * Make an API call to Foreman to create the Foreman Host associated with the user specified Org/Location
@@ -169,6 +184,25 @@ By default, bootstrap.py does not delete the system's profile from the legacy pl
 
 There are times where it is necessary to migrate clients from one Foreman + Katello installation to another. For instance, in lieu of upgrading an older Foreman + Katello installation, you choose to build a new installation in parallel. bootstrap.py can then be used to migrate clients from one Foreman + Katello installation to another. Simply provide the `--force` option, and bootstrap.py will remove the previous `katello-ca-consumer-*` package (from the old system), and will install the `katello-ca-consumer-*` package (from the new system), and continue registration as usual.
 
+### Migrating a system from one Foreman + Katello installation 6 or Capsule / to another in the same infrastructure
+
+In order to manually balance the load over multiple Capsule servers, you might
+want to move some existing systems to newly deployed Capsules. You can easily
+do this by running the bootstrap.py script like the examples below. Mind that
+you still have to manually revoke any Puppet certificates on the old capsules!
+
+~~~
+# ./bootstrap.py -l admin --new-capsule --server capsule.example.com
+~~~
+
+If you want to change the hostgroup and location of the system at the same
+time, run:
+
+~~~
+# ./bootstrap.py -l admin --new-capsule --server capsule.example.com \
+    --hostgroup mygroup --location mylocation
+~~~
+
 ### Enabling additional repositories at registration time.
 
 It is recommended to set which repositories that you want enabled on your activation keys via the UI or via `hammer activation-key product-content`. However, older versions of `subscription-manager` (versions < 1.10) do not support product content overrides. The `--enablerepos` switch accepts a comma separated lists of repositories that are passed to `subscription-manager` that will be enabled at registration time.

bootstrap.py

@@ -256,19 +256,22 @@ def is_fips():
     return fips_status == "1"
 
 
-def get_bootstrap_rpm():
+def get_bootstrap_rpm(clean=False, unreg=True):
     """
     Retrieve Client CA Certificate RPMs from the Satellite 6 server.
     Uses --insecure options to curl(1) if instructed to download via HTTPS
-    If called with --force, calls clean_katello_agent().
+    This function is usually called with clean=options.force, which ensures
+    clean_katello_agent() is called if --force is specified. You can optionally
+    pass unreg=False to bypass unregistering a system (e.g. when moving between
+    capsules.
     """
-    if options.force:
+    if clean:
         clean_katello_agent()
     if os.path.exists('/etc/rhsm/ca/katello-server-ca.pem'):
         print_generic("A Katello CA certificate is already installed. Assuming system is registered")
         print_generic("To override this behavior, run the script with the --force option. Exiting.")
         sys.exit(1)
-    if os.path.exists('/etc/pki/consumer/cert.pem'):
+    if os.path.exists('/etc/pki/consumer/cert.pem') and unreg:
         print_generic('System appears to be registered via another entitlement server. Attempting unregister')
         unregister_system()
     if options.download_method == "https":
@@ -308,6 +311,13 @@ def enable_rhsmcertd():
     exec_service("rhsmcertd", "restart")
 
 
+def is_registered():
+    # Check if all required certificates are in place (i.e. a system is
+    # registered to begin with) before we start changing things
+    return (os.path.exists('/etc/rhsm/ca/katello-server-ca.pem') and
+            os.path.exists('/etc/pki/consumer/cert.pem'))
+
+
 def migrate_systems(org_name, activationkey):
     """
     Call `rhn-migrate-classic-to-rhsm` to migrate the machine from Satellite
@@ -465,6 +475,13 @@ def install_puppet_agent():
         puppet_conf.write("""noop            = true
 """)
     puppet_conf.close()
+    noop_puppet_signing_run()
+    if 'puppet-enable' not in options.skip:
+        enable_service("puppet")
+        exec_service("puppet", "restart")
+
+
+def noop_puppet_signing_run():
     print_generic("Running Puppet in noop mode to generate SSL certs")
     print_generic("Visit the UI and approve this certificate via Infrastructure->Capsules")
     print_generic("if auto-signing is disabled")
@@ -591,9 +608,30 @@ def delete_json(url):
     return call_api(url, method='DELETE')
 
 
-def put_json(url):
+def put_json(url, jdata=None):
     """Use `call_api` to place a "PUT" REST API call."""
-    return call_api(url, method='PUT')
+    return call_api(url, data=jdata, method='PUT')
+
+
+def update_host_capsule_mapping(attribute, capsule_id, host_id):
+    url = "https://" + options.foreman_fqdn + ":" + str(API_PORT) + "/api/v2/hosts/" + str(host_id)
+    if attribute == 'content_source_id':
+        jdata = json.loads('{"host": {"content_facet_attributes": {"content_source_id": "%s"}, "content_source_id": "%s"}}' % (capsule_id, capsule_id))
+    else:
+        jdata = json.loads('{"host": {"%s": "%s"}}' % (attribute, capsule_id))
+    return put_json(url, jdata)
+
+
+def get_capsule_features(capsule_id):
+    url = "https://" + options.foreman_fqdn + ":" + str(API_PORT) + "/katello/api/capsules/%s" % str(capsule_id)
+    return [f['name'] for f in get_json(url)['features']]
+
+
+def update_host_config(attribute, value, host_id):
+    attribute_id = return_matching_foreman_key(attribute + 's', 'title="%s"' % value, 'id', False)
+    json_key = attribute + "_id"
+    jdata = json.loads('{"host": {"%s": "%s"}}' % (json_key, attribute_id))
+    put_json("https://" + options.foreman_fqdn + ":" + API_PORT + "/api/hosts/%s" % host_id, jdata)
 
 
 def return_matching_foreman_key(api_name, search_key, return_key, null_result_ok=False):
@@ -948,6 +986,7 @@ def exec_service(service, command, failonerror=True):
     parser.add_option("--deps-repository-url", dest="deps_repository_url", help="URL to a repository that contains the subscription-manager RPMs")
     parser.add_option("--deps-repository-gpg-key", dest="deps_repository_gpg_key", help="GPG Key to the repository that contains the subscription-manager RPMs", default="file:///etc/pki/rpm-gpg/RPM-GPG-KEY-redhat-release")
     parser.add_option("--install-packages", dest="install_packages", help="List of packages to be additionally installed - comma separated", metavar="installpackages")
+    parser.add_option("--new-capsule", dest="new_capsule", action="store_true", help="Switch the server to a new capsule for content and Puppet. Pass --server with the Capsule FQDN as well.")
     parser.add_option("-t", "--timeout", dest="timeout", type="int", help="Timeout (in seconds) for API calls and subscription-manager registration. Defaults to %default", metavar="timeout", default=900)
     (options, args) = parser.parse_args()
 
@@ -972,9 +1011,12 @@ def exec_service(service, command, failonerror=True):
     #     if removing from foreman:
     #        foreman_fqdn
     if not ((options.remove and ('foreman' in options.skip or options.foreman_fqdn)) or
-            (options.foreman_fqdn and options.org and options.activationkey and ('foreman' in options.skip or options.hostgroup))):
-        if not options.remove:
+            (options.foreman_fqdn and options.org and options.activationkey and ('foreman' in options.skip or options.hostgroup)) or
+            (options.foreman_fqdn and options.new_capsule)):
+        if not options.remove and not options.new_capsule:
             print "Must specify server, login, organization, hostgroup and activation key.  See usage:"
+        elif options.new_capsule:
+            print "Must use both --new-capsule and --server. See usage:"
         else:
             print "Must specify server.  See usage:"
         parser.print_help()
@@ -1083,8 +1125,8 @@ def exec_service(service, command, failonerror=True):
     # > Clean the environment from LD_... variables
     clean_environment()
 
-    # > IF RHEL 5 and not removing, prepare the migration.
-    if not options.remove and int(RELEASE[0]) == 5:
+    # > IF RHEL 5, not removing, and not moving to new capsule prepare the migration.
+    if not options.remove and int(RELEASE[0]) == 5 and not options.new_capsule:
         prepare_rhel5_migration()
 
     if options.remove:
@@ -1108,7 +1150,7 @@ def exec_service(service, command, failonerror=True):
         print_generic('This system is registered to RHN. Attempting to migrate via rhn-classic-migrate-to-rhsm')
         install_prereqs()
         check_migration_version()
-        get_bootstrap_rpm()
+        get_bootstrap_rpm(clean=options.force)
         generate_katello_facts()
         API_PORT = get_api_port()
         if 'foreman' not in options.skip:
@@ -1117,12 +1159,88 @@ def exec_service(service, command, failonerror=True):
         migrate_systems(options.org, options.activationkey)
         if options.enablerepos:
             enable_repos()
+    elif options.new_capsule:
+        # > ELIF new_capsule and foreman_fqdn set, will migrate to other capsule
+        #
+        # > will replace CA certificate, reinstall katello-agent, gofer
+        # > will optionally update hostgroup and location
+        # > wil update system definition to point to new capsule for content,
+        # > Puppet, OpenSCAP and update Puppet configuration (if applicable)
+        # > MANUAL SIGNING OF CSR OR MANUALLY CREATING AUTO-SIGN RULE STILL REQUIRED!
+        # > API doesn't have a public endpoint for creating auto-sign entries yet!
+        if not is_registered():
+            print_error("This system doesn't seem to be registered to a Capsule at this moment.")
+            sys.exit(1)
+
+        # Make system ready for switch, gather required data
+        install_prereqs()
+        get_bootstrap_rpm(clean=True, unreg=False)
+        install_katello_agent()
+        API_PORT = get_api_port()
+        capsule_id = return_matching_foreman_key('smart_proxies', 'name="%s"' % options.foreman_fqdn, 'id', False)
+        host_id = return_matching_foreman_key('hosts', 'name="%s"' % FQDN, 'id', False)
+        capsule_features = get_capsule_features(capsule_id)
+
+        # Optionally configure new hostgroup, location
+        if options.hostgroup:
+            print_running("Calling Foreman API to switch hostgroup for %s to %s" % (FQDN, options.hostgroup))
+            update_host_config('hostgroup', options.hostgroup, host_id)
+        if options.location:
+            print_running("Calling Foreman API to switch location for %s to %s" % (FQDN, options.location))
+            update_host_config('location', options.location, host_id)
+
+        # Configure new proxy_id for Puppet (if not skipped), and OpenSCAP (if available and not skipped)
+        if 'foreman' not in options.skip and 'puppet' not in options.skip:
+            print_running("Calling Foreman API to update Puppet master and Puppet CA for %s to %s" % (FQDN, options.foreman_fqdn))
+            update_host_capsule_mapping("puppet_proxy_id", capsule_id, host_id)
+            update_host_capsule_mapping("puppet_ca_proxy_id", capsule_id, host_id)
+        if 'foreman' not in options.skip and 'Openscap' in capsule_features:
+            print_running("Calling Foreman API to update OpenSCAP proxy for %s to %s" % (FQDN, options.foreman_fqdn))
+            update_host_capsule_mapping("openscap_proxy_id", capsule_id, host_id)
+        elif 'foreman' not in options.skip and 'Openscap' not in capsule_features:
+            print_warning("New capsule doesn't have OpenSCAP capability, not switching / configuring openscap_proxy_id")
+
+        print_running("Calling Foreman API to update content source for %s to %s" % (FQDN, options.foreman_fqdn))
+        update_host_capsule_mapping("content_source_id", capsule_id, host_id)
+
+        enable_rhsmcertd()
+
+        if 'puppet' not in options.skip and 'foreman' not in options.skip:
+            puppet_major_version = get_puppet_version()
+            if puppet_major_version == 3:
+                puppet_conf_file = '/etc/puppet/puppet.conf'
+                var_dir = '/var/lib/puppet'
+                ssl_dir = '/var/lib/puppet/ssl'
+            elif puppet_major_version in [4, 5]:
+                puppet_conf_file = '/etc/puppetlabs/puppet/puppet.conf'
+                var_dir = '/opt/puppetlabs/puppet/cache'
+                ssl_dir = '/etc/puppetlabs/puppet/ssl'
+            else:
+                print_error("Unsupported puppet version")
+                sys.exit(1)
+
+            print_running("Stopping the Puppet agent for configuration update")
+            exec_service("puppet", "stop")
+
+            # Not using clean_puppet() and install_puppet_agent() here, because
+            # that would nuke custom /etc/puppet/puppet.conf files, which might
+            # yield undesirable results.
+            print_running("Updating Puppet configuration")
+            exec_failexit("sed -i '/^[[:space:]]*server.*/ s/=.*/= %s/' %s" % (options.foreman_fqdn, puppet_conf_file))
+            exec_failok("sed -i '/^[[:space:]]*ca_server.*/ s/=.*/= %s/' %s" % (options.foreman_fqdn, puppet_conf_file))  # For RHEL5 stock puppet.conf
+            delete_directory(ssl_dir)
+            delete_file("%s/client_data/catalog/%s.json" % (var_dir, FQDN))
+
+            noop_puppet_signing_run()
+            print_generic("Puppet agent is not running; please start manually if required.")
+            print_generic("You also need to manually revoke the certificate on the old capsule.")
+
     else:
         # > ELSE get CA RPM, optionally create host,
         # >      register via subscription-manager
         print_generic('This system is not registered to RHN. Attempting to register via subscription-manager')
         install_prereqs()
-        get_bootstrap_rpm()
+        get_bootstrap_rpm(clean=options.force)
         generate_katello_facts()
         API_PORT = get_api_port()
         if 'foreman' not in options.skip:
@@ -1132,7 +1250,7 @@ def exec_service(service, command, failonerror=True):
         if options.enablerepos:
             enable_repos()
 
-    if not options.remove:
+    if not options.remove and not options.new_capsule:
         # > IF not removing, install Katello agent, optionally update host,
         # >                  optionally clean and install Puppet agent
         # >                  optionally remove legacy RHN packages