This is a collection of Ansible scripts which help you to setup a MapR-cluster.
Maintainer: Carsten Hufe - Lead Data Engineer @ MapR
Changes, bugs and feature requests can be requested by using the Github issue tracker. If you want to contribute to the project, open pull requests via Github.
The scripts support:
- Install MapR-Cluster including Ecosystem Components (only one playbook to execute)
- Modify an existing installation
- Adding and removing components
- Adding and decomissioning nodes
- The script also support dry-run on an installed cluster
- Aborted scripts can always re-run.
- Building Zeppelin
- Install MapR-Client
- Setting up Kerberos and generate Keytabs (only Redhat)
- Ansible modules to manage state of:
- MapR volumes
- MapR schedules
- MapR Accountable Entities
- MapR 6.1.0 with MEP 6.1.0 (branch master)
- MapR 6.1.0 with MEP 6.0.0 (branch mapr-6.1.0-MEP-6.0.x)
- MapR 6.0.1 with MEP 5.0.0 (branch mapr-6.0.1-MEP-5.x)
- MapR 6.0.0 with MEP 4.0.x (branch mapr-6.0.0-MEP-4.x)
- MapR 5.2.x (branch mapr-5.2-MEP-3.0.x)
- Redhat 7 or higher
- CentOS 7 or higher
- Ubuntu 16.x
- Suse SLES 12 or higher
Please check MapR's OS compatibility matrix: https://mapr.com/docs/61/InteropMatrix/r_os_matrix_6.x.html
- Unsecure mode
- MapR Native Security incl. Ecosystem Components
- Kerberos incl. Ecosystem Components
- Oracle (Hive, Hue and Oozie)
- MySQL (Hive, Hue and Oozie)
- Embedded databases
Ansible 2.4 or higher required!
If /usr/bin/python is not available link it (e.g. in Ubuntu 16.xx)
ansible -i myhosts/hosts_3nodes -m raw -a "ln -s /usr/bin/python3 /usr/bin/python" all -v
Execute:
ansible -i myhosts/<yourfile> -m command -a "hostname -f" all
Can be found host_templates
Use host_templates/hosts_cluster as template and copy it and the hostnames to the components you want to get installed. If components are not required, just leave the block empty.
The configuration can be found in group_vars/all.
Then run:
ansible-playbook -i hosts_template site-cluster.yml
The site-cluster.yml script always ensures the cluster state. This means you can re-run this script, after you removed or added new components or nodes.
That an unprovisioned node is still known to Ansible you have to add the removed node to [unprovision] block in inventory file.
ATTENTION: Never move critical components in one run, e.g. you have two CLDB installed on node1 and node2 and you want move it to node3 and node4. In that case first install two new CLDBs and in the next run remove the other ones.
Use host_templates/hosts_client as template and copy it and the hostnames to the components you want to get installed. If components are not required, just leave the block empty. Then run:
ansible-playbook -i hosts_template site-client.yml
Ubuntu ONLY
Use hosts_template as template and copy it and the hostnames to the components you want to get installed. If components are not required, just leave the block empty. Then run:
ansible-playbook -i hosts_template sites/ext-rstudio-server.yml
Use hosts_template as template and copy it and the hostnames to the components you want to get installed. If components are not required, just leave the block empty. Then run:
ansible-playbook -i hosts_template sites/ext-zeppelin.yml
Helpers are located in the helper folder.
This document contains the ACE file/Volume demo To create the users and groups just run:
Run:
ansible-playbook -i hosts_template helper/create-user-ace.yml
Use host_templates/hosts_kerberos as template
ansible-playbook -i host_templates/hosts_kerberos sites/ext-kerberos-sssd-setup.yml
Afterwards authenticate with users from AD.
Based on number of hosts users are created in AD.
ansible-playbook -i host_templates/hosts_kerberos sites/ext-kerberos-createadusers.yml
Keytabs are stored in folder defined in mapr_kerberos_local_keytabs_dir on ansible client machine
ansible-playbook -i host_templates/hosts_kerberos sites/ext-kerberos-keytabs-ad-generate.yml
Keytabs are stored in folder defined in mapr_kerberos_local_keytabs_dir on ansible client machine. When the customer delivers keytabs this can be also used to validate.
ansible-playbook -i host_templates/hosts_kerberos sites/ext-kerberos-keytabs-verify.yml
The module always ensures the state. If a volume exists with these settings nothing will be changed. The module also supports check mode!
- name: Create MapR volume
mapr_volume:
name: my.new.volume
state: present
topology: /data
path: /test
read_ace: p
write_ace: p
min_replication: 2
replication: 3
soft_quota_in_mb: 1024
hard_quota_in_mb: 1024
accountable_entity_type: user
accountable_entity_name: mapr
read_only: no
The module always ensures the state. If a volume exists with these settings nothing will be changed. The module also supports check mode!
- name: Modify MapR schedule
mapr_schedule:
name: testrule
state: present
rules:
- frequency: daily
time: 0
retain: 7d
- frequency: weekly
date: sun
time: 0
retain: 4w
- frequency: monthly
date: "1"
time: 0
retain: 2m
The module always ensures the state. If a volume exists with these settings nothing will be changed. The module also supports check mode!
- name: Modify MapR entity
mapr_entity:
name: mapr
type: user
email: [email protected]
soft_quota_in_mb: 1024
hard_quota_in_mb: 1024
Copyright 2018 MapR Technologies, Inc.
Licensed under the Apache License, Version 2.0 (the "License")
http://www.apache.org/licenses/LICENSE-2.0
- When uprading, keys are not generated by ElasticSearch, but works with new installation (when upgrading set spyglass_elasticsearch_secure: false)