data-cat

a datadog spin-off using the ELK stack

Setup

1. Install Vagrant and VirtualBox for your OS.
2. If you are on macOS, you will probably have some permission issues with VirtualBox. Check this article if needed.
3. Run vagrant status to check current VMs. Their initial state should be not created.
4. Run vagrant up provisioner to spin-up the provisioner VM, which has ansible. This will take a while to finish.
5. Make sure that provisioner is running, and you are on the root directory of the project and run the following command:

   vagrant ssh provisioner -c "ssh-keygen -y -f /vagrant/.vagrant/machines/provisioner/virtualbox/private_key" > roles/ssh_config/files/id_rsa.pub

   The script will generate the public key from the provisioners private key, which is used by ansible.
6. Fire-up the rest of the nodes by running vagrant up once again. At this point, you can go grab a ☕️ and come back later.
7. Verify that all nodes are running via vagrant status. All VMs should have their state running.
8. You should now be able to ssh into provisioner (vagrant ssh provisioner).
9. cd /vagrant and run ansible-playbook -i inventory/dev setup_privisioner.yml
10. ansible-playbook -i inventory/dev provision_dev.yml

TODOs

• Provision infrastructure (VMs using Vagrant)
  • bootstrap VMs via Vagrant & shell scripts to install necessary tools and updates
  • 5 VMs (1 which has ansible installed, 4 will be used as nodes with 1 being a master node):
    • provisioner (has ansible installed and is responsible for managing all other VMs)
    • master (elastic master node)
    • data (elastic data node)
    • shipper (elastic shippers - logstash/beats)
    • kibana (the UI)
• Document how to set everything up.
• Provision software using a configuration management tool (ansible, elasticsearch, kibana)
  • create ansible playbooks and roles
• Create some basic micro-services that will create sample logs (nodejs)

Some cool IntelliJ tools

• Ruby IntelliJ plugin
• Vagrant IntelliJ plugin
• YAML/Ansible support for IntelliJ
• .ini file support
• OrchidE lang. support for ansible for IntelliJ (commercial, 30 day trial)