Staging infrastructure

[skrobul]

Following our yesterday's chat with @Kooshaba about setting up a staging server, I have started exploring a few options. At this point, it is unclear what type of hosting will be available so thought I'll kick off some more discussion with this PR draft. The approach of setting this up will be different depending on the anticipated initial scale.

The way I see it, most of the options fall into one of these three categories:

1. Use cheap VPS or cloud instance(s) that is dedicated to the backend. I did some initial work on provisioning infrastructure for this with Saltstack and result of that is available in this PR. Those salt states are almost ready to deploy Postgres, Mosquitto, Rails and related dependencies on clean Debian VM. The code itself would be deployed with Capistrano.
   Pros of this approach: repeatable deployment on relatively cheap hardware
   Cons: potentially difficult to scale later

2. Docker-based solution 1 (Kubernetes). Since we already have a dockerized development environment, we may want to host staging (and potentially production later) with docker too. That dockerized environment should be managed by some sort of orchestration tool. Kubernetes is considered defacto standard these days, but I am not sure if it's right choice. I wrote kubernetes manifests as proof of concept.
   Pros: staging and prod are much closer to the development environment. Considerably easier to scale later.
   Cons: Kubernetes adds additional complexity. It's also going to be more expensive. We can control the costs by setting up a single node cluster

3. Docker-based solution 2: Same as above, but replace Kubernetes with Swarm
   Pros: same benefits as Kubernetes, but much much easier to setup and use. Can be set up as single node cluster too.
   Cons: future of Swarm is unclear for now as the Docker has been acquired recently, but it's most likely just rumours.

any thoughts on that?

Related: #12

[tabossert]

I have quite a bit of kubernetes experience and using options like EKS, AKS, GKE can be relatively cheap depending on worker node sizes. Happy to help evaluate options.