I wonder how many organizations that say they're "doing DevOps" are actually building a bespoke PaaS. And how many of those realize it.
In general PaaSTA is difficult to compare directly to individual products because PaaSTA itself is an integration of many existing tools.
The problem space of software deployment is vast, covering lots of problems that must be solved in one way or another:
- Code packaging and distribution
- Resource scheduling
- Service Discovery
- Monitoring / alerting
- Workflow / orchestration
Some tools try to solve some subset of the above topics. This document is not meant to be a comparison between every tool out there, but instead is just designed to give a general overview about what makes PaaSTA different, compared to some of the more popular tools in the same space.
As a baseline for comparison, here are the pieces that PaaSTA uses to solve the particular problems associated with running a production PaaS:
| Problem | PaaSTA Solution |
|---|---|
| Code containerizer | Docker |
| Scheduling | Mesos + Marathon |
| Service Discovery | SmartStack |
| Monitoring | Sensu |
| Workflow | Jenkins or CLI + soa-configs |
Hopefully by looking at these particular sub-problems we can better compare the different technologies out there.
| Problem | Nomad/Hashicorp Solution |
|---|---|
| Code containerizer | Docker |
| Scheduling | Nomad |
| Service Discovery | Consul |
| Monitoring | Atlas? |
| Workflow | CLI |
Hashicorp Nomad is a resource manager and cluster manager. It integrates well with Hashicorp's other products like Consul for service discovery.
PaaSTA uses Mesos for resource management, Marathon for task scheduling, and Smartstack for service discovery. PaaSTA itself just glues the pieces together, and in theory could use Nomad for launching jobs as well, but it currently does not.
Hashicorp has its own page comparing the different scheduler idiosyncrasies of Mesos versus Nomad.
| Problem | ECS/Amazon Solution |
|---|---|
| Code containerizer | Docker |
| Scheduling | ECS |
| Service Discovery | ELBs + DNS |
| Monitoring | CloudWatch |
| Workflow | Console or CLI |
Amazon ECS is an Amazon-specific method of launching Docker containers on EC2 instances.
PaaSTA is designed to be infrastructure-agnostic, and can run in EC2 just as easily as it can run on bare metal.
PaaSTA is also much more opinionated about how to deploy Docker containers in a way that meets a specified job definition. ECS is an API and is designed to be used by higher-level tools, like Empire
| Problem | Kubernetes Solution |
|---|---|
| Code containerizer | Docker |
| Scheduling | Kubernetes core |
| Service Discovery | Env Vars or Kubernetes DNS Service |
| Monitoring | Kubernetes + Webhooks |
| Workflow | CLI / API |
Kubernetes is a cluster manager and resource manager. It does load balancing and service discovery. It also can do secret distribution and monitoring.
Kubernetes can be closely compared to PaaSTA because they do most of the same functions. The main difference is that PaaSTA uses existing open source technologies, instead of a single Go binary. The consolidated approach has the benefit of making it easier to setup and deploy, at the expense of being able to swap out components.
PaaSTA trades the convenience of a unified binary for the ability to use proven existing technologies. The downside to this approach is the cost of integration and deployment.
| Problem | Heroku Solution |
|---|---|
| Code containerizer | Dynos / cgroups (Docker beta) |
| Scheduling | Heroku |
| Service Discovery | DNS + Heroku Router |
| Monitoring | via Addons |
| Workflow | CLI or Dashboard |
Heroku is a full PaaS, and has a feature set more comparable to PaaSTA. PaaSTA uses Docker containers, which leaves it up to the service author to build their own software stack, compared to Heroku's provided stacks and buildpacks.
One of the main philosophical differences between Heroku and PaaSTA is that Heroku is "imperative", in the sense that in order to make changes in Heroku, one must invoke an action, either using their dashboard or CLI. PaaSTA is designed to be "declarative", and uses config files and Docker tags to be the source of truth for "what is supposed to be running where and how".
Another major difference between the architecture of Heroku and PaaSTA is the cluster design. In PaaSTA there are many different clusters that code can be deployed to. In Heroku there is only the main Heroku platform, and users are expected to run different apps. (test-appname, staging-appname, etc)
| Problem | Flynn Solution |
|---|---|
| Code containerizer | Flynn |
| Scheduling | Flynn |
| Service Discovery | DNS / API / Router |
| Monitoring | N/A |
| Workflow | CLI / Web / API |
Flynn is a Heroku-inspired PaaS. Flynn is unique in this comparison because it has first-class support for its embedded Postgres appliance, analogous to Heroku's Postgres Add-on or Amazon's RDS. This reduces the number of components required to run a fully working setup, assuming Postgres meets the developers' needs. Most other PaaS's view this problem as "out of scope", including PaaSTA.
Depending on your opinions on the Twelve-Factor App manifesto, Flynn, Heroku, or Empire may be good solutions for environments that have apps that already conform to the twelve-factor specification.
| Problem | Docker Swarm Solution |
|---|---|
| Code containerizer | Docker |
| Scheduling | Docker Swarm (or Mesos) |
| Service Discovery | Docker links or the Ambassador pattern |
| Monitoring | N/A |
| Workflow | Docker Compose or anything that uses the Docker API |
Docker Swarm is a relatively new platform that uses many existing Docker idioms to construct a cluster and define applications. Docker goes to great lengths to make Docker Swarm work like a normal Docker API, so existing tools and workflows can use it, including docker-compose.
Docker also attempts to solve the problems of persistent storage and log aggregation, again using existing Docker idioms like docker volumes and docker logs.
Compared to PaaSTA, the Docker Swarm solution requires far fewer components, making it much easier for operators to get started. PaaSTA is more opinionated with regards to "how" Docker containers are orchestrated. Docker Swarm simply provides the raw Docker API, which allows for a lot of flexibility in deploying sets of docker containers.