Setup for running Presto with Hive Metastore on Kubernetes as introduced in this blog post.
See previous blog post for more information about running Presto on FlashBlade.
-
Build Docker images for Hive Metastore and Presto
-
Deploy Hive Metastore: MariaDB (pvs and deployment), init-schemas, Metastore
-
Deploy Presto services (coordinator, workers, and cli)
-
Deploy Redash.
Assumptions: working Kubernetes deployment and S3 object store (e.g., FlashBlade).
Things you may need to modify:
- Docker repository name ($REPONAME) in build_image scripts and yaml files.
- DataVIP and access keys for FlashBlade (fs.s3a.endpoint and hive.s3.endpoint)
- StorageClass for the MariaDB volume.
- Memory settings and worker counts.
Dockerfile for Metastore
Yaml for MariaDB
- Simple and not optimized.
Yaml for init-schemas
- One-time K8s job to initiate the MariaDB tables.
Yaml for Metastore
Dockerfile for PrestoSql.
Script: autoconfig_and_launch.sh
- Generate final presto config files at pod startup time.
Yaml for Presto Coordinator/Workers
Dockerfile for Presto CLI
- Simple image to make interactive use of Presto easier.