v0.5.0

This is the release of Theodolite 0.5.0. With this release we entirely revised Theodolite's architecture and re-implemented it from scratch. This release is a huge step toward the first stable version of Theodolite. Features of this release include:

• Entire rewrite of Theodolite in Kotlin and Quarkus for better interaction with the Kubernetes API
• Optional execution as Kubernetes Operator for better usability (recommended)
• A data model for defining benchmarks (as CRD in operator mode) to support arbitrary benchmarks and SUTs apart from the Theodolite benchmarks. This includes:
  • Description of SUTs and LoadGenerators as sets of Kubernetes resources
  • Support for arbitrary load and resources types, according to our scalability metric
  • Flexible configuration of Kafka topics
  • Benchmark CRs for the Theodolite Kafka Streams and Flink benchmarks are provided
• A data model for describing benchmark executions (again as CRD in operator mode). This includes:
  • Support for repeating SLO experiments, from which the median lag trend value is then chosen
  • Extension mechanism for custom SLO checkers
  • Support for arbitrary configuration changes of the SUT or load generator to benchmark different SUT configurations etc.
• A Theodolite Helm chart (recommended way for installation)
• Optional usage of a random scheduler to mitigate scheduling effects on the benchmark results
• Full support for implementations of the Theodolite benchmarks with Flink (partly already releases with v.0.4.0)

We thank @benediktwetzel, @lorenzboguhn, @sehrenstein, @bvonheid and @NicoBiernat for their massive support in designing, implementing, and testing Theodolite v0.5.0.