v0.5.2

This is the release of Theodolite 0.5.2. It fixes an unresolveable Helm dependency (seglo/kafka-lag-exporter#316).

v0.6.3

This is the release of Theodolite 0.6.3. It provides the following bug fix:

• Theodolite's Prometheus now watches all ServiceMonitors in the current namespace. This fixes the missing documentation of requiring specific labels to be set for ServiceMonitors that should be used by Theodolite.

v0.6.2

This is the release of Theodolite 0.6.2. With this release, we provides bug fixes as well as a preview of features coming with Theodolite v0.7. Specifically, these are:

• A benchmark no longer has to define a Kafka object. If no Kafka object is defined, Theodolite does not create any topics.
• Kubernetes files ending with .yml are now supported to be used by Theodolite. Previously, only .yaml files were supported.
• Logging of exceptions is improved.
• A bug in our Helm chart not allowing to enable the generic SLO checker while disabling the dropped records SLO is resolved.
• Immense resource usage of the SLO checker caused by a bug in uvicorn-gunicorn-fastapi-docker was fixed by creating custom images.
• Resource constraints for the Theodolite containers are set.

v0.6.1

This is the release of Theodolite 0.6.1. With this release, we provide a preview for the following features coming with Theodolite v0.7:

• Add the generic SLO, which can be declaratively configured with a PromQL query, aggregation functions, a threshold and a comparison operator.
• Create a expXYZ_demand.csv file directly from Theodolite such that the analysis notebooks have not necessarily be used to calculate the demand metric.

v0.6.0

This is the release of Theodolite 0.6.0. The most important new features are:

• Kubernetes resources describing the SUT or the workload generator don't longer need to be present inside the Theodolite container. Instead they can (and should) be put into ConfigMaps, from where Theodolite reads them
• In addition to SUT and load generator resources, benchmarks can now also define infrastructure components with run for an entire benchmark execution.
• Theodolite now generates Kubernetes events indicating the current state of an execution.
• Benchmarks now have a status indicating whether they are ready to be executed (all referring resources are accessible) or not.
• Theodolite's benchmarks are now automatically registered with Kubernetes when installing Theodolite. This can be disabled.
• A new SLO checker is available for Kafka Streams, which checks whether the amount of dropped records exceeds a certain threshold.
• Kubernetes volumes for benchmark results can be created from Helm.
• The Helm allows to define a nodeSelector for Theodolite.
• We updated some dependencies of our Helm chart.
• With this release our website and documentation is available via https://www.theodolite.rocks

Please note that the Theodolite CRDs have changed. When upgrading from a previous version, a simple helm upgrade will not be sufficient. Instead you should uninstall and re-install Theodolite.

v0.5.1

This release provides two fixes:

• minimal Helm installation does not require a PersiententVolumeClaim to be created before
• Support for Kubernetes v1.22

v0.5.0

This is the release of Theodolite 0.5.0. With this release we entirely revised Theodolite's architecture and re-implemented it from scratch. This release is a huge step toward the first stable version of Theodolite. Features of this release include:

• Entire rewrite of Theodolite in Kotlin and Quarkus for better interaction with the Kubernetes API
• Optional execution as Kubernetes Operator for better usability (recommended)
• A data model for defining benchmarks (as CRD in operator mode) to support arbitrary benchmarks and SUTs apart from the Theodolite benchmarks. This includes:
  • Description of SUTs and LoadGenerators as sets of Kubernetes resources
  • Support for arbitrary load and resources types, according to our scalability metric
  • Flexible configuration of Kafka topics
  • Benchmark CRs for the Theodolite Kafka Streams and Flink benchmarks are provided
• A data model for describing benchmark executions (again as CRD in operator mode). This includes:
  • Support for repeating SLO experiments, from which the median lag trend value is then chosen
  • Extension mechanism for custom SLO checkers
  • Support for arbitrary configuration changes of the SUT or load generator to benchmark different SUT configurations etc.
• A Theodolite Helm chart (recommended way for installation)
• Optional usage of a random scheduler to mitigate scheduling effects on the benchmark results
• Full support for implementations of the Theodolite benchmarks with Flink (partly already releases with v.0.4.0)

We thank @benediktwetzel, @lorenzboguhn, @sehrenstein, @bvonheid and @NicoBiernat for their massive support in designing, implementing, and testing Theodolite v0.5.0.

v0.4.0

This is the release of Theodolite 0.4.0. The most important features of this release are:

• Changed enumeration of benchmarks to sort them by ascending complexity and match our Theodolite publication DOI: 10.1016/j.bdr.2021.100209.
• Substantial rewrite of Theodolite's load generator such that it does no longer depend on a running Zookeeper cluster.
• Benchmark implementations for Apache Flink.
• From this release on, our Docker images are provided via GitHub Container Registry.

v0.3.0

This is the release of Theodolite 0.3.0. The most important features of this release are:

• Entire rewrite of Theodolite in Python instead of Bash
• More efficient execution using configurable search heuristics
• Support for execution as a Kubernetes Job
• Setting custom benchmark environment variables via Theodolite

Other new features are:

• Kubernetes namespace for benchmarks is now configurable
• The documentations have massively been improved
• Benchmark results can be stored in volumes
• Theodolite Docker images are now hosted on GitHub per default

v0.2.0

Most important features:

• Scalable workload generator
• Usage of Avro records instead of Kieker