KSML 0.8.0 - Major milestone for KSML

We are happy to announce the release of KSML 0.8.0!

The 0.8.0 release was the next big step for KSML, providing an updated syntax for KSML, increased stability and a lot of exciting features.
See the change and feature list below for all the changes made!

• Reworked all parsing logic, to allow for exporting the JSON schema of the KSML specification:
  • docs/specification.md is now derived from internal parser logic, guaranteeing consistency and completeness.
  • examples/ksml.json contains the JSON schema, which can be loaded into IDEs for syntax validation and completion.
• Improved schema handling:
  • Better compatibility checking between schema fields.
• Improved support for state stores:
  • Update to state store typing and handling.
  • Manual state stores can be defined and referenced in pipelines.
  • Manual state stores are also available in Python functions.
  • State stores can be used 'side-effect-free' (eg. no AVRO schema registration)
• Python function improvements:
  • Automatic variable assignment for state stores.
  • Every Python function can use a Java Logger, integrating Python output with KSML log output.
  • Type inference in situations where parameters or result types can be derived from the context.
• Lots of small language updates:
  • Improve readability for store types, filter operations and windowing operations
  • Introduction of the "as" operation, which allows for pipeline referencing and chaining.
• Better data type handling:
  • Separation of data types and KSML core, allowing for easier addition of new data types in the future.
  • Automatic conversion of data types, removing common pipeline failure scenarios.
  • New implementation for CSV handling.
• Merged the different runners into a single runner.
  • KSML definitions can now include both producers (data generators) and pipelines (Kafka Streams topologies).
  • Removal of Kafka and Axual backend distinctions.
• Configuration file updates, allowing for running multiple definitions in a single runner (each in its own namespace).
• Examples updated to reflect the latest definition format.
• Documentation updated.

Full Changelog: 0.2.2...0.8.0

0.2.2 - Bugfix release

What's Changed

• Fix KSML java process not stopping on exception by @richard-axual in #88
• Fix stream-stream join validation and align other checks by @jeroenvandisseldorp in #90
• Bump logback to 1.4.12 by @jeroenvandisseldorp in #91
• Fix to enable Streams optimisations to be applied to topology by @richard-axual in #92
• Fix resolving admin client issues causing warning messages by @richard-axual in #93

Full Changelog: 0.2.1...0.2.2

Bugfix release Avro and Field Validations

What's Changed

• This fixes an issue with AVRO and field validations by @jeroenvandisseldorp in #87

Full Changelog: 0.2.0...0.2.1

Release 0.2.0

This is an intermediate release that maintains compatibility with snapshot releases of the last couple of months.

What's Changed

• Optimized Docker build by @richard-axual in #70
• Merged KSML Runners into one module, optimize Docker builds and workflow by @richard-axual in #69
• KSML documentation updates by @jeroenvandisseldorp in #71
• Docker image, GraalPy venv, install and GU commands fail by @richard-axual in #73
• Update GitHub Actions by @richard-axual in #75
• Small robustness improvements by @jeroenvandisseldorp in #74
• Issue #72 - Fix build failures when trying to use venv and install python packages by @richard-axual in #78
• Manual state store support, Kafka client cleanups and configuration changes by @jeroenvandisseldorp in #81
• Update and clean up dependencies by @richard-axual in #82
• Update documentation to use new runner configurations by @richard-axual in #83
• Update to GraalVM for JDK 21 Community by @richard-axual in #84

Full Changelog: 0.1.0...0.2.0

0.1.0

What's Changed

• Added XML/SOAP support
• Added datagenerator
• Added Automatic Type Conversion
• Added Schema Support for XML, Avro, JSON, Schema
• Added Basic Error Handling

Full Changelog: 0.0.4...0.1.0

0.0.4

What's Changed

• Update to kafka 3.2.3
• Update to Java 17
• Support multiple architectures in KSML, linux/amd64 and linux/arm64
• Refactored internal typing system, plus some fixes to store operations
• Introduce queryable state stores
• Add better handling of NULL keys and values from Kafka
• Implement schema support
• Added Docker multistage build
• Bug fix for windowed objects
• Store improvements
• Support Liberica NIK
• Switch from Travis CI to Github workflow
• Build snapshot Docker image on pull request merged

And lots of clean up, tests added and structure updates

Full Changelog: 0.0.3...0.0.4

0.0.3

What's Changed

• Python3 support through GraalVM, better data handling
• Use GraalVM in Dockerfile and new Object Types
• Improvements to windowing, window suppresions

Full Changelog: 0.0.2...0.0.3

0.0.2

Several updates to KSML are part of this pre-release.

The main features added are:

• Support for named processors and materializations
• Support for store names
• Json Support

Initial release of KSML

Initial implementation of the KSML application