bytewax-s2

bytewax-s2 is the Python package that provides Bytewax connector for S2, which can be used in your Bytewax pipelines if you want to read from or append to S2 streams.

API overview

bytewax_s2 module in the package exposes the following types:

• S2Source - used for reading from S2 streams.
• S2Sink - used for appending to S2 streams.
• S2Config - used when initializing S2Source and S2Sink.
• S2SourceRecord - items read from S2 streams will be of this type.
• S2SinkRecord - items to be appended to S2 streams must be of this type.
• S2SinkPartitionFn - used for routing the items to appropriate S2 streams.

Please refer to the docstrings of the abovementioned types to know more.

Installation

You can install the package from the Python Package Index using the package manager of your choice. E.g., with pip:

pip install bytewax-s2

Examples

examples/ directory in the repo contain two Bytewax pipelines:

• sink.py - where firehose of events from Bluesky is processed and appended to S2 streams.
• source.py - where processed data is read from S2 streams and insights are plotted in terminal.

If you are interested in knowing more about these pipelines, take a look at this walkthrough.