0.5.0

Features

Better support for distributed, async workflows

By @turn1a in #81

The kedro-neptune plugin now respects the NEPTUNE_CUSTOM_RUN_ID environmental variable.

Previously, the plugin could not be used in a distributed asynchronous workflow (such as Kubeflow, AWS Batch, etc.), where multiple containers run independently as each container would create a new run in Neptune to log into during the after_catalog_created() execution in NeptuneHooks.

With this change, you can set NEPTUNE_CUSTOM_RUN_ID to the same value for each container before the kedro run execution and Neptune will only create one run for each Custom Run ID, letting multiple nodes executed asynchronously to log into the same Neptune run.

More about the Custom Run ID in the docs.

Enhanced pipeline execution status tracking

By @SiddhantSadangi in #82

We have added new fields to better track pipeline execution status in realtime:

• The log field shows which nodes have run and are currently running
  

• The execution_order field exports a description of the pipeline (pipeline.describe()), showing the order of execution of the nodes, and expected free input and output variables
  

• Finally, each node now has a status field that shows if the node is "running" or "done".

Fixes

• Fixed the case when node input parameters were not logged to Neptune if passed as a list (by @SiddhantSadangi in #82)

New Contributors

• @turn1a made their first contribution in #81

Full Changelog: 0.4.0...0.5.0