RAI Workflow Manager v0.0.21

What's Changed

• fix exportJointly option by @AndrewNepogoda in #35
• update version by @AndrewNepogoda in #36

Full Changelog: v0.0.20...v0.0.21

RAI Workflow Manager v0.0.20

What's Changed

• Update RAI Python SDK by @AndrewNepogoda in #34

Full Changelog: v0.0.19...v0.0.20

RAI Workflow Manager v0.0.19

What's Changed

• Log warnings when no files for declared source by @AndrewNepogoda in #32

Full Changelog: v0.0.18...v0.0.19

RAI Workflow Manager v0.0.18

What's Changed

• Added chunk partitioning support for non date partitioned sources by @AndrewNepogoda in #30

Full Changelog: v0.0.17...v0.0.18

RAI Workflow Manager v0.0.17

What's Changed

PRs:

• Add bound for import config by @AndrewNepogoda in #29

Full Changelog: v0.0.16...v0.0.17

RAI Workflow Manager v0.0.16

What's Changed

• Update setup.py by @AndrewNepogoda in #28

Full Changelog: v0.0.15...v0.0.16

RAI Workflow Manager v0.0.15

What's Changed

PRs:

• Fix boolean CLI arguments by @AndrewNepogoda in #25
• Update README by @AndrewNepogoda in #26
• added --cleanup-db and --cleanup-engine. Fix bug in db creation by @AndrewNepogoda in #27

Full Changelog: v0.0.14...v0.0.15

RAI Workflow Manager v0.0.14

What's Changed

PR: Snowflake as datasource by @AndrewNepogoda in #24

We added Snowflake support as a new data source.
Snowflake container is supported only for data ingestion.
Reference: Snowflake Integration

Full Changelog: v0.0.13...v0.0.14

RAI Workflow Manager v0.0.13

What's Changed

PRs:

• Allow multiple locations for source import/export by @AndrewNepogoda in #23

1. Deleted CLI arguments:

• --run-mode
• --dev-data-dir
• --output-root

2. CLI Arguments replacement with loader.toml:

CLI arguments --run-mode local, --dev-data-dir, --output-root now replaced with local containers in loader.toml:

Arguments

python main.py --run-mode local --dev-data-dir ./data --output-root ./output ...

loader.toml

[[container]]
name="input"
type="local"
data_path="./data"

[[container]]
name="export"
type="local"
data_path="./output"

3. Renamed properties in loader.toml:

• RAI_PROFILE -> rai_profile
• RAI_PROFILE_PATH -> rai_profile_path
• RAI_SDK_HTTP_RETRIES ->rai_sdk_http_retries

4. Azure container configuration in loader.toml

### Azure blob ###
AZURE_EXPORT_ACCOUNT="account"
AZURE_EXPORT_CONTAINER="dev"
AZURE_EXPORT_DATA_PATH="output"
AZURE_EXPORT_SAS="token"

### Azure IMPORT blob ###
AZURE_IMPORT_ACCOUNT="account"
AZURE_IMPORT_CONTAINER="dev"
AZURE_IMPORT_DATA_PATH="data"
AZURE_IMPORT_SAS="token"

Replaced by:

[[container]]
name="export"
type="azure"
account="account"
container="dev"
data_path="output"
sas="token"

[[container]]
name="input"
type="azure"
account="account"
container="dev"
data_path="data"
sas="token"

5. defaultContainer parameter for ConfigureSources and Export steps in batch config

A new defaultContainer is required for ConfigureSources and Export steps and name a container which will be used as default container for import/export.

6. container parameter for sources and exports

container for sources in ConfigureSources and exports in Export steps allow user specify the name of the container which will be used for the source for import/export. This feature allow user specify multiple datasources (local, azure, snowflake(upcoming), s3(in road map)) as inputs and outputs and mix different types of datasources.

Full Changelog: v0.0.12...v0.0.13

RAI Workflow Manager v0.0.12

What's Changed

• adding snapshot source support by @stanislav-bedunkevich in #20

Full Changelog: v0.0.11...v0.0.12