working mvp

[kgpayne]

Closes #1

• feat: native Snowflake bulk unload for BATCH messages #1

[kgpayne]

OK, this is getting close 😅 I have stream property selection working for both RECORD and BATCH and also STATE working for RECORD. State and batch need to be worked on a bit. Right now INCREMENTAL is not supported with the BATCH message type (issue sdk#976) in the SDK, as we don't initialise any existing replication keys on that code path, nor do we provide a mechanism for users to report the bookmark associated with their batch 🤔

[kgpayne]

@pnadolny13 once this merges, it would be good to start using it in squared 😅 Also, do we have a sanitised Snowflake user/database/warehouse that we can use for testing? I have been using internal creds for development, but it would be good to create some tests that utilise an example Snowflake instance that is 'safe' to be used in the MeltanoLabs context. A read-only user with access to example data should be more than sufficient 🙂

[pnadolny13]

@pnadolny13 once this merges, it would be good to start using it in squared 😅

@kgpayne sounds good I created meltano/squared#425 to track migrating to this variant.

Also, do we have a sanitised Snowflake user/database/warehouse that we can use for testing? I have been using internal creds for development, but it would be good to create some tests that utilise an example Snowflake instance that is 'safe' to be used in the MeltanoLabs context. A read-only user with access to example data should be more than sufficient 🙂

We have sample data but we dont have a read only user for that right now but I can create one. I created an issue for tracking it.

[kgpayne]

Couple of questions for feedback 🙂

[edgarrmondragon]

lgtm!