Repoint to Snowplow's new-generation data models

[alexanderdean]

The Dbt Snowplow model is years out of date at this point; the modern Snowplow data models are here:

https://github.com/snowplow/data-models

/cc @paulboocock @carabaestlein

[buremba]

Hey @alexanderdean, we model the main events table and make use of the dbt's Snowplow package for sessionization but dbt also doesn't seem to support the new data models.

We actually deprecated our in-house data modeling language in favor of our new dbt integration which extends dbt's resource files in order to build something similar to LookML but for behavioral data.

Looking at the repository that you shared, my understanding is that Snowplow takes care of the transformation itself so we don't need dbt transformation. In that case, I see two ways to make Rakam work with the new data models:

1. If there is a way to get the Iglu schemas for a Snowplow deployment, we can generate dbt sources from JSON Schema using a macro in Rakam. We were already planning to build something similar for Iteratively & Avo integration.

2. If Snowplow creates tables & columns with the descriptions defined in Iglu schema, we already have a way to create dbt sources from database tables so it might be the easier way to integrate with the new models. We implemented Segment and Firebase to Rakam running a metadata query to extract the event type & property definitions and create the models with the metrics that are relevant to the event types.

I will try out Snowplow's new version this weekend in order to understand how the new data models work. Also, it would be great to discuss it with your team before actually implementing the new models.

[alexanderdean]

Thanks for the detailed note @buremba - will leave Cara and Paul to follow up from our side!