Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

rfc: SSAS ingestion #4

Open
wants to merge 4 commits into
base: main
Choose a base branch
from

Conversation

DmytroYurchuk
Copy link

No description provided.

@DmytroYurchuk DmytroYurchuk changed the title SSAS ingestion rfc: SSAS ingestion Sep 7, 2023
Copy link
Collaborator

@hsheth2 hsheth2 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Broadly looks good. We'd accept a contribution around this, provided it is well-tested and well-documented

I responded to the open questions as inline comments


## Unresolved questions

- It would be better to create this module as a standalone, focusing solely on SSAS, or should it be integrated into the existing Mssql module?
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It probably makes sense for this to be a separate ingestion source, standalone from the existing mssql module.

## Unresolved questions

- It would be better to create this module as a standalone, focusing solely on SSAS, or should it be integrated into the existing Mssql module?
- Is it relevant to add SSAS entities (catalog, cube, dimension, measure) to the DataHub?
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The mapping you described above (catalog -> container, etc) should be fine. For dimensions and measures, we can model those as schema fields with tags of "Dimension" or "Measure". We already do something similar for Looker.

Eventually we want to add dimensions/measures as more first class things within datahub, and can migrate accordingly when the time comes.


## Future Work

Establish a complete data lineage from the data source to the analytical models.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Given how important data lineage is to the discovery process, I wonder if this should be in scope for the initial version instead of left as future work.


- It would be better to create this module as a standalone, focusing solely on SSAS, or should it be integrated into the existing Mssql module?
- Is it relevant to add SSAS entities (catalog, cube, dimension, measure) to the DataHub?
- Does the proposed communication method with SSAS align with the project's needs?
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes - the ingestion source can connect to SSAS and push metadata into datahub.

- It would be better to create this module as a standalone, focusing solely on SSAS, or should it be integrated into the existing Mssql module?
- Is it relevant to add SSAS entities (catalog, cube, dimension, measure) to the DataHub?
- Does the proposed communication method with SSAS align with the project's needs?
- Does the proposed entity mapping approach for SSAS entities suit the project's requirements?
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes - see my comment above

@DmytroYurchuk
Copy link
Author

@hsheth2 Thank you very much

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants