Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support for Metrics Layer and Semantics Layer Integration (e.g., dbt Semantic Layers and Cube) #1120

Open
yu-iskw opened this issue Jan 15, 2025 · 2 comments

Comments

@yu-iskw
Copy link

yu-iskw commented Jan 15, 2025

Is your feature request related to a problem? Please describe.
I'm always frustrated when data pipelines lack a standardized way to define metrics and semantics. This can lead to inconsistencies, difficulty in debugging, and unreliable data outputs. A solution like a metrics layer (e.g., dbt Semantic Layer or Cube) could help ensure consistent data definitions and outputs across the stack.

Describe the solution you'd like
I would like WrenAI to support integration with a metrics layer and semantics layer. This could be done by adopting or integrating with tools like dbt Semantic Layers or Cube, which provide a centralized way to define metrics and semantics. By doing so, WrenAI could:

  • Standardize metrics definitions across the stack.
  • Enable self-serve analytics with reliable, consistent data.
  • Simplify maintenance by centralizing business logic and metrics computation.
  • Enhance compatibility with modern data tools that leverage metrics layers.

Describe alternatives you've considered
Alternatives include manually managing metrics definitions across different parts of the data stack, but this is error-prone and difficult to scale. Another alternative could be to use ad-hoc methods to define semantics and metrics in WrenAI, but this risks divergence and inconsistency over time.

Additional context
Supporting a metrics and semantics layer would align WrenAI with modern data practices, as seen in tools like dbt and Cube. It could significantly improve data reliability, simplify integration with other tools, and provide a better experience for data consumers.

Links for reference:

@wwwy3y3
Copy link
Member

wwwy3y3 commented Jan 15, 2025

@yu-iskw thanks for this feature request with so much details!

I'll put it into backlog of our roadmap first.

We did think about integrating existing semantic layer by transforming the semantics of dbt semantic layer or cube into our MDL format (https://docs.getwren.ai/oss/engine/guide/modeling/model). For example, Cube also has data modeling files with their own syntax (https://cube.dev/docs/product/data-modeling/syntax). Should be feasible to read these config files and turn into our MDL.

@yu-iskw
Copy link
Author

yu-iskw commented Jan 15, 2025

@wwwy3y3 Thank you so much for taking the idea.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Status: No status
Development

No branches or pull requests

2 participants