Lineage graph fails to render and due to "missing nodes"

[kaupoondi]

Expected Behavior

We would expect that, with enough time to render, any complex lineage graph without cycles should render. For most of our tables, this seems to be the case.

Current Behavior

For some table (say table Y), we run into an issue where we get an error with the lineage rendering. The front-end is able to hit the metadata service and can fetch the entire lineage structure of Y as a JSON response. However, it seems that the D3 stratify algorithm fails to find a specific node (say table X) and errors. A good reference to the D3 algorithm can be found here.

The table X that it complains about does exist and we can go to its table page in the Amundsen front-end, view its metadata, and even view its lineage graph. However, this table X seems to be missing when table Y's lineage graph is trying to render.

Possible Solution

Steps to Reproduce

We looked through and reproduced this error for a few different tables via Selenium. The tables that the lineage render graph states were missing all exist in our Neo4j database.

Screenshots (if appropriate)

Below, we can see the error response that shows it fails to render due to a missing table.

Context

For key tables, our users are not able to pull up the lineage graph and there are no useful errors to tell the end user that there is an issue either.

Your Environment

• Amundsen version used: Frontend: frontend-4.2.0
• Data warehouse stores: Snowflake
• Deployment (k8s or native): k8s
• Link to your fork or repository:

[boring-cyborg]

Thanks for opening your first issue here!

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs.

[stale]

This issue has been automatically closed for inactivity. If you still wish to make these changes, please open a new pull request or reopen this one.

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs.