Visualise pipeline objects in notebook

[ravi-kumar-pilla]

Description

Resolves #1993

NOTE: The bundle URL will be updated once #2268 is merged

Development notes

• Created a class NotebookVisualizer and a method show responsible for visualizing Kedro-Viz using the esm bundle in notebook
• Added load_data_for_notebook_users and load_and_populate_data_for_notebook_users methods to kedro-viz -> integrations -> notebook -> data_loader.py
• Added initialize and reset methods in data_access_manager for reuse
• Added few utility functions and classes
• Update release note, tests and gcp load balancer doc link
• Removed few GraphQL checks from workflow as they were failing

QA notes

• All tests should pass
• For manual testing, open a jupyter notebook and try -

from kedro.pipeline import pipeline, node

def dummy(ds1):
   return ds1
    
n0 = node(dummy, 'flights', 'processed_flights')
dummy_pipe = pipeline([n0])

from kedro_viz.integrations.notebook import NotebookVisualizer
NotebookVisualizer(dummy_pipe).show()

• You can also test demo_project pipelines, try -

from kedro_viz.integrations.notebook import NotebookVisualizer
from demo_project.pipeline_registry import register_pipelines
demo_pipe = register_pipelines()

# Since globalNavigation depends on localStorage, the option is not working.
 
NotebookVisualizer(pipeline=demo_pipe, options={ "display": {
                "expandPipelinesBtn": False,
                "exportBtn": False,
                "labelBtn": False,
                "layerBtn": False,
                "metadataPanel": True,
                "miniMap": False,
                "sidebar": False,
                "zoomToolbar": False,
            },
            "expandAllPipelines": False,
            "behaviour": { 
                "reFocus": False,
            },
            "theme": "dark",
            "width": "100%",
            "height": "600px",   
            }).show()

Testing Results :

Jupyter lab:

Databricks:

Marimo:

VS Code

Checklist

• Read the contributing guidelines
• Opened this PR as a 'Draft Pull Request' if it is work-in-progress
• Updated the documentation to reflect the code changes
• Added new entries to the RELEASE.md file
• Added tests to cover my changes

[astrojuanlu]

1 more thing @ravi-kumar-pilla : do you think having

def _repr_html_(self):
    from kedro_viz.integrations.notebook import NotebookVisualizer

    return NotebookVisualizer(self).generate_html()

would make

In [2]: pipeline
[Just show the HTML]

work with what's currently in this PR?

(That way the user doesn't even have to do any extra imports for this to work)

[ravi-kumar-pilla]

1 more thing @ravi-kumar-pilla : do you think having

def _repr_html_(self):
    from kedro_viz.integrations.notebook import NotebookVisualizer

    return NotebookVisualizer(self).generate_html()

would make

In [2]: pipeline
[Just show the HTML]

work with what's currently in this PR?

(That way the user doesn't even have to do any extra imports for this to work)

Love the idea. Sure we can do that !

[rashidakanchwala]

thanks @ravi-kumar-pilla , works amazing! I only tried it on jupyter notebooks, but did a lot of iterative building pipeline and it was great.
codewise it is in much better state. thankssss

[astrojuanlu]

🚀

[ravi-kumar-pilla]

This will be merged after #2237 as suggested by @rashidakanchwala

[astrojuanlu]

There are some conflicts here, after addressing those this can now be merged @ravi-kumar-pilla