[Parent] - Kedro VSCode IDE Roadmap

[noklam]

Tracker:

• For Test and Review: React component work all merged in single PR. kedro-viz#1992 setup the path to use Kedro-viz as a React Component, this gives us the opportunity to integrate viz in VSCode directly via Webview

0.2.0(pre-release) &0.2.2
Release after next kedro viz (1st week of September, as rely on both the main npm/python kedro-viz)

• Kedro-Viz VS Extension Spike  kedro-viz#1968
• Decide to vendor kedro-viz or require user to install #72
• Track command usage with telemetry #68
• Reduce artifact sizes before release of 0.2.0 #62
• Introduce Kedro viz in VSCode Extension #65
• Introduce a callback as prop to React component to track node click in flowchart kedro-viz#2022
• Feature: Implement a way for visual component to communicate with LSP #56
• Reuse LSP server from Kedro Extension for "go To Definition" and "Go to reference" action from Webview. kedro-viz#2024
• Remove kedro-viz backend server, fetch data directly instead #70
• Introduce get_kedro_project_json_data function for VSCode integration  kedro-viz#2049
• Deferred dependencies import into run_server kedro-viz#2060
• Update Kedro-Viz version number after Kedro-viz latest release #90
• Check if Python dependencies is installed before setting flag  #92
• Asking user to install Kedro-Viz #104
• Optimise the time to launch Kedro-viz and avoid restarting the Webview when reload #73
• Investigate possibility of not spanning the whole webserver to get JSON from kedro-viz #67
• Update flowchart on Kedro/LSP server restart #99
• Make sure kedro-viz is an optional dependency and don't crash the server. #71
• Documentation: Kedro-Viz in VSCode extension  #93

0.2.3 (bug fix release)

• Support kedro project path #53
• Extension not working properly for Windows #102
• Silence errors when not on a Kedro project #77
• Support filtering of pipelines #123
• Review telemetry #78
• Viz flowchart re-focus on every click  #126

0.3.0

• Use symbol / cache to improve performance
• %load_node integration with flowchart - One Click debugging experience #60
• [Spike] - Explore how to validate dataset classes in catalog using LSP #5

Not scheduled (wait a bit for telemetry/issues coming in):

• Add navigation support for OmegaConfig Syntax #8
• Crashing on multi-root VSCode workspace #10
• Viz flowchart re-render too frequent and order is non-deterministic #64
• Support for namespaced pipelines #51
• VS Code extension disable hook fail to initialise when use with other Kedro plugin #69

[noklam]

2024-08-21 Kedro Extension / Viz integration

Flowchart Rendering

The non-deterministic rendenring of flowchart creates challenge when used with auto-reload in VSCode. Due to the high frequency of reloading, the node position will flip often for small/medium size pipeline and create an awkward UX.

• @rashidakanchwala mention that the rendering part is done in some old code and is not easy to control because the algorithm has some randomness (as I understand it's an optimisation algorithm)
• @merelcht challenge if it's actually required to have auto-reload

Temporary solution:

• Disable autoreload, ask user to re-run the Run viz command instead.
  • Could have a button, but probably not worth it for temporary solution, we just keep it simple and document the behavior clearly.

Other options discussed:

• Have an option to turn off autoreload if user want.
• Reduce the autoreload frequency, maybe every 30s or only reload when there are new changes. This can reduce the issue but wouldn't solve the problem completely.

Vendor Viz/Ask user to install kedro-viz separately

• @rashidakanchwala mentioned we could have a kedro-viz[core] to remove dependencies like FastAPI, uvicorn, graphql etc.
• We could do an exercise to see how much libraries we need to vendor into the extension. @ravi-kumar-pilla gives some number showing that the dependencies may not be that big (in terms of filesize)
• We seems to have consensus that vendoring is a better option at the end, but should do an exercise to check if it actually works. Benefit would be we can pin the kedro-viz version so future kedro-viz release will not break the extension easily, and it's easier to install as the process is invisible to the user.

Telemetry

• @Huongg mentioned it will be useful to differentiate telemetry between a browser and extension.
• @DimedS comment that maybe we need to have a look at kedro-viz telemetry consent flow, it should respect to the changes we made for kedro-telemetry lately, for example the environment variable DO_NOT_TRACK.
• @rashidakanchwala mentioned that she wasn't sure if the HEAP event will be sent properly in the React component

Follow up:

• Check with how consent flow works for kedro-viz
• How to identify unique user/project in UI (Is it the same as kedro-telemetry?)
• Check if the flowchart React component send HEAP event