Skip to content

Commit

Permalink
Update Kedro-viz architecture diagram (#2183)
Browse files Browse the repository at this point in the history
* update architecture diagram and add link to full diagram

Signed-off-by: Sajid Alam <[email protected]>

* update based on reviews

Signed-off-by: Sajid Alam <[email protected]>

* Update ARCHITECTURE.md

Signed-off-by: Sajid Alam <[email protected]>

---------

Signed-off-by: Sajid Alam <[email protected]>
  • Loading branch information
SajidAlamQB authored Nov 12, 2024
1 parent 852e1e1 commit 49c9183
Show file tree
Hide file tree
Showing 3 changed files with 8 additions and 1 deletion.
Binary file added .github/img/backend-architecture.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added .github/img/frontend-architecture.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
9 changes: 8 additions & 1 deletion ARCHITECTURE.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@ For further information, see also:

- [Kedro-Viz contributing documentation](CONTRIBUTING.md), which covers how to start development on the project
- [Kedro-Viz style guide](STYLE_GUIDE.md), which walks through our standards and recommended best practices for our codebase
- [Kedro-Viz Architecture Diagram](https://miro.com/app/board/uXjVKhNg1RE=/?moveToWidget=3458764606468376036&cot=10), to see a high level overview of both back-end and front-end and how they are connected.

## High-level Overview

Expand Down Expand Up @@ -62,7 +63,7 @@ The `localStorage` state is updated automatically on every Redux store update, v

## Data ingestion

![Kedro-Viz data flow diagram](/.github/img/app-architecture-data-flow.png)
![Kedro-Viz data flow diagram](/.github/img/frontend-architecture.png)

Kedro-Viz currently utilizes two different methods of data ingestion: the Redux setup for the pipeline and flowchart-view related components, and GraphQL via Apollo Client for the experiment tracking components.

Expand Down Expand Up @@ -147,3 +148,9 @@ Kedro-Viz includes a graph layout engine, for details see the [layout engine doc
Our layout engine runs inside a web worker, which asynchronously performs these expensive calculations in a separate CPU thread, in order to avoid this blocking other operations on the main thread (e.g. CSS transitions and other state updates).

The app uses [redux-watch](https://github.com/ExodusMovement/redux-watch) with a graph input selector to watch the store for state changes relevant to the graph layout. If the layout needs to change, this listener dispatches an asynchronous action which sends a message to the web worker to instruct it to calculate the new layout. Once the layout worker completes its calculations, it returns a new action to update the store's `state.graph` property with the new layout. Updates to the graph input state during worker calculations will interrupt the worker and cause it to start over from scratch.

## Backend Architecture

![Kedro-Viz backend architecture](/.github/img/backend-architecture.png)

The backend of Kedro-Viz serves as the data provider and API layer that interacts with Kedro projects and manages data access for visualisations in the frontend. It offers both REST and GraphQL APIs to support data retrieval for the frontend, allowing access to pipeline structures, node-specific details, and experiment tracking data. Key components include the `DataAccessManager`, which interfaces with data `Repositories` to fetch and structure data. The CLI enables users launch with Kedro-Viz from the command line, while deploy and build options enables seamless sharing of pipeline visualisations on any static website hosting platform.

0 comments on commit 49c9183

Please sign in to comment.