Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[CPU] Enable memory reuse for nested graphs #27521

Open
wants to merge 4 commits into
base: master
Choose a base branch
from

Conversation

EgorDuplensky
Copy link
Contributor

@EgorDuplensky EgorDuplensky commented Nov 12, 2024

Details:

  • All the nested graphs are now must be a part of a global memory reuse logic
  • The core logic of the memory reuse is untouched (a bit refactored)
  • Instead of solving memory reuse for every graph / subgraph, now all the edges and global execution indices are collected from the "virtually flatten" graph first and then memory reuse is solved once for a model.
  • All the nodes with nested graphs are updated, including:
    1. LoRa
    2. Composite
    3. If
    4. TensorIterator
    5. Convolution + Sum fallback subgraph

Tickets:

  • ticket-id

@EgorDuplensky EgorDuplensky requested review from a team as code owners November 12, 2024 10:53
@github-actions github-actions bot added category: inference OpenVINO Runtime library - Inference category: CPU OpenVINO CPU plugin category: build OpenVINO cmake script / infra labels Nov 12, 2024
@EgorDuplensky EgorDuplensky changed the title Enable memory reuse for nested graps Enable memory reuse for nested graphs Nov 12, 2024
@EgorDuplensky EgorDuplensky changed the title Enable memory reuse for nested graphs [CPU] Enable memory reuse for nested graphs Nov 12, 2024
@EgorDuplensky EgorDuplensky force-pushed the enable_memory_reuse_for_nested_graphs branch from da1d1a0 to 18563de Compare November 12, 2024 15:27
@EgorDuplensky EgorDuplensky force-pushed the enable_memory_reuse_for_nested_graphs branch from 18563de to 4734e61 Compare November 13, 2024 09:50
@EgorDuplensky EgorDuplensky force-pushed the enable_memory_reuse_for_nested_graphs branch from 4734e61 to e140018 Compare November 13, 2024 13:32
@github-actions github-actions bot removed the category: inference OpenVINO Runtime library - Inference label Nov 13, 2024
@EgorDuplensky
Copy link
Contributor Author

@maxnick Ready for review. Could you please take a look?

@EgorDuplensky EgorDuplensky force-pushed the enable_memory_reuse_for_nested_graphs branch from e140018 to d5bda07 Compare November 13, 2024 13:51
@EgorDuplensky
Copy link
Contributor Author

Added a fix for Convolution + Sum fallback graph.
Now such graph is also a part of memory reuse.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
category: build OpenVINO cmake script / infra category: CPU OpenVINO CPU plugin
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants