Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve performance of graph operations. #6243

Open
mtbc opened this issue Jul 28, 2020 · 2 comments
Open

Improve performance of graph operations. #6243

mtbc opened this issue Jul 28, 2020 · 2 comments

Comments

@mtbc
Copy link
Member

mtbc commented Jul 28, 2020

Three separate sources of slowness have been identified:

  1. Early in the process when data is being gathered, profiling shows Hibernate eating a lot of memory. Caches may need configuration and later versions of Hibernate may make that easier. GraphTraversal will tend not to be querying the same objects repeatedly.

  2. In reviewing the data gathered via Hibernate, GraphPolicyRule eats much CPU time in rule-matching. The internal implementation of the class could do non-trivial static analysis when initialized, compiling the list of rules to a decision tree of checks, branches, changes, making the subsequent matching much faster.

  3. In actioning the model changes, PostgreSQL can spend much time in managing the _reindexing_required table. Perhaps migration to https://github.com/glencoesoftware/omero-es can somehow help.

@imagesc-bot
Copy link

This issue has been mentioned on Image.sc Forum. There might be relevant details there:

https://forum.image.sc/t/omero-storage-reports/41819/2

@mtbc
Copy link
Member Author

mtbc commented Sep 1, 2020

non-trivial static analysis when initialized

This could also spot opportunities to reorder rules to reduce number of passes in cases where outcome would remain unchanged. However, there are probably not big wins as the user can already pluck low-hanging fruit, e.g., listing transitions to RELEVANT before transitions from that to IS_NOT_LAST.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants