Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve robustness of retrying operations #152

Open
3 tasks
jnation3406 opened this issue Feb 1, 2025 · 0 comments
Open
3 tasks

Improve robustness of retrying operations #152

jnation3406 opened this issue Feb 1, 2025 · 0 comments
Assignees
Labels
bug Something isn't working code cleanup Optimizations, refactors, removing code/files

Comments

@jnation3406
Copy link
Contributor

The way operations currently work is they are immediately queued to execute on creation, and then if something queries the operation after that and it is back in the initial state of PENDING, we assume that means it never executed (or it is no longer in the cache), and trigger it to re-execute then. The problem with this is that when it leaves the cache, the operations it depended on usually have left the cache as well, either because the cache resets or because its been 30 days. Then the frontend triggers retrieving the operations in "random" order, which may not allow them to complete if they depend on each other. I think we can solve this problem purely in the frontend, by having the frontend be a little smarter about polling the operations for status.

Right now any non-completed operation is polled forever until it completes - I would like to change this behavior so that the frontend traverses the operation graph and polls the first non-complete operation it finds until it completes, and then polls the next, and so on until all operations are complete. If we want to be really fancy we can poll multiple at a time as long as they have all their dependencies complete already. This should solve the problem of operations re-executing out of order, and we can use the information of what is complete and what is currently polling and what is not yet in progress/polling to show that info in the frontend, both in the graph and linear operations views.

  • Refactor operation polling to happen in graph order
  • Show some visual of what operations are currently in progress/polling, what are complete, and what are not yet started
  • Show another visual on the graph view of this polling as it occurs
@jnation3406 jnation3406 added bug Something isn't working code cleanup Optimizations, refactors, removing code/files labels Feb 1, 2025
@jnation3406 jnation3406 self-assigned this Feb 1, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working code cleanup Optimizations, refactors, removing code/files
Projects
None yet
Development

No branches or pull requests

1 participant