Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix Progress Manager Bars and Remove Duplicate execute_plan() Logic #85

Open
mdr223 opened this issue Jan 27, 2025 · 0 comments
Open

Fix Progress Manager Bars and Remove Duplicate execute_plan() Logic #85

mdr223 opened this issue Jan 27, 2025 · 0 comments
Assignees
Labels
bug Something isn't working

Comments

@mdr223
Copy link
Collaborator

mdr223 commented Jan 27, 2025

The progress managers have some small discrepancies with reality and duplicated code in a couple places, from earlier discussion:

  • In this commit and this commit, which initially added progress bars to no sentinel execution:
    • The execute_plan() method was copied from each PlanExecutor, placed in the respective Execution class, and modified to include progress bars
      • (I also see -- particularly for parallel pipelined execution -- that code improvements were made)
  • I can think of a lot of valid reasons the copying may have been done; my only concern is that now:
    • We have two execute_plan() methods floating around for each plan executor
    • We have a lot of duplicated code
  • If possible, I'd love to come to a consensus on how best to eliminate the duplication
    • (this may include refactoring the abstraction between ExecutionEngine and PlanExecutor, because it is a bit clunky at the moment)

Small notes: I do not want to let the perfect be the enemy of the good, and I am very happy that we now have some progress monitoring!

  • That being said, in the presence of aggregates, limits, and one-to-many converts, total_work_units = total_items * total_ops will not always be true
    • (I'm not sure how important this is from a correctness perspective, but thought I'd point it out)
  • Our outer op progress tracker also does not really translate well for pipelined execution (since it proceeds one record at a time as opposed to one op at a time), so we may want to flip the inner vs. outer progress bars there
  • So overall -- very glad this is here! -- but we may want to tweak some more
@mdr223 mdr223 added the bug Something isn't working label Jan 27, 2025
@mdr223 mdr223 self-assigned this Jan 27, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

1 participant