Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

No transaction in TaskRunRecorder #16829

Merged
merged 1 commit into from
Jan 23, 2025
Merged

No transaction in TaskRunRecorder #16829

merged 1 commit into from
Jan 23, 2025

Conversation

zzstoatzz
Copy link
Collaborator

related to #16299

in the TaskRunRecorder we call record_task_run_event which performs

  • an insert of the task run where the state timestamp is newer
  • an insert of the task run state
  • an update of the task run where the timestamp is non-existent or older than the current state's timestamp

record_task_run_event wraps these statements in a transaction, but these operations should be idempotent. In the interest of speeding up the many writes that may occur when running task-heavy flows, this PR removes begin_transaction=True

@zzstoatzz zzstoatzz added the development Tech debt, refactors, CI, tests, and other related work. label Jan 23, 2025
Copy link

codspeed-hq bot commented Jan 23, 2025

CodSpeed Performance Report

Merging #16829 will not alter performance

Comparing no-transaction (8ddf45f) with main (753ae01)

Summary

✅ 2 untouched benchmarks

Copy link
Collaborator

@chrisguidry chrisguidry left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yep, I'm pretty confident about this one. In Prefect Cloud (which does have some slightly different transactional semantics), these are independent statements not in a transaction either. In the case of errors or message replays, the final system state should still be coherent.

@zzstoatzz zzstoatzz changed the title No transaction No transaction in TaskRunRecorder Jan 23, 2025
@cicdw
Copy link
Member

cicdw commented Jan 23, 2025

Tests related to this change are failing - does this change require an explicit session.commit() call?

@zzstoatzz
Copy link
Collaborator Author

zzstoatzz commented Jan 23, 2025

Tests related to this change are failing - does this change require an explicit session.commit() call?

@cicdw looking now!

remove entire kwarg

add commit
@zzstoatzz zzstoatzz merged commit 525dfb8 into main Jan 23, 2025
45 checks passed
@zzstoatzz zzstoatzz deleted the no-transaction branch January 23, 2025 17:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
development Tech debt, refactors, CI, tests, and other related work.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants