Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Setup CI pipeline for reliable benchmarks #167

Closed
jzelinskie opened this issue Oct 4, 2021 · 1 comment
Closed

Setup CI pipeline for reliable benchmarks #167

jzelinskie opened this issue Oct 4, 2021 · 1 comment
Labels
area/perf Affects performance or scalability area/tooling Affects the dev or user toolchain (e.g. tests, ci, build tools) priority/2 medium This needs to be done state/needs discussion This can't be worked on yet

Comments

@jzelinskie
Copy link
Member

GitHub Actions (and most cloud VMs) have problems with noisy neighbors that make extracting consistent performance numbers from benchmarks difficult.

We should setup a dedicated worker for benchmarks or explore using a service like bencher to get reliable benchmarks built into our CI workflow.
@jzelinskie jzelinskie added priority/2 medium This needs to be done area/perf Affects performance or scalability area/tooling Affects the dev or user toolchain (e.g. tests, ci, build tools) state/needs discussion This can't be worked on yet labels Oct 4, 2021
@jzelinskie jzelinskie added hint/good first issue Someone new could handle this and removed hint/good first issue Someone new could handle this labels Oct 28, 2021
@jzelinskie
Copy link
Member Author

Closing in favor of #6

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
area/perf Affects performance or scalability area/tooling Affects the dev or user toolchain (e.g. tests, ci, build tools) priority/2 medium This needs to be done state/needs discussion This can't be worked on yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@jzelinskie