Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use pytest-benchmark #163

Open
maxrjones opened this issue Jan 22, 2025 · 3 comments
Open

Use pytest-benchmark #163

maxrjones opened this issue Jan 22, 2025 · 3 comments

Comments

@maxrjones
Copy link
Member

I don't expect it to be significant, but there is likely some overhead added to get_range and get_ranges due to the match; case in #156. I'm wondering if we should set up pytest-benchmark and codspeed-hq to monitor for performance regressions, to prevent needing to manually test performance?

@kylebarron
Copy link
Member

It would be great to have pytest-benchmark here. I'm quite skeptical about benchmarks in CI though. I think the best benchmarks are local/on a dedicated machine.

@maxrjones
Copy link
Member Author

maxrjones commented Jan 22, 2025

IIRC codspeed runs comparative benchmarks on the same machine rather than testing against cached results, but I don't like relying on GitHub apps that much so I'd definitely support only adding pytest-benchmark for developers to run locally.

@kylebarron
Copy link
Member

I guess the way I'd think about codspeed is as a way to tell if something went very wrong between PRs, but for small changes I'd trust local results more

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants