Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enhancement: Introduce performance benchmarking into continuous integration #7

Open
griffinmilsap opened this issue Dec 27, 2022 · 0 comments
Labels
ezmsg-core Relates to the ezmsg.core package good first issue

Comments

@griffinmilsap
Copy link
Collaborator

It would be lovely to get a textual or ideally graphical representation of performance testing tests/test_perf.py for various message sizes and pub-sub configurations including pub-sub in same process (TX_LOCAL), on same machine (TX_SHM), and on different machines (TX_TCP; can be forced with force_tcp in Publisher constructor so as to avoid a requirement for multiple machines in CI). Test should also vary message size (in powers of 2) from 8 bytes to ~2MB messages, and also evaluate fan-out (1, 2, 4, 8 subs connected to 1, 2, 4, 8 pubs). To get accurate numbers, the test-duration should probably be 10-30 sec.

Given this is probably a time-consuming test and performance regressions shouldn't necessarily be blocking for pull-requests (sometimes new features come at the expense of performance); this test should be configured as a separate GitHub Action.

@griffinmilsap griffinmilsap added ezmsg-core Relates to the ezmsg.core package good first issue labels Dec 27, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
ezmsg-core Relates to the ezmsg.core package good first issue
Projects
None yet
Development

No branches or pull requests

1 participant