docs(.github/release): describe performance benchmarking steps #2442
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -20,9 +20,16 @@ about: 'Start a new libp2p release.' | |
| - [ ] **Stage 0 - Finishing Touches** | ||
| - [ ] Go through relevant libp2p repos looking for unreleased changes that should make it into the release. If you find any, cut releases. | ||
| - [ ] Run `go get -u ./...` to see if there are any out-of-date deps that look important. If there are, bubble them. Try to avoid _directly_ updating indirect deps in go-libp2p's `go.mod` when possible. | ||
| - [ ] Performance Benchmarking | ||
| - [ ] Follow the instructions [on adding a new version to the libp2p performance benchmark tooling](https://github.com/libp2p/test-plans/tree/master/perf#adding-a-new-implementation-or-a-new-version). Instead of the final go-libp2p release commit, which obviously doesn't yet exist, use the most recent available commit. See https://github.com/libp2p/test-plans/pull/242 as an example. | ||
| - [ ] Run the benchmarks [via the GitHub _perf_ Action](https://github.com/libp2p/test-plans/tree/master/perf#running-via-github-action), wait for the action to push a commit to your https://github.com/libp2p/test-plans/ pull request and compare the performance of the upcoming release with the previous release [using the dashboard](https://github.com/libp2p/test-plans/tree/master/perf#running-via-github-action). | ||
| - [ ] **Stage 1 - Release** | ||
| - [ ] Publish the release through the GitHub UI, adding the release notes. Some users rely on this to receive notifications of new releases. | ||
| - [ ] Announce the release on the [discuss.libp2p.io](https://discuss.libp2p.io). | ||
| - [ ] Performance Benchmarking | ||
| - [ ] Replace the most recent (pre-release) go-libp2p commit with the final go-libp2p release commit on your previously created https://github.com/libp2p/test-plans/ pull request. | ||
| - [ ] Retrigger [the GitHub _perf_ Action](https://github.com/libp2p/test-plans/tree/master/perf#running-via-github-action) and wait for it to push the results as a new commit. | ||
|
There was a problem hiding this comment. Can this be triggered automatically? There was a problem hiding this comment. I think we can trigger this when we add a comment in the pull request per https://docs.github.com/en/actions/using-workflows/events-that-trigger-workflows#discussion_comment For example commenting There was a problem hiding this comment.
👍 preference for comment vs label for that reason. |
||
| - [ ] Merge the https://github.com/libp2p/test-plans/ pull request. | ||
| - [ ] **Stage 2 - Update Upstream** | ||
| - [ ] Update the examples to the final release | ||
| - [ ] Update the upstream dependencies to the final release and create PRs. | ||
|
|
||
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This feels kind of heavy. Any chance we can add this to the uCI release workflow?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Off the top of my head I can think of the following automation:
go get -u ...to update to the to-be-tagged-as-release go-libp2p commit.perfGitHub Action on https://github.com/libp2p/test-plans/ with the above created branch.https://observablehq.com/@libp2p-workspace/performance-dashboard?branch=<LIBP2P-TEST-PANS-BRANCH-NAME>@galargh what do you think of the above? Any suggestions? Also would you be interested and have capacity to help?
@marten-seemann in case of a breaking change in go-libp2p you would still want to manually update the go-libp2p perf protocol implementation in libp2p/test-plans. Though this is not a blocker for any automation here. E.g. one could have the libp2p/test-plans pull request creation be automated and then only require the final touches to the pull request to be manual.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Friendly ping @galargh. Any thoughts on the above?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I quickly put together how something like this could look like - libp2p/test-plans#256.