Process: Automatically open PRs against upstream projects before/on release

[Stebalien]

We used to have a section in the release process that talked about testing releases against upstream CI before releasing. That was removed, unfortunately, which has led to a few regressions (or, at least, broken tests): #2911, #2910.

However, now that we have the release flow, we should be able to completely automate this.

Idea:

1. When a draft release is published, open a draft PR against upstream projects using that draft release.
2. When a draft release is modified, update said PRs.
3. When a draft release is deleted, close those PRs.
4. When a final release is cut, open/update the upstream PRs to point to the final release (marking it as ready for review).

Ideally we'd find a way to link those PRs back to the release PR so we can track CI progress.

Projects we should check against (for now):

• filecoin-project/lotus
• go-libp2p-kad-dht
• go-libp2p-pubsub
• ipfs/kubo
• ipfs/boxo

If this flow seems to work, there's no reason we can't add more projects later.

[vyzo]

In general I agree with the idea of getting early/immediate integration, but one pain point is that our users (e.g. pubsub users) are not necessarily ready to adopt the new go-libp2p release without going through their testing process.
The concern is that they may want to get a newer pubsub (eg a bug fix) but not the go-lipbp2p that comes with it.
Then again we also care about pubsub integration with the latest release at all times, so it is not clear what is the right thing to do.

[Stebalien]

Upstream projects (e.g., pubsub) can just not merge the PRs if not ready. On the other hand.... libp2p releases tend to include important security fixes.

[vyzo]

yes, i agree and i am quite concerned about this.

[vyzo]

but lets not forget that they also have a tendency to break things... sigh.

[Stebalien]

Breaking changes seemed to have died down quite a bit. But also note: this will help catch unintentional breaking changes.

[vyzo]

yeah... all we need now is a volunteer.

It is a non invasive procedure, with less than 99% chance of implosion creating a microscopic blackhole that devours the earth.

[Stebalien]

If we can reach agreement that this is a good idea, we can ask IPDX to take a crack at it when we next schedule a round of work with them.

[vyzo]

yeah, lets do it -- just no automerge of the pr, even if it passes the tests.

Some maintainer has to merge it.

[sukunrt]

I think it's a nice idea.

Should we start with go-libp2p-kad-dht and go-libp2p-pubsub first and then expand to kubo and lotus after we've had some experience?

[cpeliciari]

Instead of opening a PR for each project, we could have the pipeline simply pull the projects and execute the tests within the pipeline. In case of a failure, we could then open a PR in the respective project with the identified error.

[Stebalien]

That won't work well for large projects with complex workflows (kubo, boxo, lotus, go-ethereum, etc.).

[MarcoPolo]

I'm in favor of the general idea here.

I think this should apply to major/minor releases, but not patch releases. Patch releases are often very small and targeted and important to get out quickly. Major/minor releases are what have the current defined release pipeline.

However, now that we have the release flow, we should be able to completely automate this.

@Stebalien can you elaborate on what you mean here? How would we automate the process you describe?

[Stebalien]

I think this should apply to major/minor releases, but not patch releases. Patch releases are often very small and targeted and important to get out quickly. Major/minor releases are what have the current defined release pipeline.

Totally agree.

[Stebalien]

However, now that we have the release flow, we should be able to completely automate this.

@Stebalien can you elaborate on what you mean here? How would we automate the process you describe?

Now that releases are done through automation, we don't have to manually make the PRs. Instead, we use github actions triggered by draft/final releases and make the PRs via some bot account.