Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add EP-10381: Overhauling changelog process #10422

Open
wants to merge 4 commits into
base: main
Choose a base branch
from

Conversation

timflannagan
Copy link
Member

@timflannagan timflannagan commented Jan 9, 2025

Description

Add an enhancement proposal for overhauling the changelog process carried over from the gloo migration. See #10381 for the details.

Checklist:

  • I have performed a self-review of my own code
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • I have added tests that prove my fix is effective or that my feature works

timflannagan pushed a commit to timflannagan/kgateway that referenced this pull request Jan 14, 2025

- Update the PR template to include a new section for the user-facing changelog description.
- Document new requirements in the CONTRIBUTING.md file to guide contributors on how to fill out the changelog section.
- Add automation to react to `/kind <fix, new_feature, breaking_change, etc.>` command within the PR template to categorize the type of change.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the automation adds a label, right? can it be done via GH action (i.e. not with an external bot)?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yup, all of this can be done via GHAs too. no need for an external bot afaik.

Copy link
Member Author

@timflannagan timflannagan Jan 15, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I ended up hacking on a POC that automates the changelog process using GHAs:

  • workflow that triggers on PR events, parses the PR description, adds the appropriate labels
  • composite workflow that lists all labels with a release-note, buckets those PRs based on kind/* labels into a known changelog format
  • release workflow can now generate the changelog and pipe that to goreleaser via the --release-notes CLI argument to pump out custom release notes

timflannagan@df45b32 & timflannagan#2.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

https://github.com/vijaykramesh/pr-lint-action is also a nice example of what others have been using for similar things


#### Alternative 3: Automate CI Behavior based on PR Changes

Automatically adjust the CI pipeline based on the files modified in a pull request. For example, a README update might only trigger linting and formatting checks, while code changes trigger full test suites.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i like this option


**Cons:**

- Likely requires sophisticated change detection logic, which introduces complexity in our CI pipeline and have long-term maintenance implications.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

unfortunately, there's some issues when a required GH workflow uses path filtering like this:

If a workflow is skipped due to path filtering, branch filtering, or a commit message, then checks associated with that workflow will remain in a "Pending" state. A pull request that requires those checks to be successful will be blocked from merging.

you can start work around this behavior by adding conditionals to the workflow's jobs to conditionally skip them in lieu of the path filtering syntax.

- **Developer Experience Regression**: With this new process, there may be a slight regression in developer experience. Previously, developers could use `git log` and `git show <commit-hash> -- changelog` to view commit history. Now, they would need to navigate to GitHub to see the full changelog in the pull request. The vscode lens extension could help with this, but it's not as convenient as `git log`. Is this trade-off acceptable, considering the PR number is embedded in the commit title and we have already adopted squash/merge commit settings?
- **CI/CD Integration and Backwards Compatibility**: The current process includes custom fields like `skipCI: true` in the changelog YAML files, which configure CI pipeline behavior. How do we ensure backwards compatibility when decoupling these fields from the changelog, while maintaining the same functionality? How can we continue to control CI behavior without embedding configurations in the changelog itself?
- **Changelog Validation**: Currently, validation is automated. Moving changelog descriptions to PRs could mean relying on PR authors and reviewers to enforce standards manually. Should this be an intermediate step, or can we automate validation post-merge? Should PR changelog descriptions be editable after approval, and if so, how do we maintain proper audit trails?
- Do we need to support NON_USER_FACING anymore? I think in most cases, release notes NONE is sufficient.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i think NONE is sufficient

- The proposal will not change the content or structure of the actual changelog entries used in the release notes; it only modifies how entries are provided, tracked, and aggregated.
- This proposal does not address CI/CD or release automation processes unrelated to changelog management. Commentary on a strawman proposal for CI/CD automation is included in the proposal, but it is not the primary focus.

## Proposal
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One drawback with this proposal that I'm now realizing is the process changelog format supported multiple types:

- type: HELM
  description: Introduced a kubeGateway.enabled toggle ...
- type: NEW_FEATURE
  description: Added support ...

I'm wondering if there's benefit in being able to support multiple types within a single PR, or whether that was an artifact that our bucketing was too broadly grouped, or whether that indicated that PRs with multiple types needed to be broken down into smaller PRs.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

its nice for backports and calling out the concerns but also could be an indication of the pr needing to be broken into more parts if there are multiple user facing changes in the pr

- Update the PR template to include a new section for the user-facing changelog description.
- Document new requirements in the CONTRIBUTING.md file to guide contributors on how to fill out the changelog section.
- Add automation to react to `/kind <fix, new_feature, breaking_change, etc.>` command within the PR template to categorize the type of change.
- Add a separate field for linking the GitHub issue associated with the change.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we want to also add the GitHub issue in the description? Kubernetes release notes don't actually have the issue link required, but you can specify one in the documentation section.

For example kubernetes/kubernetes#123098 has both a release-note section and link to the enhancement issue 4193 which then gets collected: https://github.com/kubernetes/sig-release/blob/d015590aaa89f3bce9a312f615c4e6baed061fd3/releases/release-1.30/release-notes/release-notes-draft.json#L3293

@timflannagan timflannagan changed the title WIP: Add enhancement for overhauling changelog process Add EP-10381: Overhauling changelog process Jan 31, 2025
@timflannagan
Copy link
Member Author

Update the proposal:

  • Conform with the new design doc template.
  • Added some details on the required automation and the contracts between them.
  • Added a section about PR template validation and a non-goal indicating this can be done gradually over time.
  • Mentioned that multiple kinds are supported, but multiple release notes are not. In this case, a single release note would be used for each kind bucket in the rendered changelog.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants