Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[RFC]: Enable stale bot for issues and PRs #9435

Closed
1 task done
russellb opened this issue Oct 16, 2024 · 9 comments · Fixed by #9436
Closed
1 task done

[RFC]: Enable stale bot for issues and PRs #9435

russellb opened this issue Oct 16, 2024 · 9 comments · Fixed by #9436
Labels

Comments

@russellb
Copy link
Collaborator

Motivation.

There are currently >400 open PRs and ~1.7k open issues. Many of these are not active and could be closed. This problem is often more challenging in a very active and popular project like vLLM. GitHub provides an action to help automate this process.

https://github.com/actions/stale

By adopting this action, those with the time and interest in helping with issue triage or PR reviews will have an easier time ensuring their time is used most effectively.

Proposed Change.

I propose adopting actions/stale to automate the process of pruning the issue and PR queues of items with no activity.

A sample policy would be something like this:

  • Add the stale label and leave a comment after an issue or PR has not had any activity within 90 days.
  • Close an issue and add another comment if an issue or PR continues to have no activity for an addition 30 days.

The times are configurable. An interested party can leave a comment to reset the stale timer. Configuring a label that will tell the bot to ignore that issue or PR (like keep-open or something) is also possible.

If there's interest, I am happy to do the PR for it. Agreeing on the policy is the harder and more important part.

Feedback Period.

No response

CC List.

No response

Any Other Things.

No response

Before submitting a new issue...

  • Make sure you already searched for relevant issues, and asked the chatbot living at the bottom right corner of the documentation page, which can answer lots of frequently asked questions.
@russellb russellb added the RFC label Oct 16, 2024
russellb added a commit to russellb/vllm that referenced this issue Oct 16, 2024
This commit adds configuration to run the `actions/stale` github
action that closes stale issues and PRs.

Further discussion is in the issue.

Closes vllm-project#9435

Signed-off-by: Russell Bryant <[email protected]>
@russellb
Copy link
Collaborator Author

a draft PR showing sample configuration is here: #9436

@simon-mo
Copy link
Collaborator

This sounds good. Two comment:

  • I created a keep-open label. However, not everyone has the ability to attach label. Can this be done by some other way?
  • We don't have to close PRs that are draft. Maybe draft PR can be a good way to avoid getting stale and closed.

@hmellor
Copy link
Collaborator

hmellor commented Oct 17, 2024

I also support this, in the past I have spent many hours closing 100s of issues but it's not sustainable.

I'm happy to be a reviewer on the PR to dial in the behaviour.

@hmellor
Copy link
Collaborator

hmellor commented Oct 17, 2024

actions/stale has an argument to exempt draft PRs from being marked as stale (and therefore closed) https://github.com/actions/stale?tab=readme-ov-file#exempt-draft-pr

There is also an argument to exempt specific labels like keep-open https://github.com/actions/stale?tab=readme-ov-file#exempt-pr-labels

@russellb
Copy link
Collaborator Author

This sounds good. Two comment:

  • I created a keep-open label. However, not everyone has the ability to attach label. Can this be done by some other way?
  • We don't have to close PRs that are draft. Maybe draft PR can be a good way to avoid getting stale and closed.

Drafts and issues with the keep-open label are now ignored in the proposed config. Thanks, @hmellor!

I have mixed feelings on access to apply the keep-open label. I propose for now having a github workflow that will allow anyone to use a "/keep-open" command in a comment to add the label. That could get abused, so I wouldn't necessarily advertise it since really we'd rather issues and PRs actually stay active. We can always remove that ability if it is abused, though. I'll include that in the draft PR.

@russellb
Copy link
Collaborator Author

As an interesting data point, here is the expected impact of the proposed config based on closing things after 120 days of inactivity:

  • Number of issues not updated in the last 120 days: 495
$ gh issue list --repo vllm-project/vllm --limit 2000 --state open --json number,updatedAt --jq '[.[] | select((now -
 (.updatedAt | fromdateiso8601)) > (120 * 86400))] | length'
495
  • Number of PRs not updated in the last 120 days: 106
$ gh pr list --repo vllm-project/vllm --limit 2000 --state open --json number,updatedAt --jq '[.[] | select((now - (.
updatedAt | fromdateiso8601)) > (120 * 86400))] | length'
106

@hmellor
Copy link
Collaborator

hmellor commented Oct 18, 2024

I agree that a single comment is might make it a little too easy to keep an issue open forever.

In the PR you have the workflow running on every comment, might that get expensive given the volume of comments received in this repo?

@simon-mo would it be so bad if the only people able to add the keep-open label had triage permissions and above?

@hmellor
Copy link
Collaborator

hmellor commented Oct 18, 2024

Actually, how about instead of automatically adding keep-open on "unstale" we add an intermediate label such as unstale (which would be removed if the issue went stale again).

Then people with triage and above can filter by this label and decide on a case by case basis?

@russellb
Copy link
Collaborator Author

I like restricting it to those with triage access.

unstale to indicate that "this was once stale, but then was updated again" also seems helpful for finding things that have been dragging on for a long time.

It's probably worth adding a page to the docs that covers all of this process info! I'll add that when the dust settles.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants