Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add a CI job that runs linting and formatting against latest ruff #2055

Closed
wants to merge 1 commit into from

Conversation

waxlamp
Copy link
Member

@waxlamp waxlamp commented Oct 21, 2024

This is an experimental idea based off of the work in #2053, which pins Ruff to a specific minor version so that dev and CI don't go out of sync. The idea is to give CI the ability to run against latest Ruff and let the failure of that run indicate new rules, etc., that we may want to adopt by fixing those errors and re-pinning the version used in the "real" runs.

@waxlamp
Copy link
Member Author

waxlamp commented Oct 21, 2024

Since this PR was spun out from #2053, I'm copying over the comments I left regarding this experiment:


see if this actually works

So, it works, but in a way that is kind of useless: the ruff-latest test fails inside its run (as expected) but the job shows a green checkmark in the results panel. Not sure if there's a better thing here (something that shows the failure but doesn't fail the run).

@danlamanna, @jjnesbitt, @mvandenburgh: would this be better as a nightly test? That would require duplicating the GH Action spec (which is a negative) but we could allow it to fail freely without compromising the ability to merge PRs, etc.

@waxlamp
Copy link
Member Author

waxlamp commented Oct 21, 2024

see if this actually works

So, it works, but in a way that is kind of useless: the ruff-latest test fails inside its run (as expected) but the job shows a green checkmark in the results panel. Not sure if there's a better thing here (something that shows the failure but doesn't fail the run).

@danlamanna, @jjnesbitt, @mvandenburgh: would this be better as a nightly test? That would require duplicating the GH Action spec (which is a negative) but we could allow it to fail freely without compromising the ability to merge PRs, etc.

Dan pointed out that even with a nightly run, alarm fatigue will set in. A better approach would be for us to just pay attention and run against latest Ruff when new versions come out.

I will try converting this to use the "workflow dispatch" mode which would avoid alarm fatigue, but make it a bit easier to actually run against the latest version.

This is a test added to the matrix of tests run by CI, but marked as
`continue-on-error` so that the full suite does not fail if this test
fails. The purpose is to signal that a later version of ruff may exist
that causes new linting/formatting failures, but allowing the dev team
to manually upgrade to the new version.
@waxlamp
Copy link
Member Author

waxlamp commented Oct 23, 2024

Closing this in favor of #2045 and #2057. the pre-commit autoupdate command can be used to upgrade hook versions in lieu of the mechanisms proposed in this PR.

@waxlamp waxlamp closed this Oct 23, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant