Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[pycodestyle] Implement continued indentation related rules (E12x) #13585

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

LordAro
Copy link

@LordAro LordAro commented Oct 1, 2024

Summary

I've made a start at implementing the remaining pycodestyle E12x warnings. Largely by translating the implementation in pycodestyle

See #2402

Test Plan

Copied E12.py from pycodestyle and added the relevant test cases.

E133 is untested (the same as pycodestyle!)

Copy link

codspeed-hq bot commented Oct 1, 2024

CodSpeed Performance Report

Merging #13585 will not alter performance

Comparing LordAro:e12x-continued-indentation (f2c0bbe) with main (2272880)

Summary

✅ 32 untouched benchmarks

@LordAro LordAro force-pushed the e12x-continued-indentation branch from fb038ab to bf09ebb Compare October 1, 2024 16:12
Copy link
Contributor

github-actions bot commented Oct 1, 2024

ruff-ecosystem results

Linter (stable)

✅ ecosystem check detected no linter changes.

Linter (preview)

ℹ️ ecosystem check detected linter changes. (+867 -2788 violations, +0 -0 fixes in 13 projects; 42 projects unchanged)

apache/superset (+3 -0 violations, +0 -0 fixes)

ruff check --no-cache --exit-zero --ignore RUF9 --no-fix --output-format concise --preview --select ALL

+ superset/db_engine_specs/hana.py:57:17: E128 Continuation line under-indented for visual indent
+ superset/db_engine_specs/hive.py:265:17: E128 Continuation line under-indented for visual indent
+ superset/db_engine_specs/oracle.py:55:17: E128 Continuation line under-indented for visual indent

aws/aws-sam-cli (+7 -202 violations, +0 -0 fixes)

ruff check --no-cache --exit-zero --ignore RUF9 --no-fix --output-format concise --preview

- tests/integration/testdata/buildcmd/terraform/application_outside_root_directory/src/artifacts/layer1/lib.py:1:1: I001 [*] Import block is un-sorted or un-formatted
- tests/integration/testdata/buildcmd/terraform/application_outside_root_directory/src/artifacts/layer1/lib.py:1:8: F401 [*] `boto3` imported but unused
- tests/integration/testdata/buildcmd/terraform/application_outside_root_directory/src/artifacts/layer1/lib.py:3:1: E302 [*] Expected 2 blank lines, found 1
- tests/integration/testdata/buildcmd/terraform/application_outside_root_directory_windows/src/artifacts/layer1/lib.py:1:1: I001 [*] Import block is un-sorted or un-formatted
- tests/integration/testdata/buildcmd/terraform/application_outside_root_directory_windows/src/artifacts/layer1/lib.py:1:8: F401 [*] `boto3` imported but unused
- tests/integration/testdata/buildcmd/terraform/application_outside_root_directory_windows/src/artifacts/layer1/lib.py:3:1: E302 [*] Expected 2 blank lines, found 1
- tests/integration/testdata/buildcmd/terraform/application_outside_root_directory_windows_container/src/artifacts/layer1/lib.py:1:1: I001 [*] Import block is un-sorted or un-formatted
- tests/integration/testdata/buildcmd/terraform/application_outside_root_directory_windows_container/src/artifacts/layer1/lib.py:1:8: F401 [*] `boto3` imported but unused
- tests/integration/testdata/buildcmd/terraform/application_outside_root_directory_windows_container/src/artifacts/layer1/lib.py:3:1: E302 [*] Expected 2 blank lines, found 1
- tests/integration/testdata/buildcmd/terraform/unsupported/lambda_function_with_count_and_invalid_sam_metadata/lambda_src/layer1/lib.py:1:1: I001 [*] Import block is un-sorted or un-formatted
... 199 additional changes omitted for project

binary-husky/gpt_academic (+114 -2500 violations, +0 -0 fixes)

ruff check --no-cache --exit-zero --ignore RUF9 --no-fix --output-format concise --preview

- check_proxy.py:106:11: RUF002 Docstring contains ambiguous `:` (FULLWIDTH COLON). Did you mean `:` (COLON)?
- check_proxy.py:112:42: RUF002 Docstring contains ambiguous `,` (FULLWIDTH COMMA). Did you mean `,` (COMMA)?
- check_proxy.py:117:17: RUF002 Docstring contains ambiguous `,` (FULLWIDTH COMMA). Did you mean `,` (COMMA)?
- check_proxy.py:119:5: I001 [*] Import block is un-sorted or un-formatted
- check_proxy.py:128:44: RUF001 String contains ambiguous `,` (FULLWIDTH COMMA). Did you mean `,` (COMMA)?
- check_proxy.py:128:80: RUF001 String contains ambiguous `,` (FULLWIDTH COMMA). Did you mean `,` (COMMA)?
- check_proxy.py:12:37: RUF002 Docstring contains ambiguous `(` (FULLWIDTH LEFT PARENTHESIS). Did you mean `(` (LEFT PARENTHESIS)?
- check_proxy.py:12:56: RUF002 Docstring contains ambiguous `)` (FULLWIDTH RIGHT PARENTHESIS). Did you mean `)` (RIGHT PARENTHESIS)?
- check_proxy.py:135:18: RUF001 String contains ambiguous `,` (FULLWIDTH COMMA). Did you mean `,` (COMMA)?
- check_proxy.py:136:32: E701 Multiple statements on one line (colon)
- check_proxy.py:136:47: E702 Multiple statements on one line (semicolon)
- check_proxy.py:141:5: E722 Do not use bare `except`
- check_proxy.py:142:28: RUF001 String contains ambiguous `,` (FULLWIDTH COMMA). Did you mean `,` (COMMA)?
- check_proxy.py:142:85: RUF001 String contains ambiguous `,` (FULLWIDTH COMMA). Did you mean `,` (COMMA)?
- check_proxy.py:144:16: RUF001 String contains ambiguous `,` (FULLWIDTH COMMA). Did you mean `,` (COMMA)?
... 356 additional changes omitted for rule RUF001
- check_proxy.py:148:32: E701 Multiple statements on one line (colon)
- check_proxy.py:148:47: E702 Multiple statements on one line (semicolon)
- check_proxy.py:157:30: RUF002 Docstring contains ambiguous `,` (FULLWIDTH COMMA). Did you mean `,` (COMMA)?
... 142 additional changes omitted for rule RUF002
- check_proxy.py:161:32: UP015 [*] Unnecessary mode argument
- check_proxy.py:163:5: E722 Do not use bare `except`
- check_proxy.py:179:9: I001 [*] Import block is un-sorted or un-formatted
- check_proxy.py:183:12: E701 Multiple statements on one line (colon)
- check_proxy.py:184:15: E701 Multiple statements on one line (colon)
- check_proxy.py:184:9: E722 Do not use bare `except`
- check_proxy.py:191:32: UP015 [*] Unnecessary mode argument
- check_proxy.py:194:54: E226 [*] Missing whitespace around arithmetic operator
- check_proxy.py:1:1: I001 [*] Import block is un-sorted or un-formatted
- check_proxy.py:1:1: I002 [*] Missing required import: `from __future__ import annotations`
- check_proxy.py:203:17: E722 Do not use bare `except`
- check_proxy.py:214:5: E722 Do not use bare `except`
- check_proxy.py:221:1: E302 [*] Expected 2 blank lines, found 1
- check_proxy.py:226:5: I001 [*] Import block is un-sorted or un-formatted
- check_proxy.py:234:1: E302 [*] Expected 2 blank lines, found 1
- check_proxy.py:249:52: E701 Multiple statements on one line (colon)
- check_proxy.py:39:5: E722 Do not use bare `except`
... 36 additional changes omitted for rule E722
... 2579 additional changes omitted for project

bokeh/bokeh (+449 -0 violations, +0 -0 fixes)

ruff check --no-cache --exit-zero --ignore RUF9 --no-fix --output-format concise --preview --select ALL

+ examples/advanced/extensions/widget.py:81:5: E128 Continuation line under-indented for visual indent
+ examples/basic/axes/twin_axes.py:34:5: E128 Continuation line under-indented for visual indent
+ examples/basic/axes/twin_axes.py:35:5: E128 Continuation line under-indented for visual indent
+ examples/basic/axes/twin_axes.py:36:1: E124 Closing bracket does not match visual indentation
+ examples/basic/data/transform_customjs.py:36:9: E128 Continuation line under-indented for visual indent
+ examples/basic/data/transform_customjs.py:38:9: E128 Continuation line under-indented for visual indent
+ examples/basic/layouts/sizing_mode_multiple.py:41:2: E128 Continuation line under-indented for visual indent
... 290 additional changes omitted for rule E128
+ examples/interaction/js_callbacks/color_sliders.py:27:9: E127 Continuation line over-indented for visual indent
+ examples/interaction/js_callbacks/customjs_for_selection.py:41:1: E124 Closing bracket does not match visual indentation
+ examples/interaction/js_callbacks/open_url.py:11:5: E123 Closing bracket does not match indentation of opening bracket's line
... 439 additional changes omitted for project

fronzbot/blinkpy (+3 -0 violations, +0 -0 fixes)

ruff check --no-cache --exit-zero --ignore RUF9 --no-fix --output-format concise --preview

+ blinksync/forms.py:20:25: E128 Continuation line under-indented for visual indent
+ blinksync/forms.py:21:25: E128 Continuation line under-indented for visual indent
+ blinksync/forms.py:22:25: E124 Closing bracket does not match visual indentation

latchbio/latch (+2 -0 violations, +0 -0 fixes)

ruff check --no-cache --exit-zero --ignore RUF9 --no-fix --output-format concise --preview

+ src/latch_cli/snakemake/serialize.py:86:17: E126 Continuation line over-indented for hanging indent
+ src/latch_cli/snakemake/serialize.py:87:13: E121 Continuation line under-indented for hanging indent

milvus-io/pymilvus (+70 -86 violations, +0 -0 fixes)

ruff check --no-cache --exit-zero --ignore RUF9 --no-fix --output-format concise --preview

+ examples/alter.py:22:9: E126 Continuation line over-indented for hanging indent
+ examples/bm25.py:48:9: E126 Continuation line over-indented for hanging indent
+ examples/bulk_import/example_bulkinsert_csv.py:137:42: E128 Continuation line under-indented for visual indent
+ examples/bulk_import/example_bulkinsert_csv.py:138:42: E128 Continuation line under-indented for visual indent
+ examples/bulk_import/example_bulkinsert_csv.py:139:42: E128 Continuation line under-indented for visual indent
+ examples/bulk_import/example_bulkinsert_json.py:180:42: E128 Continuation line under-indented for visual indent
+ examples/bulk_import/example_bulkinsert_json.py:181:42: E128 Continuation line under-indented for visual indent
+ examples/bulk_import/example_bulkinsert_withfunction.py:69:9: E122 Continuation line missing indentation or outdented
+ examples/bulk_import/example_bulkinsert_withfunction.py:70:9: E122 Continuation line missing indentation or outdented
+ examples/cert/example_tls1.py:42:29: E128 Continuation line under-indented for visual indent
... 12 additional changes omitted for rule E128
... 146 additional changes omitted for project

pandas-dev/pandas (+3 -0 violations, +0 -0 fixes)

ruff check --no-cache --exit-zero --ignore RUF9 --no-fix --output-format concise --preview

+ scripts/validate_rst_title_capitalization.py:269:21: E122 Continuation line missing indentation or outdented
+ scripts/validate_unwanted_patterns.py:323:13: E128 Continuation line under-indented for visual indent
+ scripts/validate_unwanted_patterns.py:325:13: E128 Continuation line under-indented for visual indent

rotki/rotki (+138 -0 violations, +0 -0 fixes)

ruff check --no-cache --exit-zero --ignore RUF9 --no-fix --output-format concise --preview

+ rotkehlchen/accounting/constants.py:6:58: E126 Continuation line over-indented for hanging indent
+ rotkehlchen/api/rest.py:645:13: E122 Continuation line missing indentation or outdented
+ rotkehlchen/api/v1/schemas.py:1220:9: E125 Continuation line with same indent as next logical line
+ rotkehlchen/api/v1/schemas.py:704:17: E126 Continuation line over-indented for hanging indent
+ rotkehlchen/api/v1/schemas.py:706:13: E121 Continuation line under-indented for hanging indent
+ rotkehlchen/api/v1/schemas.py:707:9: E123 Closing bracket does not match indentation of opening bracket's line
+ rotkehlchen/balances/historical.py:228:18: E121 Continuation line under-indented for hanging indent
+ rotkehlchen/chain/arbitrum_one/modules/umami/balances.py:138:28: E121 Continuation line under-indented for hanging indent
+ rotkehlchen/chain/arbitrum_one/modules/umami/decoder.py:232:21: E126 Continuation line over-indented for hanging indent
+ rotkehlchen/chain/base/modules/echo/decoder.py:53:17: E126 Continuation line over-indented for hanging indent
... 128 additional changes omitted for project

... Truncated remaining completed project reports due to GitHub comment length restrictions

Changes by rule (104 rules affected)

code total + violation - violation + fix - fix
E128 390 390 0 0 0
RUF001 361 0 361 0 0
RUF003 224 0 224 0 0
E302 220 0 220 0 0
I001 182 0 182 0 0
F401 182 0 182 0 0
E251 162 0 162 0 0
E231 161 0 161 0 0
RUF002 147 0 147 0 0
E701 132 0 132 0 0
E261 122 0 122 0 0
E127 116 116 0 0 0
E226 97 0 97 0 0
E126 82 82 0 0 0
E121 77 77 0 0 0
E131 63 63 0 0 0
I002 63 0 63 0 0
E252 60 0 60 0 0
E122 52 52 0 0 0
F541 49 0 49 0 0
C416 49 0 49 0 0
E722 41 0 41 0 0
F841 40 0 40 0 0
E402 40 0 40 0 0
E303 38 0 38 0 0
E401 36 0 36 0 0
T201 35 0 35 0 0
E125 31 31 0 0 0
B007 28 0 28 0 0
E123 27 27 0 0 0
E124 24 24 0 0 0
E702 17 0 17 0 0
E306 16 0 16 0 0
B006 16 0 16 0 0
UP015 15 0 15 0 0
E305 15 0 15 0 0
RUF052 13 0 13 0 0
SIM102 12 0 12 0 0
E225 12 0 12 0 0
E262 12 0 12 0 0
UP039 11 0 11 0 0
B901 9 0 9 0 0
F811 8 0 8 0 0
UP032 8 0 8 0 0
SIM103 8 0 8 0 0
E265 7 0 7 0 0
PIE790 6 0 6 0 0
SIM108 6 0 6 0 0
E129 5 5 0 0 0
E211 5 0 5 0 0
B904 5 0 5 0 0
E222 5 0 5 0 0
E221 5 0 5 0 0
E116 5 0 5 0 0
E731 5 0 5 0 0
E201 5 0 5 0 0
E111 4 0 4 0 0
E272 4 0 4 0 0
E703 4 0 4 0 0
RUF013 4 0 4 0 0
PIE810 4 0 4 0 0
E301 3 0 3 0 0
E266 3 0 3 0 0
E228 3 0 3 0 0
RUF039 3 0 3 0 0
E741 3 0 3 0 0
SIM114 3 0 3 0 0
E117 3 0 3 0 0
SIM117 3 0 3 0 0
W292 3 0 3 0 0
INP001 3 0 3 0 0
SIM105 2 0 2 0 0
RUF027 2 0 2 0 0
RUF010 2 0 2 0 0
SIM118 2 0 2 0 0
C419 2 0 2 0 0
RUF005 2 0 2 0 0
UP028 2 0 2 0 0
RUF029 2 0 2 0 0
S311 2 0 2 0 0
ANN001 2 0 2 0 0
PLR2004 1 0 1 0 0
RUF015 1 0 1 0 0
SIM115 1 0 1 0 0
B005 1 0 1 0 0
SIM300 1 0 1 0 0
E502 1 0 1 0 0
SIM223 1 0 1 0 0
UP031 1 0 1 0 0
UP034 1 0 1 0 0
E202 1 0 1 0 0
PGH003 1 0 1 0 0
C414 1 0 1 0 0
C403 1 0 1 0 0
SIM201 1 0 1 0 0
UP009 1 0 1 0 0
UP025 1 0 1 0 0
C420 1 0 1 0 0
B020 1 0 1 0 0
C408 1 0 1 0 0
B028 1 0 1 0 0
W291 1 0 1 0 0
N816 1 0 1 0 0
RET504 1 0 1 0 0

Formatter (stable)

✅ ecosystem check detected no format changes.

Formatter (preview)

✅ ecosystem check detected no format changes.

@LordAro LordAro force-pushed the e12x-continued-indentation branch from bf09ebb to 8906f45 Compare October 1, 2024 17:50
@LordAro
Copy link
Author

LordAro commented Oct 1, 2024

+589 new violations is a bit nicer than +56549

@MichaReiser
Copy link
Member

There's some overlap with #11349 and we would have to look into the performance regressions.

@LordAro
Copy link
Author

LordAro commented Oct 7, 2024

That's what I get for not checking existing PRs. I'd imagine much of the performance regression is to do with the
logical_line.lines.locator.line_start on every token. Not sure how to better do it, I'll see if I can glean anything from the previous PRs (which for my own reference includes #8557)

@LordAro LordAro force-pushed the e12x-continued-indentation branch from 8906f45 to 0be255e Compare October 31, 2024 08:20
@LordAro
Copy link
Author

LordAro commented Oct 31, 2024

Quick push to show I'm still working on it - I know the tests are still broken, but I'm happy that E12{1,2,3} are correct now. (But probably still slow)

@LordAro LordAro force-pushed the e12x-continued-indentation branch 4 times, most recently from c0b57e9 to fd85355 Compare November 1, 2024 23:54
@LordAro LordAro changed the title Add support for E12x continued indentation pycodestyle linter errors [pycodestyle] Implement continued indentation related rules (E12x) Nov 5, 2024
@LordAro LordAro force-pushed the e12x-continued-indentation branch from fd85355 to f990a5a Compare November 12, 2024 22:50
@LordAro LordAro marked this pull request as ready for review November 12, 2024 22:50
@LordAro
Copy link
Author

LordAro commented Nov 12, 2024

I'm calling this ready. The output matches pycodestyle for all added warnings (though there's no testing for E133 here or in pycodestyle)

There's still a couple of TODOs in the code, and honestly the code itself is a big blob. Hopefully someone has some better ideas for how to achieve things :)

@LordAro LordAro force-pushed the e12x-continued-indentation branch 2 times, most recently from 5f483eb to 8bd05ab Compare November 12, 2024 23:21
@LordAro LordAro force-pushed the e12x-continued-indentation branch 2 times, most recently from 1833555 to 539e21d Compare November 21, 2024 09:37
@LordAro
Copy link
Author

LordAro commented Nov 21, 2024

Guess ruff is getting close to the limit of the number of rules - I've bumped RULESET_SIZE

@LordAro LordAro force-pushed the e12x-continued-indentation branch from 539e21d to 42126d3 Compare November 29, 2024 10:37
@LordAro
Copy link
Author

LordAro commented Nov 29, 2024

@MichaReiser et al, do let me know if there's anything more I can do

(sorry, I know how annoying these sort of pings can be!)

@LordAro LordAro force-pushed the e12x-continued-indentation branch from 42126d3 to 991b923 Compare November 29, 2024 11:04
Comment on lines +544 to +547
// record initial indent
// TODO: A better way of doing this?
relative_line_indent[line_no] =
isize::try_from(token_indent).unwrap() - isize::try_from(indent_level).unwrap();
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

flagging this ugly line for review

Comment on lines +701 to +710
} else if tok.kind() == TokenKind::Colon
&& logical_line
.text_after(tok)
.chars()
.all(char::is_whitespace)
{
// If there's only whitespace following this token, it can't be a dictionary construction
// TODO: Is this always correct?
open_lines[bracket_depth].push(line_no);
}
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

flagging this to see if there's a better way of doing it (and indeed if it's logically correct)

@LordAro LordAro force-pushed the e12x-continued-indentation branch from 991b923 to 5492cde Compare November 29, 2024 11:06
@LordAro LordAro force-pushed the e12x-continued-indentation branch 2 times, most recently from 077873f to 732dae3 Compare December 16, 2024 08:35
@LordAro LordAro force-pushed the e12x-continued-indentation branch from 732dae3 to f2c0bbe Compare February 8, 2025 16:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants