-
Notifications
You must be signed in to change notification settings - Fork 1
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Start Configuration of Python Gazelle (#26)
Changes: - Establish a mechanism for defining external python requirements via a `requirements.in`, generating updates to a lock file with `bazel run //:requirements.update` - Configure gazelle python and protobufs - Configures via gazelle directives to use a file based approach, stick with the BUILD configuration, disable go, default visibility to private and resolve py imports - Update dev_setup to install openjdk (java), tree, ranger, ag New Commands: ``` bazel run //:requirements.update bazel run //:gazelle_python_manifest.update bazel run //:gazelle ``` New Tests: ``` //:requirements_test //::gazelle_python_manifest.test ``` Notes: - One of the recurring issues I have every time I deal with updating WORKSPACE is slight incompatibilities with various repos that result in incomprehensible output errors. - stackb/rules_proto is one of the only things that seems to support what I'm looking for, and while it is getting steady contributions hasn't pubilshed a release in about a year. https://github.com/rules-proto-grpc/rules_proto_grpc. Additionally, it could be handy to look into what it would take to make my own "shitty" version of these rules to learn more about what's going on in gazelle's / bazel's internals - bzlmod migration didn't go well for me, most tutorials out there still use WORKSPACE, might be good to look into in the future / figure out how to migrate piecemeal, but bypassed this and 7.0.1 bazel upgrade for now Related Issues: - Noting the issue with requiring a gazelle directive for proto libraries: bazelbuild/rules_python#1703 - bazelbuild/rules_python#1664 fixes the reason I had to "disable" my py_binary, wait for that to release and then update. I could probably patch this in, but don't quite know how 🤷 - Sounds like there's some talk around gazelle c++ bazel-contrib/bazel-gazelle#910 - I'm helping ;) bazelbuild/rules_python#1712 (doc update) References: - https://github.com/bazelbuild/bazel-gazelle - https://github.com/stackb/rules_proto - https://github.com/bazelbuild/rules_python/tree/main/gazelle Future Things to Look Into: - Setup a gazelle test that checks output of `//:gazelle -- -mode diff` is zero - Configure rust with gazelle: https://github.com/Calsign/gazelle_rust - A really cool / general parser / generator (what github uses for "semantic" tool): https://tree-sitter.github.io/tree-sitter/ Could use to make small code generator / modification? Or maybe more general utilities? - General compile command completion: https://github.com/hedronvision/bazel-compile-commands-extractor - Maybe play with https://github.com/rules-proto-grpc/rules_proto_grpc as an alternative to stackb - If I can't use gazelle for some things, mess around with https://github.com/bazelbuild/buildtools/tree/master/buildozer
- Loading branch information
1 parent
272da4d
commit d21e33d
Showing
17 changed files
with
494 additions
and
84 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,105 @@ | ||
| # Load various rules so that we can have bazel download | ||
| # various rulesets and dependencies. | ||
| # The `load` statement imports the symbol for the rule, in the defined | ||
| # ruleset. When the symbol is loaded you can use the rule. | ||
| load("@bazel_gazelle//:def.bzl", "gazelle", "gazelle_binary") | ||
| load("@pip//:requirements.bzl", "all_whl_requirements") | ||
| load("@rules_python//python:pip.bzl", "compile_pip_requirements") | ||
| load("@rules_python_gazelle_plugin//manifest:defs.bzl", "gazelle_python_manifest") | ||
| load("@rules_python_gazelle_plugin//modules_mapping:def.bzl", "modules_mapping") | ||
|
|
||
| compile_pip_requirements( | ||
| name = "requirements", | ||
| requirements_in = "requirements.in", | ||
| requirements_txt = "requirements_lock.txt", | ||
| ) | ||
|
|
||
| # This repository rule fetches the metadata for python packages we | ||
| # depend on. That data is required for the gazelle_python_manifest | ||
| # rule to update our manifest file. | ||
| # To see what this rule does, try `bazel run @modules_map//:print` | ||
| modules_mapping( | ||
| name = "modules_map", | ||
| exclude_patterns = [ | ||
| "^_|(\\._)+", # This is the default. | ||
| "(\\.tests)+", # Add a custom one to get rid of the psutil tests. | ||
| ], | ||
| wheels = all_whl_requirements, | ||
| ) | ||
|
|
||
| # Gazelle python extension needs a manifest file mapping from | ||
| # an import to the installed package that provides it. | ||
| # This macro produces two targets: | ||
| # - //:gazelle_python_manifest.update can be used with `bazel run` | ||
| # to recalculate the manifest | ||
| # - //:gazelle_python_manifest.test is a test target ensuring that | ||
| # the manifest doesn't need to be updated | ||
| gazelle_python_manifest( | ||
| name = "gazelle_python_manifest", | ||
| modules_mapping = ":modules_map", | ||
| pip_repository_name = "pip", | ||
| # NOTE: We can pass a list just like in `bzlmod_build_file_generation` example | ||
| # but we keep a single target here for regression testing. | ||
| requirements = "//:requirements_lock.txt", | ||
| ) | ||
|
|
||
| gazelle_binary( | ||
| name = "gazelle_bin", | ||
| languages = [ | ||
| "@bazel_gazelle//language/bazel/visibility", # bazel visibility rules | ||
| "@bazel_gazelle//language/go", # Built-in rule from gazelle for Golang | ||
| "@bazel_gazelle//language/proto", # Built-in rule from gazelle for Protos | ||
| # Any languages that depend on the proto plugin must come after it | ||
| "@rules_python_gazelle_plugin//python:python", # Use gazelle from rules_python | ||
| "@build_stack_rules_proto//language/protobuf", # Protobuf language generation | ||
| # TODO: Add buf suppport | ||
| # "@rules_buf//gazelle/buf:buf", # Generates buf lint and buf breaking detection rules | ||
| ], | ||
| ) | ||
|
|
||
| # Our gazelle target points to the python gazelle binary. | ||
| # This is the simple case where we only need one language supported. | ||
| # If you also had proto, go, or other gazelle-supported languages, | ||
| # you would also need a gazelle_binary rule. | ||
| # See https://github.com/bazelbuild/bazel-gazelle/blob/master/extend.rst#example | ||
| gazelle( | ||
| name = "gazelle", | ||
| args = [ | ||
| "-proto_configs=gazelle_proto_config.yaml", | ||
| ], | ||
| gazelle = ":gazelle_bin", | ||
| ) | ||
|
|
||
| # https://github.com/bazelbuild/bazel-gazelle#directives | ||
| # https://github.com/bazelbuild/rules_python/blob/main/gazelle/README.md#directives | ||
|
|
||
| # Make a py_library per python file | ||
| # gazelle:python_generation_mode file | ||
|
|
||
| # Disable BUILD.bazel files | ||
| # gazelle:build_file_name BUILD | ||
|
|
||
| # Exclude these folders from gazelle generation | ||
| # gazelle:exclude venv | ||
|
|
||
| # Don't use go | ||
| # gazelle:go_generate_proto false | ||
|
|
||
| # Generate 1 proto rule per file | ||
| # gazelle:proto file | ||
|
|
||
| # Set default BUILD rule visibility | ||
| # gazelle:default_visibility //visibility:private | ||
|
|
||
| # Configure aspect_rules_py to be used for py_test, py_binary, py_library | ||
| # gazelle:map_kind py_binary py_binary @aspect_rules_py//py:defs.bzl | ||
| # gazelle:map_kind py_library py_library @aspect_rules_py//py:defs.bzl | ||
| # gazelle:map_kind py_test py_test @aspect_rules_py//py:defs.bzl | ||
|
|
||
| # Tell gazelle where to find imports | ||
| # gazelle:resolve py google.protobuf.message @com_google_protobuf//:protobuf_python | ||
|
|
||
| # TODO: Figure out a way to not need these | ||
| # gazelle:resolve py examples.basic.hello_pb2 //examples/basic:hello_py_library | ||
|
|
||
| package(default_visibility = ["//visibility:private"]) |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.