From a92bc76403ff94d20e6f0a99b13cccf3ea8ad78e Mon Sep 17 00:00:00 2001 From: Mike Bland Date: Sun, 16 Feb 2025 13:00:11 -0500 Subject: [PATCH] Update docs for version 7.0.0, remove old APIs Describes the new `scala_toolchains()` API, breaking changes, Bazel compatibility matrix, and breaking of the `io_bazel_rules_scala` repo name dependency. Part of #1482, #1647, #1652, and #1699, it also removes obsolete `WORKSPACE` APIs in favor of `scala_toolchains()`, et. al. Much of the README text regarding `WORKSPACE` configuration, Bazel compatibility, and breaking changes comes from: - https://github.com/bazelbuild/rules_scala/issues/1482#issuecomment-2605683509 The "Breaking changes in `rules_scala` 7.x" section of `README.md` describes the removed `WORKSPACE` macros, and provides guidance for adopting the `scala_toolchains()` API. Also includes information about the upcoming Bzlmod implementation (#1482) and the Bazel 8 compatibility changes for `rules_scala` 8.x (#1652). Contains a light editing pass over almost every Markdown file to resolve `markdownlint` warnings when editing in VSCode. Also added `.markdownlint.json` to silence rule `MD033/no-inline-html`, since several docs contain `
` and/or `` elements. - https://github.com/DavidAnson/vscode-markdownlint?tab=readme-ov-file#markdownlintconfig Performed other opportunistic grammar edits and added new information, including: - `docs/coverage.md`: Describes how to determine the default JaCoCo version used by `rules_java` - `scripts/README.md`: Updates `create_repository.py` docs extensively, and adds a section for `sync_bazelversion.sh` --- Since the next major release is imminent, ensuring the documentation accurately reflects all the changes since v6.6.0 has become an urgent priority. Rather than leave the old `WORKSPACE` APIs in place, which could lead to surprising failures, this change removes all of them. This also changes some code that still depended on some of these obsolete macros, including `scala_toolchains()`. Since all the toolchainization changes led to the entire project already using `scala_toolchains()` and `scala_register_toolchains()` exclusively, this proved a low risk refactoring. With some Bzlmod and Bazel 8 information already in place, very little will need to change when these implementations land. The commits that contain those implementations will also include their relevant documentation updates. --- .markdownlint.json | 5 + README.md | 632 ++++++++++++++---- docs/coverage.md | 94 ++- docs/cross-compilation.md | 96 ++- docs/customizable_phase.md | 126 ++-- docs/dependency-tracking.md | 125 ++-- docs/phase_scalafmt.md | 60 +- docs/publish_to_maven.md | 57 +- docs/scala_binary.md | 2 +- docs/scala_doc.md | 10 +- docs/scala_import.md | 30 +- docs/scala_library.md | 2 +- docs/scala_library_suite.md | 2 +- docs/scala_macro_library.md | 2 +- docs/scala_proto_library.md | 68 +- docs/scala_repl.md | 7 +- docs/scala_test.md | 4 +- docs/scala_test_suite.md | 2 +- docs/scala_toolchain.md | 37 +- docs/testing.md | 92 ++- docs/thrift_library.md | 6 +- docs/toolchain_development.md | 176 ++--- examples/semanticdb/README.md | 8 +- jmh/toolchain/toolchain.bzl | 14 - junit/junit.bzl | 15 - manual_test/README.md | 4 +- scala/private/macros/scala_repositories.bzl | 57 +- scala/scala.bzl | 9 - scala/scalafmt/scalafmt_repositories.bzl | 33 +- scala/toolchains.bzl | 47 +- scala_proto/default/repositories.bzl | 20 - scala_proto/scala_proto.bzl | 4 - scala_proto/toolchains.bzl | 8 - scalatest/scalatest.bzl | 18 - scripts/README.md | 130 ++-- specs2/specs2.bzl | 19 - specs2/specs2_junit.bzl | 25 +- test/toolchains/jdk.bzl | 3 - .../scrooge_repositories.bzl | 14 +- testing/junit.bzl | 12 - testing/scalatest.bzl | 12 - testing/specs2_junit.bzl | 12 - third_party/README.md | 7 +- twitter_scrooge/toolchain/toolchain.bzl | 37 +- 44 files changed, 1222 insertions(+), 921 deletions(-) create mode 100644 .markdownlint.json delete mode 100644 testing/junit.bzl delete mode 100644 testing/scalatest.bzl delete mode 100644 testing/specs2_junit.bzl diff --git a/.markdownlint.json b/.markdownlint.json new file mode 100644 index 000000000..6ab7bd17c --- /dev/null +++ b/.markdownlint.json @@ -0,0 +1,5 @@ +{ + "default": true, + "MD013": false, + "MD033": false +} diff --git a/README.md b/README.md index f918748d9..8ed513cf2 100644 --- a/README.md +++ b/README.md @@ -1,10 +1,12 @@ # Scala Rules for Bazel -[![Build status](https://badge.buildkite.com/90ce5244556df74db805a3c24a703fb87458396f9e1ddd687e.svg?branch=master)](https://buildkite.com/bazel/scala-rules-scala-postsubmit) + +[![Build status](https://badge.buildkite.com/90ce5244556df74db805a3c24a703fb87458396f9e1ddd687e.svg?branch=master)](https://buildkite.com/bazel/scala-rules-scala-postsubmit) ## Where to get help - - [#scala @ Bazel Slack](https://bazelbuild.slack.com/archives/CDCKJ2KFZ) - - [Google group](https://groups.google.com/u/1/g/bazel-scala) - - [Gitter chat](https://gitter.im/bazelbuild_rules_scala/Lobby) + +- [#scala @ Bazel Slack](https://bazelbuild.slack.com/archives/CDCKJ2KFZ) +- [Google group](https://groups.google.com/u/1/g/bazel-scala) +- [Gitter chat](https://gitter.im/bazelbuild_rules_scala/Lobby) ## Overview @@ -15,93 +17,160 @@ This project defines core build rules for [Scala](https://www.scala-lang.org/) t ## Rules -* [scala_library](docs/scala_library.md) -* [scala_macro_library](docs/scala_macro_library.md) -* [scala_binary](docs/scala_binary.md) -* [scala_test](docs/scala_test.md) -* [scala_repl](docs/scala_repl.md) -* [scala_library_suite](docs/scala_library_suite.md) -* [scala_test_suite](docs/scala_test_suite.md) -* [thrift_library](docs/thrift_library.md) -* [scala_proto_library](docs/scala_proto_library.md) -* [scala_toolchain](docs/scala_toolchain.md) -* [scala_import](docs/scala_import.md) -* [scala_doc](docs/scala_doc.md) +- [scala_library](docs/scala_library.md) +- [scala_macro_library](docs/scala_macro_library.md) +- [scala_binary](docs/scala_binary.md) +- [scala_test](docs/scala_test.md) +- [scala_repl](docs/scala_repl.md) +- [scala_library_suite](docs/scala_library_suite.md) +- [scala_test_suite](docs/scala_test_suite.md) +- [thrift_library](docs/thrift_library.md) +- [scala_proto_library](docs/scala_proto_library.md) +- [scala_toolchain](docs/scala_toolchain.md) +- [scala_import](docs/scala_import.md) +- [scala_doc](docs/scala_doc.md) ## Getting started -1. [Install Bazel](https://docs.bazel.build/versions/master/install.html), -see the [compatibility table](#bazel-compatible-versions). -2. Add the following to your `WORKSPACE` file and update versions with their sha256s if needed: +[Install Bazel][], preferably using the [Bazelisk][] wrapper. See the +[compatbile Bazel versions](#compatible-bazel-versions) section to select a suitable +Bazel version. + +[Install Bazel]: https://docs.bazel.build/versions/master/install.html +[Bazelisk]: https://docs.bazel.build/versions/master/install.html + +Add the following stanza to your `WORKSPACE` file and update versions with their +sha256s if needed. This stanza is designed to ensure that users pick up the +correct order of dependencies for `rules_scala`. If you want to override any of +the following dependency versions, make sure to `load()` them before calling +`rules_scala_dependencies()`. + +As of version 7.0.0, __`rules_scala` no longer requires the +`io_bazel_rules_scala` repository name__ unless your `BUILD` files or those of +your dependencies require it (#1696). In that case, the old name will still work +when using `http_archive`. -```starlark +```py # WORKSPACE load("@bazel_tools//tools/build_defs/repo:http.bzl", "http_archive") +# See https://github.com/bazelbuild/rules_scala/releases for up to date version +# information, including `` values. http_archive( - name = "bazel_skylib", - sha256 = "b8a1527901774180afc798aeb28c4634bdccf19c4d98e7bdd1ce79d1fe9aaad7", - urls = [ - "https://mirror.bazel.build/github.com/bazelbuild/bazel-skylib/releases/download/1.4.1/bazel-skylib-1.4.1.tar.gz", - "https://github.com/bazelbuild/bazel-skylib/releases/download/1.4.1/bazel-skylib-1.4.1.tar.gz", - ], + name = "rules_scala", + sha256 = "", + strip_prefix = "rules_scala-7.0.0", + url = "https://github.com/bazelbuild/rules_scala/releases/download/7.0.0/rules_scala-7.0.0.tar.gz", ) -# See https://github.com/bazelbuild/rules_scala/releases for up to date version information. +load("@rules_scala//scala:deps.bzl", "rules_scala_dependencies") + +rules_scala_dependencies() + +load("@rules_java//java:rules_java_deps.bzl", "rules_java_dependencies") + +rules_java_dependencies() + +load("@bazel_skylib//:workspace.bzl", "bazel_skylib_workspace") + +bazel_skylib_workspace() + +# If you need a specific `rules_python` version, specify it here. +# Otherwise you may get the version defined in the `com_google_protobuf` repo. +# We use 0.38.0 to maintain compatibility with Bazel 6.5.0; this will change in +# the next release. http_archive( - name = "io_bazel_rules_scala", - sha256 = "71324bef9bc5a885097e2960d5b8effed63399b55572219919d25f43f468c716", - strip_prefix = "rules_scala-6.2.1", - url = "https://github.com/bazelbuild/rules_scala/releases/download/v6.2.1/rules_scala-v6.2.1.tar.gz", + name = "rules_python", + sha256 = "ca2671529884e3ecb5b79d6a5608c7373a82078c3553b1fa53206e6b9dddab34", + strip_prefix = "rules_python-0.38.0", + url = "https://github.com/bazelbuild/rules_python/releases/download/0.38.0/rules_python-0.38.0.tar.gz", ) -load("@io_bazel_rules_scala//:scala_config.bzl", "scala_config") -# Stores Scala version and other configuration -# 2.12 is a default version, other versions can be use by passing them explicitly: -# scala_config(scala_version = "2.11.12") -# Scala 3 requires extras... -# 3.2 should be supported on master. Please note that Scala artifacts for version (3.2.2) are not defined in -# Rules Scala, they need to be provided by your WORKSPACE. You can use external loader like -# https://github.com/bazelbuild/rules_jvm_external -scala_config() +load("@rules_python//python:repositories.bzl", "py_repositories") -load("@io_bazel_rules_scala//scala:scala.bzl", "rules_scala_setup", "rules_scala_toolchain_deps_repositories") +py_repositories() -# loads other rules Rules Scala depends on -rules_scala_setup() +# Note that `rules_java` suggests loading `protobuf_deps()` after +# `rules_java_dependencies` and before `rules_java_toolchains()`: +# - https://github.com/bazelbuild/rules_java/releases/tag/8.9.0 +load("@com_google_protobuf//:protobuf_deps.bzl", "protobuf_deps") -# Loads Maven deps like Scala compiler and standard libs. On production projects you should consider -# defining a custom deps toolchains to use your project libs instead -rules_scala_toolchain_deps_repositories(fetch_sources = True) +protobuf_deps() + +load("@rules_java//java:repositories.bzl", "rules_java_toolchains") + +rules_java_toolchains() + +load("@rules_proto//proto:repositories.bzl", "rules_proto_dependencies") -load("@rules_proto//proto:repositories.bzl", "rules_proto_dependencies", "rules_proto_toolchains") rules_proto_dependencies() + +load("@rules_proto//proto:setup.bzl", "rules_proto_setup") + +rules_proto_setup() + +load("@rules_proto//proto:toolchains.bzl", "rules_proto_toolchains") + rules_proto_toolchains() -load("@io_bazel_rules_scala//scala:toolchains.bzl", "scala_register_toolchains") -scala_register_toolchains() +load("@rules_scala//:scala_config.bzl", "scala_config") + +# Stores the selected Scala version and other configuration parameters. +# +# 2.12 is the default version. Use other versions by passing them explicitly: +# +# scala_config(scala_version = "2.13.15") +# +# You may define your own custom toolchain using Maven artifact dependencies +# configured by your `WORKSPACE` file, imported using external loader like +# https://github.com/bazelbuild/rules_jvm_external. +scala_config() -# optional: setup ScalaTest toolchain and dependencies -load("@io_bazel_rules_scala//testing:scalatest.bzl", "scalatest_repositories", "scalatest_toolchain") -scalatest_repositories() -scalatest_toolchain() +load( + "@rules_scala//scala:toolchains.bzl", + "scala_register_toolchains", + "scala_toolchains", +) + +# Defines a default toolchain repo for the configured Scala version that +# loads Maven deps like the Scala compiler and standard libs. On production +# projects, you may consider defining a custom toolchains to use your project's +# required dependencies instead. +# +# Optional builtin rules_scala toolchains may be configured by setting +# parameters as documented in the `scala_toolchains()` docstring. +scala_toolchains( + scalatest = True, +) + +scala_register_toolchains() ``` This will load the `rules_scala` repository at the commit sha `rules_scala_version` into your Bazel project and register a [scala_toolchain](docs/scala_toolchain.md) at the default Scala version (2.12.20) Then in your BUILD file just add the following so the rules will be available: -```starlark -load("@io_bazel_rules_scala//scala:scala.bzl", "scala_library", "scala_binary", "scala_test") + +```py +load( + "@rules_scala//scala:scala.bzl", + "scala_binary", + "scala_library", + "scala_test", +) ``` + You may wish to have these rules loaded by default using bazel's prelude. You can add the above to the file `tools/build_rules/prelude_bazel` in your repo (don't forget to have a, possibly empty, BUILD file there) and then it will be automatically prepended to every BUILD file in the workspace. To run with a persistent worker (much faster), you need to add -``` + +```txt build --strategy=Scalac=worker build --worker_sandboxing ``` -to your command line, or to enable by default for building/testing add it to your .bazelrc. + +to your command line, or to enable by default for building/testing add it to +your `.bazelrc`. ## Coverage support @@ -109,7 +178,7 @@ It will produce several .dat files with results for your targets. You can also add more options to receive a combined coverage report: -``` +```txt bazel coverage \ --combined_report=lcov \ --coverage_report_generator="@bazel_tools//tools/test/CoverageOutputGenerator/java/com/google/devtools/coverageoutputgenerator:Main" \ @@ -118,11 +187,13 @@ bazel coverage \ This should produce a single `bazel-out/_coverage/_coverage_report.dat` from all coverage files that are generated. -You can extract information from your coverage reports with `lcov`: +You can extract information from your coverage reports with +[`lcov`](https://github.com/linux-test-project/lcov): -``` +```txt # For a summary: lcov --summary your-coverage-report.dat + # For details: lcov --list your-coverage-report.dat ``` @@ -133,71 +204,79 @@ Coverage support has been only tested with [ScalaTest](http://www.scalatest.org/ Please check [coverage.md](docs/coverage.md) for more details on coverage support. -## Selecting Scala version +## Selecting the Scala version -### With toolchains +### With builtin toolchains -Rules scala supports the last two released minor versions for each of Scala 2.11, 2.12, 2.13. +`rules_scala` supports the last two released minor versions for each of Scala 2.11, 2.12, 2.13. Previous minor versions may work but are supported only on a best effort basis. -To configure Scala version you must call `scala_config(scala_version = "2.xx.xx")` and configure -dependencies by declaring [scala_toolchain](docs/scala_toolchain.md). -For a quick start you can use `scala_repositories()` and `scala_register_toolchains()`, which have -dependency providers configured for `2.11.12`, `2.12.20` and `2.13.15` versions. - +First select the default Scala version by calling `scala_config(scala_version = "2.xx.xx")` and configure +dependencies by calling [scala_toolchains()](docs/scala_toolchain.md) and `scala_register_toolchains()`: -```starlark +```py # WORKSPACE -load("@io_bazel_rules_scala//:scala_config.bzl", "scala_config") +load("@rules_scala//:scala_config.bzl", "scala_config") scala_config(scala_version = "2.13.15") -load("@rules_proto//proto:repositories.bzl", "rules_proto_dependencies", "rules_proto_toolchains") -rules_proto_dependencies() -rules_proto_toolchains() +load( + "@rules_scala//scala:toolchains.bzl", + "scala_register_toolchains", + "scala_toolchains", +) + +scala_toolchains() -load("@io_bazel_rules_scala//scala:toolchains.bzl", "scala_register_toolchains") scala_register_toolchains() ``` +### With custom toolchains + +You can define your own custom [scala_toolchain](docs/scala_toolchain.md) by +calling `setup_scala_toolchain()` with dependencies that you specify. + Note: Toolchains are a more flexible way to configure dependencies, so you should prefer that way. Please also note, that the `overriden_artifacts` parameter is likely to be removed in the future. ### Multiple versions (cross-compilation) -Rules scala supports configuring multiple Scala versions and offers target-level control of which one to use. +`rules_scala` supports configuring multiple Scala versions and offers target-level control of which one to use. Please check [cross-compilation.md](docs/cross-compilation.md) for more details on cross-compilation support. -## Bazel compatible versions +## Compatible Bazel versions -| minimal bazel version | rules_scala gitsha | -|--------|------------------------------------------| -| 5.3.1 | HEAD | -| 4.1.0 | a40063ef97688f056824b22b9e49fae6efd1df0f | -| 3.5.0 | 0f55e9f8cff6494bbff7cd57048d732286a520f5 | -| 2.0.0 | 116709091e5e1aab3346184217b565f4cb7ba4eb | -| 1.1.0 | d681a952da74fc61a49fc3167b03548f42fc5dde | -| 0.28.1 | bd0c388125e12f4f173648fc4474f73160a5c628 | -| 0.23.x | ca655e5a330cbf1d66ce1d9baa63522752ec6011 | -| 0.22.x | f3113fb6e9e35cb8f441d2305542026d98afc0a2 | -| 0.16.x | f3113fb6e9e35cb8f441d2305542026d98afc0a2 | -| 0.15.x | 3b9ab9be31ac217d3337c709cb6bfeb89c8dcbb1 | -| 0.14.x | 3b9ab9be31ac217d3337c709cb6bfeb89c8dcbb1 | -| 0.13.x | 3c987b6ae8a453886759b132f1572c0efca2eca2 | +Bazel compatibility is tied directly to the versions of `protobuf` required by +Bazel and `rules_java`, and their compatibility with [scalabp/ScalaPB]( +https://github.com/scalapb/ScalaPB) Maven artifacts. For extensive analysis, +see #1647. -See [the configuration file](https://github.com/bazelbuild/rules_scala/blob/master/.bazelci/presubmit.yml) for the exact versions verified with the continuous-integration builds. +The Bazel versions and dependency versions in the table below represent the +maximum available at the time of writing. -## Breaking changes +- For the actual versions used by `rules_scala`, see + [scala/deps.bzl](scala/deps.bzl). -If you're upgrading to a version containing one of these commits, you may encounter a breaking change where there was previously undefined behavior. +- See [the configuration file][ci-config] for the exact Bazel versions verified + with the continuous-integration builds. -- [929b318](https://github.com/bazelbuild/rules_scala/commit/929b3180cc099ba76859f5e88710d2ac087fbfa3) on 2020-01-30: Fixed a bug in the JMH benchmark build that was allowing build failures to creep through. Previously you were able to build a benchmark suite with JMH build errors. Running the benchmark suite would only run the successfully-built benchmarks. +[ci-config]: ./.bazelci/presubmit.yml + +| Bazel/Dependency | `rules_scala` 7.x | `rules_scala` 8.x
(Coming soon! See #1482 and #1652.) | +| :-: | :-: | :-: | +| Bazel versions using Bzlmod
(Coming soon! See #1482.) | 6.5.0, 7.5.0 | 7.5.0, 8.2.0 | +| Bazel versions using `WORKSPACE` | 6.5.0, 7.5.0 | 6.5.0, 7.5.0, 8.2.0
(see the [notes on 6.5.0 compatibility](#6.5.0)) | +| `protobuf` | v21.7
(can support up to v25.5) | v29.3 | +| `abseil-cpp` | 20220623.1 | 20250127.0 | +| `rules_java` | 7.12.4 | 8.9.0 | +| `ScalaPB` | 0.11.17
(0.9.8 for Scala 2.11) | 1.0.0-alpha.1 | ## Usage with [bazel-deps](https://github.com/johnynek/bazel-deps) Bazel-deps allows you to generate bazel dependencies transitively for maven artifacts. Generally we don't want bazel-deps to fetch scala artifacts from maven but instead use the ones we get from calling `scala_repositories`. The artifacts can be overridden in the dependencies file used by bazel-deps: + ```yaml replacements: org.scala-lang: @@ -223,48 +302,342 @@ replacements: ``` ## Publishing to Maven repository -Go [here](docs/publish_to_maven.md) + +See [Publish your Scala Libraries to a Maven Repository]( +docs/publish_to_maven.md). ## Dependency Tracking -Rules Scala supports multiple dependency modes including strict and unused dependency tracking. See + +`rules_scala` supports multiple dependency modes including strict and unused dependency tracking. See [Dependency Tracking](docs/dependency-tracking.md) for more info. ## Advanced configurable rules + To make the ruleset more flexible and configurable, we introduce a phase architecture. By using a phase architecture, where rule implementations are defined as a list of phases that are executed sequentially, functionality can easily be added (or modified) by adding (or swapping) phases. Phases provide 3 major benefits: - - Consumers are able to configure the rules to their specific use cases by defining new phases within their workspace without impacting other consumers. - - Contributors are able to implement new functionalities by creating additional default phases. - - Phases give us more clear idea what steps are shared across rules. + +- Consumers are able to configure the rules to their specific use cases by + defining new phases within their workspace without impacting other consumers. +- Contributors are able to implement new functionalities by creating additional + default phases. +- Phases give us more clear idea what steps are shared across rules. See [Customizable Phase](docs/customizable_phase.md) for more info. ### Phase extensions - - [Scala Format](docs/phase_scalafmt.md) + +- [Scala Format](docs/phase_scalafmt.md) ## Building from source -Setup bazel: -We recommend using [Bazelisk](https://docs.bazel.build/versions/master/install.html) as your default bazel binary -Test & Build: +Build main sources only: + +```txt +bazel build //src/... +``` + +Run all smaller tests: + +```txt +bazel test //test/... ``` + +To run the full test suite: + +```txt bash test_all.sh ``` -You can also use: +Note: __`bazel test //...` will not work__ since we have a sub-folder on the +root folder which is meant to be used in a failure scenario in the integration +tests. Similarly, to only build you should use `bazel build //src/...` due to +that folder. + +## Breaking changes in `rules_scala` 7.x + +__The main objective of `rules_scala` 7.x is to enable existing users to migrate +to Bazel 7 and Bzlmod.__ To facilitate a gradual migration, it remains +compatible with both `WORKSPACE` and Bzlmod. However, it contains the following +breaking changes when upgrading from `rules_scala` 6.x. + +### New `scala_toolchains()` API for `WORKSPACE` + +`rules_scala` 7.0.0 replaces existing `*_repositories()` and `*_toolchains()` +macros with the combination of `rules_scala_dependencies()`, +`scala_toolchains()`, and `scala_register_toolchains()`. + +These macros no longer exist: + +- `jmh_repositories()` +- `junit_repositories()` +- `junit_toolchain()` +- `rules_scala_setup()` +- `rules_scala_toolchain_deps_repositories()` +- `scala_proto_default_repositories()` +- `scala_proto_register_enable_all_options_toolchain()` +- `scala_proto_register_toolchains()` +- `scala_proto_repositories()` +- `scala_register_unused_deps_toolchains()` +- `scala_repositories()` +- `scalafmt_default_config()` +- `scalafmt_repositories()` +- `scalatest_repositories()` +- `scalatest_toolchain()` +- `specs2_junit_repositories()` +- `specs2_repositories()` +- `specs2_version()` +- `twitter_scrooge()` + +Replace toolchain stanzas like the following: + +```py +load( + "@rules_scala//scala:scala.bzl", + "rules_scala_setup", + "rules_scala_toolchain_deps_repositories", +) + +rules_scala_setup() + +rules_scala_toolchain_deps_repositories(fetch_sources = True) + +# Other dependency declarations... + +load("@rules_scala//:scala_config.bzl", "scala_config") + +scala_config(scala_version = "2.13.15") + +load( + "//testing:scalatest.bzl", + "scalatest_repositories", + "scalatest_toolchain", +) + +scalatest_repositories() + +scalatest_toolchain() + +load( + "//scala/scalafmt:scalafmt_repositories.bzl", + "scalafmt_default_config", + "scalafmt_repositories", +) + +scalafmt_default_config() + +scalafmt_repositories() +``` + +with calls to `rules_scala_dependencies()`, `scala_toolchains()` (with the +appropriate parameters set), and `scala_register_toolchains()`: + +```py +load("@rules_scala//scala:deps.bzl", "rules_scala_dependencies") + +rules_scala_dependencies() + +# See the `WORKSPACE` stanza from the "Getting started" section above +# for other dependency declarations. + +load("@rules_scala//:scala_config.bzl", "scala_config") + +scala_config(scala_version = "2.13.15") + +load( + "@rules_scala//scala:toolchains.bzl", + "scala_register_toolchains", + "scala_toolchains", +) + +# Note that `rules_scala` toolchain repos are _always_ configured. +scala_toolchains( + scalafmt = True, + scalatest = True, +) + +scala_register_toolchains() ``` -bazel test //test/... + +See the [`scala_toolchains()` docstring](./scala/toolchains.bzl) for the +parameter list, which is almost in complete correspondence with parameters from +the previous macros. The `WORKSPACE` files in this repository also provide many +examples. + +### Replacing toolchain registration macros + +`MODULE.bazel` not only doesn't allow `load()` statements, and therefore macros, +but _all `register_toolchains()` calls_ __must__ _appear in `MODULE.bazel`_. + +Almost all `rules_scala` toolchains are automatically configured and registered by +`scala_toolchains()` and `scala_register_toolchains()`. There are two exceptions. + +The first exception is `scala_proto_register_enable_all_options_toolchain()`. +Replace this macro with: + +```py +scala_toolchains( + scala_proto = True, + scala_proto_enable_all_options = True, +) ``` -Note `bazel test //...` will not work since we have a sub-folder on the root folder which is meant to be used in a failure scenario in the integration tests. -Similarly to only build you should use `bazel build //src/...` due to that folder. -## Updates -This section contains a list of updates that might require action from the user. +The other exception is `scala_register_unused_deps_toolchains()`. Replace this +macro with: - - [`043ba58`](https://github.com/bazelbuild/rules_scala/commit/043ba58afaf90bf571911123d3353bdf20408a33): - Updates default scrooge version to `18.6.0` - - [`76d4ab9`](https://github.com/bazelbuild/rules_scala/commit/76d4ab9f855f78113ee5990a84b0ad55d2417e4a): - Updates naming for scala artifact replacements in bazel-deps. See [Usage with bazel-deps](#usage-with-bazel-deps) +```py +register_toolchains( + "@rules_scala//scala:unused_dependency_checker_error_toolchain", +) +``` + +In `WORKSPACE`, this must come before calling `scala_register_toolchains()` to +ensure this toolchain takes precedence. The same exact call will also work in +`MODULE.bazel`. + +`scala_register_toolchains()` should remain in `WORKSPACE`, but there's no +equivalent for `MODULE.bazel`. The `MODULE.bazel` file from `rules_scala` will +automatically call `register_toolchains()` for toolchains configured via its +module extension, so you don't have to. + +### Bzlmod configuration (coming soon!) + +The upcoming Bzlmod implementation will funnel through the `scala_toolchains()` +macro as well, ensuring maximum compatibility with `WORKSPACE` configurations. +The equivalent Bzlmod stanza for the `scala_toolchains()` stanza above would be +the following. + +```py +bazel_dep(name = "rules_scala", version = "7.0.0") + +scala_config = use_extension( + "@rules_scala//scala/extensions:config.bzl", + "scala_config", +) + +scala_config.settings(scala_version = "2.13.15") + +scala_deps = use_extension( + "@rules_scala//scala/extensions:deps.bzl", + "scala_deps", +) + +scala_deps.toolchains( + scalafmt = True, + scalatest = True, +) +``` + +The module extensions will call `scala_config()` and `scala_toolchains()` in +their implementation. The `MODULE.bazel` file for `rules_scala` declares its own +dependencies via `bazel_dep()`, allowing Bazel to resolve versions according to +the main repository/root module configuration. It also calls +[`register_toolchains()`][reg_tool], so you don't have to (unless you want to +register a specific toolchain to resolve first). + +[reg_tool]: https://bazel.build/rules/lib/globals/module#register_toolchains + +The `MODULE.bazel` files in this repository will also provide many examples +(when they land per #1482). + +### Embedded resource paths no longer begin with `external/` + +[Any program compiled with an external repo asset in its `resources` attribute +will need to strip the `external/` and repo name components from its +path][ext-path]. For example, the path for `resources = +["@some_external_repo//:resource.txt"]` would change thus: + +[ext-path]: https://github.com/bazelbuild/rules_scala/pull/1621#issuecomment-2417506589 + +- Before: `external/some_external_repo/resource.txt` +- After: `resource.txt` + +This avoids encoding repo names or any other Bazel system knowledge in the compiled artifacts. This is especially important under Bzlmod, because the generated path would otherwise contain [the _canonical_ repo name, upon which users should never depend](https://bazel.build/external/module#repository_names_and_strict_deps). + +### Update `@bazel_tools//tools/jdk` targets to `@rules_java//toolchains` targets + +Per #1660, `rules_java` 7.10.0 and later precipitate the need to replace +`@bazel_tools//tools/jdk` targets with corresponding `@rules_java//toolchains` +targets. Fix any targets broken by this `rules_java` upgrade by doing a global +search and replace. + +However, `@bazel_tools//tools/jdk:toolchain_type` dependencies must remain for +now, as there's not yet a corresponding [`toolchain_type()`]( +https://bazel.build/versions/6.1.0/reference/be/platform#toolchain_type) target +in `@rules_java`. + +## Breaking changes coming in `rules_scala` 8.x + +__The main objective of 8.x will be to enable existing users to migrate to Bazel +8 and Bzlmod.__ To facilitate a gradual migration, it will remain compatible +with both `WORKSPACE` and Bzlmod. However, it will contain the following +breaking changes when upgrading from `rules_scala` 7.x. + +### Replace some `$(location)` calls with `$(rootpath)` for Bazel 8 + +This isn't actually a `rules_scala` breakage, but a Bazel 8 breakage encountered +while preparing `rules_scala` for Bazel 8 in #1652. +bazelbuild/rules_scala@7b98e3a and bazelbuild/bazel#25198 describe how the +semantics of some instances of `$(location)` changed, and how changing these +particular instances to `$(rootpath)` fixed them. + +The good news is that replacing such instances `$(location)` with `$(rootpath)` +is backwards compatible to Bazel 6.5.0 and 7.5.0. Updating them now will ensure +future compatibility. + +### Limited Bazel 6.5.0 compatibility + +`rules_scala` 8.0.0 will not support Bzlmod with Bazel 6.5.0 because +[Bazel 6.5.0 doesn't support `use_repo_rule`]( +https://bazel.build/versions/6.5.0/rules/lib/globals), which +[`rules_jvm_external` >= 6.3 requires]( +https://github.com/bazelbuild/rules_scala/issues/1482#issuecomment-2515496234). + +`WORKSPACE` builds will continue to work with Bazel 6.5.0, but not out of the +box. Per #1647, using Bazel 6.5.0 with `rules_scala` 8.x will require adding the +following flags to `.bazelrc`, required by the newer `abseil-cpp` version used +by `protobuf`: + +```txt +common --enable_platform_specific_config + +common:linux --cxxopt=-std=c++17 +common:linux --host_cxxopt=-std=c++17 +common:macos --cxxopt=-std=c++17 +common:macos --host_cxxopt=-std=c++17 +common:windows --cxxopt=/std=c++17 +common:windows --host_cxxopt=/std=c++17 +``` + +Note that this example uses `common:` config settings instead of `build:`. This +seems to prevent invalidating the action cache between `bazel` runs, which +improves performance. + +### Bazel module compatibility levels between 7.0.0 and 8.0.0 + +`rules_scala` 7.0.0 and 8.0.0 will have different +[`compatibility_level`](https://bazel.build/external/module#compatibility_level) +values for their [`module()`](https://bazel.build/rules/lib/globals/module) +directives. This is due to the gap in supported `protobuf` versions documented +in #1647 (between v25.5 and v28). + +This will ensure any users attempting to mismatch `protobuf` and `rules_scala` +versions will break during module resolution, rather than during a later +execution step. (Though, as described in #1647, there are now measures in place +to cause the build to crash during a mismatch instead of hanging.) + +The concept of proper `compatibility_level` usage is still up for discussion in +bazelbuild/bazel#24302. The `rules_scala` implementation will track semantic +versioning major version numbers, and clearly document the reason for the level +bump. If a version bump may break builds for any known reason, we should explain +why up front instead of waiting for users to be surprised. + +[A comment from #1647 illustrates how `rules_erlang` fails due to +`compatibility_level` conflicts][erlang]. The [`rules_erlang` 3.0.0 release +notes](https://github.com/rabbitmq/rules_erlang/releases/tag/3.0.0) describe the +breaking changes. This seems like a reasonable model to follow. + +[erlang]: https://github.com/bazelbuild/rules_scala/issues/1647#issuecomment-2486777859 ## Contributing @@ -274,20 +647,19 @@ See [CONTRIBUTING.md](CONTRIBUTING.md) for more info. Here's a (non-exhaustive) list of companies that use `rules_scala` in production. Don't see yours? [You can add it in a PR](https://github.com/bazelbuild/rules_scala/edit/master/README.md)! -* [Ascend](https://ascend.io/) -* [Canva](https://www.canva.com/) -* [Domino Data Lab](https://www.dominodatalab.com/) -* [Etsy](https://www.etsy.com/) -* [Gemini](https://gemini.com/) -* [Grand Rounds](http://grandrounds.com/) -* [Kitty Hawk](https://kittyhawk.aero/) -* [Meetup](https://meetup.com/) -* [Spotify](https://www.spotify.com/) -* [Stripe](https://stripe.com/) -* [Tally](https://www.meettally.com/) -* [Twitter](https://twitter.com/) -* [VirtusLab](https://virtuslab.com/) -* [VSCO](https://vsco.co) -* [Wix](https://www.wix.com/) -* [Yobi](https://www.yobi.ai/) - +- [Ascend](https://ascend.io/) +- [Canva](https://www.canva.com/) +- [Domino Data Lab](https://www.dominodatalab.com/) +- [Etsy](https://www.etsy.com/) +- [Gemini](https://gemini.com/) +- [Grand Rounds](http://grandrounds.com/) +- [Kitty Hawk](https://kittyhawk.aero/) +- [Meetup](https://meetup.com/) +- [Spotify](https://www.spotify.com/) +- [Stripe](https://stripe.com/) +- [Tally](https://www.meettally.com/) +- [Twitter](https://twitter.com/) +- [VirtusLab](https://virtuslab.com/) +- [VSCO](https://vsco.co) +- [Wix](https://www.wix.com/) +- [Yobi](https://www.yobi.ai/) diff --git a/docs/coverage.md b/docs/coverage.md index d0f741c7f..e1d84e2a3 100644 --- a/docs/coverage.md +++ b/docs/coverage.md @@ -1,10 +1,10 @@ -## Coverage support +# Coverage support -### Running tests with coverage +## Running tests with coverage rules_scala supports coverage: -``` +```txt bazel coverage //... ``` @@ -12,7 +12,7 @@ It will produce several .dat files with results for your targets. You can also add more options to receive a combined coverage report: -``` +```txt bazel coverage \ --combined_report=lcov \ --coverage_report_generator="@bazel_tools//tools/test/CoverageOutputGenerator/java/com/google/devtools/coverageoutputgenerator:Main" \ @@ -21,18 +21,18 @@ bazel coverage \ This should produce a single `bazel-out/_coverage/_coverage_report.dat` from all coverage files that are generated. -### Processing coverage reports +## Processing coverage reports -You can install `lcov` package (that supports the format Bazel uses for coverage reports) to have access to additional tools: +You can install the [`lcov`](https://github.com/linux-test-project/lcov) package (that supports the format Bazel uses for coverage reports) to have access to additional tools: -``` +```txt # Use your system package manager. E.g. on Ubuntu: sudo apt install lcov ``` Having `lcov` package installed you can extract information from your coverage reports: -``` +```txt # For a summary: lcov --summary your-coverage-report.dat # For details: @@ -55,26 +55,78 @@ echo "coverage report at file://${destdir}/index.html" ``` -### Support for testing frameworks +## Support for testing frameworks Coverage support has been only tested with [ScalaTest](http://www.scalatest.org/). -### Working around missing lambda coverage with Scala 2.12+ +## JaCoCo -The current Jacoco version in Bazel (0.8.3) has missing coverage for lambdas -(including for comprehensions; see issue https://github.com/bazelbuild/rules_scala/issues/1056). Also, the support for -filtering out code generated by the Scala compiler is quite reduced in Jacoco. +`rules_scala` uses the [JaCoCo](https://www.jacoco.org/jacoco/) library imported +by the underlying [`rules_java`](https://github.com/bazelbuild/rules_java) +module to generate code coverage. `rules_java`, in turn, imports JaCoCo via the +[`java_tools`](https://github.com/bazelbuild/java_tools) repository, built from +the [tools/jdk/BUILD.java_tools]( +https://github.com/bazelbuild/bazel/blob/master/tools/jdk/BUILD.java_tools) file +and [third_party/java/jacoco]( +https://github.com/bazelbuild/bazel/blob/master/third_party/java/jacoco/BUILD) +package in the Bazel source. `java_tools` and `rules_java` are released more +frequently than Bazel itself, decoupling them from the Bazel release cycle. -This can be worked around by building a fixed version of Jacoco yourselves (including backported fixes from 0.8.5) and reconfiguring your -build to use that one instead of the default `jacocorunner`. +To check the version of JaCoCo used by a specific version of `rules_java`: -You can build jacocorunner with a script in `scripts/build_jacocorunner/build_jacocorunner.sh` (see comments there for more explanation and options). +- Open [`java/repositories.bzl` in the `rules_java` source]( + https://github.com/bazelbuild/rules_java/blob/master/java/repositories.bzl). + +- Select a specific version of `rules_java` using the **Switch branches/tabs** + dropdown menu. + + - Alternatively, replace `master` with the `rules_java` version in the + `java/repositories.bzl` URL. + +- Make note of the version of `java_tools` specified in the `JAVA_TOOLS_CONFIG` + dictionary. For example, [`rules_java` 8.9.0 uses `java_tools` v.13.16]( + https://github.com/bazelbuild/rules_java/blob/8.9.0/java/repositories.bzl#L49). + +- Download the `java_tools` archive using the archive URL: + + ```txt + curl -LO https://mirror.bazel.build/bazel_java_tools/releases/java/v13.16/java_tools-v13.16.zip + ``` + +- Unzip the archive, then inspect the top level `BUILD` file: -An older version of this script (for Bazel 4) can be found in `scripts/build_jacocorunner/build_jacocorunner_bazel_4.0.sh`. + ```sh + $ grep 'jacoco\.core-' BUILD + + jars = ["java_tools/third_party/java/jacoco/org.jacoco.core-0.8.11.jar"], + srcjar = "java_tools/third_party/java/jacoco/org.jacoco.core-0.8.11-sources.jar", + srcs = ["java_tools/third_party/java/jacoco/org.jacoco.core-0.8.11.jar"], + ``` + +| `rules_java` version | `java_tools` version | JaCoCo version | +| :-: | :-: | :-: | +| [8.9.0](https://github.com/bazelbuild/rules_java/blob/8.9.0/java/repositories.bzl#L49) | v13.16 | [0.8.11][JaCoCo version] | +| [7.12.4](https://github.com/bazelbuild/rules_java/blob/7.12.4/java/repositories.bzl#L49) | v13.9 | [0.8.11][JaCoCo version] | + +For information on updating the [JaCoCo version][] used by `rules_java`, +`java_tools`, and Bazel, see [Bazel's "Upgrading Jacoco version" +README]( +https://github.com/bazelbuild/bazel/blob/master/third_party/java/jacoco/README.md). + +[JaCoCo version]: https://www.jacoco.org/jacoco/trunk/doc/changes.html + +## Working around missing JaCoCo features + +The version of JaCoCo that ships with `rules_java` may lack support for newer +Scala features. To work around this, build an updated version of JaCoCo and +configure the project to use this new artifact instead of the default +`jacocorunner`. + +You can build jacocorunner with a script in `scripts/build_jacocorunner/build_jacocorunner.sh` (see comments there for more explanation and options). Then, you can use the `jacocorunner` property of `scala_toolchain` to provide the jacocorunner you have built: -``` +```py # Example contents of coverage_local_jacocorunner/BUILD scala_toolchain( name = "local_jacocorunner_toolchain_impl", @@ -85,7 +137,7 @@ scala_toolchain( toolchain( name = "local_jacocorunner_scala_toolchain", toolchain = "local_jacocorunner_toolchain_impl", - toolchain_type = "@io_bazel_rules_scala//scala:toolchain_type", + toolchain_type = "@rules_scala//scala:toolchain_type", visibility = ["//visibility:public"], ) @@ -100,7 +152,7 @@ keeping binaries in your repository). Finally, provide the `scala_toolchain` in your `.bazelrc` or as an option to `bazel coverage`: -``` +```txt coverage --extra_toolchains="//coverage_local_jacocorunner:local_jacocorunner_scala_toolchain" ``` @@ -108,7 +160,7 @@ You could also register the toolchain in your `WORKSPACE`. You can verify that the locally built `jacocorunner` works with `manual_test/coverage_local_jacocorunner/test.sh`. -#### Notes +## Notes Please ensure these scripts use Java 8. diff --git a/docs/cross-compilation.md b/docs/cross-compilation.md index 9de52aa23..b59140155 100644 --- a/docs/cross-compilation.md +++ b/docs/cross-compilation.md @@ -4,20 +4,24 @@ Read *Quick start* for an information on how to use cross compilation. The remaining sections contain more detailed information, useful especially for toolchain & rule developers. ## Quick start + `scala_config` repository rule accepts two parameters related to Scala version: -* `scala_version` – a single, default version; -* `scala_versions` – a list of versions to make available for use. + +- `scala_version` – a single, default version; +- `scala_versions` – a list of versions to make available for use. The first one, `scala_version`, will be used as a default, but it can be overridden for specific targets for any version from the `scala_versions`. Multiple rules, such as: + - [scala_library](/scala/private/rules/scala_library.bzl) - [scala_binary](/scala/private/rules/scala_binary.bzl) - [scala_repl](/scala/private/rules/scala_repl.bzl) - [scala_test](/scala/private/rules/scala_test.bzl) support such override via the `scala_version` attribute, e.g.: -```starlark + +```py scala_library( name = ... ... @@ -32,16 +36,19 @@ For this library and all its dependencies 2.12.18 compiler will be used, unless `scala_config` creates the repository `@io_bazel_rules_scala_config`. File created there, `config.bzl`, consists of many variables. In particular: -* `SCALA_VERSION` – representing the default Scala version, e.g. `"3.3.1"`; -* `SCALA_VERSIONS` – representing all configured Scala versions, e.g. `["2.12.18", "3.3.1"]`. +- `SCALA_VERSION` – representing the default Scala version, e.g. `"3.3.1"`; +- `SCALA_VERSIONS` – representing all configured Scala versions, e.g. `["2.12.18", "3.3.1"]`. ## Build settings + Configured `SCALA_VERSIONS` correspond to allowed values of [build setting](https://bazel.build/extending/config#user-defined-build-setting). ### `scala_version` + `@io_bazel_rules_scala_config` in its root package defines the following build setting: -```starlark + +```py string_setting( name = "scala_version", build_setting_default = "3.3.1", @@ -50,45 +57,56 @@ string_setting( ) ... ``` + This build setting can be subject of change by [transitions](https://bazel.build/extending/config#user-defined-transitions) (within allowed `values`). ### Config settings + Then for each Scala version we have a [config setting](https://bazel.build/extending/config#build-settings-and-select): -```starlark + +```py config_setting( name = "scala_version_3_3_1", flag_values = {":scala_version": "3.3.1"}, ) ... ``` + The `name` of `config_setting` corresponds to `"scala_version" + version_suffix(scala_version)`. One may use this config setting in `select()` e.g. to provide dependencies relevant to a currently used Scala version. - ## Version-dependent behavior + Don't rely on `SCALA_VERSION` as it represents the default Scala version, not necessarily the one that is currently requested. If you need to customize the behavior for specific Scala version, there are two scenarios. ### From toolchain -If you have an access to the Scala toolchain (`@io_bazel_rules_scala//scala:toolchain_type`), there is `scala_version` field provided in there: -```starlark + +`@rules_scala//scala:toolchain_type` provides the `scala_version` field: + +```py def _rule_impl(ctx): ... - ctx.toolchains["@io_bazel_rules_scala//scala:toolchain_type"].scala_version + ctx.toolchains["@rules_scala//scala:toolchain_type"].scala_version ... ``` ### From config setting + In BUILD files, you need to use the config settings with `select()`. Majority of use cases is covered by the `select_for_scala_version` utility macro. If more flexibility is needed, you can always write the select manually. #### With select macro + See example usage of the `select_for_scala_version`: -```starlark -load("@io_bazel_rules_scala//:scala_cross_version_select.bzl", "select_for_scala_version") +```py +load( + "@rules_scala//:scala_cross_version_select.bzl", + "select_for_scala_version", +) scala_library( ... @@ -113,8 +131,10 @@ scala_library( See complete documentation in the [scala_cross_version_select.bzl](/scala/scala_cross_version_select.bzl) file #### Manually + An example usage of `select()` to provide custom dependency for specific Scala version: -```starlark + +```py deps = select({ "@io_bazel_rules_scala_config//:scala_version_3_3_1": [...], ... @@ -122,34 +142,38 @@ deps = select({ ``` For more complex logic, you can extract it to a `.bzl` file: -```starlark + +```py def srcs(scala_version): if scala_version.startswith("2"): ... ... ``` + and then in the `BUILD` file: -```starlark -load("@io_bazel_rules_scala//:scala_cross_version.bzl", "version_suffix") -load("@io_bazel_rules_scala_config//:config.bzl", "SCALA_VERSIONS") + +```py load("....bzl", "srcs") +load("@io_bazel_rules_scala_config//:config.bzl", "SCALA_VERSIONS") +load("@rules_scala//:scala_cross_version.bzl", "version_suffix") scala_library( ... srcs = select({ "@io_bazel_rules_scala_config//:scala_version" + version_suffix(v): srcs(v) for v in SCALA_VERSIONS - }), + }), ... ) ``` - ## Requesting specific version + To use other than default version of Scala, you need to change the current `@io_bazel_rules_scala_config//:scala_version` build setting. Simple transition, setting the Scala version to one found in `scala_version` attribute: -```starlark + +```py def _scala_version_transition_impl(settings, attr): if attr.scala_version: return {"@io_bazel_rules_scala_config//:scala_version": attr.scala_version} @@ -164,8 +188,13 @@ scala_version_transition = transition( ``` To use it in a rule, use the `scala_version_transition` as `cfg` and use `toolchain_transition_attr` in `attrs`: -```starlark -load("@io_bazel_rules_scala//scala:scala_cross_version.bzl", "scala_version_transition", "toolchain_transition_attr") + +```py +load( + "@rules_scala//scala:scala_cross_version.bzl", + "scala_version_transition", + "toolchain_transition_attr", +) _scala_library_attrs.update(toolchain_transition_attr) @@ -183,13 +212,13 @@ def make_scala_library(*extras): ) ``` - ## Toolchains + Standard [toolchain resolution](https://bazel.build/extending/toolchains#toolchain-resolution) procedure determines which toolchain to use for Scala targets. Toolchain should declare its compatibility with Scala version by using `target_settings` attribute of the `toolchain` rule: -```starlark +```py toolchain( ... target_settings = ["@io_bazel_rules_scala_config//:scala_version_3_3_1"], @@ -198,16 +227,17 @@ toolchain( ``` ### Cross-build support tiers + `rules_scala` consists of many toolchains implementing various toolchain types. Their support level for cross-build setup varies. We can distinguish following tiers: -* No `target_settings` set – not migrated, will work on the default `SCALA_VERSION`; undefined behavior on other versions. - * (all toolchains not mentioned elsewhere) -* `target_settings` set to the `SCALA_VERSION` – not fully migrated; will work only on the default `SCALA_VERSION` and will fail the toolchain resolution on other versions. - * (no development in progress) -* Multiple toolchain instances with `target_settings` corresponding to each of `SCALA_VERSIONS` – fully migrated; will work in cross-build setup. - * [the main Scala toolchain](/scala/BUILD) - * [Scalafmt](/scala/scalafmt/BUILD) - * [Scalatest](/testing/testing.bzl) +- No `target_settings` set – not migrated, will work on the default `SCALA_VERSION`; undefined behavior on other versions. + - (all toolchains not mentioned elsewhere) +- `target_settings` set to the `SCALA_VERSION` – not fully migrated; will work only on the default `SCALA_VERSION` and will fail the toolchain resolution on other versions. + - (no development in progress) +- Multiple toolchain instances with `target_settings` corresponding to each of `SCALA_VERSIONS` – fully migrated; will work in cross-build setup. + - [the main Scala toolchain](/scala/BUILD) + - [Scalafmt](/scala/scalafmt/BUILD) + - [Scalatest](/testing/testing.bzl) diff --git a/docs/customizable_phase.md b/docs/customizable_phase.md index e51743aba..ee9233f53 100644 --- a/docs/customizable_phase.md +++ b/docs/customizable_phase.md @@ -1,37 +1,46 @@ # Customizable Phase ## Contents -* [Overview](#overview) -* [Who needs customizable phase](#who-needs-customizable-phase) -* [As a consumer](#as-a-consumer) -* [As a contributor](#as-a-contributor) - * [Phase naming convention](#phase-naming-convention) -* [Cooperation with IntelliJ plugin](#cooperation-with-intellij-plugin) + +- [Overview](#overview) +- [Who needs customizable phase](#who-needs-customizable-phase) +- [As a consumer](#as-a-consumer) +- [As a contributor](#as-a-contributor) + - [Phase naming convention](#phase-naming-convention) +- [Cooperation with IntelliJ plugin](#cooperation-with-intellij-plugin) ## Overview + Phases increase configurability. Rule implementations are defined as a list of phases. Each phase defines a specific step, which helps breaking up implementation into smaller and more readable groups. Some phases are independent from others, which means the order doesn't matter. However, some phases depend on outputs of previous phases, in this case, we should make sure it meets all the prerequisites before executing phases. The biggest benefit of phases is that it is customizable. If default phase A is not doing what you expect, you may switch it with your self-defined phase A. One use case is to write your own compilation phase with your favorite Scala compiler. You may also extend the default phase list for more functionalities. One use case is to check the Scala format. ## Who needs customizable phase + Customizable phase is an advanced feature for people who want the rules to do more. If you are an experienced Bazel rules developer, we make this powerful API public for you to do custom work without impacting other consumers. If you have no experience on writing Bazel rules, we are happy to help but be aware it may be frustrating at first. If you don't need to customize your rules and just need the default setup to work correctly, then just load the following file for default rules: + +```py +load("@rules_scala//scala:scala.bzl") ``` -load("@io_bazel_rules_scala//scala:scala.bzl") -``` + Otherwise read on: ## As a consumer + You need to load the following 2 files: + +```py +load("@rules_scala//scala:advanced_usage/providers.bzl", "ScalaRulePhase") +load("@rules_scala//scala:advanced_usage/scala.bzl", "make_scala_binary") ``` -load("@io_bazel_rules_scala//scala:advanced_usage/providers.bzl", "ScalaRulePhase") -load("@io_bazel_rules_scala//scala:advanced_usage/scala.bzl", "make_scala_binary") -``` + `ScalaRulePhase` is a phase provider to pass in custom phases. Rules with `make_` prefix, like `make_scala_binary`, are customizable rules. `make_`s take a dictionary as input. It currently supports appending `attrs` and `outputs` to default rules, as well as modifying the phase list. For example: -``` + +```py ext_add_custom_phase = { "attrs": { "custom_content": attr.string( @@ -48,8 +57,10 @@ ext_add_custom_phase = { custom_scala_binary = make_scala_binary(ext_add_custom_phase) ``` + `make_`s append `attrs` and `outputs` to the default rule definitions. All items in `attrs` can be accessed by `ctx.attr`, and all items in `outputs` can be accessed by `ctx.outputs`. `phase_providers` takes a list of targets which define how you want to modify phase list. -``` + +```py def _add_custom_phase_singleton_implementation(ctx): return [ ScalaRulePhase( @@ -63,26 +74,31 @@ add_custom_phase_singleton = rule( implementation = _add_custom_phase_singleton_implementation, ) ``` + `add_custom_phase_singleton` is a rule solely to pass in custom phases using `ScalaRulePhase`. The `custom_phases` field in `ScalaRulePhase` takes a list of tuples. Each tuple has 4 elements: -``` + +```py (relation, peer_name, phase_name, phase_function) ``` - - relation: the position to add a new phase - - peer_name: the existing phase to compare the position with - - phase_name: the name of the new phase, also used to access phase information - - phase_function: the function of the new phase + +- relation: the position to add a new phase +- peer_name: the existing phase to compare the position with +- phase_name: the name of the new phase, also used to access phase information +- phase_function: the function of the new phase There are 5 possible relations: - - `^` or `first` - - `$` or `last` - - `-` or `before` - - `+` or `after` - - `=` or `replace` + +- `^` or `first` +- `$` or `last` +- `-` or `before` +- `+` or `after` +- `=` or `replace` The symbols and words are interchangable. If `first` or `last` is used, it puts your custom phase at the beginning or the end of the phase list, `peer_name` is not needed. Then you have to call the rule in a `BUILD` -``` + +```py add_custom_phase_singleton( name = "phase_custom_write_extra_file", visibility = ["//visibility:public"], @@ -92,15 +108,18 @@ add_custom_phase_singleton( You may now see `phase_providers` in `ext_add_custom_phase` is pointing to this target. The last step is to write the function of the phase. For example: -``` + +```py def phase_custom_write_extra_file(ctx, p): ctx.actions.write( output = ctx.outputs.custom_output, content = ctx.attr.custom_content, ) ``` + Every phase has 2 arguments, `ctx` and `p`. `ctx` gives you access to the fields defined in rules. `p` is the global provider, which contains information from initial state as well as all the previous phases. You may access the information from previous phases by `p..`. For example, if the previous phase, said `phase_jar` with phase name `jar`, returns a struct -``` + +```py def phase_jar(ctx, p): # Some works to get the jars return struct( @@ -108,26 +127,30 @@ def phase_jar(ctx, p): ijar = ijar, ) ``` + You are able to access information like `p.jar.class_jar` in `phase_custom_write_extra_file`. You can provide the information for later phases in the same way, then they can access it by `p.custom_write_extra_file.`. You should be able to define the files above entirely in your own workspace without making change to the [bazelbuild/rules_scala](https://github.com/bazelbuild/rules_scala). If you believe your custom phase will be valuable to the community, please refer to [As a contributor](#as-a-contributor). Pull requests are welcome. ## As a contributor + Besides the basics in [As a consumer](#as-a-consumer), the followings help you understand how phases are setup if you plan to contribute to [bazelbuild/rules_scala](https://github.com/bazelbuild/rules_scala). These are the relevant files - - `scala/private/phases/api.bzl`: the API of executing and modifying the phase list - - `scala/private/phases/phases.bzl`: re-expose phases for convenience - - `scala/private/phases/phase_.bzl`: all the phase definitions + +- `scala/private/phases/api.bzl`: the API of executing and modifying the phase list +- `scala/private/phases/phases.bzl`: re-expose phases for convenience +- `scala/private/phases/phase_.bzl`: all the phase definitions Currently phase architecture is used by 7 rules: - - scala_library - - scala_macro_library - - scala_library_for_plugin_bootstrapping - - scala_binary - - scala_test - - scala_junit_test - - scala_repl + +- scala_library +- scala_macro_library +- scala_library_for_plugin_bootstrapping +- scala_binary +- scala_test +- scala_junit_test +- scala_repl If you need to expose providers to downstream targets you need to return a dict of providers (provider-name to provider instance) from your phase under the `external_providers` attribute. @@ -141,40 +164,45 @@ To make a new phase, you have to define a new `phase_.bzl` in `scala In the rule implementations, put your new phase in `builtin_customizable_phases` list. The phases are executed sequentially, the order matters if the new phase depends on previous phases. ### Phase naming convention + Files in `scala/private/phases/` - - `phase_.bzl`: phase definition file + +- `phase_.bzl`: phase definition file Function names in `phase_.bzl` - - `phase__`: function with custom inputs of specific rule - - `phase__common`: function without custom inputs - - `_phase__default`: private function that takes `_args` for custom inputs - - `_phase_`: private function with the actual logic + +- `phase__`: function with custom inputs of specific rule +- `phase__common`: function without custom inputs +- `_phase__default`: private function that takes `_args` for custom inputs +- `_phase_`: private function with the actual logic See `phase_compile.bzl` for example. ## Cooperation with IntelliJ plugin -Bazel IntelliJ plugin has hard-coded the names of rules_scala targets that it detects as Scala targets: +[The Bazel IntelliJ plugin has hard-coded the names of rules_scala targets that it detects as Scala targets]( +https://github.com/bazelbuild/intellij/blame/22ea25d17ee9368a8c85262231009c5ec0225459/scala/src/com/google/idea/blaze/scala/ScalaBlazeRules.java#L32-L37). -https://github.com/bazelbuild/intellij/blame/22ea25d17ee9368a8c85262231009c5ec0225459/scala/src/com/google/idea/blaze/scala/ScalaBlazeRules.java#L32-L37 - -If you use custom-named rules, defined by using macros and phases it'll make the IntelliJ plugin not recognize those +If you use custom-named rules, defined by using macros and phases it'll make the IntelliJ plugin not recognize those as Scala targets. As a consequence e.g. you'll miss external dependency support for Scala in IntelliJ. -```python +```py ext_add_custom_phase = ... # some definition # Using this rule won't let you see external dependencies: custom_scala_binary = make_scala_binary(ext_add_custom_phase) ``` -This is tracked in https://github.com/bazelbuild/intellij/issues/1824. +This is tracked in bazelbuild/intellij#1824. If you need to use custom-named rules and the IntelliJ plugin together, then you have for now mainly two options: + 1. name your rules the same way as the IntelliJ plugin has hard-coded them (and use those from your own scope): - ```python + + ```py scala_binary = make_scala_binary(ext_add_custom_phase) ``` + 2. use a forked IntelliJ plugin where you extend the list of detected Scala targets - Example: https://github.com/gergelyfabian/intellij/commit/265d3761aeabb60b79cab53a9ae9832899bfc651 + Example: gergelyfabian/intellij@265d3761aeabb60b79cab53a9ae9832899bfc651 diff --git a/docs/dependency-tracking.md b/docs/dependency-tracking.md index 52b9d4653..ca87f5d08 100644 --- a/docs/dependency-tracking.md +++ b/docs/dependency-tracking.md @@ -2,117 +2,126 @@ ## [Experimental] Dependency options -There are a number of dependency options which can be set in the scala toolchain. These include -`dependency_mode`, `strict_deps_mode`, `unused_dependency_checker_mode`, and +There are a number of dependency options which can be set in the scala toolchain. These include +`dependency_mode`, `strict_deps_mode`, `unused_dependency_checker_mode`, and `dependency_tracking_method`. ### [Experimental] Recommended options We recommend one of the following sets of options -**Option A** +#### Option A + Accept the defaults, which might work well enough for you. The defaults are -``` + +```py dependency_mode = "direct", strict_deps_mode = "off", unused_dependency_checker_mode = "off", dependency_tracking_method = "high-level", ``` + but you do not need to include this in the toolchain as they are the defaults. -**Option B** -``` +#### Option B + +```py dependency_mode = "plus-one", strict_deps_mode = "error", unused_dependency_checker_mode = "error", dependency_tracking_method = "ast", ``` -Should the first option result in too much effort in handling build files and the like due to -confusing dependencies and you becoming confused as to why some specific dependency is needed when -the code being compiled never references it, consider this set of options. It will include both -dependencies and dependencies of dependencies, which in practice is enough to stop almost all -strange missing dependency errors at the cost of somewhat more incremental compile cost in +Should the first option result in too much effort in handling build files and the like due to +confusing dependencies and you becoming confused as to why some specific dependency is needed when +the code being compiled never references it, consider this set of options. It will include both +dependencies and dependencies of dependencies, which in practice is enough to stop almost all +strange missing dependency errors at the cost of somewhat more incremental compile cost in certain cases. -With these settings, we also will error on dependencies which are unneeded, and dependencies which +With these settings, we also will error on dependencies which are unneeded, and dependencies which should be included in `deps` due to be directly referenced in the code, but are not. -The dependency tracking method `ast` is experimental but so far proves to be better than the default -for computing the direct dependencies for `plus-one` mode code. In the future we hope to make this +The dependency tracking method `ast` is experimental but so far proves to be better than the default +for computing the direct dependencies for `plus-one` mode code. In the future we hope to make this the default for `plus-one` mode and remove the option altogether. -To try it out you can use the following toolchain: -`@io_bazel_rules_scala//scala:minimal_direct_source_deps`. +To try it out you can use the `@rules_scala//scala:minimal_direct_source_deps` toolchain. ### [Experimental] Dependency mode -There are three dependency modes. The reason for the multiple modes is that often `scalac` depends -on jars which seem unnecessary at first glance. Hence, in order to reduce the need to please +There are three dependency modes. The reason for the multiple modes is that often `scalac` depends +on jars which seem unnecessary at first glance. Hence, in order to reduce the need to please `scalac`, we provide the following options. -- `dependency_mode = "direct"` - only include direct dependencies during compiliation; that is, + +- `dependency_mode = "direct"` - only include direct dependencies during compiliation; that is, those in the `deps` attribute - `dependency_mode = "plus-one"` - only include `deps` and `deps` of `deps` during compiliation. -- `dependency_mode = "transitive"` - all transitive dependencies are included during compiliation. +- `dependency_mode = "transitive"` - all transitive dependencies are included during compiliation. That is, `deps`, `deps` of `deps`, `deps` of `deps` of `deps`, and so on. -Note when a dependency is included, that means its jars are included on the classpath, along with +Note when a dependency is included, that means its jars are included on the classpath, along with the jars of any targets that it exports. -When using `direct` mode, there can be cryptic `scalac` errors when one mistakenly depends on a -transitive dependency or, as more often the case for some, a transitive dependency is needed to -[please scalac](https://github.com/scalacenter/advisoryboard/blob/master/proposals/009-improve-direct-dependency-experience.md) +When using `direct` mode, there can be cryptic `scalac` errors when one mistakenly depends on a +transitive dependency or, as more often the case for some, a transitive dependency is needed to +[please scalac](https://github.com/scalacenter/advisoryboard/blob/master/proposals/009-improve-direct-dependency-experience.md) itself. -As one goes down the list, more dependencies are included which helps reduce confusing requirements -to add `deps`, at the cost of increased incremental builds due to a greater number of dependencies. -In practice, using `plus-one` deps results in almost no confusing `deps` entries required while +As one goes down the list, more dependencies are included which helps reduce confusing requirements +to add `deps`, at the cost of increased incremental builds due to a greater number of dependencies. +In practice, using `plus-one` deps results in almost no confusing `deps` entries required while still being relatively small in terms of the number of total dependencies included. -**Caveats for `plus_one` and `transitive`:** -
    -
  • Extra builds- Extra dependencies are inputs to the compilation action which means you can - potentially have more build triggers for changes the cross the ijar boundary
    • -
  • Label propagation- since label of targets are needed for the clear message and since it's - not currently supported by JavaInfo from bazel we manually propagate it. This means that the - error messages have a significantly lower grade if you don't use one of the scala rules or - `scala_import` (since they don't propagate these labels)
    • -
  • javac outputs incorrect targets due to a problem we're tracing down. Practically we've - noticed it's pretty trivial to understand the correct target (i.e. it's almost a formatting - problem)
    • -
- -Note: the last two issues are bugs which will be addressed by +#### Caveats for `plus_one` and `transitive` + +- Extra builds: Extra dependencies are inputs to the compilation action which + means you can potentially have more build triggers for changes the cross the ijar boundary. +- Label propagation: Since targets labels are needed for clear messages, and + since it's not currently supported by `JavaInfo` from Bazel, we manually + propagate it. This means that the error messages have a significantly lower + grade if you don't use one of the scala rules or `scala_import` (since they + don't propagate these labels). +- `javac` outputs incorrect targets due to a problem we're tracking down. + Practically we've noticed it's pretty trivial to understand the correct + target (i.e. it's almost a formatting problem). + +Note: the last two issues are bugs which will be addressed by [https://github.com/bazelbuild/rules_scala/issues/839]. ### [Experimental] Strict deps mode -We have a strict dependency checker which requires that any type referenced in the sources of a -scala target should be included in that rule's deps. To learn about the motivation for this you can -visit this Bazel blog [post](https://blog.bazel.build/2017/06/28/sjd-unused_deps.html) on the + +We have a strict dependency checker which requires that any type referenced in the sources of a +scala target should be included in that rule's deps. To learn about the motivation for this you can +visit this Bazel blog [post](https://blog.bazel.build/2017/06/28/sjd-unused_deps.html) on the subject. -The option `strict_deps_mode` can be set to `off`, `warn`, or `error`. We highly recommend setting +The option `strict_deps_mode` can be set to `off`, `warn`, or `error`. We highly recommend setting it to `error`. In both cases of `warn` or `error` you will get the following text in the event of a violation: -``` + +```txt ... Target '//some_package:transitive_dependency' is used but isn't explicitly declared, please add it to the deps. You can use the following buildozer command: buildozer 'add deps //some_package:transitive_dependency' //some_other_package:transitive_dependency_user ``` -Note that if you have `buildozer` installed you can just run the last line and have it automatically + +Note that if you have `buildozer` installed you can just run the last line and have it automatically apply the fix for you. -Note that this option only applies to scala code. Any java code, even that within `scala_library` +Note that this option only applies to scala code. Any java code, even that within `scala_library` and other rules_scala rules, is still controlled by the `--strict_java_deps` command-line flag. ### [Experimental] Unused dependency checking -To allow for better caching and faster builds we want to minimize the direct dependencies of our -targets. Unused dependency checking makes sure that all targets specified as direct dependencies are + +To allow for better caching and faster builds we want to minimize the direct dependencies of our +targets. Unused dependency checking makes sure that all targets specified as direct dependencies are actually used. If `unused_dependency_checker_mode` is set to either `error` or `warn` you will get the following message for any dependencies that are not used: -``` + +```txt error: Target '//some_package:unused_dep' is specified as a dependency to //target:target but isn't used, please remove it from the deps. You can use the following buildozer command: buildozer 'remove deps //some_package:unused_dep' //target:target @@ -145,18 +154,20 @@ It can be daunting to turn on strict deps checking or unused dependency mode che We recommend turning on strict_deps_mode first, as rule `A` might have an entry `B` in its `deps`, and `B` in turn depends on `C`. Meanwhile, the code of `A` only uses `C` but not `B`. Hence, the unused dependency checker, if on, will request that `B` be removed from `A`'s deps. But this will lead to a compile error as `A` can no longer depend on `C`. However, if strict dependency checking was on, then `A`'s deps is guaranteed to have `C` in it. ### Include/exclude filters -Both strict and unused deps tracking scope can be controlled by configuring *prefixes* of -included/excluded targets on the toolchain with attributes + +Both strict and unused deps tracking scope can be controlled by configuring *prefixes* of +included/excluded targets on the toolchain with attributes `dependency_tracking_strict_deps_patterns`, `dependency_tracking_unused_deps_patterns`. -Filters can be used for gradual migration towards strict/unused deps error mode. In general, you -should get strict deps working first before enabling unused deps mode. +Filters can be used for gradual migration towards strict/unused deps error mode. In general, you +should get strict deps working first before enabling unused deps mode. Patterns prefixed with "-" will exclude targets. -Example patterns: +Example patterns: + - `""` includes everything - default setting - `"@//"` includes all local targets -- `"@//foo/"` includes everything under package `@//foo`, if trailing slash is omitted, it will match +- `"@//foo/"` includes everything under package `@//foo`, if trailing slash is omitted, it will match other packages, which start with "some", eg. `@//foo_bar` - `"@//foo:bar"` includes target under label `@//foo:bar` - `@junit_junit` includes external targets, which start with `"@junit_junit"` diff --git a/docs/phase_scalafmt.md b/docs/phase_scalafmt.md index 0a0e6d223..0b4a69a24 100644 --- a/docs/phase_scalafmt.md +++ b/docs/phase_scalafmt.md @@ -1,49 +1,79 @@ # Phase Scalafmt ## Contents -* [Overview](#overview) -* [How to set up](#how-to-set-up) -* [IntelliJ plugin support](#intellij-plugin-support) + +- [Overview](#overview) +- [How to set up](#how-to-set-up) +- [IntelliJ plugin support](#intellij-plugin-support) ## Overview + A phase extension `phase_scalafmt` can format Scala source code via [Scalafmt](https://scalameta.org/scalafmt/). ## How to set up + Add this snippet to `WORKSPACE` -``` -load("//scala/scalafmt:scalafmt_repositories.bzl", "scalafmt_default_config", "scalafmt_repositories") -scalafmt_default_config() -scalafmt_repositories() + +```py +load( + "@rules_scala//scala:toolchains.bzl", + "scala_register_toolchains", + "scala_toolchains", +) + +scala_toolchains( + # Other toolchains settings... + scalafmt = True, +) + +scala_register_toolchains() ``` To add this phase to a rule, you have to pass the extension to a rule macro. Take `scala_binary` for example, -``` + +```py load("//scala:advanced_usage/scala.bzl", "make_scala_binary") load("//scala/scalafmt:phase_scalafmt_ext.bzl", "ext_scalafmt") scalafmt_scala_binary = make_scala_binary(ext_scalafmt) ``` + Then use `scalafmt_scala_binary` as normal. The extension adds 2 additional attributes to the rule - - `format`: enable formatting - - `config`: the Scalafmt configuration file + +- `format`: enable formatting +- `config`: the Scalafmt configuration file When `format` is set to `true`, you can do -``` + +```txt bazel run .format ``` + to format the source code, and do -``` + +```txt bazel run .format-test ``` + to check the format (without modifying source code). The extension provides default configuration, but there are 2 ways to use custom configuration - - Put `.scalafmt.conf` at root of your workspace - - Pass `.scalafmt.conf` in via `config` attribute -If using scala 3 you must append `runner.dialect = scala3` to .scalafmt.conf +- Put `.scalafmt.conf` at the root of your workspace +- Pass `.scalafmt.conf` in via `scala_toolchains`: + + ```py + scala_toolchains( + # Other toolchains settings... + scalafmt = True, + scalafmt_default_config_path = "//path/to/my/custom:scalafmt.conf", + ) + ``` + +When using Scala 3, you must append `runner.dialect = scala3` to +`.scalafmt.conf`. ## IntelliJ plugin support diff --git a/docs/publish_to_maven.md b/docs/publish_to_maven.md index 4fb63ddae..958ab27eb 100644 --- a/docs/publish_to_maven.md +++ b/docs/publish_to_maven.md @@ -1,7 +1,8 @@ -## Publish your Scala Libraries to a Maven Repository +# Publish your Scala Libraries to a Maven Repository -### 1. add a dependency on bazel-distribution repo from graknlabs -``` +## 1. add a dependency on bazel-distribution repo from graknlabs + +```py git_repository( name = "graknlabs_bazel_distribution", remote = "https://github.com/graknlabs/bazel-distribution", @@ -9,12 +10,13 @@ git_repository( ) ``` -### 2. add aggregate targets -for each artifact you want to publish to maven central, +## 2. add aggregate targets + +for each artifact you want to publish to maven central, create an aggregate `scala_library` target that includes sources for all finer-grained targets (if you have targets with coarse-grained granularity you can skip this step) -```python +```py scala_library( name = "greyhound-core", srcs = [ @@ -33,22 +35,29 @@ scala_library( ], ) ``` -**A few notes:** + +### A few notes + 1. All the labels in srcs are actually filegroup with appropriate visibility settings that allow out-of-package dependency: -```python -filegroup( - name = "sources", - srcs = glob(["*.java"]) + glob(["*.scala"]), - visibility = ["//core:__subpackages__"], -) -``` + + ```py + filegroup( + name = "sources", + srcs = glob(["*.java"]) + glob(["*.scala"]), + visibility = ["//core:__subpackages__"], + ) + ``` + 2. There is a special `maven_coordinates` tag that helps the `assemble_maven` rule to fill-in details in the pom file. + 3. You also have to add a `maven_coordinates` tag to the 3rd party dependencies (such as `dev_zio_zio_2_12` in the deps attribute of the example above) so that they will appear as dependencies in the pom file. -### 3. Add `assemble_maven` target -Add assemble_maven target for each artifact you want to publish. +## 3. Add `assemble_maven` target + +Add assemble_maven target for each artifact you want to publish. It will create all the necessary files for your artifact, including main jar, source jar and pom file. Enter all project details for the pom file. -```python + +```py load("@graknlabs_bazel_distribution//maven/templates:rules.bzl", "deploy_maven", "assemble_maven") assemble_maven( @@ -64,19 +73,25 @@ assemble_maven( license = "mit" ) ``` + Notes: + 1. For the target attribute you should put the label for the `scala_library` target you created in the previous step with all the relevant sources. 2. Make sure the `project_name` and `project_description` are unique for each of these targets/artifacts 3. The `VERSION` file just contains the SEMVER, e.g. 1.0.0 4. Currently supported licenses include `apache` and `MIT` -### 4. Add deploy_maven target +## 4. Add deploy_maven target + Add `deploy_maven` target for each artifact you want to publish. The `deployment.properties` file should contain the url for the sonatype stating area: + +```txt repo.maven.release=https://oss.sonatype.org/service/local/staging/deploy/maven2/ +``` + +## 5. Build the assemble_maven target -### 5. Build the assemble_maven target Build the assemble_maven target and review the generated pom file and jar file in the bazel-bin directory. Change the targets configuration as needed. -
-For more specific information on publishing to Maven Central repository, see [this blog post](https://medium.com/wix-engineering/how-to-publish-artifacts-from-bazel-to-maven-central-71da0cf990f5) +For more specific information, see [How to publish artifacts from Bazel to Maven Central](https://medium.com/wix-engineering/how-to-publish-artifacts-from-bazel-to-maven-central-71da0cf990f5). diff --git a/docs/scala_binary.md b/docs/scala_binary.md index 488be7444..704f3fe6c 100644 --- a/docs/scala_binary.md +++ b/docs/scala_binary.md @@ -1,6 +1,6 @@ # scala_binary -```python +```py scala_binary( name, srcs, diff --git a/docs/scala_doc.md b/docs/scala_doc.md index 95f714837..7f9046ca1 100644 --- a/docs/scala_doc.md +++ b/docs/scala_doc.md @@ -1,6 +1,6 @@ # scala_doc -```python +```py scala_doc( name, deps, @@ -15,7 +15,7 @@ i.e. only when named explicitly and not through wildcards: `tags = ["manual"]`. ## Example -```python +```py scala_doc( name = "scala_docs", tags = ["manual"], @@ -43,6 +43,6 @@ pkg_tar( | Attribute name | Description | | --------------------- | ----------------------------------------------------- | -| name | `Name, required`
A unique name for this target. -| deps | `List of labels, optional`
Labels for which you want to create scaladoc. -| scalacopts | `List of strings, optional`
Extra compiler options for this library to be passed to scalac. +| name | `Name, required`
A unique name for this target. | +| deps | `List of labels, optional`
Labels for which you want to create scaladoc. | +| scalacopts | `List of strings, optional`
Extra compiler options for this library to be passed to scalac. | diff --git a/docs/scala_import.md b/docs/scala_import.md index e04ec00a1..bf6733fce 100644 --- a/docs/scala_import.md +++ b/docs/scala_import.md @@ -1,6 +1,6 @@ # scala_import -```starlark +```py scala_import( name, jars, @@ -19,25 +19,27 @@ like `scala_library`, similar to `java_import` from Java rules. This rule reimplements `java_import` without support for ijars, which break Scala macros. Generally, ijars don’t help much for external dependencies, which rarely change. -The jar's compile MANIFEST.MF is stamped with a Target-Label attribute for dependency tracking -reporting. +The jar's compile MANIFEST.MF is stamped with a Target-Label attribute for dependency tracking +reporting. ## Configuring stamping behavior -This behaviour can be changed per `scala_import` target with an attribute or globally + +This behaviour can be changed per `scala_import` target with an attribute or globally with a setting: -``` -bazel build //my:target --@io_bazel_rules_scala//scala/settings:stamp_scala_import=False + +```txt +bazel build //my:target --@rules_scala//scala/settings:stamp_scala_import=False ``` ## Attributes | Attribute name | Description | | --------------------- | ----------------------------------------------------- | -| name | `Name, required`
A unique name for this target. -| jars | `List of labels, required`
List of .jar files to import, usually in `//external`. In practice, this usually corresponds to one jar. -| deps | `List of labels, optional`
Compile time dependencies that were used to create the jar. -| runtime_deps | `List of labels, optional`
Runtime dependencies that are needed for this library. -| exports | `List of labels, optional`
List of targets to add to the dependencies of those that depend on this target. -| neverlink | `boolean, optional (default False)`
If true only use this library for compilation and not at runtime. -| srcjar | `Label, optional`
The source jar that was used to create the jar. -| stamp | `Label, optional`
Setting to control Target-Label stamping into compile jar Manifest
Default value is `@io_bazel_rules_scala//scala/settings:stamp_scala_import` +| name | `Name, required`
A unique name for this target. | +| jars | `List of labels, required`
List of .jar files to import, usually in `//external`. In practice, this usually corresponds to one jar. | +| deps | `List of labels, optional`
Compile time dependencies that were used to create the jar. | +| runtime_deps | `List of labels, optional`
Runtime dependencies that are needed for this library. | +| exports | `List of labels, optional`
List of targets to add to the dependencies of those that depend on this target. | +| neverlink | `boolean, optional (default False)`
If true only use this library for compilation and not at runtime. | +| srcjar | `Label, optional`
The source jar that was used to create the jar. | +| stamp | `Label, optional`
Setting to control Target-Label stamping into compile jar Manifest
Default value is `@rules_scala//scala/settings:stamp_scala_import` | diff --git a/docs/scala_library.md b/docs/scala_library.md index 0f8ec28f3..a80c42b96 100644 --- a/docs/scala_library.md +++ b/docs/scala_library.md @@ -1,6 +1,6 @@ # scala_library -```python +```py scala_library( name, srcs, diff --git a/docs/scala_library_suite.md b/docs/scala_library_suite.md index 33fb2531c..df2c5795a 100644 --- a/docs/scala_library_suite.md +++ b/docs/scala_library_suite.md @@ -6,4 +6,4 @@ scala libraries. The outer target will export all of the inner targets. This rule is useful for splitting up a larger target into smaller targets (typically a series of independent files), thereby enabling better cache outputs and parallelization of building individual targets. Generally speaking, downstream targets should not be aware of a suite's presence - it should be strictly -a parent-children relationship. \ No newline at end of file +a parent-children relationship. diff --git a/docs/scala_macro_library.md b/docs/scala_macro_library.md index d13179e01..b385514f4 100644 --- a/docs/scala_macro_library.md +++ b/docs/scala_macro_library.md @@ -1,6 +1,6 @@ # scala_macro_library -```python +```py scala_macro_library( name, srcs, diff --git a/docs/scala_proto_library.md b/docs/scala_proto_library.md index 51a5145e5..4ebe76bcb 100644 --- a/docs/scala_proto_library.md +++ b/docs/scala_proto_library.md @@ -3,18 +3,19 @@ To use this rule, you'll first need to add the following to your `WORKSPACE` file, which adds a few dependencies needed for ScalaPB: -```starlark -load("@io_bazel_rules_scala//scala_proto:scala_proto.bzl", "scala_proto_repositories") -scala_proto_repositories() +```py +scala_toolchains( + # Other toolchains settings... + scala_proto = True, +) -load("@io_bazel_rules_scala//scala_proto:toolchains.bzl", "scala_proto_register_toolchains") -scala_proto_register_toolchains() +scala_register_toolchains() ``` Then you can import `scala_proto_library` in any `BUILD` file like this: -```starlark -load("@io_bazel_rules_scala//scala_proto:scala_proto.bzl", "scala_proto_library") +```py +load("@rules_scala//scala_proto:scala_proto.bzl", "scala_proto_library") scala_proto_library( name = "my_scala_proto_lib", deps = [":my_target"], @@ -28,18 +29,21 @@ generated by the [ScalaPB compiler](https://github.com/scalapb/ScalaPB). | Attribute name | Description | | --------------------- | ----------------------------------------------------- | -| name | `Name, required`
A unique name for this target. -| deps | `List of labels, required`
A list of `proto_library` targets for which to generate Scala code. +| name | `Name, required`
A unique name for this target. | +| deps | `List of labels, required`
A list of `proto_library` targets for which to generate Scala code. | ## Configuration -If you want to have a different configuration from `scala_proto_repositories`, -you can configure ScalaPB options and dependencies via toolchains. +If you want to have a different configuration from `scala_proto_repositories`, +you can configure ScalaPB options and dependencies via toolchains. To configure ScalaPB options, configure a different `scala_proto_toolchain` and declare it in a `BUILD` file: -```starlark -load("@io_bazel_rules_scala//scala_proto:scala_proto_toolchain.bzl", "scala_proto_toolchain") +```py +load( + "@rules_scala//scala_proto:scala_proto_toolchain.bzl", + "scala_proto_toolchain", +) scala_proto_toolchain( name = "scala_proto_toolchain", @@ -52,44 +56,47 @@ scala_proto_toolchain( toolchain( name = "scalapb_toolchain", toolchain = ":scala_proto_toolchain", - toolchain_type = "@io_bazel_rules_scala//scala_proto:toolchain_type", + toolchain_type = "@rules_scala//scala_proto:toolchain_type", visibility = ["//visibility:public"], ) ``` - ### `scala_proto_toolchain` Toolchain Attributes | Attribute name | Description | | ----------------------------- | ----------------------------------------------------- | -| name | `Name, required`
A unique name for this toolchain. -| with_grpc | `boolean, optional (default False)`
Enables generation of grpc service bindings for services. -| with_flat_package | `boolean, optional (default False)`
When true, ScalaPB will not append the protofile base name to the package name. -| with_single_line_to_string | `boolean, optional (default False)`
Enables generation of toString() methods that use a single line format. -| blacklisted_protos | `List of labels, optional`
List of protobuf targets to exclude from recursive building. -| code_generator | `Label, optional (has default)`
Which code generator to use. A sensible default is provided. -| named_generators | `String dict, optional`
-| extra_generator_dependencies | `List of labels, optional`
-| scalac | `Label, optional (has default)`
Target for scalac. A sensible default is provided. +| name | `Name, required`
A unique name for this toolchain. | +| with_grpc | `boolean, optional (default False)`
Enables generation of grpc service bindings for services. | +| with_flat_package | `boolean, optional (default False)`
When true, ScalaPB will not append the protofile base name to the package name. | +| with_single_line_to_string | `boolean, optional (default False)`
Enables generation of toString() methods that use a single line format. | +| blacklisted_protos | `List of labels, optional`
List of protobuf targets to exclude from recursive building. | +| code_generator | `Label, optional (has default)`
Which code generator to use. A sensible default is provided. | +| named_generators | `String dict, optional` | +| extra_generator_dependencies | `List of labels, optional` | +| scalac | `Label, optional (has default)`
Target for scalac. A sensible default is provided. | To configure dependencies, configure a different `scala_proto_deps_toolchain` and declare it in a `BUILD` file: -```starlark -load("@io_bazel_rules_scala//scala_proto:scala_proto_toolchain.bzl", "scala_proto_deps_toolchain") -load("@io_bazel_rules_scala//scala:providers.bzl", "declare_deps_provider") + +```py +load("@rules_scala//scala:providers.bzl", "declare_deps_provider") +load( + "@rules_scala//scala_proto:scala_proto_toolchain.bzl", + "scala_proto_deps_toolchain", +) scala_proto_deps_toolchain( name = "default_deps_toolchain_impl", visibility = ["//visibility:public"], dep_providers = [ ":my_compile_deps", - ":my_grpc_deps", + ":my_grpc_deps", ], ) toolchain( name = "default_deps_toolchain", toolchain = ":default_deps_toolchain_impl", - toolchain_type = "@io_bazel_rules_scala//scala_proto:deps_toolchain_type", + toolchain_type = "@rules_scala//scala_proto:deps_toolchain_type", ) declare_deps_provider( @@ -108,6 +115,7 @@ declare_deps_provider( ``` ### `scala_proto_deps_toolchain` Toolchain Attributes + | Attribute name | Description | | ----------------------------- | ----------------------------------------------------- | -| dep_providers | `List of labels, optional (has default)`
allows inject gRPC (deps_id - `scalapb_grpc_deps`) and ScalaPB (deps_id `scalapb_compile_deps`) dependencies +| dep_providers | `List of labels, optional (has default)`
allows injection of gRPC (deps_id - `scalapb_grpc_deps`) and ScalaPB (deps_id `scalapb_compile_deps`) dependencies | diff --git a/docs/scala_repl.md b/docs/scala_repl.md index af210fbcd..c821c9ee0 100644 --- a/docs/scala_repl.md +++ b/docs/scala_repl.md @@ -1,6 +1,6 @@ # scala_repl -```python +```py scala_repl( name, deps, @@ -19,7 +19,8 @@ Since `bazel run` closes stdin, it cannot be used to start a REPL. Instead, use `bazel build` to build the script, then run that script as normal to start a REPL session. An example in this repo: -``` + +```txt bazel build test:HelloLibRepl bazel-bin/test/HelloLibRepl -``` \ No newline at end of file +``` diff --git a/docs/scala_test.md b/docs/scala_test.md index e6350dcaa..8116d0e8c 100644 --- a/docs/scala_test.md +++ b/docs/scala_test.md @@ -1,6 +1,6 @@ # scala_test -```python +```py scala_test( name, srcs, @@ -28,4 +28,4 @@ For backwards compatibility, it accepts a `suites` attribute which is ignored due to the ease with which that field is not correctly populated and tests are not run. -Note: If you need to pass environment variables to your tests, you can use `env_inherit` with the list of environment variables names to pass. This flag require Bazel version 5.2 rc0 or higher. \ No newline at end of file +Note: If you need to pass environment variables to your tests, you can use `env_inherit` with the list of environment variables names to pass. This flag require Bazel version 5.2 rc0 or higher. diff --git a/docs/scala_test_suite.md b/docs/scala_test_suite.md index 183bd09ad..9040c34be 100644 --- a/docs/scala_test_suite.md +++ b/docs/scala_test_suite.md @@ -4,4 +4,4 @@ The outer target defines a native test suite to run all the inner tests. This allows splitting up of a series of independent tests from one target into several finer grained targets, enabling better caching -and parallelization of building & testing individual targets. \ No newline at end of file +and parallelization of building & testing individual targets. diff --git a/docs/scala_toolchain.md b/docs/scala_toolchain.md index 924e2bfed..a0bc728cb 100644 --- a/docs/scala_toolchain.md +++ b/docs/scala_toolchain.md @@ -4,26 +4,34 @@ **Some scala_toolchain must be registered!** -### Several options to configure `scala_toolchain`: +## Several options to configure `scala_toolchain` -#### A) Use the default `scala_toolchain`: +### A) Use the builtin Scala toolchain via `scala_toolchains` In your workspace file add the following lines: -```starlark +```py # WORKSPACE # register default scala toolchain -load("@io_bazel_rules_scala//scala:toolchains.bzl", "scala_register_toolchains") +load( + "@rules_scala//scala:toolchains.bzl", + "scala_register_toolchains", + "scala_toolchains", +) + +scala_toolchains() + scala_register_toolchains() ``` -#### B) Defining your own `scala_toolchain` requires 2 steps: +### B) Defining your own `scala_toolchain` requires 2 steps 1. Add your own definition of `scala_toolchain` to a `BUILD` file: Example assumes external libraries are resolved with [rules_jvm_external](https://github.com/bazelbuild/rules_jvm_external) - ```starlark + + ```py # //toolchains/BUILD - load("//scala:scala.bzl", "setup_scala_toolchain") + load("@rules_scala//scala:scala.bzl", "setup_scala_toolchain") setup_scala_toolchain( name = "my_toolchain", @@ -55,7 +63,8 @@ scala_register_toolchains() ``` 2. Register your custom toolchain from `WORKSPACE`: - ```python + + ```py # WORKSPACE register_toolchains("//toolchains:my_scala_toolchain") ``` @@ -76,10 +85,10 @@ scala_register_toolchains() @@ -89,7 +98,7 @@ scala_register_toolchains() @@ -150,7 +159,7 @@ scala_register_toolchains() diff --git a/docs/testing.md b/docs/testing.md index 20a0909b7..4e2bdf341 100644 --- a/docs/testing.md +++ b/docs/testing.md @@ -1,40 +1,38 @@ -## Testing toolchain configuration +# Testing toolchain configuration Toolchain type `testing_toolchain_type` is used to set up test dependencies. You can customize test dependencies by defining a custom testing toolchain. In your `WORKSPACE` default repositories and toolchains can be loaded via: -```starlark -# JUnit 4 -load("@io_bazel_rules_scala//testing:junit.bzl", "junit_repositories", "junit_toolchain") -junit_repositories() -junit_toolchain() - -# ScalaTest -load("@io_bazel_rules_scala//testing:scalatest.bzl", "scalatest_repositories", "scalatest_toolchain") -scalatest_repositories() -scalatest_toolchain() - -# Specs2 with Junit -load("@io_bazel_rules_scala//testing:specs2_junit.bzl", "specs2_junit_repositories", "specs2_junit_toolchain") -specs2_junit_repositories() -specs2_junit_toolchain() + +```py +load( + "@rules_scala//scala:toolchains.bzl", + "scala_register_toolchains", + "scala_toolchains", +) + +scala_toolchains( + junit = True, # JUnit 4 + scalatest = True, # ScalaTest + specs2 = True, # Specs2 with JUnit + testing = True, # Use for all of the above, instead of individual settings +) + +scala_register_toolchains() ``` -### Configuring testing dependencies via toolchain +## Configuring testing dependencies via toolchain -Default dependencies, which come preconfigured with Rules Scala repositories are mostly tailored -towards supporting Rules Scala codebase, and may miss specific versions or libraries for your +Default dependencies, which come preconfigured with Rules Scala repositories are mostly tailored +towards supporting Rules Scala codebase, and may miss specific versions or libraries for your usecase. You should prefer configuring dependencies via toolchains. Test framework dependencies are configured via testing toolchain. For convenience, macro `setup_scala_testing_toolchain` can be used to define such toolchains. -```starlark -load( - "@io_bazel_rules_scala//testing:testing.bzl", - "setup_scala_testing_toolchain", -) +```py +load("@rules_scala//testing:testing.bzl", "setup_scala_testing_toolchain") ``` Attributes @@ -52,18 +50,15 @@ Attributes Examples (assumes maven deps are managed with rules_jvm_external): -#### ScalaTest (flat spec with must matchers) +### ScalaTest (flat spec with must matchers) -```starlark +```py # BUILD -load( - "@io_bazel_rules_scala//testing:testing.bzl", - "setup_scala_testing_toolchain", -) +load("@rules_scala//testing:testing.bzl", "setup_scala_testing_toolchain") setup_scala_testing_toolchain( name = "scalatest_toolchain", - scalatest_classpath = [ + scalatest_classpath = [ "@maven//:org_scalactic_scalactic_2_13", "@maven//:org_scalatest_scalatest_2_13", "@maven//:org_scalatest_scalatest_compatible", @@ -74,19 +69,19 @@ setup_scala_testing_toolchain( ], ) ``` + Register the toolchain -```starlark + +```py # WORKSPACE register_toolchains('//:scalatest_toolchain') ``` -#### JUnit 4 -```starlark +### JUnit 4 + +```py # BUILD -load( - "@io_bazel_rules_scala//testing:testing.bzl", - "setup_scala_testing_toolchain", -) +load("@rules_scala//testing:testing.bzl", "setup_scala_testing_toolchain") setup_scala_testing_toolchain( name = "junit_toolchain", @@ -96,21 +91,22 @@ setup_scala_testing_toolchain( ], ) ``` + Register the toolchain -```starlark + +```py # WORKSPACE register_toolchains('//:junit_toolchain') ``` -#### Specs2 +### Specs2 + For Specs2 rules to work, `junit_classpath`, `specs2_junit_classpath` and `specs2_classpath` must be configured. -```starlark + +```py # BUILD -load( - "@io_bazel_rules_scala//testing:testing.bzl", - "setup_scala_testing_toolchain", -) +load("@rules_scala//testing:testing.bzl", "setup_scala_testing_toolchain") setup_scala_testing_toolchain( name = "specs2_toolchain", @@ -128,10 +124,12 @@ setup_scala_testing_toolchain( "@maven//:org_specs2_specs2_junit_2_12", "@maven//:org_specs2_specs2_matcher_2_12", ] -) +) ``` + Register the toolchain -```starlark + +```py # WORKSPACE register_toolchains('//:specs2_toolchain') ``` diff --git a/docs/thrift_library.md b/docs/thrift_library.md index 94a3acb7e..8a992c491 100644 --- a/docs/thrift_library.md +++ b/docs/thrift_library.md @@ -1,7 +1,7 @@ # thrift_library -```python -load("@io_bazel_rules_scala//thrift:thrift.bzl", "thrift_library") +```py +load("@rules_scala//thrift:thrift.bzl", "thrift_library") thrift_library( name, srcs, @@ -70,4 +70,4 @@ It should be consumed by a thrift compiler like `scrooge_scala_library` in its ` -

List of labels; optional

- Allows to configure dependencies lists by configuring DepInfo provider targets. - Currently supported dep ids: scala_compile_classpath, - scala_library_classpath, scala_macro_classpath, scala_xml, - parser_combinators, + Allows to configure dependencies lists by configuring DepInfo provider targets. + Currently supported dep ids: scala_compile_classpath, + scala_library_classpath, scala_macro_classpath, scala_xml, + parser_combinators, semanticdb

List of strings; optional

- Extra compiler options for this binary to be passed to scalac. + Extra compiler options for this binary to be passed to scalac.

Boolean; optional (default False)

- Enables semanticdb output. + Enables semanticdb output.

\ No newline at end of file + diff --git a/docs/toolchain_development.md b/docs/toolchain_development.md index f4fbfeda8..5a2851ae4 100644 --- a/docs/toolchain_development.md +++ b/docs/toolchain_development.md @@ -2,37 +2,39 @@ ## Motivation and design patterns -Toolchains add extra layer of indirection to configure your rules. Toolchains can be used to provide -deps required for tool (eg. compiler) without hardcoding labels. Users can define toolchains by -using labels from their chosen loaders. This type of indirections allows to configure rules_scala -without using [problematic](https://github.com/bazelbuild/bazel/issues/1952) bind. +Toolchains add extra layer of indirection to configure your rules. Toolchains can be used to provide +deps required for tool (eg. compiler) without hardcoding labels. Users can define toolchains by +using labels from their chosen loaders. This type of indirections allows to configure rules_scala +without using [problematic](https://github.com/bazelbuild/bazel/issues/1952) bind. -### Patterns ## Dependency providers on toolchains for toolchain aware rules This is a default pattern and should be considered first when designing rules. This is a recommended pattern, when a rule implementation knows how to consume information from a toolchain. - + Dependencies are configured as a list of provider target labels on the toolchain. Toolchain aware rules will lookup providers by their ids. It is important for rule developers to take into -account good reporting when users misconfigure provider mappings. Example of dependency providers +account good reporting when users misconfigure provider mappings. Example of dependency providers configuration: -```starlark + +```py dep_providers = [ ":my_compile_deps_provider", ":my_runtime_deps_provider", ] ``` -Dep providers are instances of DepInfo. Make sure to load them using absolute name including +Dep providers are instances of DepInfo. Make sure to load them using absolute name including external repository name (otherwise they may be treated as different): -```starlark -load("@io_bazel_rules_scala//scala:providers.bzl", "DepsInfo") + +```py +load("@rules_scala//scala:providers.bzl", "DepsInfo") ``` `DepsInfo` provider targets can be declared using rule `declare_deps_provider`: -```starlark -load("@io_bazel_rules_scala//scala:providers.bzl", "declare_deps_provider") + +```py +load("@rules_scala//scala:providers.bzl", "declare_deps_provider") declare_deps_provider( name = "my_compile_deps_provider", @@ -46,13 +48,15 @@ declare_deps_provider( ], ) ``` + `deps_id` is an id used by rules to lookup for dep list defined by `DepsInfo` provider. Each rule may define their own ids. Toolchain can be declared using rule `declare_deps_toolchain` and then wired with a `toolchain_type` using `toolchain`: -```starlark -load("@io_bazel_rules_scala//scala/toolchains:toolchains.bzl", "declare_deps_toolchain") + +```py +load("@rules_scala//scala/toolchains:toolchains.bzl", "declare_deps_toolchain") declare_deps_toolchain( name = "my_deps_toolchain_impl", @@ -66,7 +70,7 @@ declare_deps_toolchain( toolchain( name = "my_deps_toolchain", toolchain = ":my_deps_toolchain_impl", - toolchain_type = "@io_bazel_rules_scala//my_rules/toolchain:my_toolchain_type", + toolchain_type = "//my_rules/toolchain:my_toolchain_type", visibility = ["//visibility:public"], ) ``` @@ -74,92 +78,98 @@ toolchain( ## Rules to export deps as targets to be depended on by other rules not aware of toolchains This pattern is used to pass dependencies to rules, which are not aware of particular toolchain. For -example, Scala compile classpath deps which are defined on Scala toolchains can be made available to -non scala rules by creating a toolchain aware rule to export deps. This pattern exports dependencies -from the toolchain defined by the previous pattern. This pattern introduces additional complexity -and only needs to be used when the regular toolchain consumption is not sufficient. This pattern -should be used only for internal implementation needs. +example, Scala compile classpath deps which are defined on Scala toolchains can be made available to +non scala rules by creating a toolchain aware rule to export deps. This pattern exports dependencies +from the toolchain defined by the previous pattern. This pattern introduces additional complexity +and only needs to be used when the regular toolchain consumption is not sufficient. This pattern +should be used only for internal implementation needs. ### Exporting deps via toolchains To define toolchain deps with deps exporting, the following steps need to be taken: + 1. Create rule exposing toolchain deps using infra helper `expose_toolchain_deps` -```starlark -load("//scala/private/toolchain_deps:toolchain_deps.bzl", "expose_toolchain_deps") - -def _toolchain_deps(ctx): - toolchain_type_label = "@io_bazel_rules_scala//my_rules/toolchain:my_toolchain_type" - return expose_toolchain_deps(ctx, toolchain_type_label) - -my_toolchain_deps = rule( - implementation = _toolchain_deps, - attrs = { - "deps_id": attr.string( - mandatory = True, - ), - }, - toolchains = ["@io_bazel_rules_scala//my_rules/toolchain:my_toolchain_type"], - incompatible_use_toolchain_transition = True, -) -``` + + ```py + load("//scala/private/toolchain_deps:toolchain_deps.bzl", "expose_toolchain_deps") + + def _toolchain_deps(ctx): + toolchain_type_label = "//my_rules/toolchain:my_toolchain_type" + return expose_toolchain_deps(ctx, toolchain_type_label) + + my_toolchain_deps = rule( + implementation = _toolchain_deps, + attrs = { + "deps_id": attr.string( + mandatory = True, + ), + }, + toolchains = ["//my_rules/toolchain:my_toolchain_type"], + incompatible_use_toolchain_transition = True, + ) + ``` 2. Declare dependency exporting targets -```starlark +```py my_toolchain_deps( name = "my_deps", deps_id = "my_compile_deps_provider", ) ``` - ## Reusable infra code to define toolchains for dependencies ### Reusable symbols -- provider `DepsInfo` - provider with a field `deps`, which contains dep list to be provided via -toolchain + +- provider `DepsInfo` - provider with a field `deps`, which contains dep list to + be provided via toolchain - rule `declare_deps_provider` - used to declare target with `DepsProvider`. Eg.: -```starlark -declare_deps_provider( - name = "my_runtime_deps_provider", - deps_id = "runtime_deps" - deps = ["@dep1", "@dep2"], - visibility = ["//visibility:public"], -) -``` -- rule `declare_deps_toolchain` - used to declare toolchains for deps providers. Eg.: -```starlark -declare_deps_toolchain( - name = "my_toolchain_impl", - dep_providers = [ - ":my_compile_deps_provider", - ":my_runtime_deps_provider", - ], - visibility = ["//visibility:public"], -) -``` -Attribute `dep_providers` is a list of DepInfo targets used for indirection in toolchain exporting -rules. + ```py + declare_deps_provider( + name = "my_runtime_deps_provider", + deps_id = "runtime_deps" + deps = ["@dep1", "@dep2"], + visibility = ["//visibility:public"], + ) + ``` + +- rule `declare_deps_toolchain` - used to declare toolchains for deps providers. Eg.: -- `def expose_toolchain_deps(ctx, toolchain_type_label)` - helper to export deps from a toolchain. + ```py + declare_deps_toolchain( + name = "my_toolchain_impl", + dep_providers = [ + ":my_compile_deps_provider", + ":my_runtime_deps_provider", + ], + visibility = ["//visibility:public"], + ) + ``` + +Attribute `dep_providers` is a list of DepInfo targets used for indirection in toolchain exporting +rules. + +- `def expose_toolchain_deps(ctx, toolchain_type_label)` - helper to export deps from a toolchain. Intended to be used when defining toolchain deps exporting rules. Eg.: -```starlark -load("//scala/private/toolchain_deps:toolchain_deps.bzl", "expose_toolchain_deps") - -def _toolchain_deps(ctx): - toolchain_type_label = "@io_bazel_rules_scala//my_rules/toolchain:my_toolchain_type" - return expose_toolchain_deps(ctx, toolchain_type_label) - -my_toolchain_deps = rule( - implementation = _toolchain_deps, - attrs = { - "deps_id": attr.string( - mandatory = True, - ), - }, - toolchains = ["@io_bazel_rules_scala//my_rules/toolchain:my_toolchain_type"], - incompatible_use_toolchain_transition = True, -) -``` + + ```py + load("//scala/private/toolchain_deps:toolchain_deps.bzl", "expose_toolchain_deps") + + def _toolchain_deps(ctx): + toolchain_type_label = "//my_rules/toolchain:my_toolchain_type" + return expose_toolchain_deps(ctx, toolchain_type_label) + + my_toolchain_deps = rule( + implementation = _toolchain_deps, + attrs = { + "deps_id": attr.string( + mandatory = True, + ), + }, + toolchains = ["//my_rules/toolchain:my_toolchain_type"], + incompatible_use_toolchain_transition = True, + ) + ``` diff --git a/examples/semanticdb/README.md b/examples/semanticdb/README.md index 8aa4301e7..0d784838d 100644 --- a/examples/semanticdb/README.md +++ b/examples/semanticdb/README.md @@ -1,13 +1,13 @@ # scala_rules Semanticdb example -This example demonstrates using an aspect to access the semanticdb info for one or more targets. In this example, an aspect is used to generate a json file that contains the semanticdb info that could be consumed by a consumer such as an IDE. +This example demonstrates using an aspect to access the semanticdb info for one or more targets. In this example, an aspect is used to generate a json file that contains the semanticdb info that could be consumed by a consumer such as an IDE. In this example, note that a scala_toolchain with enable_semanticdb=True is setup in the BUILD file. -This command can be used to run the aspect (and not run the full build) +This command can be used to run the aspect (and not run the full build) -``` +```txt bazel build //... --aspects aspect.bzl%semanticdb_info_aspect --output_groups=json_output_file ``` -The semanticdb_info.json file will be created for each target, and contains the semanticdb info for the target. \ No newline at end of file +The semanticdb_info.json file will be created for each target, and contains the semanticdb info for the target. diff --git a/jmh/toolchain/toolchain.bzl b/jmh/toolchain/toolchain.bzl index 70894e939..836b8b90d 100644 --- a/jmh/toolchain/toolchain.bzl +++ b/jmh/toolchain/toolchain.bzl @@ -2,10 +2,8 @@ load("//scala/private/toolchain_deps:toolchain_deps.bzl", "expose_toolchain_deps load("//scala:providers.bzl", "declare_deps_provider", _DepsInfo = "DepsInfo") load( "//scala:scala_cross_version.bzl", - "default_maven_server_urls", _versioned_repositories = "repositories", ) -load("//third_party/repositories:repositories.bzl", "repositories") load("@io_bazel_rules_scala_config//:config.bzl", "SCALA_VERSION") DEP_PROVIDERS = [ @@ -25,18 +23,6 @@ def jmh_artifact_ids(): "io_bazel_rules_scala_org_apache_commons_commons_math3", ] -def jmh_repositories( - maven_servers = default_maven_server_urls(), - overriden_artifacts = {}): - repositories( - scala_version = SCALA_VERSION, - for_artifact_ids = jmh_artifact_ids(), - fetch_sources = False, - maven_servers = maven_servers, - overriden_artifacts = overriden_artifacts, - ) - native.register_toolchains("@rules_scala_toolchains//jmh:all") - def _jmh_toolchain_impl(ctx): toolchain = platform_common.ToolchainInfo( dep_providers = ctx.attr.dep_providers, diff --git a/junit/junit.bzl b/junit/junit.bzl index d3abd193e..ddcb4c140 100644 --- a/junit/junit.bzl +++ b/junit/junit.bzl @@ -1,20 +1,5 @@ -load( - "//scala:scala_cross_version.bzl", - _default_maven_server_urls = "default_maven_server_urls", -) -load("//third_party/repositories:repositories.bzl", "repositories") - def junit_artifact_ids(): return [ "io_bazel_rules_scala_junit_junit", "io_bazel_rules_scala_org_hamcrest_hamcrest_core", ] - -def junit_repositories( - maven_servers = _default_maven_server_urls(), - fetch_sources = True): - repositories( - for_artifact_ids = junit_artifact_ids(), - fetch_sources = fetch_sources, - maven_servers = maven_servers, - ) diff --git a/manual_test/README.md b/manual_test/README.md index acaefadf9..c600ddef7 100644 --- a/manual_test/README.md +++ b/manual_test/README.md @@ -1 +1,3 @@ -This directory contains tests that require extra setup such as extra bazel flags. \ No newline at end of file +# Manual test collection + +This directory contains tests that require extra setup such as extra bazel flags. diff --git a/scala/private/macros/scala_repositories.bzl b/scala/private/macros/scala_repositories.bzl index 75c41b6d8..d6ca48393 100644 --- a/scala/private/macros/scala_repositories.bzl +++ b/scala/private/macros/scala_repositories.bzl @@ -1,12 +1,9 @@ -load("//scala:deps.bzl", "rules_scala_dependencies") load( "//scala:scala_cross_version.bzl", "extract_major_version", "extract_minor_version", "version_suffix", - _default_maven_server_urls = "default_maven_server_urls", ) -load("//third_party/repositories:repositories.bzl", "repositories") load("@bazel_tools//tools/build_defs/repo:http.bzl", "http_archive") load("@io_bazel_rules_scala_config//:config.bzl", "SCALA_VERSIONS") @@ -51,9 +48,6 @@ def _compiler_sources_repo_impl(rctx): compiler_sources_repo = repository_rule( implementation = _compiler_sources_repo_impl, - attrs = { - "scala_versions": attr.string_list(mandatory = True), - }, ) def _validate_scalac_srcjar(srcjar): @@ -130,19 +124,9 @@ def setup_scala_compiler_sources(srcjars = {}): for scala_version in SCALA_VERSIONS: dt_patched_compiler_setup(scala_version, srcjars.get(scala_version)) - compiler_sources_repo( - name = "scala_compiler_sources", - scala_versions = SCALA_VERSIONS, - ) + compiler_sources_repo(name = "scala_compiler_sources") -def rules_scala_setup(scala_compiler_srcjar = None): - rules_scala_dependencies() - setup_scala_compiler_sources({ - version: scala_compiler_srcjar - for version in SCALA_VERSIONS - }) - -def _artifact_ids(scala_version): +def scala_version_artifact_ids(scala_version): result = [ "io_bazel_rules_scala_scala_compiler", "io_bazel_rules_scala_scala_library", @@ -183,40 +167,3 @@ def _artifact_ids(scala_version): ]) return result - -def rules_scala_toolchain_deps_repositories( - maven_servers = _default_maven_server_urls(), - overriden_artifacts = {}, - fetch_sources = False, - validate_scala_version = True): - for scala_version in SCALA_VERSIONS: - repositories( - scala_version = scala_version, - for_artifact_ids = _artifact_ids(scala_version), - maven_servers = maven_servers, - fetch_sources = fetch_sources, - overriden_artifacts = overriden_artifacts, - validate_scala_version = validate_scala_version, - ) - -def scala_repositories( - maven_servers = _default_maven_server_urls(), - overriden_artifacts = {}, - load_dep_rules = True, - load_jar_deps = True, - fetch_sources = False, - validate_scala_version = True, - scala_compiler_srcjars = {}): - if load_dep_rules: - # When `WORKSPACE` goes away, so can this case. - rules_scala_dependencies() - - setup_scala_compiler_sources(scala_compiler_srcjars) - - if load_jar_deps: - rules_scala_toolchain_deps_repositories( - maven_servers, - overriden_artifacts, - fetch_sources, - validate_scala_version, - ) diff --git a/scala/scala.bzl b/scala/scala.bzl index 643b655d1..51864c0b2 100644 --- a/scala/scala.bzl +++ b/scala/scala.bzl @@ -2,12 +2,6 @@ load( "//specs2:specs2_junit.bzl", _specs2_junit_dependencies = "specs2_junit_dependencies", ) -load( - "//scala/private:macros/scala_repositories.bzl", - _rules_scala_setup = "rules_scala_setup", - _rules_scala_toolchain_deps_repositories = "rules_scala_toolchain_deps_repositories", - _scala_repositories = "scala_repositories", -) load( "//scala/private:macros/setup_scala_toolchain.bzl", _setup_scala_toolchain = "setup_scala_toolchain", @@ -70,9 +64,6 @@ scala_library_for_plugin_bootstrapping = _scala_library_for_plugin_bootstrapping scala_library_suite = _scala_library_suite scala_macro_library = _scala_macro_library scala_repl = _scala_repl -scala_repositories = _scala_repositories -rules_scala_setup = _rules_scala_setup -rules_scala_toolchain_deps_repositories = _rules_scala_toolchain_deps_repositories scala_test = _scala_test scala_test_suite = _scala_test_suite setup_scala_toolchain = _setup_scala_toolchain diff --git a/scala/scalafmt/scalafmt_repositories.bzl b/scala/scalafmt/scalafmt_repositories.bzl index 8b7ad417e..d73b49bc7 100644 --- a/scala/scalafmt/scalafmt_repositories.bzl +++ b/scala/scalafmt/scalafmt_repositories.bzl @@ -1,15 +1,5 @@ -load( - "//scala:scala_cross_version.bzl", - "extract_major_version", - "version_suffix", - _default_maven_server_urls = "default_maven_server_urls", -) -load( - "//scala_proto/default:repositories.bzl", - "SCALAPB_COMPILE_ARTIFACT_IDS", -) -load("//third_party/repositories:repositories.bzl", "repositories") -load("@io_bazel_rules_scala_config//:config.bzl", "SCALA_VERSIONS") +load("//scala:scala_cross_version.bzl", "extract_major_version") +load("//scala_proto/default:repositories.bzl", "SCALAPB_COMPILE_ARTIFACT_IDS") def _scalafmt_config_impl(repository_ctx): config_path = repository_ctx.attr.path @@ -30,9 +20,6 @@ scalafmt_config = repository_rule( }, ) -def scalafmt_default_config(path = ".scalafmt.conf", **kwargs): - scalafmt_config(name = "scalafmt_default", path = "//:" + path, **kwargs) - _SCALAFMT_DEPS = [ "com_lihaoyi_fansi", "com_typesafe_config", @@ -77,19 +64,3 @@ def scalafmt_artifact_ids(scala_version): extra_deps.append("io_bazel_rules_scala_scala_parallel_collections") return _SCALAFMT_DEPS + _SCALAFMT_DEPS_2_12 + extra_deps - -def scalafmt_repositories( - maven_servers = _default_maven_server_urls(), - overriden_artifacts = {}): - for scala_version in SCALA_VERSIONS: - repositories( - scala_version = scala_version, - for_artifact_ids = scalafmt_artifact_ids(scala_version), - maven_servers = maven_servers, - overriden_artifacts = overriden_artifacts, - ) - - native.register_toolchains(str(Label( - "@rules_scala_toolchains//scalafmt:scalafmt_toolchain" + - version_suffix(scala_version), - ))) diff --git a/scala/toolchains.bzl b/scala/toolchains.bzl index 0a9b5cc14..637bce98e 100644 --- a/scala/toolchains.bzl +++ b/scala/toolchains.bzl @@ -2,11 +2,15 @@ load("//jmh/toolchain:toolchain.bzl", "jmh_artifact_ids") load("//junit:junit.bzl", "junit_artifact_ids") -load("//scala/private:macros/scala_repositories.bzl", "scala_repositories") +load( + "//scala/private:macros/scala_repositories.bzl", + "scala_version_artifact_ids", + "setup_scala_compiler_sources", +) load( "//scala/scalafmt:scalafmt_repositories.bzl", "scalafmt_artifact_ids", - "scalafmt_default_config", + "scalafmt_config", ) load("//scala:scala_cross_version.bzl", "default_maven_server_urls") load("//scala:toolchains_repo.bzl", "scala_toolchains_repo") @@ -28,7 +32,6 @@ def _get_unknown_entries(entries, allowed_entries): def scala_toolchains( maven_servers = default_maven_server_urls(), overridden_artifacts = {}, - load_rules_scala_dependencies = True, load_scala_toolchain_dependencies = True, fetch_sources = False, validate_scala_version = True, @@ -44,18 +47,14 @@ def scala_toolchains( jmh = False, twitter_scrooge = False, twitter_scrooge_deps = {}): - """Instantiates @io_bazel_rules_scala_toolchains and all its dependencies. + """Instantiates @rules_scala_toolchains and all its dependencies. - Provides a unified interface to configuring rules_scala both directly in a + Provides a unified interface to configuring `rules_scala` both directly in a `WORKSPACE` file and in a Bazel module extension. - Instantiates the `@rules_scala_toolchains` repository. Under - `WORKSPACE`, you will need to call `register_toolchains` at some point. - Under Bzlmod, rules_scala does this automatically. - - ```starlark - register_toolchains("@rules_scala_toolchains//...:all") - ``` + Instantiates a repository containing all configured toolchains. Under + `WORKSPACE`, you will need to call `scala_register_toolchains()`. Under + Bzlmod, the `MODULE.bazel` file from `rules_scala` does this automatically. All arguments are optional. @@ -72,10 +71,6 @@ def scala_toolchains( ], } ``` - load_rules_scala_dependencies: whether load rules_scala repository - dependencies - load_scala_toolchain_dependencies: whether to load repository - dependencies of the core Scala language toolchain fetch_sources: whether to download dependency source jars validate_scala_version: whether to check if the configured Scala version matches the default version supported by rules_scala @@ -113,19 +108,11 @@ def scala_toolchains( if unknown_ts_deps: fail("unknown twitter_scrooge_deps:", ", ".join(unknown_ts_deps)) - scala_repositories( - maven_servers = maven_servers, - # Note the internal macro parameter misspells "overriden". - overriden_artifacts = overridden_artifacts, - load_dep_rules = load_rules_scala_dependencies, - load_jar_deps = load_scala_toolchain_dependencies, - fetch_sources = fetch_sources, - validate_scala_version = validate_scala_version, - scala_compiler_srcjars = scala_compiler_srcjars, - ) + setup_scala_compiler_sources(scala_compiler_srcjars) if scalafmt: - scalafmt_default_config(scalafmt_default_config_path) + scalafmt_conf_target = "//:" + scalafmt_default_config_path + scalafmt_config(name = "scalafmt_default", path = scalafmt_conf_target) if testing: scalatest = True @@ -163,7 +150,10 @@ def scala_toolchains( }) for scala_version in SCALA_VERSIONS: - version_specific_artifact_ids = {} + version_specific_artifact_ids = { + id: fetch_sources + for id in scala_version_artifact_ids(scala_version) + } if scala_proto: version_specific_artifact_ids.update({ @@ -186,6 +176,7 @@ def scala_toolchains( maven_servers = maven_servers, fetch_sources = fetch_sources, fetch_sources_by_id = all_artifacts, + # Note the internal macro parameter misspells "overriden". overriden_artifacts = overridden_artifacts, validate_scala_version = validate_scala_version, ) diff --git a/scala_proto/default/repositories.bzl b/scala_proto/default/repositories.bzl index f56f21d23..6653eeef0 100644 --- a/scala_proto/default/repositories.bzl +++ b/scala_proto/default/repositories.bzl @@ -1,7 +1,3 @@ -load("//scala:scala_cross_version.bzl", "default_maven_server_urls") -load("//third_party/repositories:repositories.bzl", "repositories") -load("@io_bazel_rules_scala_config//:config.bzl", "SCALA_VERSION") - # Needed by scalafmt SCALAPB_COMPILE_ARTIFACT_IDS = [ "com_google_protobuf_protobuf_java", @@ -60,19 +56,3 @@ def scala_proto_artifact_ids(scala_version): "scala_proto_rules_scalapb_protoc_bridge", "scala_proto_rules_scalapb_runtime_grpc", ] + SCALAPB_COMPILE_ARTIFACT_IDS + GUAVA_ARTIFACT_IDS - -def scala_proto_default_repositories( - maven_servers = default_maven_server_urls(), - scala_version = SCALA_VERSION, - overriden_artifacts = {}): - repositories( - scala_version = scala_version, - for_artifact_ids = scala_proto_artifact_ids(scala_version), - maven_servers = maven_servers, - fetch_sources = True, - overriden_artifacts = overriden_artifacts, - ) - - native.register_toolchains( - str(Label("//scala_proto:default_deps_toolchain")), - ) diff --git a/scala_proto/scala_proto.bzl b/scala_proto/scala_proto.bzl index 98aed2f67..73826a67d 100644 --- a/scala_proto/scala_proto.bzl +++ b/scala_proto/scala_proto.bzl @@ -1,4 +1,3 @@ -load("//scala_proto/default:repositories.bzl", "scala_proto_default_repositories") load( "//scala_proto/private:scala_proto.bzl", _make_scala_proto_library = "make_scala_proto_library", @@ -13,9 +12,6 @@ make_scala_proto_library = _make_scala_proto_library make_scala_proto_aspect = _make_scala_proto_aspect -def scala_proto_repositories(**kwargs): - scala_proto_default_repositories(**kwargs) - def scala_proto_library(**kwargs): _scala_proto_library(**kwargs) diff --git a/scala_proto/toolchains.bzl b/scala_proto/toolchains.bzl index a851cdfb2..866ee19f6 100644 --- a/scala_proto/toolchains.bzl +++ b/scala_proto/toolchains.bzl @@ -4,14 +4,6 @@ load( "scala_proto_toolchain", ) -def scala_proto_register_toolchains(): - native.register_toolchains(str(Label("//scala_proto:default_toolchain"))) - -def scala_proto_register_enable_all_options_toolchain(): - native.register_toolchains( - str(Label("//scala_proto:enable_all_options_toolchain")), - ) - def setup_scala_proto_toolchains(name, enable_all_options = False): """Used by @rules_scala_toolchains//scala_proto/BUILD. diff --git a/scalatest/scalatest.bzl b/scalatest/scalatest.bzl index 6af1e7a6a..66b57fb64 100644 --- a/scalatest/scalatest.bzl +++ b/scalatest/scalatest.bzl @@ -1,10 +1,3 @@ -load( - "//scala:scala_cross_version.bzl", - _default_maven_server_urls = "default_maven_server_urls", -) -load("//third_party/repositories:repositories.bzl", "repositories") -load("@io_bazel_rules_scala_config//:config.bzl", "SCALA_VERSIONS") - def scalatest_artifact_ids(): return [ "io_bazel_rules_scala_scalactic", @@ -24,14 +17,3 @@ def scalatest_artifact_ids(): "io_bazel_rules_scala_scalatest_shouldmatchers", "io_bazel_rules_scala_scalatest_wordspec", ] - -def scalatest_repositories( - maven_servers = _default_maven_server_urls(), - fetch_sources = True): - for scala_version in SCALA_VERSIONS: - repositories( - scala_version = scala_version, - for_artifact_ids = scalatest_artifact_ids(), - maven_servers = maven_servers, - fetch_sources = fetch_sources, - ) diff --git a/scripts/README.md b/scripts/README.md index 6a8c5fec6..cc3465c96 100644 --- a/scripts/README.md +++ b/scripts/README.md @@ -1,51 +1,66 @@ -# Update/create scala_x_x.bzl repository file script +# Development helper scripts -- [About](#about) -- [Usage](#usage) -- [Examples](#examples) -- [Debugging](#debugging) -- [Requirements](#requirements) +## [`create_repository.py`](./create_repository.py) -## About +Creates and updates `scala_x_x.bzl` files in +[`//third_party/repositories`](../third_party/repositories) that list toolchain +dependency Maven artifacts. -The script allows to update a certain scala_x_x.bzl file and its content -(artifact, sha, dependencies), by changing the value of `root_scala_version` -variable. +It uses [https://get-coursier.io/docs/](coursier) to **resolve** the transitive +dependencies of root artifacts and **fetch** their JARs. -It can be used to create non-existent file for chosen Scala version. +The script will not update any entry that: -It's using a [https://get-coursier.io/docs/](coursier) in order to **resolve** -lists the transitive dependencies of dependencies and **fetch** the JARs of it. +- already matches the version resolved by `cs` (executed by + `ArtifactResolver._fetch_artifact_data()`) +- has `"testonly": True` set +- has a version newer than the resolved version -## Usage +In other words, if the script doesn't see a need to update the artifact version, +it won't change it (the "if it ain't broke" principle). -Usage from the rules_scala root directory: +When it does update an artifact's entry, it will also set its `deps` field +accordingly, even if it didn't have one before. -```sh -./scripts/create_repository.py -``` +### Requirements -## Examples +Install [Coursier](https://get-coursier.io/) and +[Python 3](https://www.python.org/downloads/) before running the script. -Current value of `root_scala_versions`: +### Usage -```py -root_scala_versions = [ - "2.11.12", - "2.12.19", - "2.13.14", - "3.1.3", - "3.2.2", - "3.3.3", - "3.4.3", - "3.5.0", -] +Update the `ROOT_SCALA_VERSIONS` or other root artifact version constants at the top of the file, then run: + +```txt +./scripts/create_repository.py ``` -To **update** content of `scala_3_4.bzl` file: +You can also run it for a specific Scala version, or generate files in a +different directory: + +```txt +$ usage: create_repository.py [-h] [--version SCALA_VERSION] + [--output_dir OUTPUT_DIR] + +Creates or updates repository configuration files for different Scala +versions. + +options: + -h, --help show this help message and exit + --version SCALA_VERSION + Scala version for which to update repository + information; if not provided, updates all supported + versions: 2.11.12, 2.12.20, 2.13.15, 3.1.3, 3.2.2, + 3.3.5, 3.4.3, 3.5.2, 3.6.3 + --output_dir OUTPUT_DIR + Directory in which to generate or update repository + files (default: .../third_party/repositories) +``` + +To **update** the `scala_3_4.bzl` file: ```py -root_scala_versions = [ +ROOT_SCALA_VERSIONS = [ "2.11.12", "2.12.19", "2.13.14", @@ -54,13 +69,14 @@ root_scala_versions = [ "3.3.3", "3.4.4", # <- updated version "3.5.0" + "3.6.0" ] ``` -To **create** new `scala_3_6.bzl` file: +To **create** a new `scala_3_7.bzl` file: ```py -root_scala_versions = [ +ROOT_SCALA_VERSIONS = [ "2.11.12", "2.12.19", "2.13.14", @@ -69,38 +85,34 @@ root_scala_versions = [ "3.3.3", "3.4.3", "3.5.0", - "3.6.0", # <- new version + "3.6.0", + "3.7.0", # <- new version ] ``` -## Debugging +There are other variables after `ROOT_SCALA_VERSIONS` for the root artifacts +used to resolve all dependencies. -Certain dependency versions may not support a specific Scala versions, e.g., +If you need to add a new root artifact, or add constraints to existing ones, +edit the `select_root_artifacts()` function accordingly. -```py -kind_projector_version = "0.13.2" if scala_major < "2.13" else "0.13.3" -``` +### `testonly` artifacts -There may be situations in which the script won't work. To debug that problem -and adjust the values of hard-coded variables: +Artifacts marked as "testonly" are manually updated. The script will not change them. -```py -scalatest_major = "3" if scala_major >= "3.0" else scala_major -scalafmt_major = "2.13" if scala_major >= "3.0" else scala_major -kind_projector_version = "0.13.2" if scala_major < "2.13" else "0.13.3" -scalafmt_version = "2.7.5" if scala_major == "2.11" else SCALAFMT_VERSION -``` +### Update an existing entry without changing its version -there is an option to print the output of these two subprocesses: +To force an update of an artifact, while keeping its same version, remove its +existing entry from the `third_party/repositories/scala_*.bzl` file, and the +script will add it back. Alternatively, artificially set the entry to reference +an older artifact version. -```py - command = f'cs resolve {' '.join(root_artifacts)}' - output = subprocess.run( - command, capture_output=True, text=True, shell=True - ).stdout.splitlines() -``` +## [`sync_bazelversion.sh`](./sync-bazelversion.sh) -## Requirements +Synchronizes all of the `.bazelversion` files in the project with the top level +`.bazelversion`. -Install [Coursier](https://get-coursier.io/) and -[Python 3](https://www.python.org/downloads/) before running the script. +The [bazelisk](https://github.com/bazelbuild/bazelisk) wrapper for Bazel uses +`.bazelversion` files select a Bazel version. While `USE_BAZEL_VERSION` can +also override the Bazel version, keeping the `.bazelversion` files synchronized +helps avoid suprises when not using `USE_BAZEL_VERSION`. diff --git a/specs2/specs2.bzl b/specs2/specs2.bzl index 43f8fc80a..0440d031c 100644 --- a/specs2/specs2.bzl +++ b/specs2/specs2.bzl @@ -1,12 +1,3 @@ -load( - "//scala:scala_cross_version.bzl", - _default_maven_server_urls = "default_maven_server_urls", -) -load("//third_party/repositories:repositories.bzl", "repositories") - -def specs2_version(): - return "4.4.1" - def specs2_artifact_ids(): return [ "io_bazel_rules_scala_org_specs2_specs2_common", @@ -15,15 +6,5 @@ def specs2_artifact_ids(): "io_bazel_rules_scala_org_specs2_specs2_matcher", ] -def specs2_repositories( - maven_servers = _default_maven_server_urls(), - overriden_artifacts = {}): - repositories( - for_artifact_ids = specs2_artifact_ids(), - maven_servers = maven_servers, - fetch_sources = True, - overriden_artifacts = overriden_artifacts, - ) - def specs2_dependencies(): return [Label("//specs2:specs2")] diff --git a/specs2/specs2_junit.bzl b/specs2/specs2_junit.bzl index d6fbf53da..92928b861 100644 --- a/specs2/specs2_junit.bzl +++ b/specs2/specs2_junit.bzl @@ -1,33 +1,10 @@ -load( - "//specs2:specs2.bzl", - "specs2_dependencies", - "specs2_repositories", -) -load("//testing:junit.bzl", "junit_repositories") -load( - "//scala:scala_cross_version.bzl", - _default_maven_server_urls = "default_maven_server_urls", -) -load("//third_party/repositories:repositories.bzl", "repositories") +load("//specs2:specs2.bzl", "specs2_dependencies") def specs2_junit_artifact_ids(): return [ "io_bazel_rules_scala_org_specs2_specs2_junit", ] -def specs2_junit_repositories( - maven_servers = _default_maven_server_urls(), - overriden_artifacts = {}): - specs2_repositories(maven_servers) - junit_repositories() - - repositories( - for_artifact_ids = specs2_junit_artifact_ids(), - maven_servers = maven_servers, - fetch_sources = True, - overriden_artifacts = overriden_artifacts, - ) - def specs2_junit_dependencies(): return specs2_dependencies() + [ Label("//testing/toolchain:specs2_junit_classpath"), diff --git a/test/toolchains/jdk.bzl b/test/toolchains/jdk.bzl index be4729297..1f8b2dde9 100644 --- a/test/toolchains/jdk.bzl +++ b/test/toolchains/jdk.bzl @@ -52,6 +52,3 @@ def remote_jdk21_repositories(): ], version = "21", ) - -def remote_jdk21_toolchains(): - native.register_toolchains("//test/toolchains:java21_toolchain_definition") diff --git a/test_version/version_specific_tests_dir/scrooge_repositories.bzl b/test_version/version_specific_tests_dir/scrooge_repositories.bzl index c50b62a29..bfb85f292 100644 --- a/test_version/version_specific_tests_dir/scrooge_repositories.bzl +++ b/test_version/version_specific_tests_dir/scrooge_repositories.bzl @@ -1,3 +1,4 @@ +load("@io_bazel_rules_scala_config//:config.bzl", "SCALA_VERSION") load( "@rules_scala//scala:scala_cross_version.bzl", "default_maven_server_urls", @@ -11,9 +12,10 @@ load( "scala_toolchains_repo", ) load( - "@rules_scala//twitter_scrooge/toolchain:toolchain.bzl", - "twitter_scrooge", + "@rules_scala//scala/toolchains.bzl", + "twitter_scrooge_artifact_ids", ) +load("@rules_scala//third_party/repositories:repositories.bzl", "repositories") def _import_external(id, artifact, sha256, deps = [], runtime_deps = []): _scala_maven_import_external( @@ -98,7 +100,13 @@ def scrooge_repositories(version = None): ] } - twitter_scrooge(register_toolchains = False, **toolchain_deps) + repositories( + scala_version = SCALA_VERSION, + for_artifact_ids = twitter_scrooge_artifact_ids(**toolchain_deps), + maven_servers = default_maven_server_urls(), + fetch_sources = False, + ) + scala_toolchains_repo( name = "twitter_scrooge_test_toolchain", twitter_scrooge = True, diff --git a/testing/junit.bzl b/testing/junit.bzl deleted file mode 100644 index 2208675b9..000000000 --- a/testing/junit.bzl +++ /dev/null @@ -1,12 +0,0 @@ -load("//junit:junit.bzl", _repositories = "junit_repositories") -load("//scala:scala_cross_version.bzl", "version_suffix") -load("@io_bazel_rules_scala_config//:config.bzl", "SCALA_VERSIONS") - -def junit_repositories(): - _repositories() - -def junit_toolchain(): - for scala_version in SCALA_VERSIONS: - native.register_toolchains(str(Label( - "//testing:junit_toolchain" + version_suffix(scala_version), - ))) diff --git a/testing/scalatest.bzl b/testing/scalatest.bzl deleted file mode 100644 index 1d66fe50c..000000000 --- a/testing/scalatest.bzl +++ /dev/null @@ -1,12 +0,0 @@ -load("//scala:scala_cross_version.bzl", "version_suffix") -load("//scalatest:scalatest.bzl", _repositories = "scalatest_repositories") -load("@io_bazel_rules_scala_config//:config.bzl", "SCALA_VERSIONS") - -def scalatest_repositories(): - _repositories() - -def scalatest_toolchain(): - for scala_version in SCALA_VERSIONS: - native.register_toolchains(str(Label( - "//testing:scalatest_toolchain" + version_suffix(scala_version), - ))) diff --git a/testing/specs2_junit.bzl b/testing/specs2_junit.bzl deleted file mode 100644 index eed731317..000000000 --- a/testing/specs2_junit.bzl +++ /dev/null @@ -1,12 +0,0 @@ -load("//scala:scala_cross_version.bzl", "version_suffix") -load("//specs2:specs2_junit.bzl", _repositories = "specs2_junit_repositories") -load("@io_bazel_rules_scala_config//:config.bzl", "SCALA_VERSIONS") - -def specs2_junit_repositories(): - _repositories() - -def specs2_junit_toolchain(): - for scala_version in SCALA_VERSIONS: - native.register_toolchains(str(Label( - "//testing:specs2_junit_toolchain" + version_suffix(scala_version), - ))) diff --git a/third_party/README.md b/third_party/README.md index 54ee73139..d6207b5d0 100644 --- a/third_party/README.md +++ b/third_party/README.md @@ -1,5 +1,8 @@ +# Third party dependencies + This file lists license and version information of all code we did not author -# dependency_analyzer +## dependency_analyzer + dependency_analyzer scala compiler plugin is based on [classpath-shrinker](https://github.com/scalacenter/classpath-shrinker) plugin. -License: 3-clause revised BSD \ No newline at end of file +License: 3-clause revised BSD diff --git a/twitter_scrooge/toolchain/toolchain.bzl b/twitter_scrooge/toolchain/toolchain.bzl index b30e2067d..b48b18c96 100644 --- a/twitter_scrooge/toolchain/toolchain.bzl +++ b/twitter_scrooge/toolchain/toolchain.bzl @@ -3,13 +3,8 @@ load( "expose_toolchain_deps", ) load("//scala:providers.bzl", "DepsInfo", "declare_deps_provider") -load( - "//scala:scala_cross_version.bzl", - "version_suffix", - _default_maven_server_urls = "default_maven_server_urls", -) +load("//scala:scala_cross_version.bzl", "version_suffix") load("//scala_proto/default:repositories.bzl", "GUAVA_ARTIFACT_IDS") -load("//third_party/repositories:repositories.bzl", "repositories") load("@io_bazel_rules_scala_config//:config.bzl", "SCALA_VERSION") DEP_PROVIDERS = [ @@ -53,36 +48,6 @@ def twitter_scrooge_artifact_ids( return artifact_ids -def twitter_scrooge( - maven_servers = _default_maven_server_urls(), - overriden_artifacts = {}, - # These target labels need maven_servers to compute sensible defaults. - # Therefore we leave them None here. - libthrift = None, - scrooge_core = None, - scrooge_generator = None, - util_core = None, - util_logging = None, - register_toolchains = True): - repositories( - scala_version = SCALA_VERSION, - for_artifact_ids = twitter_scrooge_artifact_ids( - libthrift, - scrooge_core, - scrooge_generator, - util_core, - util_logging, - ), - maven_servers = maven_servers, - fetch_sources = False, - overriden_artifacts = overriden_artifacts, - ) - - if register_toolchains: - native.register_toolchains( - "@rules_scala_toolchains//twitter_scrooge:all", - ) - def _scrooge_toolchain_impl(ctx): toolchain = platform_common.ToolchainInfo( dep_providers = ctx.attr.dep_providers,