Support Bzlmod and add rules_scala to bazel-central-registry

[sfc-gh-pbennes]

Greetings!

It looks like rules_scala isn't part of the Bzlmod effort or added to bazel-central-registry yet. I've opened an issue both here and in https://github.com/bazelbuild/bazel-central-registry to request it: bazelbuild/bazel-central-registry#522

[chrislovecnm]

Just a note to anyone that starts the work native.register_toolchains is not supported when using bzlmod.

[sluongng]

@chrislovecnm it's still available here no? https://bazel.build/rules/lib/globals/module#register_toolchains

[chrislovecnm]

@sluongng When I last tried it, it did not work when using it inside an extension. In rules_python we are skipping the native call:

https://github.com/bazelbuild/rules_python/blob/36082079b514529d4009a2b104475fde1cdd5b30/python/repositories.bzl#L569

And build the toolchains here:

https://github.com/bazelbuild/rules_python/blob/36082079b514529d4009a2b104475fde1cdd5b30/python/extensions/private/pythons_hub.bzl#L56

We build the toolchains and use the following:

https://github.com/bazelbuild/rules_python/blob/36082079b514529d4009a2b104475fde1cdd5b30/MODULE.bazel#L49

[sluongng]

Yeah you would only want to prepare the toolchain (as a new repository) inside the extension I think. Registering it must be called from the MODULE.bazel file.

So native.register_toolchains call from the extension's starlark does not work, but having the extension create @pythons_hub//:all and then MODULE.bazel calling register_toolchains("@pythons_hub//:all") worked. 👍

[mateuszkuta256]

I prepared a minimal bzlmod support here and gonna split it into a few PRs
TODOs after merging it:

• write documentation on how to use rules_scala with bzlmod (some properties like SCALA_VERSION will probably be configurable via repo-env)
• prepare release notes
• contribute repo to the Bazel Central Registry
• add tests that the project builds using both WORKSPACE and MODULE.bazel
• migrate test/external workspaces to blzmod too

[pcj]

@mateuszkuta256 thanks for getting this started! Any progress updates or blockers?

[mateuszkuta256]

@mateuszkuta256 thanks for getting this started! Any progress updates or blockers?

hi, unfortunately I'm working on other things now and won't continue with this PR in the foreseeable future
I remember this migration was on hold because interfered with 'cross compilation support'
It looks like great progress was done it this area, so maybe someone can already take over the PR

[mbland]

I'd like to take on the task of Bzlmodifying this repo through a series of pull requests.

I've already created a (mostly) working branch in my own fork. Though I saw a couple draft pull requests here, I ended up taking a different approach and got it mostly working. In fact, I did exactly what @sluongng suggested in #1482 (comment). (I didn't notice this comment before just now—I might've read it, but not understood it at the time—but I did study rules_python and rules_go, and ended up doing exactly that.)

There are still issues I need help to address (recorded in some of the commit messages), I didn't strictly maintain WORKSPACE compatibility (fixable), and I only tested against Scala 2.13. But I can start teasing chunks out of it in a methodical fashion, to ensure that I maintain WORKSPACE and cross-version compatibility.

For an example of what it looks like from a client perspective, here's what the MODULE.bazel stanza from my local EngFlow/example repo looks like (as opposed to the current stanza, explanatory comment notwithstanding):

bazel_dep(name = "rules_scala", repo_name = "io_bazel_rules_scala")
local_path_override(
    module_name = "rules_scala",
    path = "../../bazelbuild/rules_scala"
)   

scala_dev_deps = use_extension(
    "@io_bazel_rules_scala//scala/extensions:deps.bzl",
    "scala_deps",
    dev_dependency = True,
)   
scala_dev_deps.toolchains(
    scalatest = True,
)

So if folks are game for me to do this, I'll start carving off pieces as separate pull requests, and we can resolve any outstanding problems in the process.

cc: @BillyAutrey @jayconrod @benjaminp @TheGrizzlyDev

[mbland]

A quick update for visibility: I'm very close to having the Bzlmodified rules_scala passing 100% of the tests, down to the last couple of failures:

• I need to replace the bind() calls from twitter_scrooge with alias() or some such to get test_version.sh to pass.
• I need to fix the Scala 3.1.2 and Scala 3.3.3 failures for the Scalafmt targets from test/shell/test_cross_build.sh (corresponding to the test3 and library3 test cases in test_scalafmt()).

I also need to get test_lint.sh working again, and update the dt_patches repos for Bzlmod. But the following all pass:

• test_rules_scala.sh
• test_reproducibility.sh
• test_examples.sh
• test_coverage.sh

Hopefully I can get these fixed up today, and I'll start peeling off the next pull request or two. And thanks to @simuons and @liucijus for reviewing and merging #1619 and #1620 already.

[mbland]

I've clearly been slacking on the weekly updates, but not on the actual work. 😉

Overview

Toolchainization is done! Now #1703 is out to document all the changes. It adds the new scala_toolchains() API documentation to the README and removes all the macros replaced by it. It also touches up all the docs to various degrees.

And after that pull request lands, the final Bzlmodification pull request awaits in my (new!) bzlmod-enable branch in mbland/rules_scala. (This is not the same as the existing bzlmod branch, though I used bzlmod as the starting point, and have merged bzlmod-enable back into it.)

Here's how I anticipate things going from here:

• Upgrade Scala 2.13 LTS to 2.13.16 #1700 (to enable Scala 2.13.16) needs to land.
• We can cancel Use rctx.name if rctx.original_name is empty #1702 if Bazel 8.1.1 (with Fall back to rctx.name if $original_name is empty bazel#25296) goes out first.
• After Update docs for version 7.0.0, remove old APIs #1703 lands, the maintainers could cut 7.0.0 without Bzlmod, then 7.1.0 with Bzlmod, or wait and cut 7.0.0 with Bzlmod. (See below.)
• Either way, I'm ready to cut a PR from bzlmod-enable to land Bzlmod immediately after Update docs for version 7.0.0, remove old APIs #1703, including Publish to BCR GitHub app configuration under .bcr/.
• Shortly after that, I'll file a pull request to add Bazel 8 compatibility, which should precipitate the 8.0.0 release very soon thereafter.

Merged

• Toolchainize //jmh/toolchain:jmh_toolchain_type #1691
• Bump Scalafmt from 3.8.3 to 3.8.6 #1692
• Toolchainize twitter_scrooge #1693
• Use rctx.original_name if available, Go 1.23.6 #1694
• Remove @io_bazel_rules_scala or replace with Label #1696
• Bump rules_go, Go, proto-google-common-protos #1698

Open

• Use rctx.name if rctx.original_name is empty #1702
• Update docs for version 7.0.0, remove old APIs #1703

Documentation review requested!

I'd appreciate it if anyone interested could take a look at the updated README in #1703 and comment on the pull request. Go to "Files changed" and use "View file" to see the new version fully rendered. I'd like to know whether the documentation on the new API, breaking changes, etc. makes sense, or if it can be improved in specific areas.

Bazel Central Registry publishing

@Wyverald asked privately whether we could post a preliminary version of the Bzlmodified rules_scala on the BCR for super early adopters. I ultimately defer to @simuons and @liucijus whether we should do this, or just wait for Bzlmodification to land.

If they give me the green light, I'd be happy to file a pull request to add a prerelease to the BCR. That said, bzlmod-enable already contains the Publish to BCR app template files. @simuons or @liucijus would have to set up the app per the Publish to BCR app instructions before cutting the release.

Trying Bzlmod before the BCR release via git_override()

As I've mentioned before, anyone can try my bzlmod and bzlmod-bazel-8 branches. I've been updating them and making sure they pass all tests for various Bazel/WORKSPACE/Bzlmod configurations. (The new bzlmod-enable branch is not guaranteed to remain stable; please don't depend on it!)

For examples, see:

• bazelbuild/rules_webtesting
• bazelbuild/intellij

That said, I last updated those branches this past Sunday, 2025-02-16. I plan to merge the latest changes, test them locally, and push updates to both by later today. (Look at the latest commit to see if I've done so yet.)

When to release 7.0.0?

There's some discussion in #1699 regarding when to release 7.0.0 and what to include. As always, @simuons and @liucijus have the ultimate say here, but I'm ready for either of the following, once #1703 lands:

• Releasing 7.0.0 without Bzlmod, so folks can migrate to scala_toolchains() and pick up other improvements right away, followed by 7.1.0 once the Bzlmodification changes land. This prioritizes releasing all the changes since v6.6.0 (2024-07-31) as soon as possible.

• Waiting for the Bzlmod changes to land before releasing 7.0.0. This prioritizes making a single release, though it will take extra time for the Bzlmodification pull request review.

The scala_toolchains() interface is very similar to the upcoming scala_deps module extension, so migrating to scala_toolchains() should introduce hardly any friction later. In fact, scala_deps basically collects information from its tag class instances to make a call to scala_toolchains() under the hood.

I honestly don't have a preference. It's totally up to whatever the maintainers decide.

Trying out Stardoc

I created bzlmod-enable from bzlmod so I could clean up, refactor, test, and document its module extension implementations in a single, fresh commit. After having done so, I'm now looking forward to experimenting with the Stardoc documentation generator.

If this experiment works out, including the stardoc Bazel module in MODULE.bazel will likely land in rules_scala 8.0.0, not 7.x. This is because the stardoc Bazel module requires Bazel 7 and above. Since rules_scala 7.x will maintain Bzlmod compatibility with Bazel 6.5.0, we can't include it in the 7.x MODULE.bazel file, not even as a dev_dependency.

The good news is, rules_scala 8.0.0 should be ready rather shortly after we release 7.x with Bzlmod, so the wait wouldn't be for long. (8.0.0 will drop Bzlmod support for Bazel 6.5.0.)

[dmivankov]

Looks like it doesn't yet resolve #1440
What's the bzlmod equivalent for rules_scala_toolchain_deps_repositories(overriden_artifacts=...)?

[mbland]

Last things first: Under Bzlmod, the equivalent of overriden_artifacts is overridden_artifacts (notice the new dd instead of the single d):

scala_deps = use_extension("//scala/extensions:deps.bzl", "scala_deps")

scala_deps.settings(overridden_artifacts = ...)

Under WORKSPACE:

load(
    "@rules_scala//scala/toolchains.bzl",
    "scala_register_toolchains",
    "scala_toolchains",
)

scala_toolchains(overridden_artifacts = ...)

Regarding #1440, I've done a lot of builds on my own machine, but I don't appear to have any sign of 2.13.6 lingering. But if I had to guess, this is because rules_scala defines a particular patch release for each major version at the top of its third_party/repositories/scala_*.bzl files. If you specify a different scala_version, you'll need to provide those overridden_artifacts yourself, as in #1440 (comment).

I'll add this to #1440 as well.

$ pushd "$(bazel info output_base)/external"

$ find *io_bazel_rules_scala_scala_library* -name '*.jar' 
+scala_deps+io_bazel_rules_scala_scala_library_2_12_20/scala-library-2.12.20-src.jar
+scala_deps+io_bazel_rules_scala_scala_library_2_12_20/scala-library-2.12.20.jar
+scala_deps+io_bazel_rules_scala_scala_library_2_13_15/scala-library-2.13.16.jar
+scala_deps+io_bazel_rules_scala_scala_library_2_13_15/scala-library-2.13.16-src.jar
+scala_deps+io_bazel_rules_scala_scala_library_2_3_1_3/scala-library-2.13.16.jar
+scala_deps+io_bazel_rules_scala_scala_library_2_3_1_3/scala-library-2.13.16-src.jar
+scala_deps+io_bazel_rules_scala_scala_library_2_3_3_5/scala-library-2.13.16.jar
+scala_deps+io_bazel_rules_scala_scala_library_2_3_3_5/scala-library-2.13.16-src.jar
+scala_deps+io_bazel_rules_scala_scala_library_2_3_6_3/scala-library-2.13.16.jar
+scala_deps+io_bazel_rules_scala_scala_library_2_3_6_3/scala-library-2.13.16-src.jar
+scala_deps+io_bazel_rules_scala_scala_library_3_1_3/scala3-library_3-3.1.3-src.jar
+scala_deps+io_bazel_rules_scala_scala_library_3_1_3/scala3-library_3-3.1.3.jar
+scala_deps+io_bazel_rules_scala_scala_library_3_3_5/scala3-library_3-3.3.5-src.jar
+scala_deps+io_bazel_rules_scala_scala_library_3_3_5/scala3-library_3-3.3.5.jar
+scala_deps+io_bazel_rules_scala_scala_library_3_6_3/scala3-library_3-3.6.3-src.jar
+scala_deps+io_bazel_rules_scala_scala_library_3_6_3/scala3-library_3-3.6.3.jar
_main~scala_deps~io_bazel_rules_scala_scala_library_2_11_12/scala-library-2.11.12.jar
_main~scala_deps~io_bazel_rules_scala_scala_library_2_11_12/scala-library-2.11.12-src.jar
_main~scala_deps~io_bazel_rules_scala_scala_library_2_12_20/scala-library-2.12.20-src.jar
_main~scala_deps~io_bazel_rules_scala_scala_library_2_12_20/scala-library-2.12.20.jar
_main~scala_deps~io_bazel_rules_scala_scala_library_2_13_15/scala-library-2.13.16.jar
_main~scala_deps~io_bazel_rules_scala_scala_library_2_13_15/scala-library-2.13.16-src.jar
_main~scala_deps~io_bazel_rules_scala_scala_library_2_3_1_3/scala-library-2.13.16.jar
_main~scala_deps~io_bazel_rules_scala_scala_library_2_3_1_3/scala-library-2.13.16-src.jar
_main~scala_deps~io_bazel_rules_scala_scala_library_2_3_3_5/scala-library-2.13.16.jar
_main~scala_deps~io_bazel_rules_scala_scala_library_2_3_3_5/scala-library-2.13.16-src.jar
_main~scala_deps~io_bazel_rules_scala_scala_library_2_3_6_3/scala-library-2.13.16.jar
_main~scala_deps~io_bazel_rules_scala_scala_library_2_3_6_3/scala-library-2.13.16-src.jar
_main~scala_deps~io_bazel_rules_scala_scala_library_3_1_3/scala3-library_3-3.1.3-src.jar
_main~scala_deps~io_bazel_rules_scala_scala_library_3_1_3/scala3-library_3-3.1.3.jar
_main~scala_deps~io_bazel_rules_scala_scala_library_3_3_5/scala3-library_3-3.3.5-src.jar
_main~scala_deps~io_bazel_rules_scala_scala_library_3_3_5/scala3-library_3-3.3.5.jar
_main~scala_deps~io_bazel_rules_scala_scala_library_3_6_3/scala3-library_3-3.6.3-src.jar
_main~scala_deps~io_bazel_rules_scala_scala_library_3_6_3/scala3-library_3-3.6.3.jar
io_bazel_rules_scala_scala_library_2_12_20/scala-library-2.12.20-src.jar
io_bazel_rules_scala_scala_library_2_12_20/scala-library-2.12.20.jar
io_bazel_rules_scala_scala_library_2_13_15/scala-library-2.13.16.jar
io_bazel_rules_scala_scala_library_2_13_15/scala-library-2.13.16-src.jar
io_bazel_rules_scala_scala_library_2_3_1_3/scala-library-2.13.16.jar
io_bazel_rules_scala_scala_library_2_3_1_3/scala-library-2.13.16-src.jar
io_bazel_rules_scala_scala_library_2_3_3_5/scala-library-2.13.16.jar
io_bazel_rules_scala_scala_library_2_3_3_5/scala-library-2.13.16-src.jar
io_bazel_rules_scala_scala_library_2_3_6_3/scala-library-2.13.16.jar
io_bazel_rules_scala_scala_library_2_3_6_3/scala-library-2.13.16-src.jar
io_bazel_rules_scala_scala_library_3_1_3/scala3-library_3-3.1.3-src.jar
io_bazel_rules_scala_scala_library_3_1_3/scala3-library_3-3.1.3.jar
io_bazel_rules_scala_scala_library_3_3_5/scala3-library_3-3.3.5-src.jar
io_bazel_rules_scala_scala_library_3_3_5/scala3-library_3-3.3.5.jar
io_bazel_rules_scala_scala_library_3_6_3/scala3-library_3-3.6.3-src.jar
io_bazel_rules_scala_scala_library_3_6_3/scala3-library_3-3.6.3.jar

(As for why io_bazel_rules_scala_scala_library_2_13_15 shows scala-library-2.13.16.jar, apparently I accidentally bumped it as part of #1692. D'oh! All the more reason to merge #1700.)

[dmivankov]

Thanks, I was trying scala_deps.toolchains( first, with

scala_deps.settings(
    overridden_artifacts = {
        "io_bazel_rules_scala_scala_library": {
           "artifact": "org.scala-lang:scala3-library_3:%s" % scala_version,
            "sha256": "26163f54674adb4c3e2151155f7f6f17095a0be10ebfb884ee8cdca98adca248",
        },
        "io_bazel_rules_scala_scala_compiler": {
            "artifact": "org.scala-lang:scala3-compiler_3:%s" % scala_version,
            "sha256": "f067da9a52ec450efa5d1cc1a3325b7f9882874a059b11728969460d170ed424",
        },
        "io_bazel_rules_scala_scala_library_2": {
            "artifact": "org.scala-lang:scala-library:2.13.15",
            "sha256": "8e4dbc3becf70d59c787118f6ad06fab6790136a0699cd6412bc9da3d336944e",
        },
    },
)

I get

(09:30:43) ERROR: Analysis of target ... failed; build aborted: in tag at .../MODULE.bazel:104:20, error converting value for attribute overridden_artifacts: expected value of type 'string' for dict value element, but got {"artifact": "org.scala-lang:scala3-library_3:3.3.5", "sha256": "26163f54674adb4c3e2151155f7f6f17095a0be10ebfb884ee8cdca98adca248"} (dict)

So either need to find a way to encode it into strings/list of strings, or a better-suiting bzlmod way of providing compiler.

Context is by the way trying to set scala_version = "3.3.5". Out of the box build fails with

...
  exception occurred while typechecking ...Something.scala

  An unhandled exception was thrown in the compiler.
  Please file a crash report here:
  https://github.com/lampepfl/dotty/issues/new/choose
  For non-enriched exceptions, compile with -Yno-enrich-error-messages.

     while compiling: ...Something.scala
        during phase: typer
                mode: Mode(ImplicitsEnabled)
     library version: version 2.13.16
    compiler version: version 3.3.5
...
Exception in thread "main" java.lang.AssertionError: assertion failed: failure to resolve inner class:
externalName = com.google.protobuf.Descriptors$EnumDescriptor,
outerName = com.google.protobuf.Descriptors,
innerName = EnumDescriptor
owner.fullName = com.google.protobuf.Descriptors
while parsing bazel-out/k8-fastbuild/bin/external/rules_jvm_external++maven+maven/com/google/cloud/google-cloud-spanner/6.86.0/header_google-cloud-spanner-6.86.0.jar(com/google/cloud/spanner/ValueBinder.class)
	at scala.runtime.Scala3RunTime$.assertFailed(Scala3RunTime.scala:8)
...

which I didn't try to minimize yet

[mbland]

Thanks, I was trying scala_deps.toolchains( first, with

scala_deps.settings(
    overridden_artifacts = {
        "io_bazel_rules_scala_scala_library": {
           "artifact": "org.scala-lang:scala3-library_3:%s" % scala_version,
            "sha256": "26163f54674adb4c3e2151155f7f6f17095a0be10ebfb884ee8cdca98adca248",
        },
        "io_bazel_rules_scala_scala_compiler": {
            "artifact": "org.scala-lang:scala3-compiler_3:%s" % scala_version,
            "sha256": "f067da9a52ec450efa5d1cc1a3325b7f9882874a059b11728969460d170ed424",
        },
        "io_bazel_rules_scala_scala_library_2": {
            "artifact": "org.scala-lang:scala-library:2.13.15",
            "sha256": "8e4dbc3becf70d59c787118f6ad06fab6790136a0699cd6412bc9da3d336944e",
        },
    },
)

I get

(09:30:43) ERROR: Analysis of target ... failed; build aborted: in tag at .../MODULE.bazel:104:20, error converting value for attribute overridden_artifacts: expected value of type 'string' for dict value element, but got {"artifact": "org.scala-lang:scala3-library_3:3.3.5", "sha256": "26163f54674adb4c3e2151155f7f6f17095a0be10ebfb884ee8cdca98adca248"} (dict)

So either need to find a way to encode it into strings/list of strings, or a better-suiting bzlmod way of providing compiler.

@dmivankov Crap, no, it's not you...it's me. You've found a legit bug in the upcoming scala_deps design. Thanks! :-)

We must not yet have any tests to exercise overridden_artifacts (or overriden_artifacts [sic], which is how it ultimately appears in repositories() from third_party/repositories/repositories.bzl). I'll add a test, fix this problem, and ping you when I've updated the bzlmod branch (and the bzlmod-bazel-8 branch).

To let you know what to expect, this is what the API will become (there will also be an optional deps attribute of type attr.label_list):

scala_deps.overridden_artifact(
    name = "io_bazel_rules_scala_scala_library",
    artifact = "org.scala-lang:scala3-library_3:%s" % scala_version,
    sha256 = "26163f54674adb4c3e2151155f7f6f17095a0be10ebfb884ee8cdca98adca248",
)
scala_deps.overridden_artifact(
    name = "io_bazel_rules_scala_scala_compiler",
    artifact = "org.scala-lang:scala3-compiler_3:%s" % scala_version,
    sha256 = "f067da9a52ec450efa5d1cc1a3325b7f9882874a059b11728969460d170ed424",
)       
scala_deps.overridden_artifact(
    name = "io_bazel_rules_scala_scala_library_2", 
    artifact = "org.scala-lang:scala-library:2.13.15",
    sha256 = "8e4dbc3becf70d59c787118f6ad06fab6790136a0699cd6412bc9da3d336944e",
)

This because Bazel does not have an attr type of dict (where the members are arbitrary objects, including nested dicts). This essentially forces module extensions to define a schema whereby repeated instances of a tag class represent individual dict entries. The module extension can then iterate over these instances to build a proper dict as needed.

I actually already did this for the previous compiler_srcjars attribute, via the compiler_srcjar tag class. (See dt_patches/test_dt_patches_user_srcjar/MODULE.bazel in my branch for an example.) So this isn't a particularly tricky or groundbreaking fix; I just hadn't realized the need for it before now.

[mbland]

@dmivankov OK, the new scala_deps.overridden_artifact tag is available in both the bzlmod and bzlmod-bazel-8 branches.

The only thing different from what I mentioned in #1482 (comment) is that the optional deps field is an attr.string_list, not an attr.label_list. (We could ultimately try to force it to become one, but attr.string_list works better with the underlying implementation in scala/scala_maven_import_external.bzl for now.)

I've also added examples/overridden_artifacts to test it (via test/shelll/test_examples.sh), and added more tests to test/shell/test_bzlmod_macros.sh to test the new repeated_tag_values() helper underlying its implementation.

[dmivankov]

Interesting, there's some other change then compared to 6.6.0 or master, build fails with

  exception occurred while typechecking Something.scala

  An unhandled exception was thrown in the compiler.
  Please file a crash report here:
  https://github.com/lampepfl/dotty/issues/new/choose
  For non-enriched exceptions, compile with -Yno-enrich-error-messages.

     while compiling: Something.scala
        during phase: typer
                mode: Mode(ImplicitsEnabled)
     library version: version 2.13.15
    compiler version: version 3.3.5
...
Exception in thread "main" java.lang.AssertionError: assertion failed: class Ticket has non-class parent: AppliedType(TypeRef(TermRef(TermRef(ThisType(TypeRef(NoPrefix,module class scalapb)),object package),lenses),Updatable),List(TypeRef(TermRef(ThisType(TypeRef(NoPrefix,module class acl)),object ticket),class Ticket)))
	at scala.runtime.Scala3RunTime$.assertFailed(Scala3RunTime.scala:8)
	at dotty.tools.dotc.core.SymDenotations$ClassDenotation.traverse$1(SymDenotations.scala:1993)
....

which was compiling before. Adding @maven//:com_thesamet_scalapb_lenses_3 as target dependency makes this one disappear and error becomes

Exception in thread "main" dotty.tools.dotc.core.MissingType: Cannot resolve reference to type fs2.type.Stream.
The classfile defining the type might be missing from the classpath.

repeating the cycle of adding deps that weren't previously required it ends up with

...
Error while emitting Something.scala
assertion failed: ClassBType.info not yet assigned: Lcats/Functor;
2 errors found
Exception in thread "main" io.bazel.rulesscala.scalac.ScalacWorker$CompilationFailed: Build failure with errors.
	at io.bazel.rulesscala.scalac.ScalacInvoker.invokeCompiler(ScalacInvoker.java:52)
	at io.bazel.rulesscala.scalac.ScalacWorker.compileScalaSources(ScalacWorker.java:272)
	at io.bazel.rulesscala.scalac.ScalacWorker.work(ScalacWorker.java:88)
	at io.bazel.rulesscala.worker.Worker.ephemeralWorkerMain(Worker.java:131)
	at io.bazel.rulesscala.worker.Worker.workerMain(Worker.java:51)
	at io.bazel.rulesscala.scalac.ScalacWorker.main(ScalacWorker.java:48)

So there's a difference in compiler versions or settings somewhere.

If it matters following customization is applied on toolchains level

load("@rules_scala//scala:scala_toolchain.bzl", "scala_toolchain")
l
scala_toolchain(
    name = "scala_toolchain_impl",
    dependency_mode = "transitive",
    dependency_tracking_method = "ast",
    strict_deps_mode = "error",
    unused_dependency_checker_mode = "off",
    visibility = ["//visibility:public"],
)

toolchain(
    name = "scala_toolchain",
    toolchain = "scala_toolchain_impl",
    toolchain_type = "@rules_scala//scala:toolchain_type",
    visibility = ["//visibility:public"],
)

plus some scalac options

    "-deprecation",  # Emit warning and location for usages of deprecated APIs.
    "-encoding",
    "utf-8",  # Specify character encoding used by source files.
    "-explaintypes",  # Explain type errors in more detail.
    "-explain",
    "-feature",  # Emit warning and location for usages of features that should be imported explicitly.
    "-language:existentials",  # Existential types (besides wildcard types) can be written and inferred
    "-language:experimental.macros",  # Allow macro definition (besides implementation and application)
    "-language:higherKinds",  # Allow higher-kinded types
    "-language:implicitConversions",  # Allow definition of implicit functions called views
    "-unchecked",  # Enable additional warnings where generated code depends on assumptions.
    "-Werror",  # Fail the compilation if there are any warnings.
    "-Ykind-projector",
    "-source:3.0-migration",
    "-Wconf:cat=deprecation:i",
    "-Xmax-inlines:128",

[mbland]

@dmivankov Perhaps @WojciechMazur can give you more insight into the compile error. At any rate, if you've got a standalone reproduction repository, it might help either Wojciech or me see what's happening.

I can say ScalaPB and protoc-bridge versions are sensitive to the version of protobuf, which has implications for rules_scala. See #1647 for the gory details. This is why there will be one release with Bzlmod, followed by another with Bazel 8 compatibility.

[dmivankov]

I will try to make a standalone reproduction or at least minimize the breaking case, will just take some time.. so if there's any ideas I could quickly try that's very welcome 🙂

[WojciechMazur]

The first errors seems like a typical errors related to missing entries on the classpath which is confirmed by being fixed after adding dependencies explicitly. The last error not yet assigned is something new, I've never seen it but I think it might also be related to missing entry for cats-core on the classpath.
What actually confuses me is fact that we get this kind of issue underdependency_mode = "transitive" which should provide all of the dependencies transitively.
dependency_tracking_method = "ast" should give us the dependencies used by macros (AFAIR) so it's really strange. Maybe there was some bug when porting Scala 2 logic.
Any way a standalone reproducer would be required. If possible let's move the further discussion to dedicated issue to not polute this thread any furhter

[crt-31]

I finally got a chance to review the changes and have a few questions and comments.

• First off, great work on this! Looks like a lot of great work was done.
• On version numbers… what is the versioning scheme that is trying to be followed? SemVer? Something else? (Unstable versioning and distribution structure #1496 implies first component is minimum bazel number).
• I assume ideally there could just be a rules_scala 7.x release that is compatible with both Bazel 7 & 8. Can you enumerate what the breaking issues are that are making it so that rules_scala needs both a 7.x and an 8.x branch?
• For what its worth, I think rules_scala 7.x could drop support for Bazel 6.
• I am unclear of the reason the built-in toolchains were put into a separate repo, can you expound on the reasoning?
• In my case, and I’d expect in many production systems, we specify specific scala version in scala_config, and therefore manage our own jars and setup custom toolchains. So in our case, we normally wouldn’t use scala_toolchains() or scala_deps at all ( I don’t think). But in this case, if I wanted to use say protobuf, could I use scala_toolchains() to setup just the protobuf toolchain even though I am using my own custom scala compiler toolchain?
• Nit: suggest changing the “testing” parameter in scala_toolchains() to “all_testing” or “testing_all” or something like that.
• Note that my biggest concern at the moment is keeping Workspace compatibility for the time being. (We are not in a position to go Bzlmod in our production just yet). That and ensuring all works in Windows. And being robust and reliable. And supporting using user-specific scala versions.
• One specific issue I ran into about setup_testing_toolchains See setup_scala_testing_toolchain() failing when passed xxx_classpath #1706

[mbland]

I finally got a chance to review the changes and have a few questions and comments.

• First off, great work on this! Looks like a lot of great work was done.

Thanks! 🙂

• On version numbers… what is the versioning scheme that is trying to be followed? SemVer? Something else? (Unstable versioning and distribution structure #1496 implies first component is minimum bazel number).

I presumed SemVer. That said, I've been looking at 7.x as fully supporting Bazel 7, and 8.x as fully supporting Bazel 8, with both supporting Bazel 6 (more so 7.x than 8.x) to ease migration. In my mind, it's good to support older Bazels with a path to adopting newer Bazels, rather than dropping older Bazel support altogether.

But as with all executive decisions, I defer to @simuons and @liucijus. I'm happy to support whichever option they choose. I wasn't aware of #1496, but as of now, it appears to be what I've just described, per #1482 (comment).

• I assume ideally there could just be a rules_scala 7.x release that is compatible with both Bazel 7 & 8. Can you enumerate what the breaking issues are that are making it so that rules_scala needs both a 7.x and an 8.x branch?

Theoretically, there could be one release; but like I said, @simuons, at least, seems to prefer having both 7.x and 8.x to enable users to migrate gradually up to Bazel 8 and Bzlmod. I've enumerated all the known breaking changes in the README.md updates in #1703. (You'll likely want to select "View file" from the context menu.)

The main reason has to do with whether a release maintains Bazel 6 support:

• 7.x: WORKSPACE and Bzlmod both work out of the box with Bazel 6, to support migration to Bazel 7 and Bzlmod in either order. Introduces the scala_toolchains() API (breaking the existing WORKSPACE API) and the Bzlmod API.
• 8.x: WORKSPACE will require explicit C++ compiler flags to work with Bazel 6, and Bzlmod won't work at all. scala_toolchains() and Bzlmod APIs remain the same.

The "Compatible Bazel versions" section of the updated README.md in #1703 shows this.

• For what its worth, I think rules_scala 7.x could drop support for Bazel 6.

I understand, but I also like the idea of giving folks who've been using Bazel 6 and WORKSPACE more of a gentle slope.

• I am unclear of the reason the built-in toolchains were put into a separate repo, can you expound on the reasoning?

It's the only way Bzlmod can work without being annoying as hell. Without a separate toolchain repo generated by a builtin module extension, you have to import all the builtin rules_scala toolchain repositories in your own MODULE.bazel file!

This is because, under WORKSPACE, users can import and call macros that instantiate repositories into the global scope. Under Bzlmod, users can't call macros from MODULE.bazel, so any repos your module needs must be instantiated by a module extension, which users access via use_extension(). That module extension has its own scope, so if a user needs to refer to a repo from that extension, they must use use_repo() to bring it into scope explicitly.

Without "toolchainization", as I call it, the user has to call use_repo() on all required toolchain dependencies explicitly, because otherwise the module providing the toolchains won't have access to these dependency repos. With "toolchainization", there's no need, because the module's extension creates the toolchain repo with all the required dependencies in scope. The module's own MODULE.bazel then calls use_repo() to bring the toolchain repo into scope and passes it to register_toolchains(), so you don't have to.

(Arguably, separate extensions could provide individual toolchains, generating separate toolchain repos. But that can quickly get annoying, and the module's own MODULE.bazel file can't call register_toolchains() for you. As implemented, there's one toolchain repo, but it has a varying number of packages providing optional toolchains based on the configuration. The magic of the @repo_name//...:all target ensures all configured repos get registered no matter what.)

#1633 (the first "toolchainization" change introducing scala_toolchains()) speaks to this, but to make it concrete, here's what I had to do in EngFlow/example to get v6.6.0 to work with Bzlmod (plus writing my own module extensions:):

http_archive(
    name = "io_bazel_rules_scala",
    sha256 = "e734eef95cf26c0171566bdc24d83bd82bdaf8ca7873bec6ce9b0d524bdaf05d",
    strip_prefix = "rules_scala-6.6.0",
    url = "https://github.com/bazelbuild/rules_scala/releases/download/v6.6.0/rules_scala-v6.6.0.tar.gz",
    patches = ["//scala:rules_scala-6.6.0.patch"],
    patch_args = ["-p1"],
)

# This constant matches the default Scala version from rules_scala for now.
SCALA_VERSION = "2.13.12"
SCALA_VERSIONS = [SCALA_VERSION]

scala_config = use_extension("//scala/extensions:config.bzl", "scala_config")
scala_config.settings(
    scala_version = SCALA_VERSION,
    scala_versions = SCALA_VERSIONS,
)
use_repo(
    scala_config,
    "io_bazel_rules_scala_config",
)

repos = [
    "io_bazel_rules_scala_scala_compiler",
    "io_bazel_rules_scala_scala_library",
    "io_bazel_rules_scala_scala_parser_combinators",
    "io_bazel_rules_scala_scala_reflect",
    "io_bazel_rules_scala_scala_xml",
    "io_bazel_rules_scala_scalactic",
    "io_bazel_rules_scala_scalatest",
    "io_bazel_rules_scala_scalatest_core",
    "io_bazel_rules_scala_scalatest_compatible",
    "io_bazel_rules_scala_scalatest_featurespec",
    "io_bazel_rules_scala_scalatest_flatspec",
    "io_bazel_rules_scala_scalatest_freespec",
    "io_bazel_rules_scala_scalatest_funspec",
    "io_bazel_rules_scala_scalatest_funsuite",
    "io_bazel_rules_scala_scalatest_matchers_core",
    "io_bazel_rules_scala_scalatest_mustmatchers",
    "io_bazel_rules_scala_scalatest_shouldmatchers",
]

toolchains = [
    "@io_bazel_rules_scala//scala:toolchain",
    "@io_bazel_rules_scala//testing:scalatest_toolchain",
]

scala_deps = use_extension("//scala/extensions:deps.bzl", "scala_deps")
[
    (
        [use_repo(scala_deps, repo + suffix) for repo in repos],
        [register_toolchains(toolchain + suffix) for toolchain in toolchains],
    )
    # The v.replace() expression mimics the logic to generate version specific
    # repo suffixes from rules_scala.
    for suffix in ["_" + v.replace(".", "_") for v in SCALA_VERSIONS]
]

I described this process in Migrating to Bazel Modules (a.k.a. Bzlmod) - Module Extensions. By contrast, once Bzlmodification lands, all of that gets replaced by this (which I have working in a local EngFlow/example repo, but using a local_path_override()):

bazel_dep(name = "rules_scala", version = "7.0.0")

scala_config = use_extension(
    "@rules_scala//scala/extensions:config.bzl", "scala_config"
)
scala_config.settings(scala_version = "2.13.15")

scala_deps = use_extension(
    "@rules_scala//scala/extensions:deps.bzl", "scala_deps"
)
scala_deps.toolchains(scalatest = True)

Note that I don't even have to call register_toolchains() on the builtin toolchains, because the MODULE.bazel file for rules_scala will do that for me. You can still use setup_scala_toolchain(), et. al., to create your custom toolchains (possibly using rules_jvm_external to provide the artifacts), and call register_toolchains() in your MODULE.bazel file so they take precedence over the builtin toolchains. But you don't need all the use_repo() and list comprehension business to just use the builtins.

• In my case, and I’d expect in many production systems, we specify specific scala version in scala_config, and therefore manage our own jars and setup custom toolchains. So in our case, we normally wouldn’t use scala_toolchains() or scala_deps at all ( I don’t think). But in this case, if I wanted to use say protobuf, could I use scala_toolchains() to setup just the protobuf toolchain even though I am using my own custom scala compiler toolchain?

I believe so, yes. scala_toolchains() will always register the builtin toolchain for the configured Scala version, but defining and registering your own will still cause yours to take precedence. That said, just calling scala_toolchains(scala_proto = True) (or, under Bzlmod, scala_deps.toolchains(scala_proto = True)) will instantiate and register the scala_proto toolchain specifically, not the other optional toolchains.

(Now, when it comes to protobuf, that's a big reason for the proposed 7.x + 8.x release schema, because of the interplay between protobuf, Bazel, rules_java, the ScalaPB suite, and protoc-bridge. See that aforementioned matrix for "Compatible Bazel versions" in #1703 as well as #1647.)

• Nit: suggest changing the “testing” parameter in scala_toolchains() to “all_testing” or “testing_all” or something like that.

I don't have a strong preference, but am happy to change it if @simuons or @liucijus prefer.

• Note that my biggest concern at the moment is keeping Workspace compatibility for the time being. (We are not in a position to go Bzlmod in our production just yet). That and ensuring all works in Windows. And being robust and reliable. And supporting using user-specific scala versions.

Glad to hear all the WORKSPACE compatibility work will pay off for you! 🙂 After all, the build output should theoretically be the same (or as close as possible) between WORKSPACE and Bzlmod. The scala_toolchains() API and the scala_deps module extension API are designed to be very similar, and the module extension calls scala_toolchains() under the hood. It's mainly just a much improved configuration and maintenance experience.

(BTW, in case you or anyone reading didn't know, you can still use 'WORKSPACE' and Bzlmod together for the time being. You can actually keep your WORKSPACE in place and move pieces to MODULE.bazel piecemeal. But the clock is ticking, because Bazel 9 (later this year) will drop WORKSPACE support.)

And for one more shameless plug, I've written a blog series on solving Bzlmod migration issues, in case that's helpful. (I'm due to post another soon, which I've mostly drafted.)

• One specific issue I ran into about setup_testing_toolchains See setup_scala_testing_toolchain() failing when passed xxx_classpath #1706

Yes, thanks for catching that. Definitely a blocker. I'm on it!

[simuons]

Nit: suggest changing the “testing” parameter in scala_toolchains() to “all_testing” or “testing_all” or something like that.

I'd suggest to remove this attribute. If its sole purpose is to set other three to True then there is not much benefit in it.

[mbland]

Nit: suggest changing the “testing” parameter in scala_toolchains() to “all_testing” or “testing_all” or something like that.

I'd suggest to remove this attribute. If its sole purpose is to set other three to True then there is not much benefit in it.

@simuons Done in c789cb0 from #1703.