Skip to content

Add toolchain options API to WORKSPACE and Bzlmod #1730

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 5 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 0 additions & 1 deletion BUILD
Original file line number Diff line number Diff line change
@@ -1 +0,0 @@
exports_files([".scalafmt.conf"])
2 changes: 1 addition & 1 deletion MODULE.bazel
Original file line number Diff line number Diff line change
Expand Up @@ -95,7 +95,6 @@ use_repo(
"rules_scala_toolchains",
"scala_compiler_sources",
)
scala_deps.scala()

# Register some of our testing toolchains first when building our repo.
register_toolchains(
Expand All @@ -119,6 +118,7 @@ dev_deps = use_extension(
"scala_deps",
dev_dependency = True,
)
dev_deps.scala()
dev_deps.jmh()
dev_deps.junit()
dev_deps.scala_proto()
Expand Down
29 changes: 23 additions & 6 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -930,19 +930,36 @@ parameter list, which is almost in complete correspondence with parameters from
the previous macros. The `WORKSPACE` files in this repository also provide many
examples.

### Replacing toolchain registration macros in `WORKSPACE`
### Replacing toolchain registration macros

Almost all `rules_scala` toolchains configured using `scala_toolchains()` are
automatically registered by `scala_register_toolchains()`. There are two
toolchain macro replacements that require special handling.
automatically registered by `scala_register_toolchains()`. The same is true for
toolchains configured using the `scala_deps` module extension under Bzlmod.
There are two toolchain macro replacements that require special handling.

The first is replacing `scala_proto_register_enable_all_options_toolchain()`
with the following `scala_toolchains()` parameters:
with the following:

```py
# MODULE.bazel

scala_deps.scala_proto(
"default_gen_opts" = [
"flat_package",
"grpc",
"single_line_to_proto_string",
],
)

# WORKSPACE
scala_toolchains(
scala_proto = True,
scala_proto_options = [],
scala_proto = {
"default_gen_opts": [
"flat_package",
"grpc",
"single_line_to_proto_string",
],
},
)
```

Expand Down
34 changes: 15 additions & 19 deletions docs/phase_scalafmt.md
Original file line number Diff line number Diff line change
Expand Up @@ -71,25 +71,21 @@ bazel run <TARGET>.format-test

to check the format (without modifying source code).

The extension provides a default configuration, but there are 2 ways to use
a custom configuration:

- Put `.scalafmt.conf` at the root of your workspace
- Pass `.scalafmt.conf` in via `scala_toolchains`:

```py
# MODULE.bazel
scala_deps.scalafmt(
default_config = "//path/to/my/custom:scalafmt.conf",
)

# WORKSPACE
scala_toolchains(
# Other toolchains settings...
scalafmt = True,
scalafmt_default_config = "//path/to/my/custom:scalafmt.conf",
)
```
The extension provides a default configuration. To use a custom configuration,
pass its path or target Label to the toolchain configuration:

```py
# MODULE.bazel
scala_deps.scalafmt(
default_config = "path/to/my/custom/scalafmt.conf",
)

# WORKSPACE
scala_toolchains(
# Other toolchains settings...
scalafmt = {"default_config": "path/to/my/custom/scalafmt.conf"},
)
```

When using Scala 3, you must append `runner.dialect = scala3` to
`.scalafmt.conf`.
Expand Down
30 changes: 24 additions & 6 deletions docs/scala_proto_library.md
Original file line number Diff line number Diff line change
@@ -1,22 +1,40 @@
# scala_proto_library

To use this rule, you'll first need to add the following to your `WORKSPACE` file,
which adds a few dependencies needed for ScalaPB:
To use this rule, add the following configuration, which adds the dependencies
needed for ScalaPB:

```py
scala_toolchains(
# Other toolchains settings...
scala_proto = True,
scala_proto_options = [
# MODULE.bazel
scala_deps.scala_proto(
"default_gen_opts" = [
"grpc",
"flat_package",
"scala3_sources",
],
)
```

```py
# WORKSPACE
scala_toolchains(
# Other toolchains settings...
scala_proto = {
"default_gen_opts": [
"grpc",
"flat_package",
"scala3_sources",
],
},
)

scala_register_toolchains()
```

See the __scalapbc__ column of the [__ScalaPB: SBT Settings > Additional options
to the generator__](
https://scalapb.github.io/docs/sbt-settings#additional-options-to-the-generator
) table for `default_gen_opts` values.

Then you can import `scala_proto_library` in any `BUILD` file like this:

```py
Expand Down
99 changes: 51 additions & 48 deletions scala/extensions/deps.bzl
Original file line number Diff line number Diff line change
Expand Up @@ -14,8 +14,8 @@ Provides the `scala_deps` module extension with the following tag classes:
- `twitter_scrooge`
- `jmh`

For documentation, see the `_tag_classes` dict, and the `_<TAG>_attrs` dict
corresponding to each `<TAG>` listed above.
For documentation, see the `_{general,toolchain}_tag_classes` dicts and the
`_<TAG>_attrs` dict corresponding to each `<TAG>` listed above.

See the `scala/private/macros/bzlmod.bzl` docstring for a description of
the defaults, attrs, and tag class dictionaries pattern employed here.
Expand All @@ -27,6 +27,7 @@ load(
"root_module_tags",
"single_tag_values",
)
load("//scala/private:toolchain_defaults.bzl", "TOOLCHAIN_DEFAULTS")
load("//scala:scala_cross_version.bzl", "default_maven_server_urls")
load("//scala:toolchains.bzl", "scala_toolchains")

Expand Down Expand Up @@ -89,35 +90,26 @@ _compiler_srcjar_attrs = {
"integrity": attr.string(),
}

_scalafmt_defaults = {
"default_config": "//:.scalafmt.conf",
}
_scalafmt_defaults = TOOLCHAIN_DEFAULTS["scalafmt"]

_scalafmt_attrs = {
"default_config": attr.label(
default = _scalafmt_defaults["default_config"],
doc = "The default config file for Scalafmt targets",
allow_single_file = True,
),
}

_scala_proto_defaults = {
"options": [],
}
_scala_proto_defaults = TOOLCHAIN_DEFAULTS["scala_proto"]

_scala_proto_attrs = {
"options": attr.string_list(
default = _scala_proto_defaults["options"],
"default_gen_opts": attr.string_list(
default = _scala_proto_defaults["default_gen_opts"],
doc = "Protobuf options, like 'scala3_sources' or 'grpc'",
),
}

_twitter_scrooge_defaults = {
"libthrift": None,
"scrooge_core": None,
"scrooge_generator": None,
"util_core": None,
"util_logging": None,
}
_twitter_scrooge_defaults = TOOLCHAIN_DEFAULTS["twitter_scrooge"]

_twitter_scrooge_attrs = {
k: attr.label(default = v)
Expand Down Expand Up @@ -186,39 +178,51 @@ _toolchain_tag_classes = {
),
}

_tag_classes = _general_tag_classes | _toolchain_tag_classes
def _toolchain_settings(module_ctx, tags, tc_names, toolchain_defaults):
"""Configures all builtin toolchains enabled throughout the module graph.

Configures toolchain options for enabled toolchains that support them based
on the root module's settings for each toolchain. In other words, it uses:

def _toolchains(mctx):
result = {k: False for k in _toolchain_tag_classes}
- the root module's tag class settings, if present; and
- the default tag class settings otherwise.

for mod in mctx.modules:
values = {tc: len(getattr(mod.tags, tc)) != 0 for tc in result}
This avoids trying to reconcile different toolchain settings across the
module graph. Non root modules that require specific settings should either:

if mod.is_root:
return values
- publish their required toolchain settings, or
- define and register a custom toolchain instead.

# Don't overwrite `True` values with `False` from another tag.
result.update({k: v for k, v in values.items() if v})
Args:
module_ctx: the module context object
tags: a tags object, presumably the result of `root_module_tags()`
tc_names: names of all supported toolchains
toolchain_defaults: a dict of `{toolchain_name: default options dict}`

return result
Returns:
a dict of `{toolchain_name: bool or options dict}` to pass as keyword
arguments to `scala_toolchains()`
"""
toolchains = {k: False for k in tc_names}

def _scala_proto_options(mctx):
result = {}
for mod in module_ctx.modules:
values = {tc: len(getattr(mod.tags, tc)) != 0 for tc in toolchains}

for mod in mctx.modules:
for tag in mod.tags.scala_proto:
result.update({opt: True for opt in tag.options})
# Don't overwrite True values with False from another tag.
toolchains.update({k: v for k, v in values.items() if v})

return sorted(result.keys())
for tc, defaults in toolchain_defaults.items():
if toolchains[tc]:
values = single_tag_values(module_ctx, getattr(tags, tc), defaults)
toolchains[tc] = {k: v for k, v in values.items() if v != None}

return toolchains

_tag_classes = _general_tag_classes | _toolchain_tag_classes

def _scala_deps_impl(module_ctx):
tags = root_module_tags(module_ctx, _tag_classes.keys())
scalafmt = single_tag_values(module_ctx, tags.scalafmt, _scalafmt_defaults)
scrooge_deps = single_tag_values(
module_ctx,
tags.twitter_scrooge,
_twitter_scrooge_defaults,
)
tc_names = [tc for tc in _toolchain_tag_classes]

scala_toolchains(
overridden_artifacts = repeated_tag_values(
Expand All @@ -229,13 +233,9 @@ def _scala_deps_impl(module_ctx):
tags.compiler_srcjar,
_compiler_srcjar_attrs,
),
scala_proto_options = _scala_proto_options(module_ctx),
# `None` breaks the `attr.string_dict` in `scala_toolchains_repo`.
twitter_scrooge_deps = {k: v for k, v in scrooge_deps.items() if v},
**(
single_tag_values(module_ctx, tags.settings, _settings_defaults) |
{"scalafmt_%s" % k: v for k, v in scalafmt.items()} |
_toolchains(module_ctx)
_toolchain_settings(module_ctx, tags, tc_names, TOOLCHAIN_DEFAULTS)
)
)

Expand All @@ -244,23 +244,26 @@ scala_deps = module_extension(
tag_classes = _tag_classes,
doc = """Selects and configures builtin toolchains.

If the root module explicitly uses the extension, it assumes responsibility for
selecting all required toolchains by insantiating the corresponding tag classes:
Modules throughout the dependency graph can enable a builtin toolchain by
instantiating its corresponding tag class. The root module controls the
configuration of all toolchains it directly enables. Any other builtin
toolchain required by other modules will use that toolchain's default
configuration values.

```py
scala_deps = use_extension(
"@rules_scala//scala/extensions:deps.bzl",
"scala_deps",
)
scala_deps.scala()
scala_deps.scala_proto()
scala_deps.scala_proto(default_gen_opts = ["grpc", "scala3_sources"])

dev_deps = use_extension(
"@rules_scala//scala/extensions:deps.bzl",
"scala_deps",
dev_dependency = True,
)
dev_deps.scalafmt()
dev_deps.scalafmt(default_config = "path/to/scalafmt.conf")
dev_deps.scalatest()

# And so on...
Expand Down
20 changes: 20 additions & 0 deletions scala/private/toolchain_defaults.bzl
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
"""Gathers defaults for toolchain macros in one place.

Used by both //scala:toolchains.bzl and //scala/extensions:deps.bzl.
"""

load(
"//scala/scalafmt/toolchain:setup_scalafmt_toolchain.bzl",
_scalafmt = "TOOLCHAIN_DEFAULTS",
)
load("//scala_proto:toolchains.bzl", _scala_proto = "TOOLCHAIN_DEFAULTS")
load(
"//twitter_scrooge/toolchain:toolchain.bzl",
_twitter_scrooge = "TOOLCHAIN_DEFAULTS",
)

TOOLCHAIN_DEFAULTS = {
"scalafmt": _scalafmt,
"scala_proto": _scala_proto,
"twitter_scrooge": _twitter_scrooge,
}
7 changes: 7 additions & 0 deletions scala/scalafmt/toolchain/setup_scalafmt_toolchain.bzl
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,13 @@ load("//scala:providers.bzl", "declare_deps_provider")
load("//scala:scala_cross_version.bzl", "version_suffix")
load("@rules_scala_config//:config.bzl", "SCALA_VERSIONS")

TOOLCHAIN_DEFAULTS = {
# Used by `scala_toolchains{,_repo}` to generate
# `@rules_scala_toolchains//scalafmt:config`, the default config for
# `ext_scalafmt` from `phase_scalafmt_ext.bzl`.
"default_config": Label("//:.scalafmt.conf"),
}

def setup_scalafmt_toolchain(
name,
scalafmt_classpath,
Expand Down
Loading