Bazel 8 + rules_java 8 updates, protoc toolchain (#1710)

Bumps dependencies to versions that are compatible with both Bazel 7.6.0
and 8.0.0, and adds protocol compiler toolchainization in `//protoc` for
`protobuf` v29 and later.

Closes #1652. Part of #1482.

- ScalaPB jars: 0.11.17 => 1.0.0-alpha.1
- `rules_python`: 0.38.0 => 1.2.0
- `rules_cc`: 0.0.9 => 0.1.1
- `rules_java`: 7.12.4 => 8.11.0
- `protobuf`: 21.7 => 30.1
- `rules_proto`: 6.0.2 => 7.1.0

Bazel 6 is officially unsupported as of this change and the upcoming
`rules_scala` 7.0.0 release. Updates `.bazelci/presubmit.yml` to bump
the `7.x` build to `last_rc`.

Registers a precompiled protocol compiler toolchain when
`--incompatible_enable_proto_toolchain_resolution` is `True`.
Otherwise, `register_toolchains("@rules_scala_protoc_toolchains//:all")`
is a no-op, as it will be empty.

`scripts/update_protoc_integrity.py` automatically updates
`scala/private/protoc/protoc_integrity.bzl`. The `protobuf` patch is the
`git diff` output from protocolbuffers/protobuf#19679, which also
inspired the updates to `scala_proto/scala_proto_toolchain.bzl`. The
`proto_lang_toolchain` call in the `BUILD` file generated by
`protoc/private/protoc_toolchains.bzl` was inspired by the `README`
from:

- https://github.com./aspect-build/toolchains_protoc/

Loads `java_proto_library` from `com_google_protobuf`, replacing the
officially deprecated version from `rules_java`.

Adds the `scala` parameter to `scala_toolchains()` to control whether it
instantiates the builtin Scala toolchains. Removes the `if
len(toolchains) == 0` check from `_scala_toolchains_repo_impl`. The
Scala version check will now happen only when both `scala` and
`validate_scala_version` are `True`, which is essentially how the
previous API worked.

Updates to `README.md`, and updates to `WORKSPACE` and
`third_party/repositories` files precipitated by the dependency updates,
comprise the remainder of this change.

---

We're no longer planning to support Bazel 6 in the next major release
per @simuons's decision in:

- #1482 (comment)

The plan is now to land the Bazel 7 and 8 compatibility updates first,
then land the Bzlmod change. This enables us to make only one new major
version release, instead of two (whereby the first release would've
continued supporting Bazel 6).

It turns out the two major version plan wouldn't've been possible.
Bazel 8 and `rules_java` 8 require `protobuf` >= v29, but this bump
caused Windows builds to break when compiling `protoc` in #1710.
`src/google/protobuf/compiler/java/java_features.pb.h`, the path
specified in the error message, doesn't exist until `protobuf` v25.0.

@crt-31 and I found that this was related to the Windows/MSVC 260
character file path length limit.  What's more, the `protobuf` team
plans to drop MSVC support specifically because of this path length
limit.

The protocol compiler toolchain prevents `protoc` recompilation, which
fixes the Windows breakage while making all builds faster. Since Windows
builds break since at least `protobuf` v25, but `protoc`
toolchainization requires v29, the version bump and the `protoc`
toolchain must land together.

Disabling the default Scala toolchain via `scala_toolchains(scala =
False)` avoids instantiating any builtin compiler JAR repos or
validating the Scala version. This enables users defining custom Scala
toolchains using their own JARs to still use other builtin toolchains.
This was prompted by:
#1710 (comment)

Removing the `if len(toolchains) == 0` covers the case in the upcoming
Bzlmod implementation whereby the root module may explicitly disable all
builtin toolchains. This avoids potential breakage of the
`register_toolchains("@rules_scala_toolchains//...:all")` call from the
upcoming `MODULE.bazel` file. Removing the `scala_register_toolchains()`
calls from the `dt_patches/test_dt_patches*/WORKSPACE` files proves that
those calls were harmless, but ultimately unnecessary.

---

I tried several things to get protocol compiler toolchainization to work
with `protobuf` v28.2, described below. However, each path only led to
the same suffering described in the new "Why this requires `protobuf`
v29 or later" section of the README.

I discovered along the way that `protobuf` v30 isn't compatible with
Bazel 6.5.0 at all. I added an explanation to the "Limited Bazel 6.5.0
compatibility" section of `README.md`.

---

I experimented with using `protobuf` v28.2, `rules_proto` 6.0.2, and
`rules_java` 7.12.4 and 8.10.0. I updated the `protobuf` patch for v28.2
with the following statements:

```py
load("//bazel/common:proto_common.bzl", "proto_common")
load("@rules_proto//proto:proto_common.bzl", "toolchains")

_PROTO_TOOLCHAIN = "@rules_proto//proto:toolchain_type"
_PROTO_TOOLCHAIN_ATTR = "INCOMPATIBLE_ENABLE_PROTO_TOOLCHAIN_RESOLUTION"
_PROTOC_TOOLCHAINS = toolchains.use_toolchain(_PROTO_TOOLCHAIN)

def _protoc_files_to_run(ctx):
    if getattr(proto_common, _PROTO_TOOLCHAIN_ATTR, False):
```

I tried using `proto_common` from `rules_proto`. I also created a
`rules_proto` 6.0.2 patch for `proto_toolchain()` to fix a "no such
package: //proto" breakage:

```diff
diff --git i/proto/private/rules/proto_toolchain.bzl w/proto/private/rules/proto_toolchain.bzl
index a091b80..def2699 100644
--- i/proto/private/rules/proto_toolchain.bzl
+++ w/proto/private/rules/proto_toolchain.bzl
@@ -33,7 +33,7 @@ def proto_toolchain(*, name, proto_compiler, exec_compatible_with = []):

     native.toolchain(
         name = name + "_toolchain",
-        toolchain_type = "//proto:toolchain_type",
+        toolchain_type = Label("//proto:toolchain_type"),
         exec_compatible_with = exec_compatible_with,
         target_compatible_with = [],
         toolchain = name,

```

I tried adding combinations of the following
`--incompatible_autoload_externally` flag values to .bazelrc`:

```txt
common --incompatible_autoload_externally=+@protobuf,+@rules_java
```

Nothing worked.

---

After the `protobuf` v29 bump, and before the ScalaPB 1.0.0-alpha.1
bump, `scala_proto` targets would fail with the following error:

```txt
ERROR: .../external/com_google_protobuf/src/google/protobuf/BUILD.bazel:23:14:
  ProtoScalaPBRule
  external/com_google_protobuf/src/google/protobuf/any_proto_jvm_extra_protobuf_generator_scalapb.srcjar
  failed: (Exit 1): scalapb_worker failed:
  error executing ProtoScalaPBRule command
  (from target @@com_google_protobuf//src/google/protobuf:any_proto)
  bazel-out/.../bin/src/scala/scripts/scalapb_worker
    ... (remaining 2 arguments skipped)

--jvm_extra_protobuf_generator_out:
  java.lang.NoSuchMethodError:
  'java.lang.Object com.google.protobuf.DescriptorProtos$FieldOptions.getExtension(com.google.protobuf.GeneratedMessage$GeneratedExtension)'
    at scalapb.compiler.DescriptorImplicits$ExtendedFieldDescriptor.fieldOptions(DescriptorImplicits.scala:329)
  [ ...snip... ]

java.lang.RuntimeException: Exit with code 1
    at scala.sys.package$.error(package.scala:30)
    at scripts.ScalaPBWorker$.work(ScalaPBWorker.scala:44)
    at io.bazel.rulesscala.worker.Worker.persistentWorkerMain(Worker.java:96)
    at io.bazel.rulesscala.worker.Worker.workerMain(Worker.java:49)
    at scripts.ScalaPBWorker$.main(ScalaPBWorker.scala:39)
    at scripts.ScalaPBWorker.main(ScalaPBWorker.scala)

ERROR: .../external/com_google_protobuf/src/google/protobuf/BUILD.bazel:23:14
  Building source jar external/com_google_protobuf/src/google/protobuf/any_proto_scalapb-src.jar
  failed: (Exit 1): scalapb_worker failed:
  error executing ProtoScalaPBRule command
  (from target @@com_google_protobuf//src/google/protobuf:any_proto)
  bazel-out/darwin_arm64-opt-exec-ST-a828a81199fe/bin/src/scala/scripts/scalapb_worker
    ... (remaining 2 arguments skipped)
```

Author: mbland

.bazelci/presubmit.yml

@@ -29,16 +29,17 @@ tasks:
     # Install xmllint
     - sudo apt update && sudo apt install --reinstall libxml2-utils -y
     - "./test_rules_scala.sh"
-  test_rules_scala_linux_latest:
-    name: "./test_rules_scala (latest Bazel)"
+  # Switch `last_rc` to `last_green` once Bzlmod lands.
+  # https://github.com./bazelbuild/rules_scala/issues/1482
+  test_rules_scala_linux_last_rc:
+    name: "./test_rules_scala (last_rc Bazel)"
     platform: ubuntu2004
-    # Restore `bazel: latest` once Bazel 8 compatibility lands (#1625, #1652).
-    bazel: 7.x
+    bazel: last_rc
     shell_commands:
     # Install xmllint
     - sudo apt update && sudo apt install --reinstall libxml2-utils -y
     - echo "build --enable_workspace" >> .bazelrc
-    - "./test_rules_scala.sh || buildkite-agent annotate --style 'warning' \"Optional build with latest Bazel version failed, [see here](${BUILDKITE_BUILD_URL}#${BUILDKITE_JOB_ID}) (It is not mandatory but worth checking)\""
+    - "./test_rules_scala.sh || buildkite-agent annotate --style 'warning' \"Optional build with last_rc Bazel version failed, [see here](${BUILDKITE_BUILD_URL}#${BUILDKITE_JOB_ID}) (It is not mandatory but worth checking)\""
   test_rules_scala_macos:
     name: "./test_rules_scala"
     platform: macos

.bazelrc

@@ -1,9 +1,14 @@
+# Switch to --noenable_workspace when Bzlmod lands.
+# https://github.com./bazelbuild/rules_scala/issues/1482
+common --enable_workspace --noenable_bzlmod
+
+# Remove once proto toolchainization becomes the default
+# - https://bazel.build/reference/command-line-reference#flag--incompatible_enable_proto_toolchain_resolution
+# - https://docs.google.com/document/d/1CE6wJHNfKbUPBr7-mmk_0Yo3a4TaqcTPE0OWNuQkhPs/edit
+common --incompatible_enable_proto_toolchain_resolution
+
 build --enable_platform_specific_config
 
 #Windows needs --worker_quit_after_build due to workers not being shut down when the compiler tools need to be rebuilt (resulting in 'file in use' errors). See Bazel Issue#10498.
 
 build:windows --worker_quit_after_build --enable_runfiles 
-
-# Switch to --noenable_workspace when Bzlmod lands.
-# https://github.com./bazelbuild/rules_scala/issues/1482
-common --enable_workspace --noenable_bzlmod