bevy_gizmos doc tests hang compute when run locally #12207
Comments
alice-i-cecile
added
C-Bug
alice-i-cecile
changed the title
|
This issue is related to 'Microsoft® Incremental Linker' - the linker used by the rust compiler on some (most?) windows machines. The one thing that sets No single doc-test in particular seems to be responsible for the increased memory usage, as removing any given set of doc-tests results in a decrease in memory usage more or less proportional to the amount of tests removed. Instead, running doc-tests for (very) large projects on windows machines seems to be the issue. I was unable to find an "in code" fix for this, but there is a discussion on the rust forums that appears to suggest building for a different but compatible target or using However, this problem does not seem to be limited to Windows only, as #11034 details a very similar (the same?) issue on a Linux machine. |
|
Further discussion on Discord.
General conclusions seem to be the same as above, although it looks like rust-lang/rust#123974 should resolve this. |
alice-i-cecile
added
S-Blocked
|
In addition to the previous observation, [target.x86_64-pc-windows-msvc]
rustdocflags = ["-Clinker=rust-lld.exe"]This helps reduce the memory usage to 1/2 or 1/3, which is a huge improvement if it leads to avoiding swapping (depends on how much RAM your system has). |
# Objective - Rustdoc doesn't seem to follow cargo's `linker` setting - Improves the situation in #12207 ## Solution - Explicitly set the linker in rustdoc flags ## Testing - I tested this change on Windows and it significantly improves testing performances (can't give an exact estimate since they got stuck before this change) --- Note: I avoided changing the settings on Linux and MacOS because I can't test on those platforms. It would be nice if someone could test similar changes there and report so they can be done on all major platforms.
|
rust-lang/rust#126245 has been merged, though unfortunately it will be enabled only with the 2024 edition. |
|
Same issue on Fedora 40. Running Rust version info config.toml |
# Objective Running `cargo ci` on Fedora 40 causes a system crash due to many `ld` processes consuming all available memory when performing doc tests. ## Solution This PR extends #13553. - Add reference to #12207 in the CI sections of `CONTRIBUTING.md` and `config_fast_builds.toml`. - Add sample `rustdocflags` configurations for `lld` and `mold` to `config_fast_builds.toml` for Linux. ## Testing Crashing configuration - config.toml ``` [target.x86_64-unknown-linux-gnu] linker = "clang" rustflags = ["-Clink-arg=-fuse-ld=lld"] [alias] ci = "run --package ci --" ``` - Test command `cargo ci` Working configuration - config.toml ``` [target.x86_64-unknown-linux-gnu] linker = "clang" rustflags = ["-Clink-arg=-fuse-ld=lld"] rustdocflags = ["-Clink-arg=-fuse-ld=lld"] [alias] ci = "run --package ci --" ``` - Test command `cargo ci`
|
12th Gen Intel(R) Core(TM) i7-12700H cargo test --doc With It uses all 32 GB of RAM and causes the system to freeze up/soft crash after about 5-10 minutes. |
|
This appears to be fixed in latest nightly. It's not fixed for me in stable or beta. Which is odd as the fix is supposed to be in stable and I can't see anything in the beta/nightly release notes. With the fix my One tidbit I noticed while testing - Nightly: |
|
in 19ee692, with
|
|
one thing to note is that i.e. these envvars also work |
|
Appears fixed in Rust 1.85.1: https://releases.rs/docs/1.85.1/. Doc tests for I suspect that the |
|
Marking as fixed! |
## Objective Fix `bevy_ecs` doc tests failing when used with `--all-features`. ``` ---- crates\bevy_ecs\src\error\handler.rs - error::handler::GLOBAL_ERROR_HANDLER (line 87) stdout ---- error[E0425]: cannot find function `default_error_handler` in this scope --> crates\bevy_ecs\src\error\handler.rs:92:24 | 8 | let error_handler = default_error_handler(); | ^^^^^^^^^^^^^^^^^^^^^ not found in this scope ``` I happened to come across this while testing #12207. I'm not sure it actually needs fixing but seemed worth a go ## Testing ``` cargo test --doc -p bevy_ecs --all-features ``` ## Side Notes The CI misses this error as it doesn't use `--all-features`. Perhaps it should? I tried adding `--all-features` to `ci/src/commands/doc_tests.rs` but this triggered a linker error: ``` Compiling bevy_dylib v0.16.0-dev (C:\Projects\bevy\crates\bevy_dylib) error: linking with `link.exe` failed: exit code: 1189 = note: LINK : fatal error LNK1189: library limit of 65535 objects exceeded␍ ```
## Objective Fix `bevy_ecs` doc tests failing when used with `--all-features`. ``` ---- crates\bevy_ecs\src\error\handler.rs - error::handler::GLOBAL_ERROR_HANDLER (line 87) stdout ---- error[E0425]: cannot find function `default_error_handler` in this scope --> crates\bevy_ecs\src\error\handler.rs:92:24 | 8 | let error_handler = default_error_handler(); | ^^^^^^^^^^^^^^^^^^^^^ not found in this scope ``` I happened to come across this while testing #12207. I'm not sure it actually needs fixing but seemed worth a go ## Testing ``` cargo test --doc -p bevy_ecs --all-features ``` ## Side Notes The CI misses this error as it doesn't use `--all-features`. Perhaps it should? I tried adding `--all-features` to `ci/src/commands/doc_tests.rs` but this triggered a linker error: ``` Compiling bevy_dylib v0.16.0-dev (C:\Projects\bevy\crates\bevy_dylib) error: linking with `link.exe` failed: exit code: 1189 = note: LINK : fatal error LNK1189: library limit of 65535 objects exceeded␍ ```
Bevy version
b24ab2e
[Optional] Relevant system information
Windows
What you did
Run
cargo run -p ci --doc.What went wrong
Doc tests for initial crates pass, and then a dozen or so doc tests within
bevy_gizmosget stuck, producing a warning that they've been running for more than 60 seconds.My computer then runs out of RAM (over 12 GB!), freezes, and I have to hard reboot.
Additional information
Running with fewer threads via e.g.
-j 4improves responsiveness and slows down time-to-hang, but does not resolve the problem.This works fine in CI, so it may be OS specific.
The text was updated successfully, but these errors were encountered: