Skip to content

Stack overflow on large doctests in edition 2024 on Windows #138248

Closed
@ehuss

Description

@ehuss
Contributor

Over in #138162 I am trying to update the standard library to Rust 2024. I'm running into a problem where core's doctests are failing on x86_64-pc-windows-msvc.

The behavior is:

  1. Rustdoc prints thread 'main' has overflowed its stack
  2. It then starts running the tests: running 1672 tests
  3. After they are all done, it prints test result: ok. 1671 passed; 0 failed; 1 ignored; 0 measured; 0 filtered out; finished in 109.68s
  4. After all the tests complete, it exits 101.

From what I can tell, there is a const TESTS: [test::TestDescAndFn; 3641] array in the combined test code. When calling test::test_main, it fails somewhere in there. I'm struggling a bit figuring out a good way to debug exactly where the stack is overflowing.

There are a few issues here:

  1. Obviously it shouldn't overflow its stack.
  2. The "has overflowed its stack" message has no context, and doesn't explain where it is coming from (it is coming from executing the combined suite, but for some reason I guess rustdoc continues to run the uncombined tests?).
  3. The error is at the beginning of the output, but there is no summary at the end to explain what has happened (like the normal test runner would do if an individual test would fail). In fact, it prints a summary that says "everything was okay" when it was not.

I haven't put together an isolated repro. I wonder if just creating something with 4,000 doctests would suffice? So far, I've been using --persist-doctests to get a copy of core's combined suite, and then running rustc directly on that.

cc @notriddle @GuillaumeGomez

Activity

added
A-doctestsArea: Documentation tests, run by rustdoc
C-bugCategory: This is a bug.
T-rustdocRelevant to the rustdoc team, which will review and decide on the PR/issue.
on Mar 9, 2025
added
needs-triageThis issue may need triage. Remove it if it has been sufficiently triaged.
on Mar 9, 2025
saethlin

saethlin commented on Mar 9, 2025

@saethlin
Member

From what I can tell, there is a const TESTS: [test::TestDescAndFn; 3641] array in the combined test code. When calling test::test_main, it fails somewhere in there. I'm struggling a bit figuring out a good way to debug exactly where the stack is overflowing.

Just by squinting at the code, I would bet on this line, which is passing that array by value:

return std::process::Termination::report(test::test_main(test_args, Vec::from(TESTS), None));

I'd at least try passing &TESTS, you might also need to turn that const into a static.

Oh actually there's another Vec::from(TESTS) in that function. Ouch. I strongly suspect that copies of this array are what's causing the stack size explosion.

removed
needs-triageThis issue may need triage. Remove it if it has been sufficiently triaged.
on Mar 9, 2025
saethlin

saethlin commented on Mar 9, 2025

@saethlin
Member

I thought I would be able to reproduce this crash on Linux by shrinking my stack size. No luck. I can indeed get stack overflows when building the core doctests by making my stack size 1 MB, but also that stack size is too small to build the compiler so that can't be it.

So I think this is somehow actually Windows-specific, or it depends on the very specific CI build configuration.

ChrisDenton

ChrisDenton commented on Mar 9, 2025

@ChrisDenton
Member

For MSVC you increase the default stack limit by adding:

compiler.arg(format!("-Clink_arg=/STACK:{}", 8 * 1024 * 1024));

Somewhere in

fn run_test(

Or use --stack,{} for mingw.

This could be justified for rustdoc as making the test framework more consistent cross-platform. Though fixing the underlying reason why we use so much stack space would be a good idea.

ChrisDenton

ChrisDenton commented on Mar 9, 2025

@ChrisDenton
Member

Alternatively it could start a thread instead of running on the main thread. That way it could control the stack size directly on all platforms. However, that may be more hazardous for compatibility if something relies on being run on the main thread for some reason (I'm not sure what reason there would be for a doc test but I guess it's possible on some platforms).

saethlin

saethlin commented on Mar 9, 2025

@saethlin
Member

Ah! Of course this is the main thread so RUST_MIN_STACK doesn't do anything.

I can reproduce the problem on Linux with ulimit -s 1024.

self-assigned this
on Mar 9, 2025

4 remaining items

Loading
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

A-doctestsArea: Documentation tests, run by rustdocA-edition-2024Area: The 2024 editionC-bugCategory: This is a bug.T-rustdocRelevant to the rustdoc team, which will review and decide on the PR/issue.

Type

No type

Projects

Status

Done

Milestone

No milestone

Relationships

None yet

    Development

    Participants

    @ehuss@ChrisDenton@saethlin@fmease@rustbot

    Issue actions

      Stack overflow on large doctests in edition 2024 on Windows · Issue #138248 · rust-lang/rust