Closed
Description
Over in #138162 I am trying to update the standard library to Rust 2024. I'm running into a problem where core's doctests are failing on x86_64-pc-windows-msvc.
The behavior is:
- Rustdoc prints
thread 'main' has overflowed its stack
- It then starts running the tests:
running 1672 tests
- After they are all done, it prints
test result: ok. 1671 passed; 0 failed; 1 ignored; 0 measured; 0 filtered out; finished in 109.68s
- After all the tests complete, it exits 101.
From what I can tell, there is a const TESTS: [test::TestDescAndFn; 3641]
array in the combined test code. When calling test::test_main
, it fails somewhere in there. I'm struggling a bit figuring out a good way to debug exactly where the stack is overflowing.
There are a few issues here:
- Obviously it shouldn't overflow its stack.
- The "has overflowed its stack" message has no context, and doesn't explain where it is coming from (it is coming from executing the combined suite, but for some reason I guess rustdoc continues to run the uncombined tests?).
- The error is at the beginning of the output, but there is no summary at the end to explain what has happened (like the normal test runner would do if an individual test would fail). In fact, it prints a summary that says "everything was okay" when it was not.
I haven't put together an isolated repro. I wonder if just creating something with 4,000 doctests would suffice? So far, I've been using --persist-doctests
to get a copy of core's combined suite, and then running rustc
directly on that.
Metadata
Metadata
Assignees
Labels
Type
Projects
Status
Done
Activity
saethlin commentedon Mar 9, 2025
Just by squinting at the code, I would bet on this line, which is passing that array by value:
rust/src/librustdoc/doctest/runner.rs
Line 147 in dea1661
I'd at least try passing
&TESTS
, you might also need to turn thatconst
into astatic
.Oh actually there's another
Vec::from(TESTS)
in that function. Ouch. I strongly suspect that copies of this array are what's causing the stack size explosion.saethlin commentedon Mar 9, 2025
I thought I would be able to reproduce this crash on Linux by shrinking my stack size. No luck. I can indeed get stack overflows when building the core doctests by making my stack size 1 MB, but also that stack size is too small to build the compiler so that can't be it.
So I think this is somehow actually Windows-specific, or it depends on the very specific CI build configuration.
ChrisDenton commentedon Mar 9, 2025
For MSVC you increase the default stack limit by adding:
Somewhere in
rust/src/librustdoc/doctest.rs
Line 529 in dea1661
Or use
--stack,{}
for mingw.This could be justified for rustdoc as making the test framework more consistent cross-platform. Though fixing the underlying reason why we use so much stack space would be a good idea.
ChrisDenton commentedon Mar 9, 2025
Alternatively it could start a thread instead of running on the main thread. That way it could control the stack size directly on all platforms. However, that may be more hazardous for compatibility if something relies on being run on the main thread for some reason (I'm not sure what reason there would be for a doc test but I guess it's possible on some platforms).
saethlin commentedon Mar 9, 2025
Ah! Of course this is the main thread so
RUST_MIN_STACK
doesn't do anything.I can reproduce the problem on Linux with
ulimit -s 1024
.4 remaining items