Skip to content

Rustdoc output is not deterministic for failed doctests #76442

Closed
@jyn514

Description

@jyn514
Member

This is causing spurious errors in the testsuite.

Details
failures:

---- [ui] rustdoc-ui\failed-doctest-output.rs stdout ----
\rustdoc-ui\failed-doctest-output.rs
\rustdoc-ui\failed-doctest-output.rs
\rustdoc-ui\failed-doctest-output.rs
thread 'main' panicked at 'Some tests failed', src\tools\compiletest\src\main.rs:354:22
\rustdoc-ui\failed-doctest-output.rs
\rustdoc-ui\failed-doctest-output.rs
\rustdoc-ui\failed-doctest-output.rs
\rustdoc-ui\failed-doctest-output.rs
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
\rustdoc-ui\failed-doctest-output.rs
diff of stdout:

1	
2	running 2 tests
-	test $DIR/failed-doctest-output.rs - OtherStruct (line 21) ... FAILED
4	test $DIR/failed-doctest-output.rs - SomeStruct (line 11) ... FAILED
+	test $DIR/failed-doctest-output.rs - OtherStruct (line 21) ... FAILED
5	
6	failures:
7	

-	---- $DIR/failed-doctest-output.rs - OtherStruct (line 21) stdout ----
-	error[E0425]: cannot find value `no` in this scope
-	  --> $DIR/failed-doctest-output.rs:22:1
-	   |
-	LL | no
-	   | ^^ not found in this scope
-	
-	error: aborting due to previous error
-	
-	For more information about this error, try `rustc --explain E0425`.
-	Couldn't compile the test.
19	---- $DIR/failed-doctest-output.rs - SomeStruct (line 11) stdout ----
20	Test executable failed (exit code 101).
21	

30	note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
31	
32	
+	---- $DIR/failed-doctest-output.rs - OtherStruct (line 21) stdout ----
+	error[E0425]: cannot find value `no` in this scope
+	  --> $DIR/failed-doctest-output.rs:22:1
+	   |
+	LL | no
+	   | ^^ not found in this scope
+	
+	error: aborting due to previous error
+	
+	For more information about this error, try `rustc --explain E0425`.
+	Couldn't compile the test.
33	
34	failures:
35	    $DIR/failed-doctest-output.rs - OtherStruct (line 21)

Originally posted by @mati865 in #76423 (comment)

Activity

added
A-testsuiteArea: The testsuite used to check the correctness of rustc
C-bugCategory: This is a bug.
T-rustdocRelevant to the rustdoc team, which will review and decide on the PR/issue.
on Sep 7, 2020
Mark-Simulacrum

Mark-Simulacrum commented on Sep 7, 2020

@Mark-Simulacrum
Member

We should probably just remove/ignore these tests if they're causing spurious failures at a relatively high rate.

jyn514

jyn514 commented on Sep 7, 2020

@jyn514
MemberAuthor

Maybe instead of removing them we could just split them into separate files? That way the order doesn't have to be deterministic, but we're still testing the output is correct.

self-assigned this
on Sep 7, 2020
mati865

mati865 commented on Sep 9, 2020

@mati865
Member

Somehow this started to appear quite often recently, usually on i686-mingw-1 builder.

3 remaining items

Loading
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

A-testsuiteArea: The testsuite used to check the correctness of rustcC-bugCategory: This is a bug.T-rustdocRelevant to the rustdoc team, which will review and decide on the PR/issue.

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

    Development

    Participants

    @mati865@Mark-Simulacrum@jyn514

    Issue actions

      Rustdoc output is not deterministic for failed doctests · Issue #76442 · rust-lang/rust