You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
rustdoc is responsible for both compiling doc examples and executing them as tests. Cargo current has no way to control the behavior. There are a handful of issues around this:
rustdoc, as a doc comments analyzer, provides source information via a flag. Cargo invokes rustdoc with the flag, so it knows where the source is and how to compile and test it.
I'm very fine with this approach as it would remove code from rustdoc. Problem is that the code examples originally were written to allow crate writers to add code examples for their users, meaning they needed to import the crate as if it was a dependency. I think this approach is still good, but should not be the only one (for example we can't run doctest in binary crates, which is bad). If cargo can solve both issues, that'd be awesome. :)
Unfortunately, it would require some new attributes for it to work as you can add a lot of different checks on doctests such as a specific edition or even to ensure that the compilation fails. More information here.
Would it be possible to resolve how the doctest should be built with a new attribute? Maybe something like internal to opt into "compile this test as a normal unit test that uses crate internal APIs."
Unfortunately, it would require some new attributes for it to work as you can add a lot of different checks on doctests such as a specific edition or even to ensure that the compilation fails
Yeah, I was aware of and called out compile_fail. Being able to set the edition is an odd one. Where can I read up more on the use case for that? As I don't personally have a use case for it, my automatic reaction is "can this be deprecated?" but I recognize there are likely cases I'm not aware of.
Likely switching to this style of doctests would require an opt-in with it being the default in a new edition, so we could get away with changing things.
It's not finished yet. A few things need to be added:
only enable it starting 2024 edition
remove chunking to run all tests at once (and improve even further the performance improvement) and potentially fix libtest being blocked
Once done, I plan to work on binary doctests: I want to replace them with what we do for #[test] so people will be able to finally test their own code.
Activity
weihanglo commentedon Feb 8, 2024
rustdoc
is responsible for both compiling doc examples and executing them as tests. Cargo current has no way to control the behavior. There are a handful of issues around this:cargo test --all-targets
does not run doc tests cargo#6669target.runner
— RUSTC, RUSTC_WRAPPER are not respected for doctests cargo#12532Bonus
I propose to the team to looking into the possibility for
rustdoc
to delegate test executions to Cargo.epage commentedon Feb 8, 2024
If we can put as many doctests into a single binary as possible, that would also improve compilation times
We also want to make it so its as natural to get coverage for doctests as other tests.
That is the direction I would want to go as well.
weihanglo commentedon Feb 8, 2024
Yeah we can push it further.
rustdoc
, as a doc comments analyzer, provides source information via a flag. Cargo invokesrustdoc
with the flag, so it knows where the source is and how to compile and test it.GuillaumeGomez commentedon Mar 21, 2024
I'm very fine with this approach as it would remove code from rustdoc. Problem is that the code examples originally were written to allow crate writers to add code examples for their users, meaning they needed to import the crate as if it was a dependency. I think this approach is still good, but should not be the only one (for example we can't run doctest in binary crates, which is bad). If cargo can solve both issues, that'd be awesome. :)
epage commentedon Mar 26, 2024
Crazy idea: could we change the compilation of
#[doc]
to generate#[test] fn
s for each code fence.Challenges
GuillaumeGomez commentedon Mar 27, 2024
Unfortunately, it would require some new attributes for it to work as you can add a lot of different checks on doctests such as a specific edition or even to ensure that the compilation fails. More information here.
BurntSushi commentedon Mar 27, 2024
Would it be possible to resolve how the doctest should be built with a new attribute? Maybe something like
internal
to opt into "compile this test as a normal unit test that uses crate internal APIs."epage commentedon Mar 27, 2024
Yeah, I was aware of and called out
compile_fail
. Being able to set the edition is an odd one. Where can I read up more on the use case for that? As I don't personally have a use case for it, my automatic reaction is "can this be deprecated?" but I recognize there are likely cases I'm not aware of.Likely switching to this style of doctests would require an opt-in with it being the default in a new edition, so we could get away with changing things.
epage commentedon Apr 23, 2024
btw rust-lang/rust#123974 is a nice improvement to performance.
GuillaumeGomez commentedon Apr 23, 2024
It's not finished yet. A few things need to be added:
Once done, I plan to work on binary doctests: I want to replace them with what we do for
#[test]
so people will be able to finally test their own code.23 remaining items