Cargo doc

[GuillaumeGomez]

Continuation of #2256.

cc @steveklabnik

@alexcrichton: Do you want to set a thread limit (I didn't implement them yet but the code is in place)?

What's remaining:

• Find out why html generation from markdown fails.
• Add an option to disable the html generation for doc/ files.
• Add tests.

[rust-highfive]

Thanks for the pull request, and welcome! The Rust team is excited to review your changes, and you should hear from @huonw (or someone else) soon.

If any changes to this PR are deemed necessary, please add them as extra commits. This ensures that the reviewer can see what has changed since they last reviewed the code. Due to the way GitHub handles out-of-date commits, this should also make it reasonably obvious what issues have or haven't been addressed. Large or tricky changes may require several passes of review and changes.

Please see the contribution instructions for more information.

[alexcrichton]

Thanks for the PR! The more I think about this the more I'm convinced that we may want to push it into the backend infrastructure (the cargo_rustc module) as that will buy us:

• Correct tracking of inputs, e.g. memoizing the output if no input has changed.
• Parallelization
• Working with cross-compiled docs
• Extensibility in terms of filtering what to build, building only the dependencies for that, etc.

This can likely largely be done by modifying the Profile structure with a new flag and then modifying where to-compile targets are generated to generate a bunch of doc targets if we're documenting something.

[alexcrichton]

Closing due to inactivity, but feel free to reopen with comments addressed!

[GuillaumeGomez]

I'm still trying to figure all this out. I'm not used to cargo architecture so I'm mostly reading code for now.