Tracking issue for procedural macros and "hygiene 2.0"

[alexcrichton]

This is intended to be a tracking issue for enabling procedural macros to have "more hygiene" than the copy/paste hygiene they have today. This is a very vague tracking issue and there's a very long and storied history to this. The intention here though is to create a fresh tracking issue to collect the current state of play, in hopes that we can close some of the much older and very long issues which contain lots of outdated information.

[petrochenkov]

To give some background, feature(proc_macro_hygiene) tracked by this issue was merged from multiple features (proc_macro_mod, proc_macro_expr, proc_macro_non_items, proc_macro_gen) in #52121.

So, it's pretty heterogeneous and is not even entirely about hygiene.

[petrochenkov]

Copypasting what I wrote in #52121:

None of these features is blocked on hygiene, except for proc_macro_gen.

• proc_macro_mod is, IIRC, about expanding inner attributes and expanding non-inline modules (mod foo;). I'm not sure why #[my_macro] mod m { ... } with outer attribute and inline module is still gated, I'm not aware of any reasons to not stabilize it.
• proc_macro_expr/proc_macro_non_items have no reasons to be gated as well and can be stabilized after some implementation audit. Proc macros have well-defined "copy-paste" Span::call_site() hygiene, this includes "unhygienic" use of local variables.
  IIRC, @nrc assumed that additional feature gates would prevent interactions of that copy-paste hygiene with local variables, but that's not true - you can easily expand an item and inject tokens from the outside into its body (e.g. TokenStream::parse does not resolve with Span::call_site() #50050), causing exactly the same interactions.

What is really blocked on hygiene is Span::def_site() and decl_macro (which is also blocked on a ton of other stuff).

[petrochenkov]

proc_macro_gen is more about interactions between Span::call_site hygiene (entirely unhygienic) and macro_rules hygiene (unhygienic except for local variables and labels).

[CodeSandwich]

This looks like the right branch for this question. Sorry about copy-paste, but I really need to know.

It seems that since rustc 1.30.0-nightly (63d51e89a 2018-09-28) it is no longer possible to traverse code inside module in separate file. If you process mod module;, you get TokenStream containing mod module;, WYSIWYG.

I understand, that this is necessary in order to preserver spans, hygiene and consistency with processed code. Is there or will be any way to work with content of modules in separate files? Its lack may be confusing for end users when trivial refactoring of module organization causes change in macro behavior.

[petrochenkov]

@CodeSandwich
This may be a regression from #52319 perhaps?

cc @tinco