WebAssembly for procedural macros

[timClicks]

Proposal

Create an experimental implementation of not-yet-proposed language feature to use WebAssembly for procedural macros. This would involve compile procedural macro crates to wasm32-unknown-unknown (or potentially wasm32-wasip2) and then executing them to perform macro expansion with a WebAssembly runtime driven by rustc.

In principle, using WebAssembly would provide many significant benefits. A working implementation would enable these benefits, as well as costs, to be validated.

Major components

Internal

• Add a wasm_procmacros language feature
• Compile proc macro crates as wasm32-unknown-unknown within rustc
• Perform macro expansion with a a WebAssembly runtime driven by rustc

User-facing

• expand -Zproc_macro_execution_strategy to include wasm
• -Zwasm_runtime=<PATH>|<NAME> to specify the name of a wasm runtime, i.e. wasmtime or the path to an executable

Rationale

The current model of proc macro expansion—using dynamically-linked libraries—creates build reproducibility and security hazards for the Rust ecosystem. WebAssembly is one mechanism for improving our posture that shows a lot of promise.

Reproducibility

Because procedural macros can access the entire system, it's easy to introduce entropy into the build. Environment variables, the system clock, and other external sources of input mean that it's very difficult to guarantee a reproducible build.

Build speeds

As WebAssembly is inherently cross-platform, proc macros could be compiled once and re-used in each target within a build. This would reduce the number of times a proc macro is compiled from n targets to 1.

The situation even improves further when one considers that transitive dependencies of proc macros would no longer need to be fetched and compiled for each target.

Implementation notes

Guidance is welcome to advise on the most appropriate implementation strategy.

This MCP would touch at least the following parts of the rustc codebase:

• compiler/rustc_builtin_macros: [TODO: check if any special treatment is needed here]
• compiler/rustc_codegen_ssa / compiler/rustc_codegen_llvm: Logic added to compile proc macros to wasm32-unknown-unknown, irrespective of the build's actual target platform.
• compiler/rustc_expand: Logic added to proc macro expansion to use wasm when the feature is enabled
• compiler/rustc_macros/proc_macro_server.rs: create a Wasm type that mirrors the existing Rustc
• compiler/rustc_metadata: Add functionality to load a crates compiled as wasm, analogous to the native_libs and creader modules.
• compiler/rustc_session: Support wasm as possible choice of proc_macro_execution_strategy; add wasm_runtime option-
• library/proc_macro: [TODO: check if changes are needed]

Interactions with wasm_c_abi compiler feature may complicate matters. For example, consider this comment within library/proc_macro/src/bridge/mod.rs:

// proc_macros anyway don't work on wasm hosts so while both sides of this bridge can
// be built with different versions of rustc, the wasm ABI changes don't really matter.
#![allow(wasm_c_abi)]

Mentors or Reviewers

[None currently]

Related issues

• Build-time execution sandboxing (Dec 2021) - is closed. This proposal is for a narrower scope—procedural macros—rather than including build scripts.

Process

The main points of the Major Change Process are as follows:

• File an issue describing the proposal.
• A compiler team member or contributor who is knowledgeable in the area can second by writing @rustbot second.
  • Finding a "second" suffices for internal changes. If however, you are proposing a new public-facing feature, such as a -C flag, then full team check-off is required.
  • Compiler team members can initiate a check-off via @rfcbot fcp merge on either the MCP or the PR.
• Once an MCP is seconded, the Final Comment Period begins. If no objections are raised after 10 days, the MCP is considered approved.

You can read more about Major Change Proposals on forge.

[rustbot]

Important

This issue is not meant to be used for technical discussion. There is a Zulip stream for that.
Use this issue to leave procedural comments, such as volunteering to review, indicating that you second the proposal (or third, etc), or raising a concern that you would like to be addressed.

Concerns or objections can formally be registered here by adding a comment.

@rfcbot concern reason-for-concern
<description of the concern>

Concerns can be lifted with:

@rfcbot resolve reason-for-concern

See documentation at https://forge.rust-lang.org

cc @rust-lang/compiler

[petrochenkov]

@rustbot second

[apiraino]

removing the seconding to allow some time for the design discussion on Zulip to settle

@rustbot label -final-comment-period