Auto merge of rust-lang#119899 - onur-ozkan:redesign-stage0-std, r=<try>

redesign stage 0 std

### Summary

This PR changes how bootstrap builds the stage 1 compiler by switching to precompiled stage 0 standard library instead of building the in-tree one. The goal was to update bootstrap to use the beta standard library at stage 0 rather than compiling it from source (see the motivation at rust-lang/compiler-team#619).

Previously, to build a stage 1 compiler bootstrap followed this path:

```
download stage0 compiler -> build in-tree std -> compile stage1 compiler with in-tree std
```

With this PR, the new path is:

```
download stage0 compiler -> compile stage1 compiler with precompiled stage0 std
```

This also means that `cfg(bootstrap)`/`cfg(not(bootstrap))` is no longer needed for library development.

### Building "library"

Since stage0 `std` is no longer in-tree `x build/test/check library --stage 0` is now no-op. The minimum supported stage to build `std` is now 1. For the same reason, default stage values in the library profile is no longer 0.

Because building the in-tree library now requires a stage1 compiler, I highly recommend library developers to enable `download-rustc` to speed up compilation time.

<hr>

If you encounter a bug or unexpected results please open a topic in the [#t-infra/bootstrap](https://rust-lang.zulipchat.com/#narrow/channel/326414-t-infra.2Fbootstrap) Zulip channel or create a [bootstrap issue](https://github.com/rust-lang/rust/issues/new?template=bootstrap.md).

(Review thread: https://rust-lang.zulipchat.com/#narrow/channel/326414-t-infra.2Fbootstrap/topic/Review.20thread.3A.20stage.200.20redesign.20PR/with/508271433)

~~Blocked on rust-lang#122709

try-job: dist-x86_64-linux

Author: bors

compiler/rustc_span/src/analyze_source_file.rs

@@ -84,6 +84,7 @@ cfg_match! {
                 // For character in the chunk, see if its byte value is < 0, which
                 // indicates that it's part of a UTF-8 char.
                 let multibyte_test = _mm_cmplt_epi8(chunk, _mm_set1_epi8(0));
+
                 // Create a bit mask from the comparison results.
                 let multibyte_mask = _mm_movemask_epi8(multibyte_test);
 
@@ -93,6 +94,7 @@ cfg_match! {
 
                     // Check for newlines in the chunk
                     let newlines_test = _mm_cmpeq_epi8(chunk, _mm_set1_epi8(b'\n' as i8));
+
                     let mut newlines_mask = _mm_movemask_epi8(newlines_test);
 
                     let output_offset = RelativeBytePos::from_usize(chunk_index * CHUNK_SIZE + 1);

library/core/src/fmt/rt.rs

@@ -16,14 +16,6 @@ pub struct Placeholder {
     pub width: Count,
 }
 
-#[cfg(bootstrap)]
-impl Placeholder {
-    #[inline]
-    pub const fn new(position: usize, flags: u32, precision: Count, width: Count) -> Self {
-        Self { position, flags, precision, width }
-    }
-}
-
 /// Used by [width](https://doc.rust-lang.org/std/fmt/#width)
 /// and [precision](https://doc.rust-lang.org/std/fmt/#precision) specifiers.
 #[lang = "format_count"]

src/bootstrap/defaults/bootstrap.library.toml

@@ -1,17 +1,18 @@
 # These defaults are meant for contributors to the standard library and documentation.
 [build]
-# When building the standard library, you almost never want to build the compiler itself.
-build-stage = 0
-test-stage = 0
-bench-stage = 0
+build-stage = 1
+test-stage = 1
+bench-stage = 1
 
 [rust]
 # This greatly increases the speed of rebuilds, especially when there are only minor changes. However, it makes the initial build slightly slower.
 incremental = true
 # Make the compiler and standard library faster to build, at the expense of a ~20% runtime slowdown.
 lto = "off"
-# Download rustc by default for library profile if compiler-affecting
-# directories are not modified. For CI this is disabled.
+# When building the standard library, you almost never want to build the compiler itself.
+#
+# If compiler-affecting directories are not modified, use precompiled rustc to speed up
+# library development by skipping compiler builds.
 download-rustc = "if-unchanged"
 
 [llvm]

src/bootstrap/src/core/build_steps/check.rs

@@ -1,5 +1,6 @@
 //! Implementation of compiling the compiler and standard library, in "check"-based modes.
 
+use crate::core::build_steps::compile;
 use crate::core::build_steps::compile::{
     add_to_sysroot, run_cargo, rustc_cargo, rustc_cargo_env, std_cargo, std_crates_for_run_make,
 };
@@ -45,10 +46,12 @@ impl Step for Std {
     const DEFAULT: bool = true;
 
     fn should_run(run: ShouldRun<'_>) -> ShouldRun<'_> {
+        let stage = run.builder.top_stage;
         run.crate_or_deps("sysroot")
             .crate_or_deps("coretests")
             .crate_or_deps("alloctests")
             .path("library")
+            .default_condition(stage != 0)
     }
 
     fn make_run(run: RunConfig<'_>) {
@@ -62,6 +65,12 @@ impl Step for Std {
         let target = self.target;
         let compiler = builder.compiler(builder.top_stage, builder.config.build);
 
+        if builder.top_stage == 0 {
+            // Reuse the stage0 libstd
+            builder.ensure(compile::Std::new(compiler, target));
+            return;
+        }
+
         let mut cargo = builder::Cargo::new(
             builder,
             compiler,

src/bootstrap/src/core/build_steps/clippy.rs

@@ -207,16 +207,18 @@ impl Step for Rustc {
         let compiler = builder.compiler(builder.top_stage, builder.config.build);
         let target = self.target;
 
-        if compiler.stage != 0 {
-            // If we're not in stage 0, then we won't have a std from the beta
-            // compiler around. That means we need to make sure there's one in
-            // the sysroot for the compiler to find. Otherwise, we're going to
-            // fail when building crates that need to generate code (e.g., build
-            // scripts and their dependencies).
-            builder.ensure(compile::Std::new(compiler, compiler.host));
-            builder.ensure(compile::Std::new(compiler, target));
-        } else {
-            builder.ensure(check::Std::new(target).build_kind(Some(Kind::Check)));
+        if !builder.download_rustc() {
+            if compiler.stage != 0 {
+                // If we're not in stage 0, then we won't have a std from the beta
+                // compiler around. That means we need to make sure there's one in
+                // the sysroot for the compiler to find. Otherwise, we're going to
+                // fail when building crates that need to generate code (e.g., build
+                // scripts and their dependencies).
+                builder.ensure(compile::Std::new(compiler, compiler.host));
+                builder.ensure(compile::Std::new(compiler, target));
+            } else {
+                builder.ensure(check::Std::new(target).build_kind(Some(Kind::Check)));
+            }
         }
 
         let mut cargo = builder::Cargo::new(
@@ -286,7 +288,9 @@ macro_rules! lint_any {
                 let compiler = builder.compiler(builder.top_stage, builder.config.build);
                 let target = self.target;
 
-                builder.ensure(check::Rustc::new(target, builder).build_kind(Some(Kind::Check)));
+                if !builder.download_rustc() {
+                    builder.ensure(check::Rustc::new(target, builder).build_kind(Some(Kind::Check)));
+                };
 
                 let cargo = prepare_tool_cargo(
                     builder,

src/bootstrap/src/core/build_steps/compile.rs

@@ -151,16 +151,26 @@ impl Step for Std {
     )]
     fn run(self, builder: &Builder<'_>) {
         let target = self.target;
-        let compiler = self.compiler;
+
+        // We already have std ready to be used for stage 0.
+        if self.compiler.stage == 0 {
+            let compiler = self.compiler;
+            builder.ensure(StdLink::from_std(self, compiler));
+
+            return;
+        }
+
+        let compiler = if builder.download_rustc() && self.force_recompile {
+            // When there are changes in the library tree with CI-rustc, we want to build
+            // the stageN library and that requires using stageN-1 compiler.
+            builder.compiler(self.compiler.stage.saturating_sub(1), builder.config.build)
+        } else {
+            self.compiler
+        };
 
         // When using `download-rustc`, we already have artifacts for the host available. Don't
         // recompile them.
-        if builder.download_rustc() && builder.is_builder_target(target)
-            // NOTE: the beta compiler may generate different artifacts than the downloaded compiler, so
-            // its artifacts can't be reused.
-            && compiler.stage != 0
-            && !self.force_recompile
-        {
+        if builder.download_rustc() && builder.is_builder_target(target) && !self.force_recompile {
             let sysroot = builder.ensure(Sysroot { compiler, force_recompile: false });
             cp_rustc_component_to_ci_sysroot(
                 builder,
@@ -193,7 +203,13 @@ impl Step for Std {
         let compiler_to_use = builder.compiler_for(compiler.stage, compiler.host, target);
         trace!(?compiler_to_use);
 
-        if compiler_to_use != compiler {
+        if compiler_to_use != compiler
+            // Never uplift std unless we have compiled stage 1; if stage 1 is compiled,
+            // uplift it from there.
+            //
+            // FIXME: improve `fn compiler_for` to avoid adding stage condition here.
+            && compiler.stage > 1
+        {
             trace!(?compiler_to_use, ?compiler, "compiler != compiler_to_use, uplifting library");
 
             builder.ensure(Std::new(compiler_to_use, target));
@@ -226,27 +242,6 @@ impl Step for Std {
 
         target_deps.extend(self.copy_extra_objects(builder, &compiler, target));
 
-        // The LLD wrappers and `rust-lld` are self-contained linking components that can be
-        // necessary to link the stdlib on some targets. We'll also need to copy these binaries to
-        // the `stage0-sysroot` to ensure the linker is found when bootstrapping on such a target.
-        if compiler.stage == 0 && builder.is_builder_target(compiler.host) {
-            trace!(
-                "(build == host) copying linking components to `stage0-sysroot` for bootstrapping"
-            );
-            // We want to copy the host `bin` folder within the `rustlib` folder in the sysroot.
-            let src_sysroot_bin = builder
-                .rustc_snapshot_sysroot()
-                .join("lib")
-                .join("rustlib")
-                .join(compiler.host)
-                .join("bin");
-            if src_sysroot_bin.exists() {
-                let target_sysroot_bin = builder.sysroot_target_bindir(compiler, target);
-                t!(fs::create_dir_all(&target_sysroot_bin));
-                builder.cp_link_r(&src_sysroot_bin, &target_sysroot_bin);
-            }
-        }
-
         // We build a sysroot for mir-opt tests using the same trick that Miri does: A check build
         // with -Zalways-encode-mir. This frees us from the need to have a target linker, and the
         // fact that this is a check build integrates nicely with run_cargo.
@@ -741,7 +736,7 @@ impl Step for StdLink {
         let target = self.target;
 
         // NOTE: intentionally does *not* check `target == builder.build` to avoid having to add the same check in `test::Crate`.
-        let (libdir, hostdir) = if self.force_recompile && builder.download_rustc() {
+        let (libdir, hostdir) = if !self.force_recompile && builder.download_rustc() {
             // NOTE: copies part of `sysroot_libdir` to avoid having to add a new `force_recompile` argument there too
             let lib = builder.sysroot_libdir_relative(self.compiler);
             let sysroot = builder.ensure(crate::core::build_steps::compile::Sysroot {
@@ -757,23 +752,16 @@ impl Step for StdLink {
             (libdir, hostdir)
         };
 
-        add_to_sysroot(
-            builder,
-            &libdir,
-            &hostdir,
-            &build_stamp::libstd_stamp(builder, compiler, target),
-        );
+        let is_downloaded_beta_stage0 = builder
+            .build
+            .config
+            .initial_rustc
+            .starts_with(builder.out.join(compiler.host).join("stage0/bin"));
 
         // Special case for stage0, to make `rustup toolchain link` and `x dist --stage 0`
         // work for stage0-sysroot. We only do this if the stage0 compiler comes from beta,
         // and is not set to a custom path.
-        if compiler.stage == 0
-            && builder
-                .build
-                .config
-                .initial_rustc
-                .starts_with(builder.out.join(compiler.host).join("stage0/bin"))
-        {
+        if compiler.stage == 0 && is_downloaded_beta_stage0 {
             // Copy bin files from stage0/bin to stage0-sysroot/bin
             let sysroot = builder.out.join(compiler.host).join("stage0-sysroot");
 
@@ -783,21 +771,9 @@ impl Step for StdLink {
             t!(fs::create_dir_all(&sysroot_bin_dir));
             builder.cp_link_r(&stage0_bin_dir, &sysroot_bin_dir);
 
-            // Copy all files from stage0/lib to stage0-sysroot/lib
             let stage0_lib_dir = builder.out.join(host).join("stage0/lib");
-            if let Ok(files) = fs::read_dir(stage0_lib_dir) {
-                for file in files {
-                    let file = t!(file);
-                    let path = file.path();
-                    if path.is_file() {
-                        builder.copy_link(
-                            &path,
-                            &sysroot.join("lib").join(path.file_name().unwrap()),
-                            FileType::Regular,
-                        );
-                    }
-                }
-            }
+            t!(fs::create_dir_all(sysroot.join("lib")));
+            builder.cp_link_r(&stage0_lib_dir, &sysroot.join("lib"));
 
             // Copy codegen-backends from stage0
             let sysroot_codegen_backends = builder.sysroot_codegen_backends(compiler);
@@ -811,6 +787,30 @@ impl Step for StdLink {
             if stage0_codegen_backends.exists() {
                 builder.cp_link_r(&stage0_codegen_backends, &sysroot_codegen_backends);
             }
+        } else if compiler.stage == 0 {
+            let sysroot = builder.out.join(compiler.host.triple).join("stage0-sysroot");
+
+            if builder.local_rebuild {
+                // On local rebuilds this path might be a symlink to the project root,
+                // which can be read-only (e.g., on CI). So remove it before copying
+                // the stage0 lib.
+                let _ = fs::remove_dir_all(sysroot.join("lib/rustlib/src/rust"));
+            }
+
+            builder.cp_link_r(&builder.initial_sysroot.join("lib"), &sysroot.join("lib"));
+        } else {
+            if builder.download_rustc() {
+                // Ensure there are no CI-rustc std artifacts.
+                let _ = fs::remove_dir_all(&libdir);
+                let _ = fs::remove_dir_all(&hostdir);
+            }
+
+            add_to_sysroot(
+                builder,
+                &libdir,
+                &hostdir,
+                &build_stamp::libstd_stamp(builder, compiler, target),
+            );
         }
     }
 }
@@ -1033,7 +1033,7 @@ impl Step for Rustc {
         let compiler = self.compiler;
         let target = self.target;
 
-        // NOTE: the ABI of the beta compiler is different from the ABI of the downloaded compiler,
+        // NOTE: the ABI of the stage0 compiler is different from the ABI of the downloaded compiler,
         // so its artifacts can't be reused.
         if builder.download_rustc() && compiler.stage != 0 {
             trace!(stage = compiler.stage, "`download_rustc` requested");
@@ -1788,9 +1788,9 @@ impl Step for Sysroot {
         t!(fs::create_dir_all(&sysroot));
 
         // In some cases(see https://github.com/rust-lang/rust/issues/109314), when the stage0
-        // compiler relies on more recent version of LLVM than the beta compiler, it may not
+        // compiler relies on more recent version of LLVM than the stage0 compiler, it may not
         // be able to locate the correct LLVM in the sysroot. This situation typically occurs
-        // when we upgrade LLVM version while the beta compiler continues to use an older version.
+        // when we upgrade LLVM version while the stage0 compiler continues to use an older version.
         //
         // Make sure to add the correct version of LLVM into the stage0 sysroot.
         if compiler.stage == 0 {

src/bootstrap/src/core/build_steps/test.rs

@@ -1558,7 +1558,7 @@ impl Step for Compiletest {
 
         if builder.top_stage == 0 && env::var("COMPILETEST_FORCE_STAGE0").is_err() {
             eprintln!("\
-ERROR: `--stage 0` runs compiletest on the beta compiler, not your local changes, and will almost always cause tests to fail
+ERROR: `--stage 0` runs compiletest on the stage0 (precompiled) compiler, not your local changes, and will almost always cause tests to fail
 HELP: to test the compiler, use `--stage 1` instead
 HELP: to test the standard library, use `--stage 0 library/std` instead
 NOTE: if you're sure you want to do this, please open an issue as to why. In the meantime, you can override this with `COMPILETEST_FORCE_STAGE0=1`."
@@ -1586,9 +1586,9 @@ NOTE: if you're sure you want to do this, please open an issue as to why. In the
         // NOTE: Only stage 1 is special cased because we need the rustc_private artifacts to match the
         // running compiler in stage 2 when plugins run.
         let (stage, stage_id) = if suite == "ui-fulldeps" && compiler.stage == 1 {
-            // At stage 0 (stage - 1) we are using the beta compiler. Using `self.target` can lead
-            // finding an incorrect compiler path on cross-targets, as the stage 0 beta compiler is
-            // always equal to `build.build` in the configuration.
+            // At stage 0 (stage - 1) we are using the stage0 compiler. Using `self.target` can lead
+            // finding an incorrect compiler path on cross-targets, as the stage 0 is always equal to
+            // `build.build` in the configuration.
             let build = builder.build.build;
             compiler = builder.compiler(compiler.stage - 1, build);
             let test_stage = compiler.stage + 1;
@@ -1674,7 +1674,7 @@ NOTE: if you're sure you want to do this, please open an issue as to why. In the
         }
 
         if mode == "rustdoc-json" {
-            // Use the beta compiler for jsondocck
+            // Use the stage0 compiler for jsondocck
             let json_compiler = compiler.with_stage(0);
             cmd.arg("--jsondocck-path")
                 .arg(builder.ensure(tool::JsonDocCk { compiler: json_compiler, target }).tool_path);

src/bootstrap/src/core/build_steps/tool.rs

@@ -330,9 +330,9 @@ pub(crate) fn get_tool_rustc_compiler(
         return target_compiler;
     }
 
-    if builder.download_rustc() && target_compiler.stage > 0 {
-        // We already have the stage N compiler, we don't need to cut the stage.
-        return builder.compiler(target_compiler.stage, builder.config.build);
+    if builder.download_rustc() && target_compiler.stage == 1 {
+        // We shouldn't drop to stage0 compiler when using CI rustc.
+        return builder.compiler(1, builder.config.build);
     }
 
     // Similar to `compile::Assemble`, build with the previous stage's compiler. Otherwise

src/bootstrap/src/core/builder/tests.rs

@@ -234,7 +234,7 @@ fn alias_and_path_for_library() {
     );
     assert_eq!(
         first(cache.all::<doc::Std>()),
-        &[doc_std!(TEST_TRIPLE_1 => TEST_TRIPLE_1, stage = 0)]
+        &[doc_std!(TEST_TRIPLE_1 => TEST_TRIPLE_1, stage = 1)]
     );
 }
 
@@ -259,16 +259,10 @@ fn ci_rustc_if_unchanged_logic() {
     builder.run_step_descriptions(&Builder::get_step_descriptions(config.cmd.kind()), &[]);
 
     // Make sure "if-unchanged" logic doesn't try to use CI rustc while there are changes
-    // in compiler and/or library.
+    // in "compiler" tree.
     if config.download_rustc_commit.is_some() {
-        let mut paths = vec!["compiler"];
-
-        // Handle library tree the same way as in `Config::download_ci_rustc_commit`.
-        if builder.config.is_running_on_ci {
-            paths.push("library");
-        }
-
-        let has_changes = config.last_modified_commit(&paths, "download-rustc", true).is_none();
+        let has_changes =
+            config.last_modified_commit(&["compiler"], "download-rustc", true).is_none();
 
         assert!(
             !has_changes,
@@ -392,14 +386,14 @@ mod defaults {
         assert_eq!(first(cache.all::<doc::ErrorIndex>()), &[doc::ErrorIndex { target: a },]);
         assert_eq!(
             first(cache.all::<tool::ErrorIndex>()),
-            &[tool::ErrorIndex { compiler: Compiler::new(0, a) }]
+            &[tool::ErrorIndex { compiler: Compiler::new(1, a) }]
         );
-        // docs should be built with the beta compiler, not with the stage0 artifacts.
+        // docs should be built with the stage0 compiler, not with the stage0 artifacts.
         // recall that rustdoc is off-by-one: `stage` is the compiler rustdoc is _linked_ to,
         // not the one it was built by.
         assert_eq!(
             first(cache.all::<tool::Rustdoc>()),
-            &[tool::Rustdoc { compiler: Compiler::new(0, a) },]
+            &[tool::Rustdoc { compiler: Compiler::new(1, a) },]
         );
     }
 }

src/bootstrap/src/core/config/config.rs

@@ -44,6 +44,7 @@ use crate::utils::helpers::{self, exe, output, t};
 /// final output/compiler, which can be significantly affected by changes made to the bootstrap sources.
 #[rustfmt::skip] // We don't want rustfmt to oneline this list
 pub(crate) const RUSTC_IF_UNCHANGED_ALLOWED_PATHS: &[&str] = &[
+    ":!library",
     ":!src/tools",
     ":!src/librustdoc",
     ":!src/rustdoc-json-types",
@@ -2433,10 +2434,12 @@ impl Config {
             || bench_stage.is_some();
         // See https://github.com/rust-lang/compiler-team/issues/326
         config.stage = match config.cmd {
-            Subcommand::Check { .. } => flags.stage.or(check_stage).unwrap_or(0),
+            Subcommand::Check { .. } | Subcommand::Clippy { .. } | Subcommand::Fix => {
+                flags.stage.or(check_stage).unwrap_or(1)
+            }
             // `download-rustc` only has a speed-up for stage2 builds. Default to stage2 unless explicitly overridden.
             Subcommand::Doc { .. } => {
-                flags.stage.or(doc_stage).unwrap_or(if download_rustc { 2 } else { 0 })
+                flags.stage.or(doc_stage).unwrap_or(if download_rustc { 2 } else { 1 })
             }
             Subcommand::Build => {
                 flags.stage.or(build_stage).unwrap_or(if download_rustc { 2 } else { 1 })
@@ -2451,8 +2454,6 @@ impl Config {
             // These are all bootstrap tools, which don't depend on the compiler.
             // The stage we pass shouldn't matter, but use 0 just in case.
             Subcommand::Clean { .. }
-            | Subcommand::Clippy { .. }
-            | Subcommand::Fix
             | Subcommand::Run { .. }
             | Subcommand::Setup { .. }
             | Subcommand::Format { .. }
@@ -3063,24 +3064,14 @@ impl Config {
             }
         };
 
-        // RUSTC_IF_UNCHANGED_ALLOWED_PATHS
-        let mut allowed_paths = RUSTC_IF_UNCHANGED_ALLOWED_PATHS.to_vec();
-
-        // In CI, disable ci-rustc if there are changes in the library tree. But for non-CI, allow
-        // these changes to speed up the build process for library developers. This provides consistent
-        // functionality for library developers between `download-rustc=true` and `download-rustc="if-unchanged"`
-        // options.
-        //
-        // If you update "library" logic here, update `builder::tests::ci_rustc_if_unchanged_logic` test
-        // logic accordingly.
-        if !self.is_running_on_ci {
-            allowed_paths.push(":!library");
-        }
-
         let commit = if self.rust_info.is_managed_git_subrepository() {
             // Look for a version to compare to based on the current commit.
             // Only commits merged by bors will have CI artifacts.
-            match self.last_modified_commit(&allowed_paths, "download-rustc", if_unchanged) {
+            match self.last_modified_commit(
+                RUSTC_IF_UNCHANGED_ALLOWED_PATHS,
+                "download-rustc",
+                if_unchanged,
+            ) {
                 Some(commit) => commit,
                 None => {
                     if if_unchanged {

src/bootstrap/src/core/download.rs

@@ -662,7 +662,7 @@ impl Config {
             }
         };
 
-        // For the beta compiler, put special effort into ensuring the checksums are valid.
+        // For the stage0 compiler, put special effort into ensuring the checksums are valid.
         let checksum = if should_verify {
             let error = format!(
                 "src/stage0 doesn't contain a checksum for {url}. \

src/bootstrap/src/utils/change_tracker.rs

@@ -396,4 +396,9 @@ pub const CONFIG_CHANGE_HISTORY: &[ChangeInfo] = &[
         severity: ChangeSeverity::Info,
         summary: "Added a new option `build.compiletest-use-stage0-libtest` to force `compiletest` to use the stage 0 libtest.",
     },
+    ChangeInfo {
+        change_id: 119899,
+        severity: ChangeSeverity::Warning,
+        summary: "Stage0 library no longer matches the in-tree library, which means stage1 compiler now uses the beta library.",
+    },
 ];

src/ci/citool/src/jobs.rs

@@ -10,7 +10,7 @@ use crate::GitHubContext;
 /// Representation of a job loaded from the `src/ci/github-actions/jobs.yml` file.
 #[derive(serde::Deserialize, Debug, Clone)]
 pub struct Job {
-    /// Name of the job, e.g. mingw-check
+    /// Name of the job, e.g. mingw-check-1
     pub name: String,
     /// GitHub runner on which the job should be executed
     pub os: String,

src/ci/citool/tests/jobs.rs

@@ -40,7 +40,7 @@ try-job: dist-i686-msvc"#,
 fn pr_jobs() {
     let stdout = get_matrix("pull_request", "commit", "refs/heads/pr/1234");
     insta::assert_snapshot!(stdout, @r#"
-    jobs=[{"name":"mingw-check","full_name":"PR - mingw-check","os":"ubuntu-24.04","env":{"PR_CI_JOB":1},"free_disk":true},{"name":"mingw-check-tidy","full_name":"PR - mingw-check-tidy","os":"ubuntu-24.04","env":{"PR_CI_JOB":1},"continue_on_error":true,"free_disk":true,"doc_url":"https://foo.bar"}]
+    jobs=[{"name":"mingw-check-1","full_name":"PR - mingw-check-1","os":"ubuntu-24.04","env":{"PR_CI_JOB":1},"free_disk":true},{"name":"mingw-check-2","full_name":"PR - mingw-check-2","os":"ubuntu-24.04","env":{"PR_CI_JOB":1},"free_disk":true},{"name":"mingw-check-tidy","full_name":"PR - mingw-check-tidy","os":"ubuntu-24.04","env":{"PR_CI_JOB":1},"continue_on_error":true,"free_disk":true,"doc_url":"https://foo.bar"}]
     run_type=pr
     "#);
 }

src/ci/citool/tests/test-jobs.yml

@@ -64,7 +64,9 @@ envs:
 # These jobs automatically inherit envs.pr, to avoid repeating
 # it in each job definition.
 pr:
-  - name: mingw-check
+  - name: mingw-check-1
+    <<: *job-linux-4c
+  - name: mingw-check-2
     <<: *job-linux-4c
   - name: mingw-check-tidy
     continue_on_error: true

src/ci/docker/host-x86_64/mingw-check/Dockerfile renamed to src/ci/docker/host-x86_64/mingw-check-1/Dockerfile

@@ -34,31 +34,23 @@ RUN npm install es-check@6.1.1 eslint@8.6.0 typescript@5.7.3 -g
 COPY scripts/sccache.sh /scripts/
 RUN sh /scripts/sccache.sh
 
-COPY host-x86_64/mingw-check/reuse-requirements.txt /tmp/
+COPY host-x86_64/mingw-check-1/reuse-requirements.txt /tmp/
 RUN pip3 install --no-deps --no-cache-dir --require-hashes -r /tmp/reuse-requirements.txt
 
-COPY host-x86_64/mingw-check/check-default-config-profiles.sh /scripts/
-COPY host-x86_64/mingw-check/validate-toolstate.sh /scripts/
-COPY host-x86_64/mingw-check/validate-error-codes.sh /scripts/
+COPY host-x86_64/mingw-check-1/check-default-config-profiles.sh /scripts/
+COPY host-x86_64/mingw-check-1/validate-toolstate.sh /scripts/
+COPY host-x86_64/mingw-check-1/validate-error-codes.sh /scripts/
 
 # Check library crates on all tier 1 targets.
 # We disable optimized compiler built-ins because that requires a C toolchain for the target.
 # We also skip the x86_64-unknown-linux-gnu target as it is well-tested by other jobs.
 ENV SCRIPT \
-           python3 ../x.py check --stage 0 --set build.optimized-compiler-builtins=false core alloc std --target=aarch64-unknown-linux-gnu,i686-pc-windows-msvc,i686-unknown-linux-gnu,x86_64-apple-darwin,x86_64-pc-windows-gnu,x86_64-pc-windows-msvc && \
            /scripts/check-default-config-profiles.sh && \
-           python3 ../x.py check compiletest --set build.compiletest-use-stage0-libtest=true && \
-           python3 ../x.py check --target=x86_64-pc-windows-gnu --host=x86_64-pc-windows-gnu && \
-           python3 ../x.py clippy ci && \
            python3 ../x.py build --stage 0 src/tools/build-manifest && \
            python3 ../x.py test --stage 0 src/tools/compiletest && \
-           python3 ../x.py test --stage 0 core alloc std test proc_macro && \
-           # Build both public and internal documentation.
-           RUSTDOCFLAGS=\"--document-private-items --document-hidden-items\" python3 ../x.py doc --stage 0 library && \
-           mkdir -p /checkout/obj/staging/doc && \
-           cp -r build/x86_64-unknown-linux-gnu/doc /checkout/obj/staging && \
-           RUSTDOCFLAGS=\"--document-private-items --document-hidden-items\" python3 ../x.py doc --stage 0 compiler && \
-           RUSTDOCFLAGS=\"--document-private-items --document-hidden-items\" python3 ../x.py doc --stage 0 library/test && \
+           python3 ../x.py check compiletest --set build.compiletest-use-stage0-libtest=true && \
+           python3 ../x.py check --stage 1 --target=i686-pc-windows-gnu --host=i686-pc-windows-gnu && \
+           python3 ../x.py check --stage 1 --set build.optimized-compiler-builtins=false core alloc std --target=aarch64-unknown-linux-gnu,i686-pc-windows-msvc,i686-unknown-linux-gnu,x86_64-apple-darwin,x86_64-pc-windows-gnu,x86_64-pc-windows-msvc && \
            /scripts/validate-toolstate.sh && \
            /scripts/validate-error-codes.sh && \
            reuse --include-submodules lint && \

src/ci/docker/host-x86_64/mingw-check/check-default-config-profiles.sh renamed to src/ci/docker/host-x86_64/mingw-check-1/check-default-config-profiles.sh

@@ -0,0 +1,37 @@
+FROM ubuntu:22.04
+
+ARG DEBIAN_FRONTEND=noninteractive
+RUN apt-get update && apt-get install -y --no-install-recommends \
+  g++ \
+  make \
+  ninja-build \
+  file \
+  curl \
+  ca-certificates \
+  python3 \
+  python3-pip \
+  python3-pkg-resources \
+  git \
+  cmake \
+  sudo \
+  gdb \
+  xz-utils \
+  libssl-dev \
+  pkg-config \
+  mingw-w64 \
+  && rm -rf /var/lib/apt/lists/*
+
+ENV RUST_CONFIGURE_ARGS="--set rust.validate-mir-opts=3"
+
+COPY scripts/sccache.sh /scripts/
+RUN sh /scripts/sccache.sh
+
+ENV SCRIPT \
+        python3 ../x.py clippy ci && \
+        python3 ../x.py test --stage 1 core alloc std test proc_macro && \
+        # Build both public and internal documentation.
+        RUSTDOCFLAGS=\"--document-private-items --document-hidden-items\" python3 ../x.py doc --stage 0 compiler && \
+        RUSTDOCFLAGS=\"--document-private-items --document-hidden-items\" python3 ../x.py doc --stage 1 library && \
+        mkdir -p /checkout/obj/staging/doc && \
+        cp -r build/x86_64-unknown-linux-gnu/doc /checkout/obj/staging && \
+        RUSTDOCFLAGS=\"--document-private-items --document-hidden-items\" python3 ../x.py doc --stage 1 library/test

src/ci/docker/host-x86_64/mingw-check/reuse-requirements.in renamed to src/ci/docker/host-x86_64/mingw-check-1/reuse-requirements.in

@@ -27,12 +27,12 @@ RUN apt-get update && apt-get install -y --no-install-recommends \
 COPY scripts/sccache.sh /scripts/
 RUN sh /scripts/sccache.sh
 
-COPY host-x86_64/mingw-check/reuse-requirements.txt /tmp/
+COPY host-x86_64/mingw-check-1/reuse-requirements.txt /tmp/
 RUN pip3 install --no-deps --no-cache-dir --require-hashes -r /tmp/reuse-requirements.txt \
     && pip3 install virtualenv
 
-COPY host-x86_64/mingw-check/validate-toolstate.sh /scripts/
-COPY host-x86_64/mingw-check/validate-error-codes.sh /scripts/
+COPY host-x86_64/mingw-check-1/validate-toolstate.sh /scripts/
+COPY host-x86_64/mingw-check-1/validate-error-codes.sh /scripts/
 
 # NOTE: intentionally uses python2 for x.py so we can test it still works.
 # validate-toolstate only runs in our CI, so it's ok for it to only support python3.

src/ci/docker/host-x86_64/mingw-check/reuse-requirements.txt renamed to src/ci/docker/host-x86_64/mingw-check-1/reuse-requirements.txt

@@ -29,5 +29,5 @@ RUN echo "optimize = false" >> /config/nopt-std-config.toml
 ENV RUST_CONFIGURE_ARGS --build=x86_64-unknown-linux-gnu \
   --disable-optimize-tests \
   --set rust.test-compare-mode
-ENV SCRIPT python3 ../x.py test --stage 0 --config /config/nopt-std-config.toml library/std \
+ENV SCRIPT python3 ../x.py test --stage 1 --config /config/nopt-std-config.toml library/std \
   && python3 ../x.py --stage 2 test

src/ci/docker/host-x86_64/mingw-check/validate-error-codes.sh renamed to src/ci/docker/host-x86_64/mingw-check-1/validate-error-codes.sh

@@ -100,7 +100,9 @@ envs:
 # These jobs automatically inherit envs.pr, to avoid repeating
 # it in each job definition.
 pr:
-  - name: mingw-check
+  - name: mingw-check-1
+    <<: *job-linux-4c
+  - name: mingw-check-2
     <<: *job-linux-4c
   - name: mingw-check-tidy
     continue_on_error: true
@@ -250,11 +252,14 @@ auto:
     env:
       IMAGE: i686-gnu-nopt
       DOCKER_SCRIPT: >-
-        python3 ../x.py test --stage 0 --config /config/nopt-std-config.toml library/std &&
+        python3 ../x.py test --stage 1 --config /config/nopt-std-config.toml library/std &&
         /scripts/stage_2_test_set2.sh
     <<: *job-linux-4c
 
-  - name: mingw-check
+  - name: mingw-check-1
+    <<: *job-linux-4c
+
+  - name: mingw-check-2
     <<: *job-linux-4c
 
   - name: test-various

src/ci/docker/host-x86_64/mingw-check/validate-toolstate.sh renamed to src/ci/docker/host-x86_64/mingw-check-1/validate-toolstate.sh

@@ -45,13 +45,13 @@ compiler.
 
 ```mermaid
 graph TD
-    s0c["stage0 compiler (1.63)"]:::downloaded -->|A| s0l("stage0 std (1.64)"):::with-s0c;
+    s0c["stage0 compiler (1.86.0-beta.1)"]:::downloaded -->|A| s0l("stage0 std (1.86.0-beta.1)"):::downloaded;
     s0c & s0l --- stepb[ ]:::empty;
-    stepb -->|B| s0ca["stage0 compiler artifacts (1.64)"]:::with-s0c;
-    s0ca -->|copy| s1c["stage1 compiler (1.64)"]:::with-s0c;
-    s1c -->|C| s1l("stage1 std (1.64)"):::with-s1c;
+    stepb -->|B| s0ca["stage0 compiler artifacts (1.87.0-dev)"]:::with-s0c;
+    s0ca -->|copy| s1c["stage1 compiler (1.87.0-dev)"]:::with-s0c;
+    s1c -->|C| s1l("stage1 std (1.87.0-dev)"):::with-s1c;
     s1c & s1l --- stepd[ ]:::empty;
-    stepd -->|D| s1ca["stage1 compiler artifacts (1.64)"]:::with-s1c;
+    stepd -->|D| s1ca["stage1 compiler artifacts (1.87.0-dev)"]:::with-s1c;
     s1ca -->|copy| s2c["stage2 compiler"]:::with-s1c;
 
     classDef empty width:0px,height:0px;
@@ -62,19 +62,21 @@ graph TD
 
 ### Stage 0: the pre-compiled compiler
 
-The stage0 compiler is usually the current _beta_ `rustc` compiler and its
+The stage0 compiler is by default the very recent _beta_ `rustc` compiler and its
 associated dynamic libraries, which `./x.py` will download for you. (You can
-also configure `./x.py` to use something else.)
+also configure `./x.py` to change stage0 to something else.)
 
-The stage0 compiler is then used only to compile [`src/bootstrap`],
-[`library/std`], and [`compiler/rustc`]. When assembling the libraries and
-binaries that will become the stage1 `rustc` compiler, the freshly compiled
-`std` and `rustc` are used. There are two concepts at play here: a compiler
-(with its set of dependencies) and its 'target' or 'object' libraries (`std` and
-`rustc`). Both are staged, but in a staggered manner.
+The precompiled stage0 compiler is then used only to compile [`src/bootstrap`] and [`compiler/rustc`]
+with precompiled stage0 std.
+
+Note that to build the stage1 compiler we use the precompiled stage0 compiler and std.
+Therefore, to use a compiler with a std that is freshly built from the tree, you need to
+build the stage2 compiler.
+
+There are two concepts at play here: a compiler (with its set of dependencies) and its
+'target' or 'object' libraries (`std` and `rustc`). Both are staged, but in a staggered manner.
 
 [`compiler/rustc`]: https://github.com/rust-lang/rust/tree/master/compiler/rustc
-[`library/std`]: https://github.com/rust-lang/rust/tree/master/library/std
 [`src/bootstrap`]: https://github.com/rust-lang/rust/tree/master/src/bootstrap
 
 ### Stage 1: from current code, by an earlier compiler
@@ -84,26 +86,25 @@ The rustc source code is then compiled with the `stage0` compiler to produce the
 
 ### Stage 2: the truly current compiler
 
-We then rebuild our `stage1` compiler with itself to produce the `stage2`
+We then rebuild the compiler using `stage1` compiler with in-tree std to produce the `stage2`
 compiler.
 
-In theory, the `stage1` compiler is functionally identical to the `stage2`
-compiler, but in practice there are subtle differences. In particular, the
-`stage1` compiler itself was built by `stage0` and hence not by the source in
-your working directory. This means that the ABI generated by the `stage0`
-compiler may not match the ABI that would have been made by the `stage1`
-compiler, which can cause problems for dynamic libraries, tests, and tools using
-`rustc_private`.
+The `stage1` compiler itself was built by precompiled `stage0` compiler and std
+and hence not by the source in your working directory. This means that the ABI
+generated by the `stage0` compiler may not match the ABI that would have been made
+by the `stage1` compiler, which can cause problems for dynamic libraries, tests
+and tools using `rustc_private`.
 
 Note that the `proc_macro` crate avoids this issue with a `C` FFI layer called
 `proc_macro::bridge`, allowing it to be used with `stage1`.
 
 The `stage2` compiler is the one distributed with `rustup` and all other install
 methods. However, it takes a very long time to build because one must first
 build the new compiler with an older compiler and then use that to build the new
-compiler with itself. For development, you usually only want the `stage1`
-compiler, which you can build with `./x build library`. See [Building the
-compiler](../how-to-build-and-run.html#building-the-compiler).
+compiler with itself.
+
+For development, you usually only want to use `--stage 1` flag to build things.
+See [Building the compiler](../how-to-build-and-run.html#building-the-compiler).
 
 ### Stage 3: the same-result test
 
@@ -114,10 +115,11 @@ something has broken.
 ### Building the stages
 
 The script [`./x`] tries to be helpful and pick the stage you most likely meant
-for each subcommand. These defaults are as follows:
+for each subcommand. Here are some `x` commands with their default stages:
 
-- `check`: `--stage 0`
-- `doc`: `--stage 0`
+- `check`: `--stage 1`
+- `clippy`: `--stage 1`
+- `doc`: `--stage 1`
 - `build`: `--stage 1`
 - `test`: `--stage 1`
 - `dist`: `--stage 2`
@@ -191,8 +193,8 @@ include, but are not limited to:
   without building `rustc` from source ('build with `stage0`, then test the
   artifacts'). If you're working on the standard library, this is normally the
   test command you want.
-- `./x build --stage 0` means to build with the beta `rustc`.
-- `./x doc --stage 0` means to document using the beta `rustdoc`.
+- `./x build --stage 0` means to build with the stage0 `rustc`.
+- `./x doc --stage 0` means to document using the stage0 `rustdoc`.
 
 #### Examples of what *not* to do
 
@@ -208,9 +210,6 @@ include, but are not limited to:
 
 ### Building vs. running
 
-Note that `build --stage N compiler/rustc` **does not** build the stage N
-compiler: instead it builds the stage N+1 compiler _using_ the stage N compiler.
-
 In short, _stage 0 uses the `stage0` compiler to create `stage0` artifacts which
 will later be uplifted to be the stage1 compiler_.
 
@@ -268,23 +267,6 @@ However, when cross-compiling, `stage1` `std` will only run on the host. So the
 
 (See in the table how `stage2` only builds non-host `std` targets).
 
-### Why does only libstd use `cfg(bootstrap)`?
-
-For docs on `cfg(bootstrap)` itself, see [Complications of
-Bootstrapping](#complications-of-bootstrapping).
-
-The `rustc` generated by the `stage0` compiler is linked to the freshly-built
-`std`, which means that for the most part only `std` needs to be `cfg`-gated, so
-that `rustc` can use features added to `std` immediately after their addition,
-without need for them to get into the downloaded `beta` compiler.
-
-Note this is different from any other Rust program: `stage1` `rustc` is built by
-the _beta_ compiler, but using the _master_ version of `libstd`!
-
-The only time `rustc` uses `cfg(bootstrap)` is when it adds internal lints that
-use diagnostic items, or when it uses unstable library features that were
-recently changed.
-
 ### What is a 'sysroot'?
 
 When you build a project with `cargo`, the build artifacts for dependencies are
@@ -459,7 +441,6 @@ compiler itself uses to run. These aren't actually used by artifacts the new
 compiler generates. This step also copies the `rustc` and `rustdoc` binaries we
 generated into `build/$HOST/stage/bin`.
 
-The `stage1/bin/rustc` is a fully functional compiler, but it doesn't yet have
-any libraries to link built binaries or libraries to. The next 3 steps will
-provide those libraries for it; they are mostly equivalent to constructing the
-`stage1/bin` compiler so we don't go through them individually here.
+The `stage1/bin/rustc` is a fully functional compiler built with stage0 (precompiled) compiler and std.
+To use a compiler built entirely from source with the in-tree compiler and std, you need to build the
+stage2 compiler, which is compiled using the stage1 (in-tree) compiler and std.

src/ci/docker/host-x86_64/mingw-check-2/Dockerfile

@@ -217,7 +217,6 @@ probably the best "go to" command for building a local compiler:
 This may *look* like it only builds the standard library, but that is not the case.
 What this command does is the following:
 
-- Build `std` using the stage0 compiler
 - Build `rustc` using the stage0 compiler
   - This produces the stage1 compiler
 - Build `std` using the stage1 compiler
@@ -241,23 +240,22 @@ build. The **full** `rustc` build (what you get with `./x build
 --stage 2 compiler/rustc`) has quite a few more steps:
 
 - Build `rustc` with the stage1 compiler.
-  - The resulting compiler here is called the "stage2" compiler.
-- Build `std` with stage2 compiler.
+  - The resulting compiler here is called the "stage2" compiler, which uses stage1 std from the previous command.
 - Build `librustdoc` and a bunch of other things with the stage2 compiler.
 
 You almost never need to do this.
 
 ### Build specific components
 
 If you are working on the standard library, you probably don't need to build
-the compiler unless you are planning to use a recently added nightly feature.
-Instead, you can just build using the bootstrap compiler.
+every other default component. Instead, you can build a specific component by
+providing its name, like this:
 
 ```bash
-./x build --stage 0 library
+./x build --stage 1 library
 ```
 
-If you choose the `library` profile when running `x setup`, you can omit `--stage 0` (it's the
+If you choose the `library` profile when running `x setup`, you can omit `--stage 1` (it's the
 default).
 
 ## Creating a rustup toolchain
@@ -271,7 +269,6 @@ you will likely need to build at some point; for example, if you want
 to run the entire test suite).
 
 ```bash
-rustup toolchain link stage0 build/host/stage0-sysroot # beta compiler + stage0 std
 rustup toolchain link stage1 build/host/stage1
 rustup toolchain link stage2 build/host/stage2
 ```

src/ci/docker/host-x86_64/mingw-check-tidy/Dockerfile

@@ -85,7 +85,7 @@ Look for existing targets to use as examples.
 After adding your target to the `rustc_target` crate you may want to add
 `core`, `std`, ... with support for your new target. In that case you will
 probably need access to some `target_*` cfg. Unfortunately when building with
-stage0 (the beta compiler), you'll get an error that the target cfg is
+stage0 (a precompiled compiler), you'll get an error that the target cfg is
 unexpected because stage0 doesn't know about the new target specification and
 we pass `--check-cfg` in order to tell it to check.
 

src/ci/docker/host-x86_64/x86_64-gnu-nopt/Dockerfile

@@ -273,51 +273,15 @@ lets you use `cargo fmt`.
 [the section on vscode]: suggested.md#configuring-rust-analyzer-for-rustc
 [the section on rustup]: how-to-build-and-run.md?highlight=rustup#creating-a-rustup-toolchain
 
-## Faster builds with `--keep-stage`.
-
-Sometimes just checking whether the compiler builds is not enough. A common
-example is that you need to add a `debug!` statement to inspect the value of
-some state or better understand the problem. In that case, you don't really need
-a full build. By bypassing bootstrap's cache invalidation, you can often get
-these builds to complete very fast (e.g., around 30 seconds). The only catch is
-this requires a bit of fudging and may produce compilers that don't work (but
-that is easily detected and fixed).
-
-The sequence of commands you want is as follows:
-
-- Initial build: `./x build library`
-  - As [documented previously], this will build a functional stage1 compiler as
-    part of running all stage0 commands (which include building a `std`
-    compatible with the stage1 compiler) as well as the first few steps of the
-    "stage 1 actions" up to "stage1 (sysroot stage1) builds std".
-- Subsequent builds: `./x build library --keep-stage 1`
-  - Note that we added the `--keep-stage 1` flag here
-
-[documented previously]: ./how-to-build-and-run.md#building-the-compiler
-
-As mentioned, the effect of `--keep-stage 1` is that we just _assume_ that the
-old standard library can be re-used. If you are editing the compiler, this is
-almost always true: you haven't changed the standard library, after all. But
-sometimes, it's not true: for example, if you are editing the "metadata" part of
-the compiler, which controls how the compiler encodes types and other states
-into the `rlib` files, or if you are editing things that wind up in the metadata
-(such as the definition of the MIR).
-
-**The TL;DR is that you might get weird behavior from a compile when using
-`--keep-stage 1`** -- for example, strange [ICEs](../appendix/glossary.html#ice)
-or other panics. In that case, you should simply remove the `--keep-stage 1`
-from the command and rebuild. That ought to fix the problem.
-
-You can also use `--keep-stage 1` when running tests. Something like this:
-
-- Initial test run: `./x test tests/ui`
-- Subsequent test run: `./x test tests/ui --keep-stage 1`
-
-### Iterating the standard library with `--keep-stage`
-
-If you are making changes to the standard library, you can use `./x build
---keep-stage 0 library` to iteratively rebuild the standard library without
-rebuilding the compiler.
+## Faster Builds with CI-rustc  
+
+If you are not working on the compiler, you often don't need to build the compiler tree.
+For example, you can skip building the compiler and only build the `library` tree or the
+tools under `src/tools`. To achieve that, you have to enable this by setting the `download-rustc`
+option in your configuration. This tells bootstrap to use the latest nightly compiler for `stage > 0`
+steps, meaning it will have two precompiled compilers: stage0 compiler and `download-rustc` compiler
+for `stage > 0` steps. This way, it will never need to build the in-tree compiler. As a result, your
+build time will be significantly reduced by not building the in-tree compiler.
 
 ## Using incremental compilation
 

src/ci/github-actions/jobs.yml

@@ -66,9 +66,9 @@ kinds of builds (sets of jobs).
 ### Pull Request builds
 
 After each push to a pull request, a set of `pr` jobs are executed. Currently,
-these execute the `x86_64-gnu-llvm-X`, `x86_64-gnu-tools`, `mingw-check` and
-`mingw-check-tidy` jobs, all running on Linux. These execute a relatively short
-(~30 minutes) and lightweight test suite that should catch common issues. More
+these execute the `x86_64-gnu-llvm-X`, `x86_64-gnu-tools`, `mingw-check-1`, `mingw-check-2`
+and `mingw-check-tidy` jobs, all running on Linux. These execute a relatively short
+(~40 minutes) and lightweight test suite that should catch common issues. More
 specifically, they run a set of lints, they try to perform a cross-compile check
 build to Windows mingw (without producing any artifacts) and they test the
 compiler using a *system* version of LLVM. Unfortunately, it would take too many