-Zremap-cwd-prefix is not applied to the sysroot rlib paths

[danakj]

This breaks deterministic builds on Windows (see #88982 (comment)), where the PDB ends up including the paths to the object files. If a relative path is given to --sysroot, the path is turned into an absolute path for each rlib, and -Zremap-cwd-prefix is not applied.

[danakj]

cc: @michaelwoerister

[bjorn3]

Does the linker have a way to remap the paths of object files as they will end up in the .pdb file?

[danakj]

I don't think so, no.

What lld-link does have is a way to specify "for relative source file path X, map it to PREFIX\X to make an absolute path for debugging". This is /pdbsourcepath, but it is not the same as stripping paths, it requires source paths to be relative to use this.

lld-link has no way to remove parts of paths given to it.
link.exe does not either that I can find.

(The paths to the object files is not used in debugging, I believe? Just the paths to source files, which are currently remapped through -Zremap-cwd-prefix.)

[bjorn3]

If it is not used that begs the question why it is even recorded in the first place :) On a more useful note, the crate loader uses std::fs::canonicalize to resolve symlinks and prevent those from causing crates to be potentially loaded multiple times which would error. Only the canonicalized path is then recorded for use by the linker code. I guess it would be possible to check in the linker code if a library path has the working directory as prefix and turn it back into a relative path or something like that. I guess it could look at the --sysroot argument and it's canonicalized path to also allow producing paths like ../../path/to/sysroot/lib/rustlib/x86_64-unknown-linux-gnu/lib/libstd-*.rlib.

[danakj]

I feel the same but I don't make the rules for the PDBs. Ok thank you for the pointer, I will have a look at how to do this.

[bjorn3]

@rustbot label +A-reproducibility +O-windows-msvc

[danakj]

A PR to apply path remapping to linker command line outputs: #112597

I verified that the Chromium build stops seeing any delta (from these paths) with the patch applied and building the same target in two different directories.

The Windows tests for bins can't yet be enabled by this PR tho because #112587 needs to be addressed first.

[michaelwoerister]

Before we jump into a fix, I would find it helpful to get a more complete picture of the issue and possible solutions. I have some questions:

• Which paths are we talking about here exactly? Only the paths to rlib files? Or do /LIBPATH, /NATVIS, and /OUT make a difference too?
• Does it help to provide this paths as relative paths or will the linker internally just expand these to absolute paths again?
• Is it only the PDB that is affected or the executable too?
• Do link.exe and lld-link.exe behave the same?

If these are paths that tell the linker where to find files on disk, then I suspect we can't remap them on the rustc side.

[danakj]

The paths that are appearing in the PDB are:

1. The stdlib rlib files: /home/foo/bar/.../rustlib/x86_64-pc-windows-msvc/lib/libstd.rlib etc.
2. The bin crate's object file, /tmp/<random>/symbols.o

The paths to other arguments (/LIBPATH, /NATVIS, /OUT) are either already relative (unlikely) or are not appearing in the PDB, at least with lld-link.

Providing them as relative paths to the linker works, the binary differences go away.

The PDB and EXE are affected, but the deltas in the PDB are the paths (which is easy to see) while the deltas in the EXE are less clear. However they are derived from the paths, because when the paths in the PDB are fixed, the deltas in the PDB and EXE both go away.

I will have to try link.exe to verify.

If these are paths that tell the linker where to find files on disk, then I suspect we can't remap them on the rustc side.

I believe we need to differentiate remap-path-prefix (can make arbitrary paths) and remap-cwd-prefix (must make a valid path).

Here's the high level way I think we can do so, from the session options:

    /// Returns a type to perform mapping of file paths as specified by the
    /// user. These paths should not be used for IO as they may map to invalid
    /// locations that are designed to just be more clear in diagnostics.
    pub fn file_path_mapping(&self) -> FilePathMapping {
        FilePathMapping::new(&self.remap_path_prefix)
    }

    /// Returns a type to perform mapping of file paths as specified by the
    /// user. These paths may be used for IO and are required to produce a valid
    /// path to the same file or directory.
    pub fn file_path_mapping_for_io(&self) -> FilePathMapping {
        FilePathMapping::new_for_io(&self.remap_path_prefix)
    }

The majority of uses cases would use file_path_mapping. The cmd line args to other tools (the linker) would use file_path_mapping_for_io.