std::fs::canonicalize returns UNC paths on Windows, and a lot of software doesn't support UNC paths

[radix]

Hi, I hope this is the right forum/format to register this problem, let me know if it's not.

Today I tried to use std::fs::canonicalize to make a path absolute so that I could execute it with std::process::Command. canonicalize returns so-called "UNC paths", which look like this: \\?\C:\foo\bar\... (sometimes the ? can be a hostname).

It turns out you can't pass a UNC path as the current directory when starting a process (i.e., Command::new(...).current_dir(unc_path)). In fact, a lot of other apps will blow up if you pass them a UNC path: for example, Microsoft's own cl.exe compiler doesn't support it: rust-lang/cc-rs#169

It feels to me that maybe returning UNC paths from canonicalize is the wrong choice, given that they don't work in so many places. It'd probably be better to return a simple "absolute path", which begins with the drive letter, instead of returning a UNC path, and instead provide a separate function specifically for generating UNC paths for people who need them.

Maybe if this is too much of an incompatible change, a new function for creating absolute paths should be added to std? I'd bet, however, that making the change to canonicalize itself would suddenly make more software suddenly start working rather than suddenly break.

[retep998]

canonicalize simply asks the kernel to canonicalize the path, which it does and happens to return the canonical path as a root local device path. Root local device paths are capable of representing some paths which normal absolute paths are incapable of representing accurately (such as components being named ".." or "." or having "/" in them), along with the fact that they're the only way to call many system functions with paths longer than MAX_PATH (aside from being on some recent version of Windows 10 and having a certain registry key enabled). As a result having libstd automatically strip the prefix would definitely break some situations. However I'm definitely in favor of having more support in libstd for converting between these different kinds of paths so the user can easily turn a root local device path into an absolute path. I'd also love to have an fs::normalize which merely normalizes a possibly relative path into an absolute path without hitting the filesystem on Windows.

[retep998]

In reference to your commit which referenced this PR, normalization is not the same as merely joining the path onto the current directory due to drive relative paths being relative to the current directory on the given drive. For example given a drive relative path of C:foo, and an env::current_dir() of D:\bar, normalizing C:foo will have to get the current directory for C:\ and could end up being normalized to something radically different such as C:\i\dont\even\foo.

[radix]

thanks, @retep998 :) it's just a hacked-together build tool that probably will eventually be replaced with something else, and I didn't intend to notify this ticket about my commit. but I guess it goes to show that a good way to get an absolute path in std would be really helpful.

[nagisa]

Command::current_dir should be fixed. I doubt we will change canonlicalize.

Note, the i-wrong tag is only for the Command::current_dir, not the canonicalize behaviour.

[retep998]

Quick testing on Windows 10.0.15063 indicates that both SetCurrentDirectoryW and CreateProcessW are okay with a current directory starting with \\?\. They are not okay with a current directory that exceeds MAX_PATH regardless of \\?\. CreateProcessW is okay with the path to the process itself starting with \\?\ regardless of whether the first parameter is used. CreateProcessW is only okay with the path to the process exceeding MAX_PATH if it starts with \\?\ and is specified as the first parameter which Rust does not currently use. I tested std::process::Command::current_dir and it works as expected, accepting paths starting with \\?\ but rejecting any paths exceeding MAX_PATH.

[kornelski]

Technically, AFAIK it is safe to strip the prefix in common simple cases (absolute path with a drive letter, no reserved names, shorter than max_path), and leave it otherwise.

So I think there's no need to compromise on correctness as far as stdlib goes. The trade-off is between failing early and exposing other software that doesn't support UNC paths vs maximizing interoperability with non-UNC software.

In an ideal world, I would prefer the "fail early" approach, so that limitations are quickly found and removed. However, Windows/DOS path handling has exceptionally long and messy history and decades of Microsoft bending over backwards to let old software not upgrade its path handling. If Microsoft can't push developers towards UNC, and fails to enforce this even in their own products, I have no hope of Rust shifting the Windows ecosystem to UNC. It will rather just frustrate Rust users and make Rust seem less reliable on Windows.

So in this case I suggest trying to maximize interoperability instead, and canonicalize to regular paths whenever possible (using UNC only for paths that can't be handled otherwise).

Also, careful stripping of the prefix done in stdlib will be much safer than other crates stripping it unconditionally (because realistically whenever someone runs into this problem, they'll just strip it unconditionally)