Skip to content

Windows: OsStr need not be valid UTF-8 #56171

Closed
@DemiMarie

Description

@DemiMarie
Contributor

On Windows, filenames (and many other strings) are stored in UCS-2, not UTF-8. This corresponds to WTF-8, not UTF-8. Furthermore, the NT kernel uses length-specified strings via the UNICODE_STRING struct, and uses only \ as the path separator. Therefore, many APIs (such as NtCreateFile and NtCreateKey) actually allow NUL characters in names. While the NTFS driver does not allow NUL to be part of a filename, registry keys can have NUL characters in there names.

Therefore, I propose that OsStr on Windows be able to be created from any valid WTF-8, and that an API be provided that allows them to be created with NUL characters.

Also see #53261 (comment)

Activity

ollie27

ollie27 commented on Nov 22, 2018

@ollie27
Member

Are OsStrExt and OsStringExt in std::os::windows::ffi not sufficient?

retep998

retep998 commented on Nov 23, 2018

@retep998
Member

OsStr literally already is this. It stores WTF-8 instead of UTF-8 and is capable of holding NUL characters just fine.

added
O-windowsOperating system: Windows
A-docsArea: Documentation for any part of the project, including the compiler, standard library, and tools
on Jan 11, 2019
estebank

estebank commented on Jan 11, 2019

@estebank
Contributor

Leaving open as under the purview of @rust-lang/docs, like #53261.

Mark-Simulacrum

Mark-Simulacrum commented on Aug 6, 2019

@Mark-Simulacrum
Member

Closing since this seems to be satisfied by the existing APIs (per above comments and reading the API docs today). #53261 touches on the more specific issue of std::ffi docs being misleading about the underlying representation of OsString on Windows, but that's an orthogonal and separate concern.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    A-docsArea: Documentation for any part of the project, including the compiler, standard library, and toolsO-windowsOperating system: Windows

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

      Development

      No branches or pull requests

        Participants

        @retep998@estebank@Mark-Simulacrum@DemiMarie@ollie27

        Issue actions

          Windows: OsStr need not be valid UTF-8 · Issue #56171 · rust-lang/rust