Skip to content

fix: handle None sys.stdout.encoding in CLI output#1656

Open
octo-patch wants to merge 1 commit intomicrosoft:mainfrom
octo-patch:fix/issue-1597-stdout-encoding-none-fallback
Open

fix: handle None sys.stdout.encoding in CLI output#1656
octo-patch wants to merge 1 commit intomicrosoft:mainfrom
octo-patch:fix/issue-1597-stdout-encoding-none-fallback

Conversation

@octo-patch
Copy link
Copy Markdown

Fixes #1597

Problem

When sys.stdout.encoding is None (e.g. when stdout is replaced with a
raw binary stream, or in certain CI/redirected environments), the current
_handle_output code in __main__.py calls:

result.markdown.encode(sys.stdout.encoding, errors="replace")

This raises TypeError: encode() argument 'encoding' must be str, not None
instead of the expected UnicodeEncodeError, making the CLI crash with an
unhelpful error.

Solution

Fall back to "utf-8" when sys.stdout.encoding is None:

encoding = sys.stdout.encoding or "utf-8"
print(result.markdown.encode(encoding, errors="replace").decode(encoding))

The existing errors="replace" handling is preserved, so non-UTF-8
terminals (e.g. Windows cp1252) still work correctly.

Testing

Manually verified that the existing errors="replace" path continues to
work, and that the None fallback is triggered correctly:

import sys
enc = None
# Before: raises TypeError
# After: falls back to utf-8 safely
encoding = enc or "utf-8"
print("test \uff89".encode(encoding, errors="replace").decode(encoding))

)

When sys.stdout.encoding is None (e.g. when stdout is redirected to a
binary stream), calling str.encode(None) raises a TypeError. Fall back
to utf-8 in that case so the CLI does not crash.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Unicode error converting Microsoft Documentation PDF to markdown file.

1 participant