Skip to content

Feat: free-threaded python support #3526

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
May 13, 2025

Conversation

robsdedude
Copy link
Contributor

@robsdedude robsdedude commented May 11, 2025

This PR adds support for experimental no-GIL CPython builds (aka. free-threaded).

It gives special semantics to factors like py313t which will then only select free-threaded Python 3.13.

Note

This also means that factors like py313 will no longer match free-threaded CPython builds.

Related pypa/virtualenv#2809,

Closes: #3391

  • ran the linter to address style issues (tox -e fix)
  • wrote descriptive pull request text
  • ensured there are test(s) validating the fix
  • added news fragment in docs/changelog folder
  • updated/extended the documentation

Please be aware that

  • This is potentially a breaking change. If people relied on factors like py313 potentially also picking a free-threaded CPython build, that would break and they'd have to update their tox factor name to py313t. I'm not sure there's a design that circumvents that.
  • This is the first time I'm digging through tox' code-base. So it might well be that I've missed places that need adjustment, too. In such case, I'm happy for any pointers.

Sorry, something went wrong.

@gaborbernat
Copy link
Member

The approach looks good 😁

@robsdedude robsdedude force-pushed the feat/free-threaded-python branch from 233dba0 to f00b109 Compare May 12, 2025 21:14
@robsdedude robsdedude force-pushed the feat/free-threaded-python branch 3 times, most recently from d804cfb to 5a4298b Compare May 12, 2025 21:48
@robsdedude
Copy link
Contributor Author

@gaborbernat that's interesting. The readthedocs build is failing because it's installing tox-uv which is very unhappy with this change as it's a breaking API change (PythonInfo gained a new field and constructor parameter: free_threaded). I never thought of tox as an API only CLI, but I suppose especially with plugins in mind, I better should have 😜

Now the question is how to best proceed. WDYT? Give the attribute a default value? Accept breaking the API? Something else entirely?

@gaborbernat
Copy link
Member

Give the attribute a default value?

This 😊

@gaborbernat gaborbernat force-pushed the feat/free-threaded-python branch 2 times, most recently from 6d51f85 to bcc9f37 Compare May 13, 2025 14:12

Verified

This commit was signed with the committer’s verified signature.
gaborbernat Bernát Gábor
@gaborbernat gaborbernat force-pushed the feat/free-threaded-python branch 5 times, most recently from 7f6329f to 04f2bd4 Compare May 13, 2025 14:37

Verified

This commit was signed with the committer’s verified signature.
gaborbernat Bernát Gábor
Signed-off-by: Bernát Gábor <[email protected]>
@gaborbernat gaborbernat force-pushed the feat/free-threaded-python branch from 04f2bd4 to 4ccf8ad Compare May 13, 2025 14:51

Verified

This commit was signed with the committer’s verified signature.
gaborbernat Bernát Gábor
Signed-off-by: Bernát Gábor <[email protected]>
@gaborbernat gaborbernat marked this pull request as ready for review May 13, 2025 14:56
@gaborbernat gaborbernat self-requested a review as a code owner May 13, 2025 14:56
@gaborbernat gaborbernat merged commit 92d4ed3 into tox-dev:main May 13, 2025
28 checks passed
@robsdedude
Copy link
Contributor Author

robsdedude commented May 13, 2025

I was about to pick this up again. Thanks for continuing. There was also another test I was about to adjust. Further, I wanted to discuss the fact that PythonInfo is a NamedTuple. So adding a field will break code that structurally unpacks it e.g., (impl, _, version, _, _, extra) = python_info. Up to you to decide how/whether to handle this API incompatibility. Lastly, I think the free_threaded info should also be included in the json log, but it's not yet (only in my local changes 😬 )

@robsdedude robsdedude deleted the feat/free-threaded-python branch May 13, 2025 15:37
@gaborbernat
Copy link
Member

I wanted to discuss the fact that PythonInfo is a NamedTuple. So adding a field will break code that structurally unpacks it e.g., (impl, _, version, _, _, extra) = python_info. Up to you to decide how/whether to handle this API incompatibility.

Yeah made it a dataclass. I'm fine breaking unpacking here.

There was also another test I was about to adjust.

Feel free to put in a new PR if there's anything else we can improve. Thanks!

@astrofrog
Copy link

I am trying to use this, and noticed that if I do e.g. tox -e py313t-test on a machine that does not have free-threaded Python installed, the build still runs and silently uses the non-freethreaded version of Python. It might be worth making sure that py313t fails if a free-threaded Python interpreter cannot be found?

@gaborbernat
Copy link
Member

gaborbernat commented Jun 3, 2025

I am trying to use this, and noticed that if I do e.g. tox -e py313t-test on a machine that does not have free-threaded Python installed, the build still runs and silently uses the non-freethreaded version of Python. It might be worth making sure that py313t fails if a free-threaded Python interpreter cannot be found?

PR welcome 😁

@robsdedude
Copy link
Contributor Author

@astrofrog I cannot reproduce the issue. Here's what I tried (inside a ubuntu:24.04 docker container)

export DEBIAN_FRONTEND=noninteractive
cd /root
apt update
apt install -y locales
localedef -i en_US -c -f UTF-8 -A /usr/share/locale/locale.alias en_US.UTF-8
export LANG=en_US.UTF-8
apt-get install -y --no-install-recommends build-essential libssl-dev zlib1g-dev libbz2-dev libreadline-dev libsqlite3-dev wget curl llvm libncurses5-dev xz-utils tk-dev libxml2-dev libxmlsec1-dev libffi-dev liblzma-dev ca-certificates vim git software-properties-common tar
curl -fsSL https://pyenv.run | bash
export PYENV_ROOT="$HOME/.pyenv"
[[ -d $PYENV_ROOT/bin ]] && export PATH="$PYENV_ROOT/bin:$PATH"
eval "$(pyenv init - bash)"
pyenv install 3.13
pyenv install 3.12
pyenv global 3.13 3.12
pyenv rehash
python3.12 -m pip install tox

mkdir -p /root/tmp
cd /root/tmp
cat << EOF > pyproject.toml
[project]
name = "foo"
version = "0.0.1"

[build-system]
requires = ["setuptools >= 77.0.3"]
build-backend = "setuptools.build_meta"

[tool.tox]
requires = ["tox>=4.26"]
env_list = ["py313-test"]

[tool.tox.env.py313-test]
commands = [
    ["python", "-VV"]
]
EOF

# works, prints "Python 3.13.4 (main, Jun  7 2025, 13:03:46) [GCC 13.3.0]"
python3.12 -m tox

sed -i 's/py313-test/py313t-test/' pyproject.toml
# fails as expected with "py313t: skipped because could not find python interpreter with spec(s): py313t"
python3.12 -m tox

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Free threaded Python support
3 participants