Integrate 'uv' for enhanced package management by arash77 · Pull Request #21002 · galaxyproject/galaxy

arash77 · 2025-10-03T14:16:12Z

Implement uv to improve installation speed and streamline virtual environment creation across scripts. This change ensures faster package management and eliminates the need for manual pip upgrades.

How to test the changes?

(Select all options that apply)

I've included appropriate automated tests.
This is a refactoring of components with existing test coverage.
Instructions for manual testing are as follows:
1. Try running Galaxy, tests, updates, or anything related to pip packages and see how quickly it fetches them.

License

I agree to license these and all my past contributions to the core galaxy codebase under the MIT license.

mvdbeek

Overall I love it, but I think we shouldn't just bootstrap it. If it's installed, we should use it, do you think you could implement that ?

arash77 · 2025-10-06T09:20:17Z

@nsoranzo In update.sh we are already forcing UV installation. Should we move the installation of UV to one level higher in the CI job and instead use the system UV if it exists in the bash code?

if command -v uv >/dev/null; then
    uv="$(command -v uv)"
else
    uv_venv=$(mktemp -d "${TMPDIR:-/tmp}/uv_venv.XXXXXXXXXX")
    python3 -m venv "${uv_venv}"
    "${uv_venv}/bin/python" -m pip install uv
    uv="${uv_venv}/bin/uv"
fi

arash77 · 2025-10-06T12:05:29Z

To use this in GitHub Actions, we need to add astral-sh/setup-uv as a step in our workflows.

arash77 · 2025-10-09T07:24:10Z

Do we have any plans to use uv.lock in future?

jmchilton · 2025-10-14T13:09:46Z

        with:
          python-version: ${{ matrix.python-version }}
-          cache: 'pip'
-          cache-dependency-path: 'galaxy root/requirements.txt'


Are we loosing this caching or is UV doing something similar under the hood?

UV has its own caching system, which is enabled by default on GitHub-hosted runners.
I'm not sure whether we should use the cache-dependency-glob option to add our requirements, or if it picks them up automatically.

jmchilton · 2025-10-14T13:11:57Z

This will conflict with #21016 - but I am happy to update it after this is merged. Exciting stuff for sure. In this commit the uv supports made that test.sh file much uglier - but longer term after #21016 actually UV makes that whole pipeline much cleaner - it seems quite capable of allowing us to drop our ad-hoc DAG tracking and our custom incremental building of dependent packages.

jmchilton · 2025-10-14T13:17:39Z

Do we have any plans to use uv.lock in future?

I guess a non-response from anyone is a no? We do uv export --frozen --no-annotate --no-hashes --no-dev which is similar in some sense - a much worse uv experience but it does make the whole workflow compatible with pip (and maybe conda also to some degree?). Until we are ready to drop pip support I guess we should stick with that pipeline - but I don't know - there might be reasons to use uv.lock when using uv and accepting some duplication - I'm not sure.

nsoranzo · 2025-10-14T13:36:24Z

Do we have any plans to use uv.lock in future?

I guess a non-response from anyone is a no? We do uv export --frozen --no-annotate --no-hashes --no-dev which is similar in some sense - a much worse uv experience but it does make the whole workflow compatible with pip (and maybe conda also to some degree?). Until we are ready to drop pip support I guess we should stick with that pipeline - but I don't know - there might be reasons to use uv.lock when using uv and accepting some duplication - I'm not sure.

I worry that reviewing the "Update Python dependencies" pull requests with only uv.lock might be harder, but we can try with shipping both uv.lock and the requirements.txt files for a cycle?

arash77 · 2025-10-20T15:24:02Z

The failed test seems to be unrelated, but it keeps happening!

mvdbeek

Looks good to me, @nsoranzo do you want to take another look ?

nsoranzo · 2025-10-28T13:58:26Z

The "Test Galaxy packages" errors are different than usual, not sure if it's related to this PR.

arash77 · 2025-10-28T16:02:18Z

The "Test Galaxy packages" errors are different than usual, not sure if it's related to this PR.

I think the problem is that when using uv, we need to install without resolving dependencies from the lock file or pulling them from PyPI, since we're installing in dependency order from local packages

Using `uv tool install` instead of `uv pip install --system` Co-authored-by: Marius van den Beek <m.vandenbeek@gmail.com>

Co-authored-by: Nicola Soranzo <nicola.soranzo@gmail.com>

…ation Co-authored-by: Nicola Soranzo <nicola.soranzo@gmail.com>

…using 'uv'

…e for improved readability and maintainability

in case it's installed inside a virtual environment.

`uv pip` installs directories as editable packages by default. mypy does not recognize that libraries installed as editable using setuptools are fully typed even though the `py.typed` file is correctly included. This combination results in spurious mypy errors in package tests if packages are installed via `uv pip`. See also: astral-sh/uv#3898 pypa/setuptools#3518

github-actions · 2025-10-30T13:40:21Z

This PR was merged without a "kind/" label, please correct.

jmchilton · 2025-10-30T13:43:10Z

I'm sure I introduced some conflicts and I'm sorry about that. This is a pretty exciting step forward though - thank you so much for the implementation @arash77 and thanks for the detailed review @nsoranzo.

github-actions Bot added area/testing area/scripts area/dependencies area/packaging labels Oct 3, 2025

mvdbeek reviewed Oct 3, 2025

View reviewed changes

nsoranzo reviewed Oct 3, 2025

View reviewed changes

Comment thread test/release.sh Outdated

nsoranzo reviewed Oct 6, 2025

View reviewed changes

Comment thread lib/galaxy/dependencies/update.sh

arash77 marked this pull request as ready for review October 7, 2025 13:12

github-actions Bot added this to the 26.0 milestone Oct 7, 2025

jmchilton reviewed Oct 14, 2025

View reviewed changes

mvdbeek self-assigned this Oct 14, 2025

mvdbeek reviewed Oct 14, 2025

View reviewed changes

Comment thread .github/workflows/converter_tests.yaml Outdated

mvdbeek reviewed Oct 14, 2025

View reviewed changes

Comment thread .github/workflows/docs.yaml Outdated

mvdbeek reviewed Oct 14, 2025

View reviewed changes

Comment thread .github/workflows/publish_artifacts.yaml Outdated

arash77 force-pushed the replace-pip-with-uv branch 2 times, most recently from 5af8e95 to 18d6672 Compare October 20, 2025 13:17

arash77 force-pushed the replace-pip-with-uv branch from 18d6672 to 85a8b39 Compare October 21, 2025 15:59

arash77 requested a review from a team October 24, 2025 08:10

mvdbeek approved these changes Oct 27, 2025

View reviewed changes

arash77 force-pushed the replace-pip-with-uv branch from 85a8b39 to 4575c13 Compare October 27, 2025 14:27

nsoranzo reviewed Oct 28, 2025

View reviewed changes

Comment thread lib/galaxy/dependencies/update_lint_requirements.sh Outdated

Comment thread lib/galaxy/dependencies/update_lint_requirements.sh Outdated

Comment thread packages/package-build-install.sh Outdated

Comment thread packages/test.sh Outdated

Comment thread test/release.sh

arash77 and others added 12 commits October 29, 2025 12:45

Streamlines wheel + conditional dependency setup

137928e

Remove minimum required version for 'uv'

453334f

Apply suggestions from code review

f191768

Using `uv tool install` instead of `uv pip install --system` Co-authored-by: Marius van den Beek <m.vandenbeek@gmail.com>

Upgrade 'uv' setup action to version 7 across all workflow files

bb7d2e9

Update UV on running update_lint_requirements.sh

591237e

Co-authored-by: Nicola Soranzo <nicola.soranzo@gmail.com>

Refactor virtual environment setup to use VIRTUAL_ENV for pip install…

0a75c2f

…ation Co-authored-by: Nicola Soranzo <nicola.soranzo@gmail.com>

Enhance ensure_prereqs function to use 'uv' for pip list if available

e4f6aa3

Add pip installation for wheel and packaging in create_venv function …

25f5eb8

…using 'uv'

Refactor test.sh to unify pip installation commands using PIP variabl…

3236b75

…e for improved readability and maintainability

Save whole uv path in PIP env variable

0ec6c3b

in case it's installed inside a virtual environment.

Cleanup formatting of pyproject.toml files

5600065

nsoranzo force-pushed the replace-pip-with-uv branch 2 times, most recently from 2cfa6bc to 2430d3c Compare October 29, 2025 13:32

nsoranzo added 2 commits October 29, 2025 14:07

Remove last traces of test-requirements.txt

3c41fe0

Disallow also fastapi 0.120.1 and 0.120.2

c68f850

nsoranzo force-pushed the replace-pip-with-uv branch 4 times, most recently from 4c8236a to 23b47ee Compare October 29, 2025 18:39

nsoranzo added 3 commits October 29, 2025 18:45

Reduce command duplications

ae62f12

Install uv into $GALAXY_CONDA_ENV

6cf9d49

Shellcheck fixes

f20306b

nsoranzo force-pushed the replace-pip-with-uv branch from 23b47ee to f20306b Compare October 29, 2025 18:45

nsoranzo approved these changes Oct 29, 2025

View reviewed changes

jmchilton merged commit f23a469 into galaxyproject:dev Oct 30, 2025
68 of 72 checks passed

arash77 deleted the replace-pip-with-uv branch October 30, 2025 13:41

arash77 added the kind/enhancement label Oct 30, 2025

Conversation

arash77 commented Oct 3, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

How to test the changes?

License

Uh oh!

mvdbeek left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

arash77 commented Oct 6, 2025

Uh oh!

Uh oh!

arash77 commented Oct 6, 2025

Uh oh!

arash77 commented Oct 9, 2025

Uh oh!

jmchilton Oct 14, 2025 • edited by nsoranzo Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

arash77 Oct 14, 2025

Choose a reason for hiding this comment

Uh oh!

jmchilton commented Oct 14, 2025

Uh oh!

jmchilton commented Oct 14, 2025

Uh oh!

nsoranzo commented Oct 14, 2025

Uh oh!

Uh oh!

Uh oh!

Uh oh!

arash77 commented Oct 20, 2025

Uh oh!

mvdbeek left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

nsoranzo commented Oct 28, 2025

Uh oh!

arash77 commented Oct 28, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

github-actions Bot commented Oct 30, 2025

Uh oh!

jmchilton commented Oct 30, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

arash77 commented Oct 3, 2025 •

edited

Loading

jmchilton Oct 14, 2025 •

edited by nsoranzo

Loading

arash77 commented Oct 28, 2025 •

edited

Loading