Pip upgradeall

[Marriaga]

• Pip version: 9.0.1
• Python version: 2.7.13
• Operating system: Win10

Description:

I believe that there should be an option to upgrade all packages that are outdated. This seems to me like an essential command that is missing and there are threads online with many people looking for this.

What I've run:

usage:
pip install --upgradeall

code to add in InstallCommand:
__init__

cmd_opts.add_option(
            '--upgradeall',
            dest='upgradeall',
            action='store_true',
            help='Upgrade all outdated packages to the newest available '
                 'version. The handling of dependencies depends on the '
                 'upgrade-strategy used.'
        )

run

if options.upgradeall:
    args=[dist.project_name for dist in get_installed_distributions()]

[RonnyPfannschmidt]

@Marriaga as pypi is not a curated set of distribution packages, this is also likely to break the system

after all unlike a distribution like debian or fedora, that is managed in curated releases of "packages", pypi is free for all

[Marriaga]

Forgive my ignorance, I'm not so familiar with the challenges of having a non-curated set of distribution packages. From my understanding after reading #988, the issue is with managing multiple package dependencies and resolving conflicts.

However, I would note that this is a highly desired feature and most people end up circumventing this anyway by making their own scripts. I think the safer thing for most users would be to have something that works and gives proper warnings. Below is what I did. It is most likely a silly code, which proves my point. If smart pip people did a nice code that works for simple cases and gives nice warnings when things get complicated, then I wouldn't have to write this half-baked solution :)

Could this be a case of "the squeaky wheel gets the oil"? I see the possibility that most users are probably just using pip to install simple things like numpy and matplotlib and would benefit tremendously from an --upgradeall feature.

import pip

LC = pip.commands.list.ListCommand()
options, args = LC.parse_args(["--outdated"])
outd_dists = LC.get_outdated(pip.get_installed_distributions(), options)

for dist in outd_dists():
    pip.main(["install", "--upgrade", dist.project_name])

[pfmoore]

The problem is "something that works and gives proper warnings". It's very easy to blindly upgrade things and end up with a broken system (the technical point here is that "pip doesn't have a proper dependency resolver"). Until that's addressed, having something in pip that does the wrong thing is worse than having nothing.

We're working on a dependency resolver. So this is being worked on - it's not that we're ignoring the issue, just that it's a lot harder than people realise, when they look purely at their own system.

[pfmoore]

Specifically, by the way, your script would break certain systems that have Jupyter installed, because there's a dependency on html5lib that excludes the latest version, so blindly updating everything breaks that dependency. (I know because I broke my system precisely that way!!!)

[Marriaga]

I totally agree with you that doing the wrong thing is a bad idea because it creates a false sense of security. But precisely because so many people are looking for this and using scripts like you and I did, that I think that at this point people have this sense of security from the script. Therefore, in this particular case, I think that by having it in pip, at least when a user tries to do an upgradeall, they could be prompted with a message saying something like: Warning! Upgrade all is not recommended as it may potentially break your system. Do you want to proceed? (Y,n) and they would at least be aware that there might be an issue.

Maybe it could even do a backup of the current system's installed packages and versions and add to the previous message something like: Note that a backup of the current setup of the system will be made and stored in "C:/path_to_pip/backup/Date_of_today_backup". To revert do pip install --revertupgrade [backupfile]`

[pfmoore]

While I don't necessarily disagree with your suggestion, we have extremely limited developer resource for working on pip, so working on a known-wrong solution like this (even with appropriate warnings) is way down the priority list - and in particular, ranks well below working on a complete solution.

[pradyunsg]

@Marriaga Indeed what you request is a oft-requested feature in pip. #59 tracks this feature request for an "upgrade-all" command. IMO, this issue should be closed as a duplicate of it.

As @pfmoore says - properly implementing this functionality is dependent on #988 - which is a difficult problem to solve. Unless that is addressed, this functionality would not be implemented as a part of pip. I don't think any pip developer would want to a half-baked implementation that breaks your environment half the time to be where the (limited) developer time available is put.

[Marriaga]

Thanks for the answers. I'll close the issue now. Before I go I would just like to make a comment that, considering that #59 is 6 years old and #988 is 4 years old, sometimes a half-baked quick non-perfect solution can be better than a perfect solution that never arrives :)

[RonnyPfannschmidt]

@Marriaga please consider that there is a important difference between breaks in edge cases and is expected to break very regular

in this case the half baked solution would break regular and often - and the fallout is simply not something anyone should wish to impose on users, contributors and community based supporters alike

[simion]

Exactly for this kind of need I've written https://github.com/simion/pip-upgrader
Using it since release and it's a real value for periodiuc package upgrades at work 😄

[serafeimgr]

I also made a gist if for a quick workaround for this.

https://gist.github.com/serafeimgr/b4ca5d0de63950cc5349d4802d22f3f0

[defjaf]

Based on and with thanks to @serafeimgr's gist, I've written a possibly-useful command-line tool, pip_upgrade_outdated; source on github. Feedback welcome.

[RonnyPfannschmidt]

@defjaf as far as i can tell that has exactly all the horrible issues we outlind before and then some more stemming from bad use of parallel execution

its something i would bann in my own infrastructure

[defjaf]

@RonnyPfannschmidt I agree about parallel execution, which is why it’s not the default. And I agree that it can break things. But I nonetheless use it all the time with no harmful effects (and prior to writing it, I just did the same thing by hand), so I think it’s pretty useful. YMMV, etc.