Skip to content

Option "install-types" breaks project version dependencies. #17852

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
kubotty opened this issue Sep 30, 2024 · 4 comments
Closed

Option "install-types" breaks project version dependencies. #17852

kubotty opened this issue Sep 30, 2024 · 4 comments
Labels
bug mypy got something wrong

Comments

@kubotty
Copy link

kubotty commented Sep 30, 2024

Bug Report

When running mypy with the "install-types" option and the project includes pandas, the pandas-stubs package is installed into the project, causing the installation of the latest version of numpy (numpy 2.1.1). This breaks my project's dependency on numpy version 1.21.6.

To Reproduce

I have provided the minimum environment to reproduce the issue.

  • pyproject.toml
[project]
name = "mypy-debug"
version = "0.1.0"
dependencies = [
    "numpy==1.21.6",
    "pandas==1.4.4",
]
requires-python = ">=3.10, <3.11"

[project.optional-dependencies]
dev = [
    "mypy>=1.11.2",
]
  • mypy.ini
[mypy]
strict_optional = True
disallow_any_generics = True
disallow_untyped_defs = True
install_types = True

[mypy.overrides]
module = ["scipy.*", "sklearn.*"]
ignore_missing_imports = True
  • __init__.py
import pandas as pd
  • Folder Structure
.
├── src/
│   └── mypy_debug/
│        └── __ini__.py
├── mypy.ini
└── pyproject.toml
  • Environment

    • OS: Windows 11
    • Python: 3.10.11
    • Mypy
    > mypy -V
    mypy 1.11.2 (compiled: yes)
    
  • Running Command

> pip install -e ./[dev]
> mypy .\src\
src\mypy_debug\__init__.py:1: error: Library stubs not installed for "pandas"  [import-untyped]
src\mypy_debug\__init__.py:1: note: Hint: "python3 -m pip install pandas-stubs"
src\mypy_debug\__init__.py:1: note: (or run "mypy --install-types" to install all missing stub packages)
src\mypy_debug\__init__.py:1: note: See https://mypy.readthedocs.io/en/stable/running_mypy.html#missing-imports
Found 1 error in 1 file (checked 1 source file)

Installing missing stub packages:
D:\mypy_debug\.venv\Scripts\python.exe -m pip install pandas-stubs

Collecting pandas-stubs
  Using cached pandas_stubs-2.2.2.240909-py3-none-any.whl (157 kB)
Collecting numpy>=1.23.5
  Using cached numpy-2.1.1-cp310-cp310-win_amd64.whl (12.9 MB)
Collecting types-pytz>=2022.1.1
  Using cached types_pytz-2024.2.0.20240913-py3-none-any.whl (5.3 kB)
Installing collected packages: types-pytz, numpy, pandas-stubs
  Attempting uninstall: numpy
    Found existing installation: numpy 1.21.6
    Uninstalling numpy-1.21.6:
      Successfully uninstalled numpy-1.21.6
ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
mypy-debug 0.1.0 requires numpy==1.21.6, but you have numpy 2.1.1 which is incompatible.
Successfully installed numpy-2.1.1 pandas-stubs-2.2.2.240909 types-pytz-2024.2.0.20240913

[notice] A new release of pip is available: 23.0.1 -> 24.2
[notice] To update, run: python.exe -m pip install --upgrade pip

note: Run mypy again for up-to-date results with installed types

Expected Behavior

The installation process should recognize the installed version of pandas and install the matching pandas-stubs package accordingly.

Actual Behavior

The installation process installs the latest version of pandas-stubs, which forces an update of numpy and breaks the project's dependencies.

@kubotty kubotty added the bug mypy got something wrong label Sep 30, 2024
@JelleZijlstra
Copy link
Member

I don't think we have the machinery to figure out the necessary version of pandas. I'd recommend to use --install-types only as a one-time option to get the stub packages installed that you need. After that, check your type dependencies into version control just like you'd do for non-type dependencies.

@brianschubert
Copy link
Collaborator

To add to what Jelle said, if you need to maintain some pinned dependencies, it’s safest to only install new dependencies using an installation method that knows about that. For example, by adding new dependencies to your project’s pyproject.toml file and re-resolving/installing your project as you usually do.

If for whatever reason that doesn’t work for your use case, here’s some notes for how to make mypy --install-types resolve stub versions that are compatible with your project’s dependencies:

  • pip (for the most part) only considers the distributions given on the command line when resolving what versions of what distributions to install. If you ask pip to install pandas-stubs with no other information specified (which is what mypy is doing internally when you pass --install-types), it will pick the latest version of pandas-stubs and will install/upgrade other distributions as needed.

  • If you want to constrain what distributions pip will install, you need to give it more information. That can be done by specifying a requirements or constraints file, which can be done from outside mypy by defining the environment variables PIP_REQUIREMENT/PIP_CONSTRAINT.

  • As a workable solution using the above, you can try these steps:

    1. export your project’s pinned dependencies to a file in requirements.txt format
    2. Define the environment variable PIP_CONSTRAINT to point to that file
    3. Run mypy --install-types.

    That will allow pip to resolve a version of pandas-stubs that compatible with your project’s pinned numpy version, and will prevent it from upgrading/installing any other distributions that are incompatible with your project’s dependencies, the same as if you had added pandas-stubs to your project’s dependencies to begin with.

@hauntsaninja
Copy link
Collaborator

Yeah, I think this is unfortunately out of scope. Packaging is complicated and mypy shouldn't ever have gotten involved. Another fun note is that --install-types will sort of double the runtime of mypy, which is another reason not to use it.

@hauntsaninja hauntsaninja closed this as not planned Won't fix, can't repro, duplicate, stale Sep 30, 2024
@kubotty
Copy link
Author

kubotty commented Oct 1, 2024

Thank you for your response. I will add the specified version stubs to my pyproject.toml. I also understand the need to be cautious when using install-types.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug mypy got something wrong
Projects
None yet
Development

No branches or pull requests

4 participants