VS Code Support for pytest-mypy

[eleanorjboyd]

Hello!
I am the maintainer of testing for vscode-python and have been investigating: microsoft/vscode-python#23933. I have some questions in regards to how I can support pytest-mypy. This discussion is similar to the one I started on the pytest-ruff repo here: businho/pytest-ruff#32.

I explained a bit in the attached issue but what I think the best fix is would be including an extra "mypy" test node nested under each file node on the test explorer. (one side question I have is if there should be two nodes? The mypyFileItem and the mypyStatusItem?) We run tests based on their test-ids, so this would mean the mypy test would have a test-id that would be passed. Also it would fix the issue where no tests show up, this is just a side effect of the mypy node impacting the discovery process. Looking for a bit of advice as there is a piece of implementation.

What happens is while building the test tree to then display is I cycle through all the session.items see code. I am not seeing the mypy node- is this added after or should I expect it during run? I see if I run python -m pytest --mypy --collect-only from the terminal I do get 3 items:

<Dir 23933-pytest-mypy>
  <MypyFile test_ran.py>
    <MypyFileItem mypy>
    <MypyStatusItem mypy-status>
  <Module test_ran.py>
    <Function test_mypy>

so during collect it does exist but im not sure how I can see / access it from my plugin.

I appreciate any advice on how to support this. I have a few more questions but wanted to get the ball rolling with this initial one. Thanks

[dmtucker]

Hey there 👋 Thanks for reaching out! Sorry for the delay...

one side question I have is if there should be two nodes? The mypyFileItem and the mypyStatusItem?

Yes, there are 2 because mypy will check a module's imports...
For example, say I have 2 modules:

$ tree .
.
├── bad.py
└── good.py

0 directories, 2 files

# good.py
import bad

# bad.py
foo: int = "not an int"

If I run pytest --mypy good.py, mypy output will look like this:

$ mypy good.py 
bad.py:1: error: Incompatible types in assignment (expression has type "str", variable has type "int")  [assignment]
Found 1 error in 1 file (checked 1 source file)

The MypyFileitem for good.py doesn't see any lines that start with good.py:, so it passes.
In other words, the only test item passes (because bad.py was not collected),
but we don't want it to look like mypy succeeded (because it didn't).
So, we add an additional check (MypyStatusItem) that fails if the mypy exit status is not 0.

What happens is while building the test tree to then display is I cycle through all the session.items see code. I am not seeing the mypy node

MypyItems do not have a callspec attr:

(Pdb) pp items
[<MypyStatusItem mypy1.15.0-status>, <MypyFileItem mypy1.15.0>]
(Pdb) pp any(hasattr(item, "callspec") for item in items)
False

So, they are being excluded from the block you linked:
https://github.com/eleanorjboyd/vscode-python/blob/f84a48581f538a96842e03342ebf8b0dcfd51947/python_files/vscode_pytest/__init__.py#L505

Is there are reason for that exclusion?

I appreciate any advice on how to support this.

For what it's worth, this plugin is modeled after this example in the Pytest docs:
https://docs.pytest.org/en/stable/example/nonpython.html