`pipenv check` - Ruamel vendoring isn't working

[yeisonvargasf]

Issue description

2022.11.23 included the new Safety version, which requires ruamel, the wheel for this pipenv version is missing the ruamel package therefore, pipenv check is failing with the following error:

Checking PEP 508 requirements...
Passed!
Checking installed packages for vulnerabilities...
Traceback (most recent call last):
  File "/Users/foo-user/.pyenv/versions/3.11.0/bin/pipenv", line 8, in <module>
    sys.exit(cli())
             ^^^^^
  File "/Users/foo-user/.pyenv/versions/3.11.0/lib/python3.11/site-packages/pipenv/vendor/click/core.py", line 1128, in __call__
    return self.main(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/foo-user/.pyenv/versions/3.11.0/lib/python3.11/site-packages/pipenv/cli/options.py", line 57, in main
    return super().main(*args, **kwargs, windows_expand_args=False)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/foo-user/.pyenv/versions/3.11.0/lib/python3.11/site-packages/pipenv/vendor/click/core.py", line 1053, in main
    rv = self.invoke(ctx)
         ^^^^^^^^^^^^^^^^
  File "/Users/foo-user/.pyenv/versions/3.11.0/lib/python3.11/site-packages/pipenv/vendor/click/core.py", line 1659, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/foo-user/.pyenv/versions/3.11.0/lib/python3.11/site-packages/pipenv/vendor/click/core.py", line 1395, in invoke
    return ctx.invoke(self.callback, **ctx.params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/foo-user/.pyenv/versions/3.11.0/lib/python3.11/site-packages/pipenv/vendor/click/core.py", line 754, in invoke
    return __callback(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/foo-user/.pyenv/versions/3.11.0/lib/python3.11/site-packages/pipenv/vendor/click/decorators.py", line 84, in new_func
    return ctx.invoke(f, obj, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/foo-user/.pyenv/versions/3.11.0/lib/python3.11/site-packages/pipenv/vendor/click/core.py", line 754, in invoke
    return __callback(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/foo-user/.pyenv/versions/3.11.0/lib/python3.11/site-packages/pipenv/cli/command.py", line 510, in check
    do_check(
  File "/Users/foo-user/.pyenv/versions/3.11.0/lib/python3.11/site-packages/pipenv/core.py", line 2985, in do_check
    from pipenv.patched.safety.cli import cli
  File "/Users/foo-user/.pyenv/versions/3.11.0/lib/python3.11/site-packages/pipenv/patched/safety/cli.py", line 12, in <module>
    from pipenv.patched.safety import safety
  File "/Users/foo-user/.pyenv/versions/3.11.0/lib/python3.11/site-packages/pipenv/patched/safety/safety.py", line 21, in <module>
    from .util import RequirementFile, read_requirements, Package, build_telemetry_data, sync_safety_context, SafetyContext, \
  File "/Users/foo-user/.pyenv/versions/3.11.0/lib/python3.11/site-packages/pipenv/patched/safety/util.py", line 17, in <module>
    from pipenv.vendor.ruamel.yaml import YAML
ModuleNotFoundError: No module named 'pipenv.vendor.ruamel'

Inspecting the wheel, we can see there isn't the ruamel package.

Expected result

pipenv check works as expected.

[matteius]

It is checked in though, I wonder if this is another case where the wheel built is excluding these files for some reason.

[matteius]

I merged a small change to setup.py to main that worked locally to include ruamel in the wheel. Feel free to try it out @yeisonvargasf -- I plan to do a follow-up release 2022.11.24 either later tonight or tomorrow.

[yeisonvargasf]

It is checked in though, I wonder if this is another case where the wheel built is excluding these files for some reason.

@matteius I reviewed this more in detail. I think the issue is that the ruamel directory under /pipenv/vendor/ruamel/ doesn't have a __init__.py, so in the build wheel process,ruamel is ignored.

I merged a small change to setup.py to main that worked locally to include ruamel in the wheel.

This change also worked for me.

I think adding a __init__.py makes more sense.

[matteius]

@yeisonvargasf adding an init file would get blown away during revendoring unless it was created via the patch, so I like this way for now.

[yeisonvargasf]

I got it, it makes sense, and I agree. Thank you!

[matteius]

Ah bummer, the published wheel is still missing ruamel.

[matteius]

It is weird -- when I build with the same command that the github action locally uses, it does put ruamel in wheel file, but github actions does not.

[matteius]

Ok so will need a more permanent solution to keeping the init.py in place in the pipenv/vendor/rumael directory -- I'll keep this ticket open for that task. However I just verified that new release 2022.11.25 does in fact include the required ruamel.