Building wheel for hnswlib (pyproject.toml) did not run successfully - installed Microsoft C++ Build Tools not identified

[MilindGaharwar]

For Python 3.10.11 in VSCode in Windows 11, Microsoft Visual C++ 14.0 or greater is installed - Full Visual Studio is installed with C++ components

pip install hnswlib

--gives the following error--

Collecting hnswlib>=0.7 (from chromadb)
Using cached hnswlib-0.7.0.tar.gz (33 kB)
Installing build dependencies: started
Installing build dependencies: finished with status 'done'
Getting requirements to build wheel: started
Getting requirements to build wheel: finished with status 'done'
Preparing metadata (pyproject.toml): started
Preparing metadata (pyproject.toml): finished with status 'done'
Collecting clickhouse-connect>=0.5.7 (from chromadb)
Using cached clickhouse_connect-0.6.6-cp310-cp310-win_amd64.whl (227 kB)
Requirement already satisfied: sentence-transformers>=2.2.2 in c:\users\1\appdata\local\packages\pythonsoftwarefoundation.python.3.10_qbz5n2kfra8p0\localcache\local-packages\python310\site-packages (from chromadb) (2.2.2)
Collecting duckdb>=0.7.1 (from chromadb)
Using cached duckdb-0.8.1-cp310-cp310-win_amd64.whl (9.8 MB)
Collecting fastapi>=0.85.1 (from chromadb)
Using cached fastapi-0.100.0-py3-none-any.whl (65 kB)
Collecting uvicorn[standard]>=0.18.3 (from chromadb)
Using cached uvicorn-0.22.0-py3-none-any.whl (58 kB)
Requirement already satisfied: numpy>=1.21.6 in c:\users\1\appdata\local\packages\pythonsoftwarefoundation.python.3.10_qbz5n2kfra8p0\localcache\local-packages\python310\site-packages (from chromadb) (1.25.1)
Requirement already satisfied: posthog>=2.4.0 in c:\users\1\appdata\local\packages\pythonsoftwarefoundation.python.3.10_qbz5n2kfra8p0\localcache\local-packages\python310\site-packages (from chromadb) (3.0.1)
Requirement already satisfied: typing-extensions>=4.5.0 in c:\users\1\appdata\local\packages\pythonsoftwarefoundation.python.3.10_qbz5n2kfra8p0\localcache\local-packages\python310\site-packages (from chromadb) (4.7.1)
Requirement already satisfied: certifi in c:\users\1\appdata\local\packages\pythonsoftwarefoundation.python.3.10_qbz5n2kfra8p0\localcache\local-packages\python310\site-packages (from clickhouse-connect>=0.5.7->chromadb) (2023.5.7)
...
Building wheel for hnswlib (pyproject.toml): started
Building wheel for hnswlib (pyproject.toml): finished with status 'error'
Failed to build hnswlib
Note: you may need to restart the kernel to use updated packages.
Output is truncated. View as a scrollable element or open in a text editor. Adjust cell output settings...
error: subprocess-exited-with-error

× Building wheel for hnswlib (pyproject.toml) did not run successfully.
│ exit code: 1
╰─> [5 lines of output]
running bdist_wheel
running build
running build_ext
building 'hnswlib' extension
error: Microsoft Visual C++ 14.0 or greater is required. Get it with "Microsoft C++ Build Tools": https://visualstudio.microsoft.com/visual-cpp-build-tools/
[end of output]

note: This error originates from a subprocess, and is likely not a problem with pip.
ERROR: Failed building wheel for hnswlib
ERROR: Could not build wheels for hnswlib, which is required to install pyproject.toml-based projects

[yurymalkov]

Some output from chatgpt on the problem:

1. Verify your Microsoft Visual C++ Build Tools installation: You can do this by running Visual Studio Installer, selecting 'Modify' on your installed Visual Studio, and checking if 'Desktop development with C++' workload is installed.

2. Set the path for Visual Studio: Sometimes, the error can be due to the fact that Visual Studio is not in the system PATH. You can add it by searching for 'Environment Variables' in your system settings, selecting 'Edit the system environment variables', then clicking on 'Environment Variables' and adding the Visual Studio path to the 'Path' variable under 'System Variables'. The default path for Visual Studio is usually C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC<version>\bin\Hostx64\x64.

3. Try installing from a different command prompt: Sometimes, the problem might be due to the command prompt you're using. If you're using a regular command prompt, try using the Developer Command Prompt for VS or vice versa.

4. Upgrade pip, setuptools, and wheel: Sometimes, outdated versions of these tools can cause issues. Upgrade them using the following commands: pip install --upgrade pip setuptools wheel

5. Try installing the package with Conda: If you're using Anaconda or Miniconda, try installing the package using Conda. This might bypass the issue with Visual Studio: conda install -c conda-forge hnswlib
   Remember to restart your system or command prompt after making these changes to make sure they take effect. If the problem persists, please provide the full error message for more specific help.

I would start with bullet no. 3 - try to use Developer Command Prompt from the Windows start menu

[dyashuni]

Also you can try to collect more build logs by running
pip install -v hnswlib

[fiery-prometheus]

Hmm, I have the same error, building on windows, tried installing vcredist, didn't fix it either, then tried installing build tools from visual studio related to the 14.36 vcredist, and didn't fix it either. I don't know if being on windows and having installed vcredist from scoop, and then from visual studio, might create some issues, but even if it did, I thought at least one of the paths to the installations should work 🤔

 "C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.36.32532\bin\HostX86\x64\cl.exe" /c /nologo /O2 /W3 /GL /DNDEBUG /MD -IC:\Users\kenza\AppData\Local\Temp\pip-build-env-8yq0ovv5\overlay\Lib\site-packages\pybind11\include -IC:\Users\kenza\AppData\Local\Temp\pip-build-env-8yq0ovv5\overlay\Lib\site-packages\numpy\core\include -I./hnswlib/ -IC:\Users\kenza\scoop\apps\miniconda3\current\envs\lm-plugin\include -IC:\Users\kenza\scoop\apps\miniconda3\current\envs\lm-plugin\Include "-IC:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.36.32532\include" "-IC:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.36.32532\ATLMFC\include" "-IC:\Program Files\Microsoft Visual Studio\2022\Community\VC\Auxiliary\VS\include" "-IC:\Program Files (x86)\Windows Kits\NETFXSDK\4.8\include\um" /EHsc /Tp./python_bindings/bindings.cpp /Fobuild\temp.win-amd64-cpython-311\Release\./python_bindings/bindings.obj /EHsc /openmp /O2 /DVERSION_INFO=\\\"0.7.0\\\"
  bindings.cpp
  C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.36.32532\include\yvals.h(17): fatal error C1083: Cannot open include file: 'crtdbg.h': No such file or directory
  error: command 'C:\\Program Files\\Microsoft Visual Studio\\2022\\Community\\VC\\Tools\\MSVC\\14.36.32532\\bin\\HostX86\\x64\\cl.exe' failed with exit code 2
  error: subprocess-exited-with-error

  × Building wheel for hnswlib (pyproject.toml) did not run successfully.
  │ exit code: 1
  ╰─> See above for output.

  note: This error originates from a subprocess, and is likely not a problem with pip.
  full command: 'C:\Users\kenza\scoop\apps\miniconda3\current\envs\lm-plugin\python.exe' 'C:\Users\kenza\scoop\apps\miniconda3\current\envs\lm-plugin\Lib\site-packages\pip\_vendor\pyproject_hooks\_in_process\_in_process.py' build_wheel 'C:\Users\kenza\AppData\Local\Temp\tmp3g24rrq6'
  cwd: C:\Users\kenza\AppData\Local\Temp\pip-install-mhvxccup\hnswlib_ab01c048aecc4df6972cb08e9d80a0d5
  Building wheel for hnswlib (pyproject.toml) ... error
  ERROR: Failed building wheel for hnswlib
Failed to build hnswlib
ERROR: Could not build wheels for hnswlib, which is required to install pyproject.toml-based projects

[jklein33]

This worked for me: https://stackoverflow.com/questions/73969269/error-could-not-build-wheels-for-hnswlib-which-is-required-to-install-pyprojec