pip install fail because of missing musl-clang

[mcsba]

I could not find musl-clang package for linux. Do you know how can I get this package?

Python 3.7.7 musl version
Error message:
unable to execute 'musl-clang': No such file or directory
error: command 'musl-clang' failed with exit status 1

[indygreg]

What wanted to execute musl-clang?

If you compile musl libc with Clang, it will create a musl-clang script. I'm unsure why the Debian musl packages don't ship this wrapper. Maybe it is a side-effect of them building musl with GCC and not Clang?

[mcsba]

Please find below the output of the command
user@desktop#./pip install paramiko
Collecting paramiko
Using cached paramiko-2.7.1-py2.py3-none-any.whl (206 kB)
Collecting pynacl>=1.0.1
Using cached PyNaCl-1.3.0.tar.gz (3.4 MB)
Installing build dependencies ... error
ERROR: Command errored out with exit status 1:
command: /portable/python/install/bin/python3 /portable/python/install/lib/python3.7/site-packages/pip-20.0.2-py3.7.egg/pip install --ignore-installed --no-user --prefix /tmp/pip-build-env-wo50jm6q/overlay --no-warn-script-location --no-binary :none: --only-binary :none: -i https://pypi.org/simple -- setuptools wheel 'cffi>=1.4.1; python_implementation != '"'"'PyPy'"'"''
cwd: None
Complete output (65 lines):
Collecting setuptools
Using cached setuptools-46.1.3-py3-none-any.whl (582 kB)
Collecting wheel
Using cached wheel-0.34.2-py2.py3-none-any.whl (26 kB)
Collecting cffi>=1.4.1
Using cached cffi-1.14.0.tar.gz (463 kB)
Collecting pycparser
Using cached pycparser-2.20-py2.py3-none-any.whl (112 kB)
Installing collected packages: setuptools, wheel, pycparser, cffi
Running setup.py install for cffi: started
Running setup.py install for cffi: finished with status 'error'
ERROR: Command errored out with exit status 1:
command: /portable/python/install/bin/python3 -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-vp8q6239/cffi/setup.py'"'"'; file='"'"'/tmp/pip-install-vp8q6239/cffi/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(file);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, file, '"'"'exec'"'"'))' install --record /tmp/pip-record-u7j7i4o0/install-record.txt --single-version-externally-managed --prefix /tmp/pip-build-env-wo50jm6q/overlay --compile --install-headers /tmp/pip-build-env-wo50jm6q/overlay/include/python3.7m/cffi
cwd: /tmp/pip-install-vp8q6239/cffi/
Complete output (48 lines):
unable to execute 'musl-clang': No such file or directory
unable to execute 'musl-clang': No such file or directory

      No working compiler found, or bogus compiler options passed to
      the compiler from Python's standard "distutils" module.  See
      the error messages above.  Likely, the problem is not related
      to CFFI but generic to the setup.py of any Python package that
      tries to compile C code.  (Hints: on OS/X 10.8, for errors about
      -mno-fused-madd see http://stackoverflow.com/questions/22313407/
      Otherwise, see https://wiki.python.org/moin/CompLangPython or
      the IRC channel #python on irc.freenode.net.)

      Trying to continue anyway.  If you are trying to install CFFI from
      a build done in a different context, you can ignore this warning.

  running install
  running build
  running build_py
  creating build
  creating build/lib.linux-x86_64-3.7
  creating build/lib.linux-x86_64-3.7/cffi
  copying cffi/recompiler.py -> build/lib.linux-x86_64-3.7/cffi
  copying cffi/verifier.py -> build/lib.linux-x86_64-3.7/cffi
  copying cffi/cffi_opcode.py -> build/lib.linux-x86_64-3.7/cffi
  copying cffi/setuptools_ext.py -> build/lib.linux-x86_64-3.7/cffi
  copying cffi/__init__.py -> build/lib.linux-x86_64-3.7/cffi
  copying cffi/lock.py -> build/lib.linux-x86_64-3.7/cffi
  copying cffi/api.py -> build/lib.linux-x86_64-3.7/cffi
  copying cffi/vengine_gen.py -> build/lib.linux-x86_64-3.7/cffi
  copying cffi/pkgconfig.py -> build/lib.linux-x86_64-3.7/cffi
  copying cffi/ffiplatform.py -> build/lib.linux-x86_64-3.7/cffi
  copying cffi/cparser.py -> build/lib.linux-x86_64-3.7/cffi
  copying cffi/backend_ctypes.py -> build/lib.linux-x86_64-3.7/cffi
  copying cffi/error.py -> build/lib.linux-x86_64-3.7/cffi
  copying cffi/commontypes.py -> build/lib.linux-x86_64-3.7/cffi
  copying cffi/model.py -> build/lib.linux-x86_64-3.7/cffi
  copying cffi/vengine_cpy.py -> build/lib.linux-x86_64-3.7/cffi
  copying cffi/_cffi_include.h -> build/lib.linux-x86_64-3.7/cffi
  copying cffi/parse_c_type.h -> build/lib.linux-x86_64-3.7/cffi
  copying cffi/_embedding.h -> build/lib.linux-x86_64-3.7/cffi
  copying cffi/_cffi_errors.h -> build/lib.linux-x86_64-3.7/cffi
  running build_ext
  building '_cffi_backend' extension
  creating build/temp.linux-x86_64-3.7
  creating build/temp.linux-x86_64-3.7/c
  musl-clang -fno-strict-aliasing -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -fPIC -I/tools/deps/include -I/tools/deps/include/ncurses -static -fPIC -I/portable/python/install/include/python3.7m -c c/_cffi_backend.c -o build/temp.linux-x86_64-3.7/c/_cffi_backend.o
  unable to execute 'musl-clang': No such file or directory
  error: command 'musl-clang' failed with exit status 1
  ----------------------------------------

ERROR: Command errored out with exit status 1: /portable/python/install/bin/python3 -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-vp8q6239/cffi/setup.py'"'"'; file='"'"'/tmp/pip-install-vp8q6239/cffi/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(file);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, file, '"'"'exec'"'"'))' install --record /tmp/pip-record-u7j7i4o0/install-record.txt --single-version-externally-managed --prefix /tmp/pip-build-env-wo50jm6q/overlay --compile --install-headers /tmp/pip-build-env-wo50jm6q/overlay/include/python3.7m/cffi Check the logs for full command output.

ERROR: Command errored out with exit status 1: /portable/python/install/bin/python3 /portable/python/install/lib/python3.7/site-packages/pip-20.0.2-py3.7.egg/pip install --ignore-installed --no-user --prefix /tmp/pip-build-env-wo50jm6q/overlay --no-warn-script-location --no-binary :none: --only-binary :none: -i https://pypi.org/simple -- setuptools wheel 'cffi>=1.4.1; python_implementation != '"'"'PyPy'"'"'' Check the logs for full command output.

[gi0baro]

@indygreg I'm also facing issues with the musl distribution. Here is what I found so far.

Trying to build a Python project with cffi in 3rd part dependencies with pyoxidizer and the musl target:

/wrk # pyoxidizer build --target-triple x86_64-unknown-linux-musl
resolving 1 targets
resolving target install
resolving target exe
resolving target dist
resolving Python distribution Url { url: "https://github.com/indygreg/python-build-standalone/releases/download/20201020/cpython-3.8.6-x86_64-unknown-linux-musl-noopt-20201020T0627.tar.zst", sha256: "1dec303ad821b4e54d6562f7d5f85fedf9ac6e7519be60130cdb135c80bd42a5" }
Python distribution available at /wrk/./build/python_distributions/cpython-3.8.6-x86_64-unknown-linux-musl-noopt-20201020T0627.tar.zst
reading data from Python distribution...
installing modified distutils to /tmp/pyoxidizer-pip-installBOq266/packages
modifying distutils/_msvccompiler.py for oxidation
modifying distutils/command/build_ext.py for oxidation
modifying distutils/unixccompiler.py for oxidation
pip installing to /tmp/pyoxidizer-pip-installBOq266/install
Processing /wrk
  Installing build dependencies: started
  Installing build dependencies: finished with status 'error'
  ERROR: Command errored out with exit status 1:
   command: /wrk/./build/python_distributions/python.1dec303ad821/python/install/bin/python3.8 /wrk/build/python_distributions/python.1dec303ad821/python/install/lib/python3.8/site-packages/pip-20.2.4-py3.8.egg/pip install --ignore-installed --no-user --prefix /tmp/pip-build-env-13x9y7ka/overlay --no-warn-script-location --no-binary :none: --only-binary :none: -i https://pypi.org/simple -- 'poetry>=1.0.0'
       cwd: None
  Complete output (144 lines):
  Collecting poetry>=1.0.0
    Downloading poetry-1.1.4-py2.py3-none-any.whl (171 kB)
[...]
Collecting cryptography>=2.0
    Downloading cryptography-3.3.1.tar.gz (539 kB)
    Installing build dependencies: started
    Installing build dependencies: finished with status 'error'
    ERROR: Command errored out with exit status 1:
     command: /wrk/./build/python_distributions/python.1dec303ad821/python/install/bin/python3.8 /wrk/build/python_distributions/python.1dec303ad821/python/install/lib/python3.8/site-packages/pip-20.2.4-py3.8.egg/pip install --ignore-installed --no-user --prefix /tmp/pip-build-env-2wkn1jtk/overlay --no-warn-script-location --no-binary :none: --only-binary :none: -i https://pypi.org/simple -- 'setuptools>=40.6.0' wheel 'cffi>=1.12; platform_python_implementation != '"'"'PyPy'"'"''
         cwd: None
    Complete output (66 lines):
    Collecting setuptools>=40.6.0
      Downloading setuptools-51.1.1-py3-none-any.whl (2.0 MB)
    Collecting wheel
      Downloading wheel-0.36.2-py2.py3-none-any.whl (35 kB)
    Collecting cffi>=1.12
      Downloading cffi-1.14.4.tar.gz (471 kB)
    Collecting pycparser
      Downloading pycparser-2.20-py2.py3-none-any.whl (112 kB)
    Using legacy 'setup.py install' for cffi, since package 'wheel' is not installed.
    Installing collected packages: setuptools, wheel, pycparser, cffi
        Running setup.py install for cffi: started
        Running setup.py install for cffi: finished with status 'error'
        ERROR: Command errored out with exit status 1:
         command: /wrk/./build/python_distributions/python.1dec303ad821/python/install/bin/python3.8 -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-w7qs2byq/cffi/setup.py'"'"'; __file__='"'"'/tmp/pip-install-w7qs2byq/cffi/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /tmp/pip-record-gv3ah7wf/install-record.txt --single-version-externally-managed --prefix /tmp/pip-build-env-2wkn1jtk/overlay --compile --install-headers /tmp/pip-build-env-2wkn1jtk/overlay/include/python3.8/cffi
             cwd: /tmp/pip-install-w7qs2byq/cffi/
        Complete output (48 lines):
        unable to execute 'musl-clang': No such file or directory
        unable to execute 'musl-clang': No such file or directory

            No working compiler found, or bogus compiler options passed to
            the compiler from Python's standard "distutils" module.  See
            the error messages above.  Likely, the problem is not related
            to CFFI but generic to the setup.py of any Python package that
            tries to compile C code.  (Hints: on OS/X 10.8, for errors about
            -mno-fused-madd see http://stackoverflow.com/questions/22313407/
            Otherwise, see https://wiki.python.org/moin/CompLangPython or
            the IRC channel #python on irc.freenode.net.)

            Trying to continue anyway.  If you are trying to install CFFI from
            a build done in a different context, you can ignore this warning.

        running install
        running build
        running build_py
        creating build
        creating build/lib.linux-x86_64-3.8
        creating build/lib.linux-x86_64-3.8/cffi
        copying cffi/lock.py -> build/lib.linux-x86_64-3.8/cffi
        copying cffi/error.py -> build/lib.linux-x86_64-3.8/cffi
        copying cffi/cparser.py -> build/lib.linux-x86_64-3.8/cffi
        copying cffi/recompiler.py -> build/lib.linux-x86_64-3.8/cffi
        copying cffi/ffiplatform.py -> build/lib.linux-x86_64-3.8/cffi
        copying cffi/pkgconfig.py -> build/lib.linux-x86_64-3.8/cffi
        copying cffi/backend_ctypes.py -> build/lib.linux-x86_64-3.8/cffi
        copying cffi/__init__.py -> build/lib.linux-x86_64-3.8/cffi
        copying cffi/vengine_cpy.py -> build/lib.linux-x86_64-3.8/cffi
        copying cffi/setuptools_ext.py -> build/lib.linux-x86_64-3.8/cffi
        copying cffi/commontypes.py -> build/lib.linux-x86_64-3.8/cffi
        copying cffi/cffi_opcode.py -> build/lib.linux-x86_64-3.8/cffi
        copying cffi/model.py -> build/lib.linux-x86_64-3.8/cffi
        copying cffi/verifier.py -> build/lib.linux-x86_64-3.8/cffi
        copying cffi/vengine_gen.py -> build/lib.linux-x86_64-3.8/cffi
        copying cffi/api.py -> build/lib.linux-x86_64-3.8/cffi
        copying cffi/_cffi_include.h -> build/lib.linux-x86_64-3.8/cffi
        copying cffi/parse_c_type.h -> build/lib.linux-x86_64-3.8/cffi
        copying cffi/_embedding.h -> build/lib.linux-x86_64-3.8/cffi
        copying cffi/_cffi_errors.h -> build/lib.linux-x86_64-3.8/cffi
        running build_ext
        building '_cffi_backend' extension
        creating build/temp.linux-x86_64-3.8
        creating build/temp.linux-x86_64-3.8/c
        musl-clang -fno-strict-aliasing -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -fPIC -I/tools/deps/include -I/tools/deps/include/ncursesw -static -fPIC -DPy_BUILD_CORE_BUILTIN=1 -I/usr/include/ffi -I/usr/include/libffi -I/wrk/build/python_distributions/python.1dec303ad821/python/install/include/python3.8 -c c/_cffi_backend.c -o build/temp.linux-x86_64-3.8/c/_cffi_backend.o
        unable to execute 'musl-clang': No such file or directory
        error: command 'musl-clang' failed with exit status 1

At this point is not clear to me why the x86_64-unknown-linux-musl requires musl-clang, rather than selecting the proper CC system compiler. Anyway, I built musl and consequentially got the musl-clang and ld.musl-clang wrappers. But it still fails:

/wrk # pyoxidizer build --target-triple x86_64-unknown-linux-musl
resolving 1 targets
resolving target install
resolving target exe
resolving target dist
resolving Python distribution Url { url: "https://github.com/indygreg/python-build-standalone/releases/download/20201020/cpython-3.8.6-x86_64-unknown-linux-musl-noopt-20201020T0627.tar.zst", sha256: "1dec303ad821b4e54d6562f7d5f85fedf9ac6e7519be60130cdb135c80bd42a5" }
Python distribution available at /wrk/./build/python_distributions/cpython-3.8.6-x86_64-unknown-linux-musl-noopt-20201020T0627.tar.zst
reading data from Python distribution...
installing modified distutils to /tmp/pyoxidizer-pip-installMYZgiV/packages
modifying distutils/_msvccompiler.py for oxidation
modifying distutils/command/build_ext.py for oxidation
modifying distutils/unixccompiler.py for oxidation
pip installing to /tmp/pyoxidizer-pip-installMYZgiV/install
Processing /wrk
  Installing build dependencies: started
  Installing build dependencies: finished with status 'error'
  ERROR: Command errored out with exit status 1:
   command: /wrk/./build/python_distributions/python.1dec303ad821/python/install/bin/python3.8 /wrk/build/python_distributions/python.1dec303ad821/python/install/lib/python3.8/site-packages/pip-20.2.4-py3.8.egg/pip install --ignore-installed --no-user --prefix /tmp/pip-build-env-7wskld1b/overlay --no-warn-script-location --no-binary :none: --only-binary :none: -i https://pypi.org/simple -- 'poetry>=1.0.0'
       cwd: None
  Complete output (2407 lines):
  Collecting poetry>=1.0.0
    Using cached poetry-1.1.4-py2.py3-none-any.whl (171 kB)
[...]
Collecting cryptography>=2.0
    Using cached cryptography-3.3.1.tar.gz (539 kB)
    Installing build dependencies: started
    Installing build dependencies: finished with status 'error'
    ERROR: Command errored out with exit status 1:
     command: /wrk/./build/python_distributions/python.1dec303ad821/python/install/bin/python3.8 /wrk/build/python_distributions/python.1dec303ad821/python/install/lib/python3.8/site-packages/pip-20.2.4-py3.8.egg/pip install --ignore-installed --no-user --prefix /tmp/pip-build-env-_o85jpmh/overlay --no-warn-script-location --no-binary :none: --only-binary :none: -i https://pypi.org/simple -- 'setuptools>=40.6.0' wheel 'cffi>=1.12; platform_python_implementation != '"'"'PyPy'"'"''
         cwd: None
    Complete output (2329 lines):
    Collecting setuptools>=40.6.0
      Using cached setuptools-51.1.1-py3-none-any.whl (2.0 MB)
    Collecting wheel
      Using cached wheel-0.36.2-py2.py3-none-any.whl (35 kB)
    Collecting cffi>=1.12
      Using cached cffi-1.14.4.tar.gz (471 kB)
    Collecting pycparser
      Using cached pycparser-2.20-py2.py3-none-any.whl (112 kB)
    Using legacy 'setup.py install' for cffi, since package 'wheel' is not installed.
    Installing collected packages: setuptools, wheel, pycparser, cffi
        Running setup.py install for cffi: started
        Running setup.py install for cffi: finished with status 'error'
        ERROR: Command errored out with exit status 1:
         command: /wrk/./build/python_distributions/python.1dec303ad821/python/install/bin/python3.8 -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-gfopgj1a/cffi/setup.py'"'"'; __file__='"'"'/tmp/pip-install-gfopgj1a/cffi/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /tmp/pip-record-5zebgud7/install-record.txt --single-version-externally-managed --prefix /tmp/pip-build-env-_o85jpmh/overlay --compile --install-headers /tmp/pip-build-env-_o85jpmh/overlay/include/python3.8/cffi
             cwd: /tmp/pip-install-gfopgj1a/cffi/
        Complete output (2311 lines):
        clang-10: warning: argument unused during compilation: '-fuse-ld=musl-clang' [-Wunused-command-line-argument]
        clang-10: warning: argument unused during compilation: '-static-libgcc' [-Wunused-command-line-argument]
        clang-10: warning: argument unused during compilation: '-L-user-start' [-Wunused-command-line-argument]
        clang-10: warning: argument unused during compilation: '-L/usr/lib' [-Wunused-command-line-argument]
        clang-10: warning: argument unused during compilation: '-L-user-end' [-Wunused-command-line-argument]
        clang-10: warning: argument unused during compilation: '-fuse-ld=musl-clang' [-Wunused-command-line-argument]
        clang-10: warning: argument unused during compilation: '-static-libgcc' [-Wunused-command-line-argument]
        clang-10: warning: argument unused during compilation: '-L-user-start' [-Wunused-command-line-argument]
        clang-10: warning: argument unused during compilation: '-L/usr/lib' [-Wunused-command-line-argument]
        clang-10: warning: argument unused during compilation: '-L-user-end' [-Wunused-command-line-argument]
        clang-10: warning: argument unused during compilation: '-nostdinc' [-Wunused-command-line-argument]
        running install
        running build
        running build_py
        creating build
        creating build/lib.linux-x86_64-3.8
        creating build/lib.linux-x86_64-3.8/cffi
        copying cffi/lock.py -> build/lib.linux-x86_64-3.8/cffi
        copying cffi/error.py -> build/lib.linux-x86_64-3.8/cffi
        copying cffi/cparser.py -> build/lib.linux-x86_64-3.8/cffi
        copying cffi/recompiler.py -> build/lib.linux-x86_64-3.8/cffi
        copying cffi/ffiplatform.py -> build/lib.linux-x86_64-3.8/cffi
        copying cffi/pkgconfig.py -> build/lib.linux-x86_64-3.8/cffi
        copying cffi/backend_ctypes.py -> build/lib.linux-x86_64-3.8/cffi
        copying cffi/__init__.py -> build/lib.linux-x86_64-3.8/cffi
        copying cffi/vengine_cpy.py -> build/lib.linux-x86_64-3.8/cffi
        copying cffi/setuptools_ext.py -> build/lib.linux-x86_64-3.8/cffi
        copying cffi/commontypes.py -> build/lib.linux-x86_64-3.8/cffi
        copying cffi/cffi_opcode.py -> build/lib.linux-x86_64-3.8/cffi
        copying cffi/model.py -> build/lib.linux-x86_64-3.8/cffi
        copying cffi/verifier.py -> build/lib.linux-x86_64-3.8/cffi
        copying cffi/vengine_gen.py -> build/lib.linux-x86_64-3.8/cffi
        copying cffi/api.py -> build/lib.linux-x86_64-3.8/cffi
        copying cffi/_cffi_include.h -> build/lib.linux-x86_64-3.8/cffi
        copying cffi/parse_c_type.h -> build/lib.linux-x86_64-3.8/cffi
        copying cffi/_embedding.h -> build/lib.linux-x86_64-3.8/cffi
        copying cffi/_cffi_errors.h -> build/lib.linux-x86_64-3.8/cffi
        running build_ext
        building '_cffi_backend' extension
        creating build/temp.linux-x86_64-3.8
        creating build/temp.linux-x86_64-3.8/c
        musl-clang -fno-strict-aliasing -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -fPIC -I/tools/deps/include -I/tools/deps/include/ncursesw -static -fPIC -DUSE__THREAD -DHAVE_SYNC_SYNCHRONIZE -DPy_BUILD_CORE_BUILTIN=1 -I/wrk/build/python_distributions/python.1dec303ad821/python/install/include/python3.8 -c c/_cffi_backend.c -o build/temp.linux-x86_64-3.8/c/_cffi_backend.o
        clang-10: warning: argument unused during compilation: '-fuse-ld=musl-clang' [-Wunused-command-line-argument]
        clang-10: warning: argument unused during compilation: '-static-libgcc' [-Wunused-command-line-argument]
        clang-10: warning: argument unused during compilation: '-L-user-start' [-Wunused-command-line-argument]
        clang-10: warning: argument unused during compilation: '-L/usr/lib' [-Wunused-command-line-argument]
        clang-10: warning: argument unused during compilation: '-L-user-end' [-Wunused-command-line-argument]
        c/_cffi_backend.c:5887:16: warning: comparison of integers of different signs: 'ffi_status' and 'int' [-Wsign-compare]
            if (status == -1) {
                ~~~~~~ ^  ~~
        1 warning generated.
        musl-clang -shared -L/tools/deps/lib -static build/temp.linux-x86_64-3.8/c/_cffi_backend.o -L/usr/lib/../lib -lffi -o build/lib.linux-x86_64-3.8/_cffi_backend.cpython-38-x86_64-linux-gnu.so
        clang-10: warning: argument unused during compilation: '-nostdinc' [-Wunused-command-line-argument]
        /usr/bin/ld: warning: cannot find entry symbol _start; defaulting to 0000000000401010
        /usr/bin/ld: build/temp.linux-x86_64-3.8/c/_cffi_backend.o: in function `PyInit__cffi_backend':
        /tmp/pip-install-gfopgj1a/cffi/c/_cffi_backend.c:7922: undefined reference to `PySys_GetObject'
        /usr/bin/ld: /tmp/pip-install-gfopgj1a/cffi/c/_cffi_backend.c:7924: undefined reference to `PyUnicode_AsUTF8'
        /usr/bin/ld: /tmp/pip-install-gfopgj1a/cffi/c/_cffi_backend.c:7925: undefined reference to `PyExc_ImportError'
        /usr/bin/ld: /tmp/pip-install-gfopgj1a/cffi/c/_cffi_backend.c:7925: undefined reference to `PyErr_Format'
        /usr/bin/ld: /tmp/pip-install-gfopgj1a/cffi/c/_cffi_backend.c:7932: undefined reference to `PyModule_Create2'
        /usr/bin/ld: /tmp/pip-install-gfopgj1a/cffi/c/_cffi_backend.c:7941: undefined reference to `PyDict_New'
        /usr/bin/ld: /tmp/pip-install-gfopgj1a/cffi/c/_cffi_backend.c:7955: undefined reference to `PyType_Ready'
        /usr/bin/ld: /tmp/pip-install-gfopgj1a/cffi/c/_cffi_backend.c:7959: undefined reference to `PyModule_AddObject'
        /usr/bin/ld: /tmp/pip-install-gfopgj1a/cffi/c/_cffi_backend.c:7976: undefined reference to `PyCapsule_New'
        /usr/bin/ld: /tmp/pip-install-gfopgj1a/cffi/c/_cffi_backend.c:7977: undefined reference to `PyModule_AddObject'
        /usr/bin/ld: /tmp/pip-install-gfopgj1a/cffi/c/_cffi_backend.c:7980: undefined reference to `PyUnicode_FromString'
        /usr/bin/ld: /tmp/pip-install-gfopgj1a/cffi/c/_cffi_backend.c:7981: undefined reference to `PyModule_AddObject'
        /usr/bin/ld: /tmp/pip-install-gfopgj1a/cffi/c/_cffi_backend.c:7984: undefined reference to `PyModule_AddIntConstant'
        /usr/bin/ld: /tmp/pip-install-gfopgj1a/cffi/c/_cffi_backend.c:7988: undefined reference to `PyModule_AddIntConstant'
        /usr/bin/ld: /tmp/pip-install-gfopgj1a/cffi/c/_cffi_backend.c:8001: undefined reference to `PyModule_AddIntConstant'
        /usr/bin/ld: /tmp/pip-install-gfopgj1a/cffi/c/_cffi_backend.c:8001: undefined reference to `PyModule_AddIntConstant'
        /usr/bin/ld: /tmp/pip-install-gfopgj1a/cffi/c/_cffi_backend.c:8001: undefined reference to `PyModule_AddIntConstant'
[...]

In order to get more info, I also tried directly building cffi from source using the Python binary from the cpython-3.8.6-x86_64-unknown-linux-musl-noopt-20201020T0627.tar.zst distribution:

~/cffi-1.14.4 # ~/python/install/bin/python3 setup.py build
clang-10: warning: argument unused during compilation: '-fuse-ld=musl-clang' [-Wunused-command-line-argument]
clang-10: warning: argument unused during compilation: '-static-libgcc' [-Wunused-command-line-argument]
clang-10: warning: argument unused during compilation: '-L-user-start' [-Wunused-command-line-argument]
clang-10: warning: argument unused during compilation: '-L/usr/lib' [-Wunused-command-line-argument]
clang-10: warning: argument unused during compilation: '-L-user-end' [-Wunused-command-line-argument]
clang-10: warning: argument unused during compilation: '-fuse-ld=musl-clang' [-Wunused-command-line-argument]
clang-10: warning: argument unused during compilation: '-static-libgcc' [-Wunused-command-line-argument]
clang-10: warning: argument unused during compilation: '-L-user-start' [-Wunused-command-line-argument]
clang-10: warning: argument unused during compilation: '-L/usr/lib' [-Wunused-command-line-argument]
clang-10: warning: argument unused during compilation: '-L-user-end' [-Wunused-command-line-argument]
clang-10: warning: argument unused during compilation: '-nostdinc' [-Wunused-command-line-argument]
running build
running build_py
creating build
creating build/lib.linux-x86_64-3.8
creating build/lib.linux-x86_64-3.8/cffi
copying cffi/lock.py -> build/lib.linux-x86_64-3.8/cffi
copying cffi/error.py -> build/lib.linux-x86_64-3.8/cffi
copying cffi/cparser.py -> build/lib.linux-x86_64-3.8/cffi
copying cffi/recompiler.py -> build/lib.linux-x86_64-3.8/cffi
copying cffi/ffiplatform.py -> build/lib.linux-x86_64-3.8/cffi
copying cffi/pkgconfig.py -> build/lib.linux-x86_64-3.8/cffi
copying cffi/backend_ctypes.py -> build/lib.linux-x86_64-3.8/cffi
copying cffi/__init__.py -> build/lib.linux-x86_64-3.8/cffi
copying cffi/vengine_cpy.py -> build/lib.linux-x86_64-3.8/cffi
copying cffi/setuptools_ext.py -> build/lib.linux-x86_64-3.8/cffi
copying cffi/commontypes.py -> build/lib.linux-x86_64-3.8/cffi
copying cffi/cffi_opcode.py -> build/lib.linux-x86_64-3.8/cffi
copying cffi/model.py -> build/lib.linux-x86_64-3.8/cffi
copying cffi/verifier.py -> build/lib.linux-x86_64-3.8/cffi
copying cffi/vengine_gen.py -> build/lib.linux-x86_64-3.8/cffi
copying cffi/api.py -> build/lib.linux-x86_64-3.8/cffi
copying cffi/_cffi_include.h -> build/lib.linux-x86_64-3.8/cffi
copying cffi/parse_c_type.h -> build/lib.linux-x86_64-3.8/cffi
copying cffi/_embedding.h -> build/lib.linux-x86_64-3.8/cffi
copying cffi/_cffi_errors.h -> build/lib.linux-x86_64-3.8/cffi
running build_ext
building '_cffi_backend' extension
creating build/temp.linux-x86_64-3.8
creating build/temp.linux-x86_64-3.8/c
musl-clang -fno-strict-aliasing -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -fPIC -I/tools/deps/include -I/tools/deps/include/ncursesw -static -fPIC -DUSE__THREAD -DHAVE_SYNC_SYNCHRONIZE -I/root/python/install/include/python3.8 -c c/_cffi_backend.c -o build/temp.linux-x86_64-3.8/c/_cffi_backend.o
clang-10: warning: argument unused during compilation: '-fuse-ld=musl-clang' [-Wunused-command-line-argument]
clang-10: warning: argument unused during compilation: '-static-libgcc' [-Wunused-command-line-argument]
clang-10: warning: argument unused during compilation: '-L-user-start' [-Wunused-command-line-argument]
clang-10: warning: argument unused during compilation: '-L/usr/lib' [-Wunused-command-line-argument]
clang-10: warning: argument unused during compilation: '-L-user-end' [-Wunused-command-line-argument]
c/_cffi_backend.c:5887:16: warning: comparison of integers of different signs: 'ffi_status' and 'int' [-Wsign-compare]
    if (status == -1) {
        ~~~~~~ ^  ~~
1 warning generated.
musl-clang -shared -L/tools/deps/lib -static build/temp.linux-x86_64-3.8/c/_cffi_backend.o -L/usr/lib/../lib -lffi -o build/lib.linux-x86_64-3.8/_cffi_backend.cpython-38-x86_64-linux-gnu.so
clang-10: warning: argument unused during compilation: '-nostdinc' [-Wunused-command-line-argument]
/usr/bin/ld: warning: cannot find entry symbol _start; defaulting to 0000000000401010
/usr/bin/ld: build/temp.linux-x86_64-3.8/c/_cffi_backend.o: in function `PyInit__cffi_backend':
/root/cffi-1.14.4/c/_cffi_backend.c:7922: undefined reference to `PySys_GetObject'
/usr/bin/ld: /root/cffi-1.14.4/c/_cffi_backend.c:7924: undefined reference to `PyUnicode_AsUTF8'
/usr/bin/ld: /root/cffi-1.14.4/c/_cffi_backend.c:7925: undefined reference to `PyExc_ImportError'
/usr/bin/ld: /root/cffi-1.14.4/c/_cffi_backend.c:7925: undefined reference to `PyErr_Format'
/usr/bin/ld: /root/cffi-1.14.4/c/_cffi_backend.c:7932: undefined reference to `PyModule_Create2'
/usr/bin/ld: /root/cffi-1.14.4/c/_cffi_backend.c:7941: undefined reference to `PyDict_New'
/usr/bin/ld: /root/cffi-1.14.4/c/_cffi_backend.c:7955: undefined reference to `PyType_Ready'
/usr/bin/ld: /root/cffi-1.14.4/c/_cffi_backend.c:7959: undefined reference to `PyModule_AddObject'
/usr/bin/ld: /root/cffi-1.14.4/c/_cffi_backend.c:7976: undefined reference to `PyCapsule_New'
[...]

Putting the 2 musl-clang commands in verbose:

~/cffi-1.14.4 # musl-clang -fno-strict-aliasing -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -fPIC -I/tools/deps/include -I/tools/deps/include/ncursesw -static -fPIC -DUSE__THREAD -DHAVE_SYNC_SYNCHRONIZE -I/root/python/install/include/python3.8 -c c/_cffi_backend.c -o build/temp.linux-x86_64-3.8/c/_cffi_backend.o -v
Alpine clang version 10.0.0 (https://gitlab.alpinelinux.org/alpine/aports.git 7445adce501f8473efdb93b17b5eaf2f1445ed4c)
Target: x86_64-alpine-linux-musl
Thread model: posix
InstalledDir: /usr/bin
Found candidate GCC installation: /usr/bin/../lib/gcc/x86_64-alpine-linux-musl/9.3.0
Found candidate GCC installation: /usr/lib/gcc/x86_64-alpine-linux-musl/9.3.0
Selected GCC installation: /usr/lib/gcc/x86_64-alpine-linux-musl/9.3.0
Candidate multilib: .;@m64
Selected multilib: .;@m64
clang-10: warning: argument unused during compilation: '-fuse-ld=musl-clang' [-Wunused-command-line-argument]
clang-10: warning: argument unused during compilation: '-static-libgcc' [-Wunused-command-line-argument]
clang-10: warning: argument unused during compilation: '-L-user-start' [-Wunused-command-line-argument]
clang-10: warning: argument unused during compilation: '-L/usr/lib' [-Wunused-command-line-argument]
clang-10: warning: argument unused during compilation: '-L-user-end' [-Wunused-command-line-argument]
 (in-process)
 "/usr/bin/clang-10" -cc1 -triple x86_64-alpine-linux-musl -emit-obj -disable-free -disable-llvm-verifier -discard-value-names -main-file-name _cffi_backend.c -static-define -mrelocation-model pic -pic-level 2 -mthread-model posix -mframe-pointer=none -relaxed-aliasing -fmath-errno -fno-rounding-math -masm-verbose -mconstructor-aliases -munwind-tables -target-cpu x86-64 -dwarf-column-info -fno-split-dwarf-inlining -debug-info-kind=limited -dwarf-version=4 -debugger-tuning=gdb -v -nostdsysteminc -nobuiltininc -resource-dir /usr/lib/clang/10.0.0 -isystem /usr/include -D NDEBUG -I /tools/deps/include -I /tools/deps/include/ncursesw -D USE__THREAD -D HAVE_SYNC_SYNCHRONIZE -I /root/python/install/include/python3.8 -isysroot /usr -O3 -Wsign-compare -Wunreachable-code -Wall -fdebug-compilation-dir /root/cffi-1.14.4 -ferror-limit 19 -fmessage-length 118 -fwrapv -stack-protector 2 -fgnuc-version=4.2.1 -fobjc-runtime=gcc -fdiagnostics-show-option -fcolor-diagnostics -vectorize-loops -vectorize-slp -faddrsig -o build/temp.linux-x86_64-3.8/c/_cffi_backend.o -x c c/_cffi_backend.c
clang -cc1 version 10.0.0 based upon LLVM 10.0.0 default target x86_64-alpine-linux-musl
ignoring nonexistent directory "/tools/deps/include"
ignoring nonexistent directory "/tools/deps/include/ncursesw"
#include "..." search starts here:
#include <...> search starts here:
 /root/python/install/include/python3.8
 /usr/include
End of search list.
c/_cffi_backend.c:5887:16: warning: comparison of integers of different signs: 'ffi_status' and 'int' [-Wsign-compare]
    if (status == -1) {
        ~~~~~~ ^  ~~
1 warning generated.

~/cffi-1.14.4 # musl-clang -shared -L/tools/deps/lib -static build/temp.linux-x86_64-3.8/c/_cffi_backend.o -L/usr/lib/
../lib -lffi -o build/lib.linux-x86_64-3.8/_cffi_backend.cpython-38-x86_64-linux-gnu.so -v
Alpine clang version 10.0.0 (https://gitlab.alpinelinux.org/alpine/aports.git 7445adce501f8473efdb93b17b5eaf2f1445ed4c)
Target: x86_64-alpine-linux-musl
Thread model: posix
InstalledDir: /usr/bin
Found candidate GCC installation: /usr/bin/../lib/gcc/x86_64-alpine-linux-musl/9.3.0
Found candidate GCC installation: /usr/lib/gcc/x86_64-alpine-linux-musl/9.3.0
Selected GCC installation: /usr/lib/gcc/x86_64-alpine-linux-musl/9.3.0
Candidate multilib: .;@m64
Selected multilib: .;@m64
clang-10: warning: argument unused during compilation: '-nostdinc' [-Wunused-command-line-argument]
 "/usr/bin/ld.musl-clang" --sysroot=/usr -z now -z relro --hash-style=gnu --eh-frame-hdr -m elf_x86_64 -static -o build/lib.linux-x86_64-3.8/_cffi_backend.cpython-38-x86_64-linux-gnu.so /usr/lib/gcc/x86_64-alpine-linux-musl/9.3.0/../../../crti.o /usr/lib/gcc/x86_64-alpine-linux-musl/9.3.0/crtbeginT.o -L-user-start -L/tools/deps/lib -L/usr/lib/../lib -L/usr/lib -L-user-end -L/usr/lib/gcc/x86_64-alpine-linux-musl/9.3.0 -L/usr/lib/gcc/x86_64-alpine-linux-musl/9.3.0/../../../../x86_64-alpine-linux-musl/lib -L/usr/lib/gcc/x86_64-alpine-linux-musl/9.3.0/../../.. -L/usr/bin/../lib -L/usr/lib -l-user-start build/temp.linux-x86_64-3.8/c/_cffi_backend.o -lffi -l-user-end --start-group -lgcc -lgcc_eh -lc --end-group /usr/lib/gcc/x86_64-alpine-linux-musl/9.3.0/crtendS.o /usr/lib/gcc/x86_64-alpine-linux-musl/9.3.0/../../../crtn.o
/usr/bin/ld: warning: cannot find entry symbol _start; defaulting to 0000000000401010
/usr/bin/ld: build/temp.linux-x86_64-3.8/c/_cffi_backend.o: in function `PyInit__cffi_backend':
/root/cffi-1.14.4/c/_cffi_backend.c:7922: undefined reference to `PySys_GetObject'
/usr/bin/ld: /root/cffi-1.14.4/c/_cffi_backend.c:7924: undefined reference to `PyUnicode_AsUTF8'
/usr/bin/ld: /root/cffi-1.14.4/c/_cffi_backend.c:7925: undefined reference to `PyExc_ImportError'
/usr/bin/ld: /root/cffi-1.14.4/c/_cffi_backend.c:7925: undefined reference to `PyErr_Format'
/usr/bin/ld: /root/cffi-1.14.4/c/_cffi_backend.c:7932: undefined reference to `PyModule_Create2'
/usr/bin/ld: /root/cffi-1.14.4/c/_cffi_backend.c:7941: undefined reference to `PyDict_New'
/usr/bin/ld: /root/cffi-1.14.4/c/_cffi_backend.c:7955: undefined reference to `PyType_Ready'
/usr/bin/ld: /root/cffi-1.14.4/c/_cffi_backend.c:7959: undefined reference to `PyModule_AddObject'
/usr/bin/ld: /root/cffi-1.14.4/c/_cffi_backend.c:7976: undefined reference to `PyCapsule_New'
[...]
clang-10: error: linker command failed with exit code 1 (use -v to see invocation)

Considering all the above here is what I think:

• the distribution keeps the CC used during build phase, which means it won't be possible to use a different build toolchain on such distribution whenever a C extensions is needed in a package
• the distribution keeps some build flags that should not be there, the easy ones to identify are the -I /tools/deps/include -I /tools/deps/include/ncursesw which I suspect comes from the docker images used in the distribution build process

It's not clear to me if cffi should be compilable using x86_64-unknown-linux-musl, but if that's the case it is not apparently. Open up for suggestions.

[gi0baro]

@indygreg FYI, building cffi with the x86_64-unknown-linux-gnu works, but the dynamic linker produce wrong linking with libffi:

# ldd _cffi_backend.cpython-38-x86_64-linux-gnu.so
	linux-vdso.so.1 (0x00007ffc109af000)
	libffi-806b1a9d.so.6.0.4 => not found
	libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f0be7898000)
	libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f0be76d7000)
	/lib64/ld-linux-x86-64.so.2 (0x00007f0be7afc000)

then probably some cflags gets stored within the gnu python distribution as well, since I suspect the libffi-806b1a9d.so.6.0.4 comes from the specific debian version used in the docker images to build the distribution.

To add more context, I'm building into a debian buster based container, and libffi is installed:

root@680f38ffcb53:/wrk# find / -name libffi.so*
/usr/lib/x86_64-linux-gnu/libffi.so.6
/usr/lib/x86_64-linux-gnu/libffi.so.6.0.4
/usr/lib/x86_64-linux-gnu/libffi.so

[gi0baro]

@indygreg FYI on macOS I don't face any issue at all, can build cffi correctly and the dynamic linker produce correct results:

 ❯ otool -L _cffi_backend.cpython-38-darwin.so
_cffi_backend.cpython-38-darwin.so:
	/usr/lib/libffi.dylib (compatibility version 1.0.0, current version 1.0.0)
	/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1252.250.1)

[indygreg]

The musl-clang and non-existent paths references are embedded within Python because Python embeds build system settings in itself (notably the sysconfig module). What's happening here is the Docker-based build environment used by this project is being embedded in the distribution. Then when you attempt to use those settings in a different environment, they aren't accurate and you get failures like what you've seen.

Probably the most viable alternative to this is for this project to emit a Docker image or something containing its build environment. Then we can tell people to execute Python packaging commands within that environment. That being said, this type of packaging work is something I consider somewhat out of scope for this project and something more the domain of PyOxidizer. But this project would need to emit artifacts like the Clang toolchain used, certain dependencies, possibly packaged as a self-contained Docker image or equivalent.

[mcsba]

Originally I needed python3 for an old Red Hat server so my solution was that I have built python3 (and openssl) on a Red Hat 6.10 server. This python is not independent from the operating system and probably it is not suitable for all purpose but it works on Red Hat 6.10 and Ubuntu 20.10 too.

[gi0baro]

Probably the most viable alternative to this is for this project to emit a Docker image or something containing its build environment. Then we can tell people to execute Python packaging commands within that environment. That being said, this type of packaging work is something I consider somewhat out of scope for this project and something more the domain of PyOxidizer. But this project would need to emit artifacts like the Clang toolchain used, certain dependencies, possibly packaged as a self-contained Docker image or equivalent.

Understood. Maybe an alternative might also be providing some utils to configure the distribution on the system if needed, somewhat like pyenv does, WDYT @indygreg ?

[indygreg]

Cross compiling is notoriously difficult, especially on Linux. The most reliable way to facilitate this is to enable someone else to recreate the original/desired environment, as bit-for-bit identical as possible. On Linux, the most reliable way to do that is with an isolated containerized environment. e.g. a Docker image.

[tsibley]

Just noting that I'm also unable to use the x86_64-unknown-linux-musl target of PyOxidizer because of this issue (incl. the specifics of cffi). Would love build images.

[gi0baro]

Just noting that I'm also unable to use the x86_64-unknown-linux-musl target of PyOxidizer because of this issue (incl. the specifics of cffi). Would love build images.

@tsibley I build musl targets with pyoxidizer with these docker images https://github.com/gi0baro/pyoxidizer-images/pkgs/container/pyoxidizer

[tsibley]

@gi0baro Thanks for the pointer!

Unfortunately, my build errors in the cffi build even when running in your images. The compile seems to succeed, but then during linking the symbols from libpython seem to be unlocatable. There are some warnings from clang about unused arguments, and though I'm not sure yet if they're related to the errors or not, I suspect they are. e.g. there's an -I/tmp/.cache/pyoxidizer/python_distributions/python.e00f97384881/python/install/include/python3.10 in the cc flags but no corresponding ld flag for /tmp/.cache/pyoxidizer/python_distributions/python.e00f97384881/python/install/lib/python3.10/.../python.o or similar. Full set of errors.

AFAICT, these are still the same issues you encountered in #33 (comment).

[gi0baro]

@gi0baro Thanks for the pointer!

Unfortunately, my build errors in the cffi build even when running in your images. The compile seems to succeed, but then during linking the symbols from libpython seem to be unlocatable. There are some warnings from clang about unused arguments, and though I'm not sure yet if they're related to the errors or not, I suspect they are. e.g. there's an -I/tmp/.cache/pyoxidizer/python_distributions/python.e00f97384881/python/install/include/python3.10 in the cc flags but no corresponding ld flag for /tmp/.cache/pyoxidizer/python_distributions/python.e00f97384881/python/install/lib/python3.10/.../python.o or similar. Full set of errors.

AFAICT, these are still the same issues you encountered in #33 (comment).

But wait, even if you would be able to build cffi, pyoxidizer would still fail producing the final target, as you cannot dynamic link libraries with musl..

[tsibley]

@gi0baro My understanding (and it may be incorrect!) is that when building for musl, PyOxidizer tries to build each Python extension module from scratch and then treat it like a builtin extension module (ref):

This means PyOxidizer cannot consume Python wheels or other Python resource sources containing pre-built Python extension modules.

In order for PyOxidizer to support a Python extension module built for musl libc, it must compile that extension module from source and link the resulting object files / static library directly into the built binary and expose that extension module as a built-in. This is done using Building with a Custom Distutils.

and from that last link:

For example, on Linux, PyOxidizer copies the intermediate object files produced by the build and links them into the binary containing the generated libpython. PyOxidizer completely ignores the shared library that is or would typically be produced.

Those descriptions do seem to match what I see happening (e.g. PyOxidizer is building cffi from source instead of using the musllinux_1_1_x86_64 wheel that's on PyPI), but it's going wrong.