Skip to content

Build with Python ABI3 to reduce build matrix #469

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 4 commits into
base: main
Choose a base branch
from

Conversation

traversaro
Copy link
Contributor

@traversaro traversaro commented Apr 23, 2025

Fix #466 .

More details on the used PYTHON3_LIMITED_API CMake option:

Checklist

  • Used a personal fork of the feedstock to propose changes
  • Bumped the build number (if the version is unchanged)
  • Reset the build number to 0 (if the version changed)
  • Re-rendered with the latest conda-smithy (Use the phrase @conda-forge-admin, please rerender in a comment in this PR for automated rerendering)
  • Ensured the license file is being packaged.

@conda-forge-admin
Copy link
Contributor

conda-forge-admin commented Apr 23, 2025

Hi! This is the friendly automated conda-forge-linting service.

I just wanted to let you know that I linted all conda-recipes in your PR (recipe/meta.yaml) and found it was in an excellent condition.

I do have some suggestions for making it better though...

For recipe/meta.yaml:

  • ℹ️ The recipe is not parsable by parser conda-souschef (grayskull). This parser is not currently used by conda-forge, but may be in the future. We are collecting information to see which recipes are compatible with grayskull.
  • ℹ️ The recipe is not parsable by parser conda-recipe-manager. The recipe can only be automatically migrated to the new v1 format if it is parseable by conda-recipe-manager.
  • ℹ️ top-level output has some malformed specs:
  • In section host: ffmpeg ffmpeg=lgpl_*, hdf5 =*=nompi*
    Requirement spec fields should match the syntax name [version [build]]to avoid known issues in conda-build. For example, instead of name =version=build, use name version.* build. There should be no spaces between version operators and versions either: python >= 3.8 should be python >=3.8.

This message was generated by GitHub Actions workflow run https://github.com/conda-forge/conda-forge-webservices/actions/runs/14636489353. Examine the logs at this URL for more detail.

@traversaro
Copy link
Contributor Author

@conda-forge-admin, please rerender

conda-forge-webservices[bot] and others added 2 commits April 23, 2025 17:43
@traversaro
Copy link
Contributor Author

traversaro commented Apr 24, 2025

Linux builds are failing with:


During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/opt/conda/bin/conda-build", line 11, in <module>
    sys.exit(execute())
             ^^^^^^^^^
  File "/opt/conda/lib/python3.12/site-packages/conda_build/cli/main_build.py", line 622, in execute
    api.build(
  File "/opt/conda/lib/python3.12/site-packages/conda_build/api.py", line 211, in build
    return build_tree(
           ^^^^^^^^^^^
  File "/opt/conda/lib/python3.12/site-packages/conda_build/build.py", line 3664, in build_tree
    packages_from_this = build(
                         ^^^^^^
  File "/opt/conda/lib/python3.12/site-packages/conda_build/build.py", line 2775, in build
    newly_built_packages = bundlers[pkg_type](
                           ^^^^^^^^^^^^^^^^^^^
  File "/opt/conda/lib/python3.12/site-packages/conda_build/build.py", line 1878, in bundle_conda
    files = post_process_files(metadata, initial_files)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/conda/lib/python3.12/site-packages/conda_build/build.py", line 1675, in post_process_files
    noarch_python.populate_files(m, pkg_files, host_prefix, [])
  File "/opt/conda/lib/python3.12/site-packages/conda_build/noarch_python.py", line 95, in populate_files
    handle_file(f, d, prefix)
  File "/opt/conda/lib/python3.12/site-packages/conda_build/noarch_python.py", line 76, in handle_file
    fn = rewrite_script(fn, prefix)
         ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/conda/lib/python3.12/site-packages/conda_build/noarch_python.py", line 31, in rewrite_script
    raise CondaBuildUserError(f"Noarch package contains binary script: {fn}")
conda_build.exceptions.CondaBuildUserError: Noarch package contains binary script: opencv_visualisation

It seems like python_version_independent: true also triggers some noarch: python checks, even if it is not a noarch: python package? I am not sure if that is intentional or not.

@traversaro
Copy link
Contributor Author

Beside the test error in conda-build, I think we also need to revise the build strings to reflect the abi3 use, as encoding the python version is the build string may now be misleading.

@traversaro
Copy link
Contributor Author

Beside the test error in conda-build, I think we also need to revise the build strings to reflect the abi3 use, as encoding the python version is the build string may now be misleading.

Actually that is still what is done by abi3 builds, see for example https://conda-metadata-app.streamlit.app/?q=conda-forge%2Flinux-64%2Fpolars-1.27.1-py39h2a4a510_3.conda and https://conda-metadata-app.streamlit.app/?q=conda-forge%2Flinux-64%2Fatomic-counter-0.1.4-py39h677e893_2.conda . So that is good to go, for consistency with other builds.

@traversaro
Copy link
Contributor Author

By reading https://conda.org/learn/ceps/cep-0020/, I wonder if it is possible at all to use python_version_independent: true for a package that also install files (including platform-specific binary files) outside of the $SP_DIR folder.

@hmaarrfk
Copy link
Contributor

I think we can proably finally use the include/exclude
https://github.com/conda-forge/freetype-feedstock/pull/58/files

@hmaarrfk
Copy link
Contributor

and then 6 years later, you may finally be able to update the comment at the top so that the package split makes sense!

@traversaro
Copy link
Contributor Author

I think we can proably finally use the include/exclude https://github.com/conda-forge/freetype-feedstock/pull/58/files

So you suggest to move the Python files out of libopencv and into py-opencv ?

@hmaarrfk
Copy link
Contributor

Yes.

Ultimate What I did 7 years ago was a hack because of the complications of compiling opencv. This abi3 might be what opens up the possibility to move away from it in a sane way.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

build with ABI3 to reduce build matrix
3 participants