Strategies for bundling CMake configuration values for scikit-build-core

[eirrgang]

I'm opening this Discussion to continue a conversation from the monthly conference call this morning (with JC and Christian), in which I asked for strategic advice on integrating historic practices at my organization with scikit-build-core driven build systems in the future.

tl;dr

Given the useful stash of environment-specific CMake bootstrapping information in https://github.com/LLNL/blt/tree/develop/host-configs, what's the best way to start making BLT-dependent projects work well with scikit-build-core?

(See also https://llnl-blt.readthedocs.io/en/develop/tutorial/host_configs.html)

Detail

When someone needs to provide a bunch of CMake configuration values (such as to encapsulate the differences between quirky HPC environments) three common mechanisms leap to mind.

• initial cache files, provided on the command line with -C
• toolchain files
• Preset files (since CMake 3.19)

Currently, various projects that rely on BLT use initial cache files (colloquially known as host-config files) as a way to share important details for building optimal and compatible libraries in a number of computing environments. Currently, BLT needs to support CMake versions less than 3.19. Changing our practice (and documentation, tutorials, etc) from initial cache files to something else would have a lot of inertia, especially since most projects would see no initial benefit. We would not want to maintain this environment-specific knowledge in more than one place or file format, but we could conceivably use some scripting to generate different forms of configuration input files from a single source.

scikit-build-core generates its own initial cache file and calls cmake with -C under the hood.

I understand that toolchain files are supported by scikit-build-core, but have additional semantics beyond an initial cache file. I'm not sure if this would be an appropriate mechanism for current or future BLT based projects.

It sounds like scikit-build-core is on the verge of supporting CMake preset files. It seems plausible that BLT users might eventually want to migrate, generally, to preset files, but I expect that will be several years from now. It seems feasible that the BLT host-configs could be migrated to the preset file JSON format sooner than that and derive the traditional host-config cache files from the JSON with scripting, but that sounds like a lot of complexity that would be greeted very skeptically if proposed, so it would need to be very clearly motivated.

Some reasons scikit-build-core would not want to encourage user-provided -C configurations include potential conflicts with more explicit equivalent options (https://scikit-build-core.readthedocs.io/en/latest/reference/configs.html#cmake) or ambiguity with respect to its generated -C config. Is there any chance it is reasonably simple and unambiguous to merge these configs? (If this were well-defined, I could also imagine a route to invoking scikit-build-core from CMake, which occasionally seems useful (but scary).)

My expectation is that the short term solution will be to have a script that converts our cache files to pip / build command line arguments so that scikit-build-core can be invoked in the usual way and neither infrastructure project needs modification. Maybe that's the long-term solution as well.

I'm also curious about the possibility (or desirability) of a plugin package that could be identified in the pyproject.toml to process a front-end CLI option identifying a BLT "host-config" file which it then parses to inject scikit-build-core configuration values in an appropriate way. I suspect that's not something that is already straightforward with just PEP-517 hooks. Maybe this would be more in the territory of a plugin for scikit-build-core. Do you think the necessary hooks already exist?

Is it really that hard?

A bunch of set(... ... CACHE ... ...) commands are equivalent to a bunch of -D...=... CLI flags or tool.scikit-build.cmake.define mappings.

However, some initial cache files set non-cache temporary/intermediate variables, or use CMake commands like string(REPLACE ...) to prepare the cache values. Such a host-config file would need to be pre-processed with CMake, such as in a wrapper script or PEP-517 connected code. I don't know how to do that other than to generate a dummy project, and configure it.

On the other hand, the generated project could do most of the work by including something like

get_cmake_property(_variableNames CACHE_VARIABLES)
foreach (_variableName ${_variableNames})
    message("-Ccmake.define.${_variableName}=${${_variableName}}")
endforeach()

[eirrgang]

Here's the gist of a conversation I just had with @henryiii...

1. I've actually tried just doing python -m build . -Ccmake.args=-C<path/to/initialCache.cmake and it seems to be working just fine, but I may be stomping on stuff that scikit-build-core is trying to set in the initial cache file it generates. I may not have noticed because these are all projects that were initially set up without scikit-build-core, and already had CMake scripting (or documented workarounds) that took the place of the helpful things scikit-build-core provides.
2. It is trivial to add a hook like -Ccmake.initial_cache=... so that at least scikit-build-core can be made aware that the user is trying to provide their own -C to CMake.
3. We should double-check CMake's behavior wrt multiple -C arguments, and then decide whether/how to merge or interpret multiple sources of set() for CMake input variables (and whether to pursue through scikit-build-core or CMake, itself).

[LecrisUT]

1. I've actually tried just doing `python -m build . -Ccmake.args=-C<path/to/initialCache.cmake` and it seems to be working just fine, but I may be stomping on stuff that scikit-build-core is trying to set in the initial cache file it generates.

If you want to check, see if Python_ROOT and such are set. SKBUILD_STATE and such are also set via cache file iirc.

3. We should double-check CMake's behavior wrt multiple `-C` arguments, and then decide whether/how to merge or interpret multiple sources of `set()` for CMake input variables (and whether to pursue through scikit-build-core or CMake, itself).

Definitely. If the first part works, that would simplify stuff a lot. Otherwise best option imo is to append your content to ours because you can use set(CACHE FORCE) to override our values or set(CACHE) to have ours take precedence.

If you want to make a quick PR for it. I wanted to write one, but didn't get around to yet.

[henryiii]

Multiple caches work fine:

$ cd tests/packages/simple_pure
$ echo 'set(SKBUILD 1 CACHE STRING "")' > Cache1.cmake
$ echo 'set(SKBUILD2 2 CACHE STRING "")' > Cache2.cmake
$ cmake -S. -Bbuild -C Cache1.cmake  -C Cache2.cmake
loading initial cache file Cache1.cmake
loading initial cache file Cache2.cmake
-- The CXX compiler identification is AppleClang 17.0.0.17000013
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /usr/bin/c++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- SKBUILD is defined to 1
-- SKBUILD2 is defined to 2
-- Configuring done (1.3s)
-- Generating done (0.0s)
-- Build files have been written to: ~/git/scikit-build/scikit-build-core/tests/packages/simple_pure/build