Goal: Get this minpack used by SciPy

[certik]

SciPy contains minpack: https://github.com/scipy/scipy/tree/main/scipy/optimize/minpack; we should develop this fortran-lang/minpack in a way so that SciPy could use it. We should add some improvements (perhaps some new minimization algorithms), and then include them in SciPy. If we can pull this off, that would be a huge win, for both Fortran and SciPy. SciPy gets a maintainable version, we can all contribute to it in Fortran, and modern Fortran gets a "user" in a high profile Python library.

Fortran Forum discussion thread: https://fortran-lang.discourse.group/t/lets-get-fortran-lang-minpack-into-scipy/2722

[jacobwilliams]

One thing TODO: check to see what changes they have made to the original code and port those to this version. See #23.

[milancurcic]

I agree, this would be a huge boon for Fortran and SciPy. Great idea!

[awvwgk]

Here is a quick outline of the things I found in SciPy on this:

• minpack itself is compiled as static library (this is the part we actually care about here):
  • scipy/optimize/meson.build#L9-L35
• a Python extension module (scipy.optimize._minpack) is defined directly via the Python and numpy headers
  • scipy/optimize/_minpackmodule.c
• two headers are defined for this extension module:
  • scipy/optimize/minpack.h
  • scipy/optimize/__minpack.h
• the extension module for minpack is compiled in meson
  • scipy/optimize/meson.build#L37-L46
• a Python wrapper module (scipy.optimize._minpack_py) is build around this setup
  • scipy/optimize/_minpack_py.py
• this module is reexported as scipy.optimize.minpack, however direct access seems deprecated
  • scipy/optimize/minpack.py
• instead of actually using scipy.optimize.minpack for anything, scipy.optimize._minpack_py is reexported
  • scipy/optimize/__init__.py#L403

---

Note that there is a scipy.optimize.minpack2 module, this is interfaced via f2py

• interface definition in scipy/optimize/minpack2/minpack2.pyf
• generation of extension module at scipy/optimize/meson.build#L136-L151
• reexport in as Python module in scipy/optimize/minpack2.py
• not sure whether this module is later reexported somewhere else

[sakamoti]

How about considering adding optional argument class(*) for fortran user?

In the f77 version, the user defined subroutine, which is the argument like hybrd1, is defined as func(n, x, fvec, iflag), etc. The user defined derived type cannot be passed.
So, I use module variables or the internal subroutine to refer to the type.

By adding class(*) as an optional argument to func, for example, I think it is straightforward way to pass user-defined types. Personally, Such an additional argument class(*),optional will provide well userbility for fortran user.

Is it practically possible to add an optional argument like class(*),optional when considering the integration with Scipy?

[jacobwilliams]

See my comments in #4. I think that's the way to go, rather than having an optional argument.

[awvwgk]

There is also a relevant discussion on the callback design in https://fortran-lang.discourse.group/t/modern-fortran-interface-for-nlopt/2685/6 and grimme-lab/nlopt-f#16. I think this is a quite fundamental issue, and I'm hitting against the limitation of the current callback when building the C API already.

[sakamoti]

@jacobwilliams
Thank you for the link. I had missed that discussion. The code in the link is enough to serve the purpose.

@awvwgk
I really appreciate it, because discussions like the one you linked to are hard to find outside of fortran-lang.

[awvwgk]

Is it practically possible to add an optional argument like class(*),optional when considering the integration with Scipy?

I would go so far to say that is necessary to pass additional data with the procedure pointer to allow integration with SciPy, Right now we have to rely on C and Fortran using similar enough calling conventions to use the same callback function interface. Once we switch to a more Fortran-friendly interface (assumed shape arrays instead of explicit shape arrays) this is no longer true.

[Nicholaswogan]

One other thing to add that I think is relevant. Currently released GNU compilers (e.g. GNU Fortran (Homebrew GCC 11.2.0_3) 11.2.0) do not support nested functions on Apple Silicon. This will change someday but it isn't clear when (see here). If Scipy relies on GNU compilers to build binaries, then passing data via nested functions will prevent building Apple Silicon binaries.