Compilation on MacOS Monterey M1 (ARM64 architecture)

[aourednik]

Compiling from source under MacOS Monterey 12.0.1 on M1 (ARM64) fails.
R version 4.1.2 (2021-11-01)
Platform: aarch64-apple-darwin20.6.0 (64-bit)

Steps :

• reinstall Homebrew for ARM64 (instead of usr/local/ this installs homebrew to the arm64 compatible directory opt/homebrew)
• install pkg-config, GDAL, PROJ and GEOS with homebrew install
• add these lines to ~/R./Makevars:

FC      = /opt/homebrew/bin/gfortran
F77     = /opt/homebrew/bin/gfortran

• run install.packages("sf") : Fails with error "gdal-config not found or not executable"
• run install.packages("sf",configure.args = "--with-gdal-config=/opt/homebrew/opt/gdal/bin/gdal-config") : Solves the gdal-problem, but also fails with this trace:

...
checking GDAL: linking with --libs only... yes
checking GDAL: /opt/homebrew/Cellar/gdal/3.3.3/share/gdal/pcs.csv readable... no
checking GDAL: checking whether PROJ is available for linking:... yes
checking GDAL: checking whether PROJ is available fur running:... yes
configure: GDAL: 3.3.3
./configure: line 3657: pkg-config: command not found
checking proj.h usability... yes
checking proj.h presence... yes
checking for proj.h... yes
checking PROJ: checking whether PROJ and sqlite3 are available for linking:... no
configure: error: libproj or sqlite3 not found in standard or given locations.
ERROR: configuration failed for package ‘sf’

[aourednik]

This is possibly solved here by @drkrynstrng :
rspatial/terra#396
But has this been merged?

[abuabara]

on the M1 I have been using it without homebrew. You need to install some of these libraries: https://mac.r-project.org/libs-arm64/

i'm late for a class ... i will try to reply you later with more information. Just wanted to give you some update.

[abuabara]

On my M1 I just had to install "gfortran" and "pkgconfig" here "/opt/R/arm64/bin". No homebrew was necessary. The packages in R compiles from the source normally after that. MacOS may ask you permissions, just open the files, right click on it, and terminal will allow permissions automatically.

By the way, "to install" these libraries, just unzip the files and move the folders, keeping the same structure. The very first one you will only need to move the folders. If you need more libraries, keep the same file structure.

Hope it helps. Let us know if any question.

[edzer]

close?

[aourednik]

Now it compiles correctly on an M1 with Monterey even with a standard homebrew install of pkg-config, GDAL, PROJ and GEOS.
Perhaps also due to today's series of updates to many other R libraries?
In any case, it compiles.
Many thanks!

[CCheCastaldo]

Hi @aourednik. It seems as though you got sf to install from source with pkg-config, gdal, proj, and geos installed via homebrew on an M1 Mac. I have the same setup as you and am still getting the original error above (pkg-config not found). Were there additional steps you had to take? I did:

install.packages("sf",configure.args = "--with-gdal-config=/opt/homebrew/bin/gdal-config", type = "source")

and get the same error above:

...
checking GDAL: linking with --libs only... yes
checking GDAL: /opt/homebrew/Cellar/gdal/3.3.3_1/share/gdal/pcs.csv readable... no
checking GDAL: checking whether PROJ is available for linking:... yes
checking GDAL: checking whether PROJ is available fur running:... yes
configure: GDAL: 3.3.3
./configure: line 3662: pkg-config: command not found
checking proj.h usability... no
checking proj.h presence... no
checking for proj.h... no
checking proj_api.h usability... no
checking proj_api.h presence... no
checking for proj_api.h... no

and the same error again after running this in R prior to the install.packages command:

print(Sys.setenv(PKG_CONFIG_PATH = "/opt/homebrew/lib/pkgconfig/"))
print(Sys.setenv(PKG_CONFIG = "/opt/homebrew/bin/pkg-config"))

I have no problem executing the pkg-config command in a bash terminal. For example, which finds it just fine and pkg-config --modversion gdal finds the gdal .pc file to return its version. It just seems as though R doesn't know where pkg-config is and I cannot seem to tell it. I left off the .Makevars bit you did as I am not sure if this is needed.

Thanks for any advice.

MacOS Monterey 12.1 on M1 Pro (ARM64)
R version 4.1.2 (2021-11-01)

[CCheCastaldo]

OK, I think I figured it out. For whatever reason, R was not recognizing the PATH env variable updated in my .bash_profile to begin with /opt/homebrew/bin. Instead R was defaulting to the non-updated PATH that begins with usr/local/bin. This was causing R to not find pkg-config during installations of sf from source. This then worked for me:

1. brew install pkg-config, gdal, and proj
2. In R, adjust the path to point first to /opt/homebrew/bin

Sys.getenv("PATH")
path_temp <- paste0("/opt/homebrew/bin:", Sys.getenv("PATH"))
print(Sys.setenv(PATH = path_temp))
Sys.getenv("PATH")

3. Install rgdal and sf from source.

install.packages("rgdal", type = "source")
install.packages("sf", type = "source")

I did not need to add any configuration arguments for gdal or proj, as R found everything and both packages installed cleanly (although sf had 6 warnings which I have seen in the past on previous installs). Now sf links to the homebrew spatial libraries:

library(sf)
Linking to GEOS 3.10.1, GDAL 3.3.3, PROJ 8.2.0; sf_use_s2() is TRUE

This same approach also worked for terra and stars. I am not sure if the PATH always needs to have /opt/homebrew/bin in front or just when installing these packages, but that should be obvious after some use. Hope this is helpful and does not add confusion if I am wrong about all this.

My system is:
MacOS Monterey 12.1 on M1 Pro (ARM64)
R version 4.1.2 (2021-11-01) -- "Bird Hippie"
Platform: aarch64-apple-darwin20 (64-bit)

[aourednik]

Unfortunately, the problem is back with gdal 3.4 : sf is looking for the 2.9 version.

As of 2022, Homebrew updates GDAL to 3.4 /opt/homebrew/Cellar/gdal/3.4.0/lib/libgdal.30.dylib , symlinked from /opt/homebrew/opt/gdal/

The effect on the sf library is immediate:

> library("sf")
Erreur : le chargement du package ou de l'espace de noms a échoué pour ‘sf’ in dyn.load(file, DLLpath = DLLpath, ...) :
impossible de charger l'objet partagé '/opt/homebrew/lib/R/4.1/site-library/sf/libs/sf.so':
  dlopen(/opt/homebrew/lib/R/4.1/site-library/sf/libs/sf.so, 0x0006): Library not loaded: /opt/homebrew/opt/gdal/lib/libgdal.29.dylib
  Referenced from: /opt/homebrew/lib/R/4.1/site-library/sf/libs/sf.so
  Reason: tried: '/opt/homebrew/opt/gdal/lib/libgdal.29.dylib' (no such file), '/opt/homebrew/Cellar/r/4.1.2/lib/R/lib/libgdal.29.dylib' (no such file), '/opt/homebrew/Cellar/gdal/3.4.0/lib/libgdal.29.dylib' (no such file), '/opt/homebrew/Cellar/r/4.1.2/lib/R/lib/libgdal.29.dylib' (no such file)

Trying to reinstall:

> remotes::install_github("r-spatial/sf", configure.args = "--with-gdal-config=/opt/homebrew/opt/gdal/bin/gdal-config --with-sqlite3-lib=/opt/homebrew/opt/sqlite3")
Downloading GitHub repo r-spatial/sf@HEAD
   checking for file ‘/private/var/folders/p8/7m7_z7754pdffd5z_f7n15540000gp/T/R✔  checking for file ‘/private/var/folders/p8/7m7_z7754pdffd5z_f7n15540000gp/T/RtmpADyrFG/remotes1208caa9cfe8/r-spatial-sf-e2c8936/DESCRIPTION’
─  preparing ‘sf’:
✔  checking DESCRIPTION meta-information ...
─  cleaning src
─  running ‘cleanup’
─  checking for LF line-endings in source and make files and shell scripts (443ms)
─  checking for empty or unneeded directories
─  building ‘sf_1.0-6.tar.gz’
   
Installation du package dans ‘/opt/homebrew/lib/R/4.1/site-library’
(car ‘lib’ n'est pas spécifié)
* installing *source* package ‘sf’ ...
** using staged installation
configure: CC: clang
configure: CXX: clang++ -std=gnu++11
configure: gdal-config set to /opt/homebrew/opt/gdal/bin/gdal-config
checking gdal-config exists... yes
checking gdal-config executable... yes
checking gdal-config usability... yes
configure: GDAL: 3.4.0
checking GDAL version >= 2.0.1... yes
checking for gcc... clang
checking whether the C compiler works... yes
checking for C compiler default output file name... a.out
checking for suffix of executables... 
checking whether we are cross compiling... no
checking for suffix of object files... o
checking whether we are using the GNU C compiler... yes
checking whether clang accepts -g... yes
checking for clang option to accept ISO C89... none needed
checking how to run the C preprocessor... clang -E
checking for grep that handles long lines and -e... /usr/bin/grep
checking for egrep... /usr/bin/grep -E
checking for ANSI C header files... yes
checking for sys/types.h... yes
checking for sys/stat.h... yes
checking for stdlib.h... yes
checking for string.h... yes
checking for memory.h... yes
checking for strings.h... yes
checking for inttypes.h... yes
checking for stdint.h... yes
checking for unistd.h... yes
checking gdal.h usability... yes
checking gdal.h presence... yes
checking for gdal.h... yes
checking GDAL: linking with --libs only... yes
checking GDAL: /opt/homebrew/Cellar/gdal/3.4.0/share/gdal/pcs.csv readable... no
checking GDAL: checking whether PROJ is available for linking:... yes
checking GDAL: checking whether PROJ is available fur running:... yes
configure: GDAL: 3.4.0
configure: pkg-config proj exists, will use it
configure: using proj.h.
configure: PROJ: 8.2.1
checking PROJ: checking whether PROJ and sqlite3 are available for linking:... no
configure: error: libproj or sqlite3 not found in standard or given locations.
ERROR: configuration failed for package ‘sf’

I've tried to install an older version of GDAL with homebrew, to no avail:

brew install gdal@2
Warning: No available formula with the name "gdal@2". Did you mean gdal or gdl?
==> Searching for similarly named formulae...
These similarly named formulae were found:
gdal ✔                                   gdl

[rsbivand]

Please note that homebrew is the choice and complete responsibility of anyone choosing that path. M1 is supported with CRAN binaries, static linked to external software (usually not the latest version, but fairly new). You seem to be surprised that your sf installation ceased to load when the dynamically linked library against which it was built and installed is absent. It is unusual for ABI compatibility to extend over GDAL/PROJ/GEOS releases, so any local update to any of these upstream dependencies will require a source install of sf.

The final problem you hit is that software required by PROJ is not found (PROJ needs libtiff, libsqlite3 and libcurl in places where they can be found by configure). So for convenience and less loss of time, use CRAN binaries only and never install spatial packages using external software from source (sf, lwgeom, terra, vapour, rgdal, rgeos, etc.). The only viable reason for source installs using non-CRAN dynamic linking is that newer versions offer functionality not present in CRAN static-linked binaries.

You may of course install source packages using CRAN static-built libraries, but this is only relevant if you must have the development version of a package, but this should really only apply to those contributing to package development (you've made an edit to a local fork of sf and want to check its performance).

[aourednik]

Thanks for fast reply @rsbivand .

Today I've noticed that my MacOS version of R, for a reason I believed necessary up to now, defaults to building from source when running install.packages("package-name") despite there being MacOS binaries available on CRAN.
I confirm that just downloading the .tgz for arm64 from CRAN and running R CMD INSTALL sf_1.0-5.tgz from the MacOS terminal solves the problem.

SF is a great package, I've become quite dependent to it in my processes, it's always a haunting experience when it stops working :-)

[abuabara]

just have a .Renviron file in your home folder with R_COMPILE_AND_INSTALL_PACKAGES=never and don't worry anymore about compiling packages from source. Only 2 cents from a non-developer user.