Compatibility with arrow storage formats

[wlandau]

I have heard reports of folks having trouble saving sf objects using the feather and parquet formats from the arrow package. I do not personally work with spatial data, but @jaredlander could fill you in.

A lot has happened since #10, including list column support for both feather and parquet. ropensci/targets#599 is also relevant, but I feel this thread is appropriate for discussion because a solution in sf and arrow could help far more R users than just users of targets.

[jaredlander]

I'll try to create a reproducible example, but have had great failure trying this.

But question, does write_sf() support writing to parquet or do I have to do this using the {arrow} package? (Using {arrow} is how I've been failing.)

[edzer]

write_sf won't do that, as it writes through gdal, and there's no driver for arrow or parquet afaict. Have you tried writing with pkg arrow after removing the sf class, e.g. using as.data.frame or as_tibble on the sf object?

On August 13, 2021 4:56:16 AM GMT+02:00, Jared Lander ***@***.***> wrote: I'll try to create a reproducible example, but have had great failure trying this. But question, does `write_sf()` support writing to parquet or do I have to do this using the `{arrow}` package? (Using `{arrow}` is how I've been failing.) -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: #1756 (comment)

-- Sent from my Android device with K-9 Mail. Please excuse my brevity.

[mlhollestelle]

Are you reconsidering support for arrow storage formats, since GDAL 3.8 supports parquet?

[edzer]

Have you checked #2036 and the use_stream argument in st_read()? Doesn't sf support (write) support go (geo)parquet once GDAL supports this?

[mlhollestelle]

Thanks for the quick response. This looks good.