How to read Overture data by theme and bbox using Python bindings

[justb4]

Trying to read data from Overturemaps.org using geoarrow.rust.io in Python 3.12. Tried the Python lines from the example Jupyter notebook (without lonboard), but stumbled on at least these issues:

• from geoarrow.rust.io.store import S3Store S3Store seems obsolete, probably ObjectStore now
• the manifest file path is invalid: "https://raw.githubusercontent.com/OvertureMaps/io-site/refs/heads/manifest_driven_downloads/site/src/09-18-manifest.json" : repo name is now explore-site.

NB Found a working, recent manifest:
"https://raw.githubusercontent.com/OvertureMaps/explore-site/refs/heads/main/site/src/manifests/2025-04-23.json"

I mainly stumbled on how to populate what I think now should be geoarrow.rust.io.ObjectStore:

• Bucket S3 URL
• ObjectStore options dict
• path or paths

Something like:

options = {
?
}
store = ObjectStore('s3://overturemaps-us-west-2/release/2025-05-21.0', options=options)
# derive paths for theme 'Place' from manifest JSON
ds = ParquetDataset(paths, store)
etc

Would be nice to have an example in pure Python without Jupyter. Once I know the pattern I could contribute such an example to this repo as I am researching different "Parquet" access and manipulation from Python. Thanks!

[kylebarron]

There's a been a bit of an overhaul of the geoarrow Rust library, and I've been redoing the Parquet implementation (e.g. #1089, #1172). I'm working on #1170, which will update the Python bindings for reading GeoParquet

[kylebarron]

As of latest main (with #1176), this is newly supported again.

Here's an example of accessing Overture building data, in this case on Azure but it's easy to swap out the AzureStore for an S3Store. See obstore docs

from obstore.store import AzureStore
from geoarrow.rust.io import GeoParquetDataset
import requests
import lonboard

# Fetch the manifest
manifest_url = "https://raw.githubusercontent.com/OvertureMaps/explore-site/refs/heads/main/site/src/manifests/2025-04-23.json"
manifest = requests.get(manifest_url).json()

# parse the manifest
path = "2025-04-23.0"
buildings_theme = [x for x in manifest["themes"] if x["name"] == "buildings"][0]
path += buildings_theme["relative_path"]
buildings_type = [x for x in buildings_theme["types"] if x["name"] == "building"][0]
path += buildings_type["relative_path"]

store = AzureStore(
    account_name="overturemapswestus2", container_name="release", skip_signature=True
)
files = store.list(path).collect()

# open the dataset
dataset = GeoParquetDataset(files, store=store)

# spatial read
table = dataset.read(bbox=(-70.720192, -33.492286, -70.602635, -33.388274))

# visualize it
lonboard.viz(table)

[justb4]

Ok, thanks for extensive reply @kylebarron ! I was using geoarrow-rust-io 0.3.0 (sept 2024) from PyPi. Will try to install from repo. New modular conventions with obstore and above example look clear!

[kylebarron]

There's some documentation here: https://github.com/geoarrow/geoarrow-rs/blob/1d73a8c8f739ac2a9f4cbac928283c19b74db463/python/DEVELOP.md#development

I'd like to get an 0.4 python release in the next couple weeks

[justb4]

Ok thanks again! Looking forward to 0.4.0. Background is updating the GeoPython workshop we provide at FOSS4G EU (and NZ later) , and maybe in the future a pygeoapi GeoParquet Provider. But no stress, all in good time!