Adds Support for COPY TO/FROM Google Cloud Storage

[aykut-bozkurt]

Supports following Google Cloud Storage uri forms:

• gs:// <bucket> / <path>

Configuration

The simplest way to configure object storage is by creating a json config file like ~/.config/gcloud/application_default_credentials.json (can be generated by gcloud auth application-default login):

$ ~/.config/gcloud/application_default_credentials.json
{
  "gcs_base_url": "gs://testbucket/test.parquet",
  "disable_oauth": false,
  "client_email": "...",
  "private_key_id": "...",
  "private_key": "..."
}

Alternatively, you can use the following environment variables when starting postgres to configure the Google Cloud Storage client:

• GOOGLE_SERVICE_ACCOUNT_KEY: json serialized service account key
• GOOGLE_SERVICE_ACCOUNT_PATH: an alternative location for the config file

Closes #62

[codecov]

Codecov Report

Attention: Patch coverage is 94.89051% with 7 lines in your changes missing coverage. Please review.

Project coverage is 91.15%. Comparing base (7f6d421) to head (f164838).
Report is 1 commits behind head on main.

Files with missing lines	Patch %	Lines
src/object_store/gcs.rs	86.84%	5 Missing ⚠️
src/arrow_parquet/uri_utils.rs	80.00%	1 Missing ⚠️
src/object_store/object_store_cache.rs	66.66%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main      #61      +/-   ##
==========================================
+ Coverage   91.12%   91.15%   +0.03%     
==========================================
  Files          87       88       +1     
  Lines       12479    12606     +127     
==========================================
+ Hits        11371    11491     +120     
- Misses       1108     1115       +7     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[ErvalhouS]

Looking forward to this 👍

[marcoslot]

is this the default location for that file? seems unlikely

[marcoslot]

I was able to authenticate via gcloud auth application-default login which creates ~/.config/gcloud/application_default_credentials.json, probably the former should be in the docs.

[marcoslot]

Probably worth mentioning gcloud auth application-default login