Skip to content

Fixed Size List issue? #116

@tingold

Description

@tingold

Trying to use pg_parquet to import a field defined as a float[] and getting the following error:

[XX000] ERROR: type mismatch for column "embedding" between table and parquet file.
table has "List(Field { name: "element", data_type: Float32, nullable: true, dict_id: 0, dict_is_ordered: false, metadata: {} })"
parquet file has "FixedSizeList(Field { name: "element", data_type: Float32, nullable: true, dict_id: 0, dict_is_ordered: false, metadata: {} }, 384)"

The table and copy statement are this:

create table parquet_test (id bigint, embedding float4[384], geometry geometry(POINT,4326));
copy parquet_test from 'https://f004.backblazeb2.com/file/sentinel2-yearly-embeddings/01GDM_2024-01-01_2025-01-01.parquet' with (format parquet);
  • URL is public and I built off of the aykut/https PR
  • File was written with pyarrow
  • All of the metadata functions seem to work fine

Any ideas would be appreciated, I'm guessing FixedSizeList might be the issue

Thanks!

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions