-
Notifications
You must be signed in to change notification settings - Fork 32
Supports field_ids during COPY TO #102
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
6428599 to
a857cb6
Compare
ad7f2e5 to
52a44e9
Compare
a857cb6 to
2c06724
Compare
f49dcac to
3ea9bf9
Compare
92971fe to
5b5725d
Compare
3ea9bf9 to
8c2a38d
Compare
5b5725d to
3613f84
Compare
8c2a38d to
fd3a837
Compare
3613f84 to
08c012a
Compare
fd3a837 to
14500af
Compare
08c012a to
a707629
Compare
14500af to
110f50c
Compare
a707629 to
8a4d5ec
Compare
110f50c to
b655643
Compare
8a4d5ec to
59715d6
Compare
b655643 to
ffe9c4e
Compare
marcoslot
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Seems to work well, though I'm wondering whether we should throw an error for column names that cannot be found to prevent typos.
| Explicit(FieldIdMapping), | ||
| } | ||
|
|
||
| impl FromStr for FieldIds { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
generally, it would be useful to add more comments to things as the code base is growing:
// implements parsing for the field_ids option in COPY .. TO statements
cd30884 to
915fb79
Compare
462c1a0 to
59da25c
Compare
915fb79 to
7e0e0db
Compare
Adds `field_ids` option, which lets you specify how to assign field ids during COPY TO.
Supported values for it:
- `none` (default) => no field ids are written into parquet metadata.
- `auto` => pg_parquet generates fields ids starting from 0.
- `<json string>` => pg_parquet will use the given field ids. e.g.
```sql
create table test_table(a int, b text[]);
copy test_table to '/tmp/test.parquet' with (field_ids '{"a": 1, "b": {"__root_field_id": 2, "element": 3}}');
```
Closes #106.
7e0e0db to
cfa4dec
Compare
Codecov ReportAttention: Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #102 +/- ##
==========================================
- Coverage 92.90% 91.12% -1.79%
==========================================
Files 86 87 +1
Lines 11650 12479 +829
==========================================
+ Hits 10823 11371 +548
- Misses 827 1108 +281 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
Adds
field_idsoption, which lets you specify how to assign field ids during COPY TO.Supported values for it:
none(default) => no field ids are written into parquet metadata.auto=> pg_parquet generates fields ids starting from 0.<json string>=> pg_parquet will use the given field ids. e.g.Closes #106.