Skip to content

[590] Add Iceberg HMS Catalog Sync implementation #633

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 1 commit into from
Feb 4, 2025
Merged

[590] Add Iceberg HMS Catalog Sync implementation #633

merged 1 commit into from
Feb 4, 2025

Conversation

kroushan-nit
Copy link
Contributor

@kroushan-nit kroushan-nit commented Jan 30, 2025 •
edited
Loading

#590

Important Read

  • Please ensure the GitHub issue is mentioned at the beginning of the PR

What is the purpose of the pull request

Added support for syncing a source table to HMS catalog in Iceberg table format

Brief change log

  • Created a new module xtable-hive-metastore
  • Added HMSCatalogSyncClient implementation for syncing InternalTable to HMS catalog in Iceberg table format
  • Added HMSCatalogConversionSource implementation for creating SourceTable from HMS table identifier
  • Added HMSSchemaExtractor for generating Hive compatible schema from InternalSchema

Verify this pull request

  • Added unit tests
  • Manually verified the change by running a job locally

@kroushan-nit kroushan-nit force-pushed the 590-IcebergHMSSync-v2 branch from c5726ad to 4b9b455 Compare February 4, 2025 18:28
@kroushan-nit kroushan-nit force-pushed the 590-IcebergHMSSync-v2 branch from 4b9b455 to be622d9 Compare February 4, 2025 18:36
@the-other-tim-brown the-other-tim-brown merged commit 30f3e54 into apache:main Feb 4, 2025
2 checks passed
@vinishjail97 vinishjail97 mentioned this pull request Apr 7, 2025
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@the-other-tim-brown the-other-tim-brown the-other-tim-brown approved these changes

@vinishjail97 vinishjail97 vinishjail97 approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@kroushan-nit @the-other-tim-brown @vinishjail97