0.3.0-incubating

Release Notes for release-0.3.0-incubating

Apache XTable™ (Incubating) is a cross-table converter for table formats that facilitates omni-directional interoperability across data processing systems and query engines. Currently, Apache XTable™ supports widely adopted open-source table formats such as Apache Hudi, Apache Iceberg, and Delta Lake.

✨ What's Changed

🔄 Core Functionality Enhancements

• Catalog Sync:

  • Introduced CatalogSyncClient and CatalogSync interfaces.
  • Implemented Glue and HMS catalog sync for Iceberg, Delta, and Hudi.
  • Additional information can be found in docs.

• Table Format Sync Improvements:

  • Added support for continuous sync using RunSync.
  • Support restore/rollback sync during conversion across all three formats.
  • Added support for bucket partition transform (Iceberg → Delta).
  • Synced schema field comments and record counts in Delta source.
  • Improved decimal field scale handling in iceberg and fixed bug in column stats.
  • Supporting timestamp_ntz type in conversion logic.

• Breaking changes

  • We have refactored SyncStatusCode and ErrorDetails into dedicated classes, refactor code by fixing the import path if you are using these classes.

🧪 Testing & Quality

• Fixed flaky tests and normalized base paths for test stability.
• Addressed minor issues found via static analysis tooling.
• Dockerfile syntax fix and test class name corrections.

📘 Documentation & Community

• Added downloads page for 0.2.0-incubating release.
• Introduced RFC template and new change proposals.
• Updated community sync and blog pages.

🧰 Infrastructure & Maintenance

• Upgraded parquet-avro to 1.15.1 to fix critical vulnerabilities found recently.
• Upgraded Hadoop and AWS SDK dependencies.
• Refactored RunSync for better modularity and reusability.

Detailed Change Log

• Iceberg and Delta target: sync schema field comments by @emilie-wang in #575
• [551] Add downloads page for 0.2.0-incubating release by @vinishjail97 in #576
• [551] Try enabling keep_files as false in GH action for downloads by @vinishjail97 in #577
• [DOCS] Remove info on snowflake public preview for Iceberg by @sagarlakshmipathy in #580
• Add 2 New Community Blogs to the Apache XTable Blogs section by @dipankarmazumdar in #585
• Site content is not deployed due to missing .asf.yaml file by @zabetak in #587
• Execute correct delete operation in delta tests by @ashvina in #589
• Fix commit log parsing of Delta tables with delete vector by @ashvina in #596
• Normalize basePath in targetTables in ConversionController by @vinishjail97 in #602
• [590] Add RFC template for XTable by @vinishjail97 in #604
• [590] Add interfaces for CatalogSyncClient and CatalogSync by @vinishjail97 in #603
• [590] Add RFC template for XTable by @vinishjail97 in #610
• [590] Hadoop and AWS sdk version upgrade by @kroushan-nit in #614
• [619] Maintenance: Update dependencies, fix test class names by @the-other-tim-brown in #620
• [608] Ensure column stats for decimal fields have proper scale set by @the-other-tim-brown in #617
• [615] Add Iceberg stats when reading snapshot by @the-other-tim-brown in #616
• Add community sync page for XTable by @vinishjail97 in #625
• Update GMeet link and add today's recording by @vinishjail97 in #628
• [590] Add RFC for XCatalogSync - Synchronize tables across catalogs by @vinishjail97 in #605
• [590] Add RunCatalogSync utility for synchronizing tables across catalogs by @vinishjail97 in #591
• Delta source: add record count for DataFile by @emilie-wang in #623
• [site] Update blogs by @bhasudha in #635
• [590] Add Iceberg Glue Catalog Sync implementation by @kroushan-nit in #632
• [590] Add Iceberg HMS Catalog Sync implementation by @kroushan-nit in #633
• [590] Add Delta Glue Catalog Sync implementation by @kroushan-nit in #637
• [590] Add Delta HMS Catalog Sync implementation by @kroushan-nit in #638
• [590] Add Hudi HMS Catalog Sync implementation by @vamsikarnika in #639
• Update xtable-api sources by @emmanuel-ferdman in #651
• [Enhancement] Support restore/rollback sync during conversion (1/2) by @danielhumanmod in #569
• [590] Add Hudi HMS Catalog Sync Implementation by @vamsikarnika in #648
• [590] Add Hudi Glue Catalog Sync Implementation by @vamsikarnika in #649
• Add change proposal for conversion of deletion vectors by @ashvina in #634
• Add base representation for storage files by @ashvina in #653
• Fix flaky test, ignore order of elements in list validation by @ashvina in #662
• [594] Support continuous conversion with the RunSync tool by @the-other-tim-brown in #593
• [MINOR] fixes recommended from static analysis tooling by @the-other-tim-brown in #664
• Modified incremental safe check which does not depend on snapshots list order by @brishi19791 in #665
• [668] Support bucket partition transform for Iceberg Sources and Delta Targets by @the-other-tim-brown in #670
• Handle timestamp_ntz in delta and iceberg by @vinishjail97 in #647
• [MINOR] Fix docker file syntax issue by @the-other-tim-brown in #674
• [528] Refactor RunSync Class for Improved Modularity and Reusability by @vaibhavk1992 in #675
• Add how-to docs for catalog sync by @vinishjail97 in #673
• [MINOR] Add mailing list for discussions, update issues list in asf.yaml by @the-other-tim-brown in #676
• [MINOR] Fix mailing list in asf.yaml by @the-other-tim-brown in #677
• [678] Add contribution guidelines to reflect stale PR's guidelines by @vinishjail97 in #684
• [678] Avoid skipping xtable-aws and xtable-hive-metastore in mvn deploy by @vinishjail97 in #679
• [678] Introduce maturity-level in xtable-api and Refactor SyncStatusCode into separate class by @vinishjail97 in #685
• Bump org.apache.parquet:parquet-avro from 1.12.3 to 1.15.1 by @dependabot in #686
• Enable squash and merge in .asf.yaml for xtable by @vinishjail97 in #688
• [678] Add LICENSE for xtable-aws module by @vinishjail97 in #690
• [678] Add LICENSE for xtable-hive-metastore module by @vinishjail97 in #691
• Docs: Update Footer Copyright Year by @omkenge in #654
• [692] Include 2025 in NOTICE file by @vinishjail97 in #695
• [692] Exclude jol-core in xtable bundled dependencies by @vinishjail97 in #696
• Remove parameters from Glue partition columns by @kroushan-nit in #697
• [692] Ensure LICENSE content is different for bundled jars by @vinishjail97 in #699
• [DOCS]: Add 3 New Community Blogs by @dipankarmazumdar in #706
• Add open api REST spec for XTable Service by @rahil-c in #705
• [BLOG]: Add the new Catalog sync blog by @dipankarmazumdar in #708
• [700] Update license files for bundled jars by @vinishjail97 in #707
• Add latest metadata path field in InternalTable by @rahil-c in #710
• [700] Update NOTICE file for bundled jars by @vinishjail97 in #709
• Add implementation for XTable REST Service by...

0.2.0-incubating

This is the second official apache release for Apache XTable, an incubating project under the Apache Software Foundation. Apache XTable™ (Incubating) is a cross-table converter for table formats that facilitates omni-directional interoperability across data processing systems and query engines. Currently, Apache XTable™ supports widely adopted open-source table formats such as Apache Hudi, Apache Iceberg, and Delta Lake.

Quick Start Guide

https://xtable.apache.org/docs/how-to

Features

Apache XTable™ (Incubating) provides users with the ability to translate metadata from one table format to another.

Apache XTable™ (Incubating) provides two sync modes, "incremental" and "full." The incremental mode is more lightweight and has better performance, especially on large tables. If there is anything that prevents the incremental mode from working properly, the tool will fall back to the full sync mode.

This sync provides users with the following:

1. Syncing of data files along with their column level statistics and partition metadata
2. Schema updates in the source are reflected in the target table metadata
3. Metadata maintenance for the target table formats.
   • For Hudi, unreferenced files will be marked as cleaned to control the size of the metadata table.
   • For Iceberg, snapshots will be expired after a configured amount of time.
   • For Delta, the transaction log will be retained for a configured amount of time.

Summarized Change Log

1. Updated LICENSE and NOTICE file contents based on guidance given by LEGAL-684.
2. We have addressed feedback on ASF best practices by IPMC member @pjfanning, all these issues were marked as release blockers and have been resolved.
3. Fix the edge case when handling non numeric values of double type in delta stats.
4. Add a UUID Type.
5. Improved downloads page and other refactoring efforts.
6. We have excluded xtable-utilities from the release process because it's a bundled jar containing non ASF compliant dependencies, we will include it back as part of 0.3.0 release. You won't find this module in the staged artifacts or in the source releases distributions for 0.2.0.
7. Upgrade dependency versions based on CVE-2024-47561

Source Distributions

• Source Release: Apache XTable 0.2.0-incubating Source Release (asc, sha512)

Maven Artifacts

• Maven Search: Apache XTable 0.2.0-incubating

What's Changed

• [297] Refactor config classes by @the-other-tim-brown in #480
• [MINOR] Moving to 0.2.0-SNAPSHOT on main branch. by @vinishjail97 in #515
• [DOCS] add releases page and modify README.md by @sagarlakshmipathy in #520
• [DOCS] added Downloads link to the site homepage by @sagarlakshmipathy in #522
• Update Downloads page to adhere to the ASF policy by @zabetak in #527
• Remove downloads page from website by @vinishjail97 in #534
• Set keep_files as false in GH action for site deploy to remove lingering pages by @vinishjail97 in #535
• [531] xtable poms are missing license details. by @rangareddy in #533
• [530] xtable-core pom has a hardcoded scala version for jackson-module-scala by @rangareddy in #537
• [536] Exclude xtable-utilities from mvn deploy by @vinishjail97 in #540
• [532] Add DISCLAIMER file to META-INF folder in jars by @rangareddy in #539
• [536] Update LICENSE and NOTICE files by @vinishjail97 in #541
• update import statements match Delta version upgrade in PR 349 by @zhen-d in #542
• Fix the edge case when handling non numeric values of double type in delta stats by @emilie-wang in #526
• [112] Add a UUID Type by @danielhumanmod in #518
• [546] Update Lombok Dependency Scope in Project by @vamsikarnika in #547
• [529] Build xtable with scala version(s) by @rangareddy in #544
• [551] Exclude xtable-utilities from release guide by @vinishjail97 in #552
• use avro 1.11.4 due to CVE-2024-47561 by @pjfanning in #559
• [561] Add timeout to Github Action by @the-other-tim-brown in #562
• [560] Increment Protobuf version due to vulnerability by @the-other-tim-brown in #557
• [564] mirror parent dependency scopes in child pom files by @the-other-tim-brown in #563
• [551] Remove dependencies in LICENSE and NOTICE files based on feedback in LEGAL-684 by @vinishjail97 in #566
• Update xtable-hadoop-defaults.xml reference by @emmanuel-ferdman in #568
• [551] Create separate LICENSE and NOTICE files for xtable-hudi-support-extensions by @vinishjail97 in #572

New Contributors

• @rangareddy made their first contribution in #533
• @zhen-d made their first contribution in #542
• @danielhumanmod made their first contribution in #518
• @vamsikarnika made their first contribution in #547
• @pjfanning made their first contribution in #559
• @emmanuel-ferdman made their first contribution in #568

Full Changelog: 0.1.0-incubating...0.2.0-incubating

v0.1.0-beta1

First beta release for the OneTable repo. All APIs should be treated as experimental and evolving at this stage. Artifacts are meant for testing and early feedback.