Skip to content

Commit 828112b

Browse files
[Bugfix] Populate table name from the identifier in Iceberge conversion
What is the problem? While converting the iceberg table to other format such as Hudi, the icerberg source table do not populate the table name. This is due to iceberg table's behavior as it is treated as Hadoop tables. This leads to table identified as table-location, leading to confusing conversation. Solution: This commit handles the conversation logic, when icebege table manager provides HadoopTable, it populate the table name from provided input TableIdentifier. This ensures that source table name is carried over to the transformation. Testing: - Added unit test to cover this scenario. Co-authored-by: Tim Brown <[email protected]>
1 parent 14967dd commit 828112b

File tree

2 files changed

+10
-2
lines changed

2 files changed

+10
-2
lines changed

xtable-core/src/main/java/org/apache/xtable/iceberg/IcebergConversionSource.java

Lines changed: 9 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -120,10 +120,18 @@ public InternalTable getTable(Snapshot snapshot) {
120120
irPartitionFields.size() > 0
121121
? DataLayoutStrategy.HIVE_STYLE_PARTITION
122122
: DataLayoutStrategy.FLAT;
123+
// When the table name is not explicitly specified, Iceberg assumes the table is HDFS-based,
124+
// treating the table name as the location in HDFS. This assumption can lead to mismatches
125+
// during metadata conversion. To mitigate this issue, we rely on the table name provided in the
126+
// source configuration of the conversation so target matches the user's expectations.
127+
// See https://github.com/apache/incubator-xtable/issues/494
123128
return InternalTable.builder()
124129
.tableFormat(TableFormat.ICEBERG)
125130
.basePath(iceTable.location())
126-
.name(iceTable.name())
131+
.name(
132+
iceTable.name().contains(iceTable.location())
133+
? sourceTableConfig.getName()
134+
: iceTable.name())
127135
.partitioningFields(irPartitionFields)
128136
.latestCommitTime(Instant.ofEpochMilli(snapshot.timestampMillis()))
129137
.readSchema(irSchema)

xtable-core/src/test/java/org/apache/xtable/iceberg/ITIcebergConversionSource.java

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -126,7 +126,7 @@ void getCurrentTableTest() {
126126
.build();
127127
validateTable(
128128
internalTable,
129-
testIcebergTable.getBasePath(),
129+
testIcebergTable.getTableName(),
130130
TableFormat.ICEBERG,
131131
internalSchema,
132132
DataLayoutStrategy.FLAT,

0 commit comments

Comments
 (0)