You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I think this will need to be implemented for different catalogs individually, because catalogs have different ways to persist metadata. Or we are mainly talking about adding traits to Catalog here?
I think this will need to be implemented for different catalogs individually, because catalogs have different ways to persist metadata. Or we are mainly talking about adding traits to Catalog here?
I'm thinking to put some common code like saving table metadata to a location, loading table metadata from a location to a common place.
I'd be happy to take a look at this. I've already drafted a version of it while working on MemoryCatalog::update_table, but would like to move it a shared place instead.
Whether or not we would want to add something like a MetastoreCatalog trait with default implementations is a different discussion, but I think such a change would build a good foundation for it.
I'd be happy to take a look at this. I've already drafted a version of it while working on MemoryCatalog::update_table, but would like to move it a shared place instead.
Whether or not we would want to add something like a MetastoreCatalog trait with default implementations is a different discussion, but I think such a change would build a good foundation for it.
I don't think we need another trait. Maybe we just need a TableMetadataWriter which could be created from Table would be enough.
Activity
CTTY commentedon Jun 3, 2025
I think this will need to be implemented for different catalogs individually, because catalogs have different ways to persist metadata. Or we are mainly talking about adding traits to
Catalog
here?liurenjie1024 commentedon Jun 4, 2025
I'm thinking to put some common code like saving table metadata to a location, loading table metadata from a location to a common place.
DerGut commentedon Jun 9, 2025
Other implementations use a metastore catalog that encapsulates some of this shared logic. For example, there is pyiceberg's
MetastoreCatalog._write_metadata
or Java'sBaseMetastoreTableOperations.writeNewMetadata
.I'd be happy to take a look at this. I've already drafted a version of it while working on
MemoryCatalog::update_table
, but would like to move it a shared place instead.Whether or not we would want to add something like a
MetastoreCatalog
trait with default implementations is a different discussion, but I think such a change would build a good foundation for it.feat(transaction): Add TransactionAction and related classes (#1420)
feat(transaction): Make Transaction own base_table (#1421)
liurenjie1024 commentedon Jun 12, 2025
I don't think we need another trait. Maybe we just need a
TableMetadataWriter
which could be created fromTable
would be enough.feat(transaction): Implement TransactionAction for updata_loc, update…
DerGut commentedon Jun 14, 2025
I agree! I was considering to start with a new module
iceberg::catalog::metastore
that contains functions likebut I like the idea of a metadata writer even better!
feat(transaction): Implement TransactionAction for ReplaceSortOrderAc…
read_from()
andwrite_to()
toTableMetadata
#1523refactor: Add `read_from()` and `write_to()` to `TableMetadata` (#1523)