You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Add a location generator that follows the table layout properties #3729
Is your feature request related to a problem or challenge?
Related: apache/datafusion-iceberg#122 for Datafusion respecting the object storage location generator property set on a table. That could be handled by iceberg-rust by centralising its existing code that handles that.
Writers currently have to choose a location generator themselves - the default one, or the object storage one. But that can leave the write.object-storage.enabled property ignored, unless writers handle that caseworking themselves.
Iceberg Java on the other hand, as well as some of the other Iceberg language implementations, provide utilities for doing this instead.
Describe the solution you'd like
There are some different approaches - we could change the DefaultLocationGenerator to have this property-based caseworking behaviour but that would break existing callers' behaviour (it would also diverge from the reference implementation where there's a similar separation between DefaultLocationProvider and the LocationProviders.locationsFor(...) factory.
An alternative here is a new location generator, constructed via something like TableLocationGenerator::new(&TableMetadata) (edit: the PR has moved to Table::location_generator() now) to select the default or object-storage layout from table properties.
@CTTY thanks for taking a look. I responded in #3730 (comment) - my understanding is that users would pass a factory to the catalog builder, like with_storage_factory, and Table::location_generator() would use it.
(The enum is an implementation detail of TableLocationGenerator, so it can gain a variant for custom generators or be replaced with something like an Arc<dyn ErasedLocationGenerator>)
Working with these location generators did make me wonder why they aren't trait objects (#3738) - in terms of design that is, I realise this would be be a break. I am curious if folks have thought about / considered this.
Activity
CTTY commentedon Nov 19, 2026
Thanks for picking this up!
making it an enum seems fine for now. But how do users inject a custom location provider?
smaheshwar-pltr commentedon Nov 20, 2026
@CTTY thanks for taking a look. I responded in #3730 (comment) - my understanding is that users would pass a factory to the catalog builder, like
with_storage_factory, andTable::location_generator()would use it.(The enum is an implementation detail of
TableLocationGenerator, so it can gain a variant for custom generators or be replaced with something like anArc<dyn ErasedLocationGenerator>)smaheshwar-pltr commentedon Nov 20, 2026
Working with these location generators did make me wonder why they aren't trait objects (#3738) - in terms of design that is, I realise this would be be a break. I am curious if folks have thought about / considered this.