Skip to content

Add a location generator that follows the table layout properties #3729

Open

Description

@smaheshwar-pltr

Is your feature request related to a problem or challenge?

Related: apache/datafusion-iceberg#122 for Datafusion respecting the object storage location generator property set on a table. That could be handled by iceberg-rust by centralising its existing code that handles that.

Writers currently have to choose a location generator themselves - the default one, or the object storage one. But that can leave the write.object-storage.enabled property ignored, unless writers handle that caseworking themselves.

Iceberg Java on the other hand, as well as some of the other Iceberg language implementations, provide utilities for doing this instead.

Describe the solution you'd like

There are some different approaches - we could change the DefaultLocationGenerator to have this property-based caseworking behaviour but that would break existing callers' behaviour (it would also diverge from the reference implementation where there's a similar separation between DefaultLocationProvider and the LocationProviders.locationsFor(...) factory.

An alternative here is a new location generator, constructed via something like TableLocationGenerator::new(&TableMetadata) (edit: the PR has moved to Table::location_generator() now) to select the default or object-storage layout from table properties.

Willingness to contribute

I can contribute to this feature independently

Activity

  1. CTTY commented on Nov 19, 2026

    @CTTY
    Collaborator

    Thanks for picking this up!

    making it an enum seems fine for now. But how do users inject a custom location provider?

  2. smaheshwar-pltr commented on Nov 20, 2026

    @smaheshwar-pltr
    ContributorAuthor

    @CTTY thanks for taking a look. I responded in #3730 (comment) - my understanding is that users would pass a factory to the catalog builder, like with_storage_factory, and Table::location_generator() would use it.

    (The enum is an implementation detail of TableLocationGenerator, so it can gain a variant for custom generators or be replaced with something like an Arc<dyn ErasedLocationGenerator>)

  3. smaheshwar-pltr commented on Nov 20, 2026

    @smaheshwar-pltr
    ContributorAuthor

    Working with these location generators did make me wonder why they aren't trait objects (#3738) - in terms of design that is, I realise this would be be a break. I am curious if folks have thought about / considered this.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions