Skip to content

Add contract-first Iceberg REST read-only support (Polaris-aligned codegen)#497

Closed
cbb330 wants to merge 3 commits intolinkedin:mainfrom
cbb330:chbush/iceberg-rest-contract-first-phase1-v2
Closed

Add contract-first Iceberg REST read-only support (Polaris-aligned codegen)#497
cbb330 wants to merge 3 commits intolinkedin:mainfrom
cbb330:chbush/iceberg-rest-contract-first-phase1-v2

Conversation

@cbb330
Copy link
Collaborator

@cbb330 cbb330 commented Mar 11, 2026

Summary

Supersedes #457.

Adds Phase 1 Iceberg REST Catalog support using contract-first codegen aligned with the Apache Polaris pattern. The server implements generated OpenAPI interfaces for read-only endpoints (config, list tables, load table, table exists) with comprehensive round-trip integration tests.

Changes

  • Client-facing API Changes
  • Internal API Changes
  • New Features
  • Tests

Client-facing API Changes

Adds read-only Iceberg REST Catalog endpoints via IcebergRestCatalogController:

  • GET /v1/config — returns prefix override (iceberg) for route isolation
  • GET /v1/iceberg/namespaces/{namespace}/tables — list tables
  • GET /v1/iceberg/namespaces/{namespace}/tables/{table} — load table metadata
  • HEAD /v1/iceberg/namespaces/{namespace}/tables/{table} — check table existence

All unimplemented endpoints (write, namespace, views, etc.) return 501 via generated defaults.

Internal API Changes

Codegen (Polaris-aligned)

  • Upstream Iceberg REST OpenAPI spec at spec/iceberg-rest-catalog-open-api.yaml
  • importMappings/typeMappings map spec schemas to real Iceberg types (no model generation)
  • Post-processing for Iceberg 1.10-only types mapped to Object (compatibility with our 1.5.2 fork)
  • Controller implements generated CatalogApiApi + ConfigurationApiApi interfaces

Serialization

  • IcebergRestHttpMessageConverter — custom AbstractHttpMessageConverter<RESTResponse> using Iceberg's RESTSerializers (kebab-case JSON) without interfering with existing Jackson converters
  • IcebergRestSerdeConfig — registers the converter via WebMvcConfigurer.extendMessageConverters

Error Handling

  • IcebergRestExceptionHandler — scoped @RestControllerAdvice returning Iceberg ErrorResponse format

Tests

  • IcebergRestCatalogRoundTripTest (21 tests) — full round-trip integration tests using a real Iceberg RESTCatalog client:
    • List tables: single namespace, correct namespace, cross-namespace isolation, empty namespace
    • Load table: schema columns, field IDs, required fields, different schemas, partition spec, unpartitioned, location, properties, sort order, snapshot, cross-namespace
    • Table exists: HEAD for existing, missing, wrong namespace
    • Error handling: load nonexistent table, load from nonexistent namespace
    • Consistency: multiple loads return same metadata
  • IcebergRestCatalogControllerTest (6 tests) — MockMvc unit tests for config, list, load, load not found, head exists, head not found

Testing Done

  • Added new tests for the changes made.
  • Updated existing tests to reflect the changes made.
  • Manually tested with PyIceberg and DuckDB.

All 27 tests pass locally:

IcebergRestCatalogRoundTripTest > 21 tests PASSED
IcebergRestCatalogControllerTest > 6 tests PASSED

Additional Information

  • Breaking Changes
  • Deprecations

TODO: When Iceberg is upgraded to 1.7+, extend CatalogTests<RESTCatalog> in round-trip tests (matching Polaris pattern). CatalogTests is not available in our current 1.5.2 fork.

Vibe Kanban and others added 3 commits March 4, 2026 11:56
- Replace hand-written OpenAPI spec with upstream Iceberg REST spec
  (iceberg-rest-catalog-open-api.yaml) following the Apache Polaris
  codegen pattern (importMappings/typeMappings, models=false)
- Controller now implements generated CatalogApiApi and
  ConfigurationApiApi interfaces; unimplemented endpoints return 501
- Add prefix routing via /v1/config so REST client uses
  /v1/iceberg/namespaces/... (separate from /v1/databases/...)
- Add IcebergRestHttpMessageConverter for Iceberg REST type
  serialization without interfering with existing Jackson converters
- Add comprehensive round-trip integration tests (21 tests) using
  a real RESTCatalog client: schema fidelity, partitions, metadata,
  list tables, table exists, error handling, consistency
- Update mock controller tests to use prefixed routes
@cbb330
Copy link
Collaborator Author

cbb330 commented Mar 11, 2026

Superseded by a stacked PR series for easier review:

  1. Iceberg REST Phase 1 [1/3]: OpenAPI spec and codegen infrastructure #498 — OpenAPI spec and codegen infrastructure (2 files)
  2. Iceberg REST Phase 1 [2/3]: Controller, serde, and exception handling #499 — Controller, serde, and exception handling (7 files)
  3. Iceberg REST Phase 1 [3/3]: Tests and CI #500 — Tests and CI (4 files)

Merge order: #498#499#500.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant