Add an Iceberg REST Catalog front door - #14
Merged
Conversation
--scenario rest drives the listener with curl: config discovery (the advertised prefix is reused for every later path), namespace and table listings, a table load asserting metadata-location comes back, and the negative shapes - an unknown prefix, an unknown table and a write route all have to fail cleanly. Plain Hive tables of the same database must stay invisible through REST, which pins down the HiveCatalog filter. The step joins --scenario all and is skipped when HMS_SMOKE_REST_URL is not set. The stand enables the listener on its plain profile (host port 19183) and registers a minimal Iceberg table for the load check: a hand-written format-version-2 metadata.json in HDFS plus a Hive table shell carrying table_type=ICEBERG and metadata_location. A passing load therefore proves the whole chain - REST route, HiveCatalog, the proxy's routing layer, HMS, HDFS. The Kerberos profile keeps the listener off: SPNEGO would need a GSS-enabled curl in-network and is already covered by SpnegoIntegrationTest. Recorded as section G in TEST-MATRIX; both README/CHANGELOG locales updated.
RoutingMetaStoreClient.create gains an overload that accepts a CatalogNameTranslation. When present, db-name arguments are translated to the proxy's external federated form before delegating, and Database/Table results coming back are rewritten to the catalog's internal names. The untranslated single-arg create keeps delegating with a null translation, so existing behavior is unchanged.
Adds IcebergRestServices, an eager prefix -> IcebergRestService registry built from ProxyConfig: the default catalog keeps the untranslated view, every other catalog gets a CatalogNameTranslation. Backs multi-catalog prefix resolution for the REST endpoint (byWarehouse/serviceFor) ahead of wiring it into HTTP routing.
Add an optional second-prefix block to run_rest_smoke, driven by HMS_SMOKE_REST_SECOND_PREFIX (+ HMS_SMOKE_REST_SECOND_ICEBERG_TABLE): warehouse discovery advertises the chosen prefix, an unknown warehouse gets a clean 400, the clean view under the second prefix lists the namespace without leaking catalog__ external names, and the second-prefix table load returns metadata-location. Wire the stand's simple.env to the apache catalog and its second Iceberg table (smoke_iceberg_tbl_ap, registered on the second HDFS cluster namenode-b) and document the registration in both stand README locales. Extend TEST-MATRIX (EN+RU) section G with rows G8-G11 and note the same-day revalidation.
- IcebergRestService: rewrite class Javadoc to describe the actual per-prefix service model instead of the stale MVP/single-catalog note. - IcebergRestServices#close: attempt close() on every service instead of stopping at the first failure; collect the first IOException, add later ones as suppressed, and rethrow. - TEST-MATRIX.ru.md: keep "prefix" in Latin script in the G10 row instead of the Cyrillic "префиксом". - AGENTS.md: document the restcatalog module in the architecture notes.
The second-prefix block now checks the federation side too: the <catalog><separator><db> name stays visible under the default prefix, and the second catalog's Iceberg table is listed and loaded through that federated name. The clean-view side gains the cross-catalog negatives - a default-catalog table and the external namespace name both 404 under the second prefix - and the second catalog's plain Hive table must stay out of its listing (HMS_SMOKE_REST_SECOND_NON_ICEBERG_TABLE). The catalog-db separator is no longer hardwired into the leak grep (HMS_SMOKE_REST_SEPARATOR, default __). Recorded as rows G12-G16 in TEST-MATRIX, both locales.
The REST smoke now optionally fetches the management /metrics endpoint (HMS_SMOKE_REST_METRICS_URL) and checks it carries the hms_proxy_rest_requests_total and hms_proxy_rest_listener_info series, proving the Iceberg REST listener's own Prometheus metrics actually show up where an operator would look for them. Documented in both READMEs (new table rows plus a one-line note in the Iceberg REST section), the changelog, and the stand's test matrix (row G17 plus a revalidation-log note).
Rate/error-ratio/latency stats, a listener-up stat driven by hms_proxy_rest_listener_info, latency quantiles, and breakdowns by HTTP status, catalog prefix and route (pseudo-routes mark refusals). Panels are cloned from the dashboard's existing reference panels; series names were checked against a live /metrics scrape, including the histogram _bucket series the quantile panels rely on.
Bump iceberg.version to 1.9.2, pin Jackson core/databind to 2.18.3 to match what Iceberg 1.9.2 is compiled against (Hive 3.1.3 otherwise drags in databind 2.12), and exclude the newly-transitive slf4j-api from the three Iceberg dependencies so the tree keeps a single provider. Re-vendor RESTCatalogAdapter from the matching apache-iceberg-1.9.2 tag. Route stays nested and gets the same public-promotion patch as before; HTTPMethod moved out into HTTPRequest.HTTPMethod, and the adapter's public entry point is now handleRequest(...) instead of the removed execute(...) overload. Migrate IcebergRestService.dispatch and IcebergHttpHandler's route dispatch to the new signature: the handler now merges query params and route path vars itself, calls dispatch(route, vars, body, type) directly, and maps thrown RuntimeExceptions to error responses via RESTCatalogAdapter.configureResponseFromException instead of the old error-handler callback. Request outcome/metrics bookkeeping is unchanged. Include the phase 4a design spec and plan alongside the code change.
Update the README caveats/endpoint table (both locales) for the REST front door's move to Iceberg 1.9.2: the RoutingHiveCatalog reflection pin now names 1.9.2, and view GET routes are documented as supported (real data, empty listing) instead of unsupported, since HiveCatalog became a ViewCatalog. Add a Changed subsection under CHANGELOG 2026-07-27 (both locales) covering the version bump, the Jackson 2.18.3 pin, the re-vendored RESTCatalogAdapter's handleRequest migration, the view-route behavior delta, and client compatibility. Record in smoke-stand/TEST-MATRIX (both locales) that sections A-D and G were revalidated green on the upgraded jar, including the SQL layer.
The EXISTS routes went live with the 1.9.2 upgrade because IcebergHttpHandler forwards any resolved route with no allowlist; CHANGELOG.md/.ru.md wrongly said they were not yet wired in. Rewrite both to state HEAD now answers 204/404 per the REST spec across every prefix, and note the user-visible fix: HEAD on an existing table used to 404 under 1.5.2, so PyIceberg's table_exists() misreported existing tables as absent. Also fix the pleonasm "пересобран заново" in CHANGELOG.ru.md, and add a G18 row to both TEST-MATRIX files recording the exists-route checks.
The phase-4a Changed bullet claimed HEAD on an existing view returns 204, but no view exists on the smoke stand, so that case was never observed. Scope the 204/404 claim to namespaces and tables (matching matrix row G18) and note that VIEW_EXISTS is served by the same unconditional dispatch and was only exercised for the 404 (missing view) case in both locales.
…pper CatalogHandlers.paginate() in Iceberg 1.9.2 parses pageToken with Integer.parseInt and throws NumberFormatException when it is null, which is exactly what a first-page request (pageSize without a prior token) sends. IcebergHttpHandler now defaults pageToken to an empty string when pageSize is present and pageToken is absent, without disturbing the existing path-vars-win-over-query-params merge order. Also: - README.md/README.ru.md: add the three HEAD exists routes (namespace, table, view) to the Supported endpoints table; they are already live and covered by CHANGELOG and TEST-MATRIX row G18. - CHANGELOG.md/CHANGELOG.ru.md: note that listings now honor pageSize/pageToken and return next-page-token, which Iceberg 1.5.2 did not support at all. - IcebergHttpHandler: remove the unreachable statusForException helper and its RESTException import, dead since the error-mapping rewrite to RESTCatalogAdapter.configureResponseFromException.
The Iceberg REST listener may be unauthenticated, so error bodies must not include internal package/file/line details. Blank the stack after RESTCatalogAdapter maps the exception, and give readBody's parse failures a targeted 400 instead of falling through to the 500 catch-all.
Iceberg 1.9.2's ConfigResponse gained an endpoints list; a silent server
leaves modern clients assuming the full write-capable endpoint set. List
only the nine routes this read-only front door actually serves, and route
/v1/{prefix}/config through the same config writer used by
/v1/config?warehouse=<catalog> instead of falling through to the vendored
adapter's own CONFIG case, which advertised writes and ignored the prefix.
Add a HEAD helper and five tests for namespace-exists and table-exists (existing/missing namespace, existing/missing table, second-prefix table). Seed "t1" and "events" with a real metadata_location backed by an actual local metadata.json, since HiveCatalog's default tableExists does a full loadTable and the existing "orders"/"shipments" fixtures are deliberately non-loadable placeholders for the non-Iceberg case.
RFC 9110 forbids a body on HEAD responses. writeJson/writeError/ writeErrorResponse always wrote one, so the JDK HttpServer rejected the write with IOException: stream closed for every HEAD error (e.g. exists probes on missing namespaces/tables), which escaped into doHandle's catch-all as a WARN with a full stack trace on every occurrence. Route all three writers through a shared sendBody helper that, for HEAD, sends only the status with content-length -1 and closes the body unwritten, mirroring the existing 204 exists-response.
response.body() is always "" for a HEAD response regardless of what the server sent (java.net.http.HttpClient discards it per RFC 9110), so it proved nothing about the writeErrorResponse HEAD fix. Empirically, the JDK HttpServer also never sets a Content-Length header for HEAD responses, so that alternative doesn't discriminate either - reverting the production fix left both assertions green. The one thing that does differ is that the reverted code's failed body write throws, which the handler's catch-all logs as "Unhandled error serving ..." at WARN. Assert on that instead by attaching a log4j appender to the handler's logger for the duration of the request and requiring it to have logged nothing.
RFC 9110 forbids a body on HEAD responses. respond() in ManagementHttpServer always wrote one, so the JDK HttpServer rejected the write with IOException: stream closed on every HEAD request to /healthz, /metrics and /readyz - a shape health checkers commonly use. This server has no catch-all logger, so the failure was discarded silently. Route the write through a sendBody helper that, for HEAD, sends only the status with content-length -1 and closes the body unwritten, mirroring the equivalent fix just applied to IcebergHttpHandler.
Add three run_rest_smoke assertions for the phase-4b hardening: no server
stack trace in error responses, 400 on an unparseable request body, and
GET /v1/config advertising the served endpoints. Document all three plus
the prefixed /v1/{prefix}/config form and the no-body HEAD fix in
README/CHANGELOG (EN+RU), and record the stand rerun in TEST-MATRIX
(EN+RU) with rows G19-G21.
Extract the duplicated HEAD-safe response writer from ManagementHttpServer and IcebergHttpHandler into util.HttpResponseWriter, and unit-test it directly against a recording HttpExchange. The prior ManagementHttpServerTest only compared HEAD vs GET status codes, which stay identical whether or not the "stream closed" guard exists, so it could not fail on a regression; HttpResponseWriterTest asserts the actual (status, contentLength) pair and body bytes and does fail without the guard. Strengthen run_rest_smoke's /v1/config assertion: instead of only checking that an "endpoints" key exists, assert a concrete read route is advertised and that no write route (POST .../namespaces, DELETE) is, against both the unprefixed and prefixed config endpoints. Update TEST-MATRIX row G21 and its revalidation log (EN/RU) to match what the runner actually checks.
Both config branches in IcebergHttpHandler.doHandle dispatched on path
only, never on method, so POST/DELETE/HEAD on /v1/config and
/v1/{prefix}/config answered 200 instead of 404. For the prefixed form
this was a phase-4b regression: before this phase such requests fell
through to Route.from(method, path), which is GET-only for CONFIG and
returned null, producing the unknown-route 404.
Require GET on both forms; any other method now falls through to the
same unknown-route 404 as before. The unprefixed form's fix is a
deliberate correction of pre-existing behavior (POST /v1/config already
answered 200 prior to this phase), not part of the original phase scope.
RoutingMetaStoreClient now answers createTable, dropTable, alter_table_with_environmentContext, lock, checkLock, unlock, showLocks and heartbeat instead of throwing UnsupportedOperationException, so Iceberg's HiveCatalog write path and MetastoreLock can operate through the proxy's in-process IMetaStoreClient bridge. Table dbName arguments are translated on a copy for non-default catalogs, matching the existing rewriteTable convention; LockRequest is passed through untranslated since the proxy's own LockHandler resolves its database names downstream. This is client plumbing only: no gate restricts writes to the default catalog yet, that follows in a later task.
…, coverage alter_table_with_environmentContext's translation test could not fail: the fixture recorded only the two String db/tbl arguments and never the Table argument's own dbName, so deleting the translateDbName(args[2]) call left the test green. Record the Table's dbName in the fixture and assert on it. Drop the unrequested showLocks() no-arg overload; Iceberg only calls the 1-arg showLocks(ShowLocksRequest), so it now falls through to the same UnsupportedOperationException as every other unused overload. Add missing coverage for checkLock, showLocks(ShowLocksRequest) and heartbeat, and align the lock branch with its siblings by adding the paramTypes.length guard it was missing.
Iceberg commits need a real HMS lock, and only the default catalog has one: every other catalog is served by the synthetic shim, which grants locks without conflict checking, so a commit there would take an EXCLUSIVE lock that is always granted, believe it owns the table, and race a concurrent writer into lost updates. The gate keys on the catalog a namespace resolves to, never on the URL prefix. The default prefix exposes other catalogs' databases as <catalog><separator><db>, so a write to hdp/namespaces/apache__default carries the default prefix but routes into the apache catalog; checking the prefix would wave it through. Resolution reuses the proxy's own CatalogRouter rather than a second name parser. RENAME_TABLE names no namespace in its path, so both the source and the destination in its body are checked. REPORT_METRICS is a POST but not a catalog write and is deliberately outside the write-route set.
WriteRouteGate only checked CREATE_TABLE/UPDATE_TABLE/DROP_TABLE/ RENAME_TABLE/REGISTER_TABLE, but IcebergHttpHandler dispatches on the full RESTCatalogAdapter.Route enum with no allowlist, so every other write route reached dispatch() unguarded. Worst case: COMMIT_TRANSACTION, the standard multi-table atomic-commit endpoint, let a client bypass the gate entirely by bundling a federated-catalog table into an otherwise legitimate commit. View routes are gated too, since HiveCatalog now extends BaseMetastoreViewCatalog and takes the same lock path. Extend the gate to the remaining write routes across their three body shapes: path-shaped (CREATE_VIEW, UPDATE_VIEW, DROP_VIEW, DROP_NAMESPACE, UPDATE_NAMESPACE - namespace from vars, same as the existing table routes), rename-shaped (RENAME_VIEW - source and destination in the body, both checked, same as RENAME_TABLE), and body-shaped (CREATE_NAMESPACE's own namespace; COMMIT_TRANSACTION's tableChanges(), where every identifier must resolve to the default catalog or the whole request is refused). Confirmed the hole with a COMMIT_TRANSACTION request mixing one default-catalog table with one federated table: against the unpatched gate it returned 404 (dispatched unguarded, failed for an unrelated reason) instead of 403. It now refuses with 403 ForbiddenException.
hive-standalone-metastore pulls hadoop-hdfs 2.2.0 transitively via orc-core, while hadoop-common resolves to 2.6.0 elsewhere in the tree. 2.2.0's DFSOutputStream calls the Hadoop-2.2-era FSOutputSummer(Checksum,int,int) constructor, which 2.6.0's FSOutputSummer no longer has, so every HDFS write from this JVM failed with NoSuchMethodError. Exclude the transitive hadoop-hdfs and pin it explicitly at 2.6.0 to match hadoop-common. Independently, IcebergHttpHandler.doHandle only caught Exception, so an Error such as this NoSuchMethodError escaped the handler entirely and the JDK HTTP server abandoned the exchange with no response, hanging the client instead of returning a 5xx. Widen the catch-all to Throwable so no failure mode leaves a request unanswered, and add a test that drives an Error through the real dispatch path (via a proxy-based test fake, which passes Errors through unwrapped) and asserts a 500 comes back rather than a hang.
Add HMS_SMOKE_REST_WRITE_TABLE-guarded coverage to run_rest_smoke: create,
load (asserting metadata-location) and drop a table on the default catalog,
then two negatives - a direct create under a non-default prefix and a
create under that prefix's federated namespace name reached through the
default prefix - both asserting 403. Also fix the pre-existing config
assertions, which still expected a read-only listing: the default catalog's
GET /v1/config and GET /v1/{prefix}/config now must advertise the five
table-write routes, and a new check on the non-default prefix's own config
proves the write/read asymmetry is actually observable via discovery.
Document, in both locales, that table writes are default-catalog only and
why (every other catalog is served by the synthetic lock shim, which grants
locks without conflict checking), that the gate is enforced on the resolved
catalog so a federated name cannot bypass it, and that GET /v1/config
advertises the asymmetry. Record the two fixes phase 5a needed to make
writes work at all: the hadoop-hdfs/hadoop-common version alignment (writes
are the first code path opening an HDFS output stream from the proxy's own
JVM) and the IcebergHttpHandler catch-all widened from Exception to
Throwable, so a JVM Error can no longer hang a client's connection forever.
Verified on the local stand: --scenario rest and --scenario all both
completed successfully, and the SQL layer (both HiveServer2 instances)
passed as the regression check for the Hadoop dependency change, since
table writes and Hive's own ACID commits share the same lock path.
hadoop-hdfs:2.6.0 pulls in xerces:xercesImpl/xml-apis for its offline image/edits viewer only, never for the DFSClient runtime path this proxy uses. xercesImpl ships META-INF/services/javax.xml.parsers.* entries, which in the shaded fat jar would make a 2007-vintage parser with known CVEs the JVM-wide JAXP provider for every DocumentBuilderFactory/SAXParserFactory lookup in the process. Exclude both from the hadoop-hdfs dependency block and document the convention in AGENTS.md. Also add an exhaustiveness test tying WriteRouteGate.WRITE_ROUTES to every RESTCatalogAdapter.Route constant, so a route added on the next vendored adapter upgrade fails the test until explicitly classified as a write or a deliberate non-write - closing the exact drift that once let COMMIT_TRANSACTION and the view/namespace writes through unguarded.
…moke The REST write smoke previously only drove create/load/drop plus two CREATE_TABLE negatives. Extend the HMS_SMOKE_REST_WRITE_TABLE-guarded block to add a real commit against the just-created table (asserting the returned metadata-location changed, proving HiveTableOperations.commit actually wrote a new metadata file instead of silently no-oping), a rename round trip, and three more write-route gate negatives - COMMIT_TRANSACTION on a federated table, CREATE_NAMESPACE with a federated name, and rename with a federated destination namespace. COMMIT_TRANSACTION was a critical bypass found during phase 5a and had so far only been covered by unit tests. The block now drops both the write table's original and renamed names defensively before creating, so a rerun on a dirty stand cannot half-fail. Document the new coverage in TEST-MATRIX.md/.ru.md (rows G26-G30) and the 2026-07-28 revalidation-log entry.
The stand's Kerberos profile has kept the Iceberg REST listener off since phase 1 because SPNEGO needed a GSS-capable curl inside the network; the proxy container's curl is GSS-capable now, so there is no reason left to keep the listener untested there while phase 5a's REST writes and the hadoop-hdfs 2.6.0 bump both land on paths the Kerberos profile also uses. Give the proxy an HTTP/proxy@REALM principal in its existing keytab, enable rest-catalog.* with SPNEGO in the Kerberos properties, and fix two real bugs the first real write under Kerberos surfaced: IcebergRestService was building its own Kerberos-blind Configuration instead of reusing the catalog's HiveConf (breaks with "Failed to specify server's Kerberos principal name"), and the per-catalog conf was missing dfs.data.transfer.protection, which the datanodes require for SASL data transfer once an actual block write - not just a NameNode RPC - is involved. Update TEST-MATRIX(.ru).md and README(.ru).md to reflect the checks that were actually run against the Kerberos profile.
RoutingMetaStoreClient now answers IMetaStoreClient.createDatabase, dropDatabase(String,boolean,boolean,boolean) and alterDatabase instead of throwing UnsupportedOperationException, delegating to the Thrift create_database/drop_database/alter_database RPCs and translating names through the existing CatalogNameTranslation. The Database payload passed to create/alter is translated on a copy via the Thrift copy constructor, never by mutating the caller's object. RecordingThriftIface now records both the argument and the Database payload's own name for alter_database, so a missing payload translation fails the test (verified: removing it breaks alterDatabaseTranslatesBothArgumentAndPayload with the expected mismatch, restoring it goes back to green).
The dropDatabase case matched on 4-arg overloads by length alone, so IMetaStoreClient's (String,String,boolean,boolean) default overload (catName, dbName, ...) also matched and cast the dbName String to Boolean, throwing ClassCastException instead of the usual UnsupportedOperationException. Narrow the match to the exact (String,boolean,boolean,boolean) signature, mirroring dropTable's explicit paramTypes checks. Also mirror dropTable's ignoreUnknownTab handling for ignoreUnknownDb: catch NoSuchObjectException from the delegate call and rethrow only when the flag is false, instead of silently discarding it. Strengthen RecordingThriftIface's drop_database case to record the deleteData/cascade booleans (and throw NoSuchObjectException for an unknown database) so argument-order regressions and the ignoreUnknownDb behavior are covered by tests.
…erves The front door's client plumbing already dispatches view writes, namespace DDL and multi-table transaction commits through the same generic RESTCatalogAdapter/RoutingHiveCatalog path table writes use, and WriteRouteGate already gates all thirteen write routes for the default catalog - but GET /v1/config only advertised the five table-write routes, so a spec-compliant client would conclude the rest were unsupported. Extend WRITE_ENDPOINTS with the eight confirmed Iceberg 1.9.2 Endpoint constants for view CRUD+rename, namespace CRUD and transaction commit. Other catalogs are unaffected: they still advertise reads only, matching the synthetic lock shim that backs their writes with no real conflict checking.
Extend the HMS_SMOKE_REST_WRITE_TABLE-guarded block with namespace DDL, view write and multi-table transaction-commit round trips, each asserting the effect (property actually set, namespace actually gone, view actually listed, metadata-location actually changed) rather than trusting the status code alone. Prove the new transaction assertion discriminates by temporarily demanding a no-op commit, confirming the runner fails, then restoring it. Rebuilt and restaged the jar and validated on the stand: namespace DDL had never before reached a real metastore (the pre-phase jar answered 406 for createDatabase), and now passes end to end on both the plain and Kerberos profiles, the latter confirmed by genuine create_database/alter_database/ drop_database audit entries. Correct stale "read-only" claims in the smoke-stand README (both locales) and proxy properties comments left over from before phase 5a's write gate and the Kerberos REST turn-on. Update README/CHANGELOG (both locales) to state that view writes and transaction commit were already reachable after phase 5a and are now officially advertised and covered, while namespace DDL is genuinely new this phase. Update TEST-MATRIX (both locales) with the new rows and a revalidation-log entry for this run.
…the smoke Two advertised REST routes (view rename, view property update) were never exercised by the smoke, and WriteRouteGate had thirteen write routes but only five wire-level negatives. Extend the existing view round trip to update and rename the view, asserting the property actually stuck and that the rename moved the view (200 under the new name, 404 under the old) rather than copying it. Add four more 403 negatives against a federated namespace under the default prefix: CREATE_VIEW (using the full valid view body, since a stub body 400s before the gate is even consulted), DROP_VIEW, DROP_NAMESPACE and UPDATE_NAMESPACE.
…t drift WriteRouteGate previously permitted a write when the namespace's owning catalog could not be resolved; make that case a refusal instead, since an unknown catalog is exactly the ambiguous case the synthetic lock shim's lack of conflict checking makes unsafe to guess about (currently unreachable in production, but the gate's shape should not rely on that). Add a test tying WriteRouteGate.WRITE_ROUTES to IcebergRestService.WRITE_ENDPOINTS one-to-one so a future write route added upstream cannot silently lag in the advertised endpoint list, and pin the integration test's positive endpoint assertions to the exact JSON-quoted rendering so a dropped route constant cannot hide behind a sibling whose longer rendering happens to share the same prefix (e.g. CREATE_VIEW vs UPDATE_VIEW). Also reorder IcebergHttpHandler.handle()'s finally block so the per-request thread-local context is restored before the metric is recorded, so a throwing recordRestRequest can no longer leak remoteAddress/remoteUser into the next request served by that pool thread.
The Iceberg REST write round trip hardcoded the namespace ("smoke_rest_ns")
and view ("smoke_rest_view") names it creates and defensively pre-deletes,
unlike the write table which was already configurable via
HMS_SMOKE_REST_WRITE_TABLE. Against a real installation this could silently
drop a pre-existing namespace sharing the default name. Add
HMS_SMOKE_REST_WRITE_NAMESPACE and HMS_SMOKE_REST_WRITE_VIEW (defaulting to
the previous hardcoded values), document them in the runner's usage text and
env example, and set them explicitly in the stand's simple.env. The
defensive namespace pre-delete now only runs when the namespace is absent or
already empty (no tables, no views), refusing with a clear error otherwise
instead of destroying real content.
Also fix mktemp templates that put a literal suffix after the X's
(...XXXXXX.json / .sql / .out): BSD mktemp on macOS does not randomize X's
followed by a suffix, so every run reused the same filename, and a leftover
from an interrupted run (fail() bypasses the cleanup trap) broke the next
run. Put the X's last and append the suffix with mv after creation.
Finally, reword "write-роут" back to "write-поверхность"/"все write-роуты"
in three RU-only spots (README.ru.md, CHANGELOG.ru.md,
smoke-stand/README.ru.md) where "write surface" had been mistranslated as a
single route rather than the whole set of write routes.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds an Iceberg REST Catalog front door to the proxy: a parallel HTTP listener that speaks the
Iceberg REST spec and is backed by the same routing, federation, security and observability
pipeline as the Thrift front door. PyIceberg, Spark and Trino can read Iceberg tables through it,
and — in the default catalog only — create and commit to them.
Disabled by default (
rest-catalog.enabled=false); nothing changes for existing deploymentsunless the listener is switched on.
What it serves
Every configured catalog is exposed as its own prefix,
/v1/<catalog>/..., withGET /v1/config?warehouse=<catalog>for discovery. The default catalog's prefix keeps thefederated view (its own databases plus other catalogs' under
<catalog><separator><db>names);every other prefix is a clean per-catalog view.
Reads: namespace and table listing and load, view listing and load,
HEADexistence checks, andlisting pagination.
Writes, default catalog only: table create, commit, rename, register and drop; view create,
update, rename and drop; namespace create, update and drop; multi-table transaction commit.
/v1/configadvertises exactly what the front door serves, so a client learns the read/writeasymmetry between catalogs from discovery instead of from refusals.
Why writes are restricted to the default catalog
This is the load-bearing design decision, not a configuration preference.
Iceberg commits atomically one of two ways.
NoLockrelies on the metastore doing acompare-and-swap on
metadata_locationthroughalter_table's expected-parameter check —expected_parameterappears nowhere inHiveAlterHandlerof either backend jar, so that path isunavailable. The remaining option,
MetastoreLock, takes a real HMS lock.Only the default catalog has real locks: it owns the TxnHandler. Every other catalog is served by
the proxy's synthetic lock shim, which grants
EXCLUSIVElocks without conflict checking. A commitrouted there would take an always-granted lock, believe it owned the table, race a concurrent
writer, and silently lose an update. Tolerable for reads; not for writes.
WriteRouteGatetherefore refuses every write route whose target namespace resolves to anothercatalog. It keys on the resolved catalog, never the URL prefix — under the default prefix the
federated view exposes other catalogs' databases as
apache__default, and a write there wouldotherwise reach the shim through the back door. Resolution reuses the proxy's own
CatalogRouterrather than a second name parser. The gate covers all thirteen write routes the adapter exposes,
fails closed when a namespace cannot be resolved, and a drift-guard test fails the build if a
future Iceberg upgrade adds a write route that nobody classified.
Catalog access modes and write-db whitelists are inherited from the routing layer, so a REST client
faces exactly what a Thrift client faces.
Defects found and fixed along the way
Several were pre-existing and only surfaced because these code paths had never been exercised:
NoSuchMethodErroronFSOutputSummer:orc-coredraggedhadoop-hdfs:2.2.0alongsidehadoop-common:2.6.0, and Maven never comparedthem (different artifact IDs). Aligned to 2.6.0; the transitive Xerces that came with the fix is
excluded, since its JAXP services file would have made a 2007-era parser the process-wide XML
provider.
numbers — on a listener that may be unauthenticated.
HEADresponses wrote a body, so everyHEADthat errored logged a stack trace per request;the same defect existed silently on the management listener.
?pageSize=withoutpageTokenreturned 500 — the shape a client's first paginated requesttakes.
Configurationinstead of the catalog's
HiveConf), plus a missingdfs.data.transfer.protection.Validation
restcatalogpackage.smoke-stand/TEST-MATRIX.md: 38 rows covering discovery, reads,the full write surface, nine gate refusals, error shapes and metrics. Write assertions check
effects — a commit must advance
metadata-location, a rename must leave the old name gone — soa no-op that returns 200 fails the run.
writes succeed, the audit log carries the authenticated principal, and the commit lock is served
by the real backend rather than the shim.
dependency change.
Observability
Three Prometheus series —
hms_proxy_rest_requests_total{prefix,route,status},hms_proxy_rest_request_duration_seconds{prefix,route}andhms_proxy_rest_listener_info{bind_host,port}— plus an Iceberg REST row in the bundled Grafanadashboard. Labels carry the parsed route, never a raw URL, so cardinality stays bounded.
Known limits
RoutingHiveCataloginjects its client pool intoHiveCatalog's privateclientsfield byreflection, pinned to Iceberg 1.9.2;
RoutingHiveCatalogTestmust be re-run on any version bump.HA, and queries run as local MapReduce.