-
Notifications
You must be signed in to change notification settings - Fork 3k
Add Kafka Connect artifact publish to release process #15212
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,26 @@ | ||
| <!-- | ||
| - Licensed to the Apache Software Foundation (ASF) under one | ||
| - or more contributor license agreements. See the NOTICE file | ||
| - distributed with this work for additional information | ||
| - regarding copyright ownership. The ASF licenses this file | ||
| - to you under the Apache License, Version 2.0 (the | ||
| - "License"); you may not use this file except in compliance | ||
| - with the License. You may obtain a copy of the License at | ||
| - | ||
| - http://www.apache.org/licenses/LICENSE-2.0 | ||
| - | ||
| - Unless required by applicable law or agreed to in writing, | ||
| - software distributed under the License is distributed on an | ||
| - "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY | ||
| - KIND, either express or implied. See the License for the | ||
| - specific language governing permissions and limitations | ||
| - under the License. | ||
| --> | ||
|
|
||
| # Apache Iceberg Sink Connector | ||
|
|
||
| A Kafka Connect sink connector for writing data from Apache Kafka into Apache Iceberg tables. | ||
|
|
||
| Documentation: https://iceberg.apache.org/docs/latest/kafka-connect/ | ||
|
|
||
| License: Apache License 2.0 |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,27 @@ | ||
| # Licensed to the Apache Software Foundation (ASF) under one | ||
| # or more contributor license agreements. See the NOTICE file | ||
| # distributed with this work for additional information | ||
| # regarding copyright ownership. The ASF licenses this file | ||
| # to you under the Apache License, Version 2.0 (the | ||
| # "License"); you may not use this file except in compliance | ||
| # with the License. You may obtain a copy of the License at | ||
| # | ||
| # http://www.apache.org/licenses/LICENSE-2.0 | ||
| # | ||
| # Unless required by applicable law or agreed to in writing, | ||
| # software distributed under the License is distributed on an | ||
| # "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY | ||
| # KIND, either express or implied. See the License for the | ||
| # specific language governing permissions and limitations | ||
| # under the License. | ||
|
|
||
| # Iceberg Sink Connector - Example Configuration | ||
| # https://iceberg.apache.org/docs/latest/kafka-connect/ | ||
| name=iceberg-sink | ||
| connector.class=org.apache.iceberg.connect.IcebergSinkConnector | ||
| tasks.max=1 | ||
| topics=events | ||
| iceberg.tables=db.table_name | ||
| iceberg.catalog.type=rest | ||
| iceberg.catalog.uri=https://your-catalog-uri | ||
| iceberg.catalog.warehouse=your-warehouse | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -35,7 +35,7 @@ | |
| "logo": "assets/iceberg.png", | ||
| "type": "organization", | ||
| "url": "https://iceberg.apache.org", | ||
| "username": "iceberg" | ||
| "username": "apache" | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Why this change? Is there some reason this needs to be |
||
| }, | ||
|
|
||
| "support": { | ||
|
|
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -166,6 +166,11 @@ To build and publish the convenience binaries, run the `dev/stage-binaries.sh` s | |
| dev/stage-binaries.sh | ||
| ``` | ||
|
|
||
| !!! Note | ||
| The staged binaries include the Kafka Connect runtime distribution | ||
| (`iceberg-kafka-connect-runtime-<VERSION>.zip`), which bundles the connector | ||
| with all required dependencies for Kafka Connect plugin installation. | ||
|
|
||
| Next, you need to close the staging repository: | ||
|
|
||
| 1. Go to [Nexus](https://repository.apache.org/) and log in | ||
|
|
@@ -297,6 +302,25 @@ Java artifacts are available from Maven Central. | |
| Thanks to everyone for contributing! | ||
| ``` | ||
|
|
||
| #### Confluent Marketplace Submission | ||
|
|
||
| After the release is published to Maven Central, the Kafka Connect runtime distribution | ||
| can be [submitted to](https://docs.confluent.io/platform/current/connect/confluent-hub/contributing.html) | ||
| [Confluent Marketplace](https://www.confluent.io/hub/) for broader distribution. | ||
|
|
||
| To submit to Confluent Hub: | ||
|
|
||
| 1. [Contact the Confluent Hub team](mailto:confluent-hub@confluent.io) with the Maven Central URL for the distribution zip: | ||
|
|
||
| ``` | ||
| https://repo1.maven.org/maven2/org/apache/iceberg/iceberg-kafka-connect-runtime/<VERSION>/iceberg-kafka-connect-runtime-<VERSION>.zip | ||
| ``` | ||
|
|
||
| 2. Confluent will publish to the Marketplace | ||
|
|
||
| !!! Note | ||
| This step is optional and can be performed by any PMC member after the release is finalized. | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Why would this be optional? I would think if we're doing updates we always want artifacts updated in the marketplace. |
||
|
|
||
| #### Update revapi | ||
|
|
||
| Create a PR in the `iceberg` repo to make revapi run on the new release. For an example see [this PR](https://github.com/apache/iceberg/pull/6275). | ||
|
|
||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I would prefer that we use the name
iceberg-sink.properties.templatewhich is more consistent with hive/spark/etc.