Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
109 changes: 109 additions & 0 deletions .github/workflows/image-build.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,109 @@
name: Image build
on:
push:
branches:
- main
- 'custom-build-*.Final*'
paths-ignore:
- '*.md'

jobs:
build:
runs-on: ubuntu-latest

steps:
- name: Checkout core repository
uses: actions/checkout@v4
with:
repository: get-bridge/debezium
ref: main
path: core
- name: Checkout Debezium Server
uses: actions/checkout@v4
with:
repository: get-bridge/debezium-server
ref: ${{ github.ref }}
path: server
- name: Set up JDK
uses: actions/setup-java@v4
with:
distribution: 'temurin'
java-version: 21

- name: Cache Maven packages
uses: actions/cache@v4
with:
path: ~/.m2
key: ${{ runner.os }}-m2-${{ hashFiles('**/pom.xml') }}
restore-keys: ${{ runner.os }}-m2

- name: Maven build core
run: |
./server/mvnw clean install -B -ntp -f core/pom.xml \
-DskipTests=true \
-DskipITs=true \
-Dformat.formatter.goal=validate \
-Dformat.imports.goal=check \
-Dhttp.keepAlive=false \
-Dmaven.wagon.http.pool=false \
-Dmaven.wagon.httpconnectionManager.ttlSeconds=120 \
-Dcheckstyle.skip=true \
-Dformat.skip=true \
-Drevapi.skip \
-Dorg.slf4j.simpleLogger.log.org.apache.maven.cli.transfer.Slf4jMavenTransferListener=warn
- name: Maven build Debezium Server
id: build_server
run: |
./server/mvnw clean install -B -ntp -fae -f server/pom.xml \
-Passembly,pulsar-and-postgres \
-Dformat.formatter.goal=validate \
-Dformat.imports.goal=check \
-Dhttp.keepAlive=false \
-Dmaven.wagon.http.pool=false \
-Dmaven.wagon.httpconnectionManager.ttlSeconds=120 \
-DskipNonCore \
-DskipTests=true \
-DskipITs=true \
-Dorg.slf4j.simpleLogger.log.org.apache.maven.cli.transfer.Slf4jMavenTransferListener=warn \
-DfailFlakyTests=false
ARTIFACT_PATH=$(find server/debezium-server-dist/target/ -maxdepth 1 -name "debezium-server-dist-*.tar.gz" | head -n 1)
if [ -f "$ARTIFACT_PATH" ]; then
ARTIFACT_FILENAME=$(basename "$ARTIFACT_PATH")
echo "Discovered artifact: $ARTIFACT_FILENAME"
echo "artifact_filename=$ARTIFACT_FILENAME" >> $GITHUB_OUTPUT
else
echo "Error: Could not find artifact 'debezium-server-dist-*.tar.gz' in server/debezium-server-dist/target/"
echo "Available files in target directory:"
ls -la server/debezium-server-dist/target/
exit 1
fi

- name: Configure AWS credentials
uses: aws-actions/configure-aws-credentials@v2
with:
aws-access-key-id: ${{ secrets.TRUSS_AWS_ACCESS_KEY_ID }}
aws-secret-access-key: ${{ secrets.TRUSS_AWS_SECRET_ACCESS_KEY }}
aws-region: us-east-2

- name: Login to AWS ECR
id: login-ecr
uses: aws-actions/amazon-ecr-login@v1

- name: Build Server Docker image
env:
ECR_REGISTRY: ${{ steps.login-ecr.outputs.registry }}
ECR_REPOSITORY: debezium-server
run: |
ARTIFACT_FILE="${{ steps.build_server.outputs.artifact_filename }}"
if [ -z "$ARTIFACT_FILE" ]; then
echo "Error: Artifact filename not found from previous step. Docker build cannot proceed."
exit 1
fi
echo "Building Docker image with artifact: $ARTIFACT_FILE"
docker build \
--build-arg DEBEZIUM_SERVER_DIST_FILENAME="$ARTIFACT_FILE" \
--tag $ECR_REGISTRY/$ECR_REPOSITORY:latest \
--tag $ECR_REGISTRY/$ECR_REPOSITORY:$GITHUB_SHA \
server/ # Context for Dockerfile is 'server/'
docker push $ECR_REGISTRY/$ECR_REPOSITORY:$GITHUB_SHA
docker push $ECR_REGISTRY/$ECR_REPOSITORY:latest
87 changes: 87 additions & 0 deletions Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,87 @@
#
# Tracing agent prep
#
FROM curlimages/curl:latest AS downloader
RUN curl --progress-bar --location --output /tmp/otel-javaagent.jar \
https://github.com/open-telemetry/opentelemetry-java-instrumentation/releases/download/v2.9.0/opentelemetry-javaagent.jar

FROM registry.access.redhat.com/ubi8/openjdk-21 AS builder
LABEL maintainer="Debezium Community"
ENV SERVER_HOME=/debezium \
DEBEZIUM_ARCHIVE=/tmp/debezium.tar.gz

#
# Create a directory for Debezium Server
#
USER root
RUN microdnf -y install gzip && \
microdnf clean all && \
mkdir $SERVER_HOME && \
chmod 755 $SERVER_HOME

#
# Change ownership and switch user
#
RUN chown -R jboss $SERVER_HOME && \
chgrp -R jboss $SERVER_HOME
USER jboss
RUN mkdir $SERVER_HOME/conf && \
mkdir $SERVER_HOME/data

ARG DEBEZIUM_SERVER_DIST_FILENAME

#
# Copy the debezium distribution file
#
COPY --chown=jboss:jboss debezium-server-dist/target/${DEBEZIUM_SERVER_DIST_FILENAME} $DEBEZIUM_ARCHIVE

#
# Verify the contents and then install ...
#
RUN tar xzf $DEBEZIUM_ARCHIVE -C $SERVER_HOME --strip-components 1 && \
rm -f $DEBEZIUM_ARCHIVE

# Final stage
FROM registry.access.redhat.com/ubi8/openjdk-21 AS final
ENV SERVER_HOME=/debezium

USER root
RUN microdnf -y install gzip && \
microdnf clean all && \
mkdir $SERVER_HOME && \
chmod 755 $SERVER_HOME

RUN chown -R jboss $SERVER_HOME && \
chgrp -R jboss $SERVER_HOME

USER jboss

# Copy the prepared Debezium installation from builder stage
COPY --from=builder --chown=jboss:jboss $SERVER_HOME $SERVER_HOME

# Copy the OpenTelemetry agent from downloader stage
COPY --from=downloader --chown=jboss:jboss /tmp/otel-javaagent.jar $SERVER_HOME/otel-javaagent.jar

#
# Add jolokia for healthchecks over jmx
#
ADD --chown=jboss:jboss https://repo1.maven.org/maven2/org/jolokia/jolokia-jvm/1.7.2/jolokia-jvm-1.7.2.jar $SERVER_HOME/jolokia.jar

# Copy the run script
COPY --chown=jboss:jboss bridge-run.sh $SERVER_HOME

#
# Allow random UID to use Debezium Server
#
RUN chmod -R g+w,o+w $SERVER_HOME
RUN chmod +x $SERVER_HOME/bridge-run.sh

# Set the working directory to the Debezium Server home directory
WORKDIR $SERVER_HOME

#
# Expose the ports and set up volumes for the data, transaction log, and configuration
#
EXPOSE 8080
VOLUME ["/debezium/conf","/debezium/data"]
CMD ["/debezium/bridge-run.sh"]
146 changes: 146 additions & 0 deletions README-BRIDGE-PATCHES.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,146 @@
# Bridge Patches for Debezium Server

This document explains how to apply Bridge-specific customizations to new Debezium Server versions using git patches.

## Overview

We maintain Bridge-specific customizations as git patches that can be applied to any Debezium Server version. This approach provides:

- ✅ **Version independence** - Apply patches to any upstream version
- ✅ **Maintainable updates** - No manual file editing required
- ✅ **Git history preservation** - Proper commit tracking and authorship
- ✅ **Automated conflict resolution** - Using git's 3-way merge

## Available Patches

### `fast-jar-image-build.patch`
**Main production patch** containing:

- **Multi-stage Dockerfile** with OpenTelemetry and Jolokia monitoring
- **Advanced bridge-run.sh** with smart JAR detection (fast-jar vs classpath)
- **GitHub Actions workflow** for automated image building
- **Fast-jar packaging** configuration in pom.xml

**Features:**
- OpenTelemetry tracing agent
- Jolokia JMX monitoring
- Smart Quarkus fast-jar detection
- CI/CD pipeline for ECR deployment

## Applying Patches to New Versions

### Method 1: Clean Application (Recommended)

```bash
# Start from a clean upstream tag
git checkout v3.4.0.Final # or desired version
git checkout -b custom-build-v3.4.0.Final

# Apply patch with 3-way merge (handles conflicts automatically)
git am --3way bridge-patches/fast-jar-image-build.patch
```

### Method 2: If 3-way merge fails

If the patch can't be applied automatically:

```bash
# See what failed
git am --show-current-patch=diff

# Resolve manually and continue
git add .
git am --continue

# Or abort and try different approach
git am --abort
```

## Troubleshooting

### "error: patch failed: pom.xml:39"

**Solution:** Use 3-way merge which automatically resolves pom.xml structure differences:
```bash
git am --3way bridge-patches/fast-jar-image-build.patch
```

### "error: Dockerfile: already exists in index"

**Solution:** Remove conflicting files first:
```bash
git rm Dockerfile bridge-run.sh # if they exist
git commit -m "Remove old files"
git am --3way bridge-patches/fast-jar-image-build.patch
```

### "fatal: previous rebase directory .git/rebase-apply still exists"

**Solution:** Clean up previous failed attempts:
```bash
git am --abort
git am --3way bridge-patches/fast-jar-image-build.patch
```

## Creating New Patches

If you need to create a new patch from your customizations:

```bash
# After making your changes
git add .
git commit -m "Your changes"

# Create patch from the last N commits
git format-patch -1 HEAD --stdout > bridge-patches/your-patch-name.patch
```

## Patch Contents Summary

### Dockerfile
- Multi-stage build (downloader → builder → final)
- OpenJDK 21 base images
- OpenTelemetry agent download
- Jolokia JMX agent integration
- Proper permissions and security

### bridge-run.sh
- Smart JAR detection (quarkus-run.jar vs classpath mode)
- OpenTelemetry and Jolokia agent loading
- JMX remote monitoring configuration
- Pulsar service account handling

### GitHub Workflow
- Automated builds on push
- Core Debezium building
- Server building with fast-jar
- ECR deployment pipeline

### POM Configuration
- Fast-jar packaging type for optimized startup
- Quarkus Maven plugin configuration

## Version Compatibility

This patch has been tested with:
- ✅ Debezium Server v3.3.0.Final

For older versions, manual adaptation may be required.

## Building Production Images

After applying patches:

```bash
# Build the server
./mvnw clean package -DskipITs -DskipTests -Passembly

# Build Docker image
docker build -t debezium-server-bridge .

# Or use the automated CI/CD pipeline via GitHub Actions
```

---

**Note:** Always test patch application on a branch before applying to main development lines.
Loading
Loading