Kueue Workload Controller

A Knative-based Kubernetes controller that reconciles Kueue Workload CRDs that are owned by Tekton PipelineRuns.

Overview

This controller watches Kueue Workload resources that are owned by Tekton PipelineRuns and automatically syncs required secrets from the hub cluster to spoke clusters. It enables seamless multi-cluster pipeline execution by ensuring PipelineRuns have the necessary authentication secrets available on their target clusters.

Features

• Automatic Secret Syncing: Syncs Git authentication secrets from hub to spoke clusters
• Multi-Cluster Support: Works with Kueue's MultiKueue for distributed workload execution
• Selective Processing: Only handles Workloads owned by Tekton PipelineRuns
• Production Ready: Includes RBAC, deployment manifests, and Docker support

Prerequisites

• Go 1.22 or later
• Kubernetes cluster with:
  • Kueue installed
  • Tekton Pipelines installed
• kubectl configured to access your cluster
• Docker for building images

Installation

1. Build and Vendor Dependencies

make tidy    # Run go mod tidy
make vendor  # Run go mod vendor

2. Build Binary Locally

make build

3. Build Docker Image

# Build for local architecture
make docker-build

# Or build and push multi-arch image
make docker-buildx IMG=your-registry/workload-controller:latest

4. Deploy to Kubernetes

# Apply RBAC and deployment
make deploy

# Check status
make status

# View logs
make logs

Development

Local Development

# Format code
make fmt

# Run linters
make vet

# Run tests
make test

# Run controller locally (requires kubeconfig)
make run

Quick Development Cycle

# Build, dockerize, and deploy in one command
make quick-deploy

Configuration

Environment Variables

The controller reads these environment variables (set in config/deployment.yaml):

• SYSTEM_NAMESPACE: Namespace where the controller runs
• CONFIG_LOGGING_NAME: ConfigMap name for logging configuration
• CONFIG_OBSERVABILITY_NAME: ConfigMap name for observability configuration
• METRICS_DOMAIN: Domain for metrics reporting

RBAC Permissions

The controller requires access to:

• Kueue Workloads (read and watch)
• Tekton PipelineRuns (read and watch)
• Secrets (full access for syncing across clusters)
• MultiKueueClusters (read for cluster connection details)
• ConfigMaps and Leases (for controller configuration and leader election)

How It Works

When a PipelineRun is scheduled to run on a spoke cluster via Kueue MultiKueue:

1. The controller detects the Workload resource associated with the PipelineRun
2. Retrieves the Git authentication secret specified in the PipelineRun's annotations
3. Syncs the secret from the hub cluster to the target spoke cluster
4. Ensures the secret has proper ownership for lifecycle management

The PipelineRun can then access the authentication secret on the spoke cluster to clone repositories and execute pipeline tasks.

Makefile Targets

help          - Display available targets
fmt           - Run go fmt
vet           - Run go vet
test          - Run tests
build         - Build binary
run           - Run locally
tidy          - Run go mod tidy
vendor        - Run go mod vendor
docker-build  - Build docker image
docker-push   - Push docker image
docker-buildx - Build multi-arch image
deploy        - Deploy to cluster
undeploy      - Remove from cluster
logs          - Show controller logs
status        - Show controller status
clean         - Clean build artifacts
all           - Full workflow: vendor, build, push, deploy
quick-deploy  - Quick local development cycle

Troubleshooting

Check Controller Status

kubectl get deployment workload-controller -n syncer-service
kubectl get pods -n syncer-service -l app=workload-controller

View Logs

kubectl logs -n syncer-service -l app=workload-controller -f

Common Issues

1. Secrets not syncing: Ensure PipelineRun has the pipelinesascode.tekton.dev/git-auth-secret annotation
2. Controller not starting: Verify RBAC permissions and that Kueue and Tekton are installed
3. Image pull errors: Ensure the controller image is pushed and accessible from your cluster

References

• Kueue Documentation
• Kueue MultiKueue
• Tekton Pipelines

License

See LICENSE file for details.