Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions .cursor/prompts/check-versions.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ Query running DCM containers and resolve their image digests to git commit SHAs.

### Custom deploy directory
```bash
./scripts/deploy-dcm.sh --api-gateway-dir /path/to/deploy --running-versions
./scripts/deploy-dcm.sh --control-plane-dir /path/to/deploy --running-versions
```

## Prerequisites
Expand All @@ -25,7 +25,7 @@ Writes `dcm-versions.json` to the current directory. Example:

```json
{
"quay.io/dcm-project/catalog-manager:latest": {
"quay.io/dcm-project/control-plane:latest": {
"image_digest": "sha256:1cdf5482f586...",
"git_sha": "2388248"
},
Expand Down
22 changes: 11 additions & 11 deletions .cursor/prompts/deploy-dcm.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,23 +23,23 @@ Deploy the full DCM stack for E2E testing using `scripts/deploy-dcm.sh`.
# Auto-resolve the latest semver tag from Quay.io
./scripts/deploy-dcm.sh --version release

# Explicit version with a custom api-gateway branch
./scripts/deploy-dcm.sh --version v0.1.0-rc.1 --api-gateway-branch my-branch
# Explicit version with a custom control-plane branch
./scripts/deploy-dcm.sh --version v0.1.0-rc.1 --control-plane-branch my-branch
```

### Deploy from a Different Branch
```bash
./scripts/deploy-dcm.sh --api-gateway-branch feature-x
./scripts/deploy-dcm.sh --control-plane-branch feature-x
```

### Deploy from a Fork
```bash
./scripts/deploy-dcm.sh --api-gateway-repo https://github.com/myfork/api-gateway.git
./scripts/deploy-dcm.sh --control-plane-repo https://github.com/myfork/control-plane.git
```

### Deploy to a Custom Directory
```bash
./scripts/deploy-dcm.sh --api-gateway-dir /tmp/my-dcm-deploy
./scripts/deploy-dcm.sh --control-plane-dir /tmp/my-dcm-deploy
```

### Deploy with Auto-Cleanup on Failure
Expand Down Expand Up @@ -90,9 +90,9 @@ When any service provider is enabled, the script resolves cluster access in this
| Variable | Flag equivalent |
|----------|----------------|
| `DCM_VERSION` | `--version` |
| `API_GATEWAY_REPO` | `--api-gateway-repo` |
| `API_GATEWAY_BRANCH` | `--api-gateway-branch` |
| `API_GATEWAY_TMP_DIR` | `--api-gateway-dir` |
| `CONTROL_PLANE_REPO` | `--control-plane-repo` |
| `CONTROL_PLANE_BRANCH` | `--control-plane-branch` |
| `CONTROL_PLANE_TMP_DIR` | `--control-plane-dir` |
| `KUBECONFIG` | `--kubeconfig` |
| `KUBEVIRT_VM_NAMESPACE` | `--kubevirt-vm-namespace` |
| `K8S_CONTAINER_SP_NAMESPACE` | `--k8s-container-namespace` |
Expand All @@ -104,14 +104,14 @@ Flags take precedence over environment variables.

## What Happens

1. Clones api-gateway (owns `compose.yaml`)
1. Clones control-plane (`deploy/compose.yaml`)
2. Runs `podman-compose up -d`
3. Verifies all containers are running
4. Polls `/api/v1alpha1/health/*` endpoints (90s timeout)
4. Polls `/api/v1alpha1/health` (90s timeout)
5. Resolves container images to git commit SHAs via Quay.io API
6. Writes `dcm-versions.json`

## Output

- Stack available at `http://localhost:9080`
- Stack available at `http://localhost:8080`
- Version info written to `dcm-versions.json`
2 changes: 1 addition & 1 deletion .cursor/prompts/tear-down.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ Stop and clean up a running DCM deployment.

### Custom deploy directory
```bash
./scripts/deploy-dcm.sh --api-gateway-dir /path/to/deploy --tear-down
./scripts/deploy-dcm.sh --control-plane-dir /path/to/deploy --tear-down
```

## What Happens
Expand Down
31 changes: 12 additions & 19 deletions .cursor/prompts/troubleshoot-deploy.md
Original file line number Diff line number Diff line change
Expand Up @@ -51,44 +51,37 @@ oc get crd virtualmachines.kubevirt.io
### Containers fail to start
```bash
# Check container status
podman-compose -f /tmp/dcm-e2e/compose.yaml ps
podman-compose -f /tmp/dcm-e2e/deploy/compose.yaml ps

# View logs for a failing service
podman-compose -f /tmp/dcm-e2e/compose.yaml logs --tail=50 <service-name>
podman-compose -f /tmp/dcm-e2e/deploy/compose.yaml logs --tail=50 <service-name>

# Check for port conflicts
podman ps --format '{{.Ports}}' | grep 9080
podman ps --format '{{.Ports}}' | grep 8080
```

### Health check timeouts
```bash
# Manual health check
curl -v http://localhost:9080/api/v1alpha1/health/providers
curl -v http://localhost:8080/api/v1alpha1/health

# All health endpoints
for ep in providers catalog policies placement; do
echo -n "$ep: "
curl -s -o /dev/null -w "%{http_code}" "http://localhost:9080/api/v1alpha1/health/$ep"
echo
done

# Check gateway logs
podman-compose -f /tmp/dcm-e2e/compose.yaml logs --tail=50 api-gateway
# Check control-plane logs
podman-compose -f /tmp/dcm-e2e/deploy/compose.yaml logs --tail=50 control-plane
```

### Compose file not found
```bash
# Verify clone worked
ls -la /tmp/dcm-e2e/compose.yaml
ls -la /tmp/dcm-e2e/deploy/compose.yaml

# Re-deploy (cleans and re-clones)
./scripts/deploy-dcm.sh
```

### Port already in use
```bash
# Find what's using port 9080
lsof -i :9080
# Find what's using port 8080
lsof -i :8080

# Tear down and redeploy
./scripts/deploy-dcm.sh --tear-down
Expand All @@ -99,13 +92,13 @@ lsof -i :9080

```bash
# All container status
podman-compose -f /tmp/dcm-e2e/compose.yaml ps
podman-compose -f /tmp/dcm-e2e/deploy/compose.yaml ps

# Recent container logs (all services)
podman-compose -f /tmp/dcm-e2e/compose.yaml logs --tail=20
podman-compose -f /tmp/dcm-e2e/deploy/compose.yaml logs --tail=20

# Specific service logs
podman-compose -f /tmp/dcm-e2e/compose.yaml logs --tail=50 <service>
podman-compose -f /tmp/dcm-e2e/deploy/compose.yaml logs --tail=50 <service>

# Container resource usage
podman stats --no-stream
Expand Down
2 changes: 1 addition & 1 deletion .cursor/rules/deploy-script.mdc
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ See `CLAUDE.md` → "Key Script: scripts/deploy-dcm.sh" for the full description
- New functions go above the argument parsing section
- New flags need entries in both `usage()` and the `while/case` loop
- Add corresponding environment variable overrides where appropriate
- Compose variable names (`KUBEVIRT_KUBECONFIG`, `K8S_CONTAINER_SP_KUBECONFIG`, etc.) are owned by api-gateway's compose.yaml — see the contract comment at the top of the script
- Compose variable names (`KUBEVIRT_KUBECONFIG`, `K8S_CONTAINER_SP_KUBECONFIG`, etc.) are owned by control-plane's `deploy/compose.yaml` — see the contract comment at the top of the script
- Run `shellcheck scripts/deploy-dcm.sh` after changes

## Podman / Compose
Expand Down
6 changes: 3 additions & 3 deletions .cursor/rules/e2e-tests.mdc
Original file line number Diff line number Diff line change
Expand Up @@ -12,11 +12,11 @@ Files follow `{layer}_{subject}_test.go` with a matching `{layer}_helpers_test.g

| Prefix | Interface | Helpers |
|--------|-----------|---------|
| `api_` | Gateway HTTP API (`doRequest` → `localhost:9080`) | `api_helpers_test.go` |
| `api_` | Control plane HTTP API (`doRequest` → `localhost:8080`) | `api_helpers_test.go` |
| `cli_` | CLI binary (`runDCM` → `os/exec`) | `cli_helpers_test.go` |
| `sp_container_` | Container SP direct API + NATS (`doContainerSPRequest` → `localhost:8082`) | `sp_helpers_test.go` |
| `sp_acm_cluster_` | ACM Cluster SP direct API (`doAcmClusterSPRequest` → `localhost:8083`) | `sp_acm_cluster_helpers_test.go` |
| `core_` | Cross-service provisioning flow via gateway (`doRequest`) | `api_helpers_test.go`, `sp_helpers_test.go` |
| `core_` | Cross-service provisioning flow via control plane (`doRequest`) | `api_helpers_test.go`, `sp_helpers_test.go` |

When adding a new service provider, add `sp_{provider}_{concern}_test.go` (e.g. `sp_kubevirt_api_test.go`). Each provider gets its own helpers file (`sp_{provider}_helpers_test.go`). Shared infrastructure (kubectl, podman, NATS collector) stays in `sp_helpers_test.go`.

Expand All @@ -37,7 +37,7 @@ When adding a new service provider, add `sp_{provider}_{concern}_test.go` (e.g.
## Environment Variables
| Variable | Default | Purpose |
|----------|---------|---------|
| `DCM_GATEWAY_URL` | `http://localhost:9080/api/v1alpha1` | Gateway API |
| `DCM_GATEWAY_URL` | `http://localhost:8080/api/v1alpha1` | Control plane API |
| `DCM_CONTAINER_SP_URL` | `http://localhost:8082/api/v1alpha1` | Container SP direct API |
| `DCM_ACM_CLUSTER_SP_URL` | `http://localhost:8083/api/v1alpha1` | ACM Cluster SP direct API |
| `DCM_NATS_URL` | `nats://localhost:4222` | NATS for status events |
Expand Down
8 changes: 4 additions & 4 deletions .github/workflows/validate-scripts.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -94,8 +94,8 @@ jobs:

run_case "--help contains core flags" \
--expect-exit "--help" 0 \
--expect-contains "--api-gateway-repo" "api-gateway-repo flag" \
--expect-contains "--api-gateway-branch" "api-gateway-branch flag" \
--expect-contains "--control-plane-repo" "control-plane-repo flag" \
--expect-contains "--control-plane-branch" "control-plane-branch flag" \
--expect-contains "--kubeconfig" "kubeconfig flag" \
--expect-contains "--compose-file" "compose-file flag" \
--expect-contains "--cleanup-on-failure" "cleanup-on-failure flag" \
Expand Down Expand Up @@ -133,8 +133,8 @@ jobs:
run_case "--kubeconfig requires a value" \
--expect-exit "--kubeconfig" 1

run_case "--api-gateway-repo requires a value" \
--expect-exit "--api-gateway-repo" 1
run_case "--control-plane-repo requires a value" \
--expect-exit "--control-plane-repo" 1

run_case "--compose-file requires a value" \
--expect-exit "--compose-file" 1
Expand Down
12 changes: 6 additions & 6 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -26,9 +26,9 @@ CI runs ShellCheck on changed `*.sh` files via `.github/workflows/lint.yaml` (on

## Key Script: `scripts/deploy-dcm.sh`

Deploys the full DCM stack for E2E testing by cloning api-gateway (which owns `compose.yaml`), running `podman-compose up`, and polling health endpoints until all services respond 2xx.
Deploys the full DCM stack for E2E testing by cloning control-plane (`deploy/compose.yaml`), running `podman-compose up`, and polling health endpoints until all services respond 2xx.

**Flow:** clone api-gateway → `podman-compose up -d` → verify containers running → poll `/api/v1alpha1/health/*` endpoints → collect container versions from Quay.io API → write `dcm-versions.json`.
**Flow:** clone control-plane → `podman-compose up -d` → verify containers running → poll `/api/v1alpha1/health` → collect container versions from Quay.io API → write `dcm-versions.json`.

**Modes:** The script has three mutually exclusive modes:
- **Deploy** (default): full clone + bring-up + health check. Pass `--cleanup-on-failure` to auto-teardown on error (default leaves partial state for debugging).
Expand All @@ -40,7 +40,7 @@ Deploys the full DCM stack for E2E testing by cloning api-gateway (which owns `c
- `--version v0.1.0-rc.1` — pin all images to an explicit tag
- `--version release` — auto-resolve the latest semver tag from Quay.io

When a non-main version is specified, `--api-gateway-branch` is auto-derived to the corresponding release branch (e.g. `v0.1.0-rc.1` → `release/v0.1.0`) unless explicitly passed.
When a non-main version is specified, `--control-plane-branch` is auto-derived to the corresponding release branch (e.g. `v0.1.0-rc.1` → `release/v0.1.0`) unless explicitly passed.

**Service providers:** Configured via `providers/*.conf` files (see "Provider Registry" below). Enable with `--<label>-service-provider` or `--all-service-providers`.

Expand All @@ -58,7 +58,7 @@ Service providers are defined declaratively in `providers/*.conf` files. Each co
|-----|---------|
| `PROVIDER_LABEL` | Short name for display and flag generation |
| `PROVIDER_FLAG` | CLI flag name (e.g. `kubevirt-service-provider`) |
| `COMPOSE_PROFILE` | Compose profile name from api-gateway (if applicable) |
| `COMPOSE_PROFILE` | Compose profile name from control-plane deploy compose (if applicable) |
| `COMPOSE_OVERRIDE` | Compose override file relative to repo root (if applicable) |
| `CLI_REQUIREMENT` | CLI tool needed: `oc`, `oc-or-kubectl`, or empty |
| `NAMESPACE_FLAG` / `NAMESPACE_ENV` / `NAMESPACE_DEFAULT` | Namespace configuration |
Expand Down Expand Up @@ -172,7 +172,7 @@ All test targets support JUnit XML output: `make test-e2e JUNIT_REPORT=results.x
| Layer | What it tests | Label |
|-------|--------------|-------|
| **Core platform tests** | Full provisioning flow through control plane | `core`, `platform` |
| **API tests** | HTTP CRUD operations against the gateway | (none) |
| **API tests** | HTTP CRUD operations against the control plane | (none) |
| **SP tests** | Container SP direct API + NATS status events | `sp`, `container` |
| **ACM SP tests** | ACM Cluster SP API (health, registration, validation, CRUD) | `sp`, `acm-cluster` |
| **Cluster tests** | Tests requiring `kubectl`/`oc` cluster access | `cluster` |
Expand All @@ -197,7 +197,7 @@ CLI tests are skipped (not failed) if no binary is available.
- All test files use `//go:build e2e` build tag
- API tests use raw `net/http` (no generated clients) for independence from service repos
- CLI tests use `os/exec` to run the actual binary (not in-process Cobra)
- `DCM_GATEWAY_URL` env var overrides the gateway endpoint (default: `http://localhost:9080/api/v1alpha1`)
- `DCM_GATEWAY_URL` env var overrides the control plane API endpoint (default: `http://localhost:8080/api/v1alpha1`)
- `DCM_CONTAINER_SP_URL` env var overrides the container SP endpoint (default: `http://localhost:8082/api/v1alpha1`)
- `DCM_ACM_CLUSTER_SP_URL` env var overrides the ACM cluster SP endpoint (default: `http://localhost:8083/api/v1alpha1`)
- `DCM_NATS_URL` env var overrides the NATS server (default: `nats://localhost:4222`)
Expand Down
12 changes: 6 additions & 6 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# DCM Utilities

Common scripts and tooling shared across the [DCM](https://github.com/dcm-project) ecosystem. Provides the E2E deploy script for bringing up the full DCM stack locally and a Ginkgo/Gomega E2E test suite that validates the stack through the API gateway and DCM CLI.
Common scripts and tooling shared across the [DCM](https://github.com/dcm-project) ecosystem. Provides the E2E deploy script for bringing up the full DCM stack locally and a Ginkgo/Gomega E2E test suite that validates the stack through the control-plane API and DCM CLI.

## Contents

Expand All @@ -20,7 +20,7 @@ Both deploy mode and `--running-versions` produce a `dcm-versions.json` mapping

```json
{
"quay.io/dcm-project/catalog-manager:latest": {
"quay.io/dcm-project/control-plane:latest": {
"image_digest": "sha256:1cdf5482f586ce513724074c0a132b718672d2be5cbae600a47e94324078b01e",
"git_sha": "2388248"
},
Expand All @@ -35,7 +35,7 @@ Both deploy mode and `--running-versions` produce a `dcm-versions.json` mapping

`scripts/deploy-dcm.sh` automates the full DCM stack lifecycle for E2E testing:

1. Clones the [api-gateway](https://github.com/dcm-project/api-gateway) repo (which owns the `compose.yaml`)
1. Clones the [control-plane](https://github.com/dcm-project/control-plane) repo (`deploy/compose.yaml`)
2. Starts all services with `podman-compose up`
3. Polls health endpoints until every service responds 2xx
4. Resolves running container images to git commit SHAs via the Quay.io API
Expand Down Expand Up @@ -80,7 +80,7 @@ Run `./scripts/deploy-dcm.sh --help` for all flags and environment variable over

## E2E Tests

The test suite uses [Ginkgo](https://onsi.github.io/ginkgo/) and [Gomega](https://onsi.github.io/gomega/) to validate the full DCM stack through the API gateway and the DCM CLI.
The test suite uses [Ginkgo](https://onsi.github.io/ginkgo/) and [Gomega](https://onsi.github.io/gomega/) to validate the full DCM stack through the control-plane API and the DCM CLI.

### Quick Start

Expand Down Expand Up @@ -119,7 +119,7 @@ make help

| Variable | Default | Description |
|----------|---------|-------------|
| `DCM_GATEWAY_URL` | `http://localhost:9080/api/v1alpha1` | API gateway base URL |
| `DCM_GATEWAY_URL` | `http://localhost:8080/api/v1alpha1` | Control plane API base URL |
| `DCM_CONTAINER_SP_URL` | `http://localhost:8082/api/v1alpha1` | Container SP direct URL (requires published port) |
| `DCM_ACM_CLUSTER_SP_URL` | `http://localhost:8083/api/v1alpha1` | ACM Cluster SP direct URL (requires published port) |
| `DCM_NATS_URL` | `nats://localhost:4222` | NATS server URL for status event tests |
Expand All @@ -136,7 +136,7 @@ The test harness (`tests/run-e2e.sh`) supports additional flags for fine-grained
./tests/run-e2e.sh --skip-cli # Skip CLI binary resolution
./tests/run-e2e.sh --dcm-cli-path ~/bin/dcm # Use a specific CLI binary
./tests/run-e2e.sh --label-filter smoke # Run only smoke tests
./tests/run-e2e.sh --gateway-url http://... # Override gateway URL
./tests/run-e2e.sh --gateway-url http://... # Override control plane API URL
./tests/run-e2e.sh --junit-report results.xml # Write JUnit XML report

# Service provider tests
Expand Down
2 changes: 1 addition & 1 deletion providers/acm-cluster.conf
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ PROVIDER_LABEL="acm-cluster"
PROVIDER_FLAG="acm-cluster-service-provider"
PROVIDER_DESCRIPTION="Enable the ACM cluster service provider (requires OCP + ACM/MCE)"

# Compose integration — profile defined in api-gateway compose.yaml;
# Compose integration — profile defined in control-plane deploy/compose.yaml;
# override publishes the SP port to the host for direct API access.
COMPOSE_PROFILE="acm-cluster"
COMPOSE_OVERRIDE="tests/compose-acm-cluster-sp.yaml"
Expand Down
2 changes: 1 addition & 1 deletion providers/k8s-container.conf
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ PROVIDER_LABEL="k8s-container"
PROVIDER_FLAG="k8s-container-service-provider"
PROVIDER_DESCRIPTION="Enable the k8s container service provider"

# Compose integration — profile defined in api-gateway compose.yaml;
# Compose integration — profile defined in control-plane deploy/compose.yaml;
# override publishes the SP port to the host for direct API access.
COMPOSE_PROFILE="k8s-container"
COMPOSE_OVERRIDE="tests/compose-sp-test.yaml"
Expand Down
2 changes: 1 addition & 1 deletion providers/kubevirt.conf
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ PROVIDER_LABEL="kubevirt"
PROVIDER_FLAG="kubevirt-service-provider"
PROVIDER_DESCRIPTION="Enable the kubevirt service provider (requires OCP + CNV)"

# Compose integration — profile defined in api-gateway compose.yaml
# Compose integration — profile defined in control-plane deploy/compose.yaml
COMPOSE_PROFILE="kubevirt"
COMPOSE_OVERRIDE=""

Expand Down
2 changes: 1 addition & 1 deletion providers/three-tier-app-demo.conf
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ PROVIDER_LABEL="three-tier-app-demo"
PROVIDER_FLAG="three-tier-app-demo-service-provider"
PROVIDER_DESCRIPTION="Enable the three-tier app demo service provider (Pet Clinic)"

# Compose integration — not in api-gateway compose.yaml, uses override
# Compose integration — not in control-plane deploy/compose.yaml, uses override
COMPOSE_PROFILE=""
COMPOSE_OVERRIDE="tests/compose-three-tier-app-demo-sp.yaml"

Expand Down
Loading
Loading