cluster is not created

[marredowl]

Describe the bug

The cluster is not created.

To Reproduce

Steps to reproduce the behavior:

1. Performed the initial setup based on the README contents.
2. Registered Proxmox VE credentials under Cloud Credentials.
3. Configured the cluster under Clusters.
4. see error

Expected behavior

The cluster should be successfully created, as shown in the YouTube video.

Versions

• Rancher Machine Driver: 1.1.0
• Proxmox VE: 8.4.0 and 9.0.6
• Rancher: v2.12 and v2.13 on docker container

Both Proxmox VE and Rancher have been tested with multiple versions.

Additional context

• I am using certificates issued by my Proxmox and Rancher private CA.
• Rancher is running using Docker Compose.

   ---
   services:
   rancher:
       container_name: rancher
       image: rancher/rancher:v2.12-head
       privileged: true
       ports:
       - 80:80
       - 443:443
       extra_hosts:
       - "pve.sandbox.internal:10.32.255.1"
       - "rancher.sandbox.internal:127.0.0.1"
       volumes:
       - "./data/rancher:/var/lib/rancher"
       - "./certs/privateCA:/container/certs"
       - "./certs/rancher.key:/etc/rancher/ssl/key.pem"
       - "./certs/rancher.crt:/etc/rancher/ssl/cert.pem"
       - "./certs/privateCA/myRootCA.crt:/etc/rancher/ssl/cacerts.pem"
       environment:
       SSL_CERT_DIR: /container/certs

• I checked both cases—with and without bringing my own certificate—but it failed.

logs

rancher GUI

The error message displayed in the Rancher GUI is as follows:

Failed creating server [fleet-default/sandbox-internal-control-plane-gljs8-5wxpk] of kind (PveMachine) for machine sandbox-internal-control-plane-gljs8-5wxpk in infrastructure provider: CreateError: Downloading driver from https://rancher.sandbox.internal/assets/docker-machine-driver-pve Doing /etc/rancher/ssl Curl failed with error code 6 ls: docker-machine-driver-*: No such file or directory downloaded file failed sha256 checksum download of driver from https://rancher.sandbox.internal/assets/docker-machine-driver-pve failed

rancher container

The logs from the Rancher container are as follows:

click to expand

rancher  | 2025/09/16 14:41:28 [ERROR] error in transform: failed to find status.conditions block in cluster c-m-ckmx8n2g
rancher  | 2025/09/16 14:41:28 [INFO] [mgmt-cluster-rbac-delete] Creating namespace c-m-ckmx8n2g
rancher  | 2025/09/16 14:41:28 [ERROR] error in transform: failed to find status.conditions block in cluster c-m-ckmx8n2g
rancher  | 2025/09/16 14:41:28 [INFO] [mgmt-cluster-rbac-delete] Creating Default project for cluster c-m-ckmx8n2g
rancher  | 2025/09/16 14:41:28 [INFO] [mgmt-project-rbac-create] Creating namespace c-m-ckmx8n2g-p-ktk42
rancher  | 2025/09/16 14:41:28 [INFO] [mgmt-project-rbac-create] Creating creator projectRoleTemplateBinding for user user-sl6pr for project p-ktk42
rancher  | 2025/09/16 14:41:28 [ERROR] defaultSvcAccountHandler: Sync: error handling default ServiceAccount of namespace key=c-m-ckmx8n2g, err=Operation cannot be fulfilled on namespaces "c-m-ckmx8n2g": the object has been modified; please apply your changes to the latest version and try again
rancher  | 2025/09/16 14:41:28 [INFO] [mgmt-project-rbac-create] Setting InitialRolesPopulated condition on project p-ktk42
rancher  | 2025/09/16 14:41:28 [ERROR] defaultSvcAccountHandler: Sync: error handling default ServiceAccount of namespace key=c-m-ckmx8n2g, err=Operation cannot be fulfilled on namespaces "c-m-ckmx8n2g": the object has been modified; please apply your changes to the latest version and try again
rancher  | 2025/09/16 14:41:28 [INFO] [mgmt-project-rbac-create] Updating project p-ktk42
rancher  | 2025/09/16 14:41:28 [INFO] [mgmt-cluster-rbac-delete] Creating System project for cluster c-m-ckmx8n2g
rancher  | 2025/09/16 14:41:28 [INFO] [mgmt-auth-prtb-controller] Creating role/clusterRole p-ktk42-projectowner
rancher  | 2025/09/16 14:41:28 [INFO] [mgmt-auth-prtb-controller] Creating roleBinding for membership in project p-ktk42 for subject user-sl6pr
rancher  | 2025/09/16 14:41:28 [INFO] [mgmt-cluster-rbac-delete] Updating cluster c-m-ckmx8n2g
rancher  | 2025/09/16 14:41:28 [INFO] Skipping handler for clusterrepo rancher-charts. NumberOfRetries is 0, MaxRetry is 3, ClusterRepo Generation is 1, ObservedGeneration is 1, LastUpdated plus interval is 2025-09-16 15:35:01 +0000 UTC, now is 2025-09-16 14:41:28.4724389 +0000 UTC
rancher  | 2025/09/16 14:41:28 [INFO] [mgmt-project-rbac-create] Creating namespace c-m-ckmx8n2g-p-zpqcn
rancher  | 2025/09/16 14:41:28 [INFO] [mgmt-auth-prtb-controller] Creating role/clusterRole c-m-ckmx8n2g-clustermember
rancher  | 2025/09/16 14:41:28 [ERROR] error in transform: failed to find status.conditions block in cluster c-m-ckmx8n2g
rancher  | 2025/09/16 14:41:28 [INFO] [mgmt-project-rbac-create] Updating project p-ktk42
rancher  | 2025/09/16 14:41:28 [INFO] [mgmt-auth-prtb-controller] Creating clusterRoleBinding for membership in cluster c-m-ckmx8n2g for subject user-sl6pr
rancher  | 2025/09/16 14:41:28 [INFO] [mgmt-project-rbac-create] Creating creator projectRoleTemplateBinding for user user-sl6pr for project p-zpqcn
rancher  | 2025/09/16 14:41:28 [ERROR] defaultSvcAccountHandler: Sync: error handling default ServiceAccount of namespace key=c-m-ckmx8n2g-p-ktk42, err=Operation cannot be fulfilled on namespaces "c-m-ckmx8n2g-p-ktk42": the object has been modified; please apply your changes to the latest version and try again
rancher  | 2025/09/16 14:41:28 [INFO] [mgmt-auth-prtb-controller] Creating role project-owner in namespace c-m-ckmx8n2g-p-ktk42
rancher  | 2025/09/16 14:41:28 [INFO] [mgmt-project-rbac-create] Setting InitialRolesPopulated condition on project p-zpqcn
rancher  | 2025/09/16 14:41:28 [INFO] [mgmt-auth-prtb-controller] Creating role admin in namespace c-m-ckmx8n2g-p-ktk42
rancher  | 2025/09/16 14:41:28 [ERROR] defaultSvcAccountHandler: Sync: error handling default ServiceAccount of namespace key=c-m-ckmx8n2g-p-ktk42, err=Operation cannot be fulfilled on namespaces "c-m-ckmx8n2g-p-ktk42": the object has been modified; please apply your changes to the latest version and try again
rancher  | 2025/09/16 14:41:28 [ERROR] [rkecluster] fleet-default/sandbox-internal: error getting CAPI cluster no matching controller owner ref
rancher  | 2025/09/16 14:41:28 [ERROR] error syncing 'fleet-default/sandbox-internal': handler rke-cluster: no matching controller owner ref, requeuing
rancher  | 2025/09/16 14:41:28 [ERROR] [rkecluster] fleet-default/sandbox-internal: error getting CAPI cluster no matching controller owner ref
rancher  | 2025/09/16 14:41:28 [ERROR] error syncing 'fleet-default/sandbox-internal': handler rke-cluster: no matching controller owner ref, requeuing
rancher  | 2025/09/16 14:41:28 [INFO] [mgmt-auth-prtb-controller] Creating role/clusterRole p-zpqcn-projectowner
rancher  | 2025/09/16 14:41:28 [INFO] [mgmt-auth-prtb-controller] Creating roleBinding for subject user-sl6pr with role project-owner in namespace c-m-ckmx8n2g-p-ktk42
rancher  | 2025/09/16 14:41:28 [ERROR] error in transform: failed to find status.conditions block in cluster c-m-ckmx8n2g
rancher  | 2025/09/16 14:41:28 [INFO] [mgmt-project-rbac-create] Updating project p-zpqcn
rancher  | 2025/09/16 14:41:28 [ERROR] [rkecluster] fleet-default/sandbox-internal: error getting CAPI cluster no matching controller owner ref
rancher  | 2025/09/16 14:41:28 [ERROR] error syncing 'fleet-default/sandbox-internal': handler rke-cluster: no matching controller owner ref, requeuing
rancher  | 2025/09/16 14:41:28 [INFO] [mgmt-auth-prtb-controller] Creating roleBinding for membership in project p-zpqcn for subject user-sl6pr
rancher  | 2025/09/16 14:41:28 [INFO] [mgmt-auth-prtb-controller] Creating roleBinding for subject user-sl6pr with role admin in namespace c-m-ckmx8n2g-p-ktk42
rancher  | 2025/09/16 14:41:28 [ERROR] defaultSvcAccountHandler: Sync: error handling default ServiceAccount of namespace key=c-m-ckmx8n2g-p-zpqcn, err=Operation cannot be fulfilled on namespaces "c-m-ckmx8n2g-p-zpqcn": the object has been modified; please apply your changes to the latest version and try again
rancher  | 2025/09/16 14:41:28 [INFO] [mgmt-auth-prtb-controller] Updating clusterRoleBinding crb-vsnjwbnwd6 for cluster membership in cluster c-m-ckmx8n2g for subject user-sl6pr
rancher  | 2025/09/16 14:41:28 [ERROR] [rkecluster] fleet-default/sandbox-internal: error getting CAPI cluster no matching controller owner ref
rancher  | 2025/09/16 14:41:28 [ERROR] error syncing 'fleet-default/sandbox-internal': handler rke-cluster: no matching controller owner ref, requeuing
rancher  | 2025/09/16 14:41:28 [INFO] [mgmt-project-rbac-create] Updating project p-zpqcn
rancher  | 2025/09/16 14:41:28 [INFO] [mgmt-auth-prtb-controller] Creating role project-owner in namespace c-m-ckmx8n2g-p-zpqcn
rancher  | 2025/09/16 14:41:28 [INFO] [mgmt-auth-prtb-controller] Creating role admin in namespace c-m-ckmx8n2g-p-zpqcn
rancher  | 2025/09/16 14:41:28 [ERROR] [rkecluster] fleet-default/sandbox-internal: error getting CAPI cluster no matching controller owner ref
rancher  | 2025/09/16 14:41:28 [ERROR] error syncing 'fleet-default/sandbox-internal': handler rke-cluster: no matching controller owner ref, requeuing
rancher  | I0916 14:41:28.684226      49 warnings.go:110] "Warning: unknown field \"spec.template.metadata.creationTimestamp\""
rancher  | 2025/09/16 14:41:28 [ERROR] defaultSvcAccountHandler: Sync: error handling default ServiceAccount of namespace key=c-m-ckmx8n2g-p-zpqcn, err=Operation cannot be fulfilled on namespaces "c-m-ckmx8n2g-p-zpqcn": the object has been modified; please apply your changes to the latest version and try again
rancher  | 2025/09/16 14:41:28 [ERROR] [planner] rkecluster fleet-default/sandbox-internal: error during plan processing: no matching controller owner ref
rancher  | 2025/09/16 14:41:28 [ERROR] error syncing 'fleet-default/sandbox-internal': handler planner: no matching controller owner ref, requeuing
rancher  | 2025/09/16 14:41:28 [ERROR] [planner] rkecluster fleet-default/sandbox-internal: error during plan processing: no matching controller owner ref
rancher  | 2025/09/16 14:41:28 [ERROR] error syncing 'fleet-default/sandbox-internal': handler planner: no matching controller owner ref, requeuing
rancher  | 2025/09/16 14:41:28 [INFO] [mgmt-auth-prtb-controller] Creating roleBinding for subject user-sl6pr with role project-owner in namespace c-m-ckmx8n2g-p-zpqcn
rancher  | 2025/09/16 14:41:28 [ERROR] [planner] rkecluster fleet-default/sandbox-internal: error during plan processing: no matching controller owner ref
rancher  | 2025/09/16 14:41:28 [ERROR] error syncing 'fleet-default/sandbox-internal': handler planner: no matching controller owner ref, requeuing
rancher  | 2025/09/16 14:41:28 [INFO] [mgmt-auth-prtb-controller] Creating roleBinding for subject user-sl6pr with role admin in namespace c-m-ckmx8n2g-p-zpqcn
rancher  | 2025/09/16 14:41:28 [ERROR] [planner] rkecluster fleet-default/sandbox-internal: error during plan processing: no matching controller owner ref
rancher  | 2025/09/16 14:41:28 [ERROR] error syncing 'fleet-default/sandbox-internal': handler planner: no matching controller owner ref, requeuing
rancher  | 2025/09/16 14:41:28 [ERROR] [planner] rkecluster fleet-default/sandbox-internal: error during plan processing: no matching controller owner ref
rancher  | 2025/09/16 14:41:28 [INFO] [mgmt-cluster-rbac-delete] Updating cluster c-m-ckmx8n2g
rancher  | 2025/09/16 14:41:28 [ERROR] error syncing 'fleet-default/sandbox-internal': handler planner: no matching controller owner ref, requeuing
rancher  | 2025/09/16 14:41:28 [INFO] [mgmt-cluster-rbac-delete] Creating creator clusterRoleTemplateBinding for user user-sl6pr for cluster c-m-ckmx8n2g
rancher  | 2025/09/16 14:41:28 [INFO] [mgmt-auth-crtb-controller] Creating role/clusterRole c-m-ckmx8n2g-clusterowner
rancher  | 2025/09/16 14:41:28 [INFO] [mgmt-auth-crtb-controller] Creating clusterRoleBinding for membership in cluster c-m-ckmx8n2g for subject user-sl6pr
rancher  | 2025/09/16 14:41:28 [ERROR] [rkecluster] fleet-default/sandbox-internal: error getting CAPI cluster no matching controller owner ref
rancher  | 2025/09/16 14:41:28 [ERROR] error syncing 'fleet-default/sandbox-internal': handler rke-cluster: no matching controller owner ref, requeuing
rancher  | 2025/09/16 14:41:28 [ERROR] [planner] rkecluster fleet-default/sandbox-internal: error during plan processing: no matching controller owner ref
rancher  | 2025/09/16 14:41:28 [ERROR] error syncing 'fleet-default/sandbox-internal': handler planner: no matching controller owner ref, requeuing
rancher  | 2025/09/16 14:41:28 [INFO] [mgmt-cluster-rbac-delete] Setting InitialRolesPopulated condition on cluster c-m-ckmx8n2g
rancher  | 2025/09/16 14:41:28 [INFO] [mgmt-cluster-rbac-delete] Updating cluster c-m-ckmx8n2g
rancher  | 2025/09/16 14:41:28 [INFO] [mgmt-auth-crtb-controller] Creating role cluster-owner in namespace c-m-ckmx8n2g
rancher  | 2025/09/16 14:41:28 [INFO] [mgmt-auth-crtb-controller] Creating roleBinding for subject user-sl6pr with role cluster-owner in namespace c-m-ckmx8n2g
rancher  | 2025/09/16 14:41:28 [INFO] [mgmt-auth-crtb-controller] Creating role cluster-owner in namespace c-m-ckmx8n2g-p-ktk42
rancher  | 2025/09/16 14:41:28 [INFO] [mgmt-auth-crtb-controller] Creating roleBinding for subject user-sl6pr with role cluster-owner in namespace c-m-ckmx8n2g-p-ktk42
rancher  | 2025/09/16 14:41:28 [INFO] [mgmt-auth-crtb-controller] Creating role cluster-owner in namespace c-m-ckmx8n2g-p-zpqcn
rancher  | 2025/09/16 14:41:28 [INFO] [mgmt-auth-crtb-controller] Creating roleBinding for subject user-sl6pr with role cluster-owner in namespace c-m-ckmx8n2g-p-zpqcn
rancher  | 2025/09/16 14:41:28 [INFO] [mgmt-cluster-rbac-delete] Updating cluster c-m-ckmx8n2g
rancher  | 2025/09/16 14:41:29 [INFO] [planner] rkecluster fleet-default/sandbox-internal: waiting for infrastructure ready
rancher  | 2025/09/16 14:41:29 [INFO] [planner] rkecluster fleet-default/sandbox-internal: waiting for infrastructure ready
rancher  | 2025/09/16 14:41:29 [INFO] [planner] rkecluster fleet-default/sandbox-internal: waiting for infrastructure ready
rancher  | 2025/09/16 14:41:29 [INFO] [planner] rkecluster fleet-default/sandbox-internal: waiting for at least one control plane, etcd, and worker node to be registered
rancher  | 2025/09/16 14:41:29 [INFO] [planner] rkecluster fleet-default/sandbox-internal: waiting for at least one control plane, etcd, and worker node to be registered
rancher  | 2025/09/16 14:41:29 [ERROR] [rkebootstrap] fleet-default/sandbox-internal-control-plane-gljs8-5wxpk: error getting machine by owner reference no matching controller owner ref
rancher  | 2025/09/16 14:41:29 [ERROR] error syncing 'fleet-default/sandbox-internal-control-plane-gljs8-5wxpk': handler rke-bootstrap: no matching controller owner ref, requeuing
rancher  | 2025/09/16 14:41:29 [ERROR] [rkebootstrap] fleet-default/sandbox-internal-control-plane-gljs8-5wxpk: error getting machine by owner reference no matching controller owner ref
rancher  | 2025/09/16 14:41:29 [ERROR] error syncing 'fleet-default/sandbox-internal-control-plane-gljs8-5wxpk': handler rke-bootstrap: no matching controller owner ref, requeuing
rancher  | 2025/09/16 14:41:29 [ERROR] [rkebootstrap] fleet-default/sandbox-internal-control-plane-gljs8-5wxpk: error getting machine by owner reference no matching controller owner ref
rancher  | 2025/09/16 14:41:29 [ERROR] error syncing 'fleet-default/sandbox-internal-control-plane-gljs8-5wxpk': handler rke-bootstrap: no matching controller owner ref, requeuing
rancher  | 2025/09/16 14:41:29 [ERROR] [machineprovision] fleet-default/sandbox-internal-control-plane-gljs8-5wxpk: error getting machine by owner reference: no matching controller owner ref
rancher  | 2025/09/16 14:41:29 [ERROR] error syncing 'fleet-default/sandbox-internal-control-plane-gljs8-5wxpk': handler machine-provision: no matching controller owner ref, requeuing
rancher  | 2025/09/16 14:41:29 [ERROR] [machineprovision] fleet-default/sandbox-internal-control-plane-gljs8-5wxpk: error getting machine by owner reference: no matching controller owner ref
rancher  | 2025/09/16 14:41:29 [ERROR] error syncing 'fleet-default/sandbox-internal-control-plane-gljs8-5wxpk': handler machine-provision: no matching controller owner ref, requeuing
rancher  | 2025/09/16 14:41:29 [ERROR] [machineprovision] fleet-default/sandbox-internal-control-plane-gljs8-5wxpk: error getting machine by owner reference: no matching controller owner ref
rancher  | 2025/09/16 14:41:29 [ERROR] error syncing 'fleet-default/sandbox-internal-control-plane-gljs8-5wxpk': handler machine-provision: no matching controller owner ref, requeuing
rancher  | 2025/09/16 14:41:29 [ERROR] [rkebootstrap] fleet-default/sandbox-internal-control-plane-gljs8-5wxpk: error getting machine by owner reference no matching controller owner ref
rancher  | 2025/09/16 14:41:29 [ERROR] error syncing 'fleet-default/sandbox-internal-control-plane-gljs8-5wxpk': handler rke-bootstrap: no matching controller owner ref, requeuing
rancher  | 2025/09/16 14:41:29 [ERROR] [machineprovision] fleet-default/sandbox-internal-control-plane-gljs8-5wxpk: error getting machine by owner reference: no matching controller owner ref
rancher  | 2025/09/16 14:41:29 [ERROR] error syncing 'fleet-default/sandbox-internal-control-plane-gljs8-5wxpk': handler machine-provision: no matching controller owner ref, requeuing
rancher  | 2025/09/16 14:41:29 [INFO] [planner] rkecluster fleet-default/sandbox-internal: waiting for at least one control plane, etcd, and worker node to be registered
rancher  | 2025/09/16 14:41:29 [ERROR] [rkebootstrap] fleet-default/sandbox-internal-control-plane-gljs8-5wxpk: error getting machine by owner reference no matching controller owner ref
rancher  | 2025/09/16 14:41:29 [ERROR] error syncing 'fleet-default/sandbox-internal-control-plane-gljs8-5wxpk': handler rke-bootstrap: no matching controller owner ref, requeuing
rancher  | 2025/09/16 14:41:29 [INFO] [planner] rkecluster fleet-default/sandbox-internal: waiting for at least one control plane, etcd, and worker node to be registered
rancher  | 2025/09/16 14:41:29 [ERROR] [machineprovision] fleet-default/sandbox-internal-control-plane-gljs8-5wxpk: error getting machine by owner reference: no matching controller owner ref
rancher  | 2025/09/16 14:41:29 [ERROR] error syncing 'fleet-default/sandbox-internal-control-plane-gljs8-5wxpk': handler machine-provision: no matching controller owner ref, requeuing
rancher  | 2025/09/16 14:41:29 [INFO] [planner] rkecluster fleet-default/sandbox-internal: waiting for at least one control plane, etcd, and worker node to be registered
rancher  | 2025/09/16 14:41:29 [INFO] [planner] rkecluster fleet-default/sandbox-internal: waiting for at least one control plane, etcd, and worker node to be registered
rancher  | 2025/09/16 14:41:29 [ERROR] [machineprovision] fleet-default/sandbox-internal-control-plane-gljs8-5wxpk: error getting machine by owner reference: no matching controller owner ref
rancher  | 2025/09/16 14:41:29 [ERROR] error syncing 'fleet-default/sandbox-internal-control-plane-gljs8-5wxpk': handler machine-provision: no matching controller owner ref, requeuing
rancher  | 2025/09/16 14:41:29 [INFO] [planner] rkecluster fleet-default/sandbox-internal: waiting for at least one control plane, etcd, and worker node to be registered
rancher  | 2025/09/16 14:41:29 [INFO] [planner] rkecluster fleet-default/sandbox-internal: waiting for at least one control plane, etcd, and worker node to be registered
rancher  | 2025/09/16 14:41:29 [INFO] [planner] rkecluster fleet-default/sandbox-internal: waiting for viable init node
rancher  | 2025/09/16 14:41:29 [INFO] [planner] rkecluster fleet-default/sandbox-internal: waiting for viable init node
rancher  | 2025/09/16 14:41:29 [INFO] EnsureSecretForServiceAccount: waiting for secret [fleet-default:sandbox-internal-control-plane-gljs8-5wxpk-machine-bootstramwmq7] for service account [fleet-default:sandbox-internal-control-plane-gljs8-5wxpk-machine-bootstrap] to be populated with token
rancher  | 2025/09/16 14:41:29 [INFO] [planner] rkecluster fleet-default/sandbox-internal: waiting for viable init node
rancher  | 2025/09/16 14:41:29 [INFO] EnsureSecretForServiceAccount: got the service account token for service account [fleet-default:sandbox-internal-control-plane-gljs8-5wxpk-machine-bootstrap] in 6.941009ms
rancher  | 2025/09/16 14:41:29 [INFO] [planner] rkecluster fleet-default/sandbox-internal: waiting for viable init node
rancher  | 2025/09/16 14:41:29 [ERROR] error syncing '_all_': handler user-controllers-controller: userControllersController: failed to set peers for key _all_: failed to start user controllers for cluster c-m-ckmx8n2g: ClusterUnavailable 503: cluster not found, requeuing
rancher  | 2025/09/16 14:41:29 [INFO] [planner] rkecluster fleet-default/sandbox-internal: waiting for viable init node
rancher  | 2025/09/16 14:41:29 [INFO] [planner] rkecluster fleet-default/sandbox-internal: waiting for viable init node
rancher  | 2025/09/16 14:41:29 [INFO] [planner] rkecluster fleet-default/sandbox-internal: waiting for viable init node
rancher  | 2025/09/16 14:41:29 [ERROR] error syncing 'c-m-ckmx8n2g': handler cluster-deploy: cluster context c-m-ckmx8n2g is unavailable, requeuing
rancher  | 2025/09/16 14:41:29 [ERROR] error syncing '_all_': handler user-controllers-controller: userControllersController: failed to set peers for key _all_: failed to start user controllers for cluster c-m-ckmx8n2g: ClusterUnavailable 503: cluster not found, requeuing
rancher  | 2025/09/16 14:41:29 [INFO] [planner] rkecluster fleet-default/sandbox-internal: waiting for viable init node
rancher  | 2025/09/16 14:41:29 [INFO] [planner] rkecluster fleet-default/sandbox-internal: waiting for viable init node
rancher  | 2025/09/16 14:41:29 [ERROR] error syncing 'c-m-ckmx8n2g': handler cluster-deploy: cluster context c-m-ckmx8n2g is unavailable, requeuing
rancher  | 2025/09/16 14:41:29 [INFO] [planner] rkecluster fleet-default/sandbox-internal: waiting for viable init node
rancher  | 2025/09/16 14:41:29 [ERROR] error syncing '_all_': handler user-controllers-controller: userControllersController: failed to set peers for key _all_: failed to start user controllers for cluster c-m-ckmx8n2g: ClusterUnavailable 503: cluster not found, requeuing
rancher  | 2025/09/16 14:41:29 [ERROR] error syncing 'c-m-ckmx8n2g': handler cluster-deploy: cluster context c-m-ckmx8n2g is unavailable, requeuing
rancher  | 2025/09/16 14:41:29 [INFO] [planner] rkecluster fleet-default/sandbox-internal: waiting for viable init node
rancher  | 2025/09/16 14:41:29 [ERROR] error syncing '_all_': handler user-controllers-controller: userControllersController: failed to set peers for key _all_: failed to start user controllers for cluster c-m-ckmx8n2g: ClusterUnavailable 503: cluster not found, requeuing
rancher  | 2025/09/16 14:41:29 [INFO] [planner] rkecluster fleet-default/sandbox-internal: waiting for viable init node
rancher  | 2025/09/16 14:41:30 [INFO] [planner] rkecluster fleet-default/sandbox-internal: waiting for viable init node
rancher  | 2025/09/16 14:41:30 [ERROR] error syncing 'c-m-ckmx8n2g': handler cluster-deploy: cluster context c-m-ckmx8n2g is unavailable, requeuing
rancher  | 2025/09/16 14:41:30 [ERROR] error syncing '_all_': handler user-controllers-controller: userControllersController: failed to set peers for key _all_: failed to start user controllers for cluster c-m-ckmx8n2g: ClusterUnavailable 503: cluster not found, requeuing
rancher  | 2025/09/16 14:41:31 [ERROR] error syncing '_all_': handler user-controllers-controller: userControllersController: failed to set peers for key _all_: failed to start user controllers for cluster c-m-ckmx8n2g: ClusterUnavailable 503: cluster not found, requeuing
rancher  | 2025/09/16 14:41:32 [ERROR] 2025/09/16 14:41:32 http: TLS handshake error from 10.32.10.2:64440: remote error: tls: unknown certificate
rancher  | 2025/09/16 14:41:32 [INFO] CacheFor STARTS creating informer for rke-machine.cattle.io/v1, Kind=PveMachineTemplate
rancher  | 2025/09/16 14:41:32 [INFO] Started SQL cache garbage collection for rke-machine.cattle.io_v1_PveMachineTemplate (interval=15m0s, keep=1000)
rancher  | 2025/09/16 14:41:32 [INFO] CacheFor IS DONE creating informer for rke-machine.cattle.io/v1, Kind=PveMachineTemplate (took 103.464615ms)
rancher  | 2025/09/16 14:41:33 [INFO] [planner] rkecluster fleet-default/sandbox-internal: waiting for viable init node
rancher  | 2025/09/16 14:41:33 [ERROR] error syncing 'c-m-ckmx8n2g': handler cluster-deploy: cluster context c-m-ckmx8n2g is unavailable, requeuing
rancher  | 2025/09/16 14:41:34 [ERROR] error syncing '_all_': handler user-controllers-controller: userControllersController: failed to set peers for key _all_: failed to start user controllers for cluster c-m-ckmx8n2g: ClusterUnavailable 503: cluster not found, requeuing
rancher  | 2025/09/16 14:41:39 [INFO] [machineprovision] fleet-default/sandbox-internal-control-plane-gljs8-5wxpk: reconciling machine job
rancher  | 2025/09/16 14:41:39 [INFO] [machineprovision] fleet-default/sandbox-internal-control-plane-gljs8-5wxpk: reconciling machine job
rancher  | 2025/09/16 14:41:39 [INFO] [machineprovision] fleet-default/sandbox-internal-control-plane-gljs8-5wxpk: reconciling machine job
rancher  | 2025/09/16 14:41:39 [INFO] [planner] rkecluster fleet-default/sandbox-internal: waiting for viable init node
rancher  | 2025/09/16 14:41:39 [INFO] [machineprovision] fleet-default/sandbox-internal-control-plane-gljs8-5wxpk: reconciling machine job
rancher  | 2025/09/16 14:41:39 [ERROR] error syncing 'c-m-ckmx8n2g': handler cluster-deploy: cluster context c-m-ckmx8n2g is unavailable, requeuing
rancher  | 2025/09/16 14:41:39 [INFO] [machineprovision] fleet-default/sandbox-internal-control-plane-gljs8-5wxpk: reconciling machine job
rancher  | 2025/09/16 14:41:39 [INFO] [machineprovision] fleet-default/sandbox-internal-control-plane-gljs8-5wxpk: reconciling machine job
rancher  | 2025/09/16 14:41:39 [INFO] [machineprovision] fleet-default/sandbox-internal-control-plane-gljs8-5wxpk: reconciling machine job
rancher  | 2025/09/16 14:41:39 [INFO] [planner] rkecluster fleet-default/sandbox-internal: waiting for viable init node
rancher  | 2025/09/16 14:41:39 [INFO] [machineprovision] fleet-default/sandbox-internal-control-plane-gljs8-5wxpk: reconciling machine job
rancher  | 2025/09/16 14:41:44 [ERROR] error syncing '_all_': handler user-controllers-controller: userControllersController: failed to set peers for key _all_: failed to start user controllers for cluster c-m-ckmx8n2g: ClusterUnavailable 503: cluster not found, requeuing
rancher  | 2025/09/16 14:41:49 [INFO] [machineprovision] fleet-default/sandbox-internal-control-plane-gljs8-5wxpk: reconciling machine job
rancher  | 2025/09/16 14:41:50 [INFO] [machineprovision] fleet-default/sandbox-internal-control-plane-gljs8-5wxpk: reconciling machine job
rancher  | 2025/09/16 14:41:50 [INFO] Skipping handler for clusterrepo rancher-charts. NumberOfRetries is 0, MaxRetry is 3, ClusterRepo Generation is 1, ObservedGeneration is 1, LastUpdated plus interval is 2025-09-16 15:35:01 +0000 UTC, now is 2025-09-16 14:41:50.845826892 +0000 UTC
rancher  | 2025/09/16 14:41:52 [INFO] [machineprovision] fleet-default/sandbox-internal-control-plane-gljs8-5wxpk: reconciling machine job
rancher  | 2025/09/16 14:41:52 [INFO] [machineprovision] fleet-default/sandbox-internal-control-plane-gljs8-5wxpk: reconciling machine job
rancher  | 2025/09/16 14:41:52 [INFO] [planner] rkecluster fleet-default/sandbox-internal: waiting for viable init node
rancher  | 2025/09/16 14:41:52 [INFO] [planner] rkecluster fleet-default/sandbox-internal: waiting for viable init node
rancher  | 2025/09/16 14:41:53 [INFO] [machineprovision] fleet-default/sandbox-internal-control-plane-gljs8-5wxpk: Failed to create infrastructure for machine sandbox-internal-control-plane-gljs8-5wxpk, deleting and recreating...
rancher  | 2025/09/16 14:41:53 [INFO] [rkebootstrap] No associated node found for machine fleet-default/sandbox-internal-control-plane-gljs8-5wxpk in cluster sandbox-internal, ensuring machine pre-terminate annotation is removed
rancher  | 2025/09/16 14:41:53 [INFO] [planner] rkecluster fleet-default/sandbox-internal: force deleting etcd machine fleet-default/sandbox-internal-control-plane-gljs8-5wxpk as cluster was not sane and machine was deleting
rancher  | 2025/09/16 14:41:53 [ERROR] [planner] rkecluster fleet-default/sandbox-internal: error during plan processing: Operation cannot be fulfilled on machines.cluster.x-k8s.io "sandbox-internal-control-plane-gljs8-5wxpk": the object has been modified; please apply your changes to the latest version and try again
rancher  | 2025/09/16 14:41:53 [INFO] [planner] rkecluster fleet-default/sandbox-internal: force deleting etcd machine fleet-default/sandbox-internal-control-plane-gljs8-5wxpk as cluster was not sane and machine was deleting
rancher  | 2025/09/16 14:41:53 [INFO] [rkebootstrap] No associated node found for machine fleet-default/sandbox-internal-control-plane-gljs8-5wxpk in cluster sandbox-internal, ensuring machine pre-terminate annotation is removed
rancher  | 2025/09/16 14:41:53 [INFO] [planner] rkecluster fleet-default/sandbox-internal: waiting for all etcd machines to be deleted
rancher  | 2025/09/16 14:41:53 [INFO] [planner] rkecluster fleet-default/sandbox-internal: force deleting etcd machine fleet-default/sandbox-internal-control-plane-gljs8-5wxpk as cluster was not sane and machine was deleting
rancher  | 2025/09/16 14:41:53 [ERROR] [planner] rkecluster fleet-default/sandbox-internal: error during plan processing: Operation cannot be fulfilled on machines.cluster.x-k8s.io "sandbox-internal-control-plane-gljs8-5wxpk": the object has been modified; please apply your changes to the latest version and try again
rancher  | 2025/09/16 14:41:53 [INFO] [planner] rkecluster fleet-default/sandbox-internal: force deleting etcd machine fleet-default/sandbox-internal-control-plane-gljs8-5wxpk as cluster was not sane and machine was deleting
rancher  | 2025/09/16 14:41:53 [INFO] [planner] rkecluster fleet-default/sandbox-internal: waiting for all etcd machines to be deleted
rancher  | 2025/09/16 14:41:53 [INFO] [planner] rkecluster fleet-default/sandbox-internal: force deleting etcd machine fleet-default/sandbox-internal-control-plane-gljs8-5wxpk as cluster was not sane and machine was deleting
rancher  | 2025/09/16 14:41:53 [ERROR] [planner] rkecluster fleet-default/sandbox-internal: error during plan processing: Operation cannot be fulfilled on machines.cluster.x-k8s.io "sandbox-internal-control-plane-gljs8-5wxpk": the object has been modified; please apply your changes to the latest version and try again
rancher  | 2025/09/16 14:41:53 [INFO] [planner] rkecluster fleet-default/sandbox-internal: force deleting etcd machine fleet-default/sandbox-internal-control-plane-gljs8-5wxpk as cluster was not sane and machine was deleting
rancher  | 2025/09/16 14:41:53 [INFO] [planner] rkecluster fleet-default/sandbox-internal: waiting for all etcd machines to be deleted
rancher  | 2025/09/16 14:41:53 [INFO] [planner] rkecluster fleet-default/sandbox-internal: force deleting etcd machine fleet-default/sandbox-internal-control-plane-gljs8-5wxpk as cluster was not sane and machine was deleting
rancher  | 2025/09/16 14:41:53 [ERROR] error syncing 'c-m-ckmx8n2g': handler cluster-deploy: cluster context c-m-ckmx8n2g is unavailable, requeuing
rancher  | 2025/09/16 14:41:53 [INFO] [planner] rkecluster fleet-default/sandbox-internal: waiting for all etcd machines to be deleted
rancher  | 2025/09/16 14:41:54 [INFO] [machineprovision] fleet-default/sandbox-internal-control-plane-gljs8-5wxpk: reconciling machine job
rancher  | 2025/09/16 14:41:54 [INFO] [planner] rkecluster fleet-default/sandbox-internal: force deleting etcd machine fleet-default/sandbox-internal-control-plane-gljs8-5wxpk as cluster was not sane and machine was deleting
rancher  | 2025/09/16 14:41:54 [INFO] [planner] rkecluster fleet-default/sandbox-internal: waiting for all etcd machines to be deleted
rancher  | 2025/09/16 14:41:54 [INFO] [machineprovision] fleet-default/sandbox-internal-control-plane-gljs8-5wxpk: reconciling machine job
rancher  | 2025/09/16 14:41:54 [INFO] [machineprovision] fleet-default/sandbox-internal-control-plane-gljs8-5wxpk: reconciling machine job
rancher  | 2025/09/16 14:41:54 [INFO] [machineprovision] fleet-default/sandbox-internal-control-plane-gljs8-5wxpk: reconciling machine job
rancher  | 2025/09/16 14:41:54 [INFO] [machineprovision] fleet-default/sandbox-internal-control-plane-gljs8-5wxpk: reconciling machine job
rancher  | 2025/09/16 14:41:54 [INFO] [machineprovision] fleet-default/sandbox-internal-control-plane-gljs8-5wxpk: reconciling machine job
rancher  | 2025/09/16 14:41:55 [INFO] [planner] rkecluster fleet-default/sandbox-internal: force deleting etcd machine fleet-default/sandbox-internal-control-plane-gljs8-5wxpk as cluster was not sane and machine was deleting
rancher  | 2025/09/16 14:41:55 [INFO] [planner] rkecluster fleet-default/sandbox-internal: waiting for all etcd machines to be deleted

[LukasK32]

From the error it's clear that the file from https://rancher.sandbox.internal/assets/docker-machine-driver-pve is invalid in some way as the checksum check fails. Either the file is corrupted or from a mismatched version in relation to the Helm chart.

You can also check the status of the driver in Cluster Management > Drivers > Node Drivers > Proxmox VE, it should be Active.

[EagleIJoe]

I am on v1.2.0-rc2 and I am running into the exact same error. Curl error 6 means "Could not resolve host. The given remote host was not resolved." according to https://curl.se/libcurl/c/libcurl-errors.html.

[Update] I fixed the hostname resolution error in the rancher cluster.