fix(gcp): support scalable preconfigured custom-image hosts

docs/architecture/gce-machine-image-range-host-preflight-1896.md

@@ -13,6 +13,13 @@ participant container/account needed by the established setup and RDP broker
 paths. It does not add a scenario field, scenario-id branch, package executor,
 or new participant access channel.
 
+The original machine-image-only source decision is extended by #2382 for
+single-boot-disk hosts: the same `preconfigured-machine-host` capability may
+use an exact `projects/<project>/global/images/<name>` custom image. Image
+kind describes the Compute source; bootstrap capability describes guest
+readiness. A custom image is appropriate only when the complete preconfigured
+runtime is on its boot disk. Multi-disk captures retain the machine-image path.
+
 Machine-image profiles are administrator-managed runtime configuration. Their
 concrete image, container, account, and service-account values do not belong in
 catalog content. Legacy ranges may still use the bounded deployment-owned map
@@ -22,10 +29,10 @@ the latter is pinned with the adapter, pack digest, and range operation.
 
 ## Required Controls
 
-- Accept exactly one source per profile. A normal image profile uses
-  `source_image`; a preconfigured host uses the exact
+- Accept exactly one source per profile. A preconfigured host uses either an
+  exact project-qualified custom image in `source_image` or the exact
   `projects/<project>/global/machineImages/<name>` form. Families and inferred
-  names are not accepted for machine images.
+  names are not accepted for either preconfigured-host source.
 - Replace inherited metadata, SSH material, network interfaces, external-IP
   posture, labels, tags, machine type, and service account at clone time.
   Captured disks are the only inherited resources.
@@ -42,6 +49,9 @@ the latter is pinned with the adapter, pack digest, and range operation.
 - After create and on reconcile, set `autoDelete=true` on every attached disk.
   Destroy performs the same convergence before deleting the instance, so
   machine-image data disks cannot be orphaned.
+- A custom-image host creates one explicitly auto-deleting boot disk. Preserve
+  Shielded VM settings, nested virtualization, fresh metadata and SSH keys,
+  private networking, and an explicit runtime identity (or explicit absence).
 - Treat the image as owning its internal realization. The fixed volatile marker,
   running configured participant container, and host RDP listener are boot-
   liveness prerequisites only. Issue #1910 additionally requires the

docs/architecture/gcp-preconfigured-custom-image-host-preflight-2382.md

@@ -0,0 +1,76 @@
+# GCE Preconfigured Custom-Image Host Preflight (#2382)
+
+Status: implementation guidance
+
+## Decision and boundary
+
+`preconfigured-machine-host` describes guest realization and readiness; `image`
+versus `machine-image` describes the Compute Engine source. Permit the host
+capability with either an exact `projects/<project>/global/images/<name>` custom
+image or the existing exact machine-image reference. A custom image is suitable
+only when the complete runtime is on its one boot disk. Keep ordinary boot-image
+and machine-image behavior, and the existing generic RAES image/profile binding.
+Do not introduce a third image kind or a scenario-specific selector.
+This updates the machine-image-only source assumption in the #1896 preflight;
+its host security and readiness controls still apply.
+
+The machine-image source limit is six creations per source in 60 minutes; Google
+documents 20 instances per second for custom-image API/CLI creation. The latter
+still has image access, quota, and capacity constraints; it is not an unbounded
+fleet guarantee. See [machine-image restrictions](https://cloud.google.com/compute/docs/machine-images/create-instance-from-machine-image)
+and [custom-image creation](https://cloud.google.com/compute/docs/instances/create-vm-from-custom-image).
+
+## Cross-cutting gates and incumbents
+
+| Layer | Existing boundary and required guardrail |
+| --- | --- |
+| Authorization and administrator binding | CMS image-registry writes use `CMS_WRITE_PERMISSIONS`, `RaesImageMappingRegisterSerializer`, and `engine.services._raes_image.upsert_raes_image_mapping`; adapter target profiles use `cms.api.runtime_plugin_packs` and `shared.runtime_plugin_binding.RuntimeTargetImageProfile`. Keep image choice in those administrator-owned bindings, frozen with the operation or plugin pin. Pack authors cannot install executable adapters or choose a host by private identity. |
+| Shapes and validation | Reuse registry service validation, `shared.raes.operation_input_candidates`' closed projection, `shared.raes.image_policy.ResolvedImage`, `raes_gce_image`, `config._gce_image_keys`' closed JSON shape, and `config._gce_profile`'s profile validation. Validate source syntax by image kind, then validate complete host identity/readiness by bootstrap capability. Exact custom-image refs must exclude families, bare names, and URLs. Preserve the ordinary-image and AWS restrictions. The UI forms and API choices must express both independent dimensions. |
+| Plan and persistence | Reuse `GCERangeImageProfile`, `gce_image_profile_fingerprint`, deterministic `gcp_range_cell_plan`/`raes_gcp_plan` names and labels, registry rows, operation-input candidates, and pinned plugin bindings. No second profile schema, table, range status, or scenario field. An exact resource name is not a physical image ID: retain `source_image_id` checks for prepared artifacts through `raes_substrate_observation`, and do not claim name-only pins detect delete/recreate of an image. |
+| Compute request and security | Reuse `gcp_range_cell_resources.instance_resource` and `gcp_range_cells._insert_instance`. The custom-image host uses one boot disk with `source_image` and `auto_delete=true`, plus explicit `advanced_machine_features.enable_nested_virtualization=true`. Preserve current Shielded VM flags, fresh metadata/host key, private subnetwork and IP, no external IP, labels/tags, `can_ip_forward=false`, and the explicit selected service-account list or explicit empty list. Never inherit machine-image identity or metadata. Keep the existing host identity pool where the legacy path requires it; do not infer a new role or broaden IAM. |
+| Guest and access | Reuse `instance_orchestrator`, `plans.preconfigured_machine_host`, `raes_guest_plan.assert_management_login_separate`, `gcp_range_cell_outputs`, guest secret ops, private SSH host-key pinning, and declared participant access. A custom-image host follows the same bounded liveness, credential installation, and `participant-readiness/v1` canary checks; it skips ordinary guest bootstrap. The fixed canary receives only validated container/user/contract/digest arguments, never a profile-supplied command or secret in argv. A bootable image alone does not establish readiness. |
+| Reconcile and cleanup | Both `gcp_range_cells._ensure_instance` and `raes_gcp_apply._ensure_raes_instance` must reject an existing deterministic VM with conflicting range ownership or image/profile labels before returning its host key. The present keyed legacy check skips empty RAES image keys, so it cannot be the sole guard. Reuse shared binding checks and `gcp_range_cell_destroy`/`raes_gcp_destroy`; keep the custom-image boot disk `auto_delete=true` on create, adoption, and destroy. Machine-image disk convergence remains in place. A late ownership conflict must preserve the foreign VM and release only resources journaled as created by this apply attempt. Do not silently adopt a VM after an ambiguous insert result. |
+| Errors and observability | Preserve `RaesImageMappingError`, `RaesOperationInputError`, provisioner plan errors, `shared.api.errors.api_error_response`, `log_redact.safe_log_fingerprint`, and `terraform_ops._safe_failure_message`. Fail at the owning validator before cloud mutation where possible. Log bounded identifiers/operation context; never log request metadata, SSH material, credentials, or raw provider payloads. The failure-message boundary truncates but does not redact, so new exceptions must contain safe text. |
+
+The only runtime configuration transport implicated by a legacy keyed profile is
+`GCP_RANGE_IMAGE_KEY_PROFILES_JSON`: `shifter/installation/runtime_inventory_gcp.py`,
+`scripts/gcp/render_runtime_env.py`, `config._gce_image_keys`, and the provisioner
+Job admission allowlists in `platform/k8s/gcp/base` and the Helm template already
+carry that variable. Keep the change inside its existing closed shape. A new
+environment variable would have to pass each of those gates and the platform
+environment manifest; it is unnecessary here. The provider call uses the SDK in
+process, so no image reference, token, or credential belongs in process argv.
+
+## Gotchas and verification boundary
+
+- `image_kind == "image"` currently means ordinary bootstrap in the registry
+  service, operation-input parser, plugin profile validator, and both UI forms.
+  Update those gates together. Do not key host readiness or nested virtualization
+  solely on `source_machine_image`.
+- The RAES path has an empty `image_key`; its existing legacy drift check returns
+  early. Verify range ownership and the full profile fingerprint on reconcile
+  before adopting a preconfigured image-backed host.
+- The current RAES GCE apply and existing-cell activation paths do not run the
+  fixed participant canary before terminal readiness. Composition verification
+  is a different proof. Route both preconfigured source forms through the same
+  bounded canary, or reject that capability on any path lacking it. A host image
+  must also boot under the retained Shielded VM settings on a machine type that
+  supports nested virtualization; validation cannot assume the bake has this.
+- Keep `image_kind` as the source discriminator and `bootstrap_capability` as the
+  guest contract. The next boot-disk-backed capability should pass through the
+  same profile/request seam without another image kind or copied workflow.
+- Synthetic evidence must cover registry and plugin validation, closed operation
+  input, exact reference rejection, request shape and explicit nested feature,
+  RAES realization/reconcile and conflicting-VM rejection, boot-disk ownership,
+  readiness routing, and ordinary-image/machine-image regressions.
+
+## Non-goals and anti-patterns
+
+No image bake or promotion pipeline, extra disk support for custom-image hosts,
+global retry policy for machine-image throttling, new participant channel, new
+public network exposure, new service-account privilege, pack-specific code, or
+new persistence or general exception hierarchy. A typed ownership conflict may
+protect foreign VMs during partial cleanup. No fallback from a failed exact image
+to a family, another image, or a machine image. Existing ADR boundaries on
+administrator binding, range lifecycle, and participant access remain intact;
+this note clarifies their intersection without changing an ADR.

shifter/engine/provisioner/config/_gce_profile.py

@@ -125,6 +125,7 @@ def gce_image_profile_fingerprint(profile: GCERangeImageProfile) -> str:
 _GCE_MACHINE_IMAGE_REFERENCE_RE = re.compile(
     rf"^(?:(?:https://[^/]+/compute/(?:v1|beta)/)?)projects/{_GCE_PROJECT}/global/machineImages/{_GCE_NAME}$"
 )
+_GCE_EXACT_IMAGE_REFERENCE_RE = re.compile(rf"^projects/{_GCE_PROJECT}/global/images/{_GCE_NAME}$")
 _GCE_LINUX_USERNAME_RE = re.compile(r"[a-z_][a-z0-9_-]{0,31}")
 _GCE_CONTAINER_NAME_RE = re.compile(r"[A-Za-z0-9][A-Za-z0-9_.-]{0,127}")
 
@@ -192,8 +193,10 @@ def _validate_preconfigured_machine_profile(prefix: str, profile: GCERangeImageP
             prefix, profile, (profile.participant_container_name, profile.participant_username, *readiness_fields)
         )
         return
-    if not profile.source_machine_image:
-        raise RuntimeError(f"{prefix} preconfigured-machine-host requires source_machine_image")
+    if not _profile_has_source(profile):
+        raise RuntimeError(f"{prefix} preconfigured-machine-host requires a source image")
+    if profile.source_image and not _GCE_EXACT_IMAGE_REFERENCE_RE.fullmatch(profile.source_image):
+        raise RuntimeError(f"{prefix} preconfigured-machine-host requires an exact custom-image reference")
     if not all(identity_fields):
         raise RuntimeError(
             f"{prefix} preconfigured-machine-host requires participant_container_name, "

shifter/engine/provisioner/gcp_range_cell_host_binding.py

@@ -0,0 +1,58 @@
+"""Binding checks for deterministic GCE participant hosts."""
+
+from __future__ import annotations
+
+from config import GCE_BOOTSTRAP_PRECONFIGURED_MACHINE_HOST
+from gcp_range_cell_naming import _label_value
+from gcp_range_cell_resources import HOST_PUBLIC_KEY_METADATA_KEY
+from gcp_range_cell_types import InstancePlan, RangeCellPlan
+
+
+class GCEInstanceBindingError(RuntimeError):
+    """An existing deterministic VM does not belong to this range profile."""
+
+
+def _host_public_key_from_instance(existing: object) -> str:
+    """Recover the provisioner-issued host key instead of minting a mismatched one."""
+    metadata = getattr(existing, "metadata", None)
+    for item in getattr(metadata, "items", None) or []:
+        if getattr(item, "key", None) == HOST_PUBLIC_KEY_METADATA_KEY:
+            return str(getattr(item, "value", "") or "")
+    return ""
+
+
+def _existing_label(existing: object, key: str) -> str:
+    """Read one label from a dict-like Compute instance response."""
+    labels = getattr(existing, "labels", None)
+    getter = getattr(labels, "get", None)
+    if callable(getter):
+        return str(getter(key, "") or "")
+    return ""
+
+
+def _assert_instance_image_binding(existing: object, instance: InstancePlan) -> None:
+    """Reject a keyed deterministic VM whose recorded profile differs from the plan."""
+    expected_key = instance["image_key"]
+    if not expected_key:
+        return
+    actual_key = _existing_label(existing, "image-key")
+    actual_profile = _existing_label(existing, "image-profile")
+    if actual_key != expected_key or actual_profile != instance["image_profile_fingerprint"]:
+        raise RuntimeError(
+            "Existing GCE range instance has an image-profile binding that differs from the current plan; "
+            f"ami_key={expected_key!r}. Recreate the range instead of reusing the drifted instance."
+        )
+
+
+def _assert_preconfigured_host_binding(existing: object, plan: RangeCellPlan, instance: InstancePlan) -> None:
+    """Never adopt a participant host with different ownership or image policy."""
+    if instance["profile"].bootstrap_capability != GCE_BOOTSTRAP_PRECONFIGURED_MACHINE_HOST:
+        return
+    expected = {
+        "managed-by": plan["labels"]["managed-by"],
+        "range-id": plan["labels"]["range-id"],
+        "image-key": _label_value(instance["image_key"] or "default"),
+        "image-profile": instance["image_profile_fingerprint"],
+    }
+    if any(_existing_label(existing, key) != value for key, value in expected.items()):
+        raise GCEInstanceBindingError("Existing GCE participant host has a conflicting range or image-profile binding")

shifter/engine/provisioner/gcp_range_cell_outputs.py

@@ -21,8 +21,8 @@ class InstanceCredentials:
     host_public_key: str = ""
 
 
-def _machine_image_output(instance: InstancePlan) -> ResourceDict:
-    """Render the machine-image-only fields of a preconfigured range host."""
+def _preconfigured_host_output(instance: InstancePlan) -> ResourceDict:
+    """Render participant-host fields independently of the GCE source kind."""
     return {
         "gcp_source_machine_image": instance["profile"].source_machine_image,
         "gcp_participant_container_name": instance["profile"].participant_container_name,
@@ -103,8 +103,8 @@ def instance_output(
         "gcp_bootstrap_capability": instance["profile"].bootstrap_capability,
         "gcp_service_account_email": _service_account_output(instance, config),
     }
-    if instance["profile"].source_machine_image:
-        output.update(_machine_image_output(instance))
+    if instance["profile"].bootstrap_capability == GCE_BOOTSTRAP_PRECONFIGURED_MACHINE_HOST:
+        output.update(_preconfigured_host_output(instance))
     # The image's declared Guacamole SFTP root travels as realized per-instance
     # metadata (#375) so Mission Control consumes it instead of an OS map. Emitted
     # only when the profile declares one; a blank profile emits no key so the

shifter/engine/provisioner/gcp_range_cell_resources.py

@@ -227,12 +227,11 @@ def instance_resource(
         "deletion_protection": False,
         "can_ip_forward": False,
     }
-    if profile.source_machine_image:
-        # The machine image supplies every captured disk. Network, metadata,
-        # identity, labels, tags, machine type, and external-IP posture are all
-        # explicitly replaced by the body above.
+    if profile.bootstrap_capability == "preconfigured-machine-host":
         body["advanced_machine_features"] = {"enable_nested_virtualization": True}
-    else:
+    # A machine image supplies its captured disks; a custom image needs an
+    # explicitly owned boot disk with the same host hardening.
+    if not profile.source_machine_image:
         body["disks"] = [
             {
                 "boot": True,
@@ -259,9 +258,8 @@ def instance_resource(
                 "scopes": list(config.service_account_scopes),
             }
         ]
-    elif profile.source_machine_image:
-        # An omitted field inherits the captured machine-image identity. Send an
-        # explicit empty list when this range node has no authorized runtime
-        # identity so the bake-time service account is never attached.
+    elif profile.source_machine_image or profile.bootstrap_capability == "preconfigured-machine-host":
+        # Do not inherit a bake-time machine-image identity. An explicit empty
+        # list also records the administrator's choice for a custom-image host.
         body["service_accounts"] = []
     return body