From 48040cf9082388bd48e11fcbd791100032c3460f Mon Sep 17 00:00:00 2001 From: Aditya Sharma Date: Thu, 6 Aug 2026 03:05:20 -0400 Subject: [PATCH 1/4] Serve dedicated cluster and endpoint discovery services. Replace the supervisor ADS endpoint with dedicated CDS and EDS services while retaining dynamic cluster generation, protocol validation, health checks, and ORCA policy configuration. Generated clusters point their endpoint subscriptions back to the same management cluster over EDS, leaving Envoy ADS available for another control plane. Endpoint assignments are only republished when they change. Update the control-plane fixture and guide for the split discovery topology, and verify that Envoy discovers both the dynamic cluster and its supervised Falcon workers. --- .dockerignore | 1 + context/getting-started.md | 55 ++++++++++++++-- context/index.yaml | 2 +- control-plane/backend.rb | 20 +++--- control-plane/envoy.yaml | 30 ++++----- control-plane/readme.md | 12 ++-- control-plane/test/envoy.rb | 15 +++-- gems.rb | 1 + guides/getting-started/readme.md | 55 ++++++++++++++-- lib/async/service/supervisor/envoy/monitor.rb | 46 ++++++++++---- readme.md | 2 +- releases.md | 3 + .../async/service/supervisor/envoy/monitor.rb | 63 +++++++++++++++++-- 13 files changed, 242 insertions(+), 63 deletions(-) diff --git a/.dockerignore b/.dockerignore index 81008d4..e4a58f0 100644 --- a/.dockerignore +++ b/.dockerignore @@ -4,3 +4,4 @@ gems.locked pkg external +examples/falcon/*.ipc diff --git a/context/getting-started.md b/context/getting-started.md index c1ea351..fe4344a 100644 --- a/context/getting-started.md +++ b/context/getting-started.md @@ -1,6 +1,6 @@ # Getting Started -This guide explains how to use `async-service-supervisor-envoy` to publish supervised worker endpoints to Envoy using xDS. +This guide explains how to use `async-service-supervisor-envoy` to publish supervised worker clusters and endpoints to Envoy using xDS. ## Installation @@ -16,10 +16,12 @@ The gem depends on `async-service-supervisor` and `async-grpc-xds`. `async-service-supervisor-envoy` provides: - - {ruby Async::Service::Supervisor::Envoy::Monitor} - A supervisor monitor that publishes worker endpoints through xDS. + - {ruby Async::Service::Supervisor::Envoy::Monitor} - A supervisor monitor that publishes worker clusters through CDS and their endpoints through EDS. - {ruby Async::Service::Supervisor::Envoy::Endpoint} - A small value object for endpoint state. -The monitor runs an xDS control plane endpoint. Envoy connects to it using ADS and receives CDS/EDS updates derived from supervisor worker state. +The monitor serves dedicated Cluster Discovery Service and Endpoint Discovery Service streams. It does not claim Envoy's Aggregated Discovery Service, so another control plane can use ADS for listeners, routes, and other configuration. + +CDS describes the logical services exposed by supervised workers, including their supported protocol, active health checks, and load-balancing policy. EDS supplies the concrete workers currently available for each service. ## Endpoint State @@ -62,11 +64,48 @@ Add the monitor to your supervisor environment: require "async/service/supervisor/envoy" Async::Service::Supervisor::Envoy::Monitor.new( - bind: "http://127.0.0.1:18000" + bind: "http://127.0.0.1:18000", + management_cluster: "xds_cluster" ) ``` -By default, workers are grouped into clusters by `state[:name]`. +By default, workers are grouped into clusters by `state[:name]`. `management_cluster` must match the static Envoy cluster used to reach the monitor. + +## Envoy Configuration + +Configure Envoy to obtain clusters from the monitor's dedicated CDS service. The monitor configures each discovered cluster to obtain its endpoints from the dedicated EDS service on the same management server: + +``` yaml +dynamic_resources: + cds_config: + resource_api_version: V3 + api_config_source: + api_type: GRPC + transport_api_version: V3 + grpc_services: + - envoy_grpc: + cluster_name: xds_cluster + +static_resources: + clusters: + - name: xds_cluster + connect_timeout: 1s + type: STRICT_DNS + http2_protocol_options: {} + load_assignment: + cluster_name: xds_cluster + endpoints: + - lb_endpoints: + - endpoint: + address: + socket_address: + address: 127.0.0.1 + port_value: 18000 +``` + +The bootstrap cluster name must match the monitor's `management_cluster`. Envoy uses it for independent CDS and EDS gRPC streams; no `ads_config` is required. + +If a static route refers to a cluster delivered by CDS, set `validate_clusters: false` on that route configuration. Envoy can then load the route before the cluster arrives and will begin routing once CDS and EDS have warmed it. ## Custom Mapping @@ -113,10 +152,14 @@ utilization_monitor = Async::Service::Supervisor::UtilizationMonitor.new(interva ] ``` -The supervisor utilization monitor manages each worker's shared-memory allocation and registration. The Envoy monitor samples it through `sample_by_worker`, combines each worker's `requests_total` counter with processor usage from `process-metrics`, and serves the resulting ORCA reports from the same HTTP/2 endpoint as ADS. It also configures each discovered cluster to use Envoy's client-side weighted-round-robin policy. +The supervisor utilization monitor manages each worker's shared-memory allocation and registration. The Envoy monitor samples it through `sample_by_worker`, combines each worker's `requests_total` counter with processor usage from `process-metrics`, and serves the resulting ORCA reports from the same HTTP/2 endpoint as CDS and EDS. It also configures each discovered cluster to use Envoy's client-side weighted-round-robin policy. The first sample establishes a baseline. Subsequent reports contain normalized `cpu_utilization` and `rps_fractional` values for each worker. Reports are removed immediately when a worker disconnects. +Workers are identified by the `hostname` published with each endpoint, which Envoy sends as the request authority when it opens an out-of-band reporting stream. Enabling ORCA therefore publishes one endpoint per worker rather than one per shared listener. + +The generated load-balancing policy uses the monitor's bind port and reporting interval. Envoy dials that port on each endpoint's own address, which reaches the monitor because it shares a network namespace with the workers. + Out-of-band ORCA requires: - Envoy 1.39 or later. diff --git a/context/index.yaml b/context/index.yaml index 5e12684..98e11ed 100644 --- a/context/index.yaml +++ b/context/index.yaml @@ -9,4 +9,4 @@ files: - path: getting-started.md title: Getting Started description: This guide explains how to use `async-service-supervisor-envoy` to - publish supervised worker endpoints to Envoy using xDS. + publish supervised worker clusters and endpoints to Envoy using xDS. diff --git a/control-plane/backend.rb b/control-plane/backend.rb index af84c76..1b24f60 100644 --- a/control-plane/backend.rb +++ b/control-plane/backend.rb @@ -30,7 +30,7 @@ ] end -Sync do |task| +Sync do supervisor_endpoint = IO::Endpoint::Generic.parse(ENV.fetch("SUPERVISOR_ENDPOINT")) http_endpoint = IO::Endpoint.tcp("0.0.0.0", backend_port) @@ -42,16 +42,18 @@ scheme: "http" ) + state = { + endpoint: { + name: service_name, + scheme: :http, + protocols: Async::HTTP::Protocol::HTTP1.names, + addresses: [{address: backend_address, port: backend_port}], + } + } + worker = Async::Service::Supervisor::Worker.new( endpoint: supervisor_endpoint, - state: { - endpoint: { - name: service_name, - scheme: :http, - protocols: Async::HTTP::Protocol::HTTP1.names, - addresses: [{address: backend_address, port: backend_port}], - } - } + state: state ) worker.run diff --git a/control-plane/envoy.yaml b/control-plane/envoy.yaml index 205858f..ff91b6f 100644 --- a/control-plane/envoy.yaml +++ b/control-plane/envoy.yaml @@ -9,12 +9,16 @@ admin: port_value: 19000 dynamic_resources: - ads_config: - api_type: GRPC - transport_api_version: V3 - grpc_services: - - envoy_grpc: - cluster_name: xds_cluster + # Clusters come from the supervisor's dedicated CDS stream. ADS remains + # unconfigured and is available to another control plane. + cds_config: + resource_api_version: V3 + api_config_source: + api_type: GRPC + transport_api_version: V3 + grpc_services: + - envoy_grpc: + cluster_name: xds_cluster static_resources: listeners: @@ -31,6 +35,9 @@ static_resources: stat_prefix: ingress_http route_config: name: local_route + # The application cluster arrives over CDS after this static + # route is loaded. + validate_clusters: false virtual_hosts: - name: backend domains: ["*"] @@ -45,15 +52,8 @@ static_resources: "@type": type.googleapis.com/envoy.extensions.filters.http.router.v3.Router clusters: - - name: app-http1 - connect_timeout: 1s - type: EDS - lb_policy: ROUND_ROBIN - eds_cluster_config: - service_name: app-http1 - eds_config: - ads: {} - resource_api_version: V3 + # This bootstrap cluster lets Envoy reach the supervisor. Application + # clusters arrive over CDS and point their EDS subscriptions back here. - name: xds_cluster connect_timeout: 1s type: STRICT_DNS diff --git a/control-plane/readme.md b/control-plane/readme.md index 8a3c72c..ef4ac66 100644 --- a/control-plane/readme.md +++ b/control-plane/readme.md @@ -4,11 +4,13 @@ This scenario exercises the intended Envoy control plane topology: - Falcon workers register with `Async::Service::Supervisor::Worker`. - Each worker publishes `state[:endpoint]` with its name, scheme, supported protocols, and concrete addresses. -- `Async::Service::Supervisor::Envoy::Monitor` maps supervisor state into xDS endpoint resources. -- Envoy connects to the supervisor's xDS server and subscribes to endpoint updates. +- `Async::Service::Supervisor::Envoy::Monitor` maps supervisor state into Envoy endpoint assignments. +- Envoy connects to the supervisor's dedicated CDS and EDS services and subscribes to cluster and endpoint updates. - Envoy routes HTTP traffic to the supervised Falcon workers. -Envoy initiates the xDS connection. The supervisor does not call Envoy's admin API or mutate Envoy directly. Once Envoy has connected and subscribed, the supervisor streams updates over that connection. This matches the normal xDS control plane model and gives Envoy ownership of reconnects, resource ACK/NACK handling, and sidecar lifecycle. +Only the management cluster is declared statically in `envoy.yaml`. The application cluster and its endpoints are discovered from the supervisor. + +Envoy initiates the connection. The supervisor does not call Envoy's admin API or mutate Envoy directly. Once Envoy has connected and subscribed, the supervisor streams updates over that connection. This matches the normal xDS control plane model and gives Envoy ownership of reconnects, resource ACK/NACK handling, and sidecar lifecycle. ## Running Tests @@ -26,9 +28,9 @@ $ docker compose -f control-plane/docker-compose.yaml down --remove-orphans The test verifies the happy path for the desired architecture: -- The supervisor can run an xDS server. +- The supervisor can run dedicated cluster and endpoint discovery services. - Supervised Falcon workers can publish endpoints. -- Envoy can subscribe to those endpoints using ADS-backed EDS. +- Envoy can subscribe to those endpoints using EDS. - Envoy can load balance requests across the supervised workers. This is a framework for lifecycle testing rather than the complete production story. Follow-up cases should cover worker removal, worker recovery, health changes, and xDS stream reconnects. diff --git a/control-plane/test/envoy.rb b/control-plane/test/envoy.rb index 76f2258..2d33f25 100644 --- a/control-plane/test/envoy.rb +++ b/control-plane/test/envoy.rb @@ -47,18 +47,25 @@ def eventually(timeout: 20, interval: 0.5) expect(responses.filter_map{|response| response["x-backend-id"]}.uniq.sort).to be == ["backend-a", "backend-b"] end - it "loads the xDS cluster from the supervisor monitor" do + it "discovers cluster endpoints from the supervisor monitor" do uri = admin_uri + "/clusters?format=json" - cluster_status = eventually do + host_statuses = eventually do if (response = Net::HTTP.get_response(uri)).code.to_i == 200 clusters = JSON.parse(response.body) - clusters.fetch("cluster_statuses").find do |cluster| + cluster_status = clusters.fetch("cluster_statuses").find do |cluster| cluster.fetch("name") == "app-http1" end + + hosts = cluster_status&.fetch("host_statuses", nil) + hosts if hosts&.size == 2 end end - expect(cluster_status).not.to be_nil + addresses = host_statuses.map do |host| + host.fetch("address").fetch("socket_address").fetch("port_value") + end + + expect(addresses.sort).to be == [9292, 9293] end end diff --git a/gems.rb b/gems.rb index 9efc822..3fdf6c9 100644 --- a/gems.rb +++ b/gems.rb @@ -6,6 +6,7 @@ source "https://rubygems.org" gem "falcon" +gem "async-grpc-xds", github: "socketry/async-grpc-xds", ref: "fd9867c9155550982fff6eb77ed75a98c84fda49" gemspec diff --git a/guides/getting-started/readme.md b/guides/getting-started/readme.md index c1ea351..fe4344a 100644 --- a/guides/getting-started/readme.md +++ b/guides/getting-started/readme.md @@ -1,6 +1,6 @@ # Getting Started -This guide explains how to use `async-service-supervisor-envoy` to publish supervised worker endpoints to Envoy using xDS. +This guide explains how to use `async-service-supervisor-envoy` to publish supervised worker clusters and endpoints to Envoy using xDS. ## Installation @@ -16,10 +16,12 @@ The gem depends on `async-service-supervisor` and `async-grpc-xds`. `async-service-supervisor-envoy` provides: - - {ruby Async::Service::Supervisor::Envoy::Monitor} - A supervisor monitor that publishes worker endpoints through xDS. + - {ruby Async::Service::Supervisor::Envoy::Monitor} - A supervisor monitor that publishes worker clusters through CDS and their endpoints through EDS. - {ruby Async::Service::Supervisor::Envoy::Endpoint} - A small value object for endpoint state. -The monitor runs an xDS control plane endpoint. Envoy connects to it using ADS and receives CDS/EDS updates derived from supervisor worker state. +The monitor serves dedicated Cluster Discovery Service and Endpoint Discovery Service streams. It does not claim Envoy's Aggregated Discovery Service, so another control plane can use ADS for listeners, routes, and other configuration. + +CDS describes the logical services exposed by supervised workers, including their supported protocol, active health checks, and load-balancing policy. EDS supplies the concrete workers currently available for each service. ## Endpoint State @@ -62,11 +64,48 @@ Add the monitor to your supervisor environment: require "async/service/supervisor/envoy" Async::Service::Supervisor::Envoy::Monitor.new( - bind: "http://127.0.0.1:18000" + bind: "http://127.0.0.1:18000", + management_cluster: "xds_cluster" ) ``` -By default, workers are grouped into clusters by `state[:name]`. +By default, workers are grouped into clusters by `state[:name]`. `management_cluster` must match the static Envoy cluster used to reach the monitor. + +## Envoy Configuration + +Configure Envoy to obtain clusters from the monitor's dedicated CDS service. The monitor configures each discovered cluster to obtain its endpoints from the dedicated EDS service on the same management server: + +``` yaml +dynamic_resources: + cds_config: + resource_api_version: V3 + api_config_source: + api_type: GRPC + transport_api_version: V3 + grpc_services: + - envoy_grpc: + cluster_name: xds_cluster + +static_resources: + clusters: + - name: xds_cluster + connect_timeout: 1s + type: STRICT_DNS + http2_protocol_options: {} + load_assignment: + cluster_name: xds_cluster + endpoints: + - lb_endpoints: + - endpoint: + address: + socket_address: + address: 127.0.0.1 + port_value: 18000 +``` + +The bootstrap cluster name must match the monitor's `management_cluster`. Envoy uses it for independent CDS and EDS gRPC streams; no `ads_config` is required. + +If a static route refers to a cluster delivered by CDS, set `validate_clusters: false` on that route configuration. Envoy can then load the route before the cluster arrives and will begin routing once CDS and EDS have warmed it. ## Custom Mapping @@ -113,10 +152,14 @@ utilization_monitor = Async::Service::Supervisor::UtilizationMonitor.new(interva ] ``` -The supervisor utilization monitor manages each worker's shared-memory allocation and registration. The Envoy monitor samples it through `sample_by_worker`, combines each worker's `requests_total` counter with processor usage from `process-metrics`, and serves the resulting ORCA reports from the same HTTP/2 endpoint as ADS. It also configures each discovered cluster to use Envoy's client-side weighted-round-robin policy. +The supervisor utilization monitor manages each worker's shared-memory allocation and registration. The Envoy monitor samples it through `sample_by_worker`, combines each worker's `requests_total` counter with processor usage from `process-metrics`, and serves the resulting ORCA reports from the same HTTP/2 endpoint as CDS and EDS. It also configures each discovered cluster to use Envoy's client-side weighted-round-robin policy. The first sample establishes a baseline. Subsequent reports contain normalized `cpu_utilization` and `rps_fractional` values for each worker. Reports are removed immediately when a worker disconnects. +Workers are identified by the `hostname` published with each endpoint, which Envoy sends as the request authority when it opens an out-of-band reporting stream. Enabling ORCA therefore publishes one endpoint per worker rather than one per shared listener. + +The generated load-balancing policy uses the monitor's bind port and reporting interval. Envoy dials that port on each endpoint's own address, which reaches the monitor because it shares a network namespace with the workers. + Out-of-band ORCA requires: - Envoy 1.39 or later. diff --git a/lib/async/service/supervisor/envoy/monitor.rb b/lib/async/service/supervisor/envoy/monitor.rb index 433b9bf..fc3a146 100644 --- a/lib/async/service/supervisor/envoy/monitor.rb +++ b/lib/async/service/supervisor/envoy/monitor.rb @@ -3,12 +3,15 @@ # Released under the MIT License. # Copyright, 2026, by Samuel Williams. -require "async/http/endpoint" -require "async/service/supervisor/monitor" -require "async/service/supervisor/utilization_monitor" require "async/grpc/xds/client_side_weighted_round_robin" +require "async/grpc/xds/cluster_discovery_service" +require "async/grpc/xds/config_source" require "async/grpc/xds/control_plane" +require "async/grpc/xds/endpoint_discovery_service" require "async/grpc/xds/server" +require "async/http/endpoint" +require "async/service/supervisor/monitor" +require "async/service/supervisor/utilization_monitor" require "process/metrics" require "xds/data/orca/v3/orca_load_report_pb" @@ -22,12 +25,16 @@ module Service module Supervisor # Provides Envoy integration for supervisor-managed services. module Envoy - # Represents a supervisor monitor that publishes worker endpoints to Envoy using xDS. + # Represents a supervisor monitor that publishes clusters and worker endpoints to Envoy using xDS. + # + # The monitor serves dedicated CDS and EDS streams, leaving ADS available to + # another control plane for listeners, routes, and other configuration. class Monitor < Async::Service::Supervisor::Monitor # Initialize the monitor. - # @parameter bind [String | Nil] The optional address for the xDS control plane server. + # @parameter bind [String | Nil] The optional address for the discovery server. # @parameter delegate [Delegate] The delegate used to map supervisor state into Envoy endpoints. # @parameter control_plane [Async::GRPC::XDS::ControlPlane] The xDS control plane to update. + # @parameter management_cluster [String] The static Envoy cluster used to reach this discovery server. # @parameter health_checks [Array(Envoy::Config::Core::V3::HealthCheck)] The active health checks applied to published clusters. # @parameter orca [Boolean] Whether to collect and serve per-worker ORCA load reports. # @parameter processor [Process::Metrics::Processor | Nil] The optional process CPU sampler. @@ -37,6 +44,7 @@ def initialize( bind: nil, delegate: Delegate.new, control_plane: Async::GRPC::XDS::ControlPlane.new, + management_cluster: "xds_cluster", health_checks: [], orca: false, processor: nil, @@ -49,12 +57,13 @@ def initialize( @bind = bind @delegate = delegate @control_plane = control_plane + @eds_config = Async::GRPC::XDS::ConfigSource.grpc(management_cluster) @health_checks = health_checks @interval = interval @orca = orca @controllers = {} @published_clusters = {} - @server_task = nil + @published_endpoints = {} @mutex = Mutex.new if @orca @@ -105,15 +114,21 @@ def remove(supervisor_controller) end end - # Run the monitor and optional xDS server task. - # @parameter parent [Async::Task] The parent task used for the xDS server. + # Run the monitor and optional discovery server task. + # @parameter parent [Async::Task] The parent task used for the server. # @returns [Async::Task] The monitor task. def run(parent: Async::Task.current) task = super(parent: parent) if @bind - @server_task = parent.async do - server = Async::GRPC::XDS::Server.new(@control_plane) + parent.async do + server = Async::GRPC::XDS::Server.new( + @control_plane, + services: [ + Async::GRPC::XDS::ClusterDiscoveryService, + Async::GRPC::XDS::EndpointDiscoveryService, + ] + ) server.dispatcher.register(ORCAService.new(self, minimum_interval: @interval)) if @orca server.run(server_endpoint) end @@ -201,8 +216,13 @@ def reconcile end end + # Skipping an identical assignment only avoids a redundant version bump: (@published_clusters.keys | clusters.keys).each do |cluster| - @control_plane.update_endpoints(cluster, clusters.fetch(cluster, [])) + endpoints = clusters.fetch(cluster, []) + next if @published_endpoints[cluster] == endpoints + + @control_plane.update_endpoints(cluster, endpoints) + @published_endpoints[cluster] = endpoints end end @@ -218,6 +238,9 @@ def build_clusters(records_by_cluster = build_records_by_cluster) records_by_cluster.transform_values do |records| if @orca records.map do |record| + # Envoy sends the endpoint hostname as the request authority when + # it opens an out-of-band reporting stream, which is how a single + # ORCA service identifies which worker a report is for: { addresses: record[:endpoint].addresses, healthy: record[:healthy], @@ -248,6 +271,7 @@ def cluster_configuration(records) configuration = { protocol: envoy_protocol(common_protocols), + eds_config: @eds_config, health_checks: @health_checks, } diff --git a/readme.md b/readme.md index 56d214d..e588e49 100644 --- a/readme.md +++ b/readme.md @@ -8,7 +8,7 @@ Provides an Envoy xDS monitor for `async-service-supervisor`. `async-service-supervisor-envoy` publishes supervised worker endpoints to Envoy: - - **xDS control plane** - Runs an ADS server backed by `async-grpc-xds`. + - **Cluster and endpoint discovery** - Runs dedicated CDS and EDS services backed by `async-grpc-xds`, leaving ADS free for listener, route, and other configuration. - **Supervisor integration** - Registers and removes endpoints from supervisor worker lifecycle events. - **Multiple clusters** - Groups workers by `state[:name]` by default. - **Endpoint contract** - Converts concrete post-bind worker listeners into Envoy upstream endpoints, including grouped IP or Unix socket addresses. diff --git a/releases.md b/releases.md index 49c886d..8eae3f6 100644 --- a/releases.md +++ b/releases.md @@ -2,6 +2,9 @@ ## Unreleased + - Serve clusters and endpoints through dedicated CDS and EDS services instead of the aggregated discovery service, leaving ADS available for listener, route, and other configuration. + - Configure generated clusters to obtain endpoint assignments from the dedicated EDS service. + - Publish endpoint assignments only when they change, instead of on every reconciliation. - Use normalized processor utilization from `process-metrics` v0.13. ## v0.4.0 diff --git a/test/async/service/supervisor/envoy/monitor.rb b/test/async/service/supervisor/envoy/monitor.rb index fdd7d28..c3fa8e8 100644 --- a/test/async/service/supervisor/envoy/monitor.rb +++ b/test/async/service/supervisor/envoy/monitor.rb @@ -57,9 +57,11 @@ def processor(samples = []) }) monitor.register(controller) + cluster = monitor.control_plane.resources(Async::GRPC::XDS::ControlPlane::CLUSTER_TYPE).first expect(cluster.health_checks).to be == [health_check] + expect(cluster.eds_cluster_config.eds_config.api_config_source.grpc_services.first.envoy_grpc.cluster_name).to be == "xds_cluster" end it "publishes ORCA worker identity and load-balancing configuration" do @@ -394,6 +396,19 @@ def healthy?(supervisor_controller, endpoint) } end + it "publishes distinct endpoints reported by several workers" do + monitor.register(Controller.new(1, { + endpoint: {name: "myservice", scheme: "http", protocols: ["h2"], addresses: [{path: "/tmp/one.ipc"}]} + })) + monitor.register(Controller.new(2, { + endpoint: {name: "myservice", scheme: "http", protocols: ["h2"], addresses: [{path: "/tmp/two.ipc"}]} + })) + + assignment = endpoint_assignment("myservice") + + expect(assignment.endpoints.first.lb_endpoints.size).to be == 2 + end + it "selects the preferred common endpoint protocol" do monitor.register(Controller.new(1, { endpoint: {name: "myservice", scheme: "http", protocols: ["h2", "http/1.1"], addresses: [{path: "/tmp/one.ipc"}]} @@ -426,7 +441,38 @@ def healthy?(supervisor_controller, endpoint) end.to raise_exception(ArgumentError) end - it "runs an xDS server when bound" do + it "does not republish unchanged endpoints" do + monitor.register(Controller.new(1, { + endpoint: {name: "myservice", scheme: "http", protocols: ["h2"], addresses: [{address: "127.0.0.1", port: 50051}]} + })) + + version = control_plane.version(Async::GRPC::XDS::ControlPlane::ENDPOINT_TYPE) + monitor.run_once + + expect(control_plane.version(Async::GRPC::XDS::ControlPlane::ENDPOINT_TYPE)).to be == version + end + + it "keeps unchanged assignments in the state of the world when another cluster changes" do + monitor.register(Controller.new(1, { + endpoint: {name: "service-a", scheme: "http", protocols: ["h2"], addresses: [{address: "127.0.0.1", port: 50051}]} + })) + monitor.register(Controller.new(2, { + endpoint: {name: "service-b", scheme: "http", protocols: ["h2"], addresses: [{address: "127.0.0.2", port: 50052}]} + })) + + response = control_plane.response( + Async::GRPC::XDS::ControlPlane::ENDPOINT_TYPE, + ["service-a", "service-b"] + ) + + names = response.resources.map do |resource| + Envoy::Config::Endpoint::V3::ClusterLoadAssignment.decode(resource.value).cluster_name + end + + expect(names.sort).to be == ["service-a", "service-b"] + end + + it "runs dedicated cluster and endpoint discovery services when bound" do parent = Class.new do def initialize @count = 0 @@ -445,24 +491,31 @@ def async(&block) end.new calls = [] + dispatcher = Object.new + dispatcher.define_singleton_method(:register){|service| calls << [:register, service]} original_server = Async::GRPC::XDS.send(:remove_const, :Server) fake_server = Class.new do - define_method(:initialize) do |control_plane| - calls << [:initialize, control_plane] + define_method(:initialize) do |control_plane, services:| + calls << [:initialize, control_plane, services] end + define_method(:dispatcher) {dispatcher} + define_method(:run) do |endpoint| calls << [:run, endpoint] end end Async::GRPC::XDS.const_set(:Server, fake_server) - monitor = subject.new(bind: "http://127.0.0.1:18000") expect(monitor.run(parent: parent)).to be == :monitor_task - expect(calls.first).to be == [:initialize, monitor.control_plane] + expect(calls.first).to be == [ + :initialize, + monitor.control_plane, + [Async::GRPC::XDS::ClusterDiscoveryService, Async::GRPC::XDS::EndpointDiscoveryService] + ] expect(calls.last.last).to be_a(Async::HTTP::Endpoint) ensure Async::GRPC::XDS.send(:remove_const, :Server) From a58cd8769d43cff122a0ea74d6db4a1ae3c3388e Mon Sep 17 00:00:00 2001 From: Samuel Williams Date: Fri, 7 Aug 2026 20:18:34 +1200 Subject: [PATCH 2/4] Use released async-grpc-xds v0.4 --- async-service-supervisor-envoy.gemspec | 2 +- gems.rb | 2 -- 2 files changed, 1 insertion(+), 3 deletions(-) diff --git a/async-service-supervisor-envoy.gemspec b/async-service-supervisor-envoy.gemspec index fa60084..b59e377 100644 --- a/async-service-supervisor-envoy.gemspec +++ b/async-service-supervisor-envoy.gemspec @@ -25,7 +25,7 @@ Gem::Specification.new do |spec| spec.required_ruby_version = ">= 3.3" spec.add_dependency "async", "~> 2.38" - spec.add_dependency "async-grpc-xds", "~> 0.3" + spec.add_dependency "async-grpc-xds", "~> 0.4" spec.add_dependency "async-http" spec.add_dependency "async-service-supervisor", "~> 0.20" spec.add_dependency "process-metrics", "~> 0.13" diff --git a/gems.rb b/gems.rb index 3fdf6c9..2bd0a16 100644 --- a/gems.rb +++ b/gems.rb @@ -6,8 +6,6 @@ source "https://rubygems.org" gem "falcon" -gem "async-grpc-xds", github: "socketry/async-grpc-xds", ref: "fd9867c9155550982fff6eb77ed75a98c84fda49" - gemspec group :maintenance, optional: true do From 167660e01c813412cdeab31d8e650364e10a4073 Mon Sep 17 00:00:00 2001 From: Samuel Williams Date: Fri, 7 Aug 2026 20:46:59 +1200 Subject: [PATCH 3/4] Allow cluster publication to be disabled. --- .github/workflows/test-control-plane.yaml | 18 +++-- context/getting-started.md | 15 ++++- control-plane/docker-compose.yaml | 3 +- control-plane/envoy-eds.yaml | 66 +++++++++++++++++++ control-plane/readme.md | 19 ++++-- control-plane/supervisor.rb | 3 +- guides/getting-started/readme.md | 15 ++++- lib/async/service/supervisor/envoy/monitor.rb | 45 ++++++++----- readme.md | 10 ++- releases.md | 1 + .../async/service/supervisor/envoy/monitor.rb | 43 +++++++++++- 11 files changed, 203 insertions(+), 35 deletions(-) create mode 100644 control-plane/envoy-eds.yaml diff --git a/.github/workflows/test-control-plane.yaml b/.github/workflows/test-control-plane.yaml index e0d2de6..71d5ce1 100644 --- a/.github/workflows/test-control-plane.yaml +++ b/.github/workflows/test-control-plane.yaml @@ -10,9 +10,9 @@ env: jobs: test: - name: ${{matrix.ruby}} on ${{matrix.os}} - runs-on: ${{matrix.os}}-latest - continue-on-error: ${{matrix.experimental}} + name: ${{ matrix.ruby }} with ${{ matrix.discovery.name }} on ${{ matrix.os }} + runs-on: ${{ matrix.os }}-latest + continue-on-error: ${{ matrix.experimental }} strategy: matrix: @@ -24,6 +24,14 @@ jobs: - "3.4" - "4.0" + discovery: + - name: CDS and EDS + publish_clusters: "true" + envoy_config: ./envoy.yaml + - name: EDS only + publish_clusters: "false" + envoy_config: ./envoy-eds.yaml + experimental: [false] steps: @@ -32,5 +40,7 @@ jobs: - name: Run tests timeout-minutes: 10 env: - RUBY_VERSION: ${{matrix.ruby}} + RUBY_VERSION: ${{ matrix.ruby }} + PUBLISH_CLUSTERS: ${{ matrix.discovery.publish_clusters }} + ENVOY_CONFIG: ${{ matrix.discovery.envoy_config }} run: docker compose -f control-plane/docker-compose.yaml up --build --exit-code-from tests diff --git a/context/getting-started.md b/context/getting-started.md index fe4344a..48870da 100644 --- a/context/getting-started.md +++ b/context/getting-started.md @@ -19,7 +19,7 @@ The gem depends on `async-service-supervisor` and `async-grpc-xds`. - {ruby Async::Service::Supervisor::Envoy::Monitor} - A supervisor monitor that publishes worker clusters through CDS and their endpoints through EDS. - {ruby Async::Service::Supervisor::Envoy::Endpoint} - A small value object for endpoint state. -The monitor serves dedicated Cluster Discovery Service and Endpoint Discovery Service streams. It does not claim Envoy's Aggregated Discovery Service, so another control plane can use ADS for listeners, routes, and other configuration. +The monitor always serves a dedicated Endpoint Discovery Service stream. By default it also serves Cluster Discovery Service. It does not claim Envoy's Aggregated Discovery Service, so another control plane can use ADS for listeners, routes, and other configuration. CDS describes the logical services exposed by supervised workers, including their supported protocol, active health checks, and load-balancing policy. EDS supplies the concrete workers currently available for each service. @@ -69,7 +69,18 @@ Async::Service::Supervisor::Envoy::Monitor.new( ) ``` -By default, workers are grouped into clusters by `state[:name]`. `management_cluster` must match the static Envoy cluster used to reach the monitor. +By default, workers are grouped into clusters by `state[:name]`. When cluster publication is enabled, `management_cluster` must match the static Envoy cluster used to reach the monitor. + +If another control plane owns cluster configuration, disable CDS publication while retaining supervisor-owned endpoint discovery: + +``` ruby +Async::Service::Supervisor::Envoy::Monitor.new( + bind: "http://127.0.0.1:18000", + publish_clusters: false +) +``` + +In this mode, the external control plane or bootstrap configuration must define each cluster, configure it to use the monitor's dedicated EDS service, and supply its protocol, health checks, and load-balancing policy. The cluster's EDS service name must match the worker cluster name published by the monitor. ## Envoy Configuration diff --git a/control-plane/docker-compose.yaml b/control-plane/docker-compose.yaml index e5ee5d7..bbb800b 100644 --- a/control-plane/docker-compose.yaml +++ b/control-plane/docker-compose.yaml @@ -10,6 +10,7 @@ services: - CONSOLE_OUTPUT=XTerm - XDS_BIND=http://0.0.0.0:18000 - SUPERVISOR_ENDPOINT=tcp://0.0.0.0:12000 + - PUBLISH_CLUSTERS=${PUBLISH_CLUSTERS:-true} backend-a: build: @@ -49,7 +50,7 @@ services: image: envoyproxy/envoy:v1.32-latest command: ["envoy", "-c", "/etc/envoy/envoy.yaml", "--log-level", "info"] volumes: - - ./envoy.yaml:/etc/envoy/envoy.yaml:ro + - ${ENVOY_CONFIG:-./envoy.yaml}:/etc/envoy/envoy.yaml:ro ports: - "10000:10000" - "19000:19000" diff --git a/control-plane/envoy-eds.yaml b/control-plane/envoy-eds.yaml new file mode 100644 index 0000000..a423e3d --- /dev/null +++ b/control-plane/envoy-eds.yaml @@ -0,0 +1,66 @@ +node: + id: async-service-supervisor-envoy-control-plane + cluster: async-service-supervisor-envoy-control-plane + +admin: + address: + socket_address: + address: 0.0.0.0 + port_value: 19000 + +static_resources: + listeners: + - name: listener_http + address: + socket_address: + address: 0.0.0.0 + port_value: 10000 + filter_chains: + - filters: + - name: envoy.filters.network.http_connection_manager + typed_config: + "@type": type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager + stat_prefix: ingress_http + route_config: + name: local_route + virtual_hosts: + - name: backend + domains: ["*"] + routes: + - match: + prefix: "/" + route: + cluster: app-http1 + http_filters: + - name: envoy.filters.http.router + typed_config: + "@type": type.googleapis.com/envoy.extensions.filters.http.router.v3.Router + + clusters: + - name: app-http1 + connect_timeout: 1s + type: EDS + eds_cluster_config: + service_name: app-http1 + eds_config: + resource_api_version: V3 + api_config_source: + api_type: GRPC + transport_api_version: V3 + grpc_services: + - envoy_grpc: + cluster_name: xds_cluster + + - name: xds_cluster + connect_timeout: 1s + type: STRICT_DNS + load_assignment: + cluster_name: xds_cluster + endpoints: + - lb_endpoints: + - endpoint: + address: + socket_address: + address: supervisor + port_value: 18000 + http2_protocol_options: {} diff --git a/control-plane/readme.md b/control-plane/readme.md index ef4ac66..df3dfe4 100644 --- a/control-plane/readme.md +++ b/control-plane/readme.md @@ -1,14 +1,16 @@ # Control Plane Testing -This scenario exercises the intended Envoy control plane topology: +This scenario exercises both supported Envoy control plane topologies: - Falcon workers register with `Async::Service::Supervisor::Worker`. - Each worker publishes `state[:endpoint]` with its name, scheme, supported protocols, and concrete addresses. - `Async::Service::Supervisor::Envoy::Monitor` maps supervisor state into Envoy endpoint assignments. -- Envoy connects to the supervisor's dedicated CDS and EDS services and subscribes to cluster and endpoint updates. +- Envoy connects to the supervisor's dedicated discovery services. - Envoy routes HTTP traffic to the supervised Falcon workers. -Only the management cluster is declared statically in `envoy.yaml`. The application cluster and its endpoints are discovered from the supervisor. +The default configuration enables cluster publication. Only the management cluster is declared statically in `envoy.yaml`; the application cluster is discovered through CDS and its workers are discovered through EDS. + +The EDS-only configuration disables cluster publication. Both the management and application clusters are declared statically in `envoy-eds.yaml`, while the application's workers are still discovered through EDS. This represents a deployment where bootstrap configuration or another control plane owns clusters. Envoy initiates the connection. The supervisor does not call Envoy's admin API or mutate Envoy directly. Once Envoy has connected and subscribed, the supervisor streams updates over that connection. This matches the normal xDS control plane model and gives Envoy ownership of reconnects, resource ACK/NACK handling, and sidecar lifecycle. @@ -18,6 +20,12 @@ Envoy initiates the connection. The supervisor does not call Envoy's admin API o $ docker compose -f control-plane/docker-compose.yaml up --build --exit-code-from tests ``` +To run the EDS-only scenario: + +``` bash +$ PUBLISH_CLUSTERS=false ENVOY_CONFIG=./envoy-eds.yaml docker compose -f control-plane/docker-compose.yaml up --build --exit-code-from tests +``` + To clean up containers and networks: ``` bash @@ -26,9 +34,10 @@ $ docker compose -f control-plane/docker-compose.yaml down --remove-orphans ## What This Proves -The test verifies the happy path for the desired architecture: +The tests verify the happy path for both architectures: -- The supervisor can run dedicated cluster and endpoint discovery services. +- The supervisor can run dedicated CDS and EDS services together. +- The supervisor can serve EDS without serving or publishing CDS. - Supervised Falcon workers can publish endpoints. - Envoy can subscribe to those endpoints using EDS. - Envoy can load balance requests across the supervised workers. diff --git a/control-plane/supervisor.rb b/control-plane/supervisor.rb index edad7b8..eac7a3f 100644 --- a/control-plane/supervisor.rb +++ b/control-plane/supervisor.rb @@ -16,7 +16,8 @@ def endpoint(value) Sync do supervisor_endpoint = endpoint(ENV.fetch("SUPERVISOR_ENDPOINT")) monitor = Async::Service::Supervisor::Envoy::Monitor.new( - bind: ENV.fetch("XDS_BIND") + bind: ENV.fetch("XDS_BIND"), + publish_clusters: ENV.fetch("PUBLISH_CLUSTERS", "true") == "true" ) server = Async::Service::Supervisor::Server.new( diff --git a/guides/getting-started/readme.md b/guides/getting-started/readme.md index fe4344a..48870da 100644 --- a/guides/getting-started/readme.md +++ b/guides/getting-started/readme.md @@ -19,7 +19,7 @@ The gem depends on `async-service-supervisor` and `async-grpc-xds`. - {ruby Async::Service::Supervisor::Envoy::Monitor} - A supervisor monitor that publishes worker clusters through CDS and their endpoints through EDS. - {ruby Async::Service::Supervisor::Envoy::Endpoint} - A small value object for endpoint state. -The monitor serves dedicated Cluster Discovery Service and Endpoint Discovery Service streams. It does not claim Envoy's Aggregated Discovery Service, so another control plane can use ADS for listeners, routes, and other configuration. +The monitor always serves a dedicated Endpoint Discovery Service stream. By default it also serves Cluster Discovery Service. It does not claim Envoy's Aggregated Discovery Service, so another control plane can use ADS for listeners, routes, and other configuration. CDS describes the logical services exposed by supervised workers, including their supported protocol, active health checks, and load-balancing policy. EDS supplies the concrete workers currently available for each service. @@ -69,7 +69,18 @@ Async::Service::Supervisor::Envoy::Monitor.new( ) ``` -By default, workers are grouped into clusters by `state[:name]`. `management_cluster` must match the static Envoy cluster used to reach the monitor. +By default, workers are grouped into clusters by `state[:name]`. When cluster publication is enabled, `management_cluster` must match the static Envoy cluster used to reach the monitor. + +If another control plane owns cluster configuration, disable CDS publication while retaining supervisor-owned endpoint discovery: + +``` ruby +Async::Service::Supervisor::Envoy::Monitor.new( + bind: "http://127.0.0.1:18000", + publish_clusters: false +) +``` + +In this mode, the external control plane or bootstrap configuration must define each cluster, configure it to use the monitor's dedicated EDS service, and supply its protocol, health checks, and load-balancing policy. The cluster's EDS service name must match the worker cluster name published by the monitor. ## Envoy Configuration diff --git a/lib/async/service/supervisor/envoy/monitor.rb b/lib/async/service/supervisor/envoy/monitor.rb index fc3a146..aba3eb7 100644 --- a/lib/async/service/supervisor/envoy/monitor.rb +++ b/lib/async/service/supervisor/envoy/monitor.rb @@ -25,16 +25,17 @@ module Service module Supervisor # Provides Envoy integration for supervisor-managed services. module Envoy - # Represents a supervisor monitor that publishes clusters and worker endpoints to Envoy using xDS. + # Represents a supervisor monitor that publishes worker endpoints and optionally clusters to Envoy using xDS. # - # The monitor serves dedicated CDS and EDS streams, leaving ADS available to - # another control plane for listeners, routes, and other configuration. + # The monitor always serves a dedicated EDS stream and can additionally serve + # CDS, leaving ADS available to another control plane. class Monitor < Async::Service::Supervisor::Monitor # Initialize the monitor. # @parameter bind [String | Nil] The optional address for the discovery server. # @parameter delegate [Delegate] The delegate used to map supervisor state into Envoy endpoints. # @parameter control_plane [Async::GRPC::XDS::ControlPlane] The xDS control plane to update. # @parameter management_cluster [String] The static Envoy cluster used to reach this discovery server. + # @parameter publish_clusters [Boolean] Whether to publish derived cluster configuration through CDS. # @parameter health_checks [Array(Envoy::Config::Core::V3::HealthCheck)] The active health checks applied to published clusters. # @parameter orca [Boolean] Whether to collect and serve per-worker ORCA load reports. # @parameter processor [Process::Metrics::Processor | Nil] The optional process CPU sampler. @@ -45,6 +46,7 @@ def initialize( delegate: Delegate.new, control_plane: Async::GRPC::XDS::ControlPlane.new, management_cluster: "xds_cluster", + publish_clusters: true, health_checks: [], orca: false, processor: nil, @@ -58,6 +60,7 @@ def initialize( @delegate = delegate @control_plane = control_plane @eds_config = Async::GRPC::XDS::ConfigSource.grpc(management_cluster) + @publish_clusters = publish_clusters @health_checks = health_checks @interval = interval @orca = orca @@ -122,12 +125,12 @@ def run(parent: Async::Task.current) if @bind parent.async do + services = [Async::GRPC::XDS::EndpointDiscoveryService] + services.unshift(Async::GRPC::XDS::ClusterDiscoveryService) if @publish_clusters + server = Async::GRPC::XDS::Server.new( @control_plane, - services: [ - Async::GRPC::XDS::ClusterDiscoveryService, - Async::GRPC::XDS::EndpointDiscoveryService, - ] + services: services ) server.dispatcher.register(ORCAService.new(self, minimum_interval: @interval)) if @orca server.run(server_endpoint) @@ -207,17 +210,21 @@ def reconcile records_by_cluster = build_records_by_cluster clusters = build_clusters(records_by_cluster) - records_by_cluster.each do |cluster, records| - configuration = cluster_configuration(records) - - unless @published_clusters[cluster] == configuration - @control_plane.update_cluster(cluster, **configuration) - @published_clusters[cluster] = configuration + if @publish_clusters + records_by_cluster.each do |cluster, records| + configuration = cluster_configuration(records) + + unless @published_clusters[cluster] == configuration + @control_plane.update_cluster(cluster, **configuration) + @published_clusters[cluster] = configuration + end end + elsif @orca + records_by_cluster.each_value{|records| validate_orca_records(records)} end # Skipping an identical assignment only avoids a redundant version bump: - (@published_clusters.keys | clusters.keys).each do |cluster| + (@published_endpoints.keys | clusters.keys).each do |cluster| endpoints = clusters.fetch(cluster, []) next if @published_endpoints[cluster] == endpoints @@ -276,9 +283,7 @@ def cluster_configuration(records) } if @orca - if records.any?{|record| record[:endpoint].addresses.any?{|address| address[:path]}} - raise ArgumentError, "Out-of-band ORCA reporting requires IP endpoints!" - end + validate_orca_records(records) configuration[:load_balancing_policy] = Async::GRPC::XDS::ClientSideWeightedRoundRobin.build( @orca_port, @@ -289,6 +294,12 @@ def cluster_configuration(records) configuration end + def validate_orca_records(records) + if records.any?{|record| record[:endpoint].addresses.any?{|address| address[:path]}} + raise ArgumentError, "Out-of-band ORCA reporting requires IP endpoints!" + end + end + def sample_load_reports controllers = @mutex.synchronize{@controllers.dup} workers = @utilization_monitor.sample_by_worker diff --git a/readme.md b/readme.md index e588e49..62946bb 100644 --- a/readme.md +++ b/readme.md @@ -19,12 +19,20 @@ Provides an Envoy xDS monitor for `async-service-supervisor`. Please see the [project documentation](https://socketry.github.io/async-service-supervisor-envoy/) for more details. - - [Getting Started](https://socketry.github.io/async-service-supervisor-envoy/guides/getting-started/index) - This guide explains how to use `async-service-supervisor-envoy` to publish supervised worker endpoints to Envoy using xDS. + - [Getting Started](https://socketry.github.io/async-service-supervisor-envoy/guides/getting-started/index) - This guide explains how to use `async-service-supervisor-envoy` to publish supervised worker clusters and endpoints to Envoy using xDS. ## Releases Please see the [project releases](https://socketry.github.io/async-service-supervisor-envoy/releases/index) for all releases. +### Unreleased + + - Serve clusters and endpoints through dedicated CDS and EDS services instead of the aggregated discovery service, leaving ADS available for listener, route, and other configuration. + - Allow cluster publication to be disabled when clusters are owned by another control plane. + - Configure generated clusters to obtain endpoint assignments from the dedicated EDS service. + - Publish endpoint assignments only when they change, instead of on every reconciliation. + - Use normalized processor utilization from `process-metrics` v0.13. + ### v0.4.0 - Publish configured active health checks with Envoy clusters. diff --git a/releases.md b/releases.md index 8eae3f6..4b3be1f 100644 --- a/releases.md +++ b/releases.md @@ -3,6 +3,7 @@ ## Unreleased - Serve clusters and endpoints through dedicated CDS and EDS services instead of the aggregated discovery service, leaving ADS available for listener, route, and other configuration. + - Allow cluster publication to be disabled when clusters are owned by another control plane. - Configure generated clusters to obtain endpoint assignments from the dedicated EDS service. - Publish endpoint assignments only when they change, instead of on every reconciliation. - Use normalized processor utilization from `process-metrics` v0.13. diff --git a/test/async/service/supervisor/envoy/monitor.rb b/test/async/service/supervisor/envoy/monitor.rb index c3fa8e8..a8745d6 100644 --- a/test/async/service/supervisor/envoy/monitor.rb +++ b/test/async/service/supervisor/envoy/monitor.rb @@ -64,6 +64,34 @@ def processor(samples = []) expect(cluster.eds_cluster_config.eds_config.api_config_source.grpc_services.first.envoy_grpc.cluster_name).to be == "xds_cluster" end + it "can publish endpoints without publishing clusters" do + monitor = subject.new(publish_clusters: false) + control_plane = monitor.control_plane + controller = Controller.new(1, { + endpoint: {name: "myservice", scheme: "http", protocols: ["h2"], addresses: [{address: "127.0.0.1", port: 50051}]} + }) + + monitor.register(controller) + + response = control_plane.response( + Async::GRPC::XDS::ControlPlane::ENDPOINT_TYPE, + ["myservice"] + ) + assignment = Envoy::Config::Endpoint::V3::ClusterLoadAssignment.decode(response.resources.first.value) + + expect(assignment.cluster_name).to be == "myservice" + expect(control_plane.resources(Async::GRPC::XDS::ControlPlane::CLUSTER_TYPE)).to be(:empty?) + + monitor.remove(controller) + response = control_plane.response( + Async::GRPC::XDS::ControlPlane::ENDPOINT_TYPE, + ["myservice"] + ) + assignment = Envoy::Config::Endpoint::V3::ClusterLoadAssignment.decode(response.resources.first.value) + + expect(assignment.endpoints.flat_map(&:lb_endpoints)).to be(:empty?) + end + it "publishes ORCA worker identity and load-balancing configuration" do monitor = subject.new( bind: "http://127.0.0.1:18000", @@ -472,7 +500,7 @@ def healthy?(supervisor_controller, endpoint) expect(names.sort).to be == ["service-a", "service-b"] end - it "runs dedicated cluster and endpoint discovery services when bound" do + it "runs the configured dedicated discovery services when bound" do parent = Class.new do def initialize @count = 0 @@ -511,11 +539,22 @@ def async(&block) monitor = subject.new(bind: "http://127.0.0.1:18000") expect(monitor.run(parent: parent)).to be == :monitor_task - expect(calls.first).to be == [ + + endpoint_parent = parent.class.new + endpoint_monitor = subject.new(bind: "http://127.0.0.1:18001", publish_clusters: false) + expect(endpoint_monitor.run(parent: endpoint_parent)).to be == :monitor_task + + initializations = calls.select{|call| call.first == :initialize} + expect(initializations.first).to be == [ :initialize, monitor.control_plane, [Async::GRPC::XDS::ClusterDiscoveryService, Async::GRPC::XDS::EndpointDiscoveryService] ] + expect(initializations.last).to be == [ + :initialize, + endpoint_monitor.control_plane, + [Async::GRPC::XDS::EndpointDiscoveryService] + ] expect(calls.last.last).to be_a(Async::HTTP::Endpoint) ensure Async::GRPC::XDS.send(:remove_const, :Server) From b5cbbcc943aa3fa10eddfb3cc206fd9e640844c3 Mon Sep 17 00:00:00 2001 From: Samuel Williams Date: Fri, 7 Aug 2026 20:52:45 +1200 Subject: [PATCH 4/4] Cover ORCA validation without cluster publication. --- test/async/service/supervisor/envoy/monitor.rb | 1 + 1 file changed, 1 insertion(+) diff --git a/test/async/service/supervisor/envoy/monitor.rb b/test/async/service/supervisor/envoy/monitor.rb index a8745d6..56166e9 100644 --- a/test/async/service/supervisor/envoy/monitor.rb +++ b/test/async/service/supervisor/envoy/monitor.rb @@ -173,6 +173,7 @@ def processor(samples = []) it "rejects Unix endpoints for out-of-band ORCA" do monitor = subject.new( bind: "http://127.0.0.1:18000", + publish_clusters: false, orca: true, processor: processor, utilization_monitor: utilization_monitor