Skip to content

Mixed Service + custom backendRefs in one HTTPRoute rule lose traffic, and the outcome flips depending on MergeBackends #10269

Open
#10328
Open
Mixed Service + custom backendRefs in one HTTPRoute rule lose traffic, and the outcome flips depending on MergeBackends#10269
#10328
Assignees
zirain
Labels
area/xds-translatorkind/bugSomething isn't working

Description

@zirain
zirain
opened on Oct 14, 2026

Description

When one HTTPRoute rule mixes a regular backend (Service/Backend) with a custom backend from the extension manager (e.g. InferencePool), the backendRef weights are ignored and one side gets no traffic at all. Which side loses depends on whether mergeBackends is enabled on the EnvoyProxy. The route status reports Accepted / ResolvedRefs in both cases.

Reproduction

Configure the extension manager with a custom backend resource (e.g. inference.networking.k8s.io/InferencePool), then apply:

apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
  name: mixed
  namespace: default
spec:
  parentRefs:
  - name: eg
  rules:
  - matches:
    - path:
        value: /mixed
    backendRefs:
    - name: service-1
      port: 8080
      weight: 1
    - group: inference.networking.k8s.io
      kind: InferencePool
      name: inference-pool
      weight: 1

Run it once without and once with spec.mergeBackends: {} on the EnvoyProxy.

Observed behavior

xref: #10270

MergeBackends disabled MergeBackends enabled
IR Both refs stay in Destination.Settings. The custom one is isCustomBackend: true with no endpoints. The Service moves to BackendClusterRefs (shared cluster). The custom ref stays in Settings.
xDS route cluster: <route-dest>, not weighted weightedClusters with only the Service cluster
xDS clusters One EDS cluster: a Service locality plus an empty locality for the custom backend. PostClusterModifyHook gets this cluster with all of the route's extension refs. Shared Service cluster, plus a custom-backend cluster that is built and hooked but not referenced by any route
Service traffic Whatever the extension leaves in the cluster. An EPP-style extension rewrites it to ORIGINAL_DST, so the Service gets 0% and its EDS assignment is dead config. 100%
Custom backend traffic 100% (via the extension) 0%
Weights honored No No

Root cause

  • MergeBackends disabled
    • Custom settings have no AddressType and no filters, so NeedsClusterPerSetting() is false.
    • The rule collapses into a single cluster named Destination.Name, and the extension hook receives that whole cluster.
  • MergeBackends enabled
    • isMergeableBackendKind excludes custom kinds, so only the Service merges.
    • mergeIncompatibleForWeightedRule doesn't treat a custom backend in the rule as merge-unsafe.
    • The rule then has 2 cluster sources, so buildXdsWeightedRouteAction runs. That function only emits settings with len(Endpoints) > 0 || IsDynamicResolver, so the IsCustomBackend setting is skipped. Its weight is still counted as valid in BackendWeights.AddWeighted.
  • Both cases: the cluster hook's extension resources are the route's full ExtensionRefs (ExtraArgs.unstructuredRefs in translator.go), not per setting. So per-setting clusters can't be hooked correctly today either.

Questions

  1. Should the two modes behave the same? MergeBackends is meant to be an optimization that dedupes clusters, but here enabling it flips which backend receives traffic. Should it be guaranteed not to change routing outcomes?
  2. If so, which behavior is the intended one?
    • Single cluster, extension decides (today's disabled behavior): the extension owns the whole rule. We'd make a custom backend in the rule merge-unsafe so the enabled mode matches. Weights stay ignored.
    • Weighted clusters, weights honored: each backendRef gets its own cluster and the weights apply. This needs IsCustomBackend settings included in buildXdsWeightedRouteAction, and PostClusterModifyHook scoped per setting so non-custom clusters aren't rewritten. That changes what extensions receive.
  3. What about rules where every backendRef is a custom backend? Today they're collapsed into one cluster and the extension sees all resources together. Should that stay as-is regardless of the answer above, so existing extensions aren't affected?
 apiVersion: gateway.networking.k8s.io/v1
 kind: HTTPRoute
 metadata:
 name: multiple-inference-pools
 namespace: default
 spec:
 parentRefs:
 - name: eg
 rules:
 - matches:
   - path:
       value: /mixed
   backendRefs:
   - group: inference.networking.k8s.io
     kind: InferencePool
     name: primary-inference-pool
     weight: 1
   - group: inference.networking.k8s.io
     kind: InferencePool
     name: secondary-inference-pool
     weight: 1

Activity

  1. zirain
    added
    area/extensions
    area/xds-translator
    and removed
    area/extensions
    on Oct 15, 2026
  2. zirain
    self-assigned this
    on Oct 15, 2026

  3. guydc commented on Oct 15, 2026

    @guydc
    guydc
    on Oct 15, 2026
    Contributor

    Super complex scenario here. My initial thoughts are:

    Should the two modes behave the same? MergeBackends is meant to be an optimization that dedupes clusters, but here enabling it flips which backend receives traffic. Should it be guaranteed not to change routing outcomes?

    They should. If unsafe - let's not dedupe.

    If so, which behavior is the intended one?

    for me: Single cluster, extension decides. If we want this to be explicit - maybe reject HTTPRoute if it has an extensionBackend and any non-extensionBackend with weight != 0 ? Essentially, require users to opt-in to extension backend owning all traffic.

    What about rules where every backendRef

    Stay the same, extensions decide.

  4. zirain commented on Oct 15, 2026

    @zirain
    zirain
    on Oct 15, 2026
    MemberAuthor

    Thanks @guydc, I agree with your first point. The behavior should be as same as possible, since it's hard for extension server aware of MergeBackend enabled or not.

    On the second and third questions I'd like to argue for weighted clusters, because the single-cluster model leaves the extension server to redo work Envoy Gateway already knows how to do. I ran into this myself in theagentrouter/agent-router#2657, a draft I'm working on. It adds weighted routing across multiple InferencePools (needed for the GAIE GatewayWeightedAcrossTwoInferencePools conformance test). This is the all-custom case from question 3, so it's the case where "the extension decides".

    Because Envoy Gateway merges the rule into one cluster and one non-weighted route, the extension has to undo that merge in PostTranslateModify:

    1. Get the weights back. They're dropped during translation. So the extension parses the xDS route name (httproute/<ns>/<name>/rule/<idx>/...) to find the rule, fetches the HTTPRoute from the API server, and matches each backendRef to a pool to recover its weight. Cached client also may cause a race here.
    2. Track several pools on one cluster. The cluster and route metadata change from a single pool to a list of pools. Every reader of that metadata changes to match, and the len(inferencePools) != 1 "BUG" checks in the cluster and route hooks are removed.
    3. Split the cluster. It clones the merged cluster once per pool, because each pool needs its own metadata and protocol options. In PostClusterModify the protocol options can only be taken from the first pool as a placeholder, and they're fixed up later.
    4. Rebuild the route by hand. It replaces the route's cluster with weighted_clusters, points each entry at a cloned cluster, and sets typed_per_filter_config on each entry so that only that pool's EPP ext_proc filter runs.
    5. Clean up. It removes the original merged cluster and keeps the ordering of the other clusters stable.

    In total the draft is +1512/−136. That includes about 500 new lines in post_translate_modify.go and about 780 lines of tests. Almost all of it rebuilds what Envoy Gateway would produce if it emitted weighted clusters for custom backends. It also relies on details the extension contract doesn't cover, such as the route name format and a second read of the HTTPRoute.

    If Envoy Gateway emitted weighted clusters instead:

    • each backendRef gets its own cluster, and PostClusterModifyHook gets exactly one cluster and the one resource behind it;
    • Envoy applies the weights, so the extension doesn't need to know them;
    • the extension only sets per-entry typed_per_filter_config for its own filter, which the draft shows already works;
    • Service clusters never reach the hook, so an extension can't overwrite them by accident (as happens today in the mixed case);
    • MergeBackends no longer needs a special case: Service clusters are still deduped, and custom backends keep their own clusters.

    Metadata for the backend. Envoy Gateway could also write the custom backend's identity (group, version, kind, name and namespace) into the metadata of the cluster it builds for that backendRef. Then, in any hook, the extension could tell which InferencePool a cluster belongs to just by reading the cluster metadata. It wouldn't have to parse cluster or route names, or encode and decode its own metadata to carry that information between hooks (steps 1 and 2 above).

    With weighted clusters and this metadata, most of the draft (steps 1, 2, 3 and 5 above) would no longer be needed.

    Rejecting mixed rules would rule out moving traffic gradually from a Service to an InferencePool (90/10, then 50/50, and so on). Weighted clusters support that without extra work.

    Compatibility: custom backends in the extension manager were originally designed for InferencePool, and InferencePool is the use case that needs this change. So I think a breaking change is acceptable here, rather than adding a config option we'd have to keep around: emit per-backendRef clusters for custom backends, document it in the release notes. The extension server could handle this base on the cluster type.

    What do you think?

  5. cnvergence
    added
    kind/bugSomething isn't working
    on Oct 15, 2026

  6. guydc commented on Oct 15, 2026

    @guydc
    guydc
    on Oct 15, 2026
    Contributor

    OK, if inference pools are the main user of this ability and it makes translation easier and more spec-compliant, weighted clusters are also fine by me.

    My only question is: how does this impact the cluster modification hook? Would all weighted clusters be called and with which extension context?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

area/xds-translatorkind/bugSomething isn't working

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions