feat(frankgateway): split into per-traffic-class instances — inway / outway / internal (IN-2547)

[jimleitch01]

What

Frank!Gateway becomes a map of instances instead of a single deployment, so inbound (inway), outbound (outway) and east-west (internal) traffic can each run on their own pods, with their own etcd slice, dashboard and NetworkPolicy.

Closes the chart half of IN-2547. The jim00 rollout (enabling the split, repointing applications) is separate and deploy-side.

Safety: the default render is byte-identical

The instance key gateway is the reserved single-instance default — it keeps the bare object names and the /apisix prefix. A full helm template with ci/lint-values.yaml was diffed against feature/podiumd-4.8.4 and is identical.

The only differences under helm template are the randAlphaNum secrets, which have no cluster to lookup and so differ between any two runs of the same chart. Those are masked by key name for the comparison; everything else is compared verbatim, so a new key, changed indentation or reordered block would still show.

Existing environments — including those already running Frank!Gateway — are unaffected until they opt in.

Design notes

• One etcd, three prefixes. APISIX in traditional mode loads exactly the objects beneath its configured prefix, so config isolation is complete without three StatefulSets and three PVCs. etcd is not the blast-radius concern.
• Per-instance checksums. The config and shim checksums come from new helpers rather than hashing the whole template file. A range over instances in one file would give every Deployment the same file-wide checksum, so changing one traffic class would needlessly restart the others.
• Dashboard per instance. apisix-dashboard binds exactly one etcd prefix, so the whole chain (dashboard + shim + oauth2-proxy + Keycloak client + realm secret) is per instance. The prefix is written into the dashboard conf only when it differs from its built-in /apisix default, which is what keeps the default render unchanged.
• Routes are not duplicated. They move into per-class directories and each instance declares which directories it seeds. The default seeds outway + internal, i.e. all five legacy apiproxy routes with the same ids. The ConfigMap is keyed by basename in sorted order, so the result does not depend on how many directories contributed.
• NetworkPolicies are the point of the exercise and are new to this chart — one gateway carrying every class needs the union of all their egress, so nothing meaningful can be restricted. Opt-in, and only for instances with a class.
• serviceAlias keeps the pre-split frankgateway Service name pointing at a chosen instance so applications can be repointed one at a time.

Migration hazard

The Admin API Secret becomes frankgateway-<instance>-admin-credentials. Deployment pipelines that read frankgateway-admin-credentials by name must be updated — jim00's apply-routes.sh does. This is why the default instance keeps the old names and why serviceAlias exists. Called out in the upgrade notes.

Verification

helm lint charts/podiumd -f charts/podiumd/ci/lint-values.yaml                                  # clean
helm lint charts/podiumd -f charts/podiumd/ci/lint-values.yaml -f charts/podiumd/ci/split-values.yaml  # clean

Split mode renders 3 gateways + 3 dashboard chains + 3 NetworkPolicies + the shared etcd + the alias Service, with distinct etcd prefixes, correctly partitioned routes (internal = BRP, outway = BAG + 3× KVK, inway = empty in-chart), and per-instance Keycloak clients wired end to end.

CI now renders the split shape as well as the default, via ci/split-values.yaml — otherwise a broken split template could merge unnoticed.

Docs

• New frankgateway-traffic-classes.md with a mermaid architecture diagram (rendered to check it parses), the NetworkPolicy model and the migration order.
• Exploration doc's "nothing is implemented" banner retired; its four open questions answered in place.
• Upgrade notes for the 4.8.3 → 4.8.4 hop.

🤖 Generated by Claude Code with help from Jimbo