Skip to content

test: add post-quantum cryptography conformance tests for showcase - #1351

Draft
torreypayne wants to merge 2 commits into
pqc-tls-validationfrom
pqc-validation-tests
Draft

torreypayne wants to merge 2 commits into
pqc-tls-validationfrom
pqc-validation-tests

Conversation

@torreypayne

@torreypayne torreypayne commented Sep 15, 2026

Copy link
Copy Markdown
Member

Adds shared/test/showcase/pqc_test.rb, a dedicated suite proving that generated Ruby clients negotiate X25519MLKEM768 with the Showcase server over both transports.

How it verifies PQC

Ruby never performs the key exchange itself — gRPC delegates to the BoringSSL build vendored in the grpc gem, and REST delegates to the system OpenSSL that Net::HTTP links against. The suite therefore asserts on what the server observed, via the metadata Showcase reflects onto every response:

  • x-showcase-tls-group — the group that was negotiated
  • x-showcase-tls-client-supported-groups — everything the client offered in its ClientHello

This is the same mechanism Python, Node, Java, C#, and C++ already use, so Ruby's assertions are directly comparable to the rest of the ecosystem.

Scenarios covered

Test Asserts
test_grpc_negotiates_post_quantum_key_exchange gRPC negotiates X25519MLKEM768 and advertised it in ClientHello
test_rest_negotiates_post_quantum_key_exchange REST negotiates X25519MLKEM768 and advertised it in ClientHello
test_grpc_connects_when_server_requires_post_quantum_key_exchange Handshake succeeds against a server pinned to --tls-groups 0x11ec
test_grpc_falls_back_to_classical_key_exchange gRPC degrades cleanly to X25519 against --tls-groups 0x001d,0x0017
test_rest_falls_back_to_classical_key_exchange REST degrades cleanly to X25519 against --tls-groups 0x001d,0x0017

The fallback tests matter as much as the happy path: they prove we did not introduce a hard post-quantum dependency that would break clients talking to classical-only endpoints.

Notes for reviewers

  • normalize_headers fails loudly if the TLS metadata is absent. Without that guard a non-TLS connection would silently turn every assertion into a no-op.
  • REST coverage never skips. Ruby's openssl is a default gem binding to the host libssl, and ML-KEM needs OpenSSL >= 3.5 — which no runner available to us provides (ubuntu-latest is 3.0.13). Guarding with a skip would mean the REST transport is never exercised at all, i.e. a permanently green check that proves nothing. Instead the test always runs and asserts the negotiated group against a {X25519MLKEM768, X25519} allowlist, cross-checking that the group was actually offered in the ClientHello; plaintext or any unexpected curve still fails. Setting SHOWCASE_REQUIRE_REST_PQC=1 tightens this to an exact X25519MLKEM768 match, which chore(ci): hard-assert REST post-quantum key exchange on an OpenSSL 3.5 image #1352 runs in CI. This mirrors the merged gax-php conformance test, the one other language with the same constraint.
  • The auxiliary --tls-groups servers reuse the harness certificate and are reaped in an ensure block.

Stack

This is a stacked PR. Each branch targets its predecessor, so the diff here contains only the commits unique to this step. Review and merge bottom-up.

# PR Branch → Base Type Scope
1 #1330 bump-showcasemain chore: Bump GAPIC_SHOWCASE_VERSION to 0.44.0
2 #1331 pqc-tls-validationbump-showcase test: TLS harness: SAN certificate, readiness probe, TLS client factories
3 #1351 ← this PR pqc-validation-testspqc-tls-validation test: Net-new pqc_test.rb; tolerant REST assertion (Layer 1)
4 #1352 pqc-rest-cipqc-validation-tests chore(ci): Strict REST PQC job on an OpenSSL 3.5 image (Layer 2)

Companion PR — independent, different repository, not part of this stack:
googleapis/ruby-core-libraries#73fix: raise the gapic-common floor to grpc ">= 1.83", "< 2.a". This is the only customer-facing change in the effort and the only one that triggers a gem release.

Design: go/sdk:ruby-pqc · Parent (approved): go/cloudsdk-pqc-ruby

Pre-Flight Engineering Audit Sign-Off

  • Encapsulation — no accessors added. The one piece of harness state the suite needs is read with ShowcaseTest.instance_variable_get :@showcase_dir, per the team's instance_variable_get convention.
  • Concurrency clarity — auxiliary servers are awaited through the shared wait_for_showcase probe (explicit monotonic deadline) and reaped via Process.kill + Process.wait inside ensure. No orphaned processes, no bare sleep used as synchronization.
  • Test brittleness — nothing stubs private internals; every assertion travels a real TLS handshake. Replaces the previous Fiddle probe that dereferenced a hardcoded struct offset and asserted nothing.
  • DRY — header normalization, client construction, and auxiliary server lifecycle are each factored into a single private helper.
  • Native multi-version matrixtoys test showcase green across four environments spanning the OpenSSL range, with zero skips in every one: host Ruby 4.0.5 / OpenSSL 3.6.3 (tolerant and strict), ruby:3.2-trixie / 3.5.5 (strict), and ruby:3.2-bookworm / 3.0.18 (tolerant). The bookworm run is the important one — it carries the same OpenSSL generation as ubuntu-latest, and the REST test now runs and passes there via the classical-fallback branch instead of skipping.
  • Lockfile hygienegit diff origin/main touches only pqc_test.rb and test_helper.rb; no Gemfile.lock drift, and the fiddle gem added by the earlier draft is gone.

Adds a dedicated suite that proves the generated Ruby clients negotiate
X25519MLKEM768 with the Showcase server over both transports.

Assertions read the TLS metadata that Showcase reflects onto every response
(x-showcase-tls-group and x-showcase-tls-client-supported-groups) rather than
inspecting CRuby's internal OpenSSL structures, whose layout is not stable
across Ruby releases or platforms.

Covered scenarios:
- gRPC and REST negotiate the hybrid post-quantum group.
- gRPC and REST advertise the group in their ClientHello.
- Both transports degrade cleanly to classical X25519 when the server offers
  only classical groups, confirming no hard post-quantum dependency.
- gRPC completes the handshake when the server accepts nothing but
  post-quantum key exchange.

REST coverage is skipped when the host OpenSSL predates 3.5, which is the
first release to implement ML-KEM.
The REST post-quantum assertion was guarded by a skip whenever the host
OpenSSL was below 3.5. GitHub Actions ubuntu-latest ships OpenSSL 3.0.13,
so in practice that guard meant the REST transport was never exercised in
CI at all: a permanently green check that verified nothing.

Replace the skip with an assertion that carries signal on every host. The
negotiated group must be present, which proves the connection was really
TLS; it must be either X25519MLKEM768 or classical X25519; and it must be
a group the client actually offered in its ClientHello. A plaintext
connection, or any unexpected group, still fails.

Setting SHOWCASE_REQUIRE_REST_PQC=1 promotes this into a strict
post-quantum assertion, for environments guaranteed to provide
OpenSSL >= 3.5.

This mirrors the merged conformance test in gax-php. Ruby and PHP are the
only Cloud SDK languages whose REST transport binds to the system OpenSSL
instead of a vendored TLS stack, so they are the only two that cannot
hard-assert post-quantum key exchange on a stock runner.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant