Skip to content

feat(kms)!: derive each value's key from a cached authority key, so a batch binds and costs one backend call (CIP-5062) - #510

Open
freshtonic wants to merge 3 commits into
claude/cip-4071-opt-in-cloud-testsfrom
claude/cip-5062-authority-keys
Open

freshtonic wants to merge 3 commits into
claude/cip-4071-opt-in-cloud-testsfrom
claude/cip-5062-authority-keys

Conversation

@freshtonic

@freshtonic freshtonic commented Oct 21, 2026 •

Copy link
Copy Markdown
Contributor

Summary

This PR builds the design in ADR 0007 (#491): a pooled source shares the authority to mint keys, never a key. One backend call fetches an authority key, which is cached in memory. Each value then gets its own data key, which is derived locally from the authority key, a fresh 16-byte salt and the value's binding. A batch costs one backend call per cache refresh, and every value still has its own key and its own binding. #508 removed the clone-one-key pooled sources, and this replaces them.

There are two commits. Each one passes clippy with -D warnings on its own.

  1. refactor(kms)!: remove KeyIsolation. With Pooled gone, the enum had one value. So the enum, KeyProvider::ISOLATION, BatchGenerateDataKey::ISOLATION and every assertion are removed. There are no readers in this workspace, in cipherstash/stack or in cipherstash-suite. ADR 0003 and the dated design survey still name it, because they are historical records.
  2. feat(kms): add AuthorityKeySource.

API

  • The new feature authority-keys enables caching, vitaminc-random, hmac and sha2, with zeroize on both. It adds no new crate. It is off by default. The default, traits-only and wasm32 builds do not change.
  • No feature gate:
    • DerivesFromAuthorityKey, the capability marker trait, with cache_policy() and purge_expired().
    • AuthorityKeyCachePolicy { max_entries, ttl }. It has no Default, because the TTL is a security control.
  • Behind the feature:
    • AuthorityKeyVersion.
    • AuthorityKeys::new(current, versions). It refuses a duplicate version, and a current version that the map does not hold.
    • AuthorityKeySource<T>:
      • new and provision. provision mints the wrapped authority key under the correct binding, so callers never build that binding themselves.
      • It implements KeyProvider<N>, DerivesFromAuthorityKey and IndexKeyProvider (the last delegates to the inner provider).
      • CUSTODY = BackendHeld, because a KeyId carries no key material.
      • RECONSTRUCTION = T::RECONSTRUCTION.
    • AuthorityKeyError.
  • The store is a plain struct, not a trait. The deployment persists version → wrapped KeyId, the same as the FixedIndexKeySource id. To rotate, the deployment adds a version, makes it current, and builds the source again. Old versions stay for retrieval.
  • When many requests miss the cache at the same time, they share one fetch (moka try_get_with).

Wire format (new ADR 0008)

  • Authority key: 32 bytes. The inner provider mints and unwraps it under the binding "vitaminc-kms/authority-key/v1" || u32be(version). The binding names the version, so the backend refuses a wrapped key that is stored under the wrong version.
  • Data key: HKDF-SHA256 with salt = 16 bytes from SafeRand and IKM = the authority key. HKDF is built here on hmac, not on the hkdf crate. That crate drops the PRK and each expand block without wiping them, and for a 32-byte key the one expand block is the data key. Here each value is written straight into Protected memory, which zeroizes on drop. The RFC 5869 SHA-256 vectors (cases 1 to 3) check the implementation.
    • info = "vitaminc-kms/authority-key/v1/data-key" || u32be(version) || u16be(N) || u64be(len(binding)) || binding.
    • A compile-time check limits N to 1..=8160.
  • KeyId: 0x01 || u32be(version) || salt, exactly 21 bytes. A malformed id or an unknown version gives a typed error for the whole batch before any backend call.
  • Known-answer vectors are in the ADR and in the tests. They were computed with an independent Python HKDF.

Docs

  • CONTEXT.md gets the authority-key term and the three key tiers.
  • ADR 0007 is edited in place to answer its open questions.
  • The CachingKeyProvider docs now restate the auditing and in-memory stance. Per-value retrieval is no longer per-value authorization. The cached authority key is Protected and is wiped on eviction and expiry.
  • The CRAP feature lists in crap.yml and mise.toml get vitaminc-kms/authority-keys.

Closes CIP-5062.

For the reviewer

  • Wiping: the PRK and every expand block are Protected, and the HMAC states wipe on drop. Only SHA-256's own working memory is out of reach, the same as for every HMAC in the workspace. AuthorityKeyError::DataKeyLength is removed: the compile-time check on N means derivation cannot fail.
  • Retrieval cannot detect a wrong binding. It derives a key that opens nothing, and the consumer's AEAD reports the failure. The Binding docs, CONTEXT.md and ADR 0007 say this.
  • "Pluggable cache strategies" is only the trait seam and the policy struct. There is no strategy trait.
  • The LocalStack end-to-end tests copy the client builder rather than share it.
  • ADR 0008's note on AWS's Hierarchical keyring KDF was checked against AWS's specification: counter-mode KDF with HMAC-SHA256 (NIST SP 800-108).

Test plan

  • 23 new tests, including the RFC 5869 SHA-256 vectors. They cover:
    • round trip, and distinct keys in a 50-value batch
    • a wrong binding derives a different key
    • 100 values cost 1 inner retrieve, and 16 concurrent misses cost 1 fetch
    • an empty batch makes zero inner calls (this was CIP-5033's requirement)
    • TTL refetch, purge_expired, rotation, an unknown version and five malformed ids
    • backend refusal, and the wrong number of keys from the inner provider
    • three known-answer tests, and two end-to-end tests against LocalStack
  • cargo test -p vitaminc-kms --all-features --locked: 309 tests and 8 doctests pass. With default features: 58 tests and 4 doctests.
  • The test target builds with each of vault, azure, gcp and authority-keys on its own.
  • clippy with -D warnings on each commit, fmt --check, the workspace check with all features, wasm32-wasip1, and cargo doc with -D warnings. No ADR number is used twice, and every commit is signed.

🤖 Generated with Claude Code

@freshtonic
freshtonic added this pull request to stack #356 October 21, 2026 05:38
@freshtonic freshtonic changed the title claude/cip 5062 authority keys feat(kms)!: derive each value's key from a cached authority key, so a batch binds and costs one backend call (CIP-5062) Oct 21, 2026
@freshtonic
freshtonic marked this pull request as ready for review October 21, 2026 05:39
@github-actions

github-actions Bot commented Oct 21, 2026 •

Copy link
Copy Markdown
Contributor

✅ No CRAP threshold violations

1645 function(s) analyzed · threshold 30

@freshtonic
freshtonic force-pushed the claude/cip-5062-authority-keys branch from 876a0a8 to 10958c2 Compare October 21, 2026 05:51
@github-actions

github-actions Bot commented Oct 21, 2026 •

Copy link
Copy Markdown
Contributor

🧬 Mutation testing (cargo-mutants, --in-diff)

caught missed unviable timeout
37 0 47 0

✅ Every mutant in the changed lines was caught by a test.

@freshtonic
freshtonic force-pushed the claude/cip-5062-authority-keys branch 2 times, most recently from a524bd2 to cbf2147 Compare October 21, 2026 10:20
…ey per value

Nothing implemented `KeyIsolation::Pooled` after CIP-4981, so the enum had
one reachable value. ADR 0007 says a one-key-per-batch source cannot bind,
and that the round-trip saving belongs on its own capability, not on a
third variant. An enum that always says `PerValue` tells a reader nothing,
so the enum and every `ISOLATION` constant go. No crate in this workspace,
cipherstash/stack or cipherstash-suite reads the constant.

`BatchGenerateDataKey` now states the guarantee in its docs: one distinct
key per binding.

BREAKING CHANGE: `KeyIsolation`, `KeyProvider::ISOLATION` and
`BatchGenerateDataKey::ISOLATION` are removed. Remove the constant from
custom provider impls. Every source in the crate delivers one key per
value.
…per cache refresh and still binds each value

ADR 0007 keeps the round-trip saving that clone-one-key pooling gave, without
sharing a data key. `AuthorityKeySource<T>` (feature `authority-keys`) caches
an authority key that the inner provider unwraps, and derives each value's
own data key from it locally:

    data key = HKDF-SHA256(authority key, 16-byte salt, info(version, N, binding))
    KeyId    = 0x01 || u32be(version) || salt

The binding is a KDF input, so every value is bound. A wrong binding derives
a different key, and the consumer's AEAD detects it. ADR 0008 records the
wire format, with known-answer vectors from an independent HKDF.

An authority key is a 32-byte data key of any existing source, minted once
per version by `AuthorityKeySource::provision` under a fixed binding that
names the version. The deployment stores each wrapped id in an
`AuthorityKeys` (version -> wrapped id, plus the current version), as it
stores the index key id of `FixedIndexKeySource`. Rotation adds a version
and makes it current; old versions stay for retrieval, and a version the
deployment does not hold is a typed error.

The `DerivesFromAuthorityKey` capability trait carries the
`AuthorityKeyCachePolicy` (TTL and entry capacity) and `purge_expired`.
The cached authority key is `Protected` and zeroizes when it leaves the
cache. Concurrent misses share one fetch. An empty batch makes no backend
call. `CUSTODY` is `BackendHeld`; `RECONSTRUCTION` is the inner provider's.

The source, `CachingKeyProvider`, ADR 0007, `CONTEXT.md` and the README now
state the auditing and in-memory stance: per-value retrieval is not
per-value authorization, and the TTL is a security control.

HKDF is built on the workspace's existing `hmac` 0.13 and `sha2` 0.11,
not on the `hkdf` crate, so the PRK and each expand block are written
into `Protected` memory and wiped on drop. The RFC 5869 SHA-256 vectors
check it. No new crate. The coverage feature lists gain `authority-keys`, so the
CRAP gate does not score the new module as uncovered.
…DF key

The mutation gate found three gaps in the authority-key tests. No test
read `AuthorityKeyVersion::get` or its `Display`. No test derived a key
longer than 287 bytes, so the 255-block limit could change from a
product to a sum unseen. One test now reads both accessors. Another
derives an 8160-byte key, which runs the HKDF block counter to 255, and
checks that the long output starts with the short one under the same
`info`.

Refs: CIP-5062
@freshtonic
freshtonic force-pushed the claude/cip-5062-authority-keys branch from cbf2147 to c515c77 Compare October 22, 2026 01:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant