Replace LLMConfiguration.apiKey heuristic with explicit apiKeyRef (no secrets in LLM config)

[enyst]

Problem

Today we treat LLMConfiguration.apiKey as ambiguous: it can be a literal secret (e.g. sk-...) or an env-var-shaped reference (heuristic /^[A-Z0-9_]+$/).

This is a security footgun:

• encourages heuristic guessing instead of explicit meaning
• increases risk of accidentally persisting/logging secrets
• makes audits brittle because apiKey doesn’t have a single meaning

Proposed direction

Change the model so LLM config never contains the raw secret:

• Replace LLMConfiguration.apiKey with LLMConfiguration.apiKeyRef (explicit reference only).
• Resolve the reference to the actual secret only at the moment we need to send it over the network (provider client / request construction), and keep the secret value as ephemeral as possible.

This also implies: for security reasons, TS must not ever propagate a literal apiKey in LLM configuration again (beyond ephemeral runtime resolution).

Must be documented (non-negotiable)

This decision must be clearly documented in-code and in docs:

1. Comment on apiKeyRef

   • at the type/property definition (what formats are allowed; what it means; what it must not contain).

2. Comments at every point where we determine/set the value

   • either right where apiKeyRef is set, or immediately before (so future contributors don’t reintroduce literal secrets/heuristics).

3. LLM Profiles webview

   • add a comment at the right place in the webview code explaining the boundary + how apiKeyRef is handled.
   • consider whether we can store secrets directly to VS Code SecretStorage from the host side and keep the webview input strictly ephemeral.

4. Docstrings in the LLMConfiguration type

   • clarify the invariants: no raw secret values in config, only references; how resolution works.

5. Update packages/agent-sdk-ts/docs/python-parity.md

   • document that TypeScript uses apiKeyRef and must not carry apiKey in LLM config (security reason).

6. Update settings_prd.md

   • reflect the intended design and any user-facing implications.

Implementation sketch (high level)

• Introduce apiKeyRef in LLMConfiguration (TS SDK + extension).
• Define explicit reference format(s) (e.g. secret:OPENAI_API_KEY, etc.) and reject everything else.
• Remove /^[A-Z0-9_]+$/ heuristics from:
  • packages/agent-sdk-ts/src/sdk/llm/factory.ts
  • packages/agent-sdk-ts/src/sdk/llm/profiles.ts
  • src/webview/host/llmProfilesStore.ts
• Ensure profile persistence never writes secrets by default (and ideally never at all).
• Add tests covering:
  • allowed/denied apiKeyRef formats
  • no secret persistence
  • no secret echoing to webview

Acceptance criteria

• No heuristic guessing of env-var-like strings.
• No raw provider secrets stored in profile JSON when includeSecrets=false (ideally gate includeSecrets very hard and comment on it so we remember not to mess with it without human approval).
• LLM requests still work via resolved secrets from SecretStorage/SecretRegistry/env, but secrets are only materialized at request time and are not logged.

[enyst]

@OpenHands Investigate the codebase and confirm or deny: is this issue solved? If yes, close it, if not, tell what is left or your suggestions in a comment to this issue.

[openhands-ai]

I'm on it! enyst can track my progress at all-hands.dev

[openhands-ai]

• Investigated the codebase for issue Replace LLMConfiguration.apiKey heuristic with explicit apiKeyRef (no secrets in LLM config) #836 (“Replace LLMConfiguration.apiKey heuristic with explicit apiKeyRef”) and confirmed it is already solved in the current repo state:

  • LLMConfiguration uses apiKeyRef (packages/agent-sdk-ts/src/sdk/llm/types.ts).
  • Legacy apiKey is handled only as backward-compat and converted to apiKeyRef.kind="key" then removed (packages/agent-sdk-ts/src/sdk/llm/profiles.ts).
  • No env-var-shaped heuristic (/^[A-Z0-9_]+$/) remains in the relevant code paths.
  • Profiles strip secrets by default (inline apiKeyRef and headers not persisted unless explicitly allowed).
  • Secret resolution happens at runtime in LLMFactory via LLMCredentialProvider.

• Ran tests to verify behavior: npm test (passed at the time of verification).

• No code changes were made (git working tree clean; no diff).

• Commented on GitHub issue Replace LLMConfiguration.apiKey heuristic with explicit apiKeyRef (no secrets in LLM config) #836 summarizing the findings and recommending closure.

• Attempted to close the issue programmatically, but the gh CLI in this environment prompts for interactive login, so I could not close Replace LLMConfiguration.apiKey heuristic with explicit apiKeyRef (no secrets in LLM config) #836 from here. The remaining action is simply for a maintainer to close it.

View full conversation