Skip to content

aac listen --reusable-psk daemon denies all credential requests with request_id mismatch after host reboot (crash-loop 13+ days · possible proxy-side session desync) #162

Open

Description

@jimmikroo

Summary

A long-running aac listen --reusable-psk LaunchAgent daemon on macOS enters a persistent "Credential request failed: Request denied" state for ALL remote-client credential requests after a host reboot, even though:

  • PSK state file (~/.access-protocol/psk_store_user_client.json) survives reboot intact
  • Listener identity (user_client.key) unchanged
  • Fresh BW_SESSION loaded post-reboot (BW CLI unlocked successfully)
  • Listener process re-runs cleanly and reports state = running

The denial is preceded by a specific warning pattern at ap_client::clients::remote_client:

INFO aac::command::connect: Requesting credential for domain: council.hermes.foo
WARN ap_client::clients::remote_client: Ignoring response with mismatched request_id: Some("req-NNNNNNNNNNN-xxxxxxxx")
ERROR aac::command::run: Credential request failed: Request denied

The remote-client receives A response from the proxy but with a different request_id than the one it sent — the client correctly ignores it, then times out waiting for its own response, surfacing as "Request denied" (caught by #141's ambiguous-denial-class).

Environment

  • aac version: 0.11.0 (commit 3b000d15af71be5ddcf8893b099f3a66d386be9d)
  • macOS Tahoe (aarch64)
  • Deployment: aac listen --reusable-psk --provider bitwarden -v under /usr/bin/script -q /dev/null PTY wrapper, invoked by a macOS LaunchAgent
  • Bitwarden CLI: current @bitwarden/cli
  • Proxy: wss://ap.lesspassword.dev (default)

Timeline

  • T-103d — initial aac connect --token pairing ceremony completed via operator at TUI. Listener + remote-client state established. connection_cache_user_client.json + connection_cache_remote_client.json populated. All subsequent credential requests served GREEN for 103 days.
  • T-14d — host reboot (macOS scheduled restart).
  • T-13d — post-reboot: aac_listen daemon restarts cleanly, BW_SESSION re-provisioned, PSK store intact. BUT all council.* domain credential requests from LaunchAgent-driven daemons return "Request denied" with request_id mismatch warning.
  • T-0 — 13+ days of persistent crash-loop (KeepAlive=true LaunchAgent re-fires every ~30s, each attempt denied).

Repro shape

After a host reboot on a previously-paired aac listen --reusable-psk daemon:

  1. Verify PSK state intact: jq . ~/.access-protocol/psk_store_user_client.json shows 1 entry with pre-reboot created_at.
  2. Verify listener paired: aac connections list shows Paired: Nmonths ago | Last used: Nmonths ago.
  3. Trigger credential request from a remote-client (e.g., a LaunchAgent-driven daemon running aac run --domain some.domain --env X=password -- somecmd).
  4. Observe aac listen log (via tail -f ~/ops/aac_listen.log with ANSI strip):
    INFO aac::command::connect: Requesting credential for domain: some.domain
    WARN ap_client::clients::remote_client: Ignoring response with mismatched request_id: Some("...")
    ERROR aac::command::run: Credential request failed: Request denied
    

What we verified does NOT resolve

  • Re-establishing BW_SESSION (fresh bw unlock + Keychain refresh) — no change
  • Bootstrap/bootout cycle of the LaunchAgent — no change
  • Attempted full re-pair (wipe ~/.access-protocol/ + fresh aac listen + aac connect --token <NEW>) — blocked at a different failure class: new-domain approval requires TUI per the headless-LaunchAgent-denies-fresh-domains class, so headless re-pair is architecturally dead (separate class)

Hypothesis

Proxy-side session state at wss://ap.lesspassword.dev appears to track "listener online since " with some TTL or heartbeat semantic. A 24-hour-plus gap between pre-reboot listener-online and post-reboot listener-online causes proxy-side pairing expiry OR some request_id mapping desync. The client-side PSK + identity + cached connection all survive, but proxy no longer routes responses with matching request_ids.

Related to #161 but subtler: in our case the proxy DOES respond; it just responds with request_ids the client doesn't recognize.

Expected behavior

Either:

  • Proxy-side session should refresh transparently on listener reconnect (make reboot transparent)
  • OR listener should detect proxy-side desync via first request/response cycle and re-pair automatically
  • OR there should be a documented recovery procedure for post-reboot proxy-desync class

Diagnostic info we can provide

If helpful, we can share (sanitized):

  • Full log excerpts of the denial cycle
  • aac connections list -v output showing paired fingerprints
  • Timing of reboot vs first denial
  • Repro command set

References

Thanks for maintaining the project. Happy to help repro or provide additional diagnostics.

Activity

  1. bitwarden-bot commented on Nov 17, 2026

    @bitwarden-bot

    Thank you for reporting this issue! We've added this to our internal tracking system.
    ID: PM-47124
    Link (for internal use): https://bitwarden.atlassian.net/browse/PM-47124

  2. dmorriss-bw commented on Nov 17, 2026

    @dmorriss-bw

    Hello @jimmikroo
    Thank you for your report. I have flagged this to the Engineering team for review; please feel free to post additional information, such as screenshots or a screen video recordings, if you wish.
    Thank you again,

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions