diff --git a/AGENTS.md b/AGENTS.md index e31d30724..973c32196 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -195,7 +195,7 @@ The check enumerates submodule contents and fails on what it finds there, becaus ### Architecture Docs Sync -- Channel forwarders deliver to the destination captured when a run was admitted, never to a per-channel last-active slot. `src/messaging/run-pin.ts` builds the identity block (origin/requestId/scope/sessionId/remoteKey/target) that every `agent_done` carries, and `resolveForwarderTarget` refuses an event with no destination or one addressed to another channel. Slack, Discord and Telegram forwarders no longer accept a `getLastTarget`/`getLastChatId` option, so web and CLI turns are not mirrored into chat rooms. Heartbeat destinations are complete or held: a Slack destination needs a thread or an explicit `scope: "channel_root"`, an absent destination sends nothing, and `authorizeExplicitTarget` vouches for a send without rewriting its address. Slack progress cards end their live loop on `message_not_found`/`cant_update_message` or three consecutive failures rather than retrying a dead message. See `structure/telegram.md` and `structure/server_api.md`. +- Channel forwarders deliver to the destination captured when a run was admitted, never to a per-channel last-active slot. `src/messaging/run-pin.ts` builds the identity block (origin/requestId/scope/sessionId/remoteKey/target) that every `agent_done` carries, and `resolveForwarderTarget` refuses an event with no destination or one addressed to another channel. Slack, Discord and Telegram forwarders no longer accept a `getLastTarget`/`getLastChatId` option, so web and CLI turns are not mirrored into chat rooms. Heartbeat destinations are complete or held: a Slack destination needs a thread or an explicit `scope: "channel_root"`, an absent destination sends nothing, threaded jobs verify `conversations.replies` before runner work, and a 25-minute server-owned `enforceDestination` grant is injected into print, native, employee and script runtimes before work so omitted targets pin and mismatches fail without process-global locking. Live hold reasons remain visible to GET/UI until recovery. `authorizeExplicitTarget` vouches for a send without rewriting its address. Slack progress cards end their live loop on `message_not_found`/`cant_update_message` or three consecutive failures rather than retrying a dead message. See `structure/telegram.md` and `structure/server_api.md`. - Auto (`permissions:auto`) grants qualified direct-local Jaw API authority across supported runtimes, independently of per-turn secrets. Keep actual/effective loopback, exact browser origin, proxy provenance, explicit outbound destinations and server-only resource options. Safe/custom keep existing scoped/operator paths; full API authority is instance-wide, distinct from provider Safe and task scope. Preserve no-descendant/read-only assignments, captured worker context and honest capability/receipt evidence. See `docs/slack-tools.md` and `structure/server_api.md`. - Slack group DMs use `message.mpim` and optional `mpim:history`; exact `channel_type: mpim` mentions retain channel allowlist and thread policy, never the one-to-one DM bypass. An install without `mpim:history` receives no group-DM traffic at all; that gap is reported in `missingCapabilities` and logged as a reception limitation rather than failing credential validation. An absent scope header is unknown and a present empty header is a known empty grant. Keep `structure/telegram.md` and the validation API docs synchronized. diff --git a/CLAUDE.md b/CLAUDE.md index 86e3e11d2..a28025023 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -24,7 +24,7 @@ Native Code interruption seals callbacks before persisting accepted buffered con ## Current Runtime Notes -- Channel forwarders deliver to the destination captured when a run was admitted, never to a per-channel last-active slot. `src/messaging/run-pin.ts` builds the identity block (origin/requestId/scope/sessionId/remoteKey/target) that every `agent_done` carries, and `resolveForwarderTarget` refuses an event with no destination or one addressed to another channel. Slack, Discord and Telegram forwarders no longer accept a `getLastTarget`/`getLastChatId` option, so web and CLI turns are not mirrored into chat rooms. Heartbeat destinations are complete or held: a Slack destination needs a thread or an explicit `scope: "channel_root"`, an absent destination sends nothing, and `authorizeExplicitTarget` vouches for a send without rewriting its address. Slack progress cards end their live loop on `message_not_found`/`cant_update_message` or three consecutive failures rather than retrying a dead message. See `structure/telegram.md` and `structure/server_api.md`. +- Channel forwarders deliver to the destination captured when a run was admitted, never to a per-channel last-active slot. `src/messaging/run-pin.ts` builds the identity block (origin/requestId/scope/sessionId/remoteKey/target) that every `agent_done` carries, and `resolveForwarderTarget` refuses an event with no destination or one addressed to another channel. Slack, Discord and Telegram forwarders no longer accept a `getLastTarget`/`getLastChatId` option, so web and CLI turns are not mirrored into chat rooms. Heartbeat destinations are complete or held: a Slack destination needs a thread or an explicit `scope: "channel_root"`, an absent destination sends nothing, threaded jobs verify `conversations.replies` before runner work, and a 25-minute server-owned `enforceDestination` grant is injected into print, native, employee and script runtimes before work so omitted targets pin and mismatches fail without process-global locking. Live hold reasons remain visible to GET/UI until recovery. `authorizeExplicitTarget` vouches for a send without rewriting its address. Slack progress cards end their live loop on `message_not_found`/`cant_update_message` or three consecutive failures rather than retrying a dead message. See `structure/telegram.md` and `structure/server_api.md`. - File sends across Slack, Telegram and Discord share one confirmation vocabulary. A send the vendor will not name is refused rather than reported as delivered (Slack keeps its `files[]` echo requirement, Telegram requires `message_id` > 0, Discord requires a readable Create Message body): those are `ok:false` with `confirmation: 'unconfirmed'`, replacing the older `ok:true, ambiguous:true` no consumer read. Anything forwarding a file result must preserve `confirmation`, or the caption posts twice. See `structure/infra.md` and `structure/telegram.md`. diff --git a/src/agent/pi-runtime.ts b/src/agent/pi-runtime.ts index 5ac73f3e9..54c25536c 100644 --- a/src/agent/pi-runtime.ts +++ b/src/agent/pi-runtime.ts @@ -710,9 +710,10 @@ function launchPiRpcExecution(profile: PiProfile, pi: PiSettings, options: { cwd: string; sessionId?: string; root?: string; + env?: NodeJS.ProcessEnv; }) { const dir = ensurePiRuntimeConfig(pi, profile.id, options.effort || '', options.root); - const inherited = { ...process.env }, cwd = options.cwd; + const inherited = { ...(options.env ?? process.env) }, cwd = options.cwd; const cmd = resolvePiCommand(inherited); const args = [ ...cmd.baseArgs, @@ -798,6 +799,7 @@ export function spawnPersistentPiRpc(profile: PiProfile, pi: PiSettings, options cwd: string; sessionId?: string; root?: string; + env?: NodeJS.ProcessEnv; }): PiRpcSession { const profileId = profile.id, initialEffort = options.effort; const { cmd, child, owner, startVersionProbe } = launchPiRpcExecution(profile, pi, options); @@ -1041,6 +1043,7 @@ export function spawnPiRpc(profile: PiProfile, pi: PiSettings, options: { onEvent?: (event: PiRuntimeEvent) => void; onRawRecord?: (record: unknown) => void; root?: string; + env?: NodeJS.ProcessEnv; }): { child: ChildProcess; done: Promise; cleanup: Promise } { const effort = options.effort; diff --git a/src/agent/runtime-pool.ts b/src/agent/runtime-pool.ts index 4c4c6a8dd..2974d555c 100644 --- a/src/agent/runtime-pool.ts +++ b/src/agent/runtime-pool.ts @@ -69,6 +69,7 @@ export interface PiAcquireOptions { profileFp: string; }; piSettings: unknown; + env?: NodeJS.ProcessEnv; storedSessionId?: string | null; instructions?: string; forceNew?: boolean; @@ -516,6 +517,7 @@ async function createPiEntry( model: opts.key.model, effort: opts.key.effort, cwd: opts.key.cwd, + ...(opts.env ? { env: opts.env } : {}), ...(opts.forceNew || !opts.storedSessionId ? {} : { sessionId: opts.storedSessionId }), }); if (store.entries.get(key) !== creating) { diff --git a/src/agent/spawn.ts b/src/agent/spawn.ts index 7412e6f42..4eb8d90b0 100644 --- a/src/agent/spawn.ts +++ b/src/agent/spawn.ts @@ -1460,6 +1460,16 @@ export function spawnAgent(prompt: string, opts: SpawnOpts = {}): SpawnResult { const envDefaultsCli = cli; const cliEnv = applyCliEnvDefaults(envDefaultsCli, opts.env); const spawnEnv = makeCleanEnv(cliEnv); + // Capture a request grant before ANY runtime branch acquires or launches a + // process. Scheduled work also permits employee runtimes: its server-owned + // grant is narrower than their ordinary credentials and is the only way + // they may address Slack during this assignment. + const slackToolGrantEligible = origin === 'heartbeat' + || (!isEmployee && ['cursor', 'claude', 'codex', 'grok'].includes(cli)); + const slackToolGrant = slackToolGrantEligible + ? activateSlackToolGrant(opts.requestId, scopeKey, chatSessionId) + : undefined; + if (slackToolGrant) spawnEnv[SLACK_TOOL_GRANT_ENV] = slackToolGrant; const bucketRow = currentBucket ? getSessionBucket.get(currentBucket) as SessionBucketRow | undefined : null; const bucketSessionId = bucketRow?.session_id || null; const bucketModel = typeof bucketRow?.model === 'string' ? bucketRow.model : null; @@ -1789,7 +1799,8 @@ export function spawnAgent(prompt: string, opts: SpawnOpts = {}): SpawnResult { prompt: { text: withSteerContext(withHistoryPrompt(prompt, historyBlock), opts.steerContext), ...(opts.images ? { images: opts.images } : {}) }, audience: traceAudience, liveScope: effectiveLiveScope, parentLiveScope: parentLiveScopeForChild, ...(opts.runtimeParentItemId ? { parentItemId: opts.runtimeParentItemId } : {}), - storedSessionId: resumeSessionId, fresh: forceNew || opts._skipResume === true || isEmployee, + storedSessionId: resumeSessionId, + fresh: forceNew || opts._skipResume === true || isEmployee || Boolean(slackToolGrant), cleanupUnleased: cleanupClaudeWorker, isCurrent: ownedRun, isCurrentOwner: token => isCurrentSessionOwner(token, scopeKey), consumeKillReason, activity: identity => opts.lifecycle?.onActivity?.('native-runtime', identity), @@ -1977,7 +1988,10 @@ export function spawnAgent(prompt: string, opts: SpawnOpts = {}): SpawnResult { key: { scopeKey, cwd: nativeCwd, model: runtimeModel === 'default' ? '' : runtimeModel, effort, permissions }, binary: detected.path || (grok ? 'grok' : 'cursor-agent'), env: spawnEnv, promptTimeoutMs: resolvedAgyPrintTimeoutMs, persistenceOwner, isCurrentOwner: token => isCurrentSessionOwner(token, scopeKey), canAcquire: ownsRun, - storedSessionId: resumeSessionId, forceNew, signal, + storedSessionId: resumeSessionId, + forceNew: forceNew || Boolean(slackToolGrant), + ...(slackToolGrant ? { lifetime: 'request' as const } : {}), + signal, }); facade = new AcpRuntimeSession(lease.session, { provider: cli, deferTurnEnd: true, ...(grok ? grokMainOptions : { createReplacement: io => new AcpReplacement(io), prepareReplacement }), @@ -2746,7 +2760,7 @@ export function spawnAgent(prompt: string, opts: SpawnOpts = {}): SpawnResult { execution = spawnPiRpc(profile, pi, { prompt: piPrompt, model: runtimeModel, ...(piSessionId ? { sessionId: piSessionId } : {}), - effort, cwd: spawnCwd, sysPrompt: piSysPrompt, + effort, cwd: spawnCwd, sysPrompt: piSysPrompt, env: spawnEnv, onEvent: onPiEvent, onRawRecord: onPiRawRecord, }); } catch (error) { @@ -2777,9 +2791,10 @@ export function spawnAgent(prompt: string, opts: SpawnOpts = {}): SpawnResult { profileFp, }, piSettings: pi, + env: spawnEnv, storedSessionId: piSessionId || null, instructions: piSysPrompt, - forceNew, + forceNew: forceNew || Boolean(slackToolGrant), }).then((lease) => { mainRun!.starting = false; if (activeMainProcesses.get(scopeKey) !== mainRun || !isCurrentSessionOwner(persistenceOwner, scopeKey)) { @@ -3270,7 +3285,7 @@ export function spawnAgent(prompt: string, opts: SpawnOpts = {}): SpawnResult { }, storedThreadId: resumeSessionId || null, instructions: sysPrompt, - forceNew, + forceNew: forceNew || Boolean(slackToolGrant), }); return { kind: 'lease', lease }; } @@ -3332,7 +3347,7 @@ export function spawnAgent(prompt: string, opts: SpawnOpts = {}): SpawnResult { bucketKey: currentBucket!, storedThreadId: resumeSessionId || null, instructions: sysPrompt, - forceNew, + forceNew: forceNew || Boolean(slackToolGrant), waitMs: deadlineAt - Date.now(), }), (lateLease) => { lateLease.release(); }); if (acquireWasCancelled()) { @@ -3470,9 +3485,6 @@ export function spawnAgent(prompt: string, opts: SpawnOpts = {}): SpawnResult { } // The snapshot has to predate the child; the helper owns that ordering (073 §2.4). const kiroPlainText = isKiroPlainTextCli(cli, effectiveProvider); - const slackToolGrant = !isEmployee && ['cursor', 'claude', 'codex', 'grok'].includes(cli) - ? activateSlackToolGrant(opts.requestId, scopeKey, chatSessionId) : undefined; - if (slackToolGrant) launchEnv[SLACK_TOOL_GRANT_ENV] = slackToolGrant; const { child, kiroConversationIdsBefore, kiroSpawnStartedAt } = spawnWithKiroSnapshot({ kiroPlainText, isFreshMainRun: !isResume && !empSid, diff --git a/src/memory/heartbeat-destination.ts b/src/memory/heartbeat-destination.ts index 46440879c..46dd862a0 100644 --- a/src/memory/heartbeat-destination.ts +++ b/src/memory/heartbeat-destination.ts @@ -14,11 +14,17 @@ import { isHeartbeatDestination, type HeartbeatDestination } from '../core/config.js'; import { targetFromChatId } from '../messaging/send.js'; import type { RemoteTarget } from '../messaging/types.js'; +import { fetchSlackReplies } from '../slack/history.js'; +import type { SlackFetch } from '../slack/api.js'; export type HeartbeatHoldReason = | 'unbound_destination' | 'incomplete_destination' - | 'malformed_destination'; + | 'malformed_destination' + | 'thread_channel_mismatch' + | 'stale_thread' + | 'live_lookup_failed' + | 'slack_grant_unavailable'; export type HeartbeatBinding = | { state: 'bound'; target: RemoteTarget } @@ -70,5 +76,67 @@ export function heartbeatHoldMessage(reason: HeartbeatHoldReason): string { return 'destination names a channel but no thread — add a thread, or set scope:"channel_root" to post to the channel itself'; case 'malformed_destination': return 'destination is malformed'; + case 'thread_channel_mismatch': + return 'the configured Slack thread does not belong to the configured channel'; + case 'stale_thread': + return 'the configured Slack thread no longer exists'; + case 'live_lookup_failed': + return 'the configured Slack thread could not be verified for this tick'; + case 'slack_grant_unavailable': + return 'destination-bound Slack authority could not be reserved for this tick'; + } +} + +export type HeartbeatThreadVerificationOptions = { + token: string; + fetchImpl?: SlackFetch; + signal?: AbortSignal; +}; + +const STALE_THREAD_CODES = new Set(['thread_not_found', 'message_not_found']); +const MISMATCH_CODES = new Set(['channel_not_found', 'not_in_channel']); + +/** + * Prove a threaded Slack destination still names a parent in that channel. + * + * The check deliberately has no positive cache. A success from the previous + * tick says nothing about a thread that was deleted or a bot removed from its + * channel before this one. A 429 is not retried here either: the heartbeat owns + * a future tick, so waiting and issuing a second read only spends more shared + * Slack budget. Every uncertain result fails this tick closed. + */ +export async function verifyHeartbeatThreadBindingLive( + destination: unknown, + options: HeartbeatThreadVerificationOptions, +): Promise { + const binding = resolveHeartbeatBinding(destination); + if (binding.state === 'held') return binding; + const { target } = binding; + if (target.channel !== 'slack' || !target.threadId) return binding; + if (!options.token.trim()) return { state: 'held', reason: 'live_lookup_failed' }; + + try { + const result = await fetchSlackReplies(options.token, target.targetId, target.threadId, { + limit: 1, + noRetry: true, + noRetryOnRateLimit: true, + sensitiveResponse: true, + ...(options.fetchImpl ? { fetchImpl: options.fetchImpl } : {}), + ...(options.signal ? { signal: options.signal } : {}), + }); + if (!result.ok) { + if (result.code && STALE_THREAD_CODES.has(result.code)) { + return { state: 'held', reason: 'stale_thread' }; + } + if (result.code && MISMATCH_CODES.has(result.code)) { + return { state: 'held', reason: 'thread_channel_mismatch' }; + } + return { state: 'held', reason: 'live_lookup_failed' }; + } + return result.messages[0]?.ts === target.threadId + ? binding + : { state: 'held', reason: 'thread_channel_mismatch' }; + } catch { + return { state: 'held', reason: 'live_lookup_failed' }; } } diff --git a/src/memory/heartbeat.ts b/src/memory/heartbeat.ts index 3f90e9bf3..722293c69 100644 --- a/src/memory/heartbeat.ts +++ b/src/memory/heartbeat.ts @@ -16,10 +16,23 @@ import { nextDeliverySeq, wasSelfDelivered } from '../messaging/turn-delivery.js import { isHeartbeatMentionWatch } from '../core/config.js'; import type { HeartbeatMentionWatch } from '../core/config.js'; import { runMentionWatchTick } from './heartbeat-mention-watch.js'; -import { resolveHeartbeatBinding, heartbeatHoldMessage, type HeartbeatBinding } from './heartbeat-destination.js'; +import { + resolveHeartbeatBinding, + verifyHeartbeatThreadBindingLive, + heartbeatHoldMessage, + type HeartbeatBinding, + type HeartbeatHoldReason, +} from './heartbeat-destination.js'; import { watchNamespace } from './mention-watch-ledger.js'; import { detectLegacyMentionWatch, isQuarantined } from './legacy-mention-watch-quarantine.js'; import { verifiedSlackWorkspace } from '../slack/verified-workspace.js'; +import { + reserveSlackToolGrant, + activateSlackToolGrant, + revokeSlackToolGrant, + slackCredentialKey, + SLACK_TOOL_GRANT_ENV, +} from '../slack/tool-context.js'; import { buildRemoteBindingKey } from '../messaging/session-key.js'; import { getRemoteBoundSessionId, resolveOrCreateRemoteSession } from '../core/chat-sessions.js'; import { hasChatSessionWork } from '../orchestrator/session-work.js'; @@ -73,6 +86,50 @@ interface PendingHeartbeatJob { policy?: HeartbeatPendingPolicy; } const pendingJobs: PendingHeartbeatJob[] = []; +type LiveDestinationHold = { destination: string; reason: HeartbeatHoldReason; observedAt: number }; +const liveDestinationHolds = new Map(); +type HeartbeatDestinationJobRef = { id?: unknown; name?: unknown; destination?: unknown }; + +function heartbeatJobKey(job: HeartbeatDestinationJobRef): string { + return String(job.id ?? job.name ?? ''); +} + +function destinationFingerprint(destination: unknown): string { + try { return JSON.stringify(destination ?? null); } + catch { return '[unserializable]'; } +} + +/** Process-local live hold for GET/UI. A destination edit invalidates it + * immediately; a successful later tick clears it. The timer remains armed so a + * transient Slack failure can recover without an operator save. */ +export function getHeartbeatLiveDestinationHold(job: HeartbeatDestinationJobRef): HeartbeatHoldReason | null { + const key = heartbeatJobKey(job); + if (!key) return null; + const hold = liveDestinationHolds.get(key); + if (!hold) return null; + if (hold.destination !== destinationFingerprint(job.destination)) { + liveDestinationHolds.delete(key); + return null; + } + return hold.reason; +} + +export function updateHeartbeatLiveDestinationHold( + job: HeartbeatDestinationJobRef, + reason: HeartbeatHoldReason | null, +): void { + const key = heartbeatJobKey(job); + if (!key) return; + if (reason === null) { + liveDestinationHolds.delete(key); + return; + } + liveDestinationHolds.set(key, { + destination: destinationFingerprint(job.destination), + reason, + observedAt: Date.now(), + }); +} export function isHeartbeatQuietOutput(result: string, extraMarkers: string[] = []): boolean { return ['[SILENT]', ...extraMarkers].some(marker => marker.length > 0 && result.includes(marker)); @@ -185,25 +242,43 @@ export function decideHeartbeatReport(report: HeartbeatReport, policy: string): return { send: true, anchor: true, delivered: true }; } -export function runHeartbeatScript(command: string[]): Promise { +export function runHeartbeatScript( + command: string[], + extraEnv: Record = {}, +): Promise { return new Promise(resolve => { const [file, ...args] = command; if (!file) { resolve(parseHeartbeatReport('', 1)); return; } - execFile(file, args, { timeout: 10 * 60_000, maxBuffer: 64 * 1024 }, (error, stdout, stderr) => { + execFile(file, args, { + timeout: 10 * 60_000, + maxBuffer: 64 * 1024, + env: { ...process.env, ...extraEnv }, + }, (error, stdout, stderr) => { const code = error && typeof error === 'object' && 'code' in error && typeof error.code === 'number' ? error.code : error ? 1 : 0; resolve(parseHeartbeatReport([stdout, stderr].filter(Boolean).join('\n'), code)); }); }); } -async function runEmployee(job: Record, prompt: string): Promise { +async function runEmployee( + job: Record, + prompt: string, + requestId: string, + target: RemoteTarget, +): Promise { const emp = (getEmployees.all() as EmployeeRow[]).find(row => row.name === job["employee"]); if (!emp) return parseHeartbeatReport('status: failed\nsummary: employee not found'); try { const slot = claimWorker(emp, prompt, { origin: 'heartbeat', scopeId: HEARTBEAT_SCOPE, chatSessionId: 'default' }); try { const ap = { agent: emp.name, role: emp.role || 'general developer', task: prompt, parallel: false, currentPhase: 0, currentPhaseIdx: 0, phaseProfile: [0], mutable: false, scope: null, task_tags: ['heartbeat'] }; - const result = await runSingleAgent(ap, emp, { tag: `heartbeat:${job["id"] || job["name"]}` }, 1, { origin: 'heartbeat' }, []); + const result = await runSingleAgent(ap, emp, { tag: `heartbeat:${job["id"] || job["name"]}` }, 1, { + origin: 'heartbeat', + scopeKey: HEARTBEAT_SCOPE, + chatSessionId: 'default', + requestId, + target, + }, []); const text = String(result["text"] || ''); finishWorker(slot.agentId, text, Array.isArray(result["tools"]) ? result["tools"] : []); // finishWorker arms a replay for a Boss to collect. A heartbeat has no @@ -471,7 +546,32 @@ function buildMentionWatchPrompt( ].join('\n'); } -export async function runHeartbeatJob(job: Record) { +export type HeartbeatJobDeps = { + verifyDestination?: (destination: unknown) => Promise; + reserveDestinationGrant?: (binding: Extract, requestId: string) => + Promise<(() => void) | null>; + activateDestinationGrant?: (requestId: string) => string | undefined; +}; + +async function reserveHeartbeatDestinationGrant( + binding: Extract, + requestId: string, +): Promise<(() => void) | null> { + if (binding.target.channel !== 'slack') return () => {}; + const token = String(settings["slack"]?.botToken ?? '').trim(); + const workspace = await verifiedSlackWorkspace(token, { sensitiveResponse: true }).catch(() => null); + if (!workspace?.userId) return null; + const reserved = reserveSlackToolGrant({ + teamId: workspace.teamId, + actorId: workspace.userId, + destination: binding.target, + credentialKey: slackCredentialKey(token), + enforceDestination: true, + }, { requestId, scope: HEARTBEAT_SCOPE, chatSessionId: 'default' }); + return reserved ? () => revokeSlackToolGrant(requestId) : null; +} + +export async function runHeartbeatJob(job: Record, deps: HeartbeatJobDeps = {}) { const runner = job["runner"] || 'main'; if (runner === 'main' && getState('default') !== 'IDLE') { const queued = queueHeartbeatJob(job, 'pabcd_active', 'defer'); @@ -506,6 +606,20 @@ export async function runHeartbeatJob(job: Record) { await runMentionWatchJob(job, watch); return; } + // Resolve and, for a Slack thread, prove the destination BEFORE spending + // model, employee or script work. A report whose address is stale or + // unverified must not run first and discover only at delivery time that + // it has nowhere safe to go (#745). + const destinationBinding = await (deps.verifyDestination + ?? ((destination: unknown) => verifyHeartbeatThreadBindingLive(destination, { + token: String(settings["slack"]?.botToken ?? ''), + })))(job["destination"]); + if (destinationBinding.state === 'held') { + updateHeartbeatLiveDestinationHold(job, destinationBinding.reason); + log.error(`[heartbeat:${job["name"]}] refuse: ${destinationBinding.reason} — ${heartbeatHoldMessage(destinationBinding.reason)}`); + return; + } + updateHeartbeatLiveDestinationHold(job, null); const schedule = normalizeHeartbeatSchedule(job["schedule"]); const timeZone = getHeartbeatScheduleTimeZone(schedule); const now = formatHeartbeatNow(schedule); @@ -513,19 +627,78 @@ export async function runHeartbeatJob(job: Record) { const goalSection = goalPrompt ? `\n\n--- Active Goal ---\n${goalPrompt}\n--- End Goal ---\n` : ''; const prompt = `[heartbeat:${job["name"]}] 현재 시간: ${now} (${timeZone})\n\nBefore responding, you MUST search memory (cli-jaw memory search) for recent conversation context, user preferences, and ongoing tasks. Use this context to ground your response.${goalSection}\n\n${job["prompt"] || '정기 점검입니다. 할 일 없으면 [SILENT]로 응답.'}`; log.info(`[heartbeat:${job["name"]}] tick (${describeHeartbeatSchedule(schedule)})`); + const withDestinationGuard = async ( + operation: (requestId: string) => Promise, + ): Promise<{ ok: true; value: T } | { ok: false }> => { + const requestId = crypto.randomUUID(); + const release = await (deps.reserveDestinationGrant ?? reserveHeartbeatDestinationGrant)( + destinationBinding, + requestId, + ); + if (!release) { + updateHeartbeatLiveDestinationHold(job, 'slack_grant_unavailable'); + return { ok: false }; + } + try { + return { ok: true, value: await operation(requestId) }; + } finally { + release(); + } + }; let rawResult: string; if (runner === 'employee') { - rawResult = (await runEmployee(job, prompt)).raw; + const guarded = await withDestinationGuard( + requestId => runEmployee(job, prompt, requestId, destinationBinding.target).then(report => report.raw), + ); + if (!guarded.ok) { + log.error(`[heartbeat:${job["name"]}] refuse: slack_grant_unavailable — employee authority could not be reserved`); + return; + } + rawResult = guarded.value; } else if (runner === 'script') { - const scriptReport = await runHeartbeatScript(job["command"] || []); + const guarded = await withDestinationGuard(async requestId => { + let grantEnv: Record = {}; + if (destinationBinding.target.channel === 'slack') { + const secret = (deps.activateDestinationGrant + ?? (id => activateSlackToolGrant(id, HEARTBEAT_SCOPE, 'default')))(requestId); + if (!secret) throw new Error('slack_grant_activation_failed'); + grantEnv = { [SLACK_TOOL_GRANT_ENV]: secret }; + } + return runHeartbeatScript(job["command"] || [], grantEnv); + }); + if (!guarded.ok) { + log.error(`[heartbeat:${job["name"]}] refuse: slack_grant_unavailable — script authority could not be reserved`); + return; + } + const scriptReport = guarded.value; rawResult = scriptReport.status === 'failed' && !/^status:/m.test(scriptReport.raw) ? `${scriptReport.raw}\nstatus: failed\nsummary: ${scriptReport.summary || 'script failed'}` : scriptReport.raw; } else { - const first = await orchestrateAndCollectData(prompt, { origin: 'heartbeat', requestId: crypto.randomUUID(), scope: HEARTBEAT_SCOPE, chatSessionId: 'default' }); + const collect = async () => { + const guarded = await withDestinationGuard( + requestId => orchestrateAndCollectData(prompt, { + origin: 'heartbeat', + requestId, + scope: HEARTBEAT_SCOPE, + chatSessionId: 'default', + target: destinationBinding.target, + }), + ); + return guarded.ok ? guarded.value : null; + }; + const first = await collect(); + if (!first) { + log.error(`[heartbeat:${job["name"]}] refuse: slack_grant_unavailable — destination-bound Slack authority could not be reserved`); + return; + } const collected = first.data.agyPlannerOnly === true - ? await orchestrateAndCollectData(prompt, { origin: 'heartbeat', requestId: crypto.randomUUID(), scope: HEARTBEAT_SCOPE, chatSessionId: 'default' }) + ? await collect() : first; + if (!collected) { + log.error(`[heartbeat:${job["name"]}] refuse: slack_grant_unavailable — retry authority could not be reserved`); + return; + } rawResult = String(collected.text); } const result = applyOutputPolicy(rawResult, { scope: 'heartbeat', channel: 'active' }).text; @@ -550,16 +723,10 @@ export async function runHeartbeatJob(job: Record) { // active-channel fallback that used to stand in for a missing // destination is gone: it delivered to whoever spoke to the bot most // recently, which is not a property of this job at all (#437, #745). - const binding = heartbeatTarget(job["destination"]); - if (binding.state === 'held') { - log.error(`[heartbeat:${job["name"]}] refuse: ${binding.reason} — ${heartbeatHoldMessage(binding.reason)}`); - } const sendResult = !decision.send ? { ok: true as const } - : binding.state === 'bound' - ? await sendChannelOutput({ channel: binding.target.channel, type: 'text', text: formatted, - target: binding.target, allowActiveFallback: false }) - : { ok: false as const, error: `heartbeat destination held: ${binding.reason}` }; + : await sendChannelOutput({ channel: destinationBinding.target.channel, type: 'text', text: formatted, + target: destinationBinding.target, allowActiveFallback: false }); if (!sendResult.ok) { log.error(`[heartbeat:${job["name"]}] send failed: ${sendResult.error}`); } @@ -570,8 +737,8 @@ export async function runHeartbeatJob(job: Record) { try { insertHeartbeatAnchor.run( job["id"], job["name"], settings["workingDir"], - binding.state === 'bound' ? binding.target.channel : 'active', - binding.state === 'bound' ? binding.target.targetId : null, + destinationBinding.target.channel, + destinationBinding.target.targetId, job["prompt"], decision.delivered ? formatted : `[quiet] ${formatted}`, now, decision.delivered ? now : null, ); } catch (e) { diff --git a/src/orchestrator/distribute.ts b/src/orchestrator/distribute.ts index 3f5c130eb..d1b626c1c 100644 --- a/src/orchestrator/distribute.ts +++ b/src/orchestrator/distribute.ts @@ -19,6 +19,7 @@ import { updateWorkerPhase, } from './worker-registry.js'; import { sanitizeToolLogForDurableStorage } from '../shared/tool-log-sanitize.js'; +import { isRemoteTarget } from '../messaging/types.js'; // ─── Phase Constants (shared with pipeline.ts) ─────── @@ -414,6 +415,7 @@ ${worklogBlock}`.trim(); ...(typeof meta["scopeKey"] === 'string' ? { scopeKey: meta["scopeKey"] } : {}), ...(typeof meta["chatSessionId"] === 'string' ? { chatSessionId: meta["chatSessionId"] } : {}), ...(typeof meta["requestId"] === 'string' ? { requestId: meta["requestId"] } : {}), + ...(isRemoteTarget(meta["target"]) ? { target: { ...meta["target"] } } : {}), ...(assignmentPermissions !== undefined ? { permissions: assignmentPermissions } : {}), env: { JAW_EMPLOYEE_MODE: '1', diff --git a/src/routes/heartbeat.ts b/src/routes/heartbeat.ts index d5920f614..9f4ddca9a 100644 --- a/src/routes/heartbeat.ts +++ b/src/routes/heartbeat.ts @@ -2,7 +2,7 @@ import type { Express } from 'express'; import type { AuthMiddleware } from './types.js'; import { loadHeartbeatFile, saveHeartbeatFile, isHeartbeatDestination, isHeartbeatMentionWatch, settings } from '../core/config.js'; import type { HeartbeatDestination, HeartbeatMentionWatch, HeartbeatJob } from '../core/config.js'; -import { startHeartbeat } from '../memory/heartbeat.js'; +import { getHeartbeatLiveDestinationHold, startHeartbeat } from '../memory/heartbeat.js'; import { isCompleteHeartbeatDestination, resolveHeartbeatBinding } from '../memory/heartbeat-destination.js'; import { validateHeartbeatScheduleInput } from '../memory/heartbeat-schedule.js'; import { approveLegacyFreshStart, quarantineState, detectLegacyMentionWatch, isQuarantined } from '../memory/legacy-mention-watch-quarantine.js'; @@ -50,7 +50,9 @@ export function resolveHeartbeatDestination( function heldForDestination(job: HeartbeatJob): (HeartbeatJob & { held: string }) | null { if (job.mentionWatch) return null; const binding = resolveHeartbeatBinding(job.destination); - return binding.state === 'held' ? { ...job, held: binding.reason } : null; + if (binding.state === 'held') return { ...job, held: binding.reason }; + const live = getHeartbeatLiveDestinationHold(job); + return live ? { ...job, held: live } : null; } /** Resolve the mention-watch a PUT should persist. diff --git a/src/routes/messaging.ts b/src/routes/messaging.ts index 04a65d1b3..46e96346c 100644 --- a/src/routes/messaging.ts +++ b/src/routes/messaging.ts @@ -1,6 +1,6 @@ import { slackCredentialKey } from '../slack/tool-context.js'; import type { Express, Request, Response } from 'express'; -import { resolveSlackToolPrincipal, withSlackToolAccess, slackToolDenied, type SlackOperatorValidator } from '../slack/tool-access.js'; +import { resolveSlackToolPrincipal, slackToolContext, withSlackToolAccess, slackToolDenied, type SlackOperatorValidator } from '../slack/tool-access.js'; import { getHomeChannel } from '../messaging/runtime.js'; import type { AuthMiddleware } from './types.js'; import { httpStatus, httpCode, httpDetail } from './_http-error.js'; @@ -172,7 +172,8 @@ export function registerMessagingRoutes(app: Express, requireAuth: AuthMiddlewar const principal = principalFor(req, full); const client = getSlackSendClient(); if (!client.token) throw slackToolDenied('slack_unavailable', 503); - if (principal.kind === 'turn') { + const scopedGrant = slackToolContext(principal); + if (principal.kind === 'turn' || scopedGrant?.enforceDestination === true) { if (request.filePath) { const inside = (root: string) => { const rel = relative(fs.realpathSync(root), request.filePath!); @@ -180,7 +181,7 @@ export function registerMessagingRoutes(app: Express, requireAuth: AuthMiddlewar }; if (inside(JAW_HOME) && (!fs.existsSync(UPLOADS_DIR) || !inside(UPLOADS_DIR))) throw slackToolDenied('slack_private_home_file_denied'); } - const destination = principal.grant.destination; + const destination = scopedGrant!.destination; const candidate = request.target ?? request.turnTarget; if ((candidate && (candidate.targetId !== destination.targetId || (candidate.threadId ?? '') !== (destination.threadId ?? ''))) || (request.chatId !== undefined && String(request.chatId) !== destination.targetId)) throw slackToolDenied('slack_destination_mismatch'); @@ -430,7 +431,23 @@ export function registerMessagingRoutes(app: Express, requireAuth: AuthMiddlewar // it is a fact about how the send arrived, not about its body, and // an agent must not be able to claim it by putting a field in JSON. const full = fullFor(req); - const result = await sendSlackAware(req, normalizeChannelSendRequest(req.body, { fullAccess: full }), full); + // A server-owned scheduled turn carries an enforced grant even when + // the instance runs Auto/full-local. Let that grant supply the + // destination before the full-access normalizer rejects an omitted + // address; sendSlackAware then pins or rejects any supplied address. + const preflightGrant = full && req.body?.channel === 'slack' + ? slackToolContext(principalFor(req, full)) + : undefined; + const enforcedDestination = preflightGrant?.enforceDestination === true + ? preflightGrant.destination + : undefined; + const result = await sendSlackAware( + req, + normalizeChannelSendRequest(enforcedDestination && req.body?.target === undefined + ? { ...req.body, target: enforcedDestination } + : req.body, { fullAccess: full }), + full, + ); if (!result.ok) { res.status(sendResultHttpStatus(result)).json(result); return; diff --git a/src/slack/history.ts b/src/slack/history.ts index edf028e38..7fbcf0f6c 100644 --- a/src/slack/history.ts +++ b/src/slack/history.ts @@ -181,6 +181,9 @@ export type SlackHistoryOpts = { * window can be applied. */ noRetryOnRateLimit?: boolean; + /** Do not retry any Slack error. Scheduled destination verification owns a + * future tick and must not spend a second shared API call in this one. */ + noRetry?: boolean; sensitiveResponse?: boolean; }; @@ -228,7 +231,7 @@ async function callWithRetry( // A 429 is retried by default (existing callers depend on it), but an // enrichment caller opts out: it applies its own suppression window, and a // retry would fire a second request before that window exists. - const retryable = isRetryableSlackError(result.error) + const retryable = opts.noRetry !== true && isRetryableSlackError(result.error) && !(opts.noRetryOnRateLimit && result.error === 'ratelimited'); if (!result.ok && retryable && !opts.signal?.aborted) { // One bounded retry after a short pause (Hermes uses 1s/2s; a single diff --git a/src/slack/tool-context.ts b/src/slack/tool-context.ts index a3db0ac81..4c6b02b32 100644 --- a/src/slack/tool-context.ts +++ b/src/slack/tool-context.ts @@ -3,8 +3,19 @@ import type { RemoteTarget } from '../messaging/types.js'; export const SLACK_TOOL_GRANT_ENV = 'JAW_SLACK_TURN_GRANT'; const GRANT_TTL_MS = 15 * 60_000; +const ENFORCED_GRANT_TTL_MS = 25 * 60_000; const GRANT_CAP = 128; -export type SlackToolSource = { teamId: string; actorId: string; destination: RemoteTarget; credentialKey: string; actionToken?: string }; +export type SlackToolSource = { + teamId: string; + actorId: string; + destination: RemoteTarget; + credentialKey: string; + actionToken?: string; + /** Server-owned scheduled work keeps its destination constraint even under + * full-local Auto authority. Ordinary interactive turn grants omit this: + * Auto remains instance-wide for the trusted operator (#745). */ + enforceDestination?: boolean; +}; export type SlackToolGrant = Readonly; type Entry = { grant: SlackToolGrant; secret: string; active: boolean; controller: AbortController; timer: ReturnType }; const requests = new Map(); @@ -12,7 +23,8 @@ const secrets = new Map(); export function slackCredentialKey(token: string): string { return createHash('sha256').update(token).digest('hex'); } -/** Only authenticated ingress supplies source; the HTTP API cannot mint grants. */ +/** Authenticated ingress or a server-owned heartbeat supplies source; the HTTP + * API itself cannot mint grants. */ export function reserveSlackToolGrant(source: SlackToolSource, binding: { requestId: string; scope: string; chatSessionId: string }): boolean { if (!/^[UW][A-Z0-9]{1,63}$/.test(source.actorId) || !/^T[A-Z0-9]{1,63}$/.test(source.teamId) || !/^[CGD][A-Z0-9]{1,63}$/.test(source.destination.targetId) || (source.actionToken !== undefined && source.actionToken.length > 8192) @@ -20,16 +32,20 @@ export function reserveSlackToolGrant(source: SlackToolSource, binding: { reques || requests.has(binding.requestId) || requests.size >= GRANT_CAP) return false; const controller = new AbortController(); const secret = `jaw-slack-grant-${randomBytes(32).toString('hex')}`; + // A collector may wait 20 minutes for a legitimate long native turn. + // Scheduled enforcement must outlive that owner; interactive grants retain + // their existing 15-minute window. + const ttlMs = source.enforceDestination === true ? ENFORCED_GRANT_TTL_MS : GRANT_TTL_MS; const grant: SlackToolGrant = Object.freeze({ ...source, destination: Object.freeze({ ...source.destination }), ...binding, - expiresAt: Date.now() + GRANT_TTL_MS, signal: controller.signal }); - const timer = setTimeout(() => revokeSlackToolGrant(binding.requestId), GRANT_TTL_MS); + expiresAt: Date.now() + ttlMs, signal: controller.signal }); + const timer = setTimeout(() => revokeSlackToolGrant(binding.requestId), ttlMs); timer.unref?.(); const entry: Entry = { grant, secret, active: false, controller, timer }; requests.set(binding.requestId, entry); secrets.set(secret, entry); return true; } -/** Called only at a fresh main print process launch, never a pooled native lease. */ +/** Consume a reservation once before its dedicated process/acquisition launches. */ export function activateSlackToolGrant(requestId: string | undefined, scope: string, chatSessionId: string): string | undefined { const entry = requestId ? requests.get(requestId) : undefined; if (!entry || entry.active || entry.grant.scope !== scope || entry.grant.chatSessionId !== chatSessionId @@ -56,6 +72,21 @@ export function revokeSlackToolScope(scope?: string): void { for (const [id, entry] of requests) if (scope === undefined || entry.grant.scope === scope) revokeSlackToolGrant(id); } +/** Diagnostic/test view of a live server-owned destination reservation. + * + * This is not an authorization gate: process-global gating would block unrelated + * interactive Slack sends while a long heartbeat runs. Every scheduled runtime + * receives its own grant header instead. */ +export function hasActiveEnforcedSlackDestination(): boolean { + const now = Date.now(); + for (const entry of requests.values()) { + if (entry.grant.enforceDestination === true + && entry.grant.expiresAt > now + && !entry.controller.signal.aborted) return true; + } + return false; +} + export function redactSlackToolSecrets(text: string): string { return text.replace(/jaw-slack-(?:grant|operator)-[a-f0-9]{64}/g, '[REDACTED_SLACK_TOOL_CREDENTIAL]'); } diff --git a/structure/server_api.md b/structure/server_api.md index 81d4145e9..f9e5ad6ae 100644 --- a/structure/server_api.md +++ b/structure/server_api.md @@ -213,6 +213,10 @@ All trace routes set `Cache-Control: no-store` before auth/parsing. Activity dis `PUT /api/heartbeat`의 job은 `mentionWatch: { channel: "slack", userId: "U...", channelIds: ["C..."], maxHits?, since? }`를 선택적으로 받는다. `channelIds`는 비어 있지 않아야 하고 저장 시 `slack.channelIds` allowlist의 부분집합이어야 하며, 실행 tick 직전 현재 allowlist와 다시 교집합한다. job id가 같은 기존 값에 대해 필드가 없으면 상속하고, `null`이면 삭제하며, 잘못된 값은 `400 invalid heartbeat mention watch`다. 파일 로드 정규화에서 잘못된 `mentionWatch`는 해당 job을 `enabled: false`로 내린다. 기본 운영값은 비활성이고, 설정된 watch는 별도 daemon이 아니라 기존 `runHeartbeatJob`에서 실행된다. +일반 heartbeat job의 `destination`은 Slack에서 `{ channel:"slack", targetId, threadId }` 또는 명시적 채널 루트 `{ channel:"slack", targetId, scope:"channel_root" }`다. 없거나 불완전하면 `GET /api/heartbeat`가 `held: unbound_destination|incomplete_destination|malformed_destination`을 보이고 tick은 전송하지 않는다. threaded job은 runner 시작 전에 `conversations.replies`로 부모 ts가 해당 채널에 실제 존재하는지 읽는다. stale, 채널 불일치, 권한/429/인증/전송 실패는 각각 `stale_thread|thread_channel_mismatch|live_lookup_failed`로 그 tick을 fail-closed한다. live reason은 다음 성공 tick까지 GET/UI에 남되 enabled timer는 복구 확인을 위해 유지되고, destination을 편집하면 즉시 무효화된다. 성공 결과를 캐시하지 않고 어떤 오류도 tick 안에서 재시도하지 않는다. mention-watch는 방금 찾은 hit thread가 목적지라 이 검사 대상이 아니다. + +모든 Slack heartbeat runner는 같은 target에 25분짜리 `enforceDestination` grant를 예약한다(collector 상한 20분보다 길다). `spawnAgent`가 runtime 분기 전에 활성화하고 print/employee child, request-lifetime Cursor/Grok, fresh Codex App/Claude/Pi acquisition에 header를 주입한다. script도 명시적 child env로 같은 header를 받는다. 그래서 Auto/full-local에서도 target 생략은 저장 목적지로 고정되고 다른 target은 `slack_destination_mismatch`다. 일반 interactive Auto grant는 이 flag가 없고, heartbeat가 도는 동안에도 unrelated Auto send를 process-global로 잠그지 않는다. + 같은 job id를 한 요청에 두 번 보내면 `400 duplicate heartbeat job id`다. id는 ledger namespace의 일부라서, 두 job이 한 id를 쓰면 뒤쪽이 앞쪽의 cursor를 물려받아 그 아래 구간을 통째로 건너뛴다. `since`는 bootstrap 전용 floor다. 이미 cursor가 있는 watch에서는 cursor가 이기므로, `since`만 바꿔서 과거를 다시 읽게 만들 수는 없다. @@ -221,8 +225,8 @@ All trace routes set `Cache-Control: no-store` before auth/parsing. Activity dis `POST /api/heartbeat/:jobId/mention-watch-fresh-start` `{ since }` → 보류 해제. 빈 `since`는 `400`이다(floor 없는 watch는 도달 가능한 history를 거꾸로 훑어 이미 답한 것을 다시 답한다). workspace 검증(`auth.test`)이 유일한 await이고 그 뒤는 전부 동기다 — 검증 전에 hold와 파일을 snapshot한 뒤 await하면, 패배한 승인이 낡은 파일 사본으로 재개해 승자의 floor를 덮어쓴다(DB는 그 뒤에야 conflict를 알려 주므로 파일 손상은 이미 끝나 있다). 순서는 파일 저장(temp+rename) → 단일 트랜잭션(`pending`→`resolved` CAS → v1 archive → delete) → `startHeartbeat()`이며 교환 불가다. 사이에서 죽으면 새 floor만 저장되고 보류는 남으므로 재시도로 복구된다. 반대 순서는 옛 floor가 살아 있는 채로 보류를 풀어 backlog를 replay한다. 같은 `since`로 재시도하면 `already-resolved`, 다른 `since`면 `409`, 보류 이력이 없으면 `404`다. 일반 `PUT`의 `enabled: true`는 승인으로 읽지 않는다 — 모든 UI가 `mentionWatch`를 생략해 보내므로, 저장 클릭을 동의로 해석하면 무관한 편집이 보류를 풀어 버린다. -`POST /api/channel/send`에서 `channel`은 `telegram|discord|slack|active` transport다. 대화 ID는 `chat_id` 또는 `target.targetId`에 넣는다. Slack thread를 명시할 때 `target.threadId`는 reply ts가 아닌 parent message ts다. target을 생략하면 검증된 현재 대화와 thread를 사용한다. 빈 `slack.channelIds`는 임의 explicit channel을 열지 않으며, 이미 저장·검증된 `lastActive/latestSeen`과 같은 conversation/thread만 명시적으로 재사용할 수 있다. -새 채널 공지의 최상위 메시지는 `target.threadId: ""`로 명시한다. `threadId` 생략은 기존 활성 스레드 상속을 허용하므로 새 스레드 생성과 같지 않다. 게시된 루트의 주소를 확인한 뒤 답글의 부모 ts로 사용한다. +`POST /api/channel/send`에서 `channel`은 `telegram|discord|slack|active` transport다. 대화 ID는 `chat_id` 또는 `target.targetId`에 넣는다. Slack thread를 명시할 때 `target.threadId`는 reply ts가 아닌 parent message ts다. request-scoped turn grant가 있으면 target 생략은 그 grant의 대화와 thread로 고정된다. full-local은 명시적 주소가 필요하며, server-owned `enforceDestination` grant만 생략 주소를 공급한다. 빈 `slack.channelIds`는 임의 explicit channel을 열지 않으며, `lastActive/latestSeen`은 같은 대화에 보낼 수 있는지 확인하는 근거일 뿐 명시 주소의 thread를 바꾸지 않는다. +새 채널 공지의 최상위 메시지는 `target.threadId: ""` 또는 threadId 없는 명시 target으로 보낸다. 게시된 루트의 주소를 확인한 뒤 답글의 부모 ts로 사용한다. Slack text sends preserve Markdown and explicit Block Kit `blocks`, splitting multiple tables into separate messages. `ok:true` means every chunk was posted, independently of rendering verification. Inspect `delivery.verification` (`verified`, `failed`, or `unavailable`) and `delivery.messages` for each posted chunk's timestamp, verification error, table-content status, and feature evidence. A persisted mismatch is `failed`; missing permission, unavailable/malformed readback, or a missing timestamp is `unavailable`. Neither stops remaining posts or triggers reposting. Actual validation/POST failures retain `ok:false`; partial receipts include `postedChunks`, `totalChunks`, and `sent:true, retryable:false`. Never blindly resend posted chunks. Every chunk posts before any readback begins, and callers receive an onPosted hook at that point so the ACK reaction and queue notice settle at transport success instead of waiting on verification; a readback hit by its own timeout is reported as slack_readback_timeout, never slack_send_aborted. Persisted markdown blocks, data_table rows, and stored image blocks all count as delivery evidence on readback. `tableContent` compares ordered text, numeric value/display, links and supported styles; ordinary Markdown character references decode once while code and escaped ampersands stay literal. `richContent` and `sourceAccuracy` remain `not_checked`, and readback stays bounded to 1 MiB. diff --git a/structure/str_func.md b/structure/str_func.md index 64874ae5c..4e3e74033 100644 --- a/structure/str_func.md +++ b/structure/str_func.md @@ -145,7 +145,7 @@ cli-jaw/ │ │ ├── claude-runtime-run.ts ← native Claude main adaptation to shared host/lifecycle and fallback terminal ordering (295L) │ │ ├── prompt-context.ts ← history/operational/partial boundaries and bounded accepted Cursor redirects (201L) │ │ ├── steer-input-guard.ts ← transient scoped Stop fence through fallback enqueue (33L) -│ │ ├── spawn.ts ← CLI spawn + ACP/Codex App/Pi RPC/AGY/Kiro plain text/log session capture + v2 SQLite session resume + 큐 + 메모리 flush + 429 retry timer + isAgentBusy/isSteerInProgress + buildHistoryBlock compact cutoff + working_dir scoping + enqueue→processQueue race fix + QueueItem persistent DB queue + makeCleanEnv PATH augment (4115L) +│ │ ├── spawn.ts ← CLI spawn + ACP/Codex App/Pi RPC/AGY/Kiro plain text/log session capture + v2 SQLite session resume + 큐 + 메모리 flush + 429 retry timer + isAgentBusy/isSteerInProgress + buildHistoryBlock compact cutoff + working_dir scoping + enqueue→processQueue race fix + QueueItem persistent DB queue + makeCleanEnv PATH augment (4127L) │ │ ├── spawn/ ← spawn 서브모듈 (3 files) │ │ │ ├── queue.ts ← QueueItem persistent DB queue + processQueue race fix + enqueue/dequeue + drainRecoveredQueue (부팅 시 복구 큐 기동, server.ts가 transport 준비 후 호출) + `_fromQueue` 표식 (대기자 없는 턴을 채널이 답할 수 있게) (723L) │ │ │ ├── resume.ts ← session resume logic + stale resume detection (117L) @@ -168,7 +168,7 @@ cli-jaw/ │ │ ├── agy-bootstrap.ts ← AGY bootstrap/context preparation helpers (237L) │ │ ├── agy-capabilities.ts ← AGY `--help`/`--version` capability probe + cached optional flag support map + legacy emit-all fallback marker (124L) │ │ ├── agy-transcript-watcher.ts ← AGY transcript/log watcher and session-id extraction support (291L) -│ │ ├── pi-runtime.ts ← Pi profile 정규화 + isolated `PI_CODING_AGENT_DIR` models/settings 생성 + `pi --offline --list-models` discovery + `pi --mode rpc` JSONL parser/spawner (단일 launch/reader/writer 소유자) (1149L) ✨ +│ │ ├── pi-runtime.ts ← Pi profile 정규화 + isolated `PI_CODING_AGENT_DIR` models/settings 생성 + `pi --offline --list-models` discovery + `pi --mode rpc` JSONL parser/spawner (단일 launch/reader/writer 소유자) (1152L) ✨ │ │ ├── lifecycle-handler.ts ← child lifecycle + fallback/retry + queue resume orchestration + clearEmployeeSession on resume failure + stale resume fresh retry + kickGoalContinuation export + clearGoalTimers + goal continuation boundary row (1434L) │ │ ├── kiro-auth.ts ← Kiro CLI auth store reader (resolveKiroDataPath, readKiroAuthFromStore, resolveKiroProfileArn, regionFromProfileArn, listKiroConversationIdsForCwd, resolveKiroSessionIdAfterSpawn, extractKiroSessionIdFromV2Store) (314L) │ │ ├── kiro-models.ts ← Kiro live model inventory (KiroModelEntry, KiroModelInventory, parseKiroModelListJson, fetchKiroModelInventory) (98L) @@ -237,7 +237,7 @@ cli-jaw/ │ ├── orchestrator/ ← 직원 오케스트레이션 + 인터페이스 통합 (19 files) │ │ ├── state-machine.ts ← IPABCD 상태 머신 (I=Interview pre-plan) + broadcast(state,title) + worklog 타이틀 파싱 + employee terminology + OrcContext.workingDir + OrcContext.interview + Project root dispatch contract + Phase60 actor-aware canTransition(GateInput) form-only evidence gate + STATE_PROMPTS --attest instructions (806L) │ │ ├── pipeline.ts ← IPABCD orchestration (explicit entry only) + interview first-turn detection + plan context persistence + memorySnapshot injection + reset clears boss session + OrcContext workingDir init + Approved Plan Project root guard + remote-channel elicitation guard + bounded delayed worker replay notice + Phase60 phase_attestation strip/fallback + no-state narration warn (859L) -│ │ ├── distribute.ts ← runSingleAgent + buildPlanPrompt + parallel helpers + tiered findEmployee + employee resume diagnostics + virtual employee session-skip (518L) +│ │ ├── distribute.ts ← runSingleAgent + buildPlanPrompt + parallel helpers + tiered findEmployee + employee resume diagnostics + virtual employee session-skip (520L) │ │ ├── parser.ts ← triage + subtask JSON + verdict 파싱 + isResetIntent (176L) │ │ ├── gateway.ts ← submitMessage 통합 진입점 (WebUI+CLI+TG+Discord 공통) + working_dir scoped insertMessage (381L) │ │ ├── collect.ts ← orchestrateAndCollect + orchestrateAndCollectData (206L) @@ -318,7 +318,7 @@ cli-jaw/ │ ├── memory/ ← 데이터 영속화 + advanced memory runtime (17 files) │ │ ├── advanced.ts ← Advanced Memory re-export stub (1L) │ │ ├── bootstrap.ts ← legacy memory/bootstrap import + structured root 초기화 (588L) -│ │ ├── heartbeat.ts ← Heartbeat 잡 스케줄 + cron/every timer orchestration + minute-slot dedupe + fs.watch (653L) +│ │ ├── heartbeat.ts ← Heartbeat 잡 스케줄 + cron/every timer orchestration + minute-slot dedupe + fs.watch (820L) │ │ ├── heartbeat-schedule.ts ← Heartbeat schedule normalize + cron validate/match + timezone validate + immediate cron loop helper (410L) │ │ ├── heartbeat-mention-watch.ts ← Slack mention 항목 loop + busy yield + server-owned thread send + WatchNamespace 경유 ledger 접근 (249L) │ │ ├── mention-watch-ledger.ts ← v2 ledger 단일 접근 경로 (WatchNamespace = job+workspace+user, 모든 SQL이 3파트 predicate 유지, A/B 대칭 테스트가 최종 보증) (104L) ✨ @@ -365,7 +365,7 @@ cli-jaw/ │ │ ├── enrichment-cache.ts ← 공용 동시성 프리미티브 (TTL/cap 캐시, 원인별 억제, 능력 잠금 단일 재탐침, in-flight 합류, 집계 취소, 세대 무효화) (424L) │ │ ├── conversation.ts ← 대화/스레드 컨텍스트 (conversations.info + replies cursor 최대 10페이지 + parent/최신 50, 참여자는 author 유도, method별 억제·시작률) (464L) │ │ ├── context.ts ← 프롬프트 컨텍스트 블록 조립 (채널 id·thread_ts 무절단, 섹션별 코드포인트 예산 ~9200 총 overhead, 신뢰 경계 문구 보존) (273L) -│ │ ├── history.ts ← 동적 조회 (conversations.history/replies form-encoded 래퍼 + cursor 정규화 + 재시도 + 에이전트용 포맷/redact) (373L) +│ │ ├── history.ts ← 동적 조회 (conversations.history/replies form-encoded 래퍼 + cursor 정규화 + 재시도 + 에이전트용 포맷/redact) (376L) │ │ ├── mention-watch.ts ← 가입 채널 backward mention scan + frontier/resume/round-robin/429 stop/60-channel overflow (369L) │ │ ├── attachment-recovery.ts ← app_mention 봉투에 없는 첨부를 channel+ts 재조회로 복구 (oldest+inclusive+limit=1) (53L) │ │ ├── commands.ts ← slash command → 공유 parseCommand/executeCommand 파이프라인 (185L) @@ -426,7 +426,7 @@ cli-jaw/ │ │ ├── types.ts ← `AuthMiddleware` shared type (3L) │ │ ├── static.ts ← root/uploads/widgets + guarded local image/video `/api/image` 서빙 (160L) │ │ ├── employees.ts ← employee CRUD 라우트 (123L) -│ │ ├── heartbeat.ts ← heartbeat read/write 라우트 (322L) +│ │ ├── heartbeat.ts ← heartbeat read/write 라우트 (324L) │ │ ├── skills.ts ← skill list/enable/disable/reset 라우트 (90L) │ │ ├── jaw-memory.ts ← jaw memory search/read/list/save/init/reflect/flush/soul/soul-activate/bootstrap 라우트 (362L) │ │ ├── jaw-ceo.ts ← Jaw CEO channel/session support routes (321L) ✨ @@ -434,7 +434,7 @@ cli-jaw/ │ │ ├── orchestrate.ts ← IPABCD reset/state/workers/worker-runs/snapshot/queue cancel/queue steer async accept/dispatch/virtual dispatch/batch safe summary/worker result/state PUT 라우트 + Phase60 boss-token actor distinction + --attest body gate + single-use pendingAttestation null-clear (1328L) │ │ ├── memory.ts ← memory status/KV/files/settings 라우트 (193L) │ │ ├── settings.ts ← settings/prompt/project pick/git summary/heartbeat-md/MCP/registry/status/quota/copilot + Pi profile register/model discovery 라우트 + CLI_KEYS 기반 quota parity/status-only metadata (738L) -│ │ ├── messaging.ts ← upload/file-open/voice/telegram/channel/discord send 라우트 (612L) +│ │ ├── messaging.ts ← upload/file-open/voice/telegram/channel/discord send 라우트 (629L) │ │ ├── avatar.ts ← Agent/User 아바타 이미지 업로드/서빙/삭제 + settings.json 메타 저장 + safeResolveUnder 경로 보호 (147L) │ │ ├── quota.ts ← Copilot/Claude/Codex/Grok/OpenCode quota helper readers + Grok weekly credits + credential-scoped Claude cache (634L) │ │ ├── quota-native-window.ts ← Codex duration-aware and Claude model-scoped quota parsers (122L) diff --git a/structure/telegram.md b/structure/telegram.md index e45c2e62b..16fc09cd2 100644 --- a/structure/telegram.md +++ b/structure/telegram.md @@ -48,6 +48,27 @@ still inheriting an existing one. `authorizeExplicitTarget` in `src/messaging/se vouches for a send without rewriting its address: it no longer returns the last-active target's thread for an explicitly addressed channel-root post. +Threaded Slack heartbeat jobs also prove the pair live before any main, employee or +script runner starts. `verifyHeartbeatThreadBindingLive` reads +`conversations.replies(channel, threadId, limit=1)`; the first row must be the +configured parent ts. A missing/stale parent, a channel or permission mismatch, rate +limit, missing credential, malformed response or transport failure holds that tick +before inference and before send. The live reason is retained for `GET /api/heartbeat` +and the UI while the enabled timer remains armed to recover on a later tick; editing the +destination invalidates the old reason immediately. The read is not retried and positive +results are not cached. `channel_root` and non-Slack destinations need no Slack read. Mention-watch is +separate: its destination is the hit thread it just discovered. + +Every Slack heartbeat runner reserves a server-owned tool grant for the same target. +The grant carries `enforceDestination: true` and lives for 25 minutes, longer than the +20-minute collector ceiling. `spawnAgent` activates it before any runtime branch: +print and employee children inherit the header; Cursor/Grok use a request-lifetime +native process; Codex App, Claude and Pi use a fresh acquisition; script runners receive +the same header in their explicit child environment. `POST /api/channel/send` therefore +supplies an omitted destination and rejects a different one even under Auto/full-local. +Interactive turn grants omit this flag and unrelated Auto sends are never +process-globally locked. + ### Slack group DMs and scope observations diff --git a/tests/unit/channel-send-route.test.ts b/tests/unit/channel-send-route.test.ts index 7c32a5fa0..73743a781 100644 --- a/tests/unit/channel-send-route.test.ts +++ b/tests/unit/channel-send-route.test.ts @@ -331,8 +331,10 @@ test('full-local send: all four aliases lift explicit dest/root and refuse missi tgCalls.push(chatId); return { ok: true, message_id: 1 }; }) as typeof tg.api.sendMessage; - const seen: Array<{ route?: string; channel?: string; targetId?: string; filePath?: string }> = []; - registerSendTransport('slack', async req => { seen.push({ channel: 'slack', targetId: req.target?.targetId, filePath: req.filePath }); return { ok: true }; }); + const seen: Array<{ route?: string; channel?: string; targetId?: string; threadId?: string; filePath?: string }> = []; + registerSendTransport('slack', async req => { seen.push({ + channel: 'slack', targetId: req.target?.targetId, threadId: req.target?.threadId, filePath: req.filePath, + }); return { ok: true }; }); registerSendTransport('discord', async req => { seen.push({ channel: 'discord', targetId: req.target?.targetId, filePath: req.filePath }); return { ok: true }; }); registerSendTransport('telegram', async req => { seen.push({ channel: 'telegram', targetId: req.target?.targetId, filePath: req.filePath }); return { ok: true }; }); setLastActiveTarget('slack', { channel: 'slack', targetKind: 'channel', peerKind: 'channel', targetId: 'CLAST' }); @@ -431,6 +433,47 @@ test('full-local send: all four aliases lift explicit dest/root and refuse missi }, { 'x-jaw-slack-grant': secret }); assert.equal(grantPlus.status, 200, JSON.stringify(grantPlus.body)); assert.equal(seen.at(-1)?.targetId, 'CUNLISTED'); + + const scheduledDest = { + channel: 'slack' as const, + targetKind: 'channel' as const, + peerKind: 'channel' as const, + targetId: 'CHEARTBEAT', + threadId: '2.0', + }; + assert.ok(reserveSlackToolGrant({ + teamId: 'T1', + actorId: 'U1', + destination: scheduledDest, + credentialKey: slackCredentialKey('xoxb-full-send'), + enforceDestination: true, + }, { requestId: 'heartbeat-grant', scope: 'heartbeat', chatSessionId: 'default' })); + const scheduledSecret = activateSlackToolGrant('heartbeat-grant', 'heartbeat', 'default')!; + + seen.length = 0; + const concurrentInteractive = await json('/api/channel/send', { + channel: 'slack', type: 'text', text: 'interactive', + target: { ...scheduledDest, targetId: 'COTHER' }, + }); + assert.equal(concurrentInteractive.status, 200, JSON.stringify(concurrentInteractive.body)); + assert.equal(seen.at(-1)?.targetId, 'COTHER', + 'a scheduled reservation must not process-globally lock an unrelated Auto caller'); + + seen.length = 0; + const scheduledOmit = await json('/api/channel/send', { + channel: 'slack', type: 'text', text: 'scheduled', + }, { 'x-jaw-slack-grant': scheduledSecret }); + assert.equal(scheduledOmit.status, 200, JSON.stringify(scheduledOmit.body)); + assert.equal(seen.at(-1)?.targetId, scheduledDest.targetId); + assert.equal(seen.at(-1)?.threadId, scheduledDest.threadId, + 'server-owned grant supplies its destination even under full-local Auto'); + + const scheduledWrong = await json('/api/channel/send', { + channel: 'slack', type: 'text', text: 'wrong', + target: { ...scheduledDest, targetId: 'DOTHER' }, + }, { 'x-jaw-slack-grant': scheduledSecret }); + assert.equal(scheduledWrong.status, 403); + assert.equal(scheduledWrong.body.code, 'slack_destination_mismatch'); }, { isFullAccess: () => true }); await withMessagingServer(async baseUrl => { diff --git a/tests/unit/heartbeat-destination-binding.test.ts b/tests/unit/heartbeat-destination-binding.test.ts index 5aea1c389..67a526041 100644 --- a/tests/unit/heartbeat-destination-binding.test.ts +++ b/tests/unit/heartbeat-destination-binding.test.ts @@ -5,6 +5,7 @@ import { resolveHeartbeatBinding, isCompleteHeartbeatDestination, heartbeatHoldMessage, + verifyHeartbeatThreadBindingLive, } from '../../src/memory/heartbeat-destination.ts'; // HDB — a scheduled report goes to the conversation an operator named, or it @@ -71,9 +72,105 @@ test('HDB-007 non-Slack transports keep their conversation-level contract', () = }); test('HDB-008 every hold reason explains itself without leaking anything', () => { - for (const reason of ['unbound_destination', 'incomplete_destination', 'malformed_destination'] as const) { + for (const reason of ['unbound_destination', 'incomplete_destination', 'malformed_destination', + 'thread_channel_mismatch', 'stale_thread', 'live_lookup_failed', + 'slack_grant_unavailable'] as const) { const message = heartbeatHoldMessage(reason); assert.ok(message.length > 0); assert.equal(/xox[bp]-|token|secret/i.test(message), false); } }); + +function replies(payload: Record, inspect?: (body: URLSearchParams) => void) { + return (async (_url: string | URL | Request, init?: RequestInit) => { + const body = new URLSearchParams(String(init?.body ?? '')); + inspect?.(body); + return new Response(JSON.stringify(payload), { + status: 200, + headers: { 'content-type': 'application/json' }, + }); + }) as typeof fetch; +} + +const threaded = { + channel: 'slack' as const, + targetId: 'C_REPORTS', + threadId: '1787616871.254919', +}; + +test('HDB-L01 live verification accepts only the configured parent in the configured channel', async () => { + let calls = 0; + const result = await verifyHeartbeatThreadBindingLive(threaded, { + token: 'xoxb-fixture', + fetchImpl: replies({ + ok: true, + messages: [{ ts: threaded.threadId, text: 'parent' }], + has_more: false, + }, body => { + calls++; + assert.equal(body.get('channel'), threaded.targetId); + assert.equal(body.get('ts'), threaded.threadId); + assert.equal(body.get('limit'), '1'); + }), + }); + assert.equal(result.state, 'bound'); + assert.equal(calls, 1); +}); + +test('HDB-L02 a different or empty parent fails closed as a channel-thread mismatch', async () => { + for (const messages of [[], [{ ts: '1787616871.999999', text: 'other' }]]) { + const result = await verifyHeartbeatThreadBindingLive(threaded, { + token: 'xoxb-fixture', + fetchImpl: replies({ ok: true, messages, has_more: false }), + }); + assert.deepEqual(result, { state: 'held', reason: 'thread_channel_mismatch' }); + } +}); + +test('HDB-L03 Slack error classes become stable hold reasons without retry', async () => { + const cases = [ + ['message_not_found', 'stale_thread'], + ['thread_not_found', 'stale_thread'], + ['channel_not_found', 'thread_channel_mismatch'], + ['not_in_channel', 'thread_channel_mismatch'], + ['missing_scope', 'live_lookup_failed'], + ['no_permission', 'live_lookup_failed'], + ['ratelimited', 'live_lookup_failed'], + ['invalid_auth', 'live_lookup_failed'], + ['internal_error', 'live_lookup_failed'], + ] as const; + for (const [code, reason] of cases) { + let calls = 0; + const result = await verifyHeartbeatThreadBindingLive(threaded, { + token: 'xoxb-fixture', + fetchImpl: replies({ ok: false, error: code }, () => { calls++; }), + }); + assert.deepEqual(result, { state: 'held', reason }, code); + assert.equal(calls, 1, code + ' must not retry inside a heartbeat tick'); + } +}); + +test('HDB-L04 missing credentials and transport exceptions fail this tick closed', async () => { + assert.deepEqual(await verifyHeartbeatThreadBindingLive(threaded, { token: '' }), + { state: 'held', reason: 'live_lookup_failed' }); + assert.deepEqual(await verifyHeartbeatThreadBindingLive(threaded, { + token: 'xoxb-fixture', + fetchImpl: (async () => { throw new Error('network down'); }) as typeof fetch, + }), { state: 'held', reason: 'live_lookup_failed' }); +}); + +test('HDB-L05 channel_root and non-Slack destinations require no Slack read', async () => { + let calls = 0; + const fetchImpl = (async () => { calls++; throw new Error('must not fetch'); }) as typeof fetch; + for (const destination of [ + { channel: 'slack', targetId: 'C_REPORTS', scope: 'channel_root' }, + { channel: 'telegram', targetId: '123' }, + { channel: 'discord', targetId: '456' }, + ]) { + assert.equal((await verifyHeartbeatThreadBindingLive(destination, { + token: '', + fetchImpl, + })).state, 'bound'); + } + assert.equal(calls, 0); +}); diff --git a/tests/unit/heartbeat-grant-runtime-wiring.test.ts b/tests/unit/heartbeat-grant-runtime-wiring.test.ts new file mode 100644 index 000000000..88ae1a1b6 --- /dev/null +++ b/tests/unit/heartbeat-grant-runtime-wiring.test.ts @@ -0,0 +1,52 @@ +import test from 'node:test'; +import assert from 'node:assert/strict'; +import fs from 'node:fs'; +import { join } from 'node:path'; + +const root = join(import.meta.dirname, '../..'); +const spawn = fs.readFileSync(join(root, 'src/agent/spawn.ts'), 'utf8'); +const pi = fs.readFileSync(join(root, 'src/agent/pi-runtime.ts'), 'utf8'); +const heartbeat = fs.readFileSync(join(root, 'src/memory/heartbeat.ts'), 'utf8'); +const distribute = fs.readFileSync(join(root, 'src/orchestrator/distribute.ts'), 'utf8'); + +test('HGR-001 grant activation precedes every runtime branch', () => { + const activation = spawn.indexOf('const slackToolGrant = slackToolGrantEligible'); + for (const marker of [ + "if (runtimeTransport === 'native' && cli === 'claude')", + "if (cli === 'pi')", + "if (cli === 'codex-app')", + 'const { child, kiroConversationIdsBefore', + ]) { + const branch = spawn.indexOf(marker); + assert.ok(activation >= 0 && branch > activation, marker); + } +}); + +test('HGR-002 native pools cannot reuse a process carrying a scheduled grant', () => { + assert.ok(spawn.includes("...(slackToolGrant ? { lifetime: 'request' as const } : {})")); + assert.ok((spawn.match(/forceNew: forceNew || Boolean(slackToolGrant)/g) ?? []).length >= 4, + 'Cursor/Grok, Pi, and both Codex App acquisition paths force fresh ownership'); + assert.match(spawn, /fresh: forceNew || opts._skipResume === true || isEmployee || Boolean(slackToolGrant)/, + 'Claude native receives a fresh grant-bearing acquisition'); +}); + +test('HGR-003 Pi launches from the captured env instead of process-global env', () => { + assert.ok(pi.includes('const inherited = { ...(options.env ?? process.env) }')); + assert.ok(spawn.includes('piSettings: pi,\n env: spawnEnv,')); + assert.ok((spawn.match(/env: spawnEnv/g) ?? []).length >= 2, + 'both employee spawnPiRpc and main acquirePiRuntime receive the captured env'); +}); + +test('HGR-004 employee and script runners receive the same request grant', () => { + assert.ok(heartbeat.includes('runEmployee(job, prompt, requestId, destinationBinding.target)')); + assert.ok(heartbeat.includes('grantEnv = { [SLACK_TOOL_GRANT_ENV]: secret }')); + assert.ok(distribute.includes('...(isRemoteTarget(meta["target"]) ? { target: { ...meta["target"] } } : {})')); +}); + +test('HGR-005 enforced grant lifetime exceeds every runner ceiling', () => { + const toolContext = fs.readFileSync(join(root, 'src/slack/tool-context.ts'), 'utf8'); + assert.ok(toolContext.includes('ENFORCED_GRANT_TTL_MS = 25 * 60_000')); + assert.ok(heartbeat.includes('timeout: 10 * 60_000')); + const collect = fs.readFileSync(join(root, 'src/orchestrator/collect.ts'), 'utf8'); + assert.ok(collect.includes('IDLE_TIMEOUT = 1200000')); +}); diff --git a/tests/unit/heartbeat-mention-watch-fresh-start.test.ts b/tests/unit/heartbeat-mention-watch-fresh-start.test.ts index 236515af9..9fcb68bb6 100644 --- a/tests/unit/heartbeat-mention-watch-fresh-start.test.ts +++ b/tests/unit/heartbeat-mention-watch-fresh-start.test.ts @@ -12,7 +12,12 @@ import { registerHeartbeatRoutes } from '../../src/routes/heartbeat.ts'; import { loadHeartbeatFile, saveHeartbeatFile, settings } from '../../src/core/config.ts'; import { legacyMentionWatchV1Fixture, commitLegacyFreshStart } from '../../src/core/db.ts'; import { detectLegacyMentionWatch, isQuarantined, quarantineState } from '../../src/memory/legacy-mention-watch-quarantine.ts'; -import { startHeartbeat, stopHeartbeat, getHeartbeatRuntimeState } from '../../src/memory/heartbeat.ts'; +import { + startHeartbeat, + stopHeartbeat, + getHeartbeatRuntimeState, + updateHeartbeatLiveDestinationHold, +} from '../../src/memory/heartbeat.ts'; import { resetVerifiedSlackWorkspace } from '../../src/slack/verified-workspace.ts'; /** Present in the v1 table, which is what a losing claim must not have erased. */ @@ -438,6 +443,30 @@ test('a job with no hold carries no held marker', async () => { }); }); +test('GET surfaces a live thread hold while leaving the recovery timer intent enabled', async () => { + const job = { + id: 'live_thread_hold', + name: 'live_thread_hold', + enabled: true, + schedule: { kind: 'every' as const, minutes: 10 }, + prompt: 'x', + destination: { channel: 'slack' as const, targetId: 'C_REPORTS', threadId: '1787616871.254919' }, + }; + saveHeartbeatFile({ jobs: [job] }); + updateHeartbeatLiveDestinationHold(job, 'stale_thread'); + try { + await withHeartbeatServer(async baseUrl => { + const response = await fetch(baseUrl + '/api/heartbeat'); + const body = await response.json() as { jobs?: Array<{ id?: string; enabled?: boolean; held?: string }> }; + const listed = body.jobs?.find(candidate => candidate.id === job.id); + assert.equal(listed?.enabled, true, 'file intent stays enabled so the next tick can recover'); + assert.equal(listed?.held, 'stale_thread', 'operator UI sees why this tick cannot run'); + }); + } finally { + updateHeartbeatLiveDestinationHold(job, null); + } +}); + test('clearing the hold clears the marker', async () => { const jobId = 'hold_marker_cleared'; seedHeldJob(jobId); diff --git a/tests/unit/heartbeat-runner-modes.test.ts b/tests/unit/heartbeat-runner-modes.test.ts index c3f30a8ab..58cd8faf5 100644 --- a/tests/unit/heartbeat-runner-modes.test.ts +++ b/tests/unit/heartbeat-runner-modes.test.ts @@ -1,5 +1,8 @@ import { test, mock } from 'node:test'; import assert from 'node:assert/strict'; +import { settings } from '../../src/core/config.ts'; +import { hasActiveEnforcedSlackDestination } from '../../src/slack/tool-context.ts'; +import { resetVerifiedSlackWorkspace } from '../../src/slack/verified-workspace.ts'; const collectUrl = new URL('../../src/orchestrator/collect.ts', import.meta.url).href; const sendUrl = new URL('../../src/messaging/send.ts', import.meta.url).href; @@ -17,13 +20,16 @@ const [realSend, realDb, realState, realSpawn, realRegistry, realDistribute] = a let collectCalls = 0; let plannerOnly = false; let employeeBusy = false; +let collectObserver = () => {}; const sent: string[] = []; const sentRequests: Array> = []; const anchors: unknown[][] = []; +let employeeRunMeta: Record | undefined; const employee = { id: 'emp-1', name: 'reviewer', cli: 'codex', model: null, role: 'reviewer' }; mock.module(collectUrl, { namedExports: { orchestrateAndCollectData: async () => { + collectObserver(); collectCalls++; return { text: 'status: ok\nsummary: main complete', data: { agyPlannerOnly: plannerOnly } }; }, @@ -54,9 +60,29 @@ mock.module(registryUrl, { namedExports: { failWorker: () => undefined, hasPendingWorkerReplays: () => false, } }); -mock.module(distributeUrl, { namedExports: { ...realDistribute, runSingleAgent: async () => ({ text: 'status: ok\nsummary: employee complete', tools: [] }) } }); +mock.module(distributeUrl, { namedExports: { ...realDistribute, + runSingleAgent: async (...args: unknown[]) => { + employeeRunMeta = args[4] as Record | undefined; + return { text: 'status: ok\nsummary: employee complete', tools: [] }; + }, +} }); -const { decideHeartbeatReport, runHeartbeatJob, runHeartbeatScript } = await import('../../src/memory/heartbeat.js'); +const { + decideHeartbeatReport, + getHeartbeatLiveDestinationHold, + runHeartbeatJob, + runHeartbeatScript, +} = await import('../../src/memory/heartbeat.js'); +const { resolveHeartbeatBinding } = await import('../../src/memory/heartbeat-destination.js'); +const defaultDestination = { channel: 'slack' as const, targetId: 'C_REPORTS', scope: 'channel_root' as const }; +function runJob(job: Record) { + return runHeartbeatJob({ destination: defaultDestination, ...job }, { + // Runner tests own orchestration/report policy. Live Slack membership is + // exercised directly in heartbeat-destination-binding.test.ts. + verifyDestination: async destination => resolveHeartbeatBinding(destination), + reserveDestinationGrant: async () => () => {}, + }); +} type Status = 'ok' | 'warning' | 'failed'; const report = (status: Status, userVisible = false) => ({ status, changed: false, recordRequired: false, userVisible, summary: 's', evidence: '', nextAction: '', raw: 's' }); @@ -76,19 +102,74 @@ test('anomaly_only sends an ok report explicitly marked user-visible', () => { test('planner-only main heartbeat retries exactly once even when every result is planner-only', async () => { collectCalls = 0; plannerOnly = true; - await runHeartbeatJob({ id: 'retry', name: 'retry', enabled: true, schedule: { minutes: 5 }, prompt: 'check' }); + await runJob({ id: 'retry', name: 'retry', enabled: true, schedule: { minutes: 5 }, prompt: 'check' }); assert.equal(collectCalls, 2); + plannerOnly = false; +}); + +test('production default verifies, reserves, activates the guard during collection, and releases it', async t => { + const previousSlack = settings.slack; + collectCalls = 0; + sentRequests.length = 0; + resetVerifiedSlackWorkspace(); + settings.slack = { ...previousSlack, enabled: true, botToken: 'xoxb-fixture' }; + const threadId = '1787616871.254919'; + let replyCalls = 0; + let authCalls = 0; + t.mock.method(globalThis, 'fetch', async (input: string | URL | Request) => { + const url = String(input); + if (url.includes('/api/conversations.replies')) { + replyCalls++; + return new Response(JSON.stringify({ + ok: true, + messages: [{ ts: threadId, text: 'parent' }], + has_more: false, + }), { headers: { 'content-type': 'application/json' } }); + } + if (url.includes('/api/auth.test')) { + authCalls++; + return new Response(JSON.stringify({ + ok: true, + team_id: 'T1FIXTURE', + user_id: 'U1FIXTURE', + }), { headers: { 'content-type': 'application/json' } }); + } + throw new Error('unexpected Slack method: ' + url); + }); + collectObserver = () => { + assert.equal(hasActiveEnforcedSlackDestination(), true, + 'native/headerless agent calls are guarded for the whole collection'); + }; + try { + await runHeartbeatJob({ + id: 'default-path', + name: 'default-path', + enabled: true, + schedule: { minutes: 5 }, + prompt: 'check', + destination: { channel: 'slack', targetId: 'C1REPORTS', threadId }, + }); + } finally { + collectObserver = () => {}; + settings.slack = previousSlack; + resetVerifiedSlackWorkspace(); + } + assert.equal(replyCalls, 1); + assert.equal(authCalls, 1); + assert.equal(collectCalls, 1); + assert.equal(hasActiveEnforcedSlackDestination(), false, 'grant is released after collection'); + assert.equal(sentRequests[0]?.['target']?.threadId, threadId); }); test('non-planner main heartbeat runs once', async () => { collectCalls = 0; plannerOnly = false; - await runHeartbeatJob({ id: 'once', name: 'once', enabled: true, schedule: { minutes: 5 }, prompt: 'check' }); + await runJob({ id: 'once', name: 'once', enabled: true, schedule: { minutes: 5 }, prompt: 'check' }); assert.equal(collectCalls, 1); }); test('busy employee produces warning delivery without running employee', async () => { employeeBusy = true; sent.length = 0; - await runHeartbeatJob({ id: 'busy', name: 'busy', runner: 'employee', employee: employee.name, + await runJob({ id: 'busy', name: 'busy', runner: 'employee', employee: employee.name, reportPolicy: 'anomaly_only', schedule: { minutes: 5 }, prompt: 'check', destination: { channel: 'slack', targetId: 'C_REPORTS', threadId: '1787616871.254919' } }); employeeBusy = false; @@ -111,7 +192,8 @@ test('script runner maps a real nonzero exit to failed', async () => { // Timeout configuration remains a source-contract assertion: waiting ten real minutes is not an acceptable unit test. test('script runner configures the audited timeout and output bound', async () => { const source = await import('node:fs').then(fs => fs.readFileSync(new URL('../../src/memory/heartbeat.ts', import.meta.url), 'utf8')); - assert.match(source, /timeout: 10 \* 60_000, maxBuffer: 64 \* 1024/); + assert.ok(source.includes('timeout: 10 * 60_000')); + assert.ok(source.includes('maxBuffer: 64 * 1024')); }); // ─── destination routing (#437) ───────────────────── @@ -123,7 +205,7 @@ test('script runner configures the audited timeout and output bound', async () = test('a job with a destination sends there and forbids the active fallback', async () => { sent.length = 0; sentRequests.length = 0; - await runHeartbeatJob({ + await runJob({ id: 'pinned', name: 'pinned', enabled: true, schedule: { minutes: 5 }, prompt: 'check', destination: { channel: 'slack', targetId: 'C_REPORTS', threadId: '1787616871.254919' }, }); @@ -139,7 +221,7 @@ test('a job with a destination sends there and forbids the active fallback', asy test('a destination that opts into the conversation root posts there', async () => { sent.length = 0; sentRequests.length = 0; - await runHeartbeatJob({ + await runJob({ id: 'root', name: 'root', enabled: true, schedule: { minutes: 5 }, prompt: 'check', destination: { channel: 'slack', targetId: 'C_REPORTS', scope: 'channel_root' }, }); @@ -153,7 +235,7 @@ test('a Slack destination with no thread and no root opt-in is held', async () = // Guessing the root put scheduled reports at the bottom of channels their // operator had pointed at a specific thread (#745). sent.length = 0; sentRequests.length = 0; - await runHeartbeatJob({ + await runJob({ id: 'incomplete', name: 'incomplete', enabled: true, schedule: { minutes: 5 }, prompt: 'check', destination: { channel: 'slack', targetId: 'C_REPORTS' }, }); @@ -163,7 +245,7 @@ test('a Slack destination with no thread and no root opt-in is held', async () = test('an empty-string thread is not a thread', async () => { sent.length = 0; sentRequests.length = 0; - await runHeartbeatJob({ + await runJob({ id: 'blank-thread', name: 'blank-thread', enabled: true, schedule: { minutes: 5 }, prompt: 'check', destination: { channel: 'slack', targetId: 'C_REPORTS', threadId: '' }, }); @@ -173,7 +255,7 @@ test('an empty-string thread is not a thread', async () => { test('the derived target carries the kinds the operator never types', async () => { sent.length = 0; sentRequests.length = 0; - await runHeartbeatJob({ + await runJob({ id: 'kinds', name: 'kinds', enabled: true, schedule: { minutes: 5 }, prompt: 'check', destination: { channel: 'slack', targetId: 'C_REPORTS', scope: 'channel_root' }, }); @@ -195,12 +277,115 @@ test('a job without a destination delivers nowhere', async () => { assert.equal(sentRequests.length, 0); }); +test('a live thread mismatch stops before model work and before send', async () => { + collectCalls = 0; sent.length = 0; sentRequests.length = 0; + await runHeartbeatJob({ + id: 'mismatch', name: 'mismatch', enabled: true, schedule: { minutes: 5 }, prompt: 'check', + destination: { channel: 'slack', targetId: 'C_REPORTS', threadId: '1787616871.254919' }, + }, { + verifyDestination: async () => ({ state: 'held', reason: 'thread_channel_mismatch' }), + reserveDestinationGrant: async () => () => {}, + }); + assert.equal(collectCalls, 0, 'an unverified destination spends no model turn'); + assert.equal(sentRequests.length, 0, 'an unverified destination sends nowhere'); + assert.equal(getHeartbeatLiveDestinationHold({ + id: 'mismatch', + destination: { channel: 'slack', targetId: 'C_REPORTS', threadId: '1787616871.254919' }, + }), 'thread_channel_mismatch', 'GET/UI can surface the live hold'); + assert.equal(getHeartbeatLiveDestinationHold({ + id: 'mismatch', + destination: { channel: 'slack', targetId: 'C_REPORTS', threadId: '1787616871.999999' }, + }), null, 'editing the destination invalidates the old live hold immediately'); +}); + +test('a Slack main job reserves and releases destination-bound tool authority around its turn', async () => { + collectCalls = 0; sentRequests.length = 0; + let reserved = 0; + let released = 0; + const destination = { channel: 'slack' as const, targetId: 'C_REPORTS', threadId: '1787616871.254919' }; + const binding = resolveHeartbeatBinding(destination); + assert.equal(binding.state, 'bound'); + await runHeartbeatJob({ + id: 'grant', name: 'grant', enabled: true, schedule: { minutes: 5 }, prompt: 'check', + destination, + }, { + verifyDestination: async () => binding, + reserveDestinationGrant: async (actual, requestId) => { + reserved++; + assert.equal(actual.state, 'bound'); + assert.equal(actual.target.targetId, destination.targetId); + assert.ok(requestId); + return () => { released++; }; + }, + }); + assert.equal(collectCalls, 1); + assert.equal(reserved, 1); + assert.equal(released, 1); + assert.equal(sentRequests[0]?.['target']?.threadId, destination.threadId); +}); + +test('planner-only retry receives a fresh destination grant and releases both', async () => { + collectCalls = 0; plannerOnly = true; + const requestIds = new Set(); + let released = 0; + await runHeartbeatJob({ + id: 'grant-retry', name: 'grant-retry', enabled: true, schedule: { minutes: 5 }, prompt: 'check', + destination: { channel: 'slack', targetId: 'C_REPORTS', scope: 'channel_root' }, + }, { + verifyDestination: async destination => resolveHeartbeatBinding(destination), + reserveDestinationGrant: async (_binding, requestId) => { + requestIds.add(requestId); + return () => { released++; }; + }, + }); + plannerOnly = false; + assert.equal(collectCalls, 2); + assert.equal(requestIds.size, 2); + assert.equal(released, 2); +}); + +for (const runner of ['employee', 'script'] as const) { + test(`${runner} runner is covered by the same destination guard`, async () => { + employeeRunMeta = undefined; + let reserved = 0; + let released = 0; + await runHeartbeatJob({ + id: 'guard-' + runner, + name: 'guard-' + runner, + enabled: true, + runner, + ...(runner === 'employee' ? { employee: employee.name } : { + command: [process.execPath, '-e', + "if(!process.env.JAW_SLACK_TURN_GRANT)process.exit(2);console.log('status: ok\\nchanged: no\\nsummary: script complete')"], + }), + schedule: { minutes: 5 }, + prompt: 'check', + destination: { channel: 'slack', targetId: 'C_REPORTS', scope: 'channel_root' }, + }, { + verifyDestination: async destination => resolveHeartbeatBinding(destination), + reserveDestinationGrant: async () => { + reserved++; + return () => { released++; }; + }, + ...(runner === 'script' ? { activateDestinationGrant: () => 'fixture-grant' } : {}), + }); + assert.equal(reserved, 1); + assert.equal(released, 1); + if (runner === 'employee') { + assert.equal(employeeRunMeta?.['origin'], 'heartbeat'); + assert.equal(employeeRunMeta?.['scopeKey'], 'default'); + assert.ok(employeeRunMeta?.['requestId']); + assert.equal((employeeRunMeta?.['target'] as { targetId?: string } | undefined)?.targetId, 'C_REPORTS'); + } + }); +} + test('a malformed destination is refused, not redirected to the active channel', async () => { // A job that named a destination has stated an intent. When that intent // cannot be resolved, delivering to whoever spoke last is the original bug // wearing a different hat — the report still lands in an unrelated place. sent.length = 0; sentRequests.length = 0; - await runHeartbeatJob({ + await runJob({ id: 'bad', name: 'bad', enabled: true, schedule: { minutes: 5 }, prompt: 'check', destination: { channel: 'slack' }, }); @@ -210,7 +395,7 @@ test('a malformed destination is refused, not redirected to the active channel', test('a destination naming an unknown transport is refused too', async () => { sent.length = 0; sentRequests.length = 0; - await runHeartbeatJob({ + await runJob({ id: 'bad-channel', name: 'bad-channel', enabled: true, schedule: { minutes: 5 }, prompt: 'check', destination: { channel: 'irc', targetId: 'C_X' }, }); @@ -220,19 +405,19 @@ test('a destination naming an unknown transport is refused too', async () => { test('a scheduled run survives a malformed destination without throwing', async () => { // Refusing to deliver must not take the heartbeat loop down with it. - await runHeartbeatJob({ + await runJob({ id: 'bad-survives', name: 'bad-survives', enabled: true, schedule: { minutes: 5 }, prompt: 'check', destination: { targetId: 'C_X' }, }); sent.length = 0; sentRequests.length = 0; - await runHeartbeatJob({ id: 'after', name: 'after', enabled: true, schedule: { minutes: 5 }, prompt: 'check', + await runJob({ id: 'after', name: 'after', enabled: true, schedule: { minutes: 5 }, prompt: 'check', destination: { channel: 'slack', targetId: 'C_REPORTS', threadId: '1787616871.254919' } }); assert.equal(sentRequests.length, 1, 'the next job still runs'); }); test('the anchor records where the report actually went', async () => { anchors.length = 0; - await runHeartbeatJob({ + await runJob({ id: 'anchored', name: 'anchored', enabled: true, schedule: { minutes: 5 }, prompt: 'check', destination: { channel: 'slack', targetId: 'C_REPORTS', scope: 'channel_root' }, }); @@ -252,7 +437,7 @@ test('an employee heartbeat consumes its own worker replay', async () => { // arrive. const { hasPendingWorkerReplays } = await import('../../src/orchestrator/worker-registry.js'); - await runHeartbeatJob({ + await runJob({ id: 'emp', name: 'emp', runner: 'employee', employee: employee.name, enabled: true, schedule: { minutes: 5 }, prompt: 'check', }); diff --git a/tests/unit/native-grok-spawn.test.ts b/tests/unit/native-grok-spawn.test.ts index 9c15709c7..9cff86193 100644 --- a/tests/unit/native-grok-spawn.test.ts +++ b/tests/unit/native-grok-spawn.test.ts @@ -22,7 +22,7 @@ const nativeId='private-grok-native', setup={sessionId:nativeId,models:{currentM let promptId, secret='', badCancel=false; const update=u=>send({jsonrpc:'2.0',method:'session/update',params:{sessionId:nativeId,update:u}}); const text=(value,id)=>update({sessionUpdate:'agent_message_chunk',messageId:id,content:{type:'text',text:value}}); -log({kind:'spawn',pid:process.pid,argv:process.argv.slice(2)}); +log({kind:'spawn',pid:process.pid,argv:process.argv.slice(2),grant:Boolean(process.env.JAW_SLACK_TURN_GRANT)}); for await(const line of readline.createInterface({input:process.stdin})) { const r=JSON.parse(line),reply=result=>send({jsonrpc:'2.0',id:r.id,result});log(r); if(r.method==='initialize') reply({protocolVersion:1,agentCapabilities:{loadSession:true},authMethods:[{id:'cached_token'}]}); @@ -80,6 +80,7 @@ const { bumpScopeSessionGeneration } = await import('../../src/agent/session-per const { AcpRuntimeSession } = await import('../../src/agent/runtime/acp/runtime-session.ts'); const { isNativeAdapterImplemented, isNativeWorkerImplemented } = await import('../../src/agent/runtime/selection.ts'); const { admitRequest, pendingRequestIds, settleAllPending } = await import('../../src/orchestrator/request-registry.ts'); +const { reserveSlackToolGrant, slackCredentialKey } = await import('../../src/slack/tool-context.ts'); const { createChatSession, setActiveChatSession } = await import('../../src/core/chat-sessions.ts'); let serial = 0; test.beforeEach(t => { @@ -167,6 +168,33 @@ test('actual Grok main emits final-only output, canonical usage and reuses nativ } finally { off(); } }); +test('heartbeat native Grok launches request-lifetime with its destination grant in env', { timeout: 5000 }, async () => { + const opts = options(); + const target: RemoteTarget = { + channel: 'slack', + targetKind: 'channel', + peerKind: 'channel', + targetId: 'C1HEART', + threadId: '1.001', + }; + Object.assign(opts, { origin: 'heartbeat', target }); + assert.equal(reserveSlackToolGrant({ + teamId: 'T1HEART', + actorId: 'U1HEART', + destination: target, + credentialKey: slackCredentialKey('xoxb-fixture'), + enforceDestination: true, + }, { + requestId: opts.requestId, + scope: opts.scopeKey, + chatSessionId: opts.chatSessionId, + }), true); + const result = await spawnAgent('heartbeat grant fixture', opts).promise; + assert.equal(result.code, 0); + assert.equal(wire().find(item => item.kind === 'spawn')?.grant, true); + assert.equal(poolStats().busy, 0, 'request-lifetime lease retires instead of keeping the grant-bearing process resident'); +}); + test('concurrent C queues as busy while the first replacement B remains dispatched', { timeout: 5000 }, async () => { const opts = options(), run = await held(opts), { events, off } = capture(); const holdId = 'fixture-hold-' + opts.scopeKey; diff --git a/tests/unit/slack-tool-access.test.ts b/tests/unit/slack-tool-access.test.ts index bfb858356..7eb99c5a9 100644 --- a/tests/unit/slack-tool-access.test.ts +++ b/tests/unit/slack-tool-access.test.ts @@ -38,6 +38,24 @@ test('grant expiry aborts retained handles and prevents reuse', t => { assert.equal(resolveSlackToolGrant(secret), null); assert.equal(value.signal.aborted, true); }); +test('enforced scheduled grant outlives the 20-minute collector ceiling', t => { + t.mock.timers.enable({ apis: ['Date', 'setTimeout'], now: 1000 }); + assert.ok(reserveSlackToolGrant({ + teamId: 'T1', + actorId: 'U1', + destination, + credentialKey: slackCredentialKey(token), + enforceDestination: true, + }, { requestId: 'scheduled', scope: 'default', chatSessionId: 'chat' })); + const secret = activateSlackToolGrant('scheduled', 'default', 'chat')!; + const value = resolveSlackToolGrant(secret)!; + t.mock.timers.tick(20 * 60_000); + assert.equal(resolveSlackToolGrant(secret), value, 'collector can still use the grant at its maximum idle time'); + t.mock.timers.tick(5 * 60_000); + assert.equal(resolveSlackToolGrant(secret), null); + assert.equal(value.signal.aborted, true); +}); + test('operator token is separate, private and stable across initialization', t => { const home = mkdtempSync(join(tmpdir(), 'slack-operator-')); t.after(() => rmSync(home, { recursive: true, force: true })); const validate = initializeSlackOperatorAuth(home);