Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 3 additions & 2 deletions .memory/quick-view-cards.md
Original file line number Diff line number Diff line change
Expand Up @@ -27,8 +27,9 @@ Quick View is now Claude-Code-style stacked cards (owner decisions 2026-10-15; p
- Categories (ids): `system-prompt`, `builtin-tools`, `mcp-tools`, `skills`, `memory`, `project-instructions`, `mcp-instructions`, `bot-instructions`; zero-token ones are omitted. They use the same chars/4 rule as `estimateStaticContextTokens`, apportioned by largest remainder so they sum exactly to `staticTokens` (leftover tokens go to the biggest fractional remainders, ties to System prompt first).
- Measured: built-in vs MCP tools by tool size (`isMcpTool` = codemode tool with discovery); project instructions = the AGENTS.md block measured from the final system prompt (`withoutAgentsInstructions`); MCP server instructions = the admitted `<mcp_servers>` transcript section measured from the same replayed prompt (`withoutMcpServersSection`, after the AGENTS.md block is stripped), so its initial patch, refreshes and removal are reflected automatically and it is never also recorded; skills, memory and bot instructions = text recorded where the prompt is assembled (`promptSections`), scaled down if recorded longer than the prompt sent; system prompt = what is left, never negative. An idle (ambient) chat's estimated prompt never carries the `mcp_servers` section, so it has no MCP server instructions row (never a row of 0).
- Deferred MCP tools (held behind tool search) are listed per server (`discovery.label`, grouped by `discovery.namespace`) with tool count and tokens, shown in muted text with "—" for % and never counted toward the window. Tools already declared are not double-counted.
- Detail UI (`renderer/components/context-breakdown.tsx`): stacked bar over the whole `contextWindow`, categories largest first, then Free and Reserved for response (only Reserved is hatched), and a marker at the `inputBudgetTokens` boundary; legend with tokens and % of window (legend order is Reserved then Free, while the bar runs Free then Reserved); a folded "MCP tools by server" list; a note on compaction state, `compressibleHistoryMessages`, and that deferred tools load through tool search. Colors use `--chart-1..8` only: Messages 1, Built-in tools 2, System prompt 3, MCP tools 4, Skills 5, Memory 6, Project instructions 7 (shared with Bot instructions), MCP server instructions 8.
- When provider usage anchors the projection, `contextTokens` (max of local estimate and provider-anchored usage) can exceed `messageTokens + staticTokens`; the excess is its own used segment, "Provider-counted, not itemized" (faded `--chart-1`), in bar, legend and image label (omitted at 0, also on the fallback path), so used + Free + Reserved spans the whole `contextWindow` and Messages stays the local estimate.
- Detail UI (`renderer/components/context-breakdown.tsx`, tiles in `context-tiles.tsx`, redesigned 2026-10-19): 100 tiles over the whole `contextWindow` (each 1%), used tiles per category by largest remainder (any used context or reserve gets at least one tile), then Free, then the hatched reserve; a row per category with tokens, % of window and a bar relative to the largest category; quiet Reserved, Free and deferred ("not counted") rows; a folded "MCP tools by server" list; a soft status pill on compaction state, `compressibleHistoryMessages`, and deferred tools. `layout="panel"` puts a square grid beside the bars in the Context tool. The composer meter and the Quick View summary use 10 pips (`ContextPips`); the popover warns from the 80% approaching phase on. Colors use `--chart-1..8` only: Messages 1, Built-in tools 2, System prompt 3, MCP tools 4, Skills 5, Memory 6, Project instructions 7 (shared with Bot instructions), MCP server instructions 8.
- Prompt cache timer (2026-10-19): `ChatContextPressureV1.promptCache` ({ttlMs, lastRequestAt}) drives a countdown under the breakdown in Quick View and the Context tool. TTL comes from `promptCacheTtlMs` in `main/services/prompt-cache-lifetimes.ts`: the catalog's `promptCache.short` first, then a sourced table for providers that publish a fixed idle lifetime refreshed on each read (Claude via Anthropic markers or Bedrock 5 min, Bedrock Nova 5 min, OpenAI/Azure GPT-5.6+ 30 min, Kimi K3 and MiniMax M2 explicit caching 5 min, Copilot Claude 1 h / OpenAI 24 h). Best-effort ranges (older OpenAI, DeepSeek, Fireworks, Groq) and undocumented caches (Gemini implicit, xAI, Z.ai, Mistral, Codex) get no timer. Ambient `lastRequestAt` is the newest same-model cached response; live is the request's preparation time, and successful cache-warming refreshes re-send it (`onCacheRefreshed`) and are remembered per chat (`rememberPromptCacheRefresh`) so ambient reads after Stop keep the later expiry. Cache warming itself still uses only the catalog TTL.
- When provider usage anchors the projection, `contextTokens` (max of local estimate and provider-anchored usage) can exceed `messageTokens + staticTokens`; the excess is its own used segment, "Provider-counted, not itemized" (faded `--chart-1`), in tiles, legend and image label (omitted at 0, also on the fallback path), so used + Free + Reserved spans the whole `contextWindow` and Messages stays the local estimate.
- Subagent detail (`SubagentStatusGroups` in `subagent-status-groups.tsx`, grouping by `groupSubagentRunsByStatus` in `renderer/lib/subagent-panel-state.ts`): every run grouped Running, Needs attention, Failed, Stopped, Completed; Completed starts folded; Stop on hover or focus for foreground v2 runs that are still active (disabled "Stopping…" while pending, inline error until the next attempt); a row opens that run. Everything derives from `panel.subagentViews[].snapshot`.

## Left out on purpose
Expand Down
2 changes: 1 addition & 1 deletion docs/chatgpt-desktop-ui-inspiration.md
Original file line number Diff line number Diff line change
Expand Up @@ -174,7 +174,7 @@ labels recorded usage. Appearance can hide the meter without hiding the tool.
| Card layout | Each section is its own borderless raised card with a title, a chevron after the label, a trailing summary while collapsed, and a header button left of a right-edge grip. There is no `+` menu: cards start collapsed and appear only when relevant. | The grip shows on hover and keyboard focus. Drag it, or press ArrowUp or ArrowDown on it, to reorder; a polite status announces the new position. Order and the cards the user opened persist on this device. A card that appears mid-session fades and scales in; Reduce Motion removes the motion. |
| Folder | The first Environment row shows the workspace folder and its parent, with a preferred-editor action when one is installed. Its card header offers an open-external button that reveals the folder in Finder (Files off macOS). | The row opens Files on this workspace. Open failures surface as a toast. The row is disabled without file access and absent when the workspace has no folder. |
| Pull request | A Changes row names the chat's pull request number and state (open, draft, merged, closed). It appears only once a pull request exists for this chat, and there is no Create PR button. | Activating it opens the pull request on GitHub. A merged pull request without a title is named by its branch. |
| Context | A card that appears after the first message. Collapsed, it shows a mini meter and the same percentage as the composer meter. Expanded, a stacked bar over the whole context window splits messages and each static category, with Free and Reserved for response, plus a legend with tokens and % of window. | Deferred MCP tools are listed in muted text with `—` for %, since they are not counted until used, and a fold lists them per server. A note covers compaction. Without a breakdown it falls back to Messages and System prompt & tools. The header button opens the Context tool in the sidebar. |
| Context | A card that appears after the first message. Collapsed, it shows the percentage beside a 10-pip meter, the same one the composer uses. Expanded, 100 tiles lay out the whole context window, each tile 1% of it: used tiles colored by category, then free space, then the hatched response reserve. Below, each category has its own row with tokens and a bar relative to the largest one, followed by quieter Reserved, Free and deferred rows. | Deferred MCP tools are marked "not counted", since they only count once used, and a fold lists them per server. A soft status pill covers compaction. For models whose provider documents a fixed prompt cache lifetime (Claude, OpenAI GPT-5.6 and later, Kimi K3, MiniMax M2, Copilot), a countdown shows how long the cache stays warm after the last request; providers that only give a range show none. Without a breakdown it falls back to Messages and System prompt & tools. The header button opens the Context tool in the sidebar, which shows the same tiles as a square beside the bars, next to Session and Recorded usage cards. |
| Subagents | A card that appears once the chat has started a subagent and stays for the session. Collapsed, it summarizes active and finished runs. Expanded, runs are grouped Running, Needs attention, Failed, Stopped and Completed. | Completed starts folded behind one disclosure row. Active foreground runs reveal Stop on hover or focus. A row opens that run, and the header button opens the Subagents tool in the sidebar. |

### Review panel states
Expand Down
14 changes: 13 additions & 1 deletion main/services/context-pressure.ts
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,8 @@ import { gitInfo } from "./git.js";
import {
modelRetainsSystemUpdates,
projectChatContextPressure,
promptCacheWindow,
type PromptCacheRefresh,
type GenerationContextOptions,
} from "./generation-context.js";
import { resolveCompactionModelMetadata } from "./model-runtime.js";
Expand Down Expand Up @@ -53,8 +55,16 @@ export function rememberChatContextProfile(
generationProfiles.set(chatId, createGenerationContextProfile(options, scope));
}

const promptCacheRefreshes = new Map<string, PromptCacheRefresh>();

/** Cache warming refreshed this chat's cache; ambient reads keep the later expiry. */
export function rememberPromptCacheRefresh(chatId: string, refresh: PromptCacheRefresh): void {
promptCacheRefreshes.set(chatId, refresh);
}

export function forgetChatContextProfile(chatId: string): void {
generationProfiles.delete(chatId);
promptCacheRefreshes.delete(chatId);
ambientProfiles.delete(chatId);
journalSnapshots.delete(chatId);
}
Expand Down Expand Up @@ -292,5 +302,7 @@ export async function chatContextPressure(
),
]
: messages;
return projectChatContextPressure(projected, options);
const pressure = projectChatContextPressure(projected, options);
const promptCache = promptCacheWindow(model, messages, promptCacheRefreshes.get(chatId));
return promptCache ? { ...pressure, promptCache } : pressure;
}
23 changes: 23 additions & 0 deletions main/services/generation-context.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@ import type { AgentMessage, AgentTool } from "@earendil-works/pi-agent-core";
import {
assertGenerationContextCapacity,
chatContextPressureFromProjection,
promptCacheWindow,
compactGenerationContext,
createGenerationContextTransform,
estimateContextBreakdown,
Expand Down Expand Up @@ -1356,3 +1357,25 @@ test("admitted MCP service guidance is measured from the transcript's mcp_server
assert.equal(tokensOf(cleared, "mcp-instructions"), undefined);
near(tokensOf(cleared, "system-prompt"), BASE_PROMPT);
});

test("the prompt cache window follows the last request that used the cache on the same model", () => {
const model = { provider: "openai-codex", api: "openai-codex-responses", id: "gpt-5.3-codex-spark", promptCache: { short: 300, long: 3600 } };
const cached = (timestamp: number, cacheRead: number): AssistantMessage => {
const message = assistant(`call-${timestamp}`);
return { ...message, timestamp, usage: { ...message.usage, cacheRead } };
};
const messages: AgentMessage[] = [user("a"), cached(1_000, 900), user("b"), cached(5_000, 0)];
// The newest response did not touch the cache, so the window runs from the earlier one.
assert.deepEqual(promptCacheWindow(model, messages), { ttlMs: 300_000, lastRequestAt: 1_000 });
// A model without a declared lifetime, a cache never used, or a switched model shows nothing.
assert.equal(promptCacheWindow({ ...model, promptCache: undefined }, messages), undefined);
assert.equal(promptCacheWindow(model, [user("a"), cached(5_000, 0)]), undefined);
assert.equal(promptCacheWindow({ ...model, id: "other-model" }, messages), undefined);

// A warming refresh after a stopped run extends the cache; an older or other-model one does not.
const warmed = { provider: model.provider, model: model.id, requestStartedAt: 270_000 };
assert.deepEqual(promptCacheWindow(model, messages, warmed), { ttlMs: 300_000, lastRequestAt: 270_000 });
assert.deepEqual(promptCacheWindow(model, messages, { ...warmed, requestStartedAt: 500 }), { ttlMs: 300_000, lastRequestAt: 1_000 });
assert.deepEqual(promptCacheWindow(model, messages, { ...warmed, model: "other-model" }), { ttlMs: 300_000, lastRequestAt: 1_000 });
assert.equal(promptCacheWindow(model, [user("a"), cached(5_000, 0)], warmed), undefined);
});
36 changes: 36 additions & 0 deletions main/services/generation-context.ts
Original file line number Diff line number Diff line change
Expand Up @@ -14,12 +14,14 @@ import type {
ContextBreakdownV1,
ContextCategoryId,
DeferredContextServer,
PromptCacheWindowV1,
} from "../../renderer/shared/context-pressure.js";
import { withoutAgentsInstructions } from "./agents-instructions.js";
import { withoutMcpServersSection } from "./mcp-server-instructions.js";
import type { PiDiscoverableTool } from "./pi-tool-discovery.js";
import type { PiRuntimeTool } from "./pi-runtime-tool.js";

import { promptCacheTtlMs } from "./prompt-cache-lifetimes.js";
const TOOL_RESULT_TEXT_LIMIT_CHARS = 32_000;
const RECENT_TOOL_OUTPUT_BUDGET_TOKENS = 40_000;
const MIN_RECENT_TOOL_RESULTS = 2;
Expand Down Expand Up @@ -695,6 +697,40 @@ export function chatContextPressureFromProjection(
};
}

/** A cache-warming request that read or wrote the cache without a transcript entry. */
export interface PromptCacheRefresh {
provider: string;
model: string;
requestStartedAt: number;
}

/**
* The cache window left by the latest request from this provider and model
* that read or wrote the prompt cache. A model switch starts a new cache, so
* earlier requests from another model never count. A later warming refresh
* of a cache the transcript already shows extends it.
*/
export function promptCacheWindow(
model: Parameters<typeof promptCacheTtlMs>[0],
messages: readonly AgentMessage[],
refresh?: PromptCacheRefresh,
): PromptCacheWindowV1 | undefined {
const ttlMs = promptCacheTtlMs(model);
if (ttlMs === 0) return undefined;
for (let index = messages.length - 1; index >= 0; index--) {
const message = messages[index];
if (message.role !== "assistant") continue;
if (message.provider !== model.provider || message.model !== model.id) return undefined;
const usage = message.usage;
if (usage && usage.cacheRead + usage.cacheWrite > 0 && Number.isFinite(message.timestamp)) {
const refreshed =
refresh?.provider === model.provider && refresh.model === model.id && refresh.requestStartedAt > message.timestamp;
return { ttlMs, lastRequestAt: refreshed ? refresh.requestStartedAt : message.timestamp };
}
}
return undefined;
}

function removeOldHistoricalTurns(
messages: AgentMessage[],
preserveUserTurns: number,
Expand Down
22 changes: 21 additions & 1 deletion main/services/llm-client.ts
Original file line number Diff line number Diff line change
Expand Up @@ -173,6 +173,7 @@ import type {
import type { BotDefinition } from "../../renderer/shared/bots.js";
import type { UsageRequestSource } from "./usage-store-core.js";
import type { ProviderFailureV1 } from "../../renderer/shared/provider-failure.js";
import type { ChatContextPressureV1 } from "../../renderer/shared/context-pressure.js";
import { compactionFailureLogMetadata } from "./provider-failure.js";
import type { ComputerUseController } from "./computer-use/controller.js";
import {
Expand Down Expand Up @@ -255,9 +256,11 @@ import {
createGenerationContextTransform,
modelRetainsSystemUpdates,
} from "./generation-context.js";
import { promptCacheTtlMs } from "./prompt-cache-lifetimes.js";
import {
invalidateChatContextJournal,
rememberChatContextProfile,
rememberPromptCacheRefresh,
} from "./context-pressure.js";
import { generationEmergencyUserError } from "./generation-emergency-outcome.js";
import { buildGeminiWorkspaceSnapshot, GeminiContextCache } from "./gemini-context-cache.js";
Expand Down Expand Up @@ -2770,6 +2773,7 @@ export const llmClient = {
let piSession: PiSessionPort | undefined;
let candidate: PiAgentRuntimeHarness | null = null;
let cacheWarmer: PiCacheWarmer | undefined;
let lastContextPressure: ChatContextPressureV1 | undefined;
let currentPromptMessage: AgentMessage | undefined;
let journalContentOverrides: ReadonlyMap<string, string> = new Map();
let piJournalHealthy = true;
Expand Down Expand Up @@ -3321,6 +3325,15 @@ export const llmClient = {
recordUsage: async (message, freshRuntime) => {
await usageStore.record(assistantUsageRecord({ message, provider: freshRuntime.provider, model: freshRuntime.model, source: "cache-warm" }));
},
// A warming read restarts the provider's cache lifetime without a
// new projection, so the countdown is re-sent from that request.
onCacheRefreshed: (requestStartedAt) => {
rememberPromptCacheRefresh(params.chatId, { provider: model.provider, model: model.id, requestStartedAt });
const pressure = lastContextPressure;
if (!pressure?.promptCache) return;
lastContextPressure = { ...pressure, promptCache: { ...pressure.promptCache, lastRequestAt: requestStartedAt } };
sendGeneration(streamId, "chat:context-pressure", { streamId, chatId: params.chatId, pressure: lastContextPressure });
},
});
}
initialization.stopCacheWarming = () => cacheWarmer?.dispose();
Expand All @@ -3337,6 +3350,7 @@ export const llmClient = {
logger.debug("model-pace", `Could not measure pace for stream ${streamId}.`, error),
});
const observedStream: typeof realStream = pace.wrap(warmedStream);
const promptCacheTtl = promptCacheTtlMs(model);
candidate = createGenerationHarness(generationProfile, {
contributions: runtimeContributions,
deferredTools,
Expand Down Expand Up @@ -3366,10 +3380,16 @@ export const llmClient = {
permission,
toolsDisabled: runtime.provider.modelMetadata?.[model.id]?.overrides?.toolCall === false,
});
lastContextPressure = {
...chatContextPressureFromProjection(projection, projectionOptions),
// Projections are emitted as each request is prepared, so the
// cache window restarts now for models that declare one.
...(promptCacheTtl > 0 ? { promptCache: { ttlMs: promptCacheTtl, lastRequestAt: Date.now() } } : {}),
};
sendGeneration(streamId, "chat:context-pressure", {
streamId,
chatId: params.chatId,
pressure: chatContextPressureFromProjection(projection, projectionOptions),
pressure: lastContextPressure,
});
},
...agentRuntimeOptions,
Expand Down
Loading
Loading