Skip to content

feat(memory): implement token-aware sliding window memory management - #28

Merged
Nithwin merged 1 commit into
mainfrom
feat/memory-manager
Jul 26, 2026
Merged

feat(memory): implement token-aware sliding window memory management#28
Nithwin merged 1 commit into
mainfrom
feat/memory-manager

Conversation

@Nithwin

@Nithwin Nithwin commented Jul 26, 2026

Copy link
Copy Markdown
Owner

Description

This PR fundamentally changes how the WindMist agent handles conversation history. It replaces the naive 8-message hard cap with a dynamic, token-aware sliding window memory system, preventing the agent from forgetting long-term goals while protecting against token limit crashes (e.g., 400 Token Limit Exceeded when reading large files).

Key Changes

  • agent/messages.go: Introduced estimateTokens and estimateMessageTokens to roughly calculate context size (using the standard len/4 heuristic).
  • agent/messages.go: Re-wrote pruneMessages to use a token budget. It strictly pins the Message[0] (the original user goal) and aggressively keeps the most recent tool calls and thoughts, dynamically discarding intermediate steps from the middle of the history when the 15,000 token limit is reached.
  • agent/limits.go & agent/agent.go: Introduced DefaultMaxContextTokens and wired it into the Agent configuration struct.

Why this is necessary

If the agent reads a 3,000-line file, the old system would keep that massive payload in context for the next 8 turns, burning thousands of API tokens on every request and quickly crashing smaller models. The new system acts like LangChain's ConversationTokenBufferMemory, keeping API costs low and multi-step reasoning focused.

How to Test

  1. Run go run ./cmd/windmist
  2. Ask the agent a multi-step task like: "Read the README.md, then write a summary in a new file, then read chat.go, then delete the summary file."
  3. Verify that the agent successfully completes the 10+ turn journey without forgetting its instructions midway.

@Nithwin Nithwin added bug Something isn't working enhancement New feature or request labels Jul 26, 2026
@Nithwin
Nithwin force-pushed the feat/memory-manager branch from ffd2a7c to a29daa2 Compare July 26, 2026 06:26
@Nithwin
Nithwin merged commit 58af89e into main Jul 26, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant