Skip to content

feat(memory): episodic memory consolidation pipeline#49

Merged
Mathews-Tom merged 2 commits into
mainfrom
feat/memory-consolidation
Mar 25, 2026
Merged

feat(memory): episodic memory consolidation pipeline#49
Mathews-Tom merged 2 commits into
mainfrom
feat/memory-consolidation

Conversation

@Mathews-Tom

Copy link
Copy Markdown
Owner

Summary

Add a memory consolidation pipeline for episodic memory lifecycle management, inspired by memora-lab/memory-service-public's observation lifecycle patterns. Episodes now have a complete lifecycle: created → resolved → archived → summarized into lesson notes → promoted to permanent vault notes. This is the final PR (#49) in the memora-inspired enhancement plan.

Three-Step Consolidation

MemoryConsolidator.consolidate()
  ├─ Step 1: Archive old resolved episodes
  │    └─ Episodes resolved > 365 days ago → outcome_status = "archived"
  │
  ├─ Step 2: Summarize archived clusters
  │    ├─ Group archived episodes by shared entities
  │    ├─ Clusters with 3+ episodes → LLM synthesis
  │    └─ Write lesson notes to _meta/lessons/{entity}-{date}.md
  │
  └─ Step 3: Promote frequently-referenced episodes
       ├─ Find resolved episodes whose entities appear in 3+ other episodes
       └─ Write permanent notes to _meta/episodes/{decision}-{date}.md

Episode Lifecycle (Complete)

PENDING → SUCCESS/FAILURE/PARTIAL/UNKNOWN → ARCHIVED → lesson note / promoted note
   ↑              ↑                            ↑              ↑
 create()      resolve()          archive_old_resolved()  consolidate()

Changes

New Files

  • src/vaultmind/memory/consolidation.py (241 lines) — MemoryConsolidator class with consolidate() orchestrator, _archive_old_episodes(), _summarize_clusters() (groups by shared entity, LLM synthesis, writes lesson notes), _promote_referenced() (cross-reference counting, writes permanent notes). ConsolidationReport dataclass tracks counts and errors
  • tests/test_consolidation.py (230 lines) — 14 tests across 5 classes

Modified Files

  • src/vaultmind/memory/models.py — Added ARCHIVED = "archived" to OutcomeStatus enum, enabling the archive lifecycle state
  • src/vaultmind/memory/store.py — Three new methods:
    • archive_old_resolved(age_days=365) — UPDATE resolved episodes older than threshold to "archived" status. Skips pending and already-archived. Returns count
    • query_archived(limit=100) — retrieve archived episodes ordered by created desc
    • count_entity_references(entity) — count episodes mentioning a given entity (for promotion threshold)
  • src/vaultmind/config.py — Added ConsolidationConfig class with enabled (default False), retention_days (365), min_references_for_promotion (3), schedule ("0 0 1 * *" — monthly). Added consolidation field to Settings
  • config/default.toml — Added [consolidation] section with all config entries

Backward Compatibility

  • ConsolidationConfig.enabled defaults to False — zero behavior change for existing users
  • ARCHIVED status is additive to OutcomeStatus enum — existing statuses unaffected
  • New store methods are additive — existing query/resolve/create methods unchanged
  • All existing episodic tests pass unchanged (17 in test_episodic.py, 15 in test_memory_horizon.py)
  • Consolidation writes to _meta/lessons/ and _meta/episodes/ — isolated from user content

Output Examples

Lesson note (_meta/lessons/lesson-kubernetes-20260326.md):

---
title: Lessons — kubernetes
type: permanent
tags: [lesson, consolidation]
---
# Lessons — kubernetes
- Lesson 1: Always test
- Lesson 2: Keep it simple
---
*Synthesized from 4 episodes.*

Promoted note (_meta/episodes/episode-api-decision-20260326.md):

---
title: API decision 0
type: permanent
tags: [episode, promoted]
---
# API decision 0
**Context:** ...
**Outcome:** Result 0 (success)
## Lessons
- lesson 0
**Entities:** api

Test plan

  • 14 new tests in test_consolidation.py across 5 classes:
    • Archive: old resolved archived, recent skipped, pending skipped, already-archived skipped (4)
    • Query archived: returns archived, empty when none (2)
    • Entity references: counts correctly across episodes (1)
    • Consolidator: archives, summarizes clusters with LLM, skips without LLM, promotes referenced, disabled no-op (5)
    • Config: defaults, Settings integration (2)
  • All existing episodic/horizon tests pass unchanged
  • Full suite: 1028/1028 tests pass, 0 regressions
  • ruff check — clean
  • mypy --ignore-missing-imports — clean
  • Integration: wire into scheduler as monthly job
  • Manual: run consolidation on vault with 365+ day old episodes, verify lesson notes

New module memory/consolidation.py with MemoryConsolidator that runs
three-step consolidation: archive resolved episodes older than
retention_days, LLM-summarize clusters of archived episodes into
lesson notes grouped by shared entity, and promote episodes referenced
by 3+ other episodes to permanent vault notes.

Add ARCHIVED status to OutcomeStatus enum. EpisodeStore gains
archive_old_resolved(), query_archived(), and count_entity_references()
methods. Add ConsolidationConfig (disabled by default, monthly schedule).
14 tests across 5 classes: archive old resolved episodes (4),
query archived (2), count entity references (1), full consolidation
with archive/summarize/promote (5), ConsolidationConfig defaults and
Settings integration (2).
@Mathews-Tom Mathews-Tom merged commit 7be1f41 into main Mar 25, 2026
3 checks passed
@Mathews-Tom Mathews-Tom deleted the feat/memory-consolidation branch March 25, 2026 19:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant