feat(dreaming-engine-v2): add dreaming engine with scope isolation and embedded reflections#752
Open
jlin53882 wants to merge 10 commits intoCortexReach:masterfrom
Open
feat(dreaming-engine-v2): add dreaming engine with scope isolation and embedded reflections#752jlin53882 wants to merge 10 commits intoCortexReach:masterfrom
jlin53882 wants to merge 10 commits intoCortexReach:masterfrom
Conversation
… tests Clean implementation addressing all reviewer feedback from PR CortexReach#592: MR1 — Scope isolation: Each phase filters store.list() by scope. Dreaming runs per-scope using scopeManager.getAllScopes(). MR2 — REM reflection loop prevention: Reflections tagged with metadata.source = 'dreaming-engine' and excluded from all phase inputs. F2 — REM reflections now embedded via embedder.embed() instead of vector: []. Falls back to zero-vector on embedding failure. F3 — DEFAULT_DREAMING_CONFIG + mergeDreamingConfig() provides null-safe deep merge. Minimal config { enabled: true } works. F6 — Removed unimplemented fields (storageMode, separateReports, timezone) from schema. Only runtime-active fields exposed. Also includes: - 8 unit tests covering MR1, MR2, F2, F3, all 3 phases, error resilience - Dreaming wired inside async start() callback (fixes ParseError) - Cron scheduler with per-scope execution - DREAMS.md report generation per scope
getDefaultWorkspaceDir now prefers workspace-main (standard OpenClaw layout) over the generic workspace directory.
…opes getAllScopes() only returns scopes in config definitions, missing dynamic agent scopes like 'agent:main'. Now discovers scopes from actual memories in the store so dreaming processes all memory spaces.
AccessTracker was defined in retriever.ts but never instantiated in index.ts. This meant every memory had access_count=0, preventing the deep sleep phase from promoting anything. Now access_tracker is created after retriever initialization and connected via setAccessTracker(). This ensures manual recalls bump access_count, enabling proper decay scoring and tier promotion.
Blockers fixed: - dreamingTimer ReferenceError: moved declaration to service scope (same level as backupTimer) so stop() can access it - Test suite: dreaming tests now wired into 'npm test' via npx tsx - All 8 tests pass Implementation fixes: - statSync removed (used readFileSync instead for workspace detection) - Deep sleep importance now persisted via store.update() to top-level column, not just metadata - Zero-vector fallback uses config.embedding.dimensions instead of hardcoded 1024 - parseCron() now supports dayOfMonth, month, dayOfWeek fields - Scheduler runs scopes sequentially to prevent DREAMS.md write races - Added per-scope re-entrance guard (runningScopes Set) - Mock store in tests includes update() method
Blockers fixed: - Rebased onto latest master (0545c91 → includes new commits) - parseCron step=0 infinite loop: validate step > 0 before loop Non-blocking items fixed: - Scope isolation: all three phases now filter e.scope === scope explicitly, excluding null-scope/global memories that store.list() includes via OR scope IS NULL backward compat - Scope discovery: paginated through all memories (batches of 1000) instead of hard 500 limit - tsx added to devDependencies (was missing, npx tsx was cache-dependent) - Added test: testScopeExcludesNullScope (simulates real store.list null-scope leakage and verifies dreaming engine strict filter) All 9 dreaming engine tests pass. Merge conflicts resolved cleanly.
…#672 Blockers fixed: - Scoped pagination starvation: collectExactScope() paginates through store.list() results to collect exact-scope rows, preventing starvation when null-scope rows fill bounded pages before target-scope rows - Regression test added: 20 global entries (newer) + 8 target entries (older) verify all 3 phases still find and process target scope memories Implementation fixes: - fallbackDimensions: uses embedder.dimensions instead of hardcoded 1024 - Cycle-level guard: boolean flag prevents overlapping dreaming cycles and concurrent DREAMS.md read/prepend/write operations - recencyHalfLifeDays: implemented in deep sleep — recently-accessed memories get a multiplicative boost (up to +0.2) on composite score - AccessTracker cleanup: flush/destroy on plugin stop - Zero-vector REM reflections: skip storing reflections entirely when embedding fails, instead of persisting unusable zero-vectors All 10 tests pass (including new null-scope starvation regression test).
Cherry-pick from PR CortexReach#650 (nexus/runtime-dreaming-fixes): - registerMemoryCapability API with getMemorySearchManager - formatActiveMemoryPath / parseActiveMemoryPath path helpers - formatMemoryDocument / readMemoryDocumentWindow helpers - Feature-detect: only call api.registerMemoryCapability if available (fixes crash on older OpenClaw hosts that lack this API) Addresses Issue CortexReach#608 (memory-core full compatibility)
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: c60764fd9c
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Comment on lines
+4526
to
+4527
| if (accessTracker) { | ||
| accessTracker.destroy(); |
There was a problem hiding this comment.
When the service stops after a manual recall has scheduled a debounced access-count write, this accessTracker identifier is not in the register() closure—the tracker is created as a local inside _initPluginState() and is not returned in PluginSingletonState. The resulting ReferenceError is caught by the surrounding cleanup catch, so the tracker is never destroyed and its timer/pending writes can outlive plugin stop or re-registration.
Useful? React with 👍 / 👎.
Comment on lines
+145
to
+149
| if (newMatches === 0) { | ||
| emptyPages++; | ||
| if (emptyPages >= MAX_EMPTY_PAGES) { | ||
| debugLog(`paginate [${scope}]: stopping after ${MAX_EMPTY_PAGES} consecutive pages with no exact-scope matches`); | ||
| break; |
There was a problem hiding this comment.
For non-global scopes that have older memories behind many newer legacy/null-scope rows, this early stop can prevent dreaming from ever reaching the scoped memories: MemoryStore.list([scope], ...) includes OR scope IS NULL, then this helper drops those rows and stops after three empty pages. In a store with more than 3 * pageSize newer null/global rows, light/deep/REM will report no entries for the target scope even though matching entries exist later in the sorted result set.
Useful? React with 👍 / 👎.
…Reach#571/CortexReach#577 - Add test/dreaming-engine.test.ts to core-regression group via jiti runner - Append jiti test run to npm test script - Minor formatting fix to test() wrapper (same logic, cleaner layout)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Overview
ix/dreaming-engine-v2 is the v2 revision of PR #672, addressing all reviewer feedback from rwmjhb.
What this PR does
Implements a Dreaming Engine — a periodic memory consolidation system that runs during agent idle time.
Core Features
Config Schema
ts
interface DreamingConfig {
enabled?: boolean; // default: false
intervalHours?: number; // default: 24
phases?: {
analyze?: boolean;
consolidate?: boolean;
cleanup?: boolean;
};
}
Fixes from PR #672 review
Testing