Skip to content

feat(core): topical dedup context — build_dedup_context + WASM/PyO3 bindings (PRD-04 DEP-3, Part A) - #669

Open
p-diogo wants to merge 3 commits into
mainfrom
feat/dep-3-dedup-context-core
Open

p-diogo wants to merge 3 commits into
mainfrom
feat/dep-3-dedup-context-core

Conversation

@p-diogo

@p-diogo p-diogo commented Sep 30, 2026

Copy link
Copy Markdown
Owner

What

PRD-04 DEP-3 (F2), Part A: core only. This adds the new module rust/totalreclaw-core/src/dedup_context.rs. build_dedup_context(topical, pinned, recent, cap) -> String renders the "Existing memories" block the extraction prompt receives:

  • the header is byte-identical to today's inline block in Hermes and the OpenClaw plugin;
  • one [ID: <fact id>] <text> line per memory, pinned first, then topical, then recent;
  • duplicate fact ids keep the first occurrence;
  • at most cap lines (default 30); "" when nothing survives.

Bindings: WASM buildDedupContext, PyO3 build_dedup_context (JSON in, String out).

Parity

tests/parity/fixtures/dedup-context-v1.json holds 14 cases. generate-dedup-context-v1.py writes it from an independent Python reference implementation. Three suites check it:

  • Rust: cargo test dedup_context (15 tests)
  • WASM: tests/parity/dedup-context-parity.test.ts (new step in CI job cross-language-parity)
  • PyO3: python/tests/test_dedup_context_parity.py (CI job python-tests)

Spec deviation

Enclave spec §4.6 sketches Vec<Claim>. The v0 Claim has no id, and each line must carry the on-chain fact id that the LLM echoes back as existingFactId. The element type is therefore DedupContextItem { id, text }.

Not in this PR

  • No version bump. The coordinator opens the core 2.7.0 line.
  • The Hermes adapter (agent/lifecycle.py) is Part B. It is dispatched after totalreclaw-core 2.7.0 is published.

🤖 Generated with Claude Code

p-diogo and others added 3 commits September 30, 2026 18:01
…ure (PRD-04 DEP-3)

Co-Authored-By: Claude Code <noreply@anthropic.com>
Co-Authored-By: Claude Code <noreply@anthropic.com>
Co-Authored-By: Claude Code <noreply@anthropic.com>
@p-diogo

p-diogo commented Sep 30, 2026

Copy link
Copy Markdown
Owner Author

Whole-PR review (Sonnet): verdict MERGE-WITH-FOLLOW-UPS (nothing blocking)

I read the code and the CI results. Rust Core, Python, Cross-Language Parity and Phrase Safety all passed; the TS, MCP, plugin and web jobs were legitimately path-filtered.

  • Interface fidelity. build_dedup_context(topical, pinned, recent, cap) has the default cap of 30. It orders pinned → topical → recent, dedupes by id with the first occurrence winning, and stops at cap. Fixture cases pin the edge cases: cap=0, all sections empty, a duplicate id across sections, and pinned count ≥ cap.
  • Header. It is byte-identical to python/src/totalreclaw/agent/extraction.py:1028/:1185 and mcp/src/extraction/extractor.ts:761/:1440, and the per-line format [ID: id] text matches. Part B keeps the leading "\n\n" (D2). Two behaviour changes, intended and belonging to Part B: the cap drops from 50 to 30, and ids and texts are collapsed onto one line.
  • CI ordering. The new step runs in cross-language-parity after the wasm-pack build (ci.yml:295).
  • Release. No version or release files are touched, and the CHANGELOG entry sits under [Unreleased]. The 2.7.0 release is the coordinator's (Task A4).

Follow-ups (tracked in the internal repo):

  1. The module and function docs should state the caller contract: per-section limits (top-20 / last-10) are the caller's job, and once the pinned count reaches cap, pinned facts crowd out topical and recent ones.
  2. A negative cap from JS coerces to a huge usize (effectively uncapped), while PyO3 raises OverflowError. Add a one-line guard or a doc note.
  3. Add a PyO3 ValueError test for non-array input.
  4. Python str.strip() also strips U+001C–U+001F and Rust trim does not. This is reachable only through the fixture generator's reference; production runs through Rust. Fix the generator the next time it's touched, or add a fixture case with \x1c at a line edge.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant