Skip to content

feat: restore ID-only quoted messages with a local SQLite cache - #283

Merged
6 commits merged into
Tencent:mainfrom
scotthuang:feat/quote-cache-sqlite
Sep 8, 2026
Merged

6 commits merged into
Tencent:mainfrom
scotthuang:feat/quote-cache-sqlite

Conversation

@scotthuang

@scotthuang scotthuang commented Sep 2, 2026 •

Copy link
Copy Markdown

Summary

  • Restore quoted messages when newer WeChat clients provide only ref_msg.svr_id.
  • Persist inbound and outbound quote metadata in an account- and conversation-scoped SQLite side store.
  • Retain quoted images, video, voice, and attachments in plugin-owned OpenClaw managed storage and expose file/PDF tool-access hints to the agent.
  • Keep quote reconstruction enabled by default while providing bounded retention and graceful degradation when node:sqlite is unavailable.

Motivation

Newer WeChat clients may omit the quoted message body and send only its server message ID. Without local reconstruction, text quotes lose their context and attachment quotes degrade to placeholders such as [文件], leaving the agent unable to identify or read the referenced content.

This change preserves the quote experience for existing plugin users despite that client-side protocol change.

Implementation

  • Parse message_id, msg_id, and svr_id losslessly as strings to avoid JavaScript integer precision loss.
  • Store inbound and outbound records by account, conversation, and server message ID.
  • Restore full text quotes and MD5-validated partial quotes.
  • Write inbound media directly into a plugin-owned OpenClaw managed directory so initial delivery and later quotes reuse one file.
  • Restore attachment filename, MIME type, managed source path, and message ID; provide a media/inbound/ workspace hint so file/PDF tools can read the attachment when automatic extraction fails.
  • Migrate media paths created by earlier development builds.
  • Delete account-owned quote records and retained media when the account is removed.

Retention and compatibility

The cache is enabled by default with these bounded defaults:

  • Text/message metadata: 30 days, up to 10,000 records per account.
  • Media: 7 days, up to 256 MiB per account and 25 MiB per file.
  • Cleanup: startup, hourly, every 100 writes, immediately when the media budget is exceeded, and lazily on expired lookup.

The limits and enabled flag are configurable under channels.openclaw-weixin.quoteCache. If node:sqlite is unavailable or the database cannot be opened, quote caching is disabled without an in-memory fallback. Cache failures never interrupt normal message delivery.

Messages that predate cache initialization cannot be reconstructed and return an explicit cache-miss placeholder.

Change size

The PR adds 2,331 lines and removes 56 lines across 26 files. The additions break down as follows:

  • Production source: 1,191 lines (51%).
  • Tests: 989 lines (42%).
  • README/changelog documentation: 151 lines (7%).

The two largest files are the self-contained SQLite/media lifecycle implementation (quote-store.ts, 649 lines) and its focused tests (quote-store.test.ts, 516 lines). The store covers schema initialization, account/conversation isolation, legacy-path migration, retention, count and byte-budget eviction, orphan cleanup, account deletion, and compatibility fallback.

Validation

Automated

  • Product HEAD: 2645786ab5f866bc3ca96775f032dd6e9adc53d7
  • npm run typecheck
  • npm test: 29 test files and 452 tests passed
  • Coverage: 93.18% statements, 90.29% branches
  • GitHub open-source scan: passed

Manual local end-to-end evidence

The following excerpts are from one local OpenClaw session on 2026-09-02 (Asia/Shanghai). User/account identifiers, filenames, full local paths, and server message IDs are redacted; timestamps, MIME type, byte size, tool result, and behavior are preserved.

1. ID-only text quote was reconstructed from SQLite

03:11:28 quote DB row
direction=inbound
server_message_id=<19-digit ID redacted>
body=shadow
has_media=false

03:13:26 agent session result
is_quote=true
quoted body="shadow"

This proves the agent received the stored quoted body rather than only the quote marker/ID.

2. Quoted PDF metadata and its managed file were retained

15:55:42 quote DB row
direction=inbound
body=[文件]
media_mime=application/pdf
media_size=3222698
media_path=~/.openclaw/media/inbound/openclaw-weixin-quotes/<account-redacted>/<file-redacted>.pdf

3. The agent used the supplied path and reached the quoted file

15:56:33 agent tool call
ls -la "~/.openclaw/media/inbound/openclaw-weixin-quotes/<account-redacted>/"
file "~/.openclaw/media/inbound/openclaw-weixin-quotes/<account-redacted>/<file-redacted>.pdf"

15:56:33 tool result
-rw-r--r--  ... 3222698 ... <file-redacted>.pdf
<file-redacted>.pdf: PDF document, version 1.4

15:56:42 agent result
quote marker: is_quote=true
display body: [文件]
quoted attachment: original filename + accessible managed path

This demonstrates the intended fallback: WeChat still supplies [文件] as the display body, while the plugin restores the actual attachment metadata/path and the agent can access that file with a tool.

The local runtime package was installed at 15:32:43. Its quote-critical sources (quote-store.ts, inbound.ts, send.ts, api.ts, config/types/media/partial-quote processing) match this PR HEAD byte-for-byte. Its channel.ts additionally contained the unrelated local PR #269 outbound-route change, so this log is reported as manual runtime evidence rather than an exact-head CI artifact. SQLite PRAGMA integrity_check returned ok after the run.

# Conflicts:
#	src/channel.ts
#	src/messaging/error-notice.ts
#	src/messaging/inbound.test.ts
#	src/messaging/process-message.ts
@zengyi1001 zengyi1001 closed this pull request by merging all changes into Tencent:main in a9ff793 Sep 8, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants