feat: send audio as native WeChat voice bubbles - #282
kenhuangus wants to merge 13 commits into
Conversation
Route audio through SILK_V3 + VOICE upload/item instead of a file attachment, with FILE fallback if encode or send fails. OpenClaw asVoice/forceDocument flags are honored.
WeChat clients expect \x02#!SILK_V3. silk-wasm already emits it; this also prefixes raw pass-through SILK.
Needed so we can echo a real client voice payload back as a native bubble.
Live test notes (ken-mac, personal WeChat, plugin 2.4.8 + this branch)What already works
What is not verified yet
Send window
This branch also
Will update this thread when a real-device 语音条 is confirmed or ruled out. |
Real inbound voice bubbles use 16 kHz SILK bytes with encode_type=4. Default prepareOutboundVoice and sendVoiceMessageWeixin to that shape so TTS/MP3 outbound matches what the personal WeChat client already renders.
|
Follow-up: default outbound native voice now matches a real inbound voice bubble from Ken's personal WeChat:
Live echo of Ken's own inbound SILK (6576 bytes, playtime 3637 ms) was sent as native VOICE at 23:40 ET, clientId openclaw-weixin:1788320467959-024de542. Waiting on phone confirmation that a green voice bar rendered above the [ECHO] marker. |
Keep outbound defaults at 16 kHz / encode_type=4. Expand the optional inbound dump so the next real 语音条 can be compared on encrypt_type and CDN fields without writing secrets.
|
Live test update (Ken personal WeChat, iLink bot d2dabd8a1c40-im-bot):
The ECHO used Ken's exact inbound bytes ( Outbound defaults on this branch: 16 kHz + encode_type=4, FILE fallback if encode/upload/send fails. No gateway restart while a WeChat turn is in flight; live dist copied for the next process start. |
Ken's inbound 语音条 had size unset. Combined TEXT+VOICE in one sendMessage is rejected (invalid arguments). Voice-only without size is accepted.
|
Follow-up live send:
Please check whether a green voice bar is immediately above that marker. |
uploaded.aeskey is hex; Buffer.from(hex) was treating hex digits as UTF-8. Inbound images already convert hex with Buffer.from(aeskey, "hex").toString("base64"). Voice bubbles likely require that 24-char form.
|
Follow-up: encode voice/image/video/file media.aes_key as base64(raw 16-byte key), not base64(utf8(hex)). Live send of inbound SILK with corrected aes_key: look above |
…t_type. pic-decrypt parseAesKey: images are base64(raw 16 bytes); file/voice/video inbound is base64 of the 32-char hex string. Sending the image form on VOICE was likely wrong.
|
Follow-up: voice media.aes_key now matches inbound file/voice format (base64 of hex string, 44 chars). encrypt_type omitted on VOICE. Live marker: |
|
Did not ship media.full_url: constructing download URL via buildCdnDownloadUrl on an outbound x-encrypted-param returns CDN 400. Reverted. Plugin remains at encode_type=4 / 16 kHz / no size / voice aes_key=base64(hex). Still waiting on phone confirmation of markers [VOICE-HEX] [VOICE-AES] [VOICE-NOSIZE] [VOICE-16K] [ECHO]. |
ChannelMessageSendMediaContext uses audioAsVoice, not asVoice. The plugin only read asVoice, so TTS sends with asVoice=true (including .pcm) were uploaded as FILE. Map audioAsVoice and recognize pcm/aiff so the native VOICE path actually runs.
|
Deep fix: OpenClaw sendMedia ctx field is |
Needed to distinguish upload/encoding bugs from iLink dropping BOT+VOICE. Dump is env-gated (WEIXIN_DUMP_INBOUND_VOICE=1).
|
Next approach is protocol-level, not another SILK tweak. Independent iLink SDKs (lith0924 Java SDK issue 11, 2026-07-02) report Tencent changed the protocol: bot-sent VOICE is accepted (ret=0) but personal WeChat no longer renders a bubble. Hermes and issue 215 show the same. Remaining discriminators on this PR:
Also: OpenClaw sendMedia ctx field is audioAsVoice (not asVoice). Live TTS was uploading 0-byte .pcm as FILE. |
iLink accepts BOT+VOICE but personal WeChat does not render a bubble. Stop calling sendVoice from sendWeixinMediaFile so TTS cannot emit a silent VOICE item.
|
Outbound native VOICE is now disabled. sendWeixinMediaFile routes audio as FILE only. |
Inbound voice still downloads. Outbound audio (TTS, say, asVoice) is converted to a text message because iLink does not render bot voice bubbles.
|
Same result here, reproducing with the official Tencent PR #9 implementation (direct MP3 upload as We patched the exact PR #9 code into the plugin dist and sent 6 live voice messages to a real personal-WeChat account via a scanned iLink bot session: SILK Also ruled out: playtime unit (both real-ms and size/160 accepted, neither rendered) and SILK validity (our silk-wasm output decodes fine with kn007 Detailed matrix: #215 (comment) This includes the official PR #9 path, not just community guesses — I'd also like confirmation from the team whether outbound VOICE is supported for normal iLink bot accounts, or should be documented as unsupported. |
Summary
audio/*as a native WeChat voice bubble:ffmpeg+silk-wasmto Tencent SILK_V3 (\x02#!SILK_V3), uploadmedia_type=VOICE(4), sendMessageItemType.VOICE(3).encode_type=4,sample_rate=16000, nosize, voiceaes_keyas base64(hex string) perparseAesKey,encrypt_typeomitted.asVoice/forceDocumenthonored.Test plan
npm testsendVoiceMessageWeixinsuccessitem_listis rejected (invalid arguments); synthesizedmedia.full_urlCDN GET returns 400 (not shipped)Related: #91, #215, #192. If the client still drops the bubble, that is the known iLink limitation; this PR still ships the native VOICE path plus FILE fallback.