Skip to content

Bound clipboard history bytes and keep large copies as files - #11920

Open
Ahmed-Sinkeat wants to merge 11 commits into
omacom:quattrofrom
Ahmed-Sinkeat:clipboard-history-bounds
Open

Ahmed-Sinkeat wants to merge 11 commits into
omacom:quattrofrom
Ahmed-Sinkeat:clipboard-history-bounds

Conversation

@Ahmed-Sinkeat

@Ahmed-Sinkeat Ahmed-Sinkeat commented Sep 15, 2026 •

Copy link
Copy Markdown

Summary

Clipboard history limits entry count but previously carried arbitrarily large copies through capture, the watcher, every save, and startup. This change bounds those paths while keeping large copies in files with small previews. Existing history is migrated before the picker reads it, and actions use the selected entry rather than a position in a potentially stale save.

  • Keep text up to 256 KiB inline. Larger copies, up to 256 MiB, are stored as clipboard-text/<sha256>.txt with an 8 KiB preview and pasted from the full file.
  • Bound serialized history, retained large-copy content, image capture, and clipboard read time. Skip older entries that do not fit while continuing to preserve smaller entries.
  • Stream old inline text into files during load, including histories larger than the previous 32 MiB loader ceiling. Commit the migrated list before handing it to QML and keep the original as a recovery backup when entries were converted or skipped. An entry above the per-copy or retention limits no longer blocks usable entries or disables recording.
  • Send selected inline entries to paste, copy, and open helpers through stdin. A pending or failed save cannot make an action read a different row from disk. Each action gets its own process, so an open editor cannot block later copies or pastes.
  • Serialize load, migration, and saves through a shared file lock and atomic writer. Queue captures received during storage operations, and run pruning under the same lock.
  • Prune after saves and every minute while history is available, protecting names referenced by the saved history, pending captures, and recovery backups. Files evicted during the capture grace period are revisited after copying stops.
  • Skip capture whenever MIME discovery fails or times out, including watched payloads already on stdin. Password managers can signal sensitive content only through a MIME hint, so unavailable metadata must not permit recording. Explicit sensitive states and password hints are also excluded.
  • Show a recovery or load/save failure notice in the picker, preserve outside-edit detection through the bounded loader, and stop capture if history cannot safely load.

Bounds and recovery

Inline text is capped at 256 KiB, serialized history at an 8 MiB UTF-8 budget alongside 500 entries, retained large-copy content at 1 GiB, individual large copies at 256 MiB, and images at 64 MiB. Clipboard payload reads and MIME discovery have a 10-second deadline. The 32 MiB ceiling applies to the JSON returned to QML; legacy input is migrated as a stream under a 30-second deadline.

The lower inline threshold keeps even heavily JSON-escaped copies below the history budget. For existing history, the loader also converts an inline entry to a file if keeping it inline would exhaust that budget. Entries that exceed the per-copy, entry-count, or large-content limits remain in an exact recovery backup while usable entries load normally, with a visible recovery notice. I/O failures leave saved history intact and disable writes until it can load safely.

Symlinks, special files, malformed JSON, and non-array histories are renamed aside without following or opening non-regular inputs. Empty or whitespace-only histories are accepted as empty. Recovery originals remain on disk; their referenced files are protected independently of the active history's retention budget. Cleanup deletes only stale regular hash-named files in clipboard-text, plus stale capture temporaries.

Validation

  • clipboard-test.sh: 155 passing checks, including capture limits, timeouts, cleanup on termination, UTF-16 handling, re-copy timestamps, failed MIME lookup, sensitive-state exclusion, and stdin snapshot actions.
  • clipboard-privacy-test.sh: 10 passing checks covering failed, partially failed, and timed-out MIME lookups for watched text, images, and one-shot capture, plus password hints without the sensitive-state flag. The watched-text failure reproduced before the fix; no rejected payload is persisted.
  • clipboard-upgrade-test.sh: 56 passing checks covering large legacy text, streamed Unicode/control escapes, strict JSON rejection, recovery backups, retention, cleanup protection, selected-entry actions, and queued saves/reloads.
  • clipboard-roundtrip-test.sh: 14 passing checks. Actual save/load round trips preserve retained CJK, emoji, and heavily escaped control text; migration preserves all 96 entries from a legacy CJK history written with the former UTF-16 budget.
  • clipboard-storage-test.sh: 14 passing checks covering oversized-entry recovery, the exact per-copy boundary, concurrent migration/save locking, rejected saves, and atomic-commit failure cleanup.
  • clipboard-lifecycle-test.sh: 3 passing offscreen Quickshell checks using temporary history and stub clipboard commands: copy while an editor remains open, queued saves through file notifications, and periodic cleanup after copying stops. The fixture uses a regular Qt window and does not exercise native Wayland layering.
  • QML text-format checks: 19 passing; overlay-window checks: 20 passing; qmllint and shell syntax checks pass.
  • A real 257 MiB legacy entry is skipped without blocking either neighboring entry, with the complete original kept in a recovery backup. A prior isolated 64 MiB migration preserved the full copy while returning about 8 KiB JSON at about 24 MiB peak process RSS.
  • The recovery notice was previously inspected in the offscreen picker without clipping or overlap.
  • Merged quattro at 8b4eae66, preserving Omasnap image opening and the mapped overlay window.
  • The broader ./test/cli suite fails at vscode generated theme references current theme file. The identical failure reproduces on untouched quattro at 8b4eae66.

The full shell suite and VM graphical acceptance suite were not run.

Clipboard history capped how many entries it kept but never how many bytes,
and the shell held all of them: a copy was carried whole through the watcher
line, re-serialised on every later capture, and read back whole at every
start. omacom#6568 bounded search and display; this bounds the rest while keeping
its promise that nothing is lost.

- capture.sh streams text to a file first. Up to 2 MiB it is decoded as
  before; a larger copy is kept as clipboard-text/<sha256>.txt with an 8 KB
  preview in history, converted from UTF-16 as a stream when needed. A copy
  over 256 MiB is skipped, and the picker says so.
- ClipboardHistory.js bounds inline entries (2 MiB), the whole history (8 MiB
  serialised, alongside the 500-entry limit) and large copies (1 GiB, oldest
  first), refuses an oversized watcher line before parsing it, and reports an
  unreadable history as unreadable instead of empty.
- load-history.sh checks the history file before QML reads it. Symlinks,
  special files, files over 32 MiB and invalid JSON are renamed aside, never
  followed or overwritten. The 32 MiB ceiling covers the heaviest file the
  budget can produce, so the loader never rejects a history it wrote.
- Clipboard.qml reads history only through that helper, starts the watchers
  once it has loaded, never saves before a successful load, and pastes a
  large copy with omarchy-clipboard-paste-file.
- prune-text.sh deletes large copies history no longer uses. It never
  deletes a path read out of a file, and does nothing while a rejected
  history may still need them.
A clipboard owner controls the stream capture.sh reads. Images were written
whole with cat before any size check, and no read had a deadline, so an
arbitrarily large or never-ending stream could fill the disk or hold a watcher.

- read_copy reads at most the limit plus one byte into the temporary file,
  under a 10 second deadline (SIGTERM, then SIGKILL), for images and text,
  watched and one-shot. A copy over its limit or cut off by the deadline is
  deleted and reported as skipped.
- Images are capped at 64 MiB.
- A trap removes the temporary files however the script ends, killed included.
- The skipped note no longer names the text limit, since images and slow
  copies now reach it too.
@omarchybot omarchybot added the enhancement New feature or request label Sep 27, 2026
@llstrk

llstrk commented Sep 28, 2026

Copy link
Copy Markdown

Automated AI review

Community review: Independent automated community review, unaffiliated with the Omarchy team, intended to help prepare PRs for their review.

Verified: for new copies, the PR bounds clipboard data as described. Capture is limited in bytes and by the read deadline, copies over 2 MiB (up to 256 MiB) are stored as clipboard-text/<sha256>.txt with a preview, and the loader refuses symlinks, special files and oversized files without following or opening them, and renames invalid JSON aside. Saves wait for a successful load. test/shell.d/clipboard-test.sh passes at the head (98 ok) and on a local merge with current quattro (98 ok). Also verified:

Case Result
read_copy: source over the limit / stalled / empty skipped (status 141, size limit + 1) / skipped (124) / not recorded
Inline decoder for copies up to 2 MiB body byte-identical to quattro's emit_text; existing UTF-16 tests pass
Re-copy of an existing large copy same path, mtime refreshed
prune-text.sh deletes only regular 64-hex .txt files older than a minute that are not in the overlay's keep list, plus clipboard.* temp files older than 60 minutes; keeps symlinks, other names and fresh files
Invalid-JSON history kept on disk (the base returned [] and the next save overwrote the file)

Three issues remain at the edges the tests do not reach, mostly for histories that already exist.

Existing histories: oversized entries are dropped, and pastes can use the wrong entry until the next save

normalizeEntry now returns null for inline text over 2 Mi UTF-16 units (ClipboardHistory.js:43, :52), and parseHistory skips it. Nothing converts it to a large copy. Picker rows then carry indexes into the shorter in-memory list, while bin/omarchy-clipboard-paste-text (line 38) and bin/omarchy-clipboard-open still read .[$index] from the unchanged file:

file on disk (written by current code)     in memory after load     row index
[0] newest small                           [0] newest small          0
[1] 3 MiB text      -- dropped
[2] older small                            [1] older small           1  -> paste-text reads file[1]
[3] image                                  [2] image                 2

In an isolated run with stub wl-copy, paste-text --copy-only --history-index 1 copied the 3,145,728-byte dropped entry instead of "older small". The mismatch lasts until the first save, which after a login usually means the first copy. That save then removes the oversized entry for good. The base comment says "Pasting reads the full entry back from history by index, so nothing is lost" (base ClipboardHistory.js:163), and the PR description says it keeps that promise.

Two related upgrade cases:

  • A legacy file over the 32 MiB ceiling is renamed aside whole (load-history.sh:37) and the loader exits 0 with []. The picker opens empty with no message, because QML warns only on a non-zero exit (Clipboard.qml:288).
  • A legacy history over the 8 MiB budget loads truncated and is saved that way: twelve 1 MiB entries loaded as 7.

Impact: on upgrade, users with large existing entries lose them, can paste or copy the wrong entry until the next save, and can see their whole history disappear without explanation.

Suggested change: migrate at load by writing oversized inline entries out as clipboard-text files, or ship a one-time migration. At minimum, save right after a load that dropped entries so the file and in-memory indexes agree. For a legacy file over the ceiling, a streamed migration (for example with jq --stream, not tried here) could keep the small entries, and the picker could say that a history was set aside.

Any rejected history permanently disables large-copy pruning

prune-text.sh:20 exits before deleting anything while any clipboard-history.json.rejected-* file exists, and nothing ever removes those files. In a probe with a rejected file present, a 10-minute-old unreferenced large copy was kept. Rejection happens automatically, not only for crafted files:

  • an empty or whitespace-only history file (jq -e exits 4 on no input, load-history.sh:43); the base treated it as an empty history;
  • a symlinked history, including one a user pointed elsewhere (for example into tmpfs). The base read and wrote through the link; now the link is renamed aside and a regular file is written in its place;
  • a legacy file over 32 MiB (previous finding), a directory, a FIFO, invalid JSON, or JSON that is not an array.

Impact: after any rejection, the 1 GiB large-copy budget no longer bounds disk use: addEntry still drops entries, but their files stay, so every later large copy of up to 256 MiB accumulates. "Delete entire clipboard history" no longer removes large-copy files either. The rejected file itself keeps old clipboard contents on disk, outside anything the picker can clear. The same growth happens while load-history.sh exits 3, since saves (and therefore prunes) never run while capture keeps writing large copies.

Suggested change: limit the guard to the file names the rejected history actually references, or let the guard expire. Since rejected files can be oversized, invalid or empty, a format-independent scan such as grep -aoE '[0-9a-f]{64}\.txt' over the file would find those names where a jq read could not. Treat a zero-length or whitespace-only file as [], as the base did, and tell the user when a history was rejected.

The byte budget evicts all older entries at the first one that does not fit

addEntry always keeps the newest entry and then stops at the first older entry that does not fit the 8 MiB budget (break, ClipboardHistory.js:137); parseHistory does the same (:108). One inline entry can be 2 Mi units, a quarter of the budget, and up to about six times that once control characters are JSON-escaped. Probe results:

  • 400 small entries followed by four newline-rich copies of about 2 MiB each left 3 entries, none of them small.
  • One control-heavy copy of 1.5 Mi characters (9 MiB serialized) left a history of 1.

Impact: a few ordinary copies just under the 2 MiB inline limit (a log, a JSON dump) permanently wipe the rest of the history, since the result is saved to disk. Copies over 2 MiB are not affected this way, because they go to files and add only a short preview to the budget.

Suggested change: large copies already use continue so small entries survive. Either skip rather than break for oversized inline entries, or lower the threshold for storing a copy as a file (for example to a few hundred KiB) so the budget holds many entries.

Related open PRs

  • Prevent clipboard capture hangs #9488 conflicts with this head in capture.sh: both change the one-shot wl-paste --type text line. It merges cleanly only with the first commit (a157ae2), so the description's "merge cleanly" is out of date for Prevent clipboard capture hangs #9488. Prevent clipboard capture hangs #9488 also adds a timeout to wl-paste --list-types, which this PR leaves unbounded (capture.sh:35) and which runs before every capture, watched ones included. A resolution could keep that timeout.
  • Let plugins contribute actions to the clipboard overlay #10919: keeping both sides is not enough. Its entryForAction builds text: fullText(entry) and sets path only for images and single file:// entries, so for a largetext entry an extension gets the 8 KB preview plus "… N MB in all" and an empty path. It needs a largetext branch that passes the path and MIME type.
  • Type clipboard history as keystrokes with Ctrl+Enter #11701: its typeSelected sends every non-image row with non-empty fullText to omarchy-clipboard-paste-text --type. A largetext row has a preview as fullText, and paste-text accepts only type == "text" (line 38), so Ctrl+Enter on a large copy would close the picker and do nothing. The two PRs conflict in that QML hunk, so the outcome depends on the resolution.
  • Current quattro: the PR conflicts only in test/shell.d/clipboard-test.sh (the Tensaku to Omasnap rename next to the appended tests). The resolved merge passes 98.

Smaller notes

  • Prune runs before the save is known to have landed. saveHistory() starts prune-text.sh right after setText (Clipboard.qml:87-91), which only starts an asynchronous write. If the write fails and the same save dropped a large copy, that file can be deleted (once it is about a minute old) while the old history still lists it. In headless Quickshell 0.3.1, a read-only history file and an unwritable directory emitted saveFailed, and a small failed write emitted saved after logging "Atomic commit failed". After a restart the row comes back, and pasting it silently does nothing (omarchy-clipboard-paste-file exits at its -r check). A bounded read of the on-disk history in prune-text.sh, keeping every large-copy name it lists and deleting nothing if it cannot be parsed, fixed this in a probe. Also running prune from onSaved would avoid deletions lagging one save behind, which matters for Clear, but onSaved alone misses the commit failure reported as saved.
  • Loader and parser disagree on valid JSON. jq -e accepts two concatenated arrays and [NaN], so load-history.sh passes them through, but parseHistory returns null for both. historyWritable then stays false with nothing logged, and the overlay stops saving in every session. A console.warn for a null parse, or handling it like the reject path, would make this visible.
  • Large-copy UTF-16 detection lacks the inline decoder's check for control characters after decoding. Input of 01 00 pairs stays raw as an inline copy but is stored with the NUL bytes removed as a large copy.
  • trap 'exit 143' TERM INT HUP is redundant and delays exit. Bash already runs the EXIT trap on terminating signals once one is set. With the explicit trap, capture.sh waited for the read to end (4.5 s after SIGTERM with a 5 s deadline). Without it, it exited at once, and both variants removed the temp file.
  • Removing watchChanges means an outside edit or delete of the history file is no longer picked up, and the next save overwrites it. The base reloaded on change.

Optional test improvement: in a mutation run, each of these changes still passed all 98 tests: removing the loader's size ceiling or accepting non-array JSON; removing prune-text.sh's rejected-history guard, its -mmin +1 age check or its name filter; removing the re-copy touch; disabling large-copy UTF-16 conversion; removing the historyWritable guard or the prune call; and lowering historyFileLimit from 32 to 13 MiB. That last one passes because "clipboard never writes a history its loader would refuse" checks only the final state (one 12.6 MB entry), not the 23.6 MB state just before it. The test file also has no assertion on the largetext branches in applySelected and copySelected. Checking the file size after each addEntry would cover the file limit, and regex assertions for the QML guards (as this test file already uses for other QML properties) would cover those; the loader, prune and capture rules need their own shell cases.


Review information

Test scope: Source review of the head, base and current quattro, plus upstream Quickshell 0.3.1, wl-clipboard 2.3.0, Qt and bash sources where conclusions depend on them. clipboard-test.sh and qml-text-format-test.sh were run in an isolated sandbox at the head and on a local merge with quattro, and clipboard-test.sh against mutated copies. Script and helper probes used stub wl-paste/wl-copy; headless Quickshell probes covered FileView saves and the load path. No overlay was launched, no live clipboard was used, the full test/shell suite was not run, the memory figures in the description were not re-measured, and timings come from the sandbox.

AI process: Opus 5.5 Medium coordination and synthesis, Opus 5.5 Xhigh technical review and final fact check, GPT 6 Sol Xhigh search for related issues, Opus 5.5 Medium editorial check.

Opt out: To stop receiving these reviews, reply to this comment saying so.

@greptile-apps

greptile-apps Bot commented Sep 30, 2026 •

Copy link
Copy Markdown
Contributor

RetriggerConfidence Score: 5/5

[Medium risk] Clipboard history now stores large copies as files instead of inline.

The PR appears safe to merge based on the reviewed changes and resolved prior findings.

Summary

The PR bounds clipboard capture and history storage, migrates large legacy entries to files, and makes picker actions use the selected entry rather than a saved index.

  • The latest changes make backup-removal errors warnings after a successful clear and show a distinct picker notice.
Diagram
%%{init: {'theme': 'neutral'}}%%
flowchart LR
  Capture[Clipboard capture] --> History[Bounded history]
  Capture --> Files[Large-copy files]
  History --> Save[Locked atomic save]
  Save --> Load[Locked load and migration]
  Load --> Picker[Clipboard picker]
  Picker --> Action[Selected-entry action]
  Files --> Action
Loading

Reviews (5) · Last reviewed commit: "Report an undeletable recovery backup wi..."

Comment thread shell/plugins/clipboard/prune-text.sh Resolved
Comment thread shell/plugins/clipboard/capture.sh Resolved
Deleting the entire history left the .migrated-* and .rejected-* recovery
copies on disk, so cleared clipboard text survived the clear, and the large
copies those backups referenced stayed protected from pruning. The save that
commits the cleared history now removes them under the same lock.

Also clear the save failure notice once a later save succeeds, and drop
recovery notices when the history they refer to is cleared.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@Ahmed-Sinkeat

Copy link
Copy Markdown
Author

@llstrk thanks for the thorough review. All three are addressed:

1. Existing histories (4452616). Loading now goes through migrate-history.py, which streams the legacy file, moves oversized inline text into clipboard-text/<sha256>.txt at the same position, and commits the result under the history lock before QML sees it, so disk and picker always agree. Paste, copy and open also no longer read .[$index] from disk: they receive the selected in-memory entry on stdin, so a stale or failed save can't make them act on a different row. A legacy file over 32 MiB is migrated as a stream instead of being set aside, and empty or whitespace-only files load as []. If anything is set aside or kept only in the recovery backup, the picker now says so.

2. Pruning after a rejection (4452616, e9fc96a). The blanket guard is gone. prune-text.sh keeps only the large-copy names that .rejected-* / .migrated-* backups actually mention, using a chunked byte scan so malformed or oversized backups still work. On the related point that clearing history left old contents on disk: "Delete entire clipboard history" now removes those recovery backups in the same locked save (e9fc96a), so the cleared text doesn't remain and the large copies they referenced get pruned. A symlinked history is still renamed aside rather than followed, which is deliberate since the loader never follows links, but it no longer affects pruning.

3. Budget eviction (4452616). addEntry and parseHistory skip an entry that doesn't fit (continue) instead of stopping, and the inline limit is now 256 KiB, so a few near-limit copies can't evict the rest of the history.

On a merge with current quattro, all clipboard suites pass (storage 17, upgrade 59, clipboard 155, roundtrip 14, privacy 10, lifecycle 3), and the menu, overlay and CLI suites match plain quattro.

The cleared history is already committed when backups are removed, so an
unlink error made the save exit 3 and the picker claimed the previous
history was kept. Warn about the backup instead and say so in the picker.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@jandrusk

Copy link
Copy Markdown

Heads-up: #14919 covers much of the same ground (bounded clipboard history, one locked storage helper, atomic writes). This branch has also conflicted with quattro since early October. Could you compare notes on #14919 so the maintainer reviews one approach? From what I can see, the large-copies-as-files part here is the piece #14919 doesn't cover.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants