Skip to content

Ignore bytecode and temp files in local plugin watcher - #12020

Open
ram-devv1 wants to merge 2 commits into
omacom:quattrofrom
ram-devv1:fix/plugin-watcher-ignore-bytecode
Open

ram-devv1 wants to merge 2 commits into
omacom:quattrofrom
ram-devv1:fix/plugin-watcher-ignore-bytecode

Conversation

@ram-devv1

Copy link
Copy Markdown

Fixes #11929.

The local plugin watcher forwarded every write under the plugin dir to localPluginChanged, including pycache/*.pyc written by the plugin's own Python helper. Reload restarts the helper, the helper imports the next module, the import writes the next .pyc: 20 reloads/sec, ~95% GUI-thread CPU, ~100 defunct children.

  • Add --exclude for pycache, .pyc/.pyo, .git and editor temps to the inotifywait invocation.
  • Drop the same paths in localPluginIdForPath as defense in depth.

Verified with a live inotifywait smoke test (.pyc suppressed, real .qml still fires); plugins-test passes.

@llstrk

llstrk commented Sep 27, 2026

Copy link
Copy Markdown

Automated AI review

Community review: Independent automated community review, unaffiliated with the Omarchy team, intended to help prepare PRs for their review.

Verified: the change removes the reload triggers behind #11929. In live inotifywait runs with inotify-tools 4.25.9.0 and Python 3.14.7 (the versions in the issue), CPython's bytecode writes stopped producing a plugin id, while edits to plugin sources still did. Below: a conflict with two open PRs that edit the same watcher arguments, a side effect of the suffix patterns in --exclude, an over-broad __pycache__ match, and a test note.

Step (synthetic plugin tree) Base: lines mapped to a plugin id Head
First import (creates __pycache__, temp .pyc.<id>, final .pyc) 14 0 (2 directory lines printed, dropped by the JS filter)
Recompile into an existing __pycache__ 20 0
python -O import (.opt-1.pyc) 20 0
Delete __pycache__ 5 0

Verified: writing QML files, manifest.json and .py files, a vim writebackup save, an atomic save via Service.qml.tmp renamed into place, a new directory with files, and a plugin moved into place from .add.tmp.<pid> all still produce a plugin id on the head.

Open PRs #12189 and #10705 edit the same inotifywait arguments

Both PRs insert an option at the same place in localPluginWatcher.command, and both conflict textually with this PR. Keeping both options when resolving the conflict causes a different problem for each (checked with inotify-tools 4.25.9.0):

PR Adds If combined with this PR's --exclude
#12189 --include "(manifest\\.json$|\\.manifest\\.json$|\\.(qml|js)$)" plus an allowlist in localPluginIdForPath inotifywait exits 1 with include and exclude regexp cannot both be specified.; onExited restarts it every second and no events arrive, so hot reload stops
#10705 a second --exclude (.git, __pycache__, node_modules) only the last --exclude applies; the warning goes to stderr, which the shell discards

Impact: whichever PR merges second can silently disable hot reload (#12189) or drop one of the two filters (#10705) if the conflict is resolved by keeping both lines.

Suggested change: coordinate on a single filter. #12189's allowlist also stops the __pycache__ reloads on its own, so if it lands, this PR's --exclude would need to go.

Suffix patterns in --exclude leave stale paths after a directory rename

libinotifytools applies --exclude before inotifywait updates its watch names for renames. When a directory is renamed away from an excluded name (*.tmp, *.bak, *~), the filtered MOVED_FROM never updates the name, and later events inside that directory are reported under the old path:

mv acme.helper/assets.tmp acme.helper/assets
new file acme.helper/assets/b.svg
  base: acme.helper/assets/b.svg
  head: acme.helper/assets.tmp/b.svg        (stale name, still reloads: id comes from the first segment)

mv .delta.tmp delta                          (top-level hidden staging name)
edit delta/Widget.qml
  base: delta/Widget.qml         -> reload
  head: .delta.tmp/Widget.qml    -> dropped by the existing hidden-entry check, no reload

Impact: a plugin moved into place from a hidden .tmp, .bak or ~ staging directory stops hot-reloading until the watcher restarts. Omarchy's own staging and backup names (.add.tmp.<pid>, .clone.XXXXXX, .<id>.bak.<timestamp>) do not match the patterns, so only manual or third-party flows are affected.

Suggested change: keep only names that are never renamed into place in --exclude, for example (/__pycache__(/|$)|/\.git/) (argv form; not run through inotifywait), and leave the suffix patterns to the JS filter in localPluginIdForPath. In the same runs, the JS filter alone suppressed the artefacts just as well: 21 triggers against 21 in the bytecode and editor run, and 8 against 6 in the rename run, where the 2 extra are the real delta/Widget.qml edits that the head drops.

Unanchored __pycache__ match suppresses plugins whose id contains it

__pycache__ is matched as an unanchored substring in both the regex and relative.indexOf("__pycache__"). bin/omarchy-plugin-validate accepts ids with underscores, so a plugin id such as acme.__pycache__widget is valid. In sandbox runs, edits to that plugin's manifest.json and Widget.qml printed nothing on the head, while the base mapped each to the plugin id.

Impact: such a plugin gets no hot reloads at all. Installs are still picked up, because omarchy plugin add runs an explicit rescan.

Suggested change: match __pycache__ only as a whole path component in both places, for example the regex above (argv form; not run through inotifywait) plus ("/" + relative + "/").indexOf("/__pycache__/") !== -1 in the JS check.

Optional test improvement: the JS filter carries real load. On inotify-tools 4.25.9.0, recursive mode skips the --exclude regex for directory CREATE/MOVED_TO events, so each new __pycache__ directory is still printed and only the new localPluginIdForPath line drops it. The four existing localPluginIdForPath assertions in test/shell.d/fixtures/plugin-registry/shell.qml do not cover any of the new paths, and plugins-test.sh (cited in the PR) validates manifests and a few shell.qml patterns, none of which touch the watcher. Cases such as __pycache__/x.cpython-314.pyc, the temp name __pycache__/x.cpython-314.pyc.1234, a bare .../__pycache__ directory path, Widget.qml~, .Widget.qml.swp, plus a positive helper/x.py, would catch a later removal of that line.


Review information

Test scope: source review of the head against CPython 3.14.7 (importlib._bootstrap_external) and inotify-tools 4.25.9.0 source. Live inotifywait and Python runs used synthetic plugin trees in an isolated sandbox, head and base argv side by side. localPluginIdForPath was evaluated in Node from the source text, not in Quickshell. plugins-test.sh passes on head and base; plugin-registry-contract-test.sh needs a compositor and was not run. No running shell or real third-party plugin, so the full reload storm was not reproduced end to end. The related PRs were checked only by textual merge and inotifywait option behaviour, not reviewed.

AI process: Opus 5.5 Medium coordination and synthesis, Opus 5.5 Xhigh technical review and final fact check, GPT 6 Sol Xhigh search for related issues, Opus 5.5 Medium editorial check.

Opt out: To stop receiving these reviews, reply to this comment saying so.

inotifywait applies --exclude before it updates a watch's name on rename, so a directory moved out of an excluded name such as .delta.tmp keeps being reported under the old path; the hidden-entry check then drops every later edit and the plugin never reloads. Keep only names that are never renamed into place in --exclude and leave the editor-temp suffixes to localPluginIdForPath.

Match __pycache__ as a whole path component in both places, so a valid plugin id like acme.__pycache__widget is not ignored.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@omarchybot omarchybot added the verified Omarchy Triage has verified that this issue is ready for final review label Oct 1, 2026
@omarchybot

Copy link
Copy Markdown
Collaborator

Reviewed and verified on a disposable Omarchy VM running the shell from this branch. I pushed one fix on top (3689230), described below.

Reproduction. I made a plugin directory whose Python package imports five modules (plus one python -O import), which wrote 8 .pyc files. Running from quattro, the shell logged 29 Local plugin changed, reloading lines for those writes. Running from this branch it logged 0, and editing a real Widget.qml still reloaded.

Fix pushed (3689230), acting on two points from the community review above, both reproduced on the VM:

  • inotifywait applies --exclude before it updates a watch's path on rename. A plugin directory moved into place from a hidden staging name like .delta.tmp kept being reported as .delta.tmp/Widget.qml, the hidden-entry check dropped it, and editing that plugin produced 0 reloads (2 on quattro). --exclude now keeps only names that are never renamed into place (__pycache__, .pyc/.pyo, .git/). The editor-temp suffixes stay filtered in localPluginIdForPath, as you wrote them.
  • __pycache__ is now matched as a whole path component in both places, so a valid plugin id like acme.__pycache__widget is no longer ignored.
  • New assertions in test/shell.d/fixtures/plugin-registry/shell.qml cover bytecode files, Python's temporary .pyc.<id> names, the bare __pycache__ directory event, editor backup and swap files, and a positive helper/x.py and __pycache__-named id. plugin-registry-contract-test.sh fails with quattro's registry, fails with this branch's original one, and passes with the fix.

Tests. I ran ./test/cli and the full ./test/shell on the VM. CLI passes. The shell failures are the same set that fails on quattro on that VM: tests needing node, which the image lacks, and two terminal-rendering checks, none of them touching plugins.

Second opinion: Codex Medium. I asked it to choose among this PR, #12189 and #10705 without giving it my answer, and it chose this one: #12189's allowlist stops hot reload for helper scripts and assets a plugin uses, and #10705 slows every reload to 1s. It also reviewed the fixed head and found the regex valid and the bytecode cases covered. It raised two conditional cases, which I did not act on: a helper that points PYTHONPYCACHEPREFIX inside its own plugin directory, and a plugin that loads resources named *.bak or *.tmp. Independence is not guaranteed, since it can read this session.

What it waits on. CI has not run on the new head, and the maintainer has to decide which of the three competing PRs lands. #12189 and #10705 change the same inotifywait arguments: --include cannot be combined with --exclude, and a second --exclude replaces the first, so only one of them can land as is. #12189 also covers #12158 (runtime state such as data.json), which this PR does not.

@greptile-apps

greptile-apps Bot commented Oct 1, 2026 •

Copy link
Copy Markdown
Contributor

RetriggerConfidence Score: 4/5

[Medium risk] Plugin file watcher filters bytecode and editor temp files.

The PR should not merge until the filter preserves reloads for plugin directories the scanner accepts.

Findings

  1. P1 Valid plugin changes are ignored ▶
  2. P2 Watcher filter remains untested ▶

Summary

The PR filters Python bytecode, Git metadata, and editor temporary paths from local plugin reload notifications, and adds path-mapping assertions.

  • The cache-directory filter also suppresses reloads for a plugin whose valid directory name is exactly __pycache__.
  • The new tests do not exercise the watcher-level exclusion.

Reviews (1) · Last reviewed commit: "Keep renamed plugin dirs and __pycache__..."

// __pycache__/*.pyc would otherwise cause a reload storm: reload restarts
// the helper, the helper imports the next module, the import writes the
// next .pyc, repeat. Editor temp files get the same treatment.
if (("/" + relative + "/").indexOf("/__pycache__/") !== -1) return ""

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Valid plugin changes are ignored

A third-party plugin can have the valid ID and directory name __pycache__. The scanner accepts its manifest.json, but this check rejects every path inside that directory. The plugin can load at startup, yet later source edits never trigger localPluginChanged, so the running shell does not reload it.

Knowledge Base Used: Shell UI and desktop interactions

Comment on lines +459 to +465
root.assertEqual(registry.localPluginIdForPath(cloneBase + "/helper/__pycache__/x.cpython-314.pyc"), "", "python bytecode is ignored")
root.assertEqual(registry.localPluginIdForPath(cloneBase + "/helper/__pycache__/x.cpython-314.pyc.1234"), "", "python bytecode temp files are ignored")
root.assertEqual(registry.localPluginIdForPath(cloneBase + "/helper/__pycache__"), "", "python bytecode dirs are ignored")
root.assertEqual(registry.localPluginIdForPath(cloneBase + "/BarWidget.qml~"), "", "editor backups are ignored")
root.assertEqual(registry.localPluginIdForPath(cloneBase + "/.BarWidget.qml.swp"), "", "editor swap files are ignored")
root.assertEqual(registry.localPluginIdForPath(cloneBase + "/helper/x.py"), "dhh.clock", "plugin helper sources are watched")
root.assertEqual(registry.localPluginIdForPath(registry.pluginsDir + "/acme.__pycache__widget/BarWidget.qml"), "acme.__pycache__widget", "plugin ids containing __pycache__ are watched")

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Watcher filter remains untested

These assertions call only localPluginIdForPath; they do not test the new inotifywait --exclude filter, which runs before that function. A watcher-level regression could therefore pass this fixture while bytecode writes still cause reloads or real source edits stop triggering them.

Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

@jandrusk

Copy link
Copy Markdown

Same bytecode/temp ignore as #15093 (and both fix #11929). Prefer the newer #15093 and close this as a duplicate once it lands.

@omarchybot omarchybot removed the verified Omarchy Triage has verified that this issue is ready for final review label Oct 31, 2026
@omarchybot

Copy link
Copy Markdown
Collaborator

#15093 has merged into quattro and fixes #11929 with the same check at the heart of this pull request: localPluginIdForPath now ignores any path with a __pycache__ component, which covers the .pyc writes and CPython's temporary .pyc.<id> names that caused the reload loop. Thanks @ram-devv1 for getting to this first; your reproduction and the community review here shaped what landed.

This one is not made redundant by that merge, so it stays open. Read against quattro at 56a910b, it still does three things #15093 does not:

  • Editor temp files and stray bytecode. The suffix check in localPluginIdForPath (shell/services/PluginRegistry.qml:746 on this branch) ignores ~, .swp/.swx, .tmp, .bak and .pyc/.pyo outside __pycache__. On quattro, opening a plugin file in Vim creates .BarWidget.qml.swp inside the plugin directory, that path maps to the plugin id, and the 150ms timer in shell/shell.qml calls reloadPlugins(), which tears down every plugin panel, service and widget. That is a real spurious reload this change would stop.
  • The inotifywait --exclude. quattro already drops .git and __pycache__ paths in QML, so this only saves event traffic; it changes no reload.
  • A plugin directory named __pycache__ or acme.pyc. Stop reloading a local plugin on its own Python bytecode writes #15093 deliberately keeps these hot-reloading and asserts it (test/shell.d/fixtures/plugin-registry/shell.qml, "a plugin dir named pycache is still watched" and "a plugin dir named like bytecode is still watched"). This branch ignores both: the JS check matches a top-level __pycache__, the \.pyc$ suffix matches acme.pyc, and the --exclude pattern is matched against the full path, so it also drops every event under pluginsDir/__pycache__/ before QML sees it — a regression the fixture would not catch, since it calls localPluginIdForPath directly.

It now conflicts with quattro in both files. A resolution that keeps #15093's behaviour would keep quattro's __pycache__ check, add only the suffix line after it (without \.pyc$/\.pyo$ matching a top-level plugin directory), and drop or re-anchor the --exclude. I have not pushed that, because which files should trigger a hot reload is a choice for the maintainer: #12189 takes the opposite approach (reload only on manifest.json, .qml and .js), and #10705 adds its own --exclude to the same watcher arguments. I removed the verified label, since the earlier verification was against a base this merge has changed and the branch no longer merges cleanly.

What was checked: both diffs read against quattro, the merge tried locally to find the conflicts, and the Vim swap path traced through PluginRegistry.qml and shell.qml. Nothing was run on a VM this time; the earlier VM verification here is superseded. Codex Medium and muse-wonder (pi, thinking high) reviewed the overlap. Both agreed this is not obsolete and that the swap-file reload is real; muse-wonder pointed out that the --exclude would hide a __pycache__-named plugin even if the JS check were resolved in quattro's favour. They saw my conclusion beforehand, so their agreement is not independent.

Next: waiting on the maintainer to decide whether the editor-temp filtering is wanted on top of #15093, or whether #12189's allow-list should settle it instead.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Shell reloads a local plugin on every write under it, including its own __pycache__ — reload storm freezes the shell

5 participants