type	Overview
title	effect-watchman
description	Independent Effect watching and additive approved-root glob enumeration.
resource	/README.md
status	active
author	GPT-6.1 Sol xhigh
date	2026-10-28
sources	resource title opencode://ses_e7c90f1d6ffeviS28mp5Msdsk1 Implement JSON-default dual backends; human approves the ledger and permanent lossy-BSER contract, authorizes client publication/log parity, and adopts the published GitHub client resource title opencode://ses_e7c9d0041ffeVARzmCo25gRZvM Ship raw TypeScript sources; repoint package entrypoints at .ts files resource title opencode://ses_e809723ffffe8kD6j9UHE5XIV7 Clarify observed baseline membership versus finite enumeration ordering for source-materialization S1 resource title opencode://ses_e87038c1affeHRNE9v4Ggec0Sl Document additive recovery signals and the replaced shared-owner terminal-loss behavior resource title opencode://ses_ead2c9a29fferY0Og3JRINkhz4 Added the additive enumeration entrypoint and API reference; retained watch-only documentation resource title /ENUMERATION.md Glob and baseline API details

effect-watchman

Independent enhanced Effect 4.0.1 filesystem watching through an explicitly configured Watchman Unix socket. No Watch Plus runtime or layer is required.

Glob/enumeration is additive: use Glob.layer({ socketPath, approvedRoots }) or Watchman.layerWithGlob(...) for shared watch/query ownership. This includes selective daemon queries, npm glob traversal, explicit native symlink/ignored-root supplements and complete baselines. The existing Watchman.layer stays watch-only. See enumeration API and limits.

The package ships raw TypeScript sources: main/types/exports point at .ts files and there is no build output. Consumers type-strip (Bun) or use their own consumption tooling; no dist step is needed when linking the repository. effect-watchman resolves its effect-watch-plus/inventory peer subpath the same way.

import { NodeFileSystem, NodePath } from "@effect/platform-node"
import { Effect, FileSystem, Layer, Stream } from "effect"
import { Watchman } from "effect-watchman"

const filesystem = Watchman.layer({ socketPath: "/path/to/watchman.sock" }).pipe(
  Layer.provide(Layer.mergeAll(NodeFileSystem.layer, NodePath.layer))
)
const program = Effect.gen(function*() {
  const fs = yield* Watchman.fileSystem
  yield* fs.watch(".", {
    recursive: true,
    ignore: ["node_modules", ".git", "**/{node_modules,.git}/**", "**/*.log"]
  }).pipe(Stream.runForEach((event) => Effect.log(event)))
}).pipe(Effect.provide(filesystem))

• layer({ socketPath }): enhanced filesystem with one scoped connection/manager.
• backendLayer({ socketPath }): ordinary Effect WatchBackend. Provide it to NodeFileSystem.layer at construction. For both routes sharing one owner, reuse this exact layer value with fromBackend and the Node filesystem layer.
• fromBackend: capture one provided Effect WatchBackend and base filesystem/ Path; replace only watch. It never calls native watch; backend None fails.
• fileSystem: checked typed view of the same Effect filesystem object; absent ~effect-watch/options/v2: true fails rather than silently ignoring options; the older literal-only v1 marker is rejected.
• make(fs, path, backend): explicit direct construction/testing seam.
• make(fs, path, backend, policy) / configured(policy): instance-wide ignore and defaultIgnores for the enhanced filesystem. layer accepts these policy fields alongside socket/connection options.
• ignore: relative literal subtrees or whole-path globs, compiled once with minimatch 10.0.3. Supports globstar, *, ?, braces, and dotfiles; filtering is path-only, including deletes. No Git-ignore/reinclude semantics. Absolute, empty, parent-segment and leading-negation patterns fail. See the complete v2 ignore contract for anchoring, escaping and malformed patterns.
• defaultIgnores: defaults to true. Suppresses .watchman-cookie-* files and subtrees at any depth, including create/delete hints. Set false to expose cookies without disabling any supplied ignores. Instance and per-watch ignore arrays are unioned/deduplicated; the per-watch boolean overrides the instance boolean. The enhanced layer applies this policy after raw daemon delivery, so a per-watch opt-out is not defeated by unconditional filtering deeper in the backend. Standalone backendLayer also accepts defaultIgnores: false; it has no per-watch enhanced policy. Neither library layer reads process environment variables or configuration files.
• followReplacements: true: existing file parent-anchor, filter basename; parent/root replacement and missing targets are not supported. It watches the lexical entry, not writes through symbolic links to a referent.

Ordinary streams emit Create/Update/Remove hints and still fail on continuity loss. Reconcile before explicitly reopening after loss. The shared owner now recovers its connection, so new operations can use the replacement generation. Opt-in: Recovery.fileSystem returns a checked signal capability whose watches retain desired interests and emit acknowledged attachment/reset boundaries before later events. It also exposes passive owner health. No fake root update, mandatory observation service, portable cursor or native fallback. See recovery contract. Ordinary Effect consumers and non-watch methods remain usable. Plus's checked accessor is structurally interoperable but no Plus dependency is installed.

The manager uses watch-project, then clock / subscribe with since; initial full inventory is not emitted as fake creates. Events use absolute logical paths, with canonical directory routing and independent named subscriptions. Same-directory project acquisition is cached; daemon roots are shared, but overlapping subscriptions are not merged. Closing a consumer unsubscribes it; closing the layer destroys its socket. Daemon-owned root watches are not deleted. Ignores are userland filters.

Defaults: connect/request 3000ms, cleanup 500ms, frame payload 1MiB, 128 active/queued commands, 1024 buffered events per consumer. Options exposes corresponding connectTimeoutMs, requestTimeoutMs, cleanupTimeoutMs, maxFrameBytes, maxPendingCommands, maxBufferedEvents; all must be positive finite integers. For ordinary watches, disconnect, canceled subscription, fresh history, malformed/mismatched replies, deadline expiry and local overflow fail visibly. Canceling an unsent command removes only that request. Canceling a transmitted command interrupts its caller but retains the wire reply owner/deadline; late attachment acknowledgements are followed by scoped unsubscribe without disrupting healthy siblings. Cleanup may wait up to request + cleanup timeout in the queue, then has its own cleanup timeout after transmission; silence/corruption can still terminate the shared connection. Cached project acquisition is manager-owned, not tied to the first requester's lifetime. Ordinary watches are not transparently retried; recovery signals are a separate opt-in route. There is no discovery, autospawn, catch-up API, arbitrary symlink traversal or topology-replacement guarantee. Daemon ignore_dirs / ignore_vcs still constrain coverage.

Wire backend choice

JSON stays the default. wireFormat?: "json" | "bser" is a typed construction option on Watchman.layer, Watchman.backendLayer, Glob.layer, and Watchman.layerWithGlob. All watch, query, recovery and inventory consumer interfaces stay the same. Selection is fixed for that owner, including reconnects; there is no negotiation or automatic fallback.

const json = Watchman.layer({ socketPath: "/path/to/watchman.sock" })
const bser = Watchman.layer({ socketPath: "/path/to/watchman.sock", wireFormat: "bser" })

Choice	Wire implementation	Text contract
"json" (default)	Extracted watchman-client-json; reviewed S1 segmented JSON-lines framing	Fatal UTF-8 rejection; malformed text is connection-terminal
"bser" (opt-in)	BSER v1 length-prefixed PDUs with @superbfowle/bser-esm 3.0.0	Lossy UTF-8: invalid bytes silently become U+FFFD; not strict-text parity

The human accepted this BSER degradation on 2026-10-28, with JSON remaining the default and the skipped BSER fatal-text test retained permanently. Schema validation after decoding cannot detect replaced bytes or recover the original filenames. A fatal-string/bounds patch would require maintaining and distributing a real codec fork; this integration does not patch the codec. Do not choose BSER where exact non-UTF-8 rejection is required. Its exact int64 decoder returns bigint outside the safe number range; such values in numeric filesystem metadata fail validation rather than silently rounding. Query-schema failure is operation-local; malformed subscription payloads remain terminal.

Both maxFrameBytes and maxQueryBytes count encoded payload bytes: JSON excludes LF; BSER excludes its 4–11-byte v1 header. Equality is legal, and each PDU has its own limit. BSER rejects oversized declared lengths before allocating the payload. One bounded PDU buffer plus the codec's separate accumulator/decoded objects is not a decoded-heap or nesting/expansion bound. V2 is unsupported. Command FIFO ownership after cancellation, listener-before-subscribe and complete delivery ordering are retained. Detaching local routing is not unsubscribe.

Both backends ignore unilateral string log frames without consuming a command slot or buffering log text, after cancellation and named-subscription routing. The published JSON client's fix closes the original classifier gap. The filesystem APIs do not enable daemon logs themselves.

The measured tradeoff favors choice, not a speed claim: at 100k files, bser-esm decode was ~1.8× slower than fatal UTF-8 + JSON.parse (31.1 vs 17.2 ms median), but its 4,200,349-byte frame was ~52% of JSON's 8,100,385 bytes (~48% fewer). These are decoder/captured-frame results, not a benchmark of the new integration or a host latency guarantee. See bench0 and bench1.

watchman-client-json@0.1.0 is now published to its GitHub family repository, consumed as github:rektide/watchman-client-json#main; no sibling checkout or npm package is required. The human approved the interface ledger and authorized publication and effect-watchman's main push on 2026-10-28. Versions remain unchanged. @superbfowle/bser-esm uses github:rektide/bser-esm; the Bun lock records the reviewed commits. See the implementation/sign-off receipt.

bun install --frozen-lockfile
bun run typecheck
bun run build
bun run test
# Opt-in, owned isolated foreground stock daemon (never ambient daemon):
WATCHMAN_TEST_BINARY=/usr/local/bin/watchman-facebook \
  WATCH_TEST_DIRECTORY=/home/rektide/tmp-opencode bun run test:daemon

Without WATCHMAN_TEST_BINARY, live tests skip. The live fixture covers public ordinary/enhanced watches, changes/deletions, scope/ignore selection, atomic save then later edit, sibling release, no initial inventory and read-only quietness. See HANDOFF.md for verified daemon version and remaining limits.

Observed inventory

Inventory.fileSystem exposes the scoped source installed by Glob.layer and Watchman.layerWithGlob: a clock-joined baseline, epoch-tagged live changes and explicit invalidation/rebaseline. Baseline entries have no array-order promise; membership, lexical metadata and epoch are authoritative. Changes retain their frame order and per-epoch sequence. Finite enumerate/glob remains sorted and deduplicated. See the Gate A API and receipt. The S1 materialization checkpoint records linear JSON framing, phase attribution and the remaining synchronous-work limit. The extracted JSON client now owns that framing; integration-side lexical/sort diagnostics remain local. The dual-backend receipt records shared semantic tests, BSER daemon coverage and the human-approved strictness/publication follow-up.