feat: wire auto-forget lifecycle GC; position folklore vs "LMs Need Sleep"#21
Merged
Conversation
…M Need Sleep"
Prompted by studying "Language Models Need Sleep" (Behrouz et al., arXiv
2606.03979). That paper consolidates ephemeral in-context knowledge into model
*weights* (low-rank experts + RL "dreaming"); it's weights-only and the authors
themselves frame external memory as the complementary half. Folklore is
non-parametric, so the mechanism doesn't transfer — but the framing validates
and sharpens folklore's position, and folklore's own graph-consolidation
machinery had inert pieces worth wiring.
Wire (the one that's clean + safe):
- Auto-forget lifecycle GC (TTL delete + frozen-band demote) was reachable only
via `folklore gc apply` (manual). Now runs on the daemon tick:
- src/daemon/auto-forget-tick.ts — cadence gate + in-process runner (no LLM,
so no detached child, unlike consolidate; mirrors the inline enforceRetention
pass already in the loop). Best-effort, local-only, dry_run safety valve.
- config: daemon.auto_forget.{enabled(false),interval_seconds,dry_run,
demote_band,min_age_days}.
- wired into src/daemon/loop.ts, chained before retention.
- tests/auto-forget-tick.test.ts (8 tests: gate + run + skip + dry-run).
Deliberately NOT wired here (deeper, riskier — scoped in the research note with
exact integration points): the write-time gate (must gate only observation
writes with a semantic-neighbor lookup, or it drops legit writes) and the
Beta-reliability/tier-promotion selector (a retrieval-ranker change).
Docs + site:
- WHITEPAPER.md §8: prior-art bullet positioning parametric consolidation as
complementary, sharpening folklore's moat (shared / frozen-model / signed).
- research/92-sleep-and-self-improvement.md: the paper's mechanism, why it
doesn't transfer, the one idea that does (selective consolidation ≈ our
"selectivity bounds compounding"), and the wiring status of every primitive.
- site/index.html: a new "even the graph sleeps" section — episodic→semantic
consolidation + forgetting, on-brand, verified no overflow at 375/390/1280.
The website download should install everything with no command, on macOS,
Windows, and Linux. The engine (CLI/daemon/provider-integration) is already
cross-platform Node; what was missing is a GUI shell + installers. This adds a
Tauri v2 app (one Rust+webview codebase → 3 OSes) as the foundation.
- client/desktop/src-tauri (Rust): tray icon + menu (Open / Run Setup / Start /
Stop / Quit) and commands that drive the folklore CLI — run_onboard,
harness_list, daemon_{status,start,stop}. GUI apps on macOS don't inherit the
shell PATH, so the CLI is resolved by probing global-install locations, then
PATH, then `npx --yes @usefolklore/folklore`.
- client/desktop/src: a brand-styled onboard wizard — one "Install everything"
button that runs `folklore onboard --yes` (identity + daemon + every detected
AI provider), and a live provider list from `harness list`. Verified running:
the macOS build (Folklore.app + .dmg) renders the wizard and detected Claude
Desktop.
- .github/workflows/desktop-release.yml: tauri-action matrix building
.dmg (universal) / .msi+.exe / .AppImage+.deb on a `desktop-v*` tag.
- App icons generated from the folklore favicon.
Follow-ups (honest scope): bundle the Node CLI as a self-contained sidecar so it
works with zero Node on the machine (today the wizard needs Node for the npx
fallback); wire the website download to the per-OS installers; signing/
notarization; optionally reuse the island HTML as a tray popover. This replaces
client/menubar-macos (Swift) once the Tauri app reaches parity.
Deploying folklore with
|
| Latest commit: |
30a8bcc
|
| Status: | ✅ Deploy successful! |
| Preview URL: | https://39aff988.folklore-3pb.pages.dev |
| Branch Preview URL: | https://feat-self-improvement-sleep.folklore-3pb.pages.dev |
…ew cask, signing CI Follows up the Tauri app foundation. Single-file bundling is off the table — the CLI has native addons (better-sqlite3, sqlite-vec, onnx via @xenova/transformers, tree-sitter, sharp), so no SEA/compile can produce one binary. The pragmatic path (folklore's users already run Node) is to make the GUI app find Node reliably, and that's the real fix here. - lib.rs: a double-clicked GUI app inherits neither the shell PATH nor version-manager shims. CLI/npx resolution now probes nvm / fnm / volta / bun / global-npm-prefix dirs plus /opt/homebrew and /usr/local, and returns a clear "install Node" error instead of silently failing when nothing is found. - site: the download is now the desktop app and is OS-aware — JS detects the visitor's OS, resolves the newest `desktop-v*` GitHub release, and points the primary button at that OS's installer (.dmg / .msi / .AppImage), with the other two offered as links. Degrades to the releases page if no desktop release exists yet. npm + Homebrew shown as the CLI alternatives. Framing shifts from "install the CLI first" to "download installs everything, no terminal". - client/desktop/folklore.rb: Homebrew cask for the Tauri .dmg, superseding the old Swift menubar cask. - desktop-release.yml: Apple signing + notarization env wired (conditional on repo secrets) so the Developer-ID phase is a secret-drop, not a code change.
Passing an empty APPLE_CERTIFICATE made tauri-action attempt a codesign import with invalid data and fail the macOS job (Windows/Linux built fine — they don't sign). Split into signed/unsigned build steps gated on a job-level SIGN_MAC boolean (secrets.* is valid in job env, not a step if), so macOS builds unsigned until the Developer-ID secrets are dropped in, then notarizes with no code change.
Enterprise deploys and winget prefer .msi. tauri filters targets by platform, so this only adds the WiX .msi to the Windows build; macOS/Linux unaffected.
… installed Delivers the actual "download installs everything, no command" promise. Single- file bundling is impossible (the CLI has native addons), so instead the app carries a clean Node dist and runs the folklore CLI (via npx) against it. - lib.rs: resolve the app's bundled Node (resources/node) and prepend its bin dir to the child PATH for every CLI spawn, so bare `npx`/`node` resolve to the bundle. Falls back to system Node when the bundle is absent (dev builds). CLI spawns now thread the AppHandle through to reach resource_dir. - tauri.conf: bundle resources/node as the app's `node` resource. - resources/node: tracked README only; the real per-platform Node payload is git-ignored and assembled in CI. - desktop-release.yml: per-platform Node fetch before build — macOS gets a universal (lipo arm64+x86_64) node, Windows the win-x64 dist, Linux linux-x64 (Node 22.11 LTS). Each verified with `node --version`. Verified: local macOS build bundles the (README-only) node dir and still builds; with no bundled payload the app falls back to system Node exactly as before.
Single source of truth: the .sh-hearth mark from the hero section (site/index.html) — the dark card, ember glow, gradient flame, two tellers, crossed logs. Extracted verbatim to site/assets/folklore-hero.svg, then every icon regenerated from it, per request (only the hero): - Favicon (folklore-favicon.svg) → the hero hearth. - Navbar brand (site header) → the hero hearth (prefixed gradient IDs so it doesn't collide with the hero instance on the same page). - Social/OG image (assets/gen/og.png, 1200×630) → hero mark + Fraunces wordmark + "knowledge travels mouth to ear" + tagline. - Desktop app + installer icons (client/desktop/src-tauri/icons/*) → regenerated from a 1024px render of the hero mark via `tauri icon` (macOS .icns, Windows .ico, all PNG sizes). - Desktop wizard leading mark → the hero hearth.
Verified the shipped .dmg end-to-end and found the bundled npx/npm bin symlinks get flattened by the app bundler into mis-located script copies, so npx fails with "Cannot find module '../lib/cli.js'". The bundled node binary is fine. Reworked the bundled-Node path to the form proven against the real bundle with a stripped PATH (no system Node): bundled node + npm-cli.js `install -g @usefolklore/folklore --prefix <app-data>/cli` once (~20s, fetching the platform's native prebuilds), then run the installed entry bin/folklore.js directly with the bundled node — bypassing the unreliable bin symlinks. Handles unix (lib/node_modules) and Windows (node_modules) layouts. Resolution order: system folklore → bundled Node + installed CLI → system npx → clear error. Confirmed `harness list` detects providers with zero system Node on PATH.
…bar app Brings the Dynamic Island to the cross-platform Tauri app, so it can replace client/menubar-macos. Left-click the tray icon → a borderless, transparent, always-on-top island popover at the top-center of the screen (the notch spot on a MacBook; a floating pill elsewhere); the menu still opens on right-click, and a "Peer Activity" item toggles it too. - lib.rs: `recent_activity(n)` reads the last n lines of `$FOLKLORE_HOME/activity-feed.jsonl` (the same feed the Swift island tailed); `toggle_island` shows/hides + centers the popover; tray left-click wired via on_tray_icon_event. - island.html: the redesigned near-black island (hero logo tile, SF Pro, humanized peer, one hero metric, per-direction accent) polling recent_activity every 1.5s and rendering the latest pull/serve event; click to dismiss. - tauri.conf: second `island` window (hidden, borderless, transparent, alwaysOnTop, skipTaskbar). Verified: the UI renders a sample pull event correctly (peer humanized to peer-trjm, "+638 tokens"); the two-window app builds (Folklore.app + .dmg).
Icon-consistency sweep per the 'everywhere = hero' directive: research had an inline data-URI of the old simple flame; whitepaper had no favicon/OG at all. Both now use /assets/folklore-favicon.svg (the hero hearth) and the hero social image (assets/gen/og.png).
Completes the 'everywhere = hero' icon sweep — 12 pages carried the old inline data-URI flame, 3 had no favicon. All 15 now use /assets/folklore-favicon.svg (the hero hearth). Every site page is icon-consistent with the hero.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Studied Language Models Need Sleep: Learning to Self-Modify and Consolidate Memories (Behrouz, Hashemi, Javanmard, Mirrokni — Google Research + Cornell, arXiv 2606.03979) and acted on it three ways.
Verdict on the paper
Its mechanism is weights-only — Knowledge Seeding distills into new low-rank experts; Dreaming RL-fine-tunes on self-generated data. Both need backprop. Folklore is non-parametric, so the mechanism cannot and should not be adopted. The paper itself frames external memory as the complementary half a self-modifying model still needs. That sharpens folklore's moat rather than threatening it: weight consolidation is single-brain, impossible on frozen API models, and unattributable — folklore is the shared, frozen-model-compatible, signed substrate for exactly those gaps.
What this PR does
Wires the one inert primitive that's clean + safe: auto-forget lifecycle GC.
Previously reachable only via
folklore gc apply(manual). Now runs on the daemon tick:src/daemon/auto-forget-tick.ts— cadence gate + in-process runner (no LLM → no detached child, unlike consolidate; mirrors the inlineenforceRetentionpass already in the loop). Best-effort, local-only,dry_runsafety valve.daemon.auto_forget.{enabled(false default), interval_seconds, dry_run, demote_band, min_age_days}.src/daemon/loop.tsbefore retention.tests/auto-forget-tick.test.ts— 8 tests (gate, run+persist, skip, dry-run). Full suite + typecheck green.Deliberately does NOT wire the write-time gate or the Beta-reliability/tier-promotion selector. These are deeper and riskier — the write-gate must gate only observation writes with a semantic-neighbor lookup or it drops legitimate writes; the reliability selector is a retrieval-ranker change. Rushing them into a working codebase is the wrong trade. Scoped with exact integration points in the research note, for a focused follow-up.
Docs + site:
docs/WHITEPAPER.md§8 — prior-art bullet positioning parametric consolidation as complementary.research/92-sleep-and-self-improvement.md— the mechanism, why it doesn't transfer, the one idea that does (selective consolidation ≈ our "selectivity, not volume, bounds compounding"), and the wiring status of every self-improvement primitive.site/index.html— a new "even the graph sleeps" section (episodic→semantic + forgetting). Verified no horizontal overflow at 375 / 390 / 1280.Note
Site deploys from
security/audit-and-fixes, not main — merging here previews the section; it needs routing to that branch to go live (same open question as #19/#20).