Skip to content

Add 'basou refresh --watch' to keep the corpus current - #45

Merged
takashi-matsuyama merged 1 commit into
mainfrom
cli/refresh-watch
Jun 7, 2026
Merged

takashi-matsuyama merged 1 commit into
mainfrom
cli/refresh-watch

Conversation

@takashi-matsuyama

Copy link
Copy Markdown
Member

Summary

Add basou refresh --watch [--interval <seconds>] so a workspace stays current
without re-running basou refresh by hand. Import is a pull, so the corpus only
advances when refresh runs; with no scheduler, it drifts stale. After multi-root
capture (#44) fixed which repos are scanned, this closes the currency half.

Behaviour

  • One catch-up refresh on start, then polls the native-log stores
    (~/.codex/sessions, ~/.claude/projects) every interval seconds (default
    30, min 5, max 86400).
  • A cycle re-imports + regenerates only when the logs have settled
    (unchanged since the previous poll, so nothing is captured mid-write) and
    changed since the last import.
  • Handoff / decisions regenerate only when something was actually imported, so
    AI work in unrelated projects never rewrites this workspace's files.
  • Polling is dependency-free and cross-platform; latency is the interval, not
    real-time. Ctrl-C / SIGTERM stops after the in-flight cycle.
  • --watch rejects --dry-run, --json, and --force.

Design (v1) and known limits

  • Per-file {mtime, size} scan drives both settle and change detection.
  • The baseline scan is taken before the initial import, so a session that
    appears during the import window is never mistaken for "already seen" and
    missed.
  • A pending-regenerate flag retries regeneration if an import succeeded but the
    regenerate failed (handoff / decisions can't be left stale forever).
  • Scan errors are surfaced pathlessly (per the repo's no-absolute-path-in-stderr
    contract); a root that is missing or not a directory contributes nothing.
  • The abortable sleep removes its listener on each normal poll (no leak over a
    long run).
  • Because import is idempotent, a session still active at start-up or one that
    resumes after it settled is recorded only up to that point; basou refresh --force rebuilds those. Event-driven watching (fs.watch / chokidar) and
    vendor session hooks are deferred.

Testing

  • pnpm -r build && pnpm typecheck && pnpm test && pnpm lint green
    (core 743, sdk 17, cli 441; lint 0 errors).
  • New tests: scanSourceLogs / scansEqual (collection, missing root, a
    file-as-root ENOTDIR, growth); the watch loop (settle — asserting a
    just-appeared log is NOT imported before it stabilises; no regeneration when a
    change imports nothing relevant; clean abort); parseInterval range
    validation; and --watch + --dry-run/--json/--force rejection.

Out of scope

Event-driven watching, vendor SessionEnd hooks, git hooks, per-source-root
tagging, and re-import of resumed sessions (the --force path covers the last).

Import is a pull, so the corpus only advances when 'basou refresh' runs.
After multi-root capture fixed which repos are scanned, the remaining gap
is currency: with no scheduler, the dogfood corpus drifts stale the moment
more AI work happens. --watch closes that gap.

'basou refresh --watch [--interval <seconds>]' runs a catch-up refresh on
start, then polls the native-log stores (~/.codex/sessions,
~/.claude/projects) and re-imports + regenerates only when they have
SETTLED (unchanged since the previous poll, so nothing is captured
mid-write) AND changed since the last import. Handoff / decisions
regenerate only when something was actually imported, so AI work in
unrelated projects never rewrites this workspace's files. Polling is
dependency-free and cross-platform; latency is the interval (default 30s,
min 5s, max 1 day), not real-time. Ctrl-C / SIGTERM stops after the
in-flight cycle, never mid-write.

Design (v1): per-file {mtime,size} scan for settle + change detection;
the baseline scan is taken before the initial import so a session that
appears during the import window is never mistaken for "already seen" and
missed; a pending-regenerate flag retries regeneration if an import
succeeded but the regenerate failed; scan errors are surfaced pathlessly
and a root that is missing or not a directory contributes nothing; the
abortable sleep leaves no listener behind across polls. Because import is
idempotent, a session still active at start-up or one that resumes after
it settled is recorded only up to that point -- 'basou refresh --force'
rebuilds those. Event-driven watching and vendor session hooks are
deferred.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@takashi-matsuyama
takashi-matsuyama merged commit 3f83bdb into main Jun 7, 2026
7 checks passed
@takashi-matsuyama
takashi-matsuyama deleted the cli/refresh-watch branch June 7, 2026 13:16
takashi-matsuyama added a commit that referenced this pull request Jun 8, 2026
Promote the Unreleased changelog to 0.7.0 and bump @basou/core,
@basou/cli, @basou/sdk, and the bare `basou` installer to 0.7.0 in
lockstep. Publishes two dogfood-proven features already on main plus
two folded-in fixes.

Features since 0.6.0:
- Multi-root capture: one .basou/ workspace can aggregate the native
  logs of several sibling repositories (#44).
- `basou refresh --watch`: keep the workspace current automatically by
  polling the native-log stores and re-importing only settled, changed
  sessions (#45).

Fixes:
- Reject --session combined with --all on import (both adapters).
- Drop a reference to a nonexistent `basou session export` from the
  session-import JSON Schema.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant