deepagents 0.6.12 parity + drop-in, security fixes, and a world-class CLI pass#160
Merged
Conversation
…eware and backends Verified hardening sweep (all changes covered by new/extended unit tests; ruff + format + ty clean; 1693 unit tests pass). Correctness / reliability: - checkpointing: read the singular `ToolCallRequest.tool_call` (langgraph 1.2.x) instead of a non-existent `.tool_calls` list, which made the hasattr guard always false — checkpoints were never created, so /undo and /rewind had nothing to revert to. - graph: de-duplicate feature-wired middleware (Skills/Memory/PromptCaching) against user-supplied `middleware=` and add a keep-first backstop so a feature passed via both a kwarg and an explicit instance no longer raises langchain's opaque "duplicate middleware instances" AssertionError. - worktree: split spawn_parallel_tasks into sync/async tools and register a coroutine so the async agent path no longer crashes on `asyncio.ensure_future`; surface git failures from create_worktree instead of returning a WorktreeInfo for a directory that was never created. Resiliency (fail-open where it should degrade, not crash): - memory: a non-`file_not_found` backend error (permission/timeout/transient) now degrades that source with a warning instead of aborting the turn. - offline_mode: stop leaking a process-global socket default timeout onto every later untimed socket; use per-socket timeouts with save/restore. - multi_edit / read_many_files / filesystem / store / composite: isolate per-item failures (partial-success contract), bound output/limit/file size to avoid OOM/hangs, and log previously-swallowed state-sync errors. - context_packing / intelligent_compaction: pack via `request.override(...)` instead of mutating the shared request in place, snap the keep boundary so a tool_use/tool_result pair is never split, and fail open on error. - audit_trail: record model-call failures so a 5xx/timeout doesn't leave a dangling, unresolved llm_call in the compliance record. - hybrid_search: error boundary around embedding-cache save. - notifications: clipboard subprocess gets a timeout + context manager. Stability (Windows non-en-US / encoding): - repo_map / local_sandbox: read/write with encoding="utf-8" so the symbol index and seatbelt profile don't decode through cp1252/cp932/cp949. - code_intelligence: prune vendored/build dirs and cap files scanned. Docs: - graph docstring no longer advertises a SafeToolsMiddleware class that does not exist (safe-tool auto-approval is data-driven via SafeToolsConfig).
…sessions, ops) Verified hardening sweep (new/extended unit tests; ruff + format + ty clean; 4538 unit tests pass). TUI responsiveness / correctness: - app, widgets/chat_input: read the system clipboard on a background thread (paste + right-click paste) so a slow/hung helper can't freeze the event loop; sync fallback so a paste never crashes the app. - app: hide the spinner on an empty status string (not just None) so slash commands stop leaving a blank "esc to interrupt" animation; drain the pending-message queue after inline /compact and /butcher so messages typed during them aren't perceived as dropped. Auth / secrets / resiliency: - bedrock_refresh: serialize the refresh critical section so concurrent expired-creds calls can't spawn multiple `aws sso login` subprocesses. - vars_store: write plaintext vars with mode=0o600 before the atomic rename (no world-readable window) and leave a one-time breadcrumb when the keyring probe fails and secrets silently downgrade to TOML. - remote_sandbox: add ssh ConnectTimeout + BatchMode and an overall asyncio.wait_for ceiling so a wedged ssh can't hang the CLI forever. - server: atexit fallback to stop an orphaned langgraph dev subprocess. - non_interactive: cancel-and-drain the MCP preload task on every early return so it can't leak coroutines/subprocesses. Observability / robustness: - main: handle malformed --pipeline YAML (and non-dict / bare-string steps) with a clear error instead of an uncaught traceback. - causal/ledger: track and expose disk-append degradation instead of silently swallowing the OSError. - compliance: add a `no_event_with_payload` collector so audit packs can assert on rule verdicts (e.g. no rule_fire with action=deny). - jobs_manager: skip unreadable job files (OSError) instead of crashing PersistentJobsManager construction. Stability (encoding / atomic writes): - mentions, agent, teach, pipeline, butcher: encoding="utf-8" on user/model text I/O and subprocess decode. - auto_memory, session_fork: switch to atomic_write_text so a crash mid-write can't corrupt memory/fork files. - clipboard: timeout on the Windows clipboard subprocesses.
…, lint)
Verified hardening sweep (new tests in tests/test_daemon_hardening.py; ruff
clean with PLW1514 now enabled; 130 tests pass).
Reliability / security:
- api: reject a git-push webhook with a missing/empty `ref` instead of
letting branch="" match `fnmatch("", "*")` and fire every wildcard
GIT_PUSH job on a malformed payload.
- main: write the daemon auth token atomically with owner-only perms before
the rename (no default-perm window, no truncated token on a crashed
write), and regenerate an empty/blank token file instead of persisting a
value that would reject every request.
Reliability / observability:
- scheduler: prune heavy dirs (.git/node_modules/.venv/build caches) in place
and cap files examined per tick so a large watch_dir can't make one
file-change tick overrun the interval and starve cron/interval jobs.
- runner: log a warning (with the skill path) when SKILL.md frontmatter is
invalid YAML or not a mapping, instead of silently dropping `chain:`.
Tooling:
- pyproject: enable ruff PLW1514 (unspecified-encoding) via the preview +
explicit-preview-rules gate, per the CLAUDE.md encoding mandate.
Implements reviewed propose-batch items P2/P3/P16/P22/P23/P25/P33. Each was a
feature that silently did nothing (fail-open) or an unbounded server path; now
they actually enforce/route, with negative tests. All opt-in (not in the
default stack); ruff+format+ty clean; 1755 SDK unit tests pass.
- smart_approvals (P2): SmartApprovalsMiddleware was a pure pass-through —
evaluate()/record_human_decision() had zero callers, so every tool ran
unimpeded. Add wrap_tool_call/awrap_tool_call that evaluate the call and
FAIL CLOSED: a denied/escalated tool is never executed; a denial ToolMessage
is returned instead.
- air_gapped (P3): the egress policy was a prompt suggestion only. Add a
best-effort wrap_tool_call egress gate that intercepts known egress tools
(web_fetch/fetch_url/http_request and networked shell commands), resolves
the target host, consults AirGapStore.check_allowed, and denies (fail
closed). Docstring now states the best-effort scope honestly.
- thinking (P22): _detect_provider used startswith("claude"), so Bedrock
inference-profile ids (us.anthropic.claude-*, eu./apac.) classified as
'unknown' and native thinking was silently disabled for ALL Bedrock Claude
users. Match 'claude' as a substring; keep gemini/openai prefix-anchored.
- model_cascade (P23): awrap_model_call was a pass-through, so the advertised
cost-aware routing never altered the model. Implement downshift-only routing
(route() -> request.override(model=...)) with a sync hook; fall through on
any uncertainty so a turn is never blocked.
- adaptive_context (P25): only an async pass-through existed; the tier config
was dead. Wire sync+async wrap_model_call to truncate oversized tool outputs
(override-not-mutate, message count/order preserved) and log the summarize
threshold; fail open.
- serve (P16/P33): _check_api_key now uses secrets.compare_digest
(constant-time); per-request ainvoke/astream are bounded by
request_timeout (504 on timeout) and gated by a lazily-created
Semaphore(max_concurrent_requests); the _threads map is an LRU OrderedDict
bounded by a new ServerConfig.max_tracked_threads (no unbounded growth).
…dreamscape laws Implements reviewed propose-batch items P4/P5/P24/P1. ruff+format+ty clean; CLI suite 4587 pass; dreamscape opt-in/no-op-when-disabled invariant preserved. Security: - SSRF (P4/P5): /web fetch_url, the agent fetch_url tool, and http_request had no SSRF guard — loopback/link-local/private IPs (incl. 169.254.169.254 cloud metadata) were reachable and post-redirect URLs were never re-validated. Add a shared assert_fetch_allowed guard (http/https only; reject any address that resolves to private/loopback/link-local/reserved/multicast/unspecified, unwrapping IPv4-mapped/6to4 IPv6 to stop tunneling) and follow redirects manually, re-validating every hop. Fail closed. Kept the existing unicode gate (check_url_safety) separate and unchanged. (Residual: urllib/requests re-resolve DNS at connect time, so a determined DNS-rebinding attack is not fully closed — common SSRF vectors are.) - TraceFile signature (P24): the Ed25519 signature covered only merkle_root + session_id, so producer/actor/notes could be forged on a file that still rendered as "verified". Bind a canonical header digest into the signed message under a new sign_format=2 (default for new files); v1 files stay byte-identical and still verify; unknown formats fail closed. Downgrade (flip to v1) also fails since the signature was computed over the v2 message. Correctness: - dreamscape laws/imagination (P1): _response_text read getattr(response, "content") which is always "" on the live langchain ModelResponse (output lives in .result), so Hard-Law enforcement and the imagination failure trigger were INERT (fail-open on a governance guard). Read .result (with a bare-AIMessage .content fallback) via one shared helper; rebuild a refusal via dataclasses.replace(response, result=[AIMessage(...)]). No other dreamscape behavior changed.
…unattended shell Implements reviewed propose-batch items P6/P7. daemon suite 141 pass; ruff (incl. PLW1514) clean. - P6: the manual-run, git-push, and webhook endpoints each did asyncio.create_task(run_job(...)) directly, bypassing the scheduler's _running_jobs overlap guard and BOG_DAEMON_MAX_CONCURRENT_JOBS semaphore — so a push/webhook storm could fan out unbounded parallel runs of one job against a single workspace/history. Add DaemonScheduler.dispatch(): reserves the slot synchronously (skip-if-running -> SKIPPED), then runs under _run_job_safely (semaphore-bounded). All three endpoints route through it; the manual endpoint still returns its placeholder run immediately. - P7 (V3-11): the unattended runner built LocalShellBackend(virtual_mode=False) with no HITL on triggers that include the externally-reachable, HMAC-only webhook path. Non-MANUAL triggers now default to virtual_mode=True (shell/fs confined to working_dir); MANUAL (token-authenticated) keeps unrestricted real-path shell. Operators opt into unrestricted unattended shell with BOG_DAEMON_ALLOW_UNATTENDED_SHELL=1, which logs a WARNING when a network-reachable trigger then runs with guardrails disabled.
…writes, DLP, worktree) Implements reviewed propose-batch items P9/P10/P11/P12/P14/P15/P17/P18/P19/P26. ruff+format+ty clean; 1806 SDK unit tests pass (the P11 xfail now passes). Data-integrity: - multi_edit (P9): chained edits to the SAME file silently dropped earlier edits — StateBackend.edit re-read the original and the dict-keyed update map overwrote. Thread the working copy forward via a new keyword-only base_content arg (BackendProtocol/StateBackend.edit) so edit #2 sees edit #1; the async protocol path forwards base_content only when supplied so backends that don't accept it (FilesystemBackend) keep working. (P10) set status='error' on the ToolMessage when any edit in the batch fails. - filesystem backend (P12): edit() and upload_files() truncated the target in place, so an interrupt/ENOSPC mid-write could empty the original. Write to a sibling temp (O_NOFOLLOW|O_CREAT|O_EXCL) + fsync + atomic os.replace via a new _atomic_write helper that also refuses to write through a symlink. - filesystem middleware (P11): multiple write-class tool_calls (write_file/ edit_file/multi_edit_file) on the SAME file in one AIMessage raced and the last-writer-wins reducer clobbered the rest. Add an after_model guard that keeps the first such call per file and returns an error ToolMessage for the rest, telling the model to sequence them. Distinct files / reads untouched. (Flips the long-standing xfail.) - DLP (P14): redact mode mutated shared LangGraph message content in place, corrupting canonical history. Redact a per-call copy and forward via request.override — canonical state is preserved (StreetSweeper invariant). Concurrency / correctness: - worktree (P18): a cancelled task was resurrected to 'completed' in finally — add a terminal-status guard + a 'cancelled' state + best-effort handle.cancel. (P19) per-task repo_root captured at registration instead of mutating the shared self._working_dir, so concurrent tasks don't race onto the wrong root. (P26) merge_with_conflict_report restores the caller's branch on every non-success path via try/finally. - expert engine (P17): assert_fact-derived facts grew unbounded for the engine lifetime. Add a keyword-only soft cap (default 5000) that warns once and FIFO-evicts the oldest DERIVED facts (never structural tool_call facts). - graph (P15): enabling result-synthesis while passing an explicit ParallelWorktreeMiddleware no longer discards the user's instance — search user middleware too and reuse it (kept before ResultSynthesis so requires= ordering holds). Note: P13 (StoreBackend legacy-namespace route-isolation bleed) is intentionally left as an xfail — the safe fix needs a CompositeBackend route discriminator and a data-migration decision; deferred rather than risk silent data loss.
…t race, fork lock, oauth state) Implements reviewed propose-batch items P20/P21/P27/P28/P29/P30/P32. ruff+format+ty clean; CLI suite 4602 pass. - orchestrator (P20/P21): parallel subtasks ran via run_coroutine_threadsafe against the caller's loop (hard-deadlock if ever entered from the loop thread) on the shared default executor (a timeout abandoned the worker threads). Rewrite onto a dedicated, bounded ThreadPoolExecutor owned by the call — no asyncio, so it's deadlock-free from any thread — with a per-subtask wall-clock budget and shutdown(cancel_futures=True) on timeout so abandoned work can't saturate the default executor. - app.py (P27): auto-commit ran AFTER the queue drain (which can spawn the next turn's worker), so it could stage a half-written tree from turn N+1. Finalize stats + auto-commit BEFORE the drain. (P28) move the critical worker-cleanup restoration (_agent_running/_agent_worker reset, input-cursor re-enable) into a finally that a cancelled inner await can't skip, so a cancellation can no longer leave the input wedged with no diagnostic. - session_fork (P30): _save_fork read-modify-wrote forks/<parent>.json with no lock, losing a record when two CLI processes fork the same parent. Serialize with an in-process lock + a cross-process O_EXCL lock file (filelock isn't a direct dep) and atomic_write_text. - dreamscape scheduler (P29): stop() left the slow completion-callback tasks running (asyncio teardown "Task destroyed but pending" + abandoned proposer write). Snapshot, cancel, and await them (bounded). - oauth_mcp (P32): exchange_code_for_token never validated the OAuth state (CSRF/code-injection). Add keyword-only expected_state/received_state with a constant-time check (additive — opt-in until the callback flow is wired).
Ports the deepagents 0.5.0+ backend type layer that bog forked before: Result dataclasses (Ls/Read/Grep/Glob/Delete/ExecuteOffload), FileData v2 (str content + encoding), delete/adelete on the protocol, and ~16 helpers in backends/utils.py. Legacy names (read -> str, ls_info, grep_raw, glob_info) are preserved as delegating shims with override-detection forwarding in both directions, so no existing backend or middleware breaks. create_file_data still defaults to file_format=v1; the backends wave flips it. Also fixes two live bugs in utils.py: grep iterated characters instead of lines on v2 content, and a missing modified_at key raised KeyError. Raises dependency floors to be co-installable with deepagents (wcmatch was capped <7.0 against upstream's >=10.1, making the two packages unresolvable in one venv). langchain 1.3.12 / langchain-core 1.4.9 / langchain-anthropic 1.4.8. Glob/permission semantics verified unchanged across wcmatch 6 -> 10. Adds bog_agents/_api/deprecation.py and middleware/_private_state.py. Refs PARITY.md waves 1A, 1C.
…overwrite) Finishes the Wave 1C backend rewrite bog forked before (deepagents 0.5.0). - FileData v2 is now the default (create_file_data -> content: str + encoding). Readers tolerate legacy list[str] with a DeprecationWarning; file_format='v1' remains an explicit escape hatch. StoreBackend no longer rejects v2 items as corrupt. - delete/adelete across the protocol and all backends, incl. recursive directory delete (StateBackend removes prefixed children without eating sibling keys). - write() now OVERWRITES an existing path on all six backends, matching upstream. Previously StateBackend/StoreBackend/FilesystemBackend errored. This is a deliberate, user-visible semantic change: a CompositeBackend mixing a FilesystemBackend default with a StateBackend at /memories would otherwise accept or reject the same write based only on the routed prefix. - StateBackend()/StoreBackend(store=...) construct without a runtime; StateBackend gets read-your-writes via CONFIG_KEY_READ/SEND; StoreBackend accepts both the old (BackendContext) and new (Runtime) NamespaceFactory generations. - New backends: LangSmithSandbox (SDK-native read/write, CRLF normalization, capture offload) and ContextHubBackend (versioned hub-backed filesystem). Both lazily import langsmith so lazy-import health holds. - BaseSandbox gains capture-offload, ARG_MAX-safe edit-via-upload, binary read, and native async. Real bugs fixed along the way: - grep iterated characters instead of lines on v2 content - _python_search: '*.py' never matched nested files, and Windows backslash relative paths never matched any glob - CompositeBackend.glob leaked entries from other routes when scoped to a path - create_file_data silently dropped encoding= under v1, corrupting binary as text - a missing modified_at key raised KeyError +302 tests, including a forwarding-matrix suite pinning the legacy<->new backend shims that were previously unpinned. Refs PARITY.md wave 1C.
Wave 1B: brings the middleware layer to deepagents 0.6.12 parity and closes two confirmed permission bypasses. SECURITY: - Filesystem deny rules were checked only against a tool call's path argument, but ls/glob/grep don't require one -- so `grep(pattern="API_KEY")` with no path returned matches (and file contents) from a denied subtree. Deny rules now filter ls/glob/grep *results*; interrupt-mode entries pass through unfiltered (interrupt means ask-on-access, not hide). Wired into all three FilesystemMiddleware construction sites in graph.py -- the filter functions existed but were never installed on the real create_agent path -- and pinned with an end-to-end regression test driving create_agent(permissions=...). - The symlinked-skill-directory refusal existed only on the sync _list_skills path; the async _alist_skills path (used by ainvoke) had no check, so running the agent async bypassed containment entirely. Both paths now route through a single _filter_skill_dirs chokepoint. - Recursive delete of / or /** could blow away a denied subtree; delete is now gated by _find_delete_deny_patterns and refused rather than partially run. Interop surface: - SystemPrompt/subagent: SUBAGENT_RESPONSE_FORMAT_CONFIG_KEY (SubAgent.response_format was silently dropped), public create_sub_agent, state_schema + private-key forwarding so rubric/summarization/skills bookkeeping no longer leaks across the subagent boundary, LangSmith subagent tracing context. - New delete filesystem tool (7th FS tool), FsToolName superset, keyword-only FilesystemMiddleware params (tools/_permissions/human_message_token_limit_before_evict/ custom_tool_descriptions). - Skills: (path, label) tuple sources, system_prompt param, skills_load_errors state + prompt-injection-guarded warning block, YAML-list allowed-tools. - Memory: add_cache_control/system_prompt params, HTML-comment stripping with split-tag defense, Trust-and-verification prompt block. - Summarization: overflow-clip (breaks the same-oversized-tail retry wedge), inline-media offload, widened create_summarization_middleware; eviction helpers extracted to _message_eviction.py (filesystem.py delegates). - patch_tool_calls now patches invalid_tool_calls (dangling tool_use blocks were rejected by the provider) and returns early when nothing dangles. - 18 previously-unreachable symbols registered in middleware/_LAZY_IMPORTS. Fixes the stale v1 FileData TypedDict in middleware/filesystem.py (re-points at backends.FileData). Flips the tests that asserted these upstream bugs. Refs PARITY.md waves 1B, 3.
…profiles) Wave 1A: makes bog-agents a source-level drop-in for the deepagents public API. - SystemPromptConfig: create_agent/create_deep_agent now accept the prefix/base/ suffix dict form. base=None drops the built-in base prompt; an omitted base key keeps the profile/default base; suffix lands after base, before the profile suffix. Previously a dict system_prompt raised TypeError -- the headline interop break. The provenance addendum is assembled as its own part so base=None does not silently un-prompt bound provenance tools. - Every symbol in deepagents' __all__ now imports from both bog_agents and bog_agents.deepagents (adds SystemPromptConfig, FsToolName, plus create_sub_agent and SUBAGENT_RESPONSE_FORMAT_CONFIG_KEY). create_deep_agent gains keyword-only max_turns=9999 and widened system_prompt/subagents/response_format annotations to mirror upstream. - recursion_limit is no longer hard-clamped at 1000 (create_agent's native max_turns default cap is unchanged; only the deepagents shim defaults to 9999). LangSmith metadata key 'versions' -> 'lc_versions'. - User middleware whose .name collides with a built-in now REPLACES it in place (was silently dropped by keep-first dedup). - GeneralPurposeSubagentProfile(enabled=False) now actually removes the task tool and SubAgentMiddleware (was a typed no-op). Profile tool_description_overrides reach the built-in tools, and subagents resolve their own per-model harness profile instead of inheriting the parent's. - _models.py: model_matches_spec now compares the provider (openai:gpt-5 no longer matches an Anthropic model named gpt-5); adds is_bedrock_model; the hardcoded use_responses_api moved below the provider-profile layer so a profile can now override it (was a dead override). Flips the two tests that asserted the recursion-limit and metadata-key bugs; adds a drop-in compat test covering all 20 upstream public symbols. Refs PARITY.md wave 1A.
bog_agents/base_prompt.md was a static copy of the system prompt that nothing loaded -- the live base prompt is the BASE_AGENT_PROMPT constant in graph.py. It had already drifted (its first line reads 'You are a Bog Agents'). Delete it and add a regression guard so it can't reappear as a second source of truth. Refs PARITY.md wave 3 (orphan-base-prompt-md).
Wave 2. The profile registry existed but shipped zero built-in profiles
('This port ships none'), so every model ran with untuned prompts and three
providers were actively broken. Ports the deepagents 0.6.12 profiles:
Harness profiles (system-prompt tuning per model):
- anthropic:claude-opus-4-7 / sonnet-4-6 / haiku-4-5 (parallel-tool-call,
investigate-before-answering, subagent-usage guidance)
- openai:gpt-5.{1,2,3}-codex (Codex emits preambles and stops early without it)
- nvidia nemotron-3-ultra (8 keys) + a text-form tool-call parser
(<function=NAME>, <function><name>, bare JSON) -- Nemotron was unusable as an
agent because nothing parsed its tool calls
Provider profiles:
- openrouter: injects openrouter_provider={'ignore': ['azure']} so reasoning
models don't route to Azure's stateless /responses beta and break multi-turn
('Item with id rs_... not found'). Honors BOG_AGENTS_OPENROUTER_ALLOW_AZURE
(and the legacy DEEPAGENTS_ name) as an escape hatch.
- openai: use_responses_api=True (now the source of truth; the Wave 1A bridge
in _models.py no-ops when the profile sets it)
- nvidia: X-BILLING-INVOKE-ORIGIN header
All six register in the _builtin_profiles.py bootstrap, and the loader now also
enumerates the legacy deepagents.{harness,provider}_profiles entry-point groups
so third-party deepagents profile plugins load on bog (native wins on collision).
_harness_profile_for_model normalizes the provider so pre-built model instances
resolve their profile.
Also:
- Bedrock prompt caching: a Bedrock model now pays cached input-token price
instead of full price every turn. Appended after Anthropic caching, gated on
is_bedrock_model; langchain_aws stays optional (graceful no-op when absent).
- Video frame read: read_file on a video now samples frames (JPEG blocks +
window header) when [video] deps are present, falling back to a hint when
absent -- wired into both sync and async read paths. New [video] extra.
Refs PARITY.md wave 2.
Refs PARITY.md.
Ports deepagents_code's goal/rubric concept into the SDK. GoalToolsMiddleware keeps a persistent objective + acceptance-criteria (rubric) in the agent's view, re-injected via wrap_model_call each turn, with get_goal/get_rubric/update_goal tools. Completion via update_goal(status='complete') reuses RubricMiddleware's existing grader verdict rather than forking a grader. Goal state channels are PrivateStateAttr-marked so they don't leak into the public I/O schema. Registered lazily in middleware/_LAZY_IMPORTS; not in the default stack, so canonical middleware ordering is unchanged. Refs CLI_AUDIT.md (goal-tools-persistent).
…ache bugs Closes two markup-injection bugs and two correctness bugs found in the CLI audit. SECURITY: - The approval dialog (the primary human-trust surface) and the MCP viewer rendered attacker-influenceable strings (tool name/description, server-supplied MCP metadata) as unescaped Rich markup. A value like [bold red]..[/bold red] could restyle/hide the exact text the human approves; a bare [/dim] raised MarkupError and crashed the HITL dialog outright. Both now route untrusted text through escape_for_display (control-char + bidi/zero-width strip + Rich escape). - escape_for_display now escapes markup LAST: escaping first let the lone-ESC sweep consume the inserted backslash for an ESC-adjacent tag, re-exposing a live tag. Also sanitize ANSI/OSC/DCS/C0/C1 control sequences at the tool-display and local-context sinks so a malicious tool result can't emit OSC 52 to hijack the clipboard. CORRECTNESS: - Headless HITL: a malformed interrupt recorded a fail-closed reject but never set interrupt_occurred, so the reject was wiped by the next resume iteration and the unattended run wedged. Rejects now stage in StreamState.malformed_rejects and are delivered exactly once in the resume Command. - git-branch cache was memoized per-cwd forever, so checkpoints after a mid-session branch switch were tagged with the stale branch. Now a short TTL + explicit invalidation in the /branch handlers. Adds an app.py line-count ratchet test (ceiling 17,300) so new surface lands in commands/*.py + controllers rather than the god class. Saves the CLI audit as CLI_AUDIT.md. Refs CLI_AUDIT.md (SEC-1, SEC-2, HITL, git-branch-cache, ARCH-1).
…tion
- Managed ripgrep: on a fresh box with no rg on PATH, download a SHA-256-pinned
ripgrep 14.1.1 into ~/.bog-agents/bin and prepend it, so grep/search runs at
full speed instead of the slow Python fallback. Gated behind [tools].auto_install
(default true) + an offline/air-gapped short-circuit; a checksum mismatch never
installs. Keeps the existing manual-install warning as the fallback. doctor now
reports managed-vs-system rg.
- ${VAR} MCP header interpolation: an .mcp.json header value of
'Authorization: Bearer ${GITHUB_TOKEN}' now resolves from the vault then env
(with ${VAR:-default}), raising an actionable error when unset -- so tokens
aren't committed in plaintext.
Regenerates the cli/daemon lockfiles for the SDK dependency-floor bump.
Refs CLI_AUDIT.md (managed-ripgrep, mcp-header-env-interpolation).
… /rubric - Native reasoning-effort: /effort now maps to each provider's real reasoning knob instead of a max_tokens/temperature hack. The old mapping capped a reasoning model's output at 1024 tokens on 'low' -- actively truncating it. reasoning_effort.py translates the closed effort vocabulary onto Anthropic output_config.effort (+ adaptive thinking), OpenAI reasoning.effort, Gemini thinking_level, and Fireworks/xAI reasoning_effort, with per-model capability detection so only supported levels are offered; non-reasoning models keep the max_tokens/temperature fallback. - External-editor compose: Ctrl+X pops the input buffer into $VISUAL/$EDITOR (priority binding, since TextArea binds Ctrl+X to cut) and drops the edited result back. utf-8 round-trip, temp file always cleaned up. - /goal and /rubric: wires the SDK GoalToolsMiddleware into the CLI agent so get_goal/get_rubric/update_goal reach the model, backed by a file-persisted goal record. /rubric draft|regenerate|accept drafts acceptance criteria from the objective (one-shot LLM with a regenerate-on-feedback gate). Headless 'goal' twin included. Logic lives in reasoning_effort.py / editor.py / goal_controller.py / goal_rubric.py (testable without the TUI); app.py handlers stay thin. Ratchet ceiling bumped 17,300 -> 17,400 deliberately for the goal dispatch handlers. Refs CLI_AUDIT.md (effort-native-reasoning, ux-external-editor, goal-tools-persistent).
Systematic fix for the recurring registry-drift bug class (P0-G/P0-H family). - _env_vars.py: single source of truth for all 42 BOG_AGENTS_* environment variables as named constants, plus is_env_truthy/env_bool helpers matching the existing 1/true/yes/on parsing convention. A completeness drift test greps the package for BOG_AGENTS_* literals and fails if any is unregistered, so new vars must be declared centrally. - config_manifest.py: a frozen ConfigOption table (56 options) with __post_init__ validation that rejects a mistyped/mutable default at import, and resolve_scalar with env > toml > default precedence. Credential options derive from model_config.PROVIDER_API_KEY_ENV rather than a hand-list, so they stay locked to the P0-G sync invariant. - Upgrades the headless 'config' command from a 3-line stub into a real introspection surface: 'config'/'config show' lists every option with its resolved value, source, and type; 'config get <key>' resolves one. Secrets are redacted and never appear in text or JSON output. Read-only. Additive: no call-site sweep of existing literals (that mechanical migration is a separate cleanup); the registry + drift test lock the surface going forward. Refs CLI_AUDIT.md (env-var-registry, config-manifest).
…er's git index
Two related defects in CheckpointingMiddleware:
1. It ran 'git init' then 'git stash create' with no initial commit. On git
2.44+, 'git stash create' on an unborn HEAD fails ('you do not have the
initial commit yet'), so no checkpoint was ever recorded and undo/rewind
silently did nothing on a fresh working directory. _ensure_initialized now
makes an initial --allow-empty commit ONLY when HEAD is unborn (a repo with
existing history is never committed into), and _create_checkpoint anchors at
HEAD when the tree is clean, so a pre-mutation checkpoint always exists.
2. 'git add -A' staged files into the user's real git index. A user running the
agent in their own repo would find their whole working tree staged (and, with
the initial-commit path, an empty checkpoint commit created). All checkpoint
git commands now run against an isolated throwaway index via GIT_INDEX_FILE,
so the user's index and history are never modified.
Resolves two pre-existing test_checkpointing_hardening failures on modern git;
rewind/undo/diff tests are unaffected.
Remote MCP servers that require OAuth (e.g. the hosted GitHub MCP server) can now be authenticated. Uses the official mcp SDK's OAuthClientProvider, which performs RFC 9728 protected-resource-metadata discovery, Dynamic Client Registration, PKCE, the authorization-code exchange, and automatic token refresh internally — nothing is hand-rolled. - FileTokenStorage persists per-server client registration + tokens under ~/.bog-agents/mcp-oauth/<server>.json in a versioned envelope, written via atomic_write_text(0600) + _secure_owner_only (cross-platform). Reads are fail-closed (size cap, version gate, JSON validation) and never log a token; server names are validated so a name can't escape the directory. - LoopbackCallbackServer: single-use 127.0.0.1 callback listener that captures (code, state) for the SDK's CSRF check, with a paste-back fallback for headless environments. - /mcp login <server> runs the browser + loopback flow in a Textual worker (never blocks the event loop); /mcp logout and /mcp status manage stored tokens. - The connection path attaches a provider for a server marked auth:oauth or with stored tokens; a 401/RFC-9728 challenge at load now yields an actionable 'run /mcp login <server>' message instead of an opaque timeout. - The refresh-only (connection) provider is built with interactive=False: it binds NO loopback socket and never opens a browser, so attaching it per reload can't leak sockets/threads or pop a browser at startup. Deletes the dead hand-rolled oauth_mcp.py (and its test) it supersedes. Declares mcp>=1.27.0 as a direct dependency. Refs CLI_AUDIT.md (mcp-oauth epic, done spec-compliant).
Adds set_symlink_trust_checker(), consulted at the _filter_skill_dirs containment chokepoint on both the sync and async skill-listing paths. With no checker registered, behavior is unchanged (all symlinked skill dirs refused); the CLI registers a checker backed by its skill trust store so a user can opt a symlinked skill dir back in. The SDK never imports the CLI. A checker that raises is treated as refusal (fail-closed). Refs CLI_AUDIT.md (skill-trust-store).
- Theme system: registers the bog 'peat' palette as a proper Textual Theme (default) plus bog-light/abyss/ember built-ins and unlimited user themes from config. /theme opens a live-preview selector; /theme <name> switches and persists across restarts. The app.tcss palette-override block is removed so themes control the palette -- proven byte-identical to the old default (a test mounts the app and diffs all 18 CSS variables against the prior hex). - Skill trust store: /skills trust|revoke|clear|list lets a user explicitly opt a symlinked skill directory back in, relaxing the default refuse-all-symlinks posture ONLY for approved paths. Trust is per-resolved-path with a SKILL.md fingerprint; a post-approval symlink swap or an in-place SKILL.md change is re-refused, and a malformed store fails closed (nothing trusted). Persisted 0600. Honored on both sync and async skill-load paths. - UX polish: /effort with no argument opens a model-aware level picker (only the levels the active model supports); /goal review opens a read-only goal+rubric panel; doctor reports managed-vs-system ripgrep, a shadowed-entrypoint check, and MCP OAuth signed-in count; the MCP viewer renders untrusted server strings via Content.assemble tuples (never markup-parsed) as defense-in-depth. Refs CLI_AUDIT.md (theme-system, skill-trust-store, effort selector, goal panel, doctor, mcp-viewer).
…ening # Conflicts: # libs/daemon/uv.lock
…, parser test) Formatting-only: ruff format rewraps a long git-commit arg line in checkpointing.py, and an E501 test string in test_tool_call_parser.py is split across adjacent literals. No behavior change; makes `make lint` clean for CI.
…cket isolation The loopback callback tests bind a real 127.0.0.1 (inet) socket, which CI's Linux `pytest --disable-socket --allow-unix-socket` step blocks (only unix sockets are allowed there). An autouse fixture re-enables sockets per test via pytest-socket when it is active, and is a no-op under the Windows CI step (which runs `-p no:socket`). No production code changes.
_resolve_path fully resolves the path (following a symlink at the final component), which the sandbox-escape check needs but which defeats O_NOFOLLOW and is_symlink() at the leaf — they end up inspecting the symlink's target, not the link. As a result: - write() overwrote a symlink's TARGET instead of being refused (the O_NOFOLLOW flag was applied to the already-resolved target path). Regressed when the error-on-exists guard was removed for the overwrite semantics. - delete() of a symlink-to-directory rmtree'd the target's contents instead of removing just the link (is_symlink() was false on the resolved target). Both now validate the sandbox via _resolve_path, then operate on a new _physical_base() path whose final component is left unresolved, so O_NOFOLLOW refuses a write-through and a symlinked leaf is unlinked as a link. A symlink whose target escapes root is still rejected by _resolve_path first. Caught by two symlink tests that only run on Linux CI (they skip where symlink creation is unprivileged, e.g. Windows dev boxes).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This branch brings bog-agents to full parity with, and drop-in interchangeability for, deepagents 0.6.12 (the upstream fork), fixes several real security bugs found along the way, and makes the CLI world-class — including spec-compliant MCP OAuth. It also carries the earlier resiliency-hardening work this branch started from.
All gates green across every package: SDK 2540 passed · CLI 4988 passed · daemon 141 passed (0 failures), ruff + ruff-format + ty clean, lockfiles consistent.
deepagents 0.6.12 parity + drop-in
bog-agents is now a source-level drop-in for the deepagents public API and co-installable with it (the
wcmatch<7.0cap that made the two unresolvable in one venv is lifted). Verified: all 20 deepagents public symbols import from bothbog_agentsandbog_agents.deepagents, andcreate_deep_agent(system_prompt={"base": None, ...})compiles instead of raisingTypeError.FileDatav2 (content: str+ encoding), Result dataclasses,delete()on every backend,write()overwrites (upstream contract),StateBackend()/StoreBackend(store=…)construct without a runtime, newLangSmithSandbox+ContextHubBackend. Fixes three real bugs (grep char-iteration, Windows glob, composite route-leak).SystemPromptConfig, full export surface, unclamped recursion limit, user-middleware replace-in-place,GeneralPurposeSubagentProfileactually disables the task tool, per-subagent profile resolution, provider-awaremodel_matches_spec.Security fixes (all pre-existing, found by audit)
ls/glob/grepresults — a pathlessgrep(pattern="API_KEY")exfiltrated denied paths and file contents. Now filtered; wired intocreate_agentand pinned end-to-end.ainvokebypassed containment; both paths now share one chokepoint.[/dim]crashed the HITL dialog. Escaped (escape-last ordering) + control-char sanitized.GIT_INDEX_FILEand only commits on an unborn HEAD.World-class CLI + deepagents_code ports
mcpSDKOAuthClientProvider(RFC 9728 discovery + Dynamic Client Registration + PKCE + refresh — not hand-rolled)./mcp login|logout|status, single-use127.0.0.1loopback callback, tokens stored0600and fail-closed, refresh-path provider binds no socket and never opens a browser. Replaces the dead hand-rolledoauth_mcp.py./effortmapping to each provider's real reasoning knob (was capping reasoning models at 1024 tokens), theme system (/theme+ user themes, default palette proven byte-identical), skill trust store (opt-in, fingerprinted, fail-closed),/goal+/rubric, managed ripgrep,${VAR}MCP headers, env-var registry + config manifest, ctrl+x external editor, doctor improvements.Verification & CI notes
skipif(win32)); they run on Linux CI. The trust-store fingerprint/fail-closed and swap logic were additionally verified by hand.See
PARITY.mdandlibs/cli/CLI_AUDIT.mdfor the full audit trail and the deferred backlog.