Command permissions + Alembic SQLite migrations

CHANGELOG.md

@@ -6,16 +6,27 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
 
 ## [Unreleased]
 
+## [0.12.0] - 2026-07-18
+
 ### Added
 
+- **Command permissions** for `shell` / `ssh`: segment-aware allowlist, denylist, and confirm modes (`tools.*.permissions` in config). YAML baseline plus runtime overrides (SQLite), enforced **before** CommandGuard / user confirm. REST API under `/api/permissions/commands/*` (get/put overrides, dry-run validate with optional draft policy). See [docs/SECURITY.md](docs/SECURITY.md)
+- **Alembic migrations** for SQLite: chat DB (sessions, tools, monitor, connectors, **canvas**) and **prompt_lab** DB. Auto-upgrade on web boot; Docker `agentforge-web` entrypoint runs `upgrade-all` before uvicorn/SAQ. Legacy DBs are stamped (no re-CREATE). Applied history in `schema_migrations` (`revision`, **filename**, `applied_at`). CLI: `python -m web.server.database.cli upgrade-all|upgrade|current|applied|history|revision`. See [docs/architecture.md](docs/architecture.md#sqlite-schema-alembic)
 - CI **Type Check (ty)** job: [Astral `ty`](https://docs.astral.sh/ty/) (Rust-based, pairs with Ruff) on `agentforge`, `chunking`, `sidecar`, `sandbox`, `tests`
 - `[tool.ty]` configuration in `pyproject.toml` with `app/` and `web/` excluded until diagnostics are cleared; `ty` added to the `dev` extra
+- `alembic` dependency and console entry point `agentforge-db`
 
 ### Changed
 
 - **Dependencies simplified to production + dev only**: all runtime packages (framework, tools, service stack, SQLAlchemy drivers) live in `[project.dependencies]`; `[project.optional-dependencies]` now has only `dev` (ruff, ty, pytest). Removed granular extras (`bedrock`, `browser`, `service`, `all`, …)
 - CI runs on **pull requests only** (removed `push` to `master`) so merge does not duplicate the same lint/build/typecheck pass and added `concurrency` to cancel stale PR runs
 - Cleared all `ty` diagnostics in `agentforge/` (138 → 0): `chat()` overloads, tool registry typing, Playwright wait literals, config coercion helpers, and assorted narrowings
+- `ChatDatabase.create_tables()` / Canvas / Prompt Lab schema setup use Alembic instead of ad-hoc `create_all` + `ALTER TABLE` blocks
+- Shell `allowed_commands` / `blocked_patterns` still work as legacy keys under `tools.shell` when `permissions.*` lists are empty
+
+### Removed
+
+- Hand-written SQLite column migrations inside `web/server/database/manager.py` (replaced by Alembic revisions)
 
 ## [0.11.0] - 2026-06-28
 

Dockerfile.web

@@ -42,6 +42,13 @@ COPY plugins/ plugins/
 
 RUN mkdir -p /app/data /app/data/uploads
 
+# Run Alembic migrations for chat + prompt_lab before starting the process.
+# Workers reuse this image; their command is overridden in compose and still
+# goes through this entrypoint so schema is ready before job handling.
+COPY web/entrypoint.sh /entrypoint.sh
+RUN chmod +x /entrypoint.sh
+
 EXPOSE 8200
 
+ENTRYPOINT ["/entrypoint.sh"]
 CMD ["uvicorn", "web.server.app:app", "--host", "0.0.0.0", "--port", "8200"]

README.md

@@ -31,7 +31,7 @@ These projects use AgentForge as their backend and don't run without it:
 
 - **Backends**: Ollama (local + cloud relay), AWS Bedrock, and any OpenAI-compatible API (DeepInfra, OpenRouter, ...). Selected per role via named profiles. Switch the whole stack with one `provider_override`.
 - **Agent loop**: think -> act -> observe with tool calling, error recovery, and optional web-search escalation.
-- **Tools**: filesystem, shell, system info, Docker, Git, SSH, archives, network diagnostics, web search/fetch/render, media, code editing, macOS notifications, **Apple Reminders** (list/create/edit/complete/delete via `remindctl`), and more.
+- **Tools**: filesystem, shell, system info, Docker, Git, SSH, archives, network diagnostics, web search/fetch/render, media, code editing, macOS notifications, **Apple Reminders** (list/create/edit/complete/delete via `remindctl`), and more. Shell/SSH commands can be gated by allowlist, denylist, or confirm policy (config + runtime API).
 - **RAG**: index OpenAPI/SQL schemas, source code, docs, and transcripts into Qdrant. Query with refinement, reranking, and dedup.
 - **Knowledge Database**: personal store for notes, references, documentation, attached documents, cheatsheets, and snippets. One-call ingest, semantic search, tag faceting, and smart updates (re-embeds only when content changes). Multi-collection routing (`X-Knowledge-Collection` header) separates the KB SPA and AgentForge Notes into independent Qdrant collections. Original attachment files (PDFs, etc.) are stored alongside extracted text and downloadable via the API.
 - **Connectors**: link external accounts as agent tools. Gmail, Drive, BigQuery, and YouTube through one Google OAuth client, plus GitLab and GitHub via personal access tokens. Multi-account, in-process, read-only by default.
@@ -42,7 +42,7 @@ These projects use AgentForge as their backend and don't run without it:
 
 Operator guides live in [`docs/`](docs/README.md):
 
-- [Stack architecture](docs/architecture.md): how the containers fit together: services, ports, worker localities, data stores, and request flow. **Start here.**
+- [Stack architecture](docs/architecture.md): how the containers fit together: services, ports, worker localities, data stores, request flow, and **SQLite Alembic migrations**. **Start here.**
 - [HTTP API](docs/api.md): REST + the `/ws/chat` agent WebSocket, memory endpoints, the Knowledge Database (`/knowledge/*`), and the live OpenAPI spec.
 - [Modes](docs/modes.md): the `@mode` prefixes (built-in modes, custom agents, connectors) and when to use each.
 - [Tools](docs/tools.md): every built-in agent tool by category, plus locality and confirmation gates.
@@ -51,7 +51,7 @@ Operator guides live in [`docs/`](docs/README.md):
 - [Connectors](docs/connectors.md): linking external accounts. The unified Google OAuth connector (Gmail, Drive, BigQuery, YouTube) and the GitLab and GitHub token connectors.
 - [Authoring tools and private overlays](docs/plugin-authoring.md): adding private tools, the `AGENTFORGE_TOOL_PLUGINS` seam, and the local overlay files.
 - [Instruction markdown](markdown/README.md): the `skills/` and `custom-agents/` markdown you edit to tune agents without touching Python.
-- [Security](docs/SECURITY.md): the auth model, sidecar/internal tokens, interactive sudo, SSRF and read-only guards.
+- [Security](docs/SECURITY.md): auth, sidecar/internal tokens, interactive sudo, SSRF and read-only guards, and **shell/SSH command permissions**.
 
 ## Run it locally
 
@@ -160,10 +160,15 @@ ruff check .                # lint
 ruff format --check .       # formatting
 ty check                    # type check (phased scope; see pyproject.toml)
 pytest                      # tests
+
+python -m web.server.database.cli upgrade-all
+python -m web.server.database.cli applied --database chat   # revision + filename log
 ```
 
 CI runs lint, format, type check, and build + smoke on pull requests (`.github/workflows/ci.yml`).
 
+Schema changes belong in Alembic revision files under `web/server/database/migrations/` (chat/canvas) or `web/server/prompt_lab/database/migrations/` — see [architecture → SQLite schema](docs/architecture.md#sqlite-schema-alembic).
+
 To exercise the framework directly without the Docker/web stack, see the [sandbox harness](sandbox/README.md): a short Python script driving `AIClient` / `ToolRegistry` / the agent loop against your Ollama.
 
 ## License

agentforge/db_cli.py

@@ -0,0 +1,19 @@
+"""Console entry point for chat-database migrations.
+
+Installed as ``agentforge-db`` when the package is installed editable
+(``pip install -e .``). In package-mode Poetry envs you can also run::
+
+    python -m web.server.database.cli upgrade
+"""
+
+from __future__ import annotations
+
+
+def main() -> None:
+    from web.server.database.cli import main as cli_main
+
+    raise SystemExit(cli_main())
+
+
+if __name__ == "__main__":
+    main()

agentforge/tools/command_guard.py

@@ -32,6 +32,9 @@
 
 from chalkbox.logging.bridge import get_logger
 
+from agentforge.client import AIClient
+from agentforge.config import get_config
+
 if TYPE_CHECKING:
     pass
 
@@ -109,8 +112,6 @@
 def _get_guard_config() -> dict:
     """Return guard config from config.yaml → tools.shell.guard."""
     try:
-        from agentforge.config import get_config
-
         cfg = get_config()
         return cfg._raw.get("tools", {}).get("shell", {}).get("guard", {})
     except Exception:
@@ -183,9 +184,6 @@ def _get_client(self):
         We use dataclasses.replace to clone with the guard's overrides.
         """
         if self._client is None:
-            from agentforge.client import AIClient
-            from agentforge.config import get_config
-
             shared = get_config().get_profile("fast")
             guard_profile = dataclasses.replace(
                 shared,

agentforge/tools/command_policy.py

@@ -0,0 +1,154 @@
+"""Segment-aware command permission evaluator for shell and ssh tools."""
+
+from __future__ import annotations
+
+import logging
+import os
+import re
+from dataclasses import dataclass
+from typing import Literal
+
+from agentforge.config import get_config
+
+# Share shell parsing semantics with readonly_guard (segment splitting).
+from agentforge.tools.readonly_guard import _split_segments
+
+logger = logging.getLogger(__name__)
+
+PolicyMode = Literal["confirm", "allowlist", "denylist"]
+PolicyAction = Literal["allow", "deny", "confirm"]
+ToolName = Literal["shell", "ssh"]
+
+
+@dataclass(frozen=True)
+class CommandPolicy:
+    mode: PolicyMode = "confirm"
+    allowed_commands: tuple[str, ...] = ()
+    allowed_patterns: tuple[str, ...] = ()
+    blocked_patterns: tuple[str, ...] = ()
+
+
+@dataclass(frozen=True)
+class PolicyVerdict:
+    action: PolicyAction
+    reason: str = ""
+    source: str = ""
+
+
+def _segment_base_command(segment: str) -> str:
+    parts = segment.strip().split()
+    idx = 0
+    while idx < len(parts) and "=" in parts[idx] and not parts[idx].startswith("-"):
+        idx += 1
+    while idx < len(parts) and parts[idx] in ("sudo", "command", "nice", "ionice", "time"):
+        idx += 1
+    if idx >= len(parts):
+        return ""
+    return os.path.basename(parts[idx].rsplit("/", 1)[-1])
+
+
+def _pattern_matches(pattern: str, segment: str) -> bool:
+    try:
+        return bool(re.search(pattern, segment, re.IGNORECASE))
+    except re.error:
+        logger.warning("Invalid regex pattern in command policy: %s", pattern)
+        return False
+
+
+def _segment_passes_allowlist(segment: str, policy: CommandPolicy) -> bool:
+    base = _segment_base_command(segment)
+    if policy.allowed_commands and base in policy.allowed_commands:
+        return True
+    for pattern in policy.allowed_patterns:
+        if _pattern_matches(pattern, segment):
+            return True
+    return False
+
+
+def _find_blocked_segment(command: str, policy: CommandPolicy) -> tuple[str, str] | None:
+    for segment in _split_segments(command):
+        for pattern in policy.blocked_patterns:
+            if _pattern_matches(pattern, segment):
+                return segment, pattern
+    return None
+
+
+def evaluate(tool: ToolName, command: str, policy: CommandPolicy) -> PolicyVerdict:
+    if not command.strip():
+        return PolicyVerdict(action="allow", reason="empty", source="noop")
+
+    if policy.mode == "confirm":
+        blocked = _find_blocked_segment(command, policy)
+        if blocked is not None:
+            _segment, pattern = blocked
+            return PolicyVerdict(
+                action="deny",
+                reason=f"Command matches blocked pattern ({pattern})",
+                source="policy_blocked_pattern",
+            )
+        return PolicyVerdict(action="confirm", reason="", source="policy_confirm")
+
+    if policy.mode == "denylist":
+        blocked = _find_blocked_segment(command, policy)
+        if blocked is not None:
+            _segment, pattern = blocked
+            return PolicyVerdict(
+                action="deny",
+                reason=f"Command segment matches blocked pattern ({pattern})",
+                source="policy_denylist",
+            )
+        return PolicyVerdict(action="allow", reason="", source="policy_denylist")
+
+    # allowlist mode
+    for segment in _split_segments(command):
+        if not _segment_passes_allowlist(segment, policy):
+            base = _segment_base_command(segment)
+            return PolicyVerdict(
+                action="deny",
+                reason=f"Command '{base}' is not allowed",
+                source="policy_allowlist",
+            )
+    return PolicyVerdict(action="allow", reason="", source="policy_allowlist")
+
+
+def load_yaml_policy(tool: ToolName) -> CommandPolicy:
+    try:
+        cfg = get_config()
+        tool_cfg = cfg._raw.get("tools", {}).get(tool, {})
+        perms = tool_cfg.get("permissions", {})
+
+        mode = perms.get("mode", "confirm")
+        allowed_commands = tuple(perms.get("allowed_commands") or ())
+        allowed_patterns = tuple(perms.get("allowed_patterns") or ())
+        blocked_patterns = tuple(perms.get("blocked_patterns") or ())
+
+        if not allowed_commands:
+            legacy_allowed = tool_cfg.get("allowed_commands") or []
+            if legacy_allowed:
+                allowed_commands = tuple(legacy_allowed)
+
+        if not blocked_patterns:
+            legacy_blocked = tool_cfg.get("blocked_patterns") or []
+            if legacy_blocked:
+                blocked_patterns = tuple(legacy_blocked)
+
+        return CommandPolicy(
+            mode=mode,
+            allowed_commands=allowed_commands,
+            allowed_patterns=allowed_patterns,
+            blocked_patterns=blocked_patterns,
+        )
+    except Exception:
+        logger.warning("Failed to load YAML command policy for %s; using defaults", tool)
+        return CommandPolicy()
+
+
+def merge_policies(base: CommandPolicy, override: CommandPolicy | None) -> CommandPolicy:
+    if override is None:
+        return base
+    return CommandPolicy(
+        mode=override.mode,
+        allowed_commands=override.allowed_commands or base.allowed_commands,
+        allowed_patterns=override.allowed_patterns or base.allowed_patterns,
+        blocked_patterns=override.blocked_patterns or base.blocked_patterns,
+    )