Replace finance-only hook with a generic config-driven artifact hook

.github/workflows/ci.yml

@@ -0,0 +1,19 @@
+name: CI
+
+on:
+  push:
+    branches: [main]
+  pull_request:
+
+jobs:
+  test:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+      - uses: actions/setup-python@v5
+        with:
+          python-version: "3.12"
+          cache: pip
+      - run: pip install -r requirements.txt -r requirements-dev.txt
+      - run: ruff check .
+      - run: pytest

README.md

@@ -76,19 +76,47 @@ tools=[{
 
 ## Claude Code hooks
 
-`hooks/` contains PostToolUse hook scripts that log agent activity into ai-data-store automatically, and `scripts/install_hook.sh` installs one into your global `~/.claude/settings.json`:
+`hooks/log_artifact_entry.py` is a generic PostToolUse hook that logs JSON artifacts produced by agent scripts into ai-data-store automatically. Install it once globally:
 
 ```bash
-./scripts/install_hook.sh log_finance_entry.py
+./scripts/install_hook.sh
 ```
 
 This copies the script to `~/.claude/hooks/` and registers it as a `command`-type `PostToolUse` hook, without touching any other hooks or settings already there — safe to re-run. It requires the `ai-data-store` MCP server to already be configured in `~/.claude.json` (see Laptop setup above), since the hook reads that config to find the server URL and API key.
 
-Run it with no argument to install the default `log_finance_entry.py`, or name a different script in `hooks/` to install that one instead. Hooks are self-filtering — each one only acts on the tool calls it recognizes and is a silent no-op otherwise, so it's safe to have installed even outside the project it targets.
+The hook fires on every tool call in every project, but only acts on Bash calls in projects that **opt in** by having a `.ai-data-store.json` config at their root; everywhere else it's a silent no-op. The convention: an artifact-producing script prints just the path of the JSON file it wrote, and the hook picks it up from stdout the moment it's created.
 
-| Hook | Fires on | Logs |
-|------|----------|------|
-| `log_finance_entry.py` | Bash calls in `finance-agent` whose stdout is the path to a known `skills/*/tmp/*.json` artifact (verdict, debate, panel, check_stock, check_crypto, price history, portfolio, news) — every artifact script in that repo prints just this path, so the hook reads the file straight off disk | A ticker/stage-aware description and keywords built from the artifact's content, e.g. `Verdict - MSFT (bear, medium confidence)` |
+The config declares how artifacts become entries — regex rules over the artifact path, with `str.format` templates for the description and keywords. Named regex groups (`{ticker}`) and dotted paths into the artifact JSON (`{content.verdict.winner}`) are both available; `description` takes a list of templates and uses the first whose fields all resolve, so richer descriptions degrade gracefully:
+
+```json
+{
+  "source": "finance-agent",
+  "rules": [
+    {
+      "pattern": "/tmp/verdict_(?P<ticker>[^_]+)_[^/]*\\.json$",
+      "description": [
+        "Verdict - {content.symbol} ({content.verdict.winner}, {content.verdict.confidence} confidence)",
+        "Verdict - {ticker}"
+      ],
+      "keywords": ["{ticker}", "verdict"]
+    }
+  ]
+}
+```
+
+Ready-made configs live in `hooks/configs/` — copy one to the target project's root:
+
+```bash
+cp hooks/configs/finance-agent.json <finance-agent-repo>/.ai-data-store.json
+```
+
+**Migrating from `log_finance_entry.py`** (the old finance-only hook): uninstall it, install the generic hook, and drop the config into the finance-agent repo:
+
+```bash
+./scripts/uninstall_hook.sh log_finance_entry.py
+./scripts/install_hook.sh
+cp hooks/configs/finance-agent.json <finance-agent-repo>/.ai-data-store.json
+```
 
 ## MCP tools
 
@@ -111,4 +139,14 @@ curl -X POST http://<casaos-ip>:8742/entries \
   -d '{"source": "finance-agent", "description": "Verdict - MSFT", "keywords": ["MSFT", "verdict"], "data": {}}'
 ```
 
-Returns the created entry (`201`) or a `400` with an error message for a missing field or invalid JSON body.
+Returns the created entry (`201`) or a `400` with an error message for a missing or wrongly-typed field or invalid JSON body.
+
+## Development
+
+```bash
+pip install -r requirements.txt -r requirements-dev.txt
+ruff check .
+pytest
+```
+
+Tests run against an in-memory MongoDB (`mongomock-motor`) — no containers needed. CI runs the same two commands on every push and PR.

app/database.py

@@ -1,4 +1,5 @@
 from motor.motor_asyncio import AsyncIOMotorClient
+
 from app.config import settings
 
 _client: AsyncIOMotorClient | None = None
@@ -15,6 +16,13 @@ def get_db():
     return get_client()[settings.mongodb_db]
 
 
+async def ensure_indexes():
+    db = get_db()
+    await db.entries.create_index([("created_at", -1)])
+    await db.entries.create_index("source")
+    await db.entries.create_index("keywords")
+
+
 async def close_client():
     global _client
     if _client is not None:

app/formatting.py

@@ -13,5 +13,5 @@ def format_doc(doc: dict) -> dict:
 def parse_id(entry_id: str) -> ObjectId:
     try:
         return ObjectId(entry_id)
-    except (InvalidId, Exception):
-        raise ValueError(f"Invalid entry ID: {entry_id}")
+    except (InvalidId, TypeError):
+        raise ValueError(f"Invalid entry ID: {entry_id}") from None

app/server.py

@@ -1,19 +1,24 @@
+import secrets
 from contextlib import asynccontextmanager
-from datetime import datetime, timezone
+from datetime import UTC, datetime
 from typing import Any
 
 import uvicorn
 from fastmcp import FastMCP
+from pydantic import BaseModel, ValidationError
 from starlette.requests import Request
 from starlette.responses import JSONResponse
 
 from app.config import settings
-from app.database import close_client, get_db
+from app.database import close_client, ensure_indexes, get_db
 from app.formatting import format_doc, parse_id
 
+MAX_LIMIT = 100
+
 
 @asynccontextmanager
 async def lifespan(server: FastMCP):
+    await ensure_indexes()
     yield
     await close_client()
 
@@ -31,7 +36,8 @@ async def __call__(self, scope, receive, send):
         if scope["type"] == "http":
             headers = dict(scope.get("headers", []))
             auth = headers.get(b"authorization", b"").decode()
-            if not auth.startswith("Bearer ") or auth[7:] != settings.api_key:
+            token = auth[7:] if auth.startswith("Bearer ") else ""
+            if not secrets.compare_digest(token.encode(), settings.api_key.encode()):
                 response = JSONResponse({"error": "Unauthorized"}, status_code=401)
                 await response(scope, receive, send)
                 return
@@ -42,14 +48,14 @@ async def _create_entry(
     source: str,
     description: str,
     data: dict[str, Any],
-    keywords: list[str] = [],
+    keywords: list[str] | None = None,
 ) -> dict:
     db = get_db()
-    now = datetime.now(timezone.utc)
+    now = datetime.now(UTC)
     doc = {
         "source": source,
         "description": description,
-        "keywords": keywords,
+        "keywords": keywords or [],
         "data": data,
         "created_at": now,
         "updated_at": now,
@@ -64,7 +70,7 @@ async def create_entry(
     source: str,
     description: str,
     data: dict[str, Any],
-    keywords: list[str] = [],
+    keywords: list[str] | None = None,
 ) -> dict:
     """Store a new data entry produced by an AI agent."""
     return await _create_entry(source, description, data, keywords)
@@ -78,6 +84,8 @@ async def list_entries(
     skip: int = 0,
 ) -> dict:
     """List stored entries. Optionally filter by source agent or keywords."""
+    limit = max(1, min(limit, MAX_LIMIT))
+    skip = max(0, skip)
     db = get_db()
     query: dict = {}
     if source:
@@ -121,7 +129,7 @@ async def update_entry(
     if not updates:
         raise ValueError("No fields provided to update")
     db = get_db()
-    updates["updated_at"] = datetime.now(timezone.utc)
+    updates["updated_at"] = datetime.now(UTC)
     result = await db.entries.find_one_and_update(
         {"_id": parse_id(entry_id)},
         {"$set": updates},
@@ -142,6 +150,13 @@ async def delete_entry(entry_id: str) -> dict:
     return {"deleted": True, "id": entry_id}
 
 
+class EntryPayload(BaseModel):
+    source: str
+    description: str
+    data: dict[str, Any]
+    keywords: list[str] = []
+
+
 async def create_entry_endpoint(request: Request) -> JSONResponse:
     """REST equivalent of create_entry, for callers that can't speak MCP (e.g. hook scripts)."""
     try:
@@ -150,17 +165,19 @@ async def create_entry_endpoint(request: Request) -> JSONResponse:
         return JSONResponse({"error": "Invalid JSON body"}, status_code=400)
 
     try:
-        source = body["source"]
-        description = body["description"]
-        data = body["data"]
-    except KeyError as e:
-        return JSONResponse({"error": f"Missing required field: {e.args[0]}"}, status_code=400)
+        payload = EntryPayload.model_validate(body)
+    except ValidationError as e:
+        errors = "; ".join(
+            f"{'.'.join(str(loc) for loc in err['loc']) or 'body'}: {err['msg']}"
+            for err in e.errors()
+        )
+        return JSONResponse({"error": errors}, status_code=400)
 
     entry = await _create_entry(
-        source=source,
-        description=description,
-        data=data,
-        keywords=body.get("keywords", []),
+        source=payload.source,
+        description=payload.description,
+        data=payload.data,
+        keywords=payload.keywords,
     )
     return JSONResponse(entry, status_code=201)
 

hooks/configs/finance-agent.json

@@ -0,0 +1,76 @@
+{
+  "source": "finance-agent",
+  "rules": [
+    {
+      "pattern": "/skills/(?P<skill>[^/]+)/tmp/verdict_(?P<ticker>[^_]+)_[^/]*\\.json$",
+      "description": [
+        "Verdict - {content.symbol} ({content.verdict.winner}, {content.verdict.confidence} confidence)",
+        "Verdict - {ticker}"
+      ],
+      "keywords": ["{ticker}", "{skill}", "verdict"]
+    },
+    {
+      "pattern": "/skills/(?P<skill>[^/]+)/tmp/debate_(?P<ticker>[^_]+)_[^/]*\\.json$",
+      "description": [
+        "Debate - {content.symbol} ({content.rounds} rounds)",
+        "Debate - {ticker}"
+      ],
+      "keywords": ["{ticker}", "{skill}", "debate"]
+    },
+    {
+      "pattern": "/skills/(?P<skill>[^/]+)/tmp/panel_(?P<ticker>[^_]+)_[^/]*\\.json$",
+      "description": [
+        "Panel analysis - {content.symbol}",
+        "Panel analysis - {ticker}"
+      ],
+      "keywords": ["{ticker}", "{skill}", "panel"]
+    },
+    {
+      "pattern": "/skills/(?P<skill>[^/]+)/tmp/check_stock_(?P<ticker>[^_]+)_[^/]*\\.json$",
+      "description": [
+        "{content.name} ({content.symbol}) - ${content.price:,.2f} ({content.change_percent:+.2f}%)",
+        "{content.name} ({content.symbol}) price check",
+        "{ticker} price check"
+      ],
+      "keywords": ["{ticker}", "{skill}", "check_stock"]
+    },
+    {
+      "pattern": "/skills/(?P<skill>[^/]+)/tmp/check_crypto_(?P<ticker>[^_]+)_[^/]*\\.json$",
+      "description": [
+        "{content.name} ({content.symbol}) - ${content.price:,.2f} ({content.change_percent_24h:+.2f}% 24h)",
+        "{content.name} ({content.symbol}) price check",
+        "{ticker} price check"
+      ],
+      "keywords": ["{ticker}", "{skill}", "check_crypto"]
+    },
+    {
+      "pattern": "/skills/(?P<skill>[^/]+)/tmp/history_(?P<ticker>[^_]+)_(?P<period>[^_]+)_[^/]*\\.json$",
+      "description": "Price history - {ticker}",
+      "keywords": ["{ticker}", "{skill}", "history", "{period}"]
+    },
+    {
+      "pattern": "/skills/(?P<skill>[^/]+)/tmp/portfolio_[^/]*\\.json$",
+      "description": [
+        "Portfolio snapshot - {content.account_summary.currency} {content.account_summary.totalValue:,.2f}",
+        "Portfolio snapshot"
+      ],
+      "keywords": ["{skill}"]
+    },
+    {
+      "pattern": "/skills/(?P<skill>[^/]+)/tmp/news_results_[^/]*\\.json$",
+      "description": [
+        "News search - {content_length} articles",
+        "News search"
+      ],
+      "keywords": ["{skill}", "news_results"]
+    },
+    {
+      "pattern": "/skills/(?P<skill>[^/]+)/tmp/analysis_[^/]*\\.json$",
+      "description": [
+        "News analysis - {content_length} articles ranked",
+        "News analysis"
+      ],
+      "keywords": ["{skill}", "analysis"]
+    }
+  ]
+}