diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml new file mode 100644 index 0000000..9172313 --- /dev/null +++ b/.github/workflows/ci.yml @@ -0,0 +1,46 @@ +name: Repository CI + +on: + pull_request: + push: + branches: + - main + +permissions: + contents: read + +concurrency: + group: ci-${{ github.workflow }}-${{ github.ref }} + cancel-in-progress: true + +jobs: + validate: + name: Build and test + runs-on: ubuntu-latest + timeout-minutes: 30 + steps: + - name: Checkout + uses: actions/checkout@v4 + + - name: Install pnpm + uses: pnpm/action-setup@v4 + with: + version: 11.8.0 + + - name: Set up Node.js + uses: actions/setup-node@v4 + with: + node-version: 24.14.0 + cache: pnpm + + - name: Install dependencies + run: pnpm install --frozen-lockfile + + - name: Build workspace + run: pnpm build + + - name: Run test suite + run: pnpm test + + - name: Test ast-grep rules + run: pnpm ast-grep:test diff --git a/Dockerfile b/Dockerfile index 53e67b2..02fd9de 100644 --- a/Dockerfile +++ b/Dockerfile @@ -30,6 +30,7 @@ COPY packages/agx-core/package.json ./packages/agx-core/ COPY packages/agx-cli/package.json ./packages/agx-cli/ COPY packages/agx-herdr/package.json ./packages/agx-herdr/ COPY packages/mcp/package.json ./packages/mcp/ +COPY packages/shared-ui/package.json ./packages/shared-ui/ RUN --mount=type=cache,id=pnpm,target=/pnpm/store pnpm install --frozen-lockfile # ─── Stage 2: Build all packages ──────────────────────────────────────────── diff --git a/README.md b/README.md index 94fb72e..11dba69 100644 --- a/README.md +++ b/README.md @@ -1,81 +1,174 @@ # agentx -> Event-Driven AI Agent Runtime SDK — modeled on Node.js and the Chrome DevTools Protocol. +> An event-driven TypeScript runtime for controllable, tool-using AI agents. + +AgentX separates an agent's conversation engine from an out-of-band control plane. The runtime +streams one model step at a time, preserves native tool-call history, dispatches registered tools +through worker threads, and accepts live operator commands over the Agent Debugger Protocol (ADP), +a JSON-RPC 2.0 WebSocket protocol inspired by Chrome DevTools. + +AgentX is an active project, not yet a production-ready agent platform. Its strongest implemented +idea is remote control of a running agent; durable sessions, authenticated remote ADP access, and +fully owned orchestration lifecycles remain roadmap work. ## Architecture +```text +┌─────────────────────────────────────────────────────────────────┐ +│ AgentSession │ +│ │ +│ I/O callbacks Inference Check │ +│ ingest tool results ─▶ one streamed LLM step ─▶ queued guards │ +│ ▲ │ │ +│ │ ▼ │ +│ └──────── AgenticThreadPool (worker_threads) │ +└──────────────────────────────┬──────────────────────────────────┘ + │ injected LLM, tools, notifier + ┌──────────┴──────────┐ + │ │ + AgentEventLoop AgentSessionHost + one session/server shared infrastructure, + isolated client sessions + │ │ + └──────────┬──────────┘ + │ WebSocket / JSON-RPC 2.0 + ┌──────────▼──────────┐ + │ ADP control plane │ + │ halt, pause, prompt,│ + │ inspect, tools │ + └─────────────────────┘ ``` -┌──────────────────────────────────────────────────────────────────┐ -│ Event-Driven Agent Runtime │ -│ │ -│ ┌─────────┐ ┌──────────────┐ ┌──────────────────────┐ │ -│ │ Timers │───▶│ I/O Callbacks│───▶│ Inference (LLM) │ │ -│ │ (TTL) │ │ (Tool Results)│ │ (Streaming + Abort) │ │ -│ └─────────┘ └──────────────┘ └──────────────────────┘ │ -│ ▲ │ │ -│ │ ▼ │ -│ ┌───────┴────┐ ┌────────────────┐ │ -│ │ Macrotask │ │ Microtask │ │ -│ │ Queue │ │ Queue (Guards) │ │ -│ └───────┬────┘ └────────────────┘ │ -│ │ │ -│ ┌───────┴────────────┐ │ -│ │ Agentic Thread Pool│ │ -│ │ (worker_threads) │ │ -│ └────────────────────┘ │ -└──────────────────────────────────────────────────────────────────┘ - ▲ - │ WebSocket (JSON-RPC 2.0) - │ Out-of-Band — bypasses event loop - ▼ -┌────────────────────────┐ -│ Agent Debugger Protocol│ -│ (ADP) │ -│ ├─ Inference.halt │ -│ ├─ Metacognition.pause│ -│ ├─ Memory.compact │ -│ └─ Toolchain.intercept│ -└────────────────────────┘ -``` -## Packages +The source labels each tick as four phases (Timers → I/O → Inference → Check), but the Timers +phase is currently a placeholder. Treat the implementation as a turn scheduler, not as a complete +reimplementation of the Node.js event loop. + +### Package map + +| Area | Packages and apps | +| --- | --- | +| Runtime | `@agentx/core`, `@agentx/adp` | +| Coordination | `@agentx/orchestrator`, `@agentx/mcp` | +| Operator clients | `@agentx/agx-core`, `@agentx/agx-cli`, `@agentx/agx-herdr`, `apps/agx-web`, `apps/pi-extension` | +| Examples | `apps/demo`, `apps/orchestrator-demo` | +| Product experiments | music scanner apps, Simon CLI, Zettel | + +The dependency direction for the runtime is `adp ← core ← orchestrator`. `AgentSessionHost` shares +an LLM and worker pool while keeping conversation state and outbound notifications scoped to each +ADP connection. + +## Five-minute start + +### Prerequisites + +- [mise](https://mise.jdx.dev/) (recommended), which installs the exact Node, pnpm, Bun, and Turso + versions declared in `mise.toml`; or Node 24.14.0 and pnpm 11.8.0 installed manually. +- Linux, macOS, or another environment supported by Node worker threads. + +```bash +mise install +mise exec -- pnpm install --frozen-lockfile +``` -| Package | Description | -| ------------------- | --------------------------------------------------------------------- | -| `@agentx/adp` | Agent Debugger Protocol — JSON-RPC schemas, WebSocket server & client | -| `@agentx/core` | Runtime — AgentEventLoop, AgenticThreadPool, LLMOrchestrator | -| `apps/demo` | Prototype agent demonstrating non-blocking tools + ADP control | -| `apps/pi-extension` | Pi TUI frontend — control agentx from inside pi | +### Fast, credential-free example -## Quick Start +The orchestrator demo uses local mock executors and reviewers. It requires no model account or +cloud service and completes in a few seconds: ```bash -# Install -pnpm install +mise exec -- pnpm exec vp run @agentx/orchestrator#build +mise exec -- pnpm --filter orchestrator-demo start +``` + +It demonstrates dependency ordering and review remediation. It is a coordination simulation—not +proof that the orchestrator owns remote executors or waits durably for work. + +### Live runtime and ADP -# Build packages -pnpm build +To exercise model inference and out-of-band control: -# Configure your LLM (OpenAI-compatible) +```bash cp apps/demo/.env.example apps/demo/.env -# Edit .env with your API key and base URL +# Set OPENAI_API_KEY, OPENAI_BASE_URL, and AGENT_MODEL. + +# Build the demo and its workspace dependencies. +mise exec -- pnpm exec vp run demo#build + +# Terminal 1: start the runtime and ADP server. +mise exec -- pnpm --filter demo start + +# Terminal 2: send a prompt, inspect state, then shut down. +mise exec -- pnpm --filter demo admin prompt "Explain the AgentX control plane in one paragraph" +mise exec -- pnpm --filter demo admin inspect +mise exec -- pnpm --filter demo admin shutdown +``` -# Run the interactive agent (stays alive, waits for ADP prompts) -cd apps/demo && pnpm start +The demo ADP endpoint is intended for a trusted local development machine. ADP currently has no +built-in authentication or method authorization; do not expose port 9222 to an untrusted network. +Worker threads keep expensive tools off the main event loop but are **not** a security sandbox. -# In another terminal — control from pi -cd apps/pi-extension && pi -e ./src/extension.ts +## Develop and validate -# Or use the low-level admin CLI -cd apps/demo && pnpm admin # Inference.halt -cd apps/demo && pnpm admin pause # Metacognition.pause -cd apps/demo && pnpm admin inspect # Get call frame -cd apps/demo && pnpm admin compact # Compact memory +Run workspace commands from the repository root so the root Vite+ and Vitest configuration is +used. The checks currently run by PR CI are: + +```bash +mise exec -- pnpm build +mise exec -- pnpm test +mise exec -- pnpm ast-grep:test +``` + +Two stricter checks are configured but currently report known repository debt rather than passing: + +```bash +mise exec -- pnpm test:coverage # current totals are below the configured global thresholds +mise exec -- pnpm lint # current type-aware lint reports pre-existing errors +``` + +They remain visible rather than being weakened; closing those gaps and promoting both to required +CI checks is tracked in the roadmap. + +Build one Vite+ task and its dependencies with, for example: + +```bash +mise exec -- pnpm exec vp run @agentx/core#build ``` -## How It Works +Tests use deterministic fake model boundaries by default; ordinary CI does not spend provider +credits. Real-provider compatibility and model output quality require separate opt-in evaluation. + +## Design strengths and boundaries + +**Implemented strengths** + +- A small embeddable `AgentSession` with injected model, tools, worker pool, and notifier. +- Native provider response messages and tool-call IDs are retained across tool rounds. +- Inference cancellation propagates through an `AbortSignal`. +- ADP supports broadcasts and session-targeted notifications for multi-client hosts. +- DAG, retry, and review primitives have focused unit and integration tests. + +**Important current boundaries** + +- ADP is unauthenticated and should remain loopback/trusted-network only. +- Session context is in memory; `Memory.compact` is fixed-window truncation, not durable memory. +- Tool workers have no complete timeout/cancellation/backpressure policy. +- Orchestration is event-driven scaffolding; executor ownership and completion semantics need + hardening before production use. +- The Timers phase is not implemented. +- `@agentx/core` and `@agentx/adp` have package entry points, but the repository does not yet have + a public release/versioning workflow. + +See [the roadmap](docs/roadmap.md) for prioritized work and explicit anti-goals. + +## Documentation + +- [Roadmap and readiness](docs/roadmap.md) +- [Vite+ task graph and caching](docs/vite-plus-caching.md) +- [Pi extension](apps/pi-extension/README.md) +- [AGX Herdr](packages/agx-herdr/README.md) +- [Zettel CI/CD](docs/zettel-cicd.md) +- [GCP setup for Zettel](docs/gcp-setup.md) + +## License -1. **Non-blocking tools**: Heavy computation runs in `worker_threads`. The main loop continues LLM inference without waiting. -2. **4-phase event loop**: Timers → I/O Callbacks → Inference → Check (microtasks/guards). -3. **ADP control plane**: A WebSocket on port 9222 accepts JSON-RPC commands that bypass the event queue entirely, enabling instant `/stop`, `/pause`, and memory compaction. -4. **Interactive prompt loop**: The agent stays alive and waits for `Session.prompt` commands via ADP. This lets external frontends (like the pi extension) drive the agent interactively. +[MIT](LICENSE) diff --git a/apps/demo/.env.example b/apps/demo/.env.example index 37ea310..cd24d18 100644 --- a/apps/demo/.env.example +++ b/apps/demo/.env.example @@ -1,2 +1,3 @@ -OPENAI_API_KEY=your_opencode_go_api_key -OPENAI_BASE_URL=your_opencode_go_base_url_if_applicable +OPENAI_API_KEY=your_api_key +OPENAI_BASE_URL=https://api.openai.com/v1 +AGENT_MODEL=gpt-4o diff --git a/apps/music-scanner-cli/tsconfig.json b/apps/music-scanner-cli/tsconfig.json index f1faac4..841647f 100644 --- a/apps/music-scanner-cli/tsconfig.json +++ b/apps/music-scanner-cli/tsconfig.json @@ -5,7 +5,8 @@ "jsxImportSource": "@opentui/react", "module": "ESNext", "moduleResolution": "Bundler", - "rootDir": "." + "rootDir": ".", + "types": ["node", "@testing-library/jest-dom/vitest"] }, "include": ["src/**/*"] } diff --git a/apps/zettel/src/frontend/components/SemanticVisualizer.tsx b/apps/zettel/src/frontend/components/SemanticVisualizer.tsx index e2aedce..2cdcb4f 100644 --- a/apps/zettel/src/frontend/components/SemanticVisualizer.tsx +++ b/apps/zettel/src/frontend/components/SemanticVisualizer.tsx @@ -8,7 +8,6 @@ interface Note { tags?: string[]; links?: string[]; body: string; - createdAt: number; } interface SemanticVisualizerProps { diff --git a/docs/roadmap.md b/docs/roadmap.md new file mode 100644 index 0000000..12c99ce --- /dev/null +++ b/docs/roadmap.md @@ -0,0 +1,94 @@ +# AgentX roadmap and readiness + +This roadmap reflects the repository as of August 2026. It is intentionally ordered around +correctness, security, and developer trust rather than feature count. Comparative research against +larger coding-agent projects informed the priorities, but every current-state claim below was +checked against AgentX source and configuration. + +## Positioning + +AgentX should remain a compact, embeddable runtime with a debugger-like remote control plane. Its +distinctive pieces are ADP, session-routed live control, injected runtime dependencies, and native +tool-call history. It should learn from mature agent products' lifecycle, policy, persistence, and +release contracts without copying their full coding TUI, provider catalog, daemon, plugin system, +or trusted-code execution environments. + +## Current readiness + +| Area | Verified state | Readiness implication | +| --- | --- | --- | +| Runtime | `AgentSession` runs one streamed inference step, ingests paired tool results, and supports cancellation. The Timers phase is a placeholder. | Promising kernel; scheduler claims must remain narrow. | +| Control plane | ADP validates JSON-RPC envelopes and routes notifications per client. It has no built-in authentication, authorization, origin policy, or request limits. | Trusted local use only. Network exposure is a release blocker. | +| Tools | Registered module-backed tools can run in worker threads. Rejection, timeout, cancellation, worker-exit, and queue-bound semantics are incomplete. | Worker isolation is responsiveness, not sandboxing or complete reliability. | +| State | Conversation context is in memory. Compaction retains a small recent window. | No restart recovery, resume/fork, or durable memory contract. | +| Orchestration | Typed plans, DAG scheduling, retry ledgers, and review events exist. Executor routing and terminal completion ownership are incomplete. | Useful prototype and simulation; not yet a durable orchestration service. | +| Tests | Focused Vitest suites, property tests, coverage thresholds, and some real WebSocket tests exist. | Good local foundation; CI must execute it consistently. | +| Delivery | Zettel has deployment automation. SDK packages lack a changelog/version policy, pack-consumer smoke test, and publish workflow. | Do not infer public SDK stability from the current `1.0.0` package versions. | + +## Now — correctness and safety foundation + +1. **Secure and version ADP.** Default standalone servers to loopback; add authenticated handshake, + capabilities/protocol version, method scopes, payload limits, and audit events. Preserve an + explicit insecure-local development mode. +2. **Make run completion deterministic.** Define run IDs and one terminal + `completed | halted | failed` outcome; serialize overlapping runs; ensure tool-bearing turns + settle without caller timing assumptions; make pause and shutdown race-safe. +3. **Bound tool execution.** Validate arguments before dispatch, add fail-closed policy hooks, + queue bounds, per-tool timeout, cancellation, worker replacement, output limits, and typed + failures. Never describe worker threads as a sandbox. +4. **Harden ADP transport.** Ensure one handler produces at most one response; distinguish parse, + invalid-request, and handler errors; reject pending client requests on close/error/timeout. +5. **Make orchestration status truthful.** Route by declared executor role, reject invalid + dependency references and duplicate IDs, ignore stale/foreign events, and define whether + `start()` means accepted or terminal completion. +6. **Establish required PR quality gates.** Build, lint/typecheck, test with coverage, ast-grep rule + tests, a process smoke, and packed-package consumer checks without paid-provider calls. + +## Next — durable, usable contracts + +1. Add an injectable, versioned append-only session store with crash recovery, owner-only file + permissions, redaction/retention policy, and reload tests. +2. Add token-aware compaction with complete tool-turn boundaries and an optional failure-safe + summarizer. Raw durable events remain the source of truth. +3. Replace untyped lifecycle-event consumption with a discriminated event union, stable IDs, + timestamps, documented ordering, and separate observer versus veto-hook semantics. +4. Introduce an executor registry that owns assignment promises, retries, clarification, + cancellation, reviews, and cleanup. +5. Introduce a small injectable provider registry while retaining the current gateway-friendly + routing as a compatibility resolver. +6. Stabilize `@agentx/core` and `@agentx/adp`: package metadata, package READMEs, changelog and + semver policy, tarball import/require smoke tests, provenance, and explicit release approval. +7. Polish the operator path around secure connection profiles, session selection, run state, + tool policy, and confirmation for privileged commands. + +## Later — only after measured demand + +- Multi-process daemon recovery and leases. +- Durable subagent trees and distributed orchestration transports. +- Optional encrypted or remote persistence. +- Broader OAuth/provider-catalog UX for a first-party product. +- Opt-in behavioral evaluations with replay corpora and explicit cost/latency/quality budgets. +- A constrained, versioned extension SDK for trusted host-packaged extensions. + +## Anti-goals + +- Do not recreate a full coding-agent product inside the runtime kernel. +- Do not add Python/IPython or model-generated code execution as the default orchestration model. +- Do not expose ADP remotely before authentication and authorization exist. +- Do not call truncation “durable memory” or make generated summaries the only source of truth. +- Do not retry side-effecting tools without idempotency and visible attempt state. +- Do not run paid-provider tests in ordinary CI. +- Do not add remote telemetry without a product need, consent, redaction, and disable semantics. + +## Release criteria for a flagship SDK + +A public stable release should require all of the following: + +- clean-clone setup succeeds on the pinned toolchain; +- required PR checks pass for build, type-aware lint, unit/integration tests, and coverage; +- ADP remote exposure has an authenticated, scoped protocol and security documentation; +- run/tool/session terminal semantics are deterministic under race and failure tests; +- packed ESM and CommonJS consumers pass in a temporary project; +- package metadata, API docs, changelog, compatibility policy, and release provenance exist; +- examples distinguish deterministic mocks from real-provider tests and document expected output; +- no roadmap-only capability is presented as implemented. diff --git a/mise.toml b/mise.toml index a355a3e..59b88ca 100644 --- a/mise.toml +++ b/mise.toml @@ -1,5 +1,5 @@ [tools] -bun = "latest" -node = "latest" -pnpm = "latest" -turso = "latest" +bun = "1.3.14" +node = "24.14.0" +pnpm = "11.8.0" +turso = "1.0.31"