diff --git a/docs/adr/0003-extension-architecture.md b/docs/adr/0003-extension-architecture.md new file mode 100644 index 0000000..8333343 --- /dev/null +++ b/docs/adr/0003-extension-architecture.md @@ -0,0 +1,102 @@ +# ADR-0003: Extension Architecture + +**Status**: Proposed +**Date**: 2026-04-28 +**Authors**: Igor Brandao +**Reviewers**: + +## Context + +Lola's current Python implementation has hardcoded registries for assistant targets (`TARGETS` dict in `targets/__init__.py`) and source handlers (`SOURCE_HANDLERS` list in `parsers.py`). Adding support for a new AI assistant or a new module source type requires modifying the core codebase. As the AI assistant ecosystem grows rapidly — with new assistants, agent frameworks, skill registries, and security scanning needs emerging regularly — this hardcoded approach does not scale. + +We need an extension system that allows any developer to add support for new assistants, catalogs, runtimes, source transports, and security scanners without forking or modifying the Lola core binary. + +## Decision + +Introduce a formal extension system with 5 extension kinds, YAML manifests, and a language-agnostic communication protocol. + +**Extension kinds**: + +| Kind | What it extends | Built-in examples | +|------|----------------|-------------------| +| `target` | Where skills are installed (assistant file formats and paths) | claude-code, cursor, gemini-cli, openclaw, opencode | +| `repo` | Where skills are discovered (catalogs and registries) | yaml-catalog, oci-registry | +| `runtime` | Where skills are executed (agent framework environments) | — (future) | +| `source` | How skills are fetched (transport protocols) | git, zip, tar, folder, oci | +| `scan` | How skills are validated (security scanning) | — (future) | + +**Extension manifest** (`extension.yaml`): + +```yaml +name: "Windsurf Target" +kind: target +description: "Adds Windsurf IDE as a Lola target" +executable: "lola-ext-windsurf" +version: "1.0.0" +author: "Community Member" +license: "Apache-2.0" +``` + +**Built-in vs external**: + +- **Built-in extensions** are compiled into the Lola binary. They implement Go interfaces defined in `pkg/sdk/` and are registered at startup via factory maps. No manifest or discovery needed. +- **External extensions** live in the extension directory (default `~/.lola/extensions/`, configurable) with an `extension.yaml` manifest and an executable binary or script. They communicate with core via stdin/stdout. Any programming language that reads stdin and writes stdout can implement an extension. + +**Extension protocol**: The initial protocol uses stdin/stdout for simplicity. The architecture is designed to support gRPC as a future transport option for extensions that need streaming, concurrent calls, or richer type safety. The extension interface abstractions in `pkg/sdk/` are transport-agnostic — switching from stdin/stdout to gRPC would not require changes to extension interfaces, only to the transport layer in `internal/extensions/`. + +**Extension management**: `lola ext add|rm|ls|info` manages installed external extensions. + +**Extensible kind system**: Adding a new kind in the future requires only: define a new interface in `pkg/sdk/`, add an implementation in `pkg/builtin/`, and register it in the factory map. No core architecture changes. + +## Rationale + +- **Language-agnostic**: External extensions can be written in bash, Python, Go, or any language +- **Process isolation**: External extensions run as separate processes, protecting core stability +- **Clean interface boundaries**: Go interfaces in `pkg/sdk/` define a stable, transport-agnostic contract +- **Forward-compatible**: New kinds can be added without architectural changes; gRPC transport can be added without changing extension interfaces + +## Consequences + +### Positive Consequences + +- Community can add support for new assistants without forking Lola +- Custom skill catalogs (enterprise registries, community hubs) are addable as repo extensions +- Security scanning is pluggable via scan extensions +- Built-in extensions share the same interface as externals, ensuring consistency +- Extension directory is configurable for enterprise environments +- gRPC can be adopted as an additional transport without breaking existing extensions + +### Negative Consequences + +- External extensions have subprocess overhead compared to compiled built-ins +- Extension protocol must be versioned to avoid breaking changes +- Extension discovery adds startup cost (scanning directories) + +## Alternatives Considered + +### Alternative 1: Hardcoded handlers only +- Description: Continue adding new targets and sources directly to the core codebase +- Pros: Simple, no extension infrastructure needed +- Cons: Every new assistant requires a core release; community cannot contribute independently +- Reason for rejection: Does not scale with the growing AI assistant ecosystem + +### Alternative 2: Shared library plugins (.so files) +- Description: Extensions as dynamically linked shared libraries +- Pros: No subprocess overhead, full Go type safety +- Cons: Not language-agnostic (Go-only), platform-specific binary compatibility issues +- Reason for rejection: Language-agnostic extensions are a core requirement + +### Alternative 3: gRPC as initial protocol +- Description: Use gRPC from day one instead of stdin/stdout +- Pros: Strongly typed, supports streaming, concurrent calls, well-established +- Cons: Higher initial complexity, requires protobuf compilation for extension authors +- Reason for deferral: Start with stdin/stdout for simplicity; gRPC is a planned future transport. The interface layer is designed to support both. + +## Implementation Notes + +See paired design document: `docs/dev-guide/design/extension-architecture.md` + +## References + +- [ADR-0002: Go Migration](0002-go-migration.md) — prerequisite decision +- [Current Architecture](../dev-guide/architecture.md) — existing SourceHandler strategy pattern that extensions generalize diff --git a/docs/dev-guide/design/extension-architecture.md b/docs/dev-guide/design/extension-architecture.md new file mode 100644 index 0000000..789c62e --- /dev/null +++ b/docs/dev-guide/design/extension-architecture.md @@ -0,0 +1,213 @@ +# Extension Architecture — Implementation Design + +Paired with [ADR-0003: Extension Architecture](../../adr/0003-extension-architecture.md). + +## Extension Kind Taxonomy + +```mermaid +graph TD + Core["Lola Core"] + Core --> Target["kind: target
Where skills are installed"] + Core --> Repo["kind: repo
Where skills are discovered"] + Core --> Runtime["kind: runtime
Where skills are executed"] + Core --> Source["kind: source
How skills are fetched"] + Core --> Scan["kind: scan
How skills are validated"] + + Target --> T1["claude-code (built-in)"] + Target --> T2["cursor (built-in)"] + Target --> T3["gemini-cli (built-in)"] + Target --> T4["openclaw (built-in)"] + Target --> T5["opencode (built-in)"] + Target --> T6["windsurf (external)"] + + Repo --> R1["yaml-catalog (built-in)"] + Repo --> R2["oci-registry (built-in)"] + Repo --> R3["clawhub (external)"] + + Source --> S1["git (built-in)"] + Source --> S2["zip/tar (built-in)"] + Source --> S3["folder (built-in)"] + Source --> S4["oci (built-in)"] + Source --> S5["s3 (external)"] +``` + +## Extension Discovery Flow + +```mermaid +flowchart TD + Start([Lola startup]) --> BuiltIn[Register built-in extensions
via factory maps] + BuiltIn --> ScanDir[Scan extension directory
default: ~/.lola/extensions/] + ScanDir --> ForEach{For each subdirectory} + ForEach --> ReadManifest[Read extension.yaml] + ReadManifest --> Validate{Valid manifest?} + Validate -->|yes| Register[Register in extension catalog] + Validate -->|no| Warn[Log warning, skip] + Register --> ForEach + Warn --> ForEach + ForEach -->|done| ScanPath[Scan PATH for lola-ext-* binaries] + ScanPath --> Ready([Extensions ready]) +``` + +## Extension Manifest Schema + +```yaml +# Required fields +name: string # Display name +kind: string # One of: target, repo, runtime, source, scan +description: string # Brief description of what this extension does +executable: string # Filename of the executable to invoke + +# Optional fields +version: string # Semantic version +author: string # Author name or organization +license: string # SPDX license identifier +``` + +## Hello World: Bash Target Extension + +A minimal target extension that installs skills to a custom directory. + +**Directory structure:** +```text +~/.lola/extensions/hello-target/ +├── extension.yaml +└── hello-target.sh +``` + +**extension.yaml:** +```yaml +name: "Hello Target" +kind: target +description: "A simple hello world target extension" +executable: "hello-target.sh" +author: "Your Name" +license: "MIT" +version: "0.1.0" +``` + +**hello-target.sh:** + +> **Prerequisites:** requires [`jq`](https://jqlang.github.io/jq/) for JSON parsing. + +```bash +#!/bin/bash +set -e + +input=$(cat) +action=$(echo "$input" | jq -r '.action') + +case "$action" in + "install") + skill_name=$(echo "$input" | jq -r '.skill_name') + dest_path=$(echo "$input" | jq -r '.dest_path') + content=$(echo "$input" | jq -r '.content') + + mkdir -p "$dest_path" + echo "$content" > "$dest_path/$skill_name.md" + + echo '{"status": "ok", "installed": ["'"$skill_name"'"]}' ;; + + "remove") + skill_name=$(echo "$input" | jq -r '.skill_name') + dest_path=$(echo "$input" | jq -r '.dest_path') + rm -f "$dest_path/$skill_name.md" + + echo '{"status": "ok", "removed": ["'"$skill_name"'"]}' ;; + + "paths") + echo '{"skills": ".hello/skills", "commands": ".hello/commands"}' ;; + + *) + echo '{"status": "error", "message": "unknown action"}' >&2 + exit 1 ;; +esac +``` + +**Setup:** +```bash +chmod +x ~/.lola/extensions/hello-target/hello-target.sh +``` + +**Usage:** +```bash +lola ext add ./hello-target/ +lola install my-module -a hello-target +``` + +## Hello World: Python Repo Extension + +A repo extension providing search and resolve for a custom skill catalog. + +**Directory structure:** +```text +~/.lola/extensions/my-catalog/ +├── extension.yaml +└── my-catalog.py +``` + +**extension.yaml:** +```yaml +name: "My Catalog" +kind: repo +description: "Search and install skills from my custom catalog" +executable: "my-catalog.py" +version: "0.1.0" +``` + +**my-catalog.py:** +```python +#!/usr/bin/env python3 +import json +import sys + +CATALOG = [ + {"name": "react-skills", "version": "2.0", "description": "React development skills", + "repository": "https://github.com/example/react-skills.git"}, + {"name": "security-audit", "version": "1.0", "description": "Security auditing skills", + "repository": "https://github.com/example/security-audit.git"}, +] + +try: + request = json.loads(sys.stdin.read()) + action = request["action"] +except (json.JSONDecodeError, KeyError) as e: + print(json.dumps({"error": str(e)}), file=sys.stderr) + sys.exit(1) + +if action == "search": + query = request["query"].lower() + results = [m for m in CATALOG if query in m["name"] or query in m["description"].lower()] + print(json.dumps({"results": results})) + +elif action == "resolve": + name = request["name"] + match = next((m for m in CATALOG if m["name"] == name), None) + if match: + print(json.dumps(match)) + else: + print(json.dumps({"error": f"module '{name}' not found"})) + sys.exit(1) + +elif action == "list": + print(json.dumps({"results": CATALOG})) +``` + +**Setup:** +```bash +chmod +x ~/.lola/extensions/my-catalog/my-catalog.py +``` + +**Usage:** +```bash +lola ext add ./my-catalog/ +lola search react +lola install react-skills +``` + +## Protocol Transport + +The initial protocol uses stdin/stdout for simplicity. The architecture supports evolving to gRPC as a future transport option without changing extension interfaces — only the transport layer in `internal/extensions/` would change. + +```text +Core → [write request to stdin] → Extension process → [read response from stdout] → Core +```