docs: ADR-0003 extension architecture

docs/adr/0003-extension-architecture.md

@@ -0,0 +1,102 @@
+# ADR-0003: Extension Architecture
+
+**Status**: Proposed
+**Date**: 2026-04-28
+**Authors**: Igor Brandao
+**Reviewers**:
+
+## Context
+
+Lola's current Python implementation has hardcoded registries for assistant targets (`TARGETS` dict in `targets/__init__.py`) and source handlers (`SOURCE_HANDLERS` list in `parsers.py`). Adding support for a new AI assistant or a new module source type requires modifying the core codebase. As the AI assistant ecosystem grows rapidly — with new assistants, agent frameworks, skill registries, and security scanning needs emerging regularly — this hardcoded approach does not scale.
+
+We need an extension system that allows any developer to add support for new assistants, catalogs, runtimes, source transports, and security scanners without forking or modifying the Lola core binary.
+
+## Decision
+
+Introduce a formal extension system with 5 extension kinds, YAML manifests, and a language-agnostic communication protocol.
+
+**Extension kinds**:
+
+| Kind | What it extends | Built-in examples |
+|------|----------------|-------------------|
+| `target` | Where skills are installed (assistant file formats and paths) | claude-code, cursor, gemini-cli, openclaw, opencode |
+| `repo` | Where skills are discovered (catalogs and registries) | yaml-catalog, oci-registry |
+| `runtime` | Where skills are executed (agent framework environments) | — (future) |
+| `source` | How skills are fetched (transport protocols) | git, zip, tar, folder, oci |
+| `scan` | How skills are validated (security scanning) | — (future) |
+
+**Extension manifest** (`extension.yaml`):
+
+```yaml
+name: "Windsurf Target"
+kind: target
+description: "Adds Windsurf IDE as a Lola target"
+executable: "lola-ext-windsurf"
+version: "1.0.0"
+author: "Community Member"
+license: "Apache-2.0"
+```
+
+**Built-in vs external**:
+
+- **Built-in extensions** are compiled into the Lola binary. They implement Go interfaces defined in `pkg/sdk/` and are registered at startup via factory maps. No manifest or discovery needed.
+- **External extensions** live in the extension directory (default `~/.lola/extensions/`, configurable) with an `extension.yaml` manifest and an executable binary or script. They communicate with core via stdin/stdout. Any programming language that reads stdin and writes stdout can implement an extension.
+
+**Extension protocol**: The initial protocol uses stdin/stdout for simplicity. The architecture is designed to support gRPC as a future transport option for extensions that need streaming, concurrent calls, or richer type safety. The extension interface abstractions in `pkg/sdk/` are transport-agnostic — switching from stdin/stdout to gRPC would not require changes to extension interfaces, only to the transport layer in `internal/extensions/`.
+
+**Extension management**: `lola ext add|rm|ls|info` manages installed external extensions.
+
+**Extensible kind system**: Adding a new kind in the future requires only: define a new interface in `pkg/sdk/`, add an implementation in `pkg/builtin/`, and register it in the factory map. No core architecture changes.
+
+## Rationale
+
+- **Language-agnostic**: External extensions can be written in bash, Python, Go, or any language
+- **Process isolation**: External extensions run as separate processes, protecting core stability
+- **Clean interface boundaries**: Go interfaces in `pkg/sdk/` define a stable, transport-agnostic contract
+- **Forward-compatible**: New kinds can be added without architectural changes; gRPC transport can be added without changing extension interfaces
+
+## Consequences
+
+### Positive Consequences
+
+- Community can add support for new assistants without forking Lola
+- Custom skill catalogs (enterprise registries, community hubs) are addable as repo extensions
+- Security scanning is pluggable via scan extensions
+- Built-in extensions share the same interface as externals, ensuring consistency
+- Extension directory is configurable for enterprise environments
+- gRPC can be adopted as an additional transport without breaking existing extensions
+
+### Negative Consequences
+
+- External extensions have subprocess overhead compared to compiled built-ins
+- Extension protocol must be versioned to avoid breaking changes
+- Extension discovery adds startup cost (scanning directories)
+
+## Alternatives Considered
+
+### Alternative 1: Hardcoded handlers only
+- Description: Continue adding new targets and sources directly to the core codebase
+- Pros: Simple, no extension infrastructure needed
+- Cons: Every new assistant requires a core release; community cannot contribute independently
+- Reason for rejection: Does not scale with the growing AI assistant ecosystem
+
+### Alternative 2: Shared library plugins (.so files)
+- Description: Extensions as dynamically linked shared libraries
+- Pros: No subprocess overhead, full Go type safety
+- Cons: Not language-agnostic (Go-only), platform-specific binary compatibility issues
+- Reason for rejection: Language-agnostic extensions are a core requirement
+
+### Alternative 3: gRPC as initial protocol
+- Description: Use gRPC from day one instead of stdin/stdout
+- Pros: Strongly typed, supports streaming, concurrent calls, well-established
+- Cons: Higher initial complexity, requires protobuf compilation for extension authors
+- Reason for deferral: Start with stdin/stdout for simplicity; gRPC is a planned future transport. The interface layer is designed to support both.
+
+## Implementation Notes
+
+See paired design document: `docs/dev-guide/design/extension-architecture.md`
+
+## References
+
+- [ADR-0002: Go Migration](0002-go-migration.md) — prerequisite decision
+- [Current Architecture](../dev-guide/architecture.md) — existing SourceHandler strategy pattern that extensions generalize

docs/dev-guide/design/extension-architecture.md

@@ -0,0 +1,213 @@
+# Extension Architecture — Implementation Design
+
+Paired with [ADR-0003: Extension Architecture](../../adr/0003-extension-architecture.md).
+
+## Extension Kind Taxonomy
+
+```mermaid
+graph TD
+    Core["Lola Core"]
+    Core --> Target["kind: target<br/>Where skills are installed"]
+    Core --> Repo["kind: repo<br/>Where skills are discovered"]
+    Core --> Runtime["kind: runtime<br/>Where skills are executed"]
+    Core --> Source["kind: source<br/>How skills are fetched"]
+    Core --> Scan["kind: scan<br/>How skills are validated"]
+
+    Target --> T1["claude-code (built-in)"]
+    Target --> T2["cursor (built-in)"]
+    Target --> T3["gemini-cli (built-in)"]
+    Target --> T4["openclaw (built-in)"]
+    Target --> T5["opencode (built-in)"]
+    Target --> T6["windsurf (external)"]
+
+    Repo --> R1["yaml-catalog (built-in)"]
+    Repo --> R2["oci-registry (built-in)"]
+    Repo --> R3["clawhub (external)"]
+
+    Source --> S1["git (built-in)"]
+    Source --> S2["zip/tar (built-in)"]
+    Source --> S3["folder (built-in)"]
+    Source --> S4["oci (built-in)"]
+    Source --> S5["s3 (external)"]
+```
+
+## Extension Discovery Flow
+
+```mermaid
+flowchart TD
+    Start([Lola startup]) --> BuiltIn[Register built-in extensions<br/>via factory maps]
+    BuiltIn --> ScanDir[Scan extension directory<br/>default: ~/.lola/extensions/]
+    ScanDir --> ForEach{For each subdirectory}
+    ForEach --> ReadManifest[Read extension.yaml]
+    ReadManifest --> Validate{Valid manifest?}
+    Validate -->|yes| Register[Register in extension catalog]
+    Validate -->|no| Warn[Log warning, skip]
+    Register --> ForEach
+    Warn --> ForEach
+    ForEach -->|done| ScanPath[Scan PATH for lola-ext-* binaries]
+    ScanPath --> Ready([Extensions ready])
+```
+
+## Extension Manifest Schema
+
+```yaml
+# Required fields
+name: string          # Display name
+kind: string          # One of: target, repo, runtime, source, scan
+description: string   # Brief description of what this extension does
+executable: string    # Filename of the executable to invoke
+
+# Optional fields
+version: string       # Semantic version
+author: string        # Author name or organization
+license: string       # SPDX license identifier
+```
+
+## Hello World: Bash Target Extension
+
+A minimal target extension that installs skills to a custom directory.
+
+**Directory structure:**
+```text
+~/.lola/extensions/hello-target/
+├── extension.yaml
+└── hello-target.sh
+```
+
+**extension.yaml:**
+```yaml
+name: "Hello Target"
+kind: target
+description: "A simple hello world target extension"
+executable: "hello-target.sh"
+author: "Your Name"
+license: "MIT"
+version: "0.1.0"
+```
+
+**hello-target.sh:**
+
+> **Prerequisites:** requires [`jq`](https://jqlang.github.io/jq/) for JSON parsing.
+
+```bash
+#!/bin/bash
+set -e
+
+input=$(cat)
+action=$(echo "$input" | jq -r '.action')
+
+case "$action" in
+  "install")
+    skill_name=$(echo "$input" | jq -r '.skill_name')
+    dest_path=$(echo "$input" | jq -r '.dest_path')
+    content=$(echo "$input" | jq -r '.content')
+
+    mkdir -p "$dest_path"
+    echo "$content" > "$dest_path/$skill_name.md"
+
+    echo '{"status": "ok", "installed": ["'"$skill_name"'"]}' ;;
+
+  "remove")
+    skill_name=$(echo "$input" | jq -r '.skill_name')
+    dest_path=$(echo "$input" | jq -r '.dest_path')
+    rm -f "$dest_path/$skill_name.md"
+
+    echo '{"status": "ok", "removed": ["'"$skill_name"'"]}' ;;
+
+  "paths")
+    echo '{"skills": ".hello/skills", "commands": ".hello/commands"}' ;;
+
+  *)
+    echo '{"status": "error", "message": "unknown action"}' >&2
+    exit 1 ;;
+esac
+```
+
+**Setup:**
+```bash
+chmod +x ~/.lola/extensions/hello-target/hello-target.sh
+```
+
+**Usage:**
+```bash
+lola ext add ./hello-target/
+lola install my-module -a hello-target
+```
+
+## Hello World: Python Repo Extension
+
+A repo extension providing search and resolve for a custom skill catalog.
+
+**Directory structure:**
+```text
+~/.lola/extensions/my-catalog/
+├── extension.yaml
+└── my-catalog.py
+```
+
+**extension.yaml:**
+```yaml
+name: "My Catalog"
+kind: repo
+description: "Search and install skills from my custom catalog"
+executable: "my-catalog.py"
+version: "0.1.0"
+```
+
+**my-catalog.py:**
+```python
+#!/usr/bin/env python3
+import json
+import sys
+
+CATALOG = [
+    {"name": "react-skills", "version": "2.0", "description": "React development skills",
+     "repository": "https://github.com/example/react-skills.git"},
+    {"name": "security-audit", "version": "1.0", "description": "Security auditing skills",
+     "repository": "https://github.com/example/security-audit.git"},
+]
+
+try:
+    request = json.loads(sys.stdin.read())
+    action = request["action"]
+except (json.JSONDecodeError, KeyError) as e:
+    print(json.dumps({"error": str(e)}), file=sys.stderr)
+    sys.exit(1)
+
+if action == "search":
+    query = request["query"].lower()
+    results = [m for m in CATALOG if query in m["name"] or query in m["description"].lower()]
+    print(json.dumps({"results": results}))
+
+elif action == "resolve":
+    name = request["name"]
+    match = next((m for m in CATALOG if m["name"] == name), None)
+    if match:
+        print(json.dumps(match))
+    else:
+        print(json.dumps({"error": f"module '{name}' not found"}))
+        sys.exit(1)
+
+elif action == "list":
+    print(json.dumps({"results": CATALOG}))
+```
+
+**Setup:**
+```bash
+chmod +x ~/.lola/extensions/my-catalog/my-catalog.py
+```
+
+**Usage:**
+```bash
+lola ext add ./my-catalog/
+lola search react
+lola install react-skills
+```
+
+## Protocol Transport
+
+The initial protocol uses stdin/stdout for simplicity. The architecture supports evolving to gRPC as a future transport option without changing extension interfaces — only the transport layer in `internal/extensions/` would change.
+
+```text
+Core → [write request to stdin] → Extension process → [read response from stdout] → Core
+```