Revive the Atomic Forge as a shadcn-style tool registry

.claude/.codebase-info/.map-state.json

@@ -1,9 +1,9 @@
 {
   "schemaVersion": 1,
   "tool": "codebase-mapper",
-  "version": "2.14.4",
-  "mappedAt": "2026-08-11",
-  "gitCommit": "c80c7520a703018d36abc97f2199dcaf3d9c29a3",
+  "version": "2.15.5",
+  "mappedAt": "2026-08-20",
+  "gitCommit": "4746cbdba91333b9c65af47175636e90cb18e487",
   "documents": [
     "architecture.md",
     "coding-style.md",
@@ -14,19 +14,21 @@
     "modules.md",
     "onboarding.md",
     "patterns.md",
+    "structure.md",
     "tech-landscape.md"
   ],
   "hashes": {
-    "INDEX.md": "50e7beb7784d3d01f8ce0e85b842889f5c033ca98cbb30981fd0b532c1495c8f",
-    "architecture.md": "8e6be8d2795231ea5919d7cdc23a201595f9fed9724b635e6cc686b2bf24581f",
+    "INDEX.md": "6160cf20e5a9ae015ff14993171ddc21c7b9f649bd4218e99e90e1b867b6873f",
+    "architecture.md": "8e5e9fc1f7eb7ef5fc97c9072eb2f35d6e3c464cdba65d505c7850082cf1ed84",
     "coding-style.md": "b63e8dff06d16a2d882f02624c62f490291317aec8b3032f6d79b3ee035e8567",
-    "communication.md": "6c73cad50151b73a6be63c8f0745d3f7a6653a861d37afce660c512ac30b8819",
-    "dependencies.md": "b445a920c121e27006fec824add6865d01cf71212a517e5ff8b2058d3999037a",
-    "directory-structure.md": "ca200817789c4d1fdc41d1683c37740c9837bf323b18881cb2351ef8a56c4926",
-    "entry-points.md": "62fd8ff478fc27c3658776626059bdd4101cab12a74a48cdd8787f9ca7b83f97",
-    "modules.md": "7a8772e5f31be0070003c9a7ce78eca5df95dfb6a1c9dd471a6be97df81084aa",
-    "onboarding.md": "3d28c671dc095a2aadb297bd7ab200c677779a99c7534e03e55a3d64c935f07e",
-    "patterns.md": "f3a339a7bc711c4586062139d135e1d79129653cf82fc430cd256028d25e2dcc",
-    "tech-landscape.md": "700b3ba7bb5e47ec14086e2b94ad65ad956c0bfbe1c463f96c39493143e0c912"
+    "communication.md": "1d335d8cc977ec65cba5b4d77fb95939a25f66046792a9df1b52e8ea2f38dbd8",
+    "dependencies.md": "34d9d3441f33d0c1082c899764e626032c24337baff090c55fc85b727f0d4330",
+    "directory-structure.md": "5ac97b0ee815073b85d22db09ef3693cd6dd784176053481afeb346278e573ee",
+    "entry-points.md": "61c97480846155a567b97b633d5254a0a513d1305e9f70b74f87ba66e132d555",
+    "modules.md": "a78b88b9a63af5060c5b58f4173c55890f45a04aa3b75d9bf8f8388d5185e15a",
+    "onboarding.md": "ccc30bd73115d603afa7dc1b39344174542f3142a7dde25182fb94fad3e63ded",
+    "patterns.md": "a5fea9892789c24cda012c494c33b3ed7de7f8e59e411ba590165b82bde6e573",
+    "structure.md": "a6d1f09a2b0f5aa3d68ec8f008244acf6cf032050670e9dcf76c8812ab8467b1",
+    "tech-landscape.md": "47009c38224a2e1c7f5f843166566d0f408b91bb4761a3259b4af1edb9f8ff41"
   }
 }

.claude/.codebase-info/INDEX.md

@@ -1,14 +1,15 @@
 # Codebase Map — Atomic Agents
 
-*Last Updated: 2026-08-11*
+*Last Updated: 2026-08-20*
 
 Atomic Agents is a lightweight, modular Python framework for building agentic AI applications as
 composable, schema-driven building blocks (built on Instructor + Pydantic). This repository is a
-`uv`-workspace **monorepo**: the core framework, a TUI tool installer, a tool library, and examples.
+`uv`-workspace **monorepo**: the core framework, a Forge CLI client, a vendored-code tool registry, and
+examples.
 
 **Stack:** Python ≥3.12 · Instructor · Pydantic v2 · LiteLLM · MCP · Textual · uv + Hatchling · Pyright · `VideoURL` multimodal content
 **Shape:** Monorepo — `atomic-agents/` (core lib) · `atomic-assembler/` (CLI) · `atomic-forge/` (tools) · `atomic-examples/` (examples)
-**Package:** `atomic-agents` v2.9.1 on PyPI · core import package is `atomic_agents`
+**Package:** `atomic-agents` v2.10.1 on PyPI · core import package is `atomic_agents`
 
 ## Documents
 
@@ -24,6 +25,7 @@ composable, schema-driven building blocks (built on Instructor + Pydantic). This
 | [patterns.md](./patterns.md) | Atomicity, schema-driven I/O, context providers, testing |
 | [coding-style.md](./coding-style.md) | Formatting, linting, naming conventions |
 | [onboarding.md](./onboarding.md) | Setup with uv, tests, docs, common tasks |
+| [structure.md](./structure.md) | Layout intent: where new code goes, non-obvious conventions |
 
 ## How to use this map
 

.claude/.codebase-info/architecture.md

@@ -1,6 +1,6 @@
 # Architecture
 
-*Last Updated: 2026-08-11*
+*Last Updated: 2026-08-20*
 
 ## Summary
 
@@ -14,10 +14,10 @@ abstractions: the developer controls the system prompt, the conversation history
 The repository is a **monorepo** managed as a `uv` workspace with four parts:
 
 - **`atomic-agents/`** — the core framework (PyPI package `atomic-agents`, imported as `atomic_agents`).
-- **`atomic-assembler/`** — a Textual TUI (the `atomic` command) that downloads tools from the forge
-  into a user's project.
-- **`atomic-forge/`** — a library of 13 standalone, copy-into-your-project tools.
-- **`atomic-examples/`** — 15 runnable example applications.
+- **`atomic-assembler/`** — the `atomic` command: a Textual TUI plus a noninteractive CLI that
+  vendors tool packages from one or more configured Forge sources into a user's project.
+- **`atomic-forge/`** — a vendored-code registry of 13 standalone, copy-into-your-project tools.
+- **`atomic-examples/`** — 16 runnable example applications.
 
 ## High-Level Flow
 
@@ -37,8 +37,14 @@ Agent run (atomic_agents core):
                                   ├──► appended to ChatHistory
                                   └──► returned to caller
 
-Tool install (atomic-assembler TUI):
-  `atomic`  →  clone github.com/eigenwise/atomic-agents  →  copy atomic-forge/tools/<tool>  →  user project
+Tool install (atomic-assembler):
+  `atomic list` / `atomic download <source>/<tool>`
+        │
+        ├─► read ~/.atomic-assembler/sources.json (defaults to the official Forge)
+        ├─► git clone each source at its branch  →  read <tools-path>/../index.json
+        ├─► validate index paths stay inside the tools directory, reject unsafe symlinks
+        └─► copy the complete tool package  →  user project (they now own the code)
+  `atomic` with no subcommand opens the TUI instead.
 ```
 
 ## Components
@@ -50,8 +56,9 @@ Tool install (atomic-assembler TUI):
 | Context | `atomic-agents/atomic_agents/context/` | `SystemPromptGenerator`, `BaseDynamicContextProvider`, `BaseChatHistory` (pluggable memory contract) + `ChatHistory` with Instructor media and `VideoURL` |
 | Connectors | `atomic-agents/atomic_agents/connectors/mcp/` | Model Context Protocol tools / resources / prompts |
 | Utils | `atomic-agents/atomic_agents/utils/` | Token counting (LiteLLM), tool-message formatting |
-| Assembler (CLI) | `atomic-assembler/atomic_assembler/` | Textual TUI to fetch/install forge tools |
+| Assembler (CLI) | `atomic-assembler/atomic_assembler/` | `atomic` TUI + CLI to list and vendor Forge tools from configured Git sources |
 | Forge (tools) | `atomic-forge/tools/` | 13 standalone tools, each `BaseTool`-based |
+| Forge registry | `atomic-forge/index.json`, `scripts/`, `conformance/` | generated catalog, its deterministic generator, and the package-contract suite |
 
 ## The Agent Run Lifecycle
 
@@ -75,6 +82,9 @@ Tool install (atomic-assembler TUI):
 - **Python ≥3.12** — uses PEP 695 generic syntax (`AtomicAgent[In, Out]`).
 - **Tools are not a dependency.** Forge tools are *copied* into the user's repo (full control, no
   version lock-in) by the assembler, rather than pip-installed.
+- **A Forge source is untrusted input.** The assembler validates index paths and symlinks before
+  copying, and never stores or prompts for credentials — private sources authenticate through the
+  user's existing Git SSH keys or credential helper.
 - **No database and no containers** in the framework itself.
 - **v2 was a breaking change** from v1 — see `UPGRADE_DOC.md` (`BaseAgent`→`AtomicAgent`,
   `AgentMemory`→`ChatHistory`, flattened imports, schemas moved to generic type parameters).

.claude/.codebase-info/communication.md

@@ -1,6 +1,6 @@
 # Communication & Integrations
 
-*Last Updated: 2026-08-11*
+*Last Updated: 2026-08-20*
 
 The framework exposes no HTTP API of its own; "communication" here means how it talks to LLM
 providers and external tools.
@@ -34,8 +34,22 @@ providers and external tools.
   `clear_hooks`. Events include `parse:error`, `completion:kwargs`, `completion:response`,
   `token:counted`. See the `hooks-example` example and `docs/guides/hooks.md`.
 
-## Assembler ↔ GitHub
-- `atomic-assembler` clones `https://github.com/eigenwise/atomic-agents.git` (via GitPython), reads
-  `atomic-forge/tools/`, and copies the chosen tool into the user's project (skipping build files).
-  Source URL and paths are in `atomic-assembler/atomic_assembler/constants.py` and `utils.py`
-  (`GithubRepoCloner`, `AtomicToolManager`).
+## Assembler ↔ Forge sources (Git)
+- A **Forge source** is a Git repository, branch, and tools directory (`ForgeSource` in
+  `atomic-assembler/atomic_assembler/constants.py`). Sources are stored in
+  `~/.atomic-assembler/sources.json` and default to the official
+  `https://github.com/eigenwise/atomic-agents.git` on `main` when that file is absent.
+- Per source, `GithubRepoCloner` clones into a temp directory via GitPython, then
+  `AtomicToolManager.get_indexed_forge_tools` reads the generated `index.json` beside the tools
+  directory and `copy_atomic_tool_to_destination` copies the selected package into the user's project
+  (skipping `.coveragerc` and `uv.lock`). Both live in `utils.py`.
+- **Authentication is out of scope by design.** The assembler stores, prompts for, and prints no
+  credentials; private sources rely on the user's Git SSH keys or credential helper.
+  `validate_source_url` rejects source URLs carrying userinfo, query strings, or fragments, and
+  `redact_source_url` masks them in any output.
+- **An index is untrusted input.** Tool paths must be relative, free of `..`, and resolve inside the
+  configured tools directory; symlinks escaping the tool directory are refused before the copy; and
+  every string reaching the terminal passes through `display_safe_text`, which strips ANSI/OSC escape
+  sequences and control characters.
+- `atomic list` reports per-source failures on stderr but still prints healthy sources, exiting
+  nonzero only when every configured source fails.

.claude/.codebase-info/dependencies.md

@@ -1,6 +1,6 @@
 # Dependencies
 
-*Last Updated: 2026-06-13*
+*Last Updated: 2026-08-20*
 
 Declared in `pyproject.toml` (Hatchling build, `uv` workspace; versions locked in `uv.lock`).
 
@@ -18,7 +18,7 @@ Declared in `pyproject.toml` (Hatchling build, `uv` workspace; versions locked i
 | textual (≥5.3,<6) | TUI framework for the `atomic` assembler |
 | mcp[cli] (≥1.6) | Model Context Protocol client + CLI |
 | requests (≥2.32,<3) | HTTP |
-| gitpython (≥3.1.43,<4) | Clone the repo to fetch forge tools |
+| gitpython (≥3.1.43,<4) | Clone each configured Forge source to fetch tools |
 | pyyaml (≥6,<7) | Read tool `config.yaml` metadata |
 | pyfiglet (≥1,<2) | ASCII-art banners in the TUI |
 

.claude/.codebase-info/directory-structure.md

@@ -1,6 +1,6 @@
 # Directory Structure
 
-*Last Updated: 2026-08-11*
+*Last Updated: 2026-08-20*
 
 ## Root Layout
 
@@ -14,13 +14,16 @@ atomic-agents/                  # repo root (uv workspace)
 │       ├── connectors/mcp/     #     Model Context Protocol integration
 │       └── utils/              #     token counter, tool-message formatting
 │   └── tests/                  #   pytest suite (agents/, base/, context/, connectors/, utils/; VideoURL tests in base/)
-├── atomic-assembler/           # Textual TUI (`atomic` command) to install forge tools
-│   └── atomic_assembler/       #   main.py, app.py, screens/, widgets/, utils.py, constants.py
-├── atomic-forge/               # library of standalone tools (NOT a package)
-│   ├── tools/<tool>/           #   one folder per tool: tool/<tool>.py, tests/, pyproject.toml
-│   └── guides/                 #   tool authoring guides (e.g. tool_structure.md)
+├── atomic-assembler/           # Textual TUI + noninteractive `atomic` Forge client
+│   └── atomic_assembler/       #   main.py, source/index/download utilities, TUI screens/widgets
+├── atomic-forge/               # vendored-code tool registry (NOT a package)
+│   ├── tools/<tool>/           #   standalone package: tool/, tests/, pyproject.toml, requirements.txt
+│   ├── conformance/            #   registry package-contract test suite
+│   ├── scripts/                #   deterministic index generator
+│   ├── index.json              #   generated tool catalog
+│   └── guides/                 #   tool authoring guides
 ├── atomic-examples/            # 16 runnable example apps (each its own project)
-├── claude-plugin/atomic-agents/ # AI-assistant plugin: 7 skills + 2 subagents (Claude Code plugin,
+├── claude-plugin/atomic-agents/ # AI-assistant plugin: 8 skills + 2 subagents (Claude Code plugin,
 │                               #   also installable cross-tool via `npx skills add eigenwise/atomic-agents`)
 ├── .claude-plugin/             # marketplace.json — plugin marketplace manifest (drives npx skills discovery)
 ├── docs/                       # Sphinx + MyST documentation (api/, guides/, examples/)
@@ -43,15 +46,16 @@ prompts and stores conversation history. `connectors/mcp/` bridges to MCP server
 token accounting via LiteLLM.
 
 ### `atomic-assembler/atomic_assembler/`
-A Textual terminal UI launched by the `atomic` command (`main.py:main`). `app.py` routes between
-`screens/` (main menu, tool explorer, file picker, README viewer); `utils.py` clones the GitHub repo
-and copies a selected tool into the user's project.
-
-### `atomic-forge/tools/`
-13 self-contained tools (`arxiv_search`, `calculator`, `tavily_search`, `weather`,
-`webpage_scraper`, `wikipedia_search`, …). Each tool folder contains `tool/<name>.py` (Input/Output
-`BaseIOSchema` + a `BaseToolConfig` + a `BaseTool` subclass), `tests/`, and its own
-`pyproject.toml`/`requirements.txt`. Tools are copied into user projects, not pip-installed.
+The `atomic` entry point runs the Textual UI with no subcommand and supports scripting with `list`,
+`download`, and `sources` subcommands. Source utilities clone configured Git repositories, resolve an
+indexed package only inside the configured tools directory, and copy the full standalone package into the
+user’s project. Source/index metadata is treated as untrusted before it reaches the terminal or filesystem.
+
+### `atomic-forge/`
+A shadcn-style collection of 13 self-contained tool packages. Each package contains `tool/<name>.py`
+(Input/Output `BaseIOSchema`, `BaseToolConfig`, `BaseTool`), tests, `pyproject.toml`, and
+`requirements.txt`; those files stay with the package when it is downloaded. `index.json` is generated
+from each package’s metadata, and `conformance/` plus CI enforce the registry contract.
 
 ### `atomic-examples/`
 16 standalone example apps (`quickstart`, `rag-chatbot`, `deep-research`, `web-search-agent`,

.claude/.codebase-info/entry-points.md

@@ -25,7 +25,12 @@ result = agent.run(BasicChatInputSchema(chat_message="Hello"))
 
 | Entry | Type | Purpose | File |
 |-------|------|---------|------|
-| `atomic` | Textual TUI | Browse & install forge tools into a project | `atomic-assembler/atomic_assembler/main.py:main` |
+| `atomic` | Textual TUI + command-line interface | Browse and install vendored Forge tools | `atomic-assembler/atomic_assembler/main.py:main` |
+
+The noninteractive surface is `atomic list`, `atomic download <name> [--dest DIR]`, and
+`atomic sources list|add|remove`. Sources are Git repositories, including private ones accessed through
+normal Git SSH or credential-helper setup. A qualified name such as `company/weather` resolves a collision
+between source catalogs. The TUI remains available when no subcommand is supplied.
 
 Declared in `pyproject.toml` (`[project.scripts]  atomic = "atomic_assembler.main:main"`). Flags:
 `--enable-logging`, `--version`.

.claude/.codebase-info/modules.md

@@ -48,18 +48,20 @@
 
 ### atomic-assembler (CLI)
 - **Location:** `atomic-assembler/atomic_assembler/`
-- **Purpose:** Textual TUI to browse and install forge tools into a user project.
-- **Key files:** `main.py` (`main()`; argparse `--enable-logging`, `--version`), `app.py`
-  (`AtomicAssembler(App)`), `screens/` (`main_menu`, `atomic_tool_explorer`, `file_explorer`,
-  `tool_info_screen`), `widgets/`, `utils.py` (`GithubRepoCloner`, `AtomicToolManager`),
-  `constants.py` (GitHub URL, `TOOLS_SUBFOLDER`).
+- **Purpose:** Textual TUI plus the `atomic` command-line client for vendoring Forge tool packages.
+- **Key files:** `main.py` (argparse commands and TUI entry), `app.py` (`AtomicAssembler(App)`),
+  `screens/`, `widgets/`, `utils.py` (source clone/index resolution/download), `constants.py`
+  (`ForgeSource`, source validation and display safety).
+- **Source model:** configured sources live in `~/.atomic-assembler/sources.json`; each source names a
+  Git URL, branch, and tools directory. Git authentication stays with SSH or the user’s credential helper.
 
 ### atomic-forge (tools)
-- **Location:** `atomic-forge/tools/`
-- **Purpose:** 13 standalone tools, each an independent mini-project following the `BaseTool` pattern,
-  copied into user projects by the assembler (build files such as `pyproject.toml` / `requirements.txt`
-  / `uv.lock` are skipped on copy).
-- **Authoring guide:** `atomic-forge/guides/tool_structure.md`.
+- **Location:** `atomic-forge/`
+- **Purpose:** a vendored-code registry of 13 standalone tools. `index.json` is generated from package
+  metadata by `scripts/generate_index.py`; `conformance/` verifies package structure and metadata.
+  CI requires a fresh index and runs every tool’s test suite.
+- **Authoring guide:** `atomic-forge/guides/tool_structure.md`; user-facing workflow:
+  `docs/guides/atomic_forge.md`.
 
 ### atomic-examples
 - **Location:** `atomic-examples/`

.claude/.codebase-info/onboarding.md

@@ -1,6 +1,6 @@
 # Onboarding
 
-*Last Updated: 2026-06-13*
+*Last Updated: 2026-07-23*
 
 ## Prerequisites
 - Python **≥3.12**
@@ -22,6 +22,8 @@ it. To launch the tool-installer TUI: `uv run atomic` (or `atomic` once it's on
 |---------|---------|
 | `uv sync` | Install/update all workspace dependencies |
 | `uv run pytest --cov=atomic_agents atomic-agents` | Run the core test suite with coverage |
+| `uv run pytest atomic-assembler` | Run Atomic Assembler CLI and source-handling tests |
+| `uv run pytest atomic-forge/conformance` | Verify Forge package and registry contract |
 | `uv run black --check atomic-agents atomic-assembler atomic-examples atomic-forge` | Format check |
 | `uv run flake8 --extend-exclude=.venv atomic-agents atomic-assembler atomic-examples atomic-forge` | Lint |
 | `cd docs && uv run make html` | Build the Sphinx docs |
@@ -30,9 +32,12 @@ it. To launch the tool-installer TUI: `uv run atomic` (or `atomic` once it's on
 ## Common tasks
 - **Build a new agent:** define input/output `BaseIOSchema` subclasses, wrap an LLM client with
   Instructor, pass it to `AtomicAgent[In, Out](AgentConfig(...))`. See `patterns.md` + `entry-points.md`.
+- **Use a Forge tool:** run `atomic list`, then `atomic download <name>` (or
+  `atomic download <source>/<name>` for a collision). Add private Git registries with
+  `atomic sources add <name> <url> --tools-path tools`.
 - **Add a forge tool:** create `atomic-forge/tools/<name>/` following
-  `atomic-forge/guides/tool_structure.md` (input/output schemas, `BaseToolConfig`, a `BaseTool`
-  subclass, `tests/`).
+  `atomic-forge/guides/tool_structure.md`, give it tests and dependency metadata, regenerate
+  `atomic-forge/index.json`, then run the conformance suite.
 - **Add an example:** create `atomic-examples/<name>/` with its own `pyproject.toml`.
 - **Release:** `build_and_deploy.ps1 <major|minor|patch>` (needs `PYPI_TOKEN`).