Research Toolkit — Autonomous Research CLI for AI Agents

A production-grade CLI toolkit that enables AI agents to autonomously research, store, retrieve, and refresh up-to-date information for any software project.

Quick Start

# 1. Install
pip install -e ".[dev]"

# 2. Configure (optional — works without API keys using DuckDuckGo + extractive summaries)
cp .env.example .env
# Edit .env to add your OPENAI_API_KEY for AI-powered summaries

# 3. Run your first research query
research query "Python 3.13 new features" --depth 3 --project python-updates

# 4. Check library status
research status

# 5. Generate onboarding docs for an agent
research onboard --agent-name MyAgent

Architecture

This project follows Clean Architecture (Robert C. Martin) with strict dependency inversion:

┌─────────────────────────────────────────────┐
│          interfaces/cli/  (Typer app)       │  ← Outermost
├─────────────────────────────────────────────┤
│     infrastructure/  (adapters, I/O)        │
├─────────────────────────────────────────────┤
│     application/  (use-cases, orchestration)│
├─────────────────────────────────────────────┤
│     domain/  (entities, ports, services)    │  ← Innermost
└─────────────────────────────────────────────┘

Dependency direction: outer → inner only (via abstract ports).

Layer Responsibilities

Layer	Purpose	Dependencies
domain/	Entities, value objects, interfaces (ports), pure business rules	None
application/	Use-cases that orchestrate domain logic through ports	Domain only
infrastructure/	Concrete adapters (web search, OpenAI, filesystem, SQLite)	Domain ports
interfaces/cli/	Typer CLI commands, composition root (dependency wiring)	All layers

Commands

Command	Description
research query "<topic>"	Search, summarise, and store results
research refresh <project>	Update stale items in a project
research cite <item_id>	Get citation for a stored item
research index rebuild	Rebuild the search index
research status	Show library summary
research help	Display usage guide
research onboard --agent-name <name>	Generate onboarding docs

All commands support --json for machine-readable output.

Research Library Structure

research_library/
  {project}/
    {YYYY-MM-DD}/
      sources/       ← Raw fetched artefacts
      notes/         ← Normalised summaries
      citations/     ← Citation snippets
      index.json     ← Searchable metadata
      manifest.json  ← Run metadata

Testing

make test          # Run all tests
make test-unit     # Domain + use-case tests
make test-integration  # Full cycle tests
make test-cli      # CLI behaviour tests
make lint          # Ruff + Black checks
make coverage      # Coverage report

Security

• API keys loaded from .env (never committed)
• Log output passes through RedactingFilter — secrets are replaced with ***REDACTED***
• Settings.__repr__ redacts all key/secret/token fields
• No hardcoded credentials anywhere in the codebase

Known Limitations

1. DuckDuckGo rate limiting — Aggressive queries may be throttled; the Serper adapter is more reliable for high-volume use.
2. Content extraction — Raw HTML is fetched; a proper readability parser (e.g. trafilatura) would improve summary quality.
3. SQLite FTS — The full-text index is opt-in and basic; a vector-embedding index would enable semantic search.
4. Offline mode — No offline research capability; could be added by indexing local documents.
5. Concurrent queries — Use-cases run sequentially within a single query; parallel fetching would improve throughput.

Next-Step Improvements

• Add trafilatura for clean content extraction from HTML
• Vector embeddings for semantic search across the library
• Parallel fetch/summarise pipeline with asyncio.gather
• Export to BibTeX / RIS citation formats
• Web UI dashboard for browsing the research library
• Plugin system for custom search providers