Research Assistant MVP

Production‑ready local research agent that plans queries, searches the web, fetches pages, extracts readable text, takes notes with inline citations, and synthesizes an answer using a local Ollama model.

Table of Contents

• Research Assistant MVP
  • Table of Contents
  • Overview
  • Requirements
    • Install
    • Quick Start
  • Agents
    • CLI Reference
    • Configuration
      • Example
    • Architecture
    • Extending Agents
    • Development
    • Troubleshooting
    • Notes
  • License

---

Overview

• Fast, deterministic web research via DuckDuckGo HTML results + httpx
• Clean HTML extraction and source deduplication
• LLM‑assisted notes and decision making (Ollama)
• Pluggable agents for focused sources (GitHub, Databricks, AWS, Terraform)
• Polite defaults (timeouts, rate limiting) and testable design (hermetic unit tests)

Requirements

• Python 3.13
• uv (https://github.com/astral-sh/uv)
• An Ollama server (e.g., http://localhost:11434) with a model available
  • Default model: gpt-oss:20b

Install

• Editable install for development:

  uv pip install -e .

Quick Start

• Verify Ollama connectivity:

  uv run research-ast test ollama -m gpt-oss:20b

Agents

• Internet (default): Open‑web research using DuckDuckGo + HTTP

  • Command: research-ast research
  • Example uvrunresearch-astresearch-g"What is Rust programming language?"-mgpt-oss:20b--hosthttp://localhost:11434--max-rounds1-orust_info.md

• GitHub: Summarize recent GitHub updates (RSS)

  • Feed: https://github.blog/changelog/feed/

  • Example:

    uv run research-ast github-changes --days 30 -m gpt-oss:20b --host [http://localhost:11434](http://localhost:11434) -o github-changes.csv

• Databricks: Summarize Databricks release notes (RSS)

  • Feed: https://docs.databricks.com/aws/en/feed.xml

  • Example:

    uv run research-ast databricks-changes --days 30 -m gpt-oss:20b --host [http://localhost:11434](http://localhost:11434) -o databricks-changelog.csv

• AWS: Summarize AWS “What’s New” announcements (RSS)

  • Feed: https://aws.amazon.com/about-aws/whats-new/recent/feed/

  • Example:

    uv run research-ast aws-changes --days 30 -m gpt-oss:20b --host [http://localhost:11434](http://localhost:11434) -o aws-changelog.csv

• Terraform: Summarize Terraform CLI CHANGELOG across minor versions

  • Fetches CHANGELOG.md from version branches >= a starting minor

  • Example:

    uv run research-ast terraform-changes --days 30 -s 1.12 -m gpt-oss:20b --host [http://localhost:11434](http://localhost:11434) -o terraform-changelog.csv

CLI Reference

• research-ast test ollama — health check

  • --model/-m, --host, --prompt, --no-stream

• research-ast answer — open‑web research

  • --goal/-g (required)
  • --model/-m, --host
  • --max-rounds, --queries-per-round, --pages-per-query, --min-confidence
  • --no-color

• research-ast github-changes — GitHub changelog CSV

• research-ast databricks-changes — Databricks release notes CSV

• research-ast aws-changes — AWS “What’s New” CSV

• research-ast terraform-changes — Terraform changes CSV (with Version column)

Configuration

All settings are Pydantic BaseSettings and can be overridden via environment variables prefixed REAST_.

Key settings (see research_ast/core/config.py):

• REAST_MAX_ROUNDS (int, default 3)
• REAST_QUERIES_PER_ROUND (int, default 3)
• REAST_PAGES_PER_QUERY (int, default 3)
• REAST_MIN_CONFIDENCE (float, default 0.65)
• REAST_REQUEST_TIMEOUT_S (float, default 15.0)
• REAST_READ_MAX_CHARS (int, default 35000)
• REAST_USER_AGENT (str, default research-ast/0.1 (+local))
• REAST_SEARCH_URL (str, default DuckDuckGo HTML endpoint)
• REAST_RATE_LIMIT_SLEEP_S (float, default 0.8)
• REAST_OLLAMA_MODEL (str, default gpt-oss:20b)
• REAST_OLLAMA_HOST (str, optional)

Example

export REAST_MAX_ROUNDS=2 export REAST_PAGES_PER_QUERY=2 uv run research-ast answer -g "Recent changes to GitHub Actions cache" -m gpt-oss:20b

Architecture

• research_ast/core — settings, types, and LLM wrapper
• research_ast/web — DuckDuckGo search, HTTP fetch, HTML extraction
• research_ast/summarize — prompts and helpers for notes + decision
• research_ast/agents — base agent and concrete implementations
• Compatibility shim: research_ast/agent.py re‑exports Agent, InternetAgent, run

Extending Agents

Subclass Agent and override minimal hooks:

from research_ast.agents import Agent
from research_ast.core import Settings

class MyAgent(Agent):
    def search(self, query: str, limit: int):
        return [("Title", "https://example.com")]
    def fetch(self, url: str):
        return "<html><body>Example</body></html>"
    def extract(self, html: str):
        return "Example"

agent = MyAgent(Settings())
print(agent.run(goal="Find examples"))

Development

• Install for development:

  uv pip install -e .

• Run tests (see tests/README.md for more examples):

  PYTHONPATH=. uv run python -m pytest

• Formatting: Black is configured (line length 100)

Troubleshooting

• “No module named pytest”: Install and run with uvx or add pytest to your env:
  • uvx pytest -q
  • or uv pip install pytest then uv run -m pytest
• Ollama connection issues: verify server is running and model exists
  • uv run research-ast test ollama -m gpt-oss:20b --host http://localhost:11434

Notes

• Prefer specific goals (entities, dates, locations) for better query planning.
• Keep pages_per_query small and max_rounds modest to be polite to sites.

License

This project is licensed under the Apache License 2.0 - see the LICENSE file for details.