Search, scrape, crawl, map, batch, and get AI answers from the live web — directly from your OpenClaw agent.
Your agent's training data goes stale. This skill gives it the current web: scrape any URL to clean markdown, search Google with structured results, crawl entire documentation sites, batch-process up to 10,000 URLs in parallel, and get AI-synthesized answers with citations.
clawhub install olostepOr manually: clone this repo into your OpenClaw skills directory.
- Get a free API key at olostep.com/auth (500 requests/month, no credit card)
- Set it in your OpenClaw config:
openclaw config set skills.entries.olostep.env.OLOSTEP_API_KEY "your-key-here"Or set the environment variable:
export OLOSTEP_API_KEY="your-key-here"| Tool | What it does |
|---|---|
| Scrape | Extract content from any URL as markdown, HTML, text, or JSON. Full browser rendering, anti-bot bypass, residential proxies. JS-heavy SPAs, CAPTCHAs, geo-restricted pages — handled. |
| Search | Structured Google search results: titles, URLs, snippets, knowledge graph, People Also Ask. |
| Crawl | Follow links from a starting URL and scrape every page. Great for ingesting docs sites, blog archives, product catalogs. |
| Batch | Scrape up to 10,000 URLs in parallel. All pages rendered concurrently with full anti-bot protection. |
| Map | Discover every URL on a website instantly, filterable by glob patterns and search query. |
| Answers | Ask a question, get an AI-synthesized answer from live web sources — with citations and optional structured JSON output. |
Debugging an error — Search for the exact error message across GitHub issues and StackOverflow, then scrape the most relevant thread for the full fix.
Writing code from docs — Scrape the actual API documentation and write working integration code from what is really there, not from stale training data.
Research and comparisons — Get structured, cited answers about pricing, features, tech stacks, or market data. Define a JSON shape and get it filled with live web data.
Ingesting a knowledge base — Map a docs site to discover all URLs, then batch-scrape or crawl the relevant sections into clean markdown for RAG pipelines.
Framework migration — Scrape the migration guide and changelog, understand breaking changes, and refactor code based on real documentation.
1. Search Google → find authoritative sources
2. Scrape the top results → get full content as markdown
3. Synthesize into a deliverable
1. Map the docs site → discover all page URLs
2. Batch or Crawl the relevant sections
3. Retrieve clean markdown content
1. Search the exact error message (in quotes)
2. Scrape the GitHub issue or StackOverflow answer
3. Apply the fix
1. Map the site to find all product/listing URLs
2. Batch scrape with a parser for structured JSON
3. Process and store the results
- Olostep API Docs
- Get API Key (free, 30 seconds)
- OpenClaw Docs
- ClawHub
- Full ClawHub Plugin (13 skills + MCP server)
MIT — see LICENSE for details.