Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion skills/answers/SKILL.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
---
name: answers
name: olostep-answers
category: usage
description: Get AI-powered answers with citations from live web data using Olostep. Use when the user needs up-to-date facts, competitive intelligence, pricing comparisons, structured data from the web, or wants web-sourced answers in a specific JSON shape they can use directly in code or docs.
---
Expand Down
2 changes: 1 addition & 1 deletion skills/batch/SKILL.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
---
name: batch
name: olostep-batch
category: usage
description: Scrape up to 10,000 URLs in parallel using Olostep. Use when the user has a list of URLs to extract content from at once — product pages, job listings, pricing pages, changelogs, docs, or any set of known pages. Much faster than scraping one-by-one.
---
Expand Down
2 changes: 1 addition & 1 deletion skills/crawl/SKILL.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
---
name: crawl
name: olostep-crawl
category: usage
description: Autonomously crawl an entire website by following links from a starting URL using Olostep. Use when the user wants to ingest an entire docs site, blog, knowledge base, or any multi-page site as context for coding, writing, analysis, or building a knowledge base.
---
Expand Down
2 changes: 1 addition & 1 deletion skills/debug-error/SKILL.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
---
name: debug-error
name: olostep-debug-error
category: workflow
description: Search the web for a specific error message or bug using Olostep. Use when the user pastes an error stack trace, says their code is failing, or asks how to fix a bug that requires looking up recent GitHub issues, StackOverflow threads, or framework documentation.
---
Expand Down
2 changes: 1 addition & 1 deletion skills/docs-to-code/SKILL.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
---
name: docs-to-code
name: olostep-docs-to-code
category: workflow
description: Scrape API documentation or library docs and use them to write working, up-to-date code. Use when the user wants to integrate a third-party API, learn a new library, generate SDK usage examples, write code based on a docs URL, or update code after a library version change. This is the flagship workflow — never hallucinate API parameters when you can scrape the real docs.
---
Expand Down
2 changes: 1 addition & 1 deletion skills/extract-schema/SKILL.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
---
name: extract-schema
name: olostep-extract-schema
category: workflow
description: Scrape a webpage with Olostep and extract specific structured data matching a TypeScript interface, JSON schema, or database model. Use when the user wants to turn a website (like a product list, directory, or article) into clean, structured JSON or seed data.
---
Expand Down
2 changes: 1 addition & 1 deletion skills/integrate/SKILL.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
---
name: integrate
name: olostep-integrate
category: build
description: Automatically integrate the Olostep SDK into the user's codebase. Analyzes the project, detects the language and framework, chooses the right integration pattern, installs the SDK, writes all the code, and verifies it works — with minimal prompting. Use when the user wants to add Olostep web scraping, search, or AI answers to their project.
argument-hint: "API-KEY"
Expand Down
2 changes: 1 addition & 1 deletion skills/map/SKILL.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
---
name: map
name: olostep-map
category: usage
description: Discover and list all URLs on a website using Olostep. Use when the user wants to see what pages exist on a site, find specific sections (docs, blog, API reference), audit a site's structure, or get a URL list to feed into batch or crawl.
---
Expand Down
2 changes: 1 addition & 1 deletion skills/migrate-code/SKILL.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
---
name: migrate-code
name: olostep-migrate-code
category: workflow
description: Scrape a migration guide, changelog, or breaking changes document using Olostep and automatically update the user's local code. Use when the user is upgrading a framework, moving to a new API version, or says "help me migrate this code based on these docs".
---
Expand Down
2 changes: 1 addition & 1 deletion skills/research/SKILL.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
---
name: research
name: olostep-research
category: workflow
description: Do thorough, cited web research to help the user make a decision. Compare tools, libraries, or services. Evaluate competitors. Research tech stacks, market data, or architecture patterns. Use when the user needs a well-sourced answer before making a technical, product, or business decision.
---
Expand Down
2 changes: 1 addition & 1 deletion skills/scrape/SKILL.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
---
name: scrape
name: olostep-scrape
category: usage
description: Scrape a single webpage and extract its content as clean markdown, HTML, JSON, or text using Olostep. Use when the user shares a URL and wants to read it, extract content from it, understand its structure, use it as context for coding, or turn a page into data. Handles JavaScript-rendered pages, SPAs, and bot-protected sites automatically.
---
Expand Down
2 changes: 1 addition & 1 deletion skills/search/SKILL.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
---
name: search
name: olostep-search
category: usage
description: Search the web using Olostep for live, up-to-date information. Three tools available — answers (AI-synthesised answers with citations), google_search (structured Google results), and get_website_urls (find pages within a specific site). Use when the user needs current data, comparisons, or facts the AI's training data may not have.
---
Expand Down
2 changes: 1 addition & 1 deletion skills/setup/SKILL.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
---
name: setup
name: olostep-setup
category: build
description: Configure the Olostep MCP server so all Olostep skills work correctly. Use when the user is setting up Olostep for the first time, has no MCP server configured, or is getting authentication errors. Supports both the hosted endpoint (recommended) and a local stdio install.
---
Expand Down
13 changes: 5 additions & 8 deletions src/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ import { registerMcpUninstall } from "./commands/mcp-uninstall.js";
import { registerListMcp } from "./commands/list-mcp.js";
import { registerDoctor } from "./commands/doctor.js";

import { maybeNotifyUpdate } from "./lib/version-check.js";
import { showPendingUpdateNotice, scheduleUpdateCheck } from "./lib/version-check.js";

// Replaced at build time by tsup's `define` with the package.json version.
// The fallback only fires under `npm run dev` (tsx) where define doesn't apply.
Expand Down Expand Up @@ -81,15 +81,12 @@ async function main(): Promise<void> {
registerMcpUninstall(mcpCmd);
registerListMcp(listCmd);

// Best-effort update notice — runs in parallel with the command,
// never blocks command output, never throws.
const notifyPromise = maybeNotifyUpdate(VERSION, process.argv[2]);
// Show any pending update notice from the previous check (appears before command output).
showPendingUpdateNotice(VERSION, process.argv[2]);
// Fire the registry check for the next run — no await, never blocks.
scheduleUpdateCheck(VERSION, process.argv[2]);

await program.parseAsync(process.argv);

// Brief window for the update notice to land before exit. It's
// background work; we don't wait forever.
await Promise.race([notifyPromise, new Promise((r) => setTimeout(r, 500))]);
}

main().catch((err: any) => {
Expand Down
13 changes: 9 additions & 4 deletions src/lib/skills.ts
Original file line number Diff line number Diff line change
Expand Up @@ -489,13 +489,18 @@ export function runInstall(opts: InstallOptions): InstallResult {

const discovered = discoverSkills(source);

const include = new Set((opts.only ?? []).map((x) => sanitizeName(x)));
const exclude = new Set((opts.exclude ?? []).map((x) => sanitizeName(x)));
// Strip the olostep- prefix before comparing so both `--skill search`
// and `--skill olostep-search` resolve to the same entry.
const stripPrefix = (x: string) =>
x.startsWith(SKILL_FOLDER_PREFIX) ? x.slice(SKILL_FOLDER_PREFIX.length) : x;

const include = new Set((opts.only ?? []).map((x) => stripPrefix(sanitizeName(x))));
const exclude = new Set((opts.exclude ?? []).map((x) => stripPrefix(sanitizeName(x))));
const category = opts.category as SkillCategory | undefined;

const selected = discovered.filter((s) => {
if (include.size && !include.has(s.sanitizedName)) return false;
if (exclude.has(s.sanitizedName)) return false;
if (include.size && !include.has(stripPrefix(s.sanitizedName))) return false;
if (exclude.has(stripPrefix(s.sanitizedName))) return false;
if (category && s.category !== category) return false;
return true;
});
Expand Down
153 changes: 153 additions & 0 deletions src/lib/version-check.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -9,9 +9,12 @@ import {
isNewer,
maybeNotifyUpdate,
parseVersion,
scheduleUpdateCheck,
showPendingUpdateNotice,
} from "./version-check.js";

const CACHE_FILE = path.join(os.tmpdir(), "olostep-cli-version-cache.json");
const NOTICE_FILE = path.join(os.tmpdir(), "olostep-cli-update-notice.json");

function clearCache() {
try { fs.unlinkSync(CACHE_FILE); } catch { /* ignore */ }
Expand Down Expand Up @@ -211,3 +214,153 @@ describe("maybeNotifyUpdate", () => {
await expect(maybeNotifyUpdate("1.0.0")).resolves.toBeUndefined();
});
});

describe("showPendingUpdateNotice", () => {
const ORIGINAL_TTY = (process.stderr as any).isTTY;

function clearNotice() {
try { fs.unlinkSync(NOTICE_FILE); } catch { /* ignore */ }
}

function writeNotice(data: object) {
fs.writeFileSync(NOTICE_FILE, JSON.stringify(data), "utf8");
}

beforeEach(() => {
clearNotice();
delete process.env.OLOSTEP_NO_UPDATE_NOTICE;
delete process.env.OLOSTEP_NO_UPDATE_CHECK;
});

afterEach(() => {
(process.stderr as any).isTTY = ORIGINAL_TTY;
delete process.env.OLOSTEP_NO_UPDATE_NOTICE;
delete process.env.OLOSTEP_NO_UPDATE_CHECK;
clearNotice();
});

it("prints to stderr when a newer notice file exists", () => {
(process.stderr as any).isTTY = true;
writeNotice({ latest: "9.9.9" });
const chunks: string[] = [];
const orig = process.stderr.write.bind(process.stderr);
process.stderr.write = (chunk: any) => { chunks.push(String(chunk)); return true; };
showPendingUpdateNotice("1.0.0");
process.stderr.write = orig;
expect(chunks.join("")).toContain("9.9.9");
expect(fs.existsSync(NOTICE_FILE)).toBe(false);
});

it("does nothing when no notice file exists", () => {
(process.stderr as any).isTTY = true;
const chunks: string[] = [];
const orig = process.stderr.write.bind(process.stderr);
process.stderr.write = (chunk: any) => { chunks.push(String(chunk)); return true; };
showPendingUpdateNotice("1.0.0");
process.stderr.write = orig;
expect(chunks).toHaveLength(0);
});

it("deletes the notice file even when version is not newer", () => {
(process.stderr as any).isTTY = true;
writeNotice({ latest: "0.0.1" });
showPendingUpdateNotice("1.0.0");
expect(fs.existsSync(NOTICE_FILE)).toBe(false);
});

it("skips when stderr is not a TTY", () => {
(process.stderr as any).isTTY = false;
writeNotice({ latest: "9.9.9" });
const chunks: string[] = [];
const orig = process.stderr.write.bind(process.stderr);
process.stderr.write = (chunk: any) => { chunks.push(String(chunk)); return true; };
showPendingUpdateNotice("1.0.0");
process.stderr.write = orig;
expect(chunks).toHaveLength(0);
// Notice file should NOT be deleted when we skip (TTY check)
expect(fs.existsSync(NOTICE_FILE)).toBe(true);
});

it("skips when OLOSTEP_NO_UPDATE_CHECK is set", () => {
(process.stderr as any).isTTY = true;
process.env.OLOSTEP_NO_UPDATE_CHECK = "1";
writeNotice({ latest: "9.9.9" });
showPendingUpdateNotice("1.0.0");
expect(fs.existsSync(NOTICE_FILE)).toBe(true);
});

it("skips when invokedSubcommand === 'update'", () => {
(process.stderr as any).isTTY = true;
writeNotice({ latest: "9.9.9" });
showPendingUpdateNotice("1.0.0", "update");
expect(fs.existsSync(NOTICE_FILE)).toBe(true);
});

it("never throws on malformed notice file", () => {
(process.stderr as any).isTTY = true;
fs.writeFileSync(NOTICE_FILE, "not-json", "utf8");
expect(() => showPendingUpdateNotice("1.0.0")).not.toThrow();
});
});

describe("scheduleUpdateCheck", () => {
function clearNotice() {
try { fs.unlinkSync(NOTICE_FILE); } catch { /* ignore */ }
}

beforeEach(() => {
clearCache();
clearNotice();
delete process.env.OLOSTEP_NO_UPDATE_NOTICE;
delete process.env.OLOSTEP_NO_UPDATE_CHECK;
});

afterEach(() => {
vi.unstubAllGlobals();
delete process.env.OLOSTEP_NO_UPDATE_NOTICE;
delete process.env.OLOSTEP_NO_UPDATE_CHECK;
clearCache();
clearNotice();
});

it("writes a notice file when a newer version is found", async () => {
fs.writeFileSync(
CACHE_FILE,
JSON.stringify({ latest: "9.9.9", checkedAt: Date.now() }),
);
scheduleUpdateCheck("1.0.0");
// scheduleUpdateCheck is fire-and-forget; wait for the microtask to settle
await new Promise((r) => setTimeout(r, 50));
expect(fs.existsSync(NOTICE_FILE)).toBe(true);
const data = JSON.parse(fs.readFileSync(NOTICE_FILE, "utf8"));
expect(data.latest).toBe("9.9.9");
});

it("does not write a notice file when already on latest", async () => {
fs.writeFileSync(
CACHE_FILE,
JSON.stringify({ latest: "1.0.0", checkedAt: Date.now() }),
);
scheduleUpdateCheck("1.0.0");
await new Promise((r) => setTimeout(r, 50));
expect(fs.existsSync(NOTICE_FILE)).toBe(false);
});

it("skips when OLOSTEP_NO_UPDATE_CHECK is set", async () => {
process.env.OLOSTEP_NO_UPDATE_CHECK = "1";
const fetchSpy = vi.fn();
vi.stubGlobal("fetch", fetchSpy);
scheduleUpdateCheck("1.0.0");
await new Promise((r) => setTimeout(r, 50));
expect(fetchSpy).not.toHaveBeenCalled();
expect(fs.existsSync(NOTICE_FILE)).toBe(false);
});

it("skips when invokedSubcommand === 'update'", async () => {
const fetchSpy = vi.fn();
vi.stubGlobal("fetch", fetchSpy);
scheduleUpdateCheck("1.0.0", "update");
await new Promise((r) => setTimeout(r, 50));
expect(fetchSpy).not.toHaveBeenCalled();
});
});
61 changes: 54 additions & 7 deletions src/lib/version-check.ts
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@ import * as path from "node:path";
import { c } from "./colors.js";

const CACHE_FILE = path.join(os.tmpdir(), "olostep-cli-version-cache.json");
const NOTICE_FILE = path.join(os.tmpdir(), "olostep-cli-update-notice.json");
const CACHE_TTL_MS = 24 * 60 * 60 * 1000; // 24h
const REGISTRY_URL = "https://registry.npmjs.org/olostep-cli/latest";

Expand Down Expand Up @@ -79,23 +80,69 @@ export async function checkForUpdate(current: string): Promise<string | null> {
return isNewer(latest, current) ? latest : null;
}

function isSuppressed(subcommand?: string): boolean {
if (subcommand === "update") return true;
if (process.env.OLOSTEP_NO_UPDATE_CHECK) return true;
if (process.env.OLOSTEP_NO_UPDATE_NOTICE) return true;
return false;
}

/**
* Shows any pending update notice written by a previous scheduleUpdateCheck call.
* Call at process start (before the command) so the notice lands before output.
* Deletes the notice file after reading it. Fail-silent.
*/
export function showPendingUpdateNotice(current: string, invokedSubcommand?: string): void {
if (isSuppressed(invokedSubcommand)) return;
try {
if (!process.stderr.isTTY) return;
} catch { return; }
try {
if (!fs.existsSync(NOTICE_FILE)) return;
const raw = fs.readFileSync(NOTICE_FILE, "utf8");
// Delete regardless of whether we can show it.
try { fs.unlinkSync(NOTICE_FILE); } catch { /* ignore */ }
const data = JSON.parse(raw);
const latest = (data?.latest || "").toString().trim();
if (!latest || !isNewer(latest, current)) return;
process.stderr.write(
c.yellow(` ↑ olostep ${latest} available — run \`olostep update\`\n`),
);
} catch { /* fail silent */ }
}

/**
* Fires a background registry check with no await. If a newer version is found,
* writes a notice file so showPendingUpdateNotice displays it on the next run.
* Never blocks the command. Fail-silent.
*/
export function scheduleUpdateCheck(current: string, invokedSubcommand?: string): void {
if (isSuppressed(invokedSubcommand)) return;
checkForUpdate(current)
.then((latest) => {
if (latest) {
try {
fs.writeFileSync(NOTICE_FILE, JSON.stringify({ latest }), "utf8");
} catch { /* ignore */ }
}
})
.catch(() => { /* ignore */ });
}

/**
* Prints a one-line "update available" notice to stderr — interactive use only.
* Suppressed in pipes/CI and when OLOSTEP_NO_UPDATE_NOTICE is set.
* Never throws.
* Legacy combined helper. Kept for tests that import it directly.
* New code should use showPendingUpdateNotice + scheduleUpdateCheck.
*/
export async function maybeNotifyUpdate(current: string, invokedSubcommand?: string): Promise<void> {
if (invokedSubcommand === "update") return;
if (process.env.OLOSTEP_NO_UPDATE_CHECK) return;
if (process.env.OLOSTEP_NO_UPDATE_NOTICE) return;
if (isSuppressed(invokedSubcommand)) return;
try {
if (!process.stderr.isTTY) return;
} catch { return; }
try {
const latest = await checkForUpdate(current);
if (latest) {
process.stderr.write(
c.yellow(` ↑ olostep ${latest} is available (you have ${current}) — run \`olostep update\`\n`),
c.yellow(` ↑ olostep ${latest} available — run \`olostep update\`\n`),
);
}
} catch { /* fail silent */ }
Expand Down
Loading