diff --git a/README.md b/README.md index 425e482..2a34bed 100644 --- a/README.md +++ b/README.md @@ -1,13 +1,6 @@ # VES AI -Make product analytics actionable for AI agents. - -**Core differentiator:** VES AI closes the product improvement loop by making product analytics data actionable for AI agents. - -VES AI is local-first: -- You run the CLI on your machine. -- You use your own PostHog + Google Cloud. -- You keep outputs in `~/.vesai/workspace` as durable, git-friendly artifacts. +VES AI is a local-first session replay analysis runtime for agent workflows. ## Install @@ -15,12 +8,11 @@ VES AI is local-first: curl -fsSL https://ves.ai/install | bash ``` -Installer flow: -1. Clone/update repo at `~/.vesai/app/vesai` -2. Install dependencies with Bun -3. Install Playwright Chromium -4. Link `vesai` at `~/.local/bin/vesai` -5. Start `vesai quickstart` +The installer clones VES AI to `~/.vesai/app/vesai`, installs dependencies, links `vesai`, and runs `vesai quickstart`. + +## Auto Update + +Every `vesai` command syncs the installed CLI with the latest `origin/main` before execution. ## Prerequisites @@ -29,7 +21,7 @@ Installer flow: - `gcloud` - `ffmpeg` -Before quickstart: +Before `quickstart`: ```bash gcloud auth login @@ -37,163 +29,149 @@ gcloud auth application-default login gcloud config set project ``` -If `gcloud` throws `unsupported hash type blake2b` / `blake2s`: +## Setup Model -```bash -export CLOUDSDK_PYTHON=/usr/bin/python3 -``` +VES AI now has two setup steps: + +1. `vesai quickstart` configures global machine-level runtime. +2. `vesai init` configures the current repository as a VES AI project. -## Quickstart +### 1) Global setup (`vesai quickstart`) ```bash vesai quickstart ``` -Quickstart configures: -- PostHog host + User API key -- PostHog project selection -- PostHog group key -- Replay domain filter -- GCP project + Vertex region -- GCS bucket create/select -- Render concurrency (defaults to ~50% of available RAM, ~512MB per renderer) -- Product description context for analysis prompts +This sets: +- Global Google Cloud + Vertex config +- Global bucket config for rendered artifacts +- Global machine render memory budget (`runtime.maxRenderMemoryMb`) +- Local render runtime dependencies (Chromium) -PostHog API key requirements: -- Key type: User API key -- Scope: `All access + MCP server scope` -- URL: `https://app.posthog.com/settings/user-api-keys` +Render services scale dynamically up/down based on current free RAM, capped by your configured memory budget. -Non-interactive usage: +Example: ```bash -vesai quickstart \ - --non-interactive \ - --posthog-api-key phx_... \ - --posthog-project-id 123 \ - --posthog-group-key organization \ - --domain-filter app.example.com \ - --product-description "B2B SaaS for support teams" +vesai quickstart --max-render-memory-mb 8192 ``` -## CLI Surface +Global config is stored in `~/.vesai/core.json`. + +### 2) Project setup (`vesai init`) -Replay intelligence: +Run this inside your product repo: ```bash -vesai replays session -vesai replays user -vesai replays group -vesai replays query "" -vesai replays list +vesai init ``` -PostHog analytics intelligence: +This creates and configures project-local artifacts: +- `.vesai/project.json` +- `.vesai/workspace/{sessions,users,groups,research}` +- `.vesai/jobs`, `.vesai/cache`, `.vesai/logs` + +`vesai init` also: +- Generates a UUID `projectId` by default (or uses `--project-id`) +- Prompts for PostHog project settings +- Prompts for `lookbackDays` (default `180`) +- Adds `.vesai/` to repository `.gitignore` +- Throws a descriptive error if `.gitignore` cannot be updated (locked/read-only) + +Example: ```bash -vesai events -vesai properties -vesai schema data -vesai schema warehouse -vesai insights hogql "" -vesai insights sql "" -vesai errors list -vesai logs query --from ... --to ... +vesai init --lookback-days 180 ``` -Agent mode patterns: +## CLI Surface ```bash -vesai replays query --group acme --min-active 30 --dry-run -vesai replays query --group acme --min-active 30 -vesai insights sql "SELECT event, count() FROM events GROUP BY event LIMIT 20" -``` +vesai user +vesai group +vesai research "" -JSON is default for data commands. Use `--no-json` for human-readable summaries. +vesai daemon start +vesai daemon watch +vesai daemon status +vesai daemon stop -## Replay Querying Notes +vesai quickstart +vesai init +vesai config show +vesai doctor +``` -`vesai replays query "checkout friction"` is **literal metadata search** plus filters. It does not infer intent from language on its own. +Data commands return JSON by default. Use `--no-json` for readable text. -For strong signal, pair text with structured filters: +Examples: ```bash -vesai replays query "checkout" --url /checkout --min-active 30 --from 2026-02-01 --to 2026-02-15 -vesai replays query --group acme --where plan=enterprise --url /checkout +vesai user bryce@company.com +vesai group acme-co +vesai research "What drives checkout abandonment?" ``` -## User Analysis Contract - -`vesai replays user ` does: -1. Find all matching sessions for the user -2. Ensure every session is rendered to video -3. Analyze each session individually -4. Run one aggregate Gemini call across all session analyses + metadata -5. Write comprehensive user story markdown to workspace - -## Filesystem Layout - -```text -~/.vesai/ - vesai.json - cache/ - logs/ - tmp/ - workspace/ - sessions/ - users/ - groups/ - app/ - vesai/ -``` +## Command Behavior -## Daemon Commands +### `vesai user ` +- Finds all sessions for the user +- Ensures those sessions are rendered and analyzed +- Produces one aggregate user story -```bash -vesai daemon start # background -vesai daemon watch # foreground (Ctrl+C to stop) -vesai daemon status -vesai daemon stop -``` +### `vesai group ` +- Resolves users under the group ID (PostHog group key mapping) +- Builds each user story from all their sessions +- Produces one aggregate group story -## Troubleshooting +### `vesai research ""` +- Uses only already analyzed sessions in `.vesai/workspace/sessions` +- Selects relevant sessions as context +- Sends context to Gemini and returns a research answer -### Bucket location errors +## Daemon Model -If bucket creation fails with invalid location constraint, use a valid location: -- Multi-region: `US`, `EU`, `ASIA` -- Or supported region like `us-central1` +`vesai daemon` is project-scoped and runs against the current repo’s `.vesai` directory. -### Permission mismatch (`storage.objects.create` denied) +Behavior: +- First run performs backfill from `now - lookbackDays` to now. +- Heartbeat then continuously pulls sessions from `lastPulledAt` to now. +- New sessions are queued for render + analysis. +- After session jobs complete, affected user and group stories are re-run. -Common cause: ADC identity differs from `gcloud auth list` active account. +## Global vs Project Separation -Reset ADC: +### Global core (`~/.vesai`) +- Machine-level memory budget (`maxRenderMemoryMb`) +- Render service/runtime dependencies +- GCloud bucket/project/model settings +- Shared cross-process render slot locks (`~/.vesai/render-locks`) -```bash -gcloud auth application-default revoke -gcloud auth application-default login -gcloud auth application-default set-quota-project -``` +### Project-local (`/.vesai`) +- Project UUID +- PostHog API key/project/domain/group config +- Session/user/group/research markdown artifacts +- Daemon state, job queue, cache, logs -### Missing Playwright executable +## Storage Layout in GCS -```bash -bunx playwright install chromium -``` +Rendered artifacts are prefixed by VES AI project UUID: -### Vertex model access errors (`gemini-3-pro-preview` not found) +- `projects//events/.json` +- `projects//videos/.webm` -- Verify Vertex AI API is enabled on the selected project -- Verify selected region supports the configured model -- Update config if needed: +This keeps all project artifacts isolated under top-level project folders in one bucket. + +## Config Commands ```bash -vesai config set vertex.model gemini-3-pro-preview -vesai config set vertex.location us-central1 +vesai config show +vesai config validate +vesai config set core.runtime.maxRenderMemoryMb 8192 +vesai config set project.daemon.lookbackDays 180 ``` -Run `vesai doctor` to confirm local setup state. +Use `core.` paths for global config and `project.` paths for repo-local config. ## Development @@ -202,17 +180,4 @@ bun install bun run lint bun run typecheck bun run test -bun run vesai -- --help -``` - -Website: - -```bash -bun run website:dev -bun run website:build -bun run website:start ``` - -Quality gates: -- Husky pre-commit: `bun run precommit` -- CI: `.github/workflows/ci.yml` runs the same `lint + typecheck + test` diff --git a/bin/vesai b/bin/vesai index 885b12d..7bb6728 100755 --- a/bin/vesai +++ b/bin/vesai @@ -1,5 +1,67 @@ #!/usr/bin/env bash set -euo pipefail ROOT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)" -cd "$ROOT_DIR" -exec bun cli/index.ts "$@" + +resolve_realpath_or_empty() { + local path="$1" + if [ -d "$path" ]; then + (cd "$path" && pwd -P) + else + echo "" + fi +} + +sync_latest_main() { + if ! command -v git >/dev/null 2>&1; then + echo "VES AI auto-update requires git." >&2 + return 1 + fi + + if ! git -C "$ROOT_DIR" rev-parse --is-inside-work-tree >/dev/null 2>&1; then + return 0 + fi + + local vesai_home="${VESAI_HOME:-$HOME/.vesai}" + local installed_root + installed_root="$(resolve_realpath_or_empty "$vesai_home/app/vesai")" + local repo_root + repo_root="$(resolve_realpath_or_empty "$ROOT_DIR")" + local is_installed_repo="0" + if [ -n "$installed_root" ] && [ "$repo_root" = "$installed_root" ]; then + is_installed_repo="1" + fi + + local lock_dir="$ROOT_DIR/.git/vesai-auto-update.lock" + local lock_wait_attempts=0 + until mkdir "$lock_dir" 2>/dev/null; do + lock_wait_attempts=$((lock_wait_attempts + 1)) + if [ "$lock_wait_attempts" -ge 300 ]; then + echo "VES AI auto-update lock timed out." >&2 + return 1 + fi + sleep 0.1 + done + + trap 'rmdir "$lock_dir" >/dev/null 2>&1 || true' RETURN + + git -C "$ROOT_DIR" fetch --quiet --depth 1 origin main + + if [ "$is_installed_repo" = "1" ]; then + git -C "$ROOT_DIR" checkout --quiet main + git -C "$ROOT_DIR" reset --quiet --hard origin/main + else + local current_branch + current_branch="$(git -C "$ROOT_DIR" symbolic-ref --short -q HEAD || true)" + if [ "$current_branch" = "main" ] && + git -C "$ROOT_DIR" diff --quiet && + git -C "$ROOT_DIR" diff --cached --quiet; then + git -C "$ROOT_DIR" merge --quiet --ff-only origin/main + fi + fi + + trap - RETURN + rmdir "$lock_dir" >/dev/null 2>&1 || true +} + +sync_latest_main +exec bun "$ROOT_DIR/cli/index.ts" "$@" diff --git a/cli/analysis/group.ts b/cli/analysis/group.ts index 7cf617c..8248d38 100644 --- a/cli/analysis/group.ts +++ b/cli/analysis/group.ts @@ -18,6 +18,8 @@ export async function analyzeGroupById(params: { groupId: string; usersAnalyzed: number; score: number; + story: string; + health: string; markdownPath: string; }> { const recordings = await findRecordingsByGroupId({ @@ -72,8 +74,8 @@ export async function analyzeGroupById(params: { users: userResults.map((user) => ({ email: user.email, sessions: user.sessionCount, - story: `User ${user.email} comprehensive story generated.`, - health: `Average session score ${user.averageSessionScore}`, + story: user.story, + health: user.health, score: user.userScore, })), }); @@ -97,6 +99,8 @@ export async function analyzeGroupById(params: { groupId: params.groupId, usersAnalyzed: userResults.length, score: aggregate.score, + story: aggregate.story, + health: aggregate.health, markdownPath, }; } diff --git a/cli/analysis/index.ts b/cli/analysis/index.ts index 4f5881d..f5221b2 100644 --- a/cli/analysis/index.ts +++ b/cli/analysis/index.ts @@ -1,5 +1,5 @@ export * from "./group"; -export * from "./query"; +export * from "./research"; export * from "./session"; export * from "./types"; export * from "./user"; diff --git a/cli/analysis/query.ts b/cli/analysis/query.ts deleted file mode 100644 index b67eaa1..0000000 --- a/cli/analysis/query.ts +++ /dev/null @@ -1,46 +0,0 @@ -import { findRecordingsByQuery } from "../../connectors"; -import type { PostHogQueryFilters } from "../../connectors/posthog"; -import type { SessionBatchProgressCallback } from "./session"; -import { renderAndAnalyzeSessions, summarizeSessionResults } from "./session"; -import type { CoreContext } from "./types"; - -export async function analyzeQuery(params: { - query?: string; - filters?: PostHogQueryFilters; - context: CoreContext; - sessionConcurrency?: number; - onSessionProgress?: SessionBatchProgressCallback; -}): Promise<{ - query: string; - sessionCount: number; - averageScore: number; - filters?: PostHogQueryFilters; -}> { - const recordings = await findRecordingsByQuery({ - host: params.context.config.posthog.host, - apiKey: params.context.config.posthog.apiKey, - projectId: params.context.config.posthog.projectId, - query: params.query, - filters: params.filters, - domainFilter: params.context.config.posthog.domainFilter, - }); - - if (!recordings.length) { - throw new Error("No sessions found for the provided query filters."); - } - - const sessions = await renderAndAnalyzeSessions({ - recordings, - context: params.context, - concurrency: params.sessionConcurrency, - onProgress: params.onSessionProgress, - }); - const summary = summarizeSessionResults(sessions); - - return { - query: params.query || "", - sessionCount: summary.count, - averageScore: summary.averageScore, - filters: params.filters, - }; -} diff --git a/cli/analysis/research.ts b/cli/analysis/research.ts new file mode 100644 index 0000000..0c328ce --- /dev/null +++ b/cli/analysis/research.ts @@ -0,0 +1,198 @@ +import { readdir, readFile } from "node:fs/promises"; +import { getVesaiPaths } from "../../config"; +import { answerResearchQuestion, createVertexClient } from "../../connectors"; +import type { CoreContext } from "./types"; + +type SessionResearchDocument = { + sessionId: string; + score: number | null; + startTime: string | null; + markdownPath: string; + text: string; +}; + +export type ResearchResult = { + question: string; + answer: string; + findings: string[]; + confidence: "low" | "medium" | "high"; + supportingSessionIds: string[]; + sessionsConsidered: number; + sessionsUsed: number; +}; + +function tokenize(value: string): string[] { + return Array.from( + new Set( + value + .toLowerCase() + .split(/[^a-z0-9]+/g) + .map((part) => part.trim()) + .filter((part) => part.length >= 3) + ) + ); +} + +function parseFrontmatter(content: string): Record { + const match = content.match(/^---\n([\s\S]*?)\n---/); + if (!match) { + return {}; + } + + const out: Record = {}; + for (const line of match[1].split("\n")) { + const idx = line.indexOf(":"); + if (idx <= 0) { + continue; + } + const key = line.slice(0, idx).trim(); + const raw = line.slice(idx + 1).trim(); + if (!key) { + continue; + } + + try { + out[key] = JSON.parse(raw); + } catch { + out[key] = raw; + } + } + + return out; +} + +function scoreDocument(questionTerms: string[], content: string): number { + const normalized = content.toLowerCase(); + let score = 0; + for (const term of questionTerms) { + if (normalized.includes(term)) { + score += 1; + } + } + return score; +} + +async function loadSessionResearchDocuments( + homeDir?: string +): Promise { + const paths = getVesaiPaths(homeDir); + const files = await readdir(paths.sessionsDir).catch(() => []); + const markdownFiles = files.filter((name) => name.endsWith(".md")); + + const docs = await Promise.all( + markdownFiles.map(async (fileName) => { + const markdownPath = `${paths.sessionsDir}/${fileName}`; + const content = await readFile(markdownPath, "utf8"); + const frontmatter = parseFrontmatter(content); + const sessionId = String( + frontmatter.session_id ?? frontmatter.id ?? fileName + ); + const scoreValue = Number(frontmatter.score); + return { + sessionId, + score: Number.isFinite(scoreValue) ? scoreValue : null, + startTime: + typeof frontmatter.start_time === "string" + ? frontmatter.start_time + : null, + markdownPath, + text: content, + }; + }) + ); + + return docs; +} + +function pickResearchContext(params: { + question: string; + docs: SessionResearchDocument[]; + limit: number; +}): SessionResearchDocument[] { + const terms = tokenize(params.question); + const ranked = params.docs + .map((doc) => ({ + doc, + relevance: scoreDocument(terms, doc.text), + })) + .sort((a, b) => { + if (b.relevance !== a.relevance) { + return b.relevance - a.relevance; + } + + const aTime = Date.parse(a.doc.startTime || ""); + const bTime = Date.parse(b.doc.startTime || ""); + if (Number.isFinite(aTime) && Number.isFinite(bTime) && bTime !== aTime) { + return bTime - aTime; + } + + return (b.doc.score || 0) - (a.doc.score || 0); + }); + + const relevant = ranked + .filter((entry) => entry.relevance > 0) + .slice(0, params.limit) + .map((entry) => entry.doc); + + if (relevant.length) { + return relevant; + } + + return ranked + .slice(0, Math.max(1, Math.min(5, params.limit))) + .map((entry) => entry.doc); +} + +export async function researchFromAnalyzedSessions(params: { + question: string; + context: CoreContext; + limit?: number; +}): Promise { + const docs = await loadSessionResearchDocuments(params.context.homeDir); + if (!docs.length) { + throw new Error( + "No analyzed sessions found in workspace. Run daemon backfill or `vesai user ` first." + ); + } + + const parsedLimit = + params.limit === undefined ? Number.NaN : Number(params.limit); + const limit = + Number.isFinite(parsedLimit) && parsedLimit > 0 + ? Math.floor(parsedLimit) + : 12; + const selected = pickResearchContext({ + question: params.question, + docs, + limit, + }); + + const ai = createVertexClient( + params.context.config.gcloud.projectId, + params.context.config.vertex.location + ); + + const response = await answerResearchQuestion({ + ai, + model: params.context.config.vertex.model, + productDescription: params.context.config.product.description, + question: params.question, + sessions: selected.map((doc) => ({ + sessionId: doc.sessionId, + startTime: doc.startTime, + score: doc.score, + markdownPath: doc.markdownPath, + summary: doc.text.slice(0, 12_000), + })), + }); + + return { + question: params.question, + answer: response.answer, + findings: response.findings, + confidence: response.confidence, + supportingSessionIds: response.supportingSessionIds, + sessionsConsidered: docs.length, + sessionsUsed: selected.length, + }; +} diff --git a/cli/analysis/session.ts b/cli/analysis/session.ts index d0488a4..654e4d4 100644 --- a/cli/analysis/session.ts +++ b/cli/analysis/session.ts @@ -1,6 +1,10 @@ import { mkdir, readFile, writeFile } from "node:fs/promises"; import { join } from "node:path"; import { DEFAULT_VERTEX_MODEL, getVesaiPaths } from "../../config"; +import { + computeDynamicRenderServiceCapacity, + estimateRenderServiceCapacity, +} from "../../config/runtime"; import { analyzeSessionVideo, createVertexClient, @@ -8,6 +12,7 @@ import { type PostHogRecording, } from "../../connectors"; import constructEvents from "../../render/events"; +import { withGlobalRenderSlot } from "../../render/global-render-slot"; import constructVideo from "../../render/replay"; import { writeSessionMarkdown } from "../../workspace"; import type { @@ -178,7 +183,7 @@ export async function ensureSessionRendered( } const semaphore = getRenderSemaphore( - context.config.runtime.maxConcurrentRenders + estimateRenderServiceCapacity(context.config.runtime.maxRenderMemoryMb) ); const release = await semaphore.acquire(); try { @@ -190,49 +195,62 @@ export async function ensureSessionRendered( return cachedAfterWait; } - process.env.VESAI_GCS_BUCKET = context.config.gcloud.bucket; - - const { events, video } = await withSuppressedRenderLogs(async () => { - const events = await constructEvents({ - source_type: "posthog", - source_host: context.config.posthog.host, - source_key: context.config.posthog.apiKey, - source_project: context.config.posthog.projectId, - external_id: recording.id, - project_id: context.config.posthog.projectId, - session_id: recording.id, - bucket_name: context.config.gcloud.bucket, - }); - - const video = await constructVideo({ - projectId: context.config.posthog.projectId, - sessionId: recording.id, - eventsPath: events.eventsPath, - bucketName: context.config.gcloud.bucket, - config: { - speed: 1, - skipInactive: true, - mouseTail: { - strokeStyle: "red", - lineWidth: 2, - lineCap: "round", - }, - }, - }); + return withGlobalRenderSlot({ + maxRenderMemoryMb: context.config.runtime.maxRenderMemoryMb, + task: async () => { + const cachedWithGlobalLock = await readRenderCache( + recording.id, + context.homeDir + ); + if (cachedWithGlobalLock) { + return cachedWithGlobalLock; + } + + process.env.VESAI_GCS_BUCKET = context.config.gcloud.bucket; + + const { events, video } = await withSuppressedRenderLogs(async () => { + const events = await constructEvents({ + source_type: "posthog", + source_host: context.config.posthog.host, + source_key: context.config.posthog.apiKey, + source_project: context.config.posthog.projectId, + external_id: recording.id, + project_id: context.config.projectId, + session_id: recording.id, + bucket_name: context.config.gcloud.bucket, + }); + + const video = await constructVideo({ + projectId: context.config.projectId, + sessionId: recording.id, + eventsPath: events.eventsPath, + bucketName: context.config.gcloud.bucket, + config: { + speed: 1, + skipInactive: true, + mouseTail: { + strokeStyle: "red", + lineWidth: 2, + lineCap: "round", + }, + }, + }); + + return { events, video }; + }); - return { events, video }; + const render: RenderedSession = { + sessionId: recording.id, + eventsUri: events.eventsUri, + videoUri: video.videoUri, + videoDuration: Math.round(video.videoDuration), + renderedAt: new Date().toISOString(), + }; + + await writeRenderCache(recording.id, render, context.homeDir); + return render; + }, }); - - const render: RenderedSession = { - sessionId: recording.id, - eventsUri: events.eventsUri, - videoUri: video.videoUri, - videoDuration: Math.round(video.videoDuration), - renderedAt: new Date().toISOString(), - }; - - await writeRenderCache(recording.id, render, context.homeDir); - return render; } finally { release(); } @@ -317,7 +335,10 @@ export async function renderAndAnalyzeSessions(params: { const concurrency = Math.max( 1, Math.floor( - params.concurrency ?? params.context.config.runtime.maxConcurrentRenders + params.concurrency ?? + computeDynamicRenderServiceCapacity({ + maxRenderMemoryMb: params.context.config.runtime.maxRenderMemoryMb, + }) ) ); const effectiveConcurrency = Math.min(concurrency, recordings.length); diff --git a/cli/analysis/user.ts b/cli/analysis/user.ts index a8b0ca0..37c3c87 100644 --- a/cli/analysis/user.ts +++ b/cli/analysis/user.ts @@ -16,6 +16,8 @@ export type UserAnalysisResult = { sessionCount: number; averageSessionScore: number; userScore: number; + story: string; + health: string; markdownPath: string; }; @@ -166,6 +168,8 @@ export async function analyzeUserByEmailWithDeps(params: { sessionCount: sessionResults.length, averageSessionScore: summary.averageScore, userScore: aggregate.score, + story: aggregate.story, + health: aggregate.health, markdownPath, }; } diff --git a/cli/commands/analytics-commands.ts b/cli/commands/analytics-commands.ts deleted file mode 100644 index 2f89acc..0000000 --- a/cli/commands/analytics-commands.ts +++ /dev/null @@ -1,16 +0,0 @@ -import type { Command } from "commander"; -import { registerErrorsCommand } from "./errors-command"; -import { registerEventsCommand } from "./events-command"; -import { registerInsightsCommand } from "./insights-command"; -import { registerLogsCommand } from "./logs-command"; -import { registerPropertiesCommand } from "./properties-command"; -import { registerSchemaCommand } from "./schema-command"; - -export function registerAnalyticsCommands(program: Command): void { - registerEventsCommand(program); - registerPropertiesCommand(program); - registerSchemaCommand(program); - registerInsightsCommand(program); - registerErrorsCommand(program); - registerLogsCommand(program); -} diff --git a/cli/commands/config-command.ts b/cli/commands/config-command.ts index e7c6e12..4e673c2 100644 --- a/cli/commands/config-command.ts +++ b/cli/commands/config-command.ts @@ -1,11 +1,64 @@ import type { Command } from "commander"; -import { loadConfig, requireConfig, updateConfig } from "../../config"; -import { printJson, redactConfigSecrets } from "./helpers"; +import { + loadCoreConfig, + loadProjectConfig, + requireCoreConfig, + updateCoreConfig, + updateProjectConfig, +} from "../../config"; +import { printJson } from "./helpers"; type ConfigShowOptions = { showSecrets?: boolean; }; +function setNestedValue( + root: Record, + path: string, + value: unknown +): void { + const keys = path.split(".").filter(Boolean); + if (!keys.length) { + throw new Error("Invalid config path"); + } + + let current = root; + for (let i = 0; i < keys.length - 1; i++) { + const key = keys[i]!; + const existing = current[key]; + if (!existing || typeof existing !== "object") { + current[key] = {}; + } + current = current[key] as Record; + } + + current[keys[keys.length - 1]!] = value; +} + +function parseScalarValue(raw: string): unknown { + if (raw === "true") { + return true; + } + if (raw === "false") { + return false; + } + if (!Number.isNaN(Number(raw))) { + return Number(raw); + } + return raw; +} + +function maskSecret(value: string): string { + const trimmed = value.trim(); + if (!trimmed) { + return ""; + } + if (trimmed.length <= 8) { + return "*".repeat(Math.max(4, trimmed.length)); + } + return `${trimmed.slice(0, 4)}...${trimmed.slice(-4)}`; +} + export function registerConfigCommand(program: Command): void { const configCmd = program.command("config").description("Config commands"); @@ -16,93 +69,116 @@ Examples: $ vesai config show $ vesai config show --show-secrets $ vesai config validate - $ vesai config set runtime.maxConcurrentRenders 6 + $ vesai config set core.runtime.maxRenderMemoryMb 8192 + $ vesai config set project.daemon.lookbackDays 180 ` ); configCmd .command("show") - .description("Show current config") + .description("Show current core and project config") .option( "--show-secrets", "Show sensitive values (default redacts credentials)" ) - .addHelpText( - "after", - ` -By default, sensitive keys are redacted. -Use --show-secrets only in trusted local terminals. -` - ) .action(async (options: ConfigShowOptions) => { - const config = await loadConfig(); - printJson(options.showSecrets ? config : redactConfigSecrets(config)); + const core = await loadCoreConfig(); + const project = await loadProjectConfig().catch(() => null); + let projectOutput: typeof project | null = null; + + if (project) { + if (options.showSecrets) { + projectOutput = project; + } else { + projectOutput = { + ...project, + posthog: { + ...project.posthog, + apiKey: maskSecret(project.posthog.apiKey), + }, + }; + } + } + + const output = { + core, + project: projectOutput, + }; + + printJson(output); }); configCmd .command("validate") .description("Validate current config") - .addHelpText( - "after", - ` -Use this after quickstart or manual config edits. -` - ) .action(async () => { - const config = await requireConfig(); - printJson({ valid: true, project: config.gcloud.projectId }); + const core = await requireCoreConfig(); + const project = await loadProjectConfig().catch(() => null); + printJson({ + core: { valid: true, gcloudProject: core.gcloud.projectId }, + project: project + ? { valid: true, projectId: project.projectId } + : { + valid: false, + reason: "Project config missing. Run `vesai init` in your repo.", + }, + }); }); configCmd .command("set ") - .description("Set a config value using dot path") + .description( + "Set config value using dot path prefixed with core. or project." + ) .addHelpText( "after", ` Examples: - $ vesai config set posthog.groupKey organization - $ vesai config set runtime.maxConcurrentRenders 6 - $ vesai config set vertex.location us-central1 + $ vesai config set core.runtime.maxRenderMemoryMb 8192 + $ vesai config set core.vertex.location us-central1 + $ vesai config set project.daemon.lookbackDays 180 + $ vesai config set project.posthog.groupKey organization ` ) .action(async (path: string, value: string) => { - const next = await updateConfig({ - updater: (config) => { - const clone = structuredClone(config); - const keys = path.split(".").filter(Boolean); - if (!keys.length) { - throw new Error("Invalid config path"); - } - - let current: Record = clone as unknown as Record< - string, - unknown - >; - - for (let i = 0; i < keys.length - 1; i++) { - const key = keys[i]!; - const existing = current[key]; - if (!existing || typeof existing !== "object") { - current[key] = {}; - } - current = current[key] as Record; - } - - const finalKey = keys[keys.length - 1]!; - let parsed: unknown = value; - if (value === "true") { - parsed = true; - } else if (value === "false") { - parsed = false; - } else if (!Number.isNaN(Number(value))) { - parsed = Number(value); - } - - current[finalKey] = parsed; - return clone; - }, - }); + const parsed = parseScalarValue(value); + + if (path.startsWith("core.")) { + const keyPath = path.slice("core.".length); + const next = await updateCoreConfig({ + updater: (config) => { + const clone = structuredClone(config) as unknown as Record< + string, + unknown + >; + setNestedValue(clone, keyPath, parsed); + return clone as never; + }, + }); + printJson({ updated: true, scope: "core", updatedAt: next.updatedAt }); + return; + } + + if (path.startsWith("project.")) { + const keyPath = path.slice("project.".length); + const next = await updateProjectConfig({ + updater: (config) => { + const clone = structuredClone(config) as unknown as Record< + string, + unknown + >; + setNestedValue(clone, keyPath, parsed); + return clone as never; + }, + }); + printJson({ + updated: true, + scope: "project", + updatedAt: next.updatedAt, + }); + return; + } - printJson({ updated: true, updatedAt: next.updatedAt }); + throw new Error("Config path must start with core. or project."); }); } diff --git a/cli/commands/daemon-command.ts b/cli/commands/daemon-command.ts index 4a3a91c..38c3213 100644 --- a/cli/commands/daemon-command.ts +++ b/cli/commands/daemon-command.ts @@ -5,7 +5,8 @@ import { kill } from "node:process"; import { fileURLToPath } from "node:url"; import type { Command } from "commander"; import { - ensureVesaiDirectories, + ensureCoreDirectories, + ensureProjectDirectories, getVesaiPaths, requireConfig, } from "../../config"; @@ -51,7 +52,10 @@ async function startDaemonInBackground(): Promise<{ const child = spawn(process.execPath, [DAEMON_ENTRYPOINT], { cwd: REPO_ROOT, detached: true, - env: process.env, + env: { + ...process.env, + VESAI_PROJECT_ROOT: paths.projectRoot, + }, stdio: ["ignore", logHandle.fd, logHandle.fd], }); child.unref(); @@ -78,6 +82,10 @@ Examples: $ vesai daemon watch $ vesai daemon status $ vesai daemon stop + +Behavior: + - First run backfills replay analysis from init lookbackDays. + - Ongoing heartbeat pulls sessions from last cursor to now and queues analysis. ` ); @@ -87,12 +95,13 @@ Examples: .addHelpText( "after", ` -Runs daemon detached from the terminal and writes logs under ~/.vesai/logs. +Runs daemon detached from the terminal and writes logs under .vesai/logs. Use \`vesai daemon status\` to confirm pid and \`vesai daemon stop\` to shut down. ` ) .action(async () => { - await ensureVesaiDirectories(); + await ensureCoreDirectories(); + await ensureProjectDirectories(); await requireConfig(); await ensurePlaywrightChromiumInstalled(); @@ -119,7 +128,8 @@ Process exits on Ctrl+C or shell exit. ` ) .action(async () => { - await ensureVesaiDirectories(); + await ensureCoreDirectories(); + await ensureProjectDirectories(); await requireConfig(); await ensurePlaywrightChromiumInstalled(); diff --git a/cli/commands/doctor-command.ts b/cli/commands/doctor-command.ts index c49c18b..1d9fba0 100644 --- a/cli/commands/doctor-command.ts +++ b/cli/commands/doctor-command.ts @@ -1,5 +1,5 @@ import type { Command } from "commander"; -import { requireConfig, resolveVesaiHome } from "../../config"; +import { getVesaiPaths, requireConfig, resolveVesaiHome } from "../../config"; import { getPlaywrightChromiumExecutablePath, isPlaywrightChromiumInstalled, @@ -20,6 +20,7 @@ Examples: ) .action(async () => { const home = resolveVesaiHome(); + const projectPaths = getVesaiPaths(); const status = await getDaemonStatus(); const config = await requireConfig(); const playwrightExecutable = getPlaywrightChromiumExecutablePath(); @@ -27,7 +28,9 @@ Examples: isPlaywrightChromiumInstalled(playwrightExecutable); printJson({ - home, + coreHome: home, + projectRoot: projectPaths.projectRoot, + projectConfig: projectPaths.configFile, daemon: status, posthogProject: config.posthog.projectId, gcloudProject: config.gcloud.projectId, diff --git a/cli/commands/errors-command.ts b/cli/commands/errors-command.ts deleted file mode 100644 index bf9b58d..0000000 --- a/cli/commands/errors-command.ts +++ /dev/null @@ -1,105 +0,0 @@ -import type { Command } from "commander"; -import { requireConfig } from "../../config"; -import { getErrorDetails, listErrors } from "../../connectors"; -import { printResult, shouldEmitJson } from "./runtime"; - -type ErrorListOptions = { - orderBy?: "occurrences" | "first_seen" | "last_seen" | "users" | "sessions"; - orderDirection?: "ASC" | "DESC"; - status?: "active" | "resolved" | "all" | "suppressed"; - from?: string; - to?: string; - includeTestAccounts?: boolean; - json?: boolean; -}; - -type ErrorDetailsOptions = { - from?: string; - to?: string; - json?: boolean; -}; - -export function registerErrorsCommand(program: Command): void { - const errors = program - .command("errors") - .description("Query PostHog error-tracking data"); - - errors.addHelpText( - "after", - ` -Examples: - $ vesai errors list --status active --from 2026-02-01T00:00:00Z --to 2026-02-15T00:00:00Z - $ vesai errors list --order-by occurrences --order-direction DESC - $ vesai errors details -` - ); - - errors - .command("list") - .description("List top errors") - .option( - "--order-by ", - "occurrences|first_seen|last_seen|users|sessions" - ) - .option("--order-direction ", "ASC|DESC") - .option("--status ", "active|resolved|all|suppressed") - .option("--from ", "Start date (ISO timestamp)") - .option("--to ", "End date (ISO timestamp)") - .option("--include-test-accounts", "Include test-account traffic") - .option("--no-json", "Output human-readable text") - .addHelpText( - "after", - ` -Examples: - $ vesai errors list --status active - $ vesai errors list --order-by occurrences --order-direction DESC - $ vesai errors list --from 2026-02-01T00:00:00Z --to 2026-02-15T00:00:00Z -` - ) - .action(async (options: ErrorListOptions) => { - const config = await requireConfig(); - const result = await listErrors({ - host: config.posthog.host, - apiKey: config.posthog.apiKey, - projectId: config.posthog.projectId, - input: { - orderBy: options.orderBy, - orderDirection: options.orderDirection, - status: options.status, - dateFrom: options.from, - dateTo: options.to, - filterTestAccounts: options.includeTestAccounts, - }, - }); - - printResult(result, shouldEmitJson(options.json)); - }); - - errors - .command("details ") - .description("Get detailed data for one error issue UUID") - .option("--from ", "Start date (ISO timestamp)") - .option("--to ", "End date (ISO timestamp)") - .option("--no-json", "Output human-readable text") - .addHelpText( - "after", - ` -Examples: - $ vesai errors details 0195f3f9-d39b-7210-ac23-c3be31b1c6ba - $ vesai errors details --from 2026-02-01T00:00:00Z --to 2026-02-15T00:00:00Z -` - ) - .action(async (issueId: string, options: ErrorDetailsOptions) => { - const config = await requireConfig(); - const result = await getErrorDetails({ - host: config.posthog.host, - apiKey: config.posthog.apiKey, - projectId: config.posthog.projectId, - issueId, - dateFrom: options.from, - dateTo: options.to, - }); - - printResult(result, shouldEmitJson(options.json)); - }); -} diff --git a/cli/commands/events-command.ts b/cli/commands/events-command.ts deleted file mode 100644 index 93c2e85..0000000 --- a/cli/commands/events-command.ts +++ /dev/null @@ -1,50 +0,0 @@ -import type { Command } from "commander"; -import { requireConfig } from "../../config"; -import { listEventDefinitions } from "../../connectors"; -import { printJson } from "./helpers"; -import { shouldEmitJson } from "./runtime"; - -type EventsCommandOptions = { - search?: string; - limit?: number; - offset?: number; - json?: boolean; -}; - -export function registerEventsCommand(program: Command): void { - program - .command("events") - .description("List PostHog event definitions (MCP-compatible)") - .option("--search ", "Search by event name") - .option("--limit ", "Limit number of events", Number) - .option("--offset ", "Pagination offset", Number) - .option("--no-json", "Output human-readable text") - .addHelpText( - "after", - ` -Examples: - $ vesai events - $ vesai events --search checkout --limit 20 - $ vesai events --search "$pageview" -` - ) - .action(async (options: EventsCommandOptions) => { - const config = await requireConfig(); - const events = await listEventDefinitions({ - host: config.posthog.host, - apiKey: config.posthog.apiKey, - projectId: config.posthog.projectId, - search: options.search, - limit: options.limit, - offset: options.offset, - }); - - if (shouldEmitJson(options.json)) { - printJson(events); - return; - } - - console.log(`Events: ${events.length}`); - printJson(events); - }); -} diff --git a/cli/commands/group-command.ts b/cli/commands/group-command.ts new file mode 100644 index 0000000..a1ac13e --- /dev/null +++ b/cli/commands/group-command.ts @@ -0,0 +1,72 @@ +import type { Command } from "commander"; +import { ensurePlaywrightChromiumInstalled } from "../../connectors"; +import { analyzeGroupById } from "../analysis"; +import { printJson } from "./helpers"; +import { createReplayProgressRenderer } from "./replays-progress"; +import { + ensureReplayContext, + type ReplayRunOptions, + resolveSessionConcurrency, + shouldEmitJson, + withRenderLogMode, +} from "./runtime"; + +export function registerGroupCommand(program: Command): void { + program + .command("group ") + .description( + "Analyze one group story from all users and sessions under this group id" + ) + .option( + "--max-concurrent ", + "Max concurrent session pipelines for each user", + Number + ) + .option("--verbose", "Show low-level render/debug logs") + .option("--no-json", "Output human-readable text") + .addHelpText( + "after", + ` +What this does: + - Resolves users associated with the group id. + - Builds each user story from that user's sessions. + - Returns one aggregate group story. +` + ) + .action(async (groupId: string, options: ReplayRunOptions) => { + const { config } = await ensureReplayContext(); + await ensurePlaywrightChromiumInstalled(); + + const progress = shouldEmitJson(options.json) + ? null + : createReplayProgressRenderer({ + label: `group ${groupId}`, + }); + + try { + const result = await withRenderLogMode(options.verbose, async () => + analyzeGroupById({ + groupId, + context: { config }, + sessionConcurrency: resolveSessionConcurrency(options, config), + onSessionProgress: progress?.handle, + }) + ); + + if (shouldEmitJson(options.json)) { + printJson(result); + return; + } + + console.log(`Group: ${result.groupId}`); + console.log(`Users analyzed: ${result.usersAnalyzed}`); + console.log(`Score: ${result.score}`); + console.log(`Health: ${result.health}`); + console.log(`Markdown: ${result.markdownPath}`); + console.log(""); + console.log(result.story); + } finally { + progress?.close(); + } + }); +} diff --git a/cli/commands/init-command.ts b/cli/commands/init-command.ts new file mode 100644 index 0000000..d395fa7 --- /dev/null +++ b/cli/commands/init-command.ts @@ -0,0 +1,45 @@ +import type { Command } from "commander"; +import { ensureCoreDirectories } from "../../config"; +import { type InitCommandOptions, runInitCli } from "./init"; + +export function registerInitCommand(program: Command): void { + program + .command("init") + .description("Initialize current directory as a VES AI project") + .option("--project-id ", "VES AI project id UUID override") + .option("--posthog-host ", "PostHog host URL") + .option("--posthog-api-key ", "PostHog user API key") + .option("--posthog-project-id ", "PostHog project id") + .option("--posthog-group-key ", "PostHog group key") + .option("--domain-filter ", "Replay domain filter") + .option( + "--lookback-days ", + "Initial backfill lookback window in days (positive integer)", + Number + ) + .option( + "--product-description ", + "Product description for analysis context" + ) + .option("-y, --yes", "Use defaults where possible without prompting") + .option( + "--non-interactive", + "Disable prompts and require flags for required fields" + ) + .addHelpText( + "after", + ` +Creates .vesai/ in the current repository, writes project config, and ensures +.vesai/ is added to .gitignore. + +Examples: + $ vesai init + $ vesai init --yes --domain-filter app.example.com --product-description "B2B SaaS for support teams" + $ vesai init --non-interactive --posthog-api-key phx_... --posthog-project-id 123 --posthog-group-key organization --domain-filter app.example.com --product-description "B2B SaaS for support teams" +` + ) + .action(async (options: InitCommandOptions) => { + await ensureCoreDirectories(); + await runInitCli(options); + }); +} diff --git a/cli/commands/init.ts b/cli/commands/init.ts new file mode 100644 index 0000000..79c595e --- /dev/null +++ b/cli/commands/init.ts @@ -0,0 +1,293 @@ +import { randomUUID } from "node:crypto"; +import { stdin as input, stdout as output } from "node:process"; +import { createInterface } from "node:readline/promises"; +import { + ensureProjectDirectories, + ensureProjectGitignore, + ensureWorkspaceGitRepo, + requireCoreConfig, + saveProjectConfig, +} from "../../config"; +import { listProjects, type PostHogProject } from "../../connectors"; +import { normalizeConcurrencyValue } from "./quickstart"; + +export type InitCommandOptions = { + projectId?: string; + posthogHost?: string; + posthogApiKey?: string; + posthogProjectId?: string; + posthogGroupKey?: string; + domainFilter?: string; + lookbackDays?: number; + productDescription?: string; + yes?: boolean; + nonInteractive?: boolean; +}; + +type PromptClient = { + ask: (prompt: string) => Promise; + close: () => void; +}; + +function createPromptClient(): PromptClient { + const rl = createInterface({ input, output }); + return { + ask: async (prompt: string) => rl.question(prompt), + close: () => rl.close(), + }; +} + +async function resolveValue(params: { + promptClient: PromptClient | null; + label: string; + flag: string; + value: string | undefined; + defaultValue?: string; + required?: boolean; + nonInteractive: boolean; + useDefaultsWithoutPrompt: boolean; +}): Promise { + const required = params.required !== false; + const direct = params.value?.trim(); + if (direct) { + return direct; + } + + if (params.useDefaultsWithoutPrompt && params.defaultValue !== undefined) { + return params.defaultValue; + } + + if (params.nonInteractive) { + if (params.defaultValue !== undefined) { + return params.defaultValue; + } + throw new Error( + `Missing required option --${params.flag} in non-interactive mode.` + ); + } + + if (!params.promptClient) { + throw new Error(`Missing prompt client for --${params.flag}.`); + } + + while (true) { + const suffix = params.defaultValue ? ` [${params.defaultValue}]` : ""; + const raw = await params.promptClient.ask(`${params.label}${suffix}: `); + const picked = raw.trim() || params.defaultValue || ""; + + if (!required || picked) { + return picked; + } + + console.log(`${params.label} is required.`); + } +} + +export function parseProjectSelectionIndex( + value: string, + size: number +): number { + const parsed = Number(value.trim()); + if (!Number.isInteger(parsed) || parsed < 1 || parsed > size) { + throw new Error(`Invalid project selection "${value}". Choose 1-${size}.`); + } + return parsed - 1; +} + +async function choosePostHogProjectId(params: { + projects: PostHogProject[]; + optionValue: string | undefined; + nonInteractive: boolean; + promptClient: PromptClient | null; + useDefaultsWithoutPrompt: boolean; +}): Promise { + const normalizedOption = params.optionValue?.trim(); + if (normalizedOption) { + const exists = params.projects.some( + (project) => String(project.id) === normalizedOption + ); + if (!exists) { + const available = params.projects + .map((project) => `${project.name} (${project.id})`) + .join(", "); + throw new Error( + `Unknown PostHog project id "${normalizedOption}". Available projects: ${available}` + ); + } + return normalizedOption; + } + + if (params.useDefaultsWithoutPrompt) { + return String(params.projects[0]!.id); + } + + if (params.nonInteractive) { + throw new Error( + "Missing required option --posthog-project-id in non-interactive mode." + ); + } + + if (!params.promptClient) { + throw new Error("Prompt client unavailable for project selection."); + } + + console.log(""); + console.log("Available PostHog projects:"); + for (let i = 0; i < params.projects.length; i++) { + const project = params.projects[i]!; + console.log(` ${i + 1}. ${project.name} (${project.id})`); + } + + while (true) { + const raw = await params.promptClient.ask("Select project number [1]: "); + const selection = raw.trim() || "1"; + try { + const index = parseProjectSelectionIndex( + selection, + params.projects.length + ); + return String(params.projects[index]!.id); + } catch (error) { + console.log(error instanceof Error ? error.message : String(error)); + } + } +} + +export async function runInitCli(options: InitCommandOptions): Promise { + const nonInteractive = Boolean(options.nonInteractive); + const useDefaultsWithoutPrompt = Boolean( + options.yes || options.nonInteractive + ); + const promptClient = nonInteractive ? null : createPromptClient(); + + try { + console.log("VES AI init (project setup)"); + await requireCoreConfig(); + await ensureProjectGitignore(process.cwd()); + await ensureProjectDirectories(process.cwd()); + + const generatedProjectId = options.projectId?.trim() || randomUUID(); + const projectId = await resolveValue({ + promptClient, + label: "VES AI project id (UUID)", + flag: "project-id", + value: options.projectId, + defaultValue: generatedProjectId, + nonInteractive, + useDefaultsWithoutPrompt, + }); + + const posthogHost = await resolveValue({ + promptClient, + label: "PostHog host URL", + flag: "posthog-host", + value: options.posthogHost, + defaultValue: "https://us.posthog.com", + nonInteractive, + useDefaultsWithoutPrompt, + }); + + const posthogApiKey = await resolveValue({ + promptClient, + label: "PostHog API key", + flag: "posthog-api-key", + value: options.posthogApiKey, + nonInteractive, + useDefaultsWithoutPrompt, + }); + + console.log("Loading PostHog projects..."); + const projects = await listProjects({ + host: posthogHost, + apiKey: posthogApiKey, + }); + + if (!projects.length) { + throw new Error("No PostHog projects found with this key."); + } + + const posthogProjectId = await choosePostHogProjectId({ + projects, + optionValue: options.posthogProjectId, + nonInteractive, + promptClient, + useDefaultsWithoutPrompt, + }); + + const posthogGroupKey = await resolveValue({ + promptClient, + label: "PostHog group key", + flag: "posthog-group-key", + value: options.posthogGroupKey, + defaultValue: "company_id", + nonInteractive, + useDefaultsWithoutPrompt, + }); + + const domainFilter = await resolveValue({ + promptClient, + label: "Domain filter for session replays", + flag: "domain-filter", + value: options.domainFilter, + nonInteractive, + useDefaultsWithoutPrompt, + }); + + const lookbackDaysInput = await resolveValue({ + promptClient, + label: "Backfill lookback window in days (>=1)", + flag: "lookback-days", + value: + options.lookbackDays === undefined + ? undefined + : String(options.lookbackDays), + defaultValue: "180", + nonInteractive, + useDefaultsWithoutPrompt, + }); + const lookbackDays = normalizeConcurrencyValue( + lookbackDaysInput, + 180, + "--lookback-days" + ); + + const productDescription = await resolveValue({ + promptClient, + label: "Describe your product for better analysis", + flag: "product-description", + value: options.productDescription, + nonInteractive, + useDefaultsWithoutPrompt, + }); + + await saveProjectConfig({ + projectRoot: process.cwd(), + config: { + version: 1, + projectId, + posthog: { + host: posthogHost, + apiKey: posthogApiKey, + projectId: posthogProjectId, + groupKey: posthogGroupKey, + domainFilter, + }, + daemon: { + lookbackDays, + }, + product: { + description: productDescription, + }, + }, + }); + + await ensureWorkspaceGitRepo(process.cwd()); + + console.log("Project init complete."); + console.log("Next steps:"); + console.log(" vesai daemon start"); + console.log(" vesai user you@example.com"); + } finally { + promptClient?.close(); + } +} diff --git a/cli/commands/insights-command.ts b/cli/commands/insights-command.ts deleted file mode 100644 index 5e72033..0000000 --- a/cli/commands/insights-command.ts +++ /dev/null @@ -1,212 +0,0 @@ -import { readFile as readFileFs } from "node:fs/promises"; -import type { Command } from "commander"; -import { requireConfig } from "../../config"; -import { - executeSqlQuery, - generateHogQLFromQuestion, - runInsightQuery, -} from "../../connectors"; -import { printJson } from "./helpers"; -import { normalizeHogqlInsight, normalizeSqlInsight } from "./insights-format"; -import { mapRowToObject, printResult, shouldEmitJson } from "./runtime"; - -type InsightRunOptions = { - queryJson?: string; - queryFile?: string; - hogql?: string; - json?: boolean; -}; - -async function parseInsightQueryInput( - options: InsightRunOptions -): Promise> { - const candidates = [ - options.queryJson, - options.queryFile, - options.hogql, - ].filter(Boolean); - if (candidates.length !== 1) { - throw new Error( - "Provide exactly one of --query-json, --query-file, or --hogql." - ); - } - - if (options.hogql) { - return { - kind: "DataVisualizationNode", - source: { - kind: "HogQLQuery", - query: options.hogql, - }, - }; - } - - if (options.queryFile) { - const raw = await readFileFs(options.queryFile, "utf8"); - const parsed = JSON.parse(raw); - if (!(parsed && typeof parsed === "object")) { - throw new Error("--query-file must contain a JSON object."); - } - return parsed as Record; - } - - const parsed = JSON.parse(options.queryJson || "{}"); - if (!(parsed && typeof parsed === "object")) { - throw new Error("--query-json must be a JSON object."); - } - return parsed as Record; -} - -export function registerInsightsCommand(program: Command): void { - const insights = program - .command("insights") - .description("Run PostHog insight/HogQL analytics commands"); - - insights.addHelpText( - "after", - ` -Examples: - $ vesai insights hogql "top events in the last 7 days" - $ vesai insights sql "SELECT event, count() FROM events GROUP BY event LIMIT 20" - $ vesai insights run --hogql "SELECT distinct_id, count() FROM events GROUP BY distinct_id LIMIT 10" - -Output tips: - - JSON output is default for structured agent workflows. - - Use --no-json for human-readable previews. - - Use --raw to inspect the original PostHog payload. -` - ); - - insights - .command("run") - .description("Run a PostHog insight query object") - .option("--query-json ", "Insight query JSON object") - .option("--query-file ", "Path to insight query JSON file") - .option("--hogql ", "Shortcut: run HogQL as a DataVisualizationNode") - .option("--no-json", "Output human-readable text") - .addHelpText( - "after", - ` -Examples: - $ vesai insights run --hogql "SELECT event, count() FROM events GROUP BY event LIMIT 20" - $ vesai insights run --query-file ./query.json -` - ) - .action(async (options: InsightRunOptions) => { - const config = await requireConfig(); - const query = await parseInsightQueryInput(options); - const result = await runInsightQuery({ - host: config.posthog.host, - apiKey: config.posthog.apiKey, - projectId: config.posthog.projectId, - query, - }); - - printResult(result, shouldEmitJson(options.json)); - }); - - insights - .command("hogql ") - .description("Generate HogQL insight from a natural-language question") - .option("--raw", "Return raw PostHog response payload") - .option("--no-json", "Output human-readable text") - .addHelpText( - "after", - ` -Examples: - $ vesai insights hogql "weekly active users by plan" - $ vesai insights hogql "show top pages for enterprise users last 30 days" - $ vesai insights hogql "signup conversion by week" --raw -` - ) - .action( - async (question: string, options: { raw?: boolean; json?: boolean }) => { - const config = await requireConfig(); - const raw = await generateHogQLFromQuestion({ - host: config.posthog.host, - apiKey: config.posthog.apiKey, - projectId: config.posthog.projectId, - question, - }); - - if (options.raw) { - printResult(raw, shouldEmitJson(options.json)); - return; - } - - const normalized = normalizeHogqlInsight(raw); - if (shouldEmitJson(options.json)) { - printJson(normalized); - return; - } - - console.log(`HogQL question: ${question}`); - if (normalized.query) { - console.log("Generated query object:"); - printJson(normalized.query); - } - if (normalized.table) { - const table = normalized.table; - const preview = table.rows - .slice(0, 10) - .map((row) => mapRowToObject(table.columns, row)); - console.log( - `Result rows: ${table.rows.length} (showing ${preview.length})` - ); - printJson(preview); - } else { - console.log("No results table detected in response payload."); - } - } - ); - - insights - .command("sql ") - .description("Execute SQL/HogQL through PostHog MCP execute_sql") - .option("--raw", "Return raw PostHog response payload") - .option("--no-json", "Output human-readable text") - .addHelpText( - "after", - ` -Examples: - $ vesai insights sql "SELECT event, count() FROM events GROUP BY event LIMIT 20" - $ vesai insights sql "SELECT person.properties.email, count() FROM events GROUP BY person.properties.email LIMIT 20" - $ vesai insights sql "SELECT * FROM events LIMIT 5" --raw -` - ) - .action( - async (query: string, options: { raw?: boolean; json?: boolean }) => { - const config = await requireConfig(); - const raw = await executeSqlQuery({ - host: config.posthog.host, - apiKey: config.posthog.apiKey, - projectId: config.posthog.projectId, - query, - }); - - if (options.raw) { - printResult(raw, shouldEmitJson(options.json)); - return; - } - - const normalized = normalizeSqlInsight(raw); - if (shouldEmitJson(options.json)) { - printJson(normalized); - return; - } - - if (normalized.kind === "table") { - const preview = normalized.rows - .slice(0, 20) - .map((row) => mapRowToObject(normalized.columns, row)); - console.log( - `Rows: ${normalized.rowCount} (showing ${preview.length}) from insights sql` - ); - printJson(preview); - return; - } - - printResult(raw, false); - } - ); -} diff --git a/cli/commands/insights-format.ts b/cli/commands/insights-format.ts deleted file mode 100644 index 01a4c4c..0000000 --- a/cli/commands/insights-format.ts +++ /dev/null @@ -1,183 +0,0 @@ -export type ParsedPipeTable = { - columns: string[]; - rows: string[][]; -}; - -export type NormalizedSqlInsight = { - kind: "table" | "text"; - columns: string[]; - rows: string[][]; - rowCount: number; - raw: unknown; -}; - -export type NormalizedHogqlInsight = { - kind: "hogql"; - query: Record | null; - table: ParsedPipeTable | null; - artifactId: string | null; - plan: string | null; - raw: unknown; -}; - -function asRecord(value: unknown): Record | null { - if (!(value && typeof value === "object" && !Array.isArray(value))) { - return null; - } - return value as Record; -} - -function parsePipeLines(lines: string[]): ParsedPipeTable | null { - if (lines.length < 2) { - return null; - } - - const parsed = lines.map((line) => - line.split("|").map((cell) => cell.trim()) - ); - const width = parsed[0]?.length ?? 0; - if (width < 2) { - return null; - } - if (parsed.some((row) => row.length !== width)) { - return null; - } - - return { - columns: parsed[0] ?? [], - rows: parsed.slice(1), - }; -} - -export function extractFirstPipeTable(text: string): ParsedPipeTable | null { - const fenceRegex = /```(?:[\w-]+)?\n([\s\S]*?)```/g; - const fencedBlocks = [...text.matchAll(fenceRegex)].map((match) => match[1]); - const candidates = fencedBlocks.length ? fencedBlocks : [text]; - let best: ParsedPipeTable | null = null; - let bestScore = -1; - - for (let i = 0; i < candidates.length; i++) { - const candidate = candidates[i]!; - const lines = candidate - .split("\n") - .map((line) => line.trim()) - .filter((line) => line.length > 0 && line.includes("|")); - const table = parsePipeLines(lines); - if (table) { - const placeholderPenalty = - table.columns[0]?.toLowerCase() === "column1" ? 1000 : 0; - const score = table.rows.length - placeholderPenalty + i * 0.0001; - if (score > bestScore) { - best = table; - bestScore = score; - } - } - } - - return best; -} - -export function normalizeSqlInsight(raw: unknown): NormalizedSqlInsight { - if (typeof raw !== "string") { - return { - kind: "text", - columns: [], - rows: [], - rowCount: 0, - raw, - }; - } - - const table = extractFirstPipeTable(raw); - if (!table) { - return { - kind: "text", - columns: [], - rows: [], - rowCount: 0, - raw, - }; - } - - return { - kind: "table", - columns: table.columns, - rows: table.rows, - rowCount: table.rows.length, - raw, - }; -} - -export function normalizeHogqlInsight(raw: unknown): NormalizedHogqlInsight { - const normalized: NormalizedHogqlInsight = { - kind: "hogql", - query: null, - table: null, - artifactId: null, - plan: null, - raw, - }; - - if (!Array.isArray(raw)) { - return normalized; - } - - for (const item of raw) { - const record = asRecord(item); - const data = asRecord(record?.data); - if (!data) { - continue; - } - - if (!normalized.query) { - const answer = asRecord(data.answer); - if (answer) { - normalized.query = answer; - } - } - - if (!normalized.query) { - const content = asRecord(data.content); - const query = asRecord(content?.query); - if (query) { - normalized.query = query; - } - } - - if (!normalized.artifactId) { - const artifactId = data.artifact_id; - if (typeof artifactId === "string" && artifactId.trim()) { - normalized.artifactId = artifactId; - } - } - - if (!normalized.plan) { - const content = asRecord(data.content); - const plan = content?.plan; - if (typeof plan === "string" && plan.trim()) { - normalized.plan = plan; - } - } - - if (!normalized.table) { - const candidates: string[] = []; - if (typeof data.content === "string") { - candidates.push(data.content); - } - const content = asRecord(data.content); - if (typeof content?.content === "string") { - candidates.push(content.content); - } - - for (const candidate of candidates) { - const table = extractFirstPipeTable(candidate); - if (table) { - normalized.table = table; - break; - } - } - } - } - - return normalized; -} diff --git a/cli/commands/logs-command.ts b/cli/commands/logs-command.ts deleted file mode 100644 index a509a65..0000000 --- a/cli/commands/logs-command.ts +++ /dev/null @@ -1,175 +0,0 @@ -import type { Command } from "commander"; -import { requireConfig } from "../../config"; -import { - listLogAttributes, - listLogAttributeValues, - queryLogs, -} from "../../connectors"; -import { collectOption } from "./query-filters"; -import { printResult, shouldEmitJson } from "./runtime"; - -type LogsQueryOptions = { - from: string; - to: string; - severity: string[]; - service: string[]; - search?: string; - order?: "latest" | "earliest"; - limit?: number; - after?: string; - json?: boolean; -}; - -type LogsAttributesOptions = { - type?: "log" | "resource"; - search?: string; - limit?: number; - offset?: number; - json?: boolean; -}; - -type LogsValuesOptions = { - type?: "log" | "resource"; - search?: string; - limit?: number; - json?: boolean; -}; - -export function registerLogsCommand(program: Command): void { - const logs = program - .command("logs") - .description("Query PostHog logs and log metadata"); - - logs.addHelpText( - "after", - ` -Examples: - $ vesai logs query --from 2026-02-15T00:00:00Z --to 2026-02-15T06:00:00Z --severity error - $ vesai logs attributes --type resource - $ vesai logs values service.name --limit 20 -` - ); - - logs - .command("query") - .description("Run a logs query") - .requiredOption("--from ", "Start date (ISO timestamp)") - .requiredOption("--to ", "End date (ISO timestamp)") - .option( - "--severity ", - "Repeatable: trace|debug|info|warn|error|fatal", - collectOption, - [] - ) - .option( - "--service ", - "Repeatable service-name filter", - collectOption, - [] - ) - .option("--search ", "Free-text search") - .option("--order ", "latest|earliest") - .option("--limit ", "Max rows", Number) - .option("--after ", "Pagination cursor") - .option("--no-json", "Output human-readable text") - .addHelpText( - "after", - ` -Examples: - $ vesai logs query --from 2026-02-15T00:00:00Z --to 2026-02-15T06:00:00Z --severity error - $ vesai logs query --from 2026-02-15T00:00:00Z --to 2026-02-15T06:00:00Z --service api --search timeout - -Tip: - - Start with \`vesai logs attributes\` and \`vesai logs values \` to discover filterable fields. -` - ) - .action(async (options: LogsQueryOptions) => { - const config = await requireConfig(); - const result = await queryLogs({ - host: config.posthog.host, - apiKey: config.posthog.apiKey, - projectId: config.posthog.projectId, - input: { - dateFrom: options.from, - dateTo: options.to, - severityLevels: options.severity as Array< - "trace" | "debug" | "info" | "warn" | "error" | "fatal" - >, - serviceNames: options.service, - searchTerm: options.search, - orderBy: options.order, - limit: options.limit, - after: options.after, - }, - }); - - printResult(result, shouldEmitJson(options.json)); - }); - - logs - .command("attributes") - .description("List available log/resource attributes") - .option("--type ", "log|resource") - .option("--search ", "Filter by name") - .option("--limit ", "Max rows", Number) - .option("--offset ", "Pagination offset", Number) - .option("--no-json", "Output human-readable text") - .addHelpText( - "after", - ` -Examples: - $ vesai logs attributes - $ vesai logs attributes --type resource --search service - $ vesai logs attributes --type log -` - ) - .action(async (options: LogsAttributesOptions) => { - const config = await requireConfig(); - const result = await listLogAttributes({ - host: config.posthog.host, - apiKey: config.posthog.apiKey, - projectId: config.posthog.projectId, - attributeType: options.type, - search: options.search, - limit: options.limit, - offset: options.offset, - }); - - printResult(result, shouldEmitJson(options.json)); - }); - - logs - .command("values ") - .description("List values for a log/resource attribute key") - .option("--type ", "log|resource") - .option("--search ", "Filter values") - .option("--limit ", "Max rows", Number) - .option("--no-json", "Output human-readable text") - .addHelpText( - "after", - ` -Examples: - $ vesai logs values service.name - $ vesai logs values service.name --search api --limit 20 - $ vesai logs values resource.cloud.region -` - ) - .action(async (key: string, options: LogsValuesOptions) => { - const config = await requireConfig(); - const values = await listLogAttributeValues({ - host: config.posthog.host, - apiKey: config.posthog.apiKey, - projectId: config.posthog.projectId, - key, - attributeType: options.type, - search: options.search, - }); - - const result = - options.limit && Number.isFinite(options.limit) - ? values.slice(0, Math.max(1, options.limit)) - : values; - - printResult(result, shouldEmitJson(options.json)); - }); -} diff --git a/cli/commands/properties-command.ts b/cli/commands/properties-command.ts deleted file mode 100644 index 3a927b4..0000000 --- a/cli/commands/properties-command.ts +++ /dev/null @@ -1,61 +0,0 @@ -import type { Command } from "commander"; -import { requireConfig } from "../../config"; -import { listPropertyDefinitions } from "../../connectors"; -import { printJson } from "./helpers"; -import { shouldEmitJson } from "./runtime"; - -type PropertiesCommandOptions = { - type: string; - eventName?: string; - includePredefined?: boolean; - limit?: number; - offset?: number; - json?: boolean; -}; - -export function registerPropertiesCommand(program: Command): void { - program - .command("properties") - .description("List PostHog property definitions (MCP-compatible)") - .requiredOption("--type ", "Property type: event|person") - .option("--event-name ", "Event name (required when --type event)") - .option("--include-predefined", "Include predefined/system properties") - .option("--limit ", "Limit number of properties", Number) - .option("--offset ", "Pagination offset", Number) - .option("--no-json", "Output human-readable text") - .addHelpText( - "after", - ` -Examples: - $ vesai properties --type person --limit 20 - $ vesai properties --type event --event-name '$pageview' - $ vesai properties --type event --event-name '$autocapture' --include-predefined -` - ) - .action(async (options: PropertiesCommandOptions) => { - const type = options.type === "person" ? "person" : "event"; - if (type === "event" && !options.eventName) { - throw new Error("--event-name is required when --type event."); - } - - const config = await requireConfig(); - const properties = await listPropertyDefinitions({ - host: config.posthog.host, - apiKey: config.posthog.apiKey, - projectId: config.posthog.projectId, - type, - eventName: options.eventName, - includePredefinedProperties: options.includePredefined, - limit: options.limit, - offset: options.offset, - }); - - if (shouldEmitJson(options.json)) { - printJson(properties); - return; - } - - console.log(`Properties (${type}): ${properties.length}`); - printJson(properties); - }); -} diff --git a/cli/commands/query-filters.ts b/cli/commands/query-filters.ts deleted file mode 100644 index bb45838..0000000 --- a/cli/commands/query-filters.ts +++ /dev/null @@ -1,203 +0,0 @@ -import type { VesaiConfig } from "../../config"; -import type { PostHogQueryFilters } from "../../connectors/posthog"; - -export type QueryCommandOptions = { - email?: string; - group?: string; - groupKey?: string; - domain?: string; - allDomains?: boolean; - url?: string; - sessionId?: string; - sessionContains?: string; - distinctId?: string; - from?: string; - to?: string; - minActive?: number; - maxActive?: number; - includeOngoing?: boolean; - requirePerson?: boolean; - where?: string[]; - limit?: number; -}; - -export function collectOption( - value: string, - previous: string[] = [] -): string[] { - return [...previous, value]; -} - -export function parseWhereAssignments( - values: string[] = [] -): Record { - const out: Record = {}; - for (const entry of values) { - const idx = entry.indexOf("="); - if (idx <= 0 || idx === entry.length - 1) { - throw new Error( - `Invalid --where value "${entry}". Expected format: key=value` - ); - } - - const key = entry.slice(0, idx).trim(); - const value = entry.slice(idx + 1).trim(); - if (!(key && value)) { - throw new Error( - `Invalid --where value "${entry}". Expected format: key=value` - ); - } - - out[key] = value; - } - return out; -} - -function parseDateOrThrow( - value: string | undefined, - flag: string -): string | undefined { - if (!value) { - return undefined; - } - const parsed = Date.parse(value); - if (Number.isNaN(parsed)) { - throw new Error(`Invalid ${flag} date "${value}". Use ISO format.`); - } - return new Date(parsed).toISOString(); -} - -function hasMeaningfulFilter(filters: PostHogQueryFilters): boolean { - return Boolean( - filters.text || - filters.email || - filters.groupId || - filters.domain || - filters.urlContains || - filters.sessionId || - filters.sessionContains || - filters.distinctId || - filters.startsAfter || - filters.startsBefore || - filters.minActiveSeconds !== undefined || - filters.maxActiveSeconds !== undefined || - filters.includeOngoing || - filters.requirePerson || - (filters.properties && Object.keys(filters.properties).length > 0) - ); -} - -export function buildQueryFilters(params: { - text?: string; - options: QueryCommandOptions; - config: VesaiConfig; -}): PostHogQueryFilters { - const hasExplicitInput = Boolean( - params.text?.trim() || - params.options.email || - params.options.group || - params.options.groupKey || - params.options.domain || - params.options.allDomains || - params.options.url || - params.options.sessionId || - params.options.sessionContains || - params.options.distinctId || - params.options.from || - params.options.to || - params.options.minActive !== undefined || - params.options.maxActive !== undefined || - params.options.includeOngoing || - params.options.requirePerson || - (params.options.where && params.options.where.length > 0) || - params.options.limit !== undefined - ); - if (!hasExplicitInput) { - throw new Error( - "No query filters provided. Pass text and/or filters (run `vesai replays query --help`)." - ); - } - - const properties = parseWhereAssignments(params.options.where || []); - const startsAfter = parseDateOrThrow(params.options.from, "--from"); - const startsBefore = parseDateOrThrow(params.options.to, "--to"); - - if ( - params.options.minActive !== undefined && - !Number.isFinite(params.options.minActive) - ) { - throw new Error("--min-active must be a valid number."); - } - - if ( - params.options.maxActive !== undefined && - !Number.isFinite(params.options.maxActive) - ) { - throw new Error("--max-active must be a valid number."); - } - - if ( - params.options.limit !== undefined && - !Number.isFinite(params.options.limit) - ) { - throw new Error("--limit must be a valid number."); - } - - if ( - startsAfter && - startsBefore && - new Date(startsAfter).getTime() > new Date(startsBefore).getTime() - ) { - throw new Error("--from must be earlier than or equal to --to."); - } - - if ( - params.options.minActive !== undefined && - params.options.maxActive !== undefined && - params.options.minActive > params.options.maxActive - ) { - throw new Error("--min-active cannot be greater than --max-active."); - } - - if (params.options.limit !== undefined && params.options.limit <= 0) { - throw new Error("--limit must be a positive integer."); - } - - const filters: PostHogQueryFilters = { - text: params.text?.trim() || undefined, - email: params.options.email?.trim() || undefined, - groupId: params.options.group?.trim() || undefined, - groupKey: - params.options.groupKey?.trim() || - (params.options.group ? params.config.posthog.groupKey : undefined), - domain: params.options.allDomains - ? undefined - : params.options.domain?.trim() || params.config.posthog.domainFilter, - urlContains: params.options.url?.trim() || undefined, - sessionId: params.options.sessionId?.trim() || undefined, - sessionContains: params.options.sessionContains?.trim() || undefined, - distinctId: params.options.distinctId?.trim() || undefined, - startsAfter, - startsBefore, - minActiveSeconds: - params.options.minActive === undefined - ? undefined - : params.options.minActive, - maxActiveSeconds: - params.options.maxActive === undefined - ? undefined - : params.options.maxActive, - includeOngoing: params.options.includeOngoing, - requirePerson: params.options.requirePerson, - properties: Object.keys(properties).length ? properties : undefined, - limit: params.options.limit, - }; - - if (!hasMeaningfulFilter(filters)) { - throw new Error( - "No query filters provided. Pass text and/or filters (run `vesai replays query --help`)." - ); - } - - return filters; -} diff --git a/cli/commands/quickstart-command.ts b/cli/commands/quickstart-command.ts index f31f626..ad0ffb7 100644 --- a/cli/commands/quickstart-command.ts +++ b/cli/commands/quickstart-command.ts @@ -1,29 +1,20 @@ import type { Command } from "commander"; -import { ensureVesaiDirectories } from "../../config"; +import { ensureCoreDirectories } from "../../config"; import { type QuickstartCommandOptions, runQuickstartCli } from "./quickstart"; export function registerQuickstartCommand(program: Command): void { program .command("quickstart") - .description("Run CLI setup wizard") - .option("--posthog-host ", "PostHog host URL") - .option("--posthog-api-key ", "PostHog user API key") - .option("--posthog-project-id ", "PostHog project id") - .option("--posthog-group-key ", "PostHog group key") - .option("--domain-filter ", "Replay domain filter") + .description("Run global machine setup (core config + render service)") .option("--gcloud-project-id ", "Google Cloud project id override") .option("--vertex-location ", "Vertex AI location") .option("--bucket-location ", "GCS bucket location") .option("--bucket ", "GCS bucket name") .option( - "--max-concurrent-renders ", - "Max concurrent renders (positive integer)", + "--max-render-memory-mb ", + "Max memory budget for render services in MiB", Number ) - .option( - "--product-description ", - "Product description for analysis context" - ) .option("-y, --yes", "Use defaults where possible without prompting") .option( "--non-interactive", @@ -38,17 +29,17 @@ Requirements before quickstart: - gcloud config set project PostHog key guidance: - - Create User API key: https://app.posthog.com/settings/user-api-keys - - Required scope: All access + MCP server scope + - PostHog setup is now project-scoped via \`vesai init\`. + - Run \`vesai init\` in each repository after global quickstart. Examples: $ vesai quickstart - $ vesai quickstart --yes --posthog-api-key phx_... --posthog-project-id 123 --domain-filter app.example.com --max-concurrent-renders 8 --product-description "B2B SaaS for..." - $ vesai quickstart --non-interactive --posthog-api-key phx_... --posthog-project-id 123 --posthog-group-key company_id --domain-filter app.example.com --product-description "B2B SaaS for..." + $ vesai quickstart --yes --gcloud-project-id my-gcp-project --bucket vesai-my-gcp-project --max-render-memory-mb 8192 + $ vesai quickstart --non-interactive --gcloud-project-id my-gcp-project --bucket vesai-my-gcp-project --vertex-location us-central1 ` ) .action(async (options: QuickstartCommandOptions) => { - await ensureVesaiDirectories(); + await ensureCoreDirectories(); await runQuickstartCli(options); }); } diff --git a/cli/commands/quickstart.ts b/cli/commands/quickstart.ts index 75fc3d2..33138cc 100644 --- a/cli/commands/quickstart.ts +++ b/cli/commands/quickstart.ts @@ -3,10 +3,18 @@ import { stdin as input, stdout as output } from "node:process"; import { createInterface } from "node:readline/promises"; import { DEFAULT_VERTEX_MODEL, - ensureVesaiDirectories, - ensureWorkspaceGitRepo, - saveConfig, + ensureCoreDirectories, + saveCoreConfig, } from "../../config"; +import { + BYTES_PER_MIB, + computeDefaultRenderMemoryMb, + estimateRenderServiceCapacity, + formatGiB, + normalizePositiveIntegerValue, + normalizeRenderMemoryMbValue, + RENDER_MEMORY_PER_SERVICE_MB, +} from "../../config/runtime"; import { ensureBucket, ensurePlaywrightChromiumInstalled, @@ -15,27 +23,15 @@ import { getActiveGcloudProject, hasApplicationDefaultCredentials, isGcloudInstalled, - listProjects, normalizeBucketLocation, - type PostHogProject, } from "../../connectors"; -const POSTHOG_API_KEY_SCOPE_TEXT = "All access + MCP server scope"; -const POSTHOG_API_KEY_SETTINGS_URL = - "https://app.posthog.com/settings/user-api-keys"; - export type QuickstartCommandOptions = { - posthogHost?: string; - posthogApiKey?: string; - posthogProjectId?: string; - posthogGroupKey?: string; - domainFilter?: string; gcloudProjectId?: string; vertexLocation?: string; bucketLocation?: string; bucket?: string; - maxConcurrentRenders?: number; - productDescription?: string; + maxRenderMemoryMb?: number; yes?: boolean; nonInteractive?: boolean; }; @@ -60,49 +56,12 @@ export function defaultBucketLocationFromVertex(location: string): string { return "US"; } -const BYTES_PER_GIB = 1024 * 1024 * 1024; -const BYTES_PER_RENDER_INSTANCE = 512 * 1024 * 1024; - -export function computeDefaultRenderConcurrency( - availableRamBytes: number = freemem() -): number { - if (!Number.isFinite(availableRamBytes) || availableRamBytes <= 0) { - return 1; - } - const budget = availableRamBytes * 0.5; - return Math.max(1, Math.floor(budget / BYTES_PER_RENDER_INSTANCE)); -} - -export function formatGiB(bytes: number): string { - if (!Number.isFinite(bytes) || bytes <= 0) { - return "0.0"; - } - return (bytes / BYTES_PER_GIB).toFixed(1); -} - export function normalizeConcurrencyValue( value: string | number | undefined, fallback: number, flag: string ): number { - if (value === undefined || value === null || String(value).trim() === "") { - return fallback; - } - - const parsed = Number(value); - if (!(Number.isFinite(parsed) && Number.isInteger(parsed))) { - throw new Error( - `Invalid ${flag} value "${value}". Must be a positive integer.` - ); - } - - if (parsed < 1) { - throw new Error( - `Invalid ${flag} value "${value}". Must be a positive integer.` - ); - } - - return parsed; + return normalizePositiveIntegerValue(value, fallback, flag); } export function parseProjectSelectionIndex( @@ -162,11 +121,7 @@ async function resolveValue(params: { const raw = await params.promptClient.ask(`${params.label}${suffix}: `); const picked = raw.trim() || params.defaultValue || ""; - if (!required) { - return picked; - } - - if (picked) { + if (!required || picked) { return picked; } @@ -174,65 +129,6 @@ async function resolveValue(params: { } } -async function choosePostHogProjectId(params: { - projects: PostHogProject[]; - optionValue: string | undefined; - nonInteractive: boolean; - promptClient: PromptClient | null; - useDefaultsWithoutPrompt: boolean; -}): Promise { - const normalizedOption = params.optionValue?.trim(); - if (normalizedOption) { - const exists = params.projects.some( - (project) => String(project.id) === normalizedOption - ); - if (!exists) { - const available = params.projects - .map((project) => `${project.name} (${project.id})`) - .join(", "); - throw new Error( - `Unknown PostHog project id "${normalizedOption}". Available projects: ${available}` - ); - } - return normalizedOption; - } - - if (params.useDefaultsWithoutPrompt) { - return String(params.projects[0]!.id); - } - - if (params.nonInteractive) { - throw new Error( - "Missing required option --posthog-project-id in non-interactive mode." - ); - } - - if (!params.promptClient) { - throw new Error("Prompt client unavailable for project selection."); - } - - console.log(""); - console.log("Available PostHog projects:"); - for (let i = 0; i < params.projects.length; i++) { - const project = params.projects[i]!; - console.log(` ${i + 1}. ${project.name} (${project.id})`); - } - - while (true) { - const raw = await params.promptClient.ask("Select project number [1]: "); - const selection = raw.trim() || "1"; - try { - const index = parseProjectSelectionIndex( - selection, - params.projects.length - ); - return String(params.projects[index]!.id); - } catch (error) { - console.log(error instanceof Error ? error.message : String(error)); - } - } -} - export async function runQuickstartCli( options: QuickstartCommandOptions ): Promise { @@ -243,16 +139,14 @@ export async function runQuickstartCli( const promptClient = nonInteractive ? null : createPromptClient(); try { - console.log("VES AI quickstart (CLI)"); + console.log("VES AI quickstart (global core setup)"); console.log("Running preflight checks..."); - const gcloudInstalled = await isGcloudInstalled(); - if (!gcloudInstalled) { + if (!(await isGcloudInstalled())) { throw new Error("gcloud CLI is required but not installed."); } - const account = await getActiveGcloudAccount(); - if (!account) { + if (!(await getActiveGcloudAccount())) { throw new Error( "No active gcloud account. Run `gcloud auth login` before quickstart." ); @@ -265,8 +159,7 @@ export async function runQuickstartCli( ); } - const hasAdc = await hasApplicationDefaultCredentials(); - if (!hasAdc) { + if (!(await hasApplicationDefaultCredentials())) { throw new Error( "Application default credentials are missing. Run `gcloud auth application-default login`." ); @@ -278,72 +171,15 @@ export async function runQuickstartCli( const gcloudProjectId = options.gcloudProjectId?.trim() || activeProjectId || ""; const availableRamBytes = freemem(); - const defaultRenderConcurrency = - computeDefaultRenderConcurrency(availableRamBytes); + const defaultRenderMemoryMb = + computeDefaultRenderMemoryMb(availableRamBytes); + const defaultRenderCapacity = estimateRenderServiceCapacity( + defaultRenderMemoryMb + ); console.log( - `Render concurrency default: ${defaultRenderConcurrency} (~50% of ${formatGiB(availableRamBytes)} GiB available RAM at ~512MB per renderer).` + `Render memory budget default: ${defaultRenderMemoryMb} MiB (~${formatGiB(defaultRenderMemoryMb * BYTES_PER_MIB)} GiB, ~${defaultRenderCapacity} render services at ${RENDER_MEMORY_PER_SERVICE_MB} MiB each).` ); - const posthogHost = await resolveValue({ - promptClient, - label: "PostHog host URL", - flag: "posthog-host", - value: options.posthogHost, - defaultValue: "https://us.posthog.com", - nonInteractive, - useDefaultsWithoutPrompt, - }); - - console.log(""); - console.log(`PostHog API key requirements: ${POSTHOG_API_KEY_SCOPE_TEXT}`); - console.log(`Create key: ${POSTHOG_API_KEY_SETTINGS_URL}`); - - const posthogApiKey = await resolveValue({ - promptClient, - label: "PostHog API key", - flag: "posthog-api-key", - value: options.posthogApiKey, - nonInteractive, - useDefaultsWithoutPrompt, - }); - - console.log("Loading PostHog projects..."); - const projects = await listProjects({ - host: posthogHost, - apiKey: posthogApiKey, - }); - - if (!projects.length) { - throw new Error("No PostHog projects found with this key."); - } - - const posthogProjectId = await choosePostHogProjectId({ - projects, - optionValue: options.posthogProjectId, - nonInteractive, - promptClient, - useDefaultsWithoutPrompt, - }); - - const posthogGroupKey = await resolveValue({ - promptClient, - label: "PostHog group key", - flag: "posthog-group-key", - value: options.posthogGroupKey, - defaultValue: "company_id", - nonInteractive, - useDefaultsWithoutPrompt, - }); - - const domainFilter = await resolveValue({ - promptClient, - label: "Domain filter for session replays", - flag: "domain-filter", - value: options.domainFilter, - nonInteractive, - useDefaultsWithoutPrompt, - }); - const vertexLocation = await resolveValue({ promptClient, label: "Vertex AI location", @@ -379,36 +215,26 @@ export async function runQuickstartCli( }) ); - const maxConcurrentRendersInput = await resolveValue({ + const maxRenderMemoryInput = await resolveValue({ promptClient, - label: "Max concurrent renders (>=1)", - flag: "max-concurrent-renders", + label: `Max render memory budget in MiB (>=${RENDER_MEMORY_PER_SERVICE_MB})`, + flag: "max-render-memory-mb", value: - options.maxConcurrentRenders === undefined + options.maxRenderMemoryMb === undefined ? undefined - : String(options.maxConcurrentRenders), - defaultValue: String(defaultRenderConcurrency), + : String(options.maxRenderMemoryMb), + defaultValue: String(defaultRenderMemoryMb), nonInteractive, useDefaultsWithoutPrompt, }); - const maxConcurrentRenders = normalizeConcurrencyValue( - maxConcurrentRendersInput, - 2, - "--max-concurrent-renders" + const maxRenderMemoryMb = normalizeRenderMemoryMbValue( + maxRenderMemoryInput, + defaultRenderMemoryMb, + "--max-render-memory-mb" ); - const productDescription = await resolveValue({ - promptClient, - label: "Describe your product for better analysis", - flag: "product-description", - value: options.productDescription, - nonInteractive, - useDefaultsWithoutPrompt, - }); - - console.log("Provisioning local config and cloud resources..."); - await ensureVesaiDirectories(); - await ensureWorkspaceGitRepo(); + console.log("Provisioning core config and cloud resources..."); + await ensureCoreDirectories(); await ensureRequiredApis(gcloudProjectId); await ensureBucket({ bucket, @@ -416,15 +242,8 @@ export async function runQuickstartCli( location: bucketLocation, }); - await saveConfig({ + await saveCoreConfig({ version: 1, - posthog: { - host: posthogHost, - apiKey: posthogApiKey, - projectId: posthogProjectId, - groupKey: posthogGroupKey, - domainFilter, - }, gcloud: { projectId: gcloudProjectId, region: bucketLocation, @@ -435,17 +254,15 @@ export async function runQuickstartCli( location: vertexLocation, }, runtime: { - maxConcurrentRenders, - }, - product: { - description: productDescription, + maxRenderMemoryMb, }, }); - console.log("Quickstart complete."); + console.log("Global quickstart complete."); console.log("Next steps:"); - console.log(" vesai daemon start"); - console.log(" vesai replays user you@example.com"); + console.log(" 1) cd "); + console.log(" 2) vesai init"); + console.log(" 3) vesai daemon start"); } finally { promptClient?.close(); } diff --git a/cli/commands/replays-command.ts b/cli/commands/replays-command.ts deleted file mode 100644 index 6509e3a..0000000 --- a/cli/commands/replays-command.ts +++ /dev/null @@ -1,483 +0,0 @@ -import type { Command } from "commander"; -import { - ensurePlaywrightChromiumInstalled, - findRecordingById, - findRecordingsByQuery, - getRecordingUserEmail, -} from "../../connectors"; -import { - analyzeGroupById, - analyzeQuery, - analyzeUserByEmail, - renderAndAnalyzeSessions, -} from "../analysis"; -import { printJson } from "./helpers"; -import { - buildQueryFilters, - collectOption, - type QueryCommandOptions, -} from "./query-filters"; -import { createReplayProgressRenderer } from "./replays-progress"; -import { - ensureReplayContext, - printResult, - type ReplayRunOptions, - resolveSessionConcurrency, - shouldEmitJson, - withRenderLogMode, -} from "./runtime"; - -type ReplayQueryOptions = QueryCommandOptions & - ReplayRunOptions & { - dryRun?: boolean; - }; - -function addReplayFilterOptions(command: T): T { - return command - .option("--email ", "Filter by user email") - .option("--group ", "Filter by group identifier") - .option("--group-key ", "Group property key override") - .option("--domain ", "Filter URLs by domain (default from config)") - .option("--all-domains", "Disable default domain filter") - .option("--url ", "Filter by URL substring") - .option("--session-id ", "Filter by exact session ID") - .option("--session-contains ", "Filter by session ID substring") - .option("--distinct-id ", "Filter by exact distinct_id") - .option("--from ", "Session start on/after ISO timestamp") - .option("--to ", "Session start on/before ISO timestamp") - .option("--min-active ", "Minimum active seconds", Number) - .option("--max-active ", "Maximum active seconds", Number) - .option("--include-ongoing", "Include sessions still recording") - .option("--require-person", "Require person profile on each session") - .option( - "--where ", - "Exact person property filter (repeatable)", - collectOption, - [] - ) - .option("--limit ", "Limit number of sessions", Number); -} - -export function registerReplaysCommand(program: Command): void { - const replays = program - .command("replays") - .description("Run synchronous replay analysis flows"); - - replays.addHelpText( - "after", - ` -Examples: - $ vesai replays session ph_abc123 - $ vesai replays user user@example.com - $ vesai replays group acme-inc - $ vesai replays query "checkout friction" --email user@example.com --min-active 30 - $ vesai replays list --group acme-inc --limit 50 - -Agent tips: - - Use --dry-run to estimate runtime before analysis. - - JSON output is default. Use --no-json for human-readable output. - - Text query is literal search over replay/session metadata, not semantic intent. -` - ); - - replays - .command("session ") - .description("Render and analyze one session replay synchronously") - .option("--no-json", "Output human-readable text") - .option("--verbose", "Show low-level render/debug logs") - .addHelpText( - "after", - ` -What this does: - - Fetch one replay by session id. - - Render replay events + video. - - Analyze session quality and write markdown to workspace. - -Examples: - $ vesai replays session ph_abc123 - $ vesai replays session ph_abc123 --no-json -` - ) - .action( - async ( - sessionId: string, - options: { json?: boolean; verbose?: boolean } - ) => { - const { config } = await ensureReplayContext(); - await ensurePlaywrightChromiumInstalled(); - - const recording = await findRecordingById({ - host: config.posthog.host, - apiKey: config.posthog.apiKey, - projectId: config.posthog.projectId, - sessionId, - }); - - if (!recording) { - throw new Error(`Session ${sessionId} was not found in PostHog.`); - } - - const progress = shouldEmitJson(options.json) - ? null - : createReplayProgressRenderer({ - label: "replays session", - }); - - try { - const results = await withRenderLogMode(options.verbose, async () => - renderAndAnalyzeSessions({ - recordings: [recording], - context: { config }, - concurrency: 1, - onProgress: progress?.handle, - }) - ); - - const result = results[0]!; - const output = { - sessionId: result.recording.id, - name: result.analysis.name, - score: result.analysis.score, - health: result.analysis.health, - markdownPath: result.markdownPath, - videoUri: result.render.videoUri, - eventsUri: result.render.eventsUri, - }; - - if (shouldEmitJson(options.json)) { - printJson(output); - return; - } - - console.log(`Session: ${output.sessionId}`); - console.log(`Name: ${output.name}`); - console.log(`Score: ${output.score}`); - console.log(`Health: ${output.health}`); - console.log(`Markdown: ${output.markdownPath}`); - console.log(`Video: ${output.videoUri}`); - console.log(`Events: ${output.eventsUri}`); - } finally { - progress?.close(); - } - } - ); - - replays - .command("user ") - .description("Analyze a user across all matching replay sessions") - .option( - "--max-concurrent ", - "Max concurrent session pipelines for this run", - Number - ) - .option("--no-json", "Output human-readable text") - .option("--verbose", "Show low-level render/debug logs") - .addHelpText( - "after", - ` -User analysis contract: - - Analyze each session for this user first. - - Then run one aggregate user inference across all sessions + metadata. - -Learning flow: - 1) Preview scope: vesai replays list --email user@example.com --limit 10 - 2) Estimate runtime: vesai replays query --email user@example.com --dry-run - 3) Run full analysis: vesai replays user user@example.com -` - ) - .action(async (email: string, options: ReplayRunOptions) => { - const { config } = await ensureReplayContext(); - await ensurePlaywrightChromiumInstalled(); - - const progress = shouldEmitJson(options.json) - ? null - : createReplayProgressRenderer({ - label: `replays user ${email}`, - }); - - try { - const result = await withRenderLogMode(options.verbose, async () => - analyzeUserByEmail({ - email, - context: { config }, - sessionConcurrency: resolveSessionConcurrency(options, config), - onSessionProgress: progress?.handle, - }) - ); - - const output = { - email: result.email, - sessionCount: result.sessionCount, - averageSessionScore: result.averageSessionScore, - userScore: result.userScore, - markdownPath: result.markdownPath, - }; - - if (shouldEmitJson(options.json)) { - printJson(output); - return; - } - - console.log(`User: ${output.email}`); - console.log(`Sessions: ${output.sessionCount}`); - console.log(`Average session score: ${output.averageSessionScore}`); - console.log(`User score: ${output.userScore}`); - console.log(`Markdown: ${output.markdownPath}`); - } finally { - progress?.close(); - } - }); - - replays - .command("group ") - .description("Analyze one group via replay sessions and user rollups") - .option( - "--max-concurrent ", - "Max concurrent session pipelines for each user", - Number - ) - .option("--no-json", "Output human-readable text") - .option("--verbose", "Show low-level render/debug logs") - .addHelpText( - "after", - ` -What this does: - - Resolve users in the group. - - Analyze each user from replay evidence. - - Produce one group summary and markdown artifact. - -Examples: - $ vesai replays group acme - $ vesai replays group acme --max-concurrent 6 -` - ) - .action(async (groupId: string, options: ReplayRunOptions) => { - const { config } = await ensureReplayContext(); - await ensurePlaywrightChromiumInstalled(); - - const progress = shouldEmitJson(options.json) - ? null - : createReplayProgressRenderer({ - label: `replays group ${groupId}`, - }); - - try { - const result = await withRenderLogMode(options.verbose, async () => - analyzeGroupById({ - groupId, - context: { config }, - sessionConcurrency: resolveSessionConcurrency(options, config), - onSessionProgress: progress?.handle, - }) - ); - - if (shouldEmitJson(options.json)) { - printJson(result); - return; - } - - console.log(`Group: ${result.groupId}`); - console.log(`Users analyzed: ${result.usersAnalyzed}`); - console.log(`Score: ${result.score}`); - console.log(`Markdown: ${result.markdownPath}`); - } finally { - progress?.close(); - } - }); - - addReplayFilterOptions( - replays - .command("query [text]") - .description("Analyze replays matching text + structured filters") - ) - .option( - "--max-concurrent ", - "Max concurrent session pipelines for this run", - Number - ) - .option( - "--dry-run", - "Resolve matching sessions and estimate workload without AI analysis" - ) - .option("--verbose", "Show low-level render/debug logs") - .option("--no-json", "Output human-readable text") - .addHelpText( - "after", - ` -Examples: - $ vesai replays query "checkout friction" - $ vesai replays query "checkout" --url /checkout --min-active 30 --where plan=enterprise - $ vesai replays query --email jane@acme.com --from 2026-01-01 --to 2026-01-31 - $ vesai replays query --group acme --group-key company_id --where plan=enterprise - $ vesai replays query --url /checkout --min-active 45 --limit 25 - $ vesai replays query --group acme --dry-run - -Filter strategy for "checkout friction": - - Text query is literal search over replay/session metadata. - - Start with --url /checkout or a checkout event/property filter via --where. - - Add --min-active to prioritize high-effort sessions. - - Use --from/--to to isolate a release window. -` - ) - .action( - async (queryText: string | undefined, options: ReplayQueryOptions) => { - const { config } = await ensureReplayContext(); - - const filters = buildQueryFilters({ - text: queryText, - options, - config, - }); - - if (options.dryRun) { - const recordings = await findRecordingsByQuery({ - host: config.posthog.host, - apiKey: config.posthog.apiKey, - projectId: config.posthog.projectId, - query: queryText, - filters, - domainFilter: config.posthog.domainFilter, - }); - - const concurrency = resolveSessionConcurrency(options, config); - const totalActiveSeconds = recordings.reduce( - (sum, recording) => - sum + Math.max(0, Number(recording.active_seconds ?? 0)), - 0 - ); - const estimatedRenderSeconds = recordings.reduce((sum, recording) => { - const activeSeconds = Math.max( - 1, - Number(recording.active_seconds ?? 0) - ); - return sum + Math.max(25, Math.round(activeSeconds * 1.65 + 24)); - }, 0); - - const estimatedWallClockSeconds = - recordings.length > 0 - ? Math.ceil(estimatedRenderSeconds / Math.max(1, concurrency)) - : 0; - - const sample = recordings.slice(0, 20).map((recording) => ({ - sessionId: recording.id, - startTime: recording.start_time ?? null, - activeSeconds: recording.active_seconds ?? 0, - url: recording.start_url ?? null, - email: getRecordingUserEmail(recording), - })); - - const output = { - mode: "dry-run", - query: queryText || "", - filters, - matchedSessions: recordings.length, - totalActiveSeconds, - estimatedRenderSeconds, - estimatedWallClockSeconds, - recommendedMaxConcurrent: concurrency, - sample, - }; - - printResult(output, shouldEmitJson(options.json)); - return; - } - - await ensurePlaywrightChromiumInstalled(); - - const progress = shouldEmitJson(options.json) - ? null - : createReplayProgressRenderer({ - label: "replays query", - }); - - try { - const result = await withRenderLogMode(options.verbose, async () => - analyzeQuery({ - query: queryText, - filters, - context: { config }, - sessionConcurrency: resolveSessionConcurrency(options, config), - onSessionProgress: progress?.handle, - }) - ); - - if (shouldEmitJson(options.json)) { - printJson(result); - return; - } - - console.log(`Query: ${result.query || ""}`); - console.log(`Sessions analyzed: ${result.sessionCount}`); - console.log(`Average score: ${result.averageScore}`); - } finally { - progress?.close(); - } - } - ); - - addReplayFilterOptions( - replays - .command("list [text]") - .description( - "List replay sessions matching filters without running AI analysis" - ) - ) - .option("--no-json", "Output human-readable text") - .addHelpText( - "after", - ` -Use this to discover candidates before expensive analysis. - -Examples: - $ vesai replays list --email user@example.com --limit 25 - $ vesai replays list "checkout" --url /checkout --min-active 30 - -Next step: - - Use the same filters with \`vesai replays query\` to run AI analysis. -` - ) - .action( - async (queryText: string | undefined, options: ReplayQueryOptions) => { - const { config } = await ensureReplayContext(); - - const effectiveOptions: QueryCommandOptions = { - ...options, - limit: options.limit ?? 50, - }; - - const filters = buildQueryFilters({ - text: queryText, - options: effectiveOptions, - config, - }); - - const recordings = await findRecordingsByQuery({ - host: config.posthog.host, - apiKey: config.posthog.apiKey, - projectId: config.posthog.projectId, - query: queryText, - filters, - domainFilter: config.posthog.domainFilter, - }); - - const rows = recordings.map((recording) => ({ - id: recording.id, - start_time: recording.start_time, - end_time: recording.end_time, - active_seconds: recording.active_seconds, - start_url: recording.start_url, - distinct_id: recording.distinct_id, - email: getRecordingUserEmail(recording), - ongoing: recording.ongoing ?? false, - })); - - if (shouldEmitJson(options.json)) { - printJson({ count: rows.length, rows }); - return; - } - - console.log(`Matched sessions: ${rows.length}`); - printJson(rows); - } - ); -} diff --git a/cli/commands/research-command.ts b/cli/commands/research-command.ts new file mode 100644 index 0000000..cfb1e39 --- /dev/null +++ b/cli/commands/research-command.ts @@ -0,0 +1,64 @@ +import type { Command } from "commander"; +import { researchFromAnalyzedSessions } from "../analysis"; +import { printJson } from "./helpers"; +import { ensureReplayContext, shouldEmitJson } from "./runtime"; + +type ResearchCommandOptions = { + limit?: number; + json?: boolean; +}; + +export function registerResearchCommand(program: Command): void { + program + .command("research ") + .description( + "Answer a product research question using already analyzed replay sessions" + ) + .option( + "--limit ", + "Max analyzed sessions to include in context", + Number + ) + .option("--no-json", "Output human-readable text") + .addHelpText( + "after", + ` +Important: + - Uses only already analyzed sessions from workspace. + - Does not fetch or analyze new sessions. +` + ) + .action(async (question: string, options: ResearchCommandOptions) => { + const { config } = await ensureReplayContext(); + const result = await researchFromAnalyzedSessions({ + question, + context: { config }, + limit: options.limit, + }); + + if (shouldEmitJson(options.json)) { + printJson(result); + return; + } + + console.log(`Question: ${result.question}`); + console.log(`Confidence: ${result.confidence}`); + console.log( + `Sessions: used ${result.sessionsUsed} of ${result.sessionsConsidered} analyzed sessions` + ); + if (result.supportingSessionIds.length) { + console.log( + `Supporting sessions: ${result.supportingSessionIds.join(", ")}` + ); + } + if (result.findings.length) { + console.log(""); + console.log("Findings:"); + for (const finding of result.findings) { + console.log(`- ${finding}`); + } + } + console.log(""); + console.log(result.answer); + }); +} diff --git a/cli/commands/runtime.ts b/cli/commands/runtime.ts index 46a8d94..a15133f 100644 --- a/cli/commands/runtime.ts +++ b/cli/commands/runtime.ts @@ -1,9 +1,10 @@ import { - ensureVesaiDirectories, + ensureCoreDirectories, + ensureProjectDirectories, requireConfig, type VesaiConfig, } from "../../config"; -import { printJson } from "./helpers"; +import { computeDynamicRenderServiceCapacity } from "../../config/runtime"; export type ReplayRunOptions = { json?: boolean; @@ -55,7 +56,8 @@ export async function withRenderLogMode( } export async function ensureReplayContext(): Promise<{ config: VesaiConfig }> { - await ensureVesaiDirectories(); + await ensureCoreDirectories(); + await ensureProjectDirectories(); const config = await requireConfig(); return { config }; } @@ -65,43 +67,14 @@ export function resolveSessionConcurrency( config: VesaiConfig ): number { const override = toPositiveInt(options.maxConcurrent, "--max-concurrent"); - return override ?? config.runtime.maxConcurrentRenders; + return ( + override ?? + computeDynamicRenderServiceCapacity({ + maxRenderMemoryMb: config.runtime.maxRenderMemoryMb, + }) + ); } export function shouldEmitJson(value: boolean | undefined): boolean { return value !== false; } - -export function printResult(value: unknown, json = true): void { - if (json) { - printJson(value); - return; - } - - if (typeof value === "string") { - console.log(value); - return; - } - - console.log(JSON.stringify(value, null, 2)); -} - -function toNumberIfFinite(value: string): number | string { - const parsed = Number(value); - if (Number.isFinite(parsed)) { - return parsed; - } - return value; -} - -export function mapRowToObject( - columns: string[], - row: string[] -): Record { - const out: Record = {}; - for (let i = 0; i < columns.length; i++) { - const key = columns[i] || `column_${i + 1}`; - out[key] = toNumberIfFinite(row[i] ?? ""); - } - return out; -} diff --git a/cli/commands/schema-command.ts b/cli/commands/schema-command.ts deleted file mode 100644 index 24504be..0000000 --- a/cli/commands/schema-command.ts +++ /dev/null @@ -1,207 +0,0 @@ -import type { Command } from "commander"; -import { requireConfig } from "../../config"; -import { - listEventDefinitions, - readDataSchema, - readDataWarehouseSchema, -} from "../../connectors"; -import { printResult, shouldEmitJson } from "./runtime"; - -type SchemaDataCommandOptions = { - kind?: string; - eventName?: string; - entity?: string; - propertyName?: string; - actionId?: string; - search?: string; - limit?: number; - json?: boolean; -}; - -type SchemaWarehouseOptions = { - json?: boolean; -}; - -function buildTaxonomyQuery( - options: SchemaDataCommandOptions -): Record { - const rawKind = (options.kind || "events").trim().toLowerCase(); - if (rawKind === "events") { - return { kind: "events" }; - } - if (rawKind === "event-properties" || rawKind === "event_properties") { - if (!options.eventName) { - throw new Error("--event-name is required when --kind event-properties."); - } - return { - kind: "event_properties", - event_name: options.eventName, - }; - } - if (rawKind === "entity-properties" || rawKind === "entity_properties") { - if (!options.entity) { - throw new Error("--entity is required when --kind entity-properties."); - } - return { - kind: "entity_properties", - entity: options.entity, - }; - } - if (rawKind === "action-properties" || rawKind === "action_properties") { - const actionId = Number(options.actionId); - if (!(Number.isInteger(actionId) && actionId > 0)) { - throw new Error( - "--action-id must be a positive integer when --kind action-properties." - ); - } - return { - kind: "action_properties", - action_id: actionId, - }; - } - if ( - rawKind === "entity-property-values" || - rawKind === "entity_property_values" - ) { - if (!(options.entity && options.propertyName)) { - throw new Error( - "--entity and --property-name are required when --kind entity-property-values." - ); - } - return { - kind: "entity_property_values", - entity: options.entity, - property_name: options.propertyName, - }; - } - if ( - rawKind === "event-property-values" || - rawKind === "event_property_values" - ) { - if (!(options.eventName && options.propertyName)) { - throw new Error( - "--event-name and --property-name are required when --kind event-property-values." - ); - } - return { - kind: "event_property_values", - event_name: options.eventName, - property_name: options.propertyName, - }; - } - if ( - rawKind === "action-property-values" || - rawKind === "action_property_values" - ) { - const actionId = Number(options.actionId); - if (!(Number.isInteger(actionId) && actionId > 0 && options.propertyName)) { - throw new Error( - "--action-id (positive integer) and --property-name are required when --kind action-property-values." - ); - } - return { - kind: "action_property_values", - action_id: actionId, - property_name: options.propertyName, - }; - } - throw new Error( - `Invalid --kind '${options.kind}'. See \`vesai schema data --help\`.` - ); -} - -export function registerSchemaCommand(program: Command): void { - const schema = program - .command("schema") - .description("Inspect PostHog data schemas via MCP-backed APIs"); - - schema - .command("data [query]") - .description("Read PostHog event/property taxonomy") - .option( - "--kind ", - "events|event-properties|entity-properties|action-properties|entity-property-values|event-property-values|action-property-values" - ) - .option("--event-name ", "Event name for event-* kinds") - .option("--entity ", "Entity name for entity-* kinds") - .option( - "--property-name ", - "Property name for *-property-values kinds" - ) - .option("--action-id ", "Action id for action-* kinds") - .option("--search ", "Shortcut: search event definitions by text") - .option( - "--limit ", - "Search limit (used with positional query or --search)", - Number - ) - .option("--no-json", "Output human-readable text") - .addHelpText( - "after", - ` -Examples: - $ vesai schema data --kind events - $ vesai schema data --kind event-properties --event-name "$pageview" - $ vesai schema data --kind event-property-values --event-name "$pageview" --property-name "$browser" - $ vesai schema data checkout --limit 20 -` - ) - .action( - async (query: string | undefined, options: SchemaDataCommandOptions) => { - const config = await requireConfig(); - const searchText = options.search?.trim() || query?.trim(); - if (searchText) { - const events = await listEventDefinitions({ - host: config.posthog.host, - apiKey: config.posthog.apiKey, - projectId: config.posthog.projectId, - search: searchText, - limit: options.limit ?? 25, - offset: 0, - }); - - const result = { - mode: "event_search", - search: searchText, - count: events.length, - events, - }; - printResult(result, shouldEmitJson(options.json)); - return; - } - - const taxonomyQuery = buildTaxonomyQuery(options); - const result = await readDataSchema({ - host: config.posthog.host, - apiKey: config.posthog.apiKey, - projectId: config.posthog.projectId, - query: taxonomyQuery, - }); - - printResult(result, shouldEmitJson(options.json)); - } - ); - - schema - .command("warehouse") - .description("Read PostHog warehouse/session/person schema information") - .option("--no-json", "Output human-readable text") - .addHelpText( - "after", - ` -Examples: - $ vesai schema warehouse - $ vesai schema warehouse --no-json -` - ) - .action(async (options: SchemaWarehouseOptions) => { - const config = await requireConfig(); - const result = await readDataWarehouseSchema({ - host: config.posthog.host, - apiKey: config.posthog.apiKey, - projectId: config.posthog.projectId, - }); - - printResult(result, shouldEmitJson(options.json)); - }); -} diff --git a/cli/commands/user-command.ts b/cli/commands/user-command.ts new file mode 100644 index 0000000..43a084b --- /dev/null +++ b/cli/commands/user-command.ts @@ -0,0 +1,81 @@ +import type { Command } from "commander"; +import { ensurePlaywrightChromiumInstalled } from "../../connectors"; +import { analyzeUserByEmail } from "../analysis"; +import { printJson } from "./helpers"; +import { createReplayProgressRenderer } from "./replays-progress"; +import { + ensureReplayContext, + type ReplayRunOptions, + resolveSessionConcurrency, + shouldEmitJson, + withRenderLogMode, +} from "./runtime"; + +export function registerUserCommand(program: Command): void { + program + .command("user ") + .description("Analyze one user story from all matching replay sessions") + .option( + "--max-concurrent ", + "Max concurrent session pipelines for this run", + Number + ) + .option("--verbose", "Show low-level render/debug logs") + .option("--no-json", "Output human-readable text") + .addHelpText( + "after", + ` +What this does: + - Finds all sessions for the user. + - Analyzes each session replay. + - Returns one aggregate user story. +` + ) + .action(async (email: string, options: ReplayRunOptions) => { + const { config } = await ensureReplayContext(); + await ensurePlaywrightChromiumInstalled(); + + const progress = shouldEmitJson(options.json) + ? null + : createReplayProgressRenderer({ + label: `user ${email}`, + }); + + try { + const result = await withRenderLogMode(options.verbose, async () => + analyzeUserByEmail({ + email, + context: { config }, + sessionConcurrency: resolveSessionConcurrency(options, config), + onSessionProgress: progress?.handle, + }) + ); + + const output = { + email: result.email, + sessionCount: result.sessionCount, + averageSessionScore: result.averageSessionScore, + userScore: result.userScore, + health: result.health, + story: result.story, + markdownPath: result.markdownPath, + }; + + if (shouldEmitJson(options.json)) { + printJson(output); + return; + } + + console.log(`User: ${output.email}`); + console.log(`Sessions: ${output.sessionCount}`); + console.log(`Average session score: ${output.averageSessionScore}`); + console.log(`User score: ${output.userScore}`); + console.log(`Health: ${output.health}`); + console.log(`Markdown: ${output.markdownPath}`); + console.log(""); + console.log(output.story); + } finally { + progress?.close(); + } + }); +} diff --git a/cli/index.ts b/cli/index.ts index 7a4b59f..9aa2c60 100644 --- a/cli/index.ts +++ b/cli/index.ts @@ -1,17 +1,19 @@ #!/usr/bin/env node import { Command } from "commander"; -import { registerAnalyticsCommands } from "./commands/analytics-commands"; import { registerConfigCommand } from "./commands/config-command"; import { registerDaemonCommand } from "./commands/daemon-command"; import { registerDoctorCommand } from "./commands/doctor-command"; +import { registerGroupCommand } from "./commands/group-command"; +import { registerInitCommand } from "./commands/init-command"; import { registerQuickstartCommand } from "./commands/quickstart-command"; -import { registerReplaysCommand } from "./commands/replays-command"; +import { registerResearchCommand } from "./commands/research-command"; +import { registerUserCommand } from "./commands/user-command"; const program = new Command(); program .name("vesai") - .description("VES AI: AI-ready product analytics") + .description("VES AI: session replay intelligence for agents") .version("0.1.0"); program.showSuggestionAfterError(true); program.showHelpAfterError( @@ -22,22 +24,17 @@ program.addHelpText( ` Quickstart: $ vesai quickstart + $ vesai init -Replay Workflows: - $ vesai replays user bryce@lenny.com - $ vesai replays session ph_abc123 - $ vesai replays query "checkout friction" --from 2026-01-01 --to 2026-01-31 +Replay Story Workflows: + $ vesai user bryce@lenny.com + $ vesai group acme-inc + $ vesai research "what causes checkout abandonment?" -Analytics Workflows: - $ vesai events --search checkout - $ vesai insights hogql "weekly active users by plan" - $ vesai insights sql "SELECT event, count() FROM events GROUP BY event LIMIT 20" - -Agent Workflows: - $ vesai replays query --group acme --min-active 30 --dry-run - $ vesai replays query --group acme --min-active 30 - $ vesai insights sql "SELECT event, count() FROM events GROUP BY event LIMIT 20" - $ vesai config show --show-secrets +Daemon Workflows: + $ vesai daemon start + $ vesai daemon status + $ vesai daemon stop Output mode: - JSON is default for data commands. @@ -46,8 +43,10 @@ Output mode: ); registerQuickstartCommand(program); -registerReplaysCommand(program); -registerAnalyticsCommands(program); +registerInitCommand(program); +registerUserCommand(program); +registerGroupCommand(program); +registerResearchCommand(program); registerDaemonCommand(program); registerConfigCommand(program); registerDoctorCommand(program); diff --git a/config/index.ts b/config/index.ts index ae23d5a..2040f26 100644 --- a/config/index.ts +++ b/config/index.ts @@ -1,36 +1,76 @@ import { spawn } from "node:child_process"; import { mkdir, readFile, writeFile } from "node:fs/promises"; -import { dirname } from "node:path"; -import { getVesaiPaths, resolveVesaiHome } from "./paths"; -import { type VesaiConfig, VesaiConfigSchema } from "./schema"; +import { dirname, join, resolve } from "node:path"; +import { + getVesaiCorePaths, + getVesaiPaths, + resolveProjectRoot, + resolveVesaiHome, +} from "./paths"; +import { legacyConcurrencyToRenderMemoryMb } from "./runtime"; +import { + type CoreConfig, + CoreConfigSchema, + type ProjectConfig, + ProjectConfigSchema, + type VesaiConfig, +} from "./schema"; -export { getVesaiPaths, resolveVesaiHome } from "./paths"; -export type { VesaiConfig } from "./schema"; -export { DEFAULT_VERTEX_MODEL, VesaiConfigSchema } from "./schema"; +export { + findProjectRoot, + getVesaiCorePaths, + getVesaiPaths, + resolveProjectRoot, + resolveVesaiHome, +} from "./paths"; +export type { CoreConfig, ProjectConfig, VesaiConfig } from "./schema"; +export { + CoreConfigSchema, + DEFAULT_VERTEX_MODEL, + ProjectConfigSchema, +} from "./schema"; -export async function ensureVesaiDirectories(homeDir?: string): Promise { - const paths = getVesaiPaths(homeDir ?? resolveVesaiHome()); +export async function ensureCoreDirectories(homeDir?: string): Promise { + const paths = getVesaiCorePaths(homeDir ?? resolveVesaiHome()); await Promise.all([ mkdir(paths.home, { recursive: true }), + mkdir(paths.logsDir, { recursive: true }), + mkdir(paths.tmpDir, { recursive: true }), + mkdir(paths.renderLocksDir, { recursive: true }), + mkdir(dirname(paths.appDir), { recursive: true }), + ]); +} + +export async function ensureProjectDirectories( + projectRoot?: string +): Promise { + const paths = getVesaiPaths(projectRoot ?? resolveProjectRoot()); + await Promise.all([ + mkdir(paths.vesaiDir, { recursive: true }), mkdir(paths.workspace, { recursive: true }), mkdir(paths.sessionsDir, { recursive: true }), mkdir(paths.usersDir, { recursive: true }), mkdir(paths.groupsDir, { recursive: true }), + mkdir(paths.researchDir, { recursive: true }), mkdir(paths.jobsDir, { recursive: true }), mkdir(paths.cacheDir, { recursive: true }), mkdir(paths.logsDir, { recursive: true }), mkdir(paths.tmpDir, { recursive: true }), - mkdir(dirname(paths.appDir), { recursive: true }), ]); } +// Backward-compatible alias retained for existing call sites. +export async function ensureVesaiDirectories(homeDir?: string): Promise { + await ensureCoreDirectories(homeDir); +} + function run(command: string, args: string[]): Promise { - return new Promise((resolve, reject) => { + return new Promise((resolvePromise, reject) => { const child = spawn(command, args, { stdio: "ignore" }); child.on("error", reject); child.on("close", (code) => { if (code === 0) { - resolve(); + resolvePromise(); } else { reject(new Error(`${command} ${args.join(" ")} failed`)); } @@ -38,78 +78,251 @@ function run(command: string, args: string[]): Promise { }); } -export async function ensureWorkspaceGitRepo(homeDir?: string): Promise { - const paths = getVesaiPaths(homeDir ?? resolveVesaiHome()); - const gitDir = `${paths.workspace}/.git`; +export async function ensureWorkspaceGitRepo( + projectRoot?: string +): Promise { + const paths = getVesaiPaths(projectRoot ?? resolveProjectRoot()); + const gitDir = join(paths.workspace, ".git"); try { - await readFile(`${gitDir}/HEAD`, "utf8"); + await readFile(join(gitDir, "HEAD"), "utf8"); } catch { await run("git", ["init", paths.workspace]); } await writeFile( - `${paths.workspace}/.gitignore`, + join(paths.workspace, ".gitignore"), ["# VES AI workspace cache", ".DS_Store", "*.tmp"].join("\n") + "\n", "utf8" ); } -export async function loadConfig(homeDir?: string): Promise { - const paths = getVesaiPaths(homeDir ?? resolveVesaiHome()); - const raw = await readFile(paths.configFile, "utf8"); - const parsed = JSON.parse(raw); - return VesaiConfigSchema.parse(parsed); +export async function ensureProjectGitignore( + projectRoot: string +): Promise { + const targetRoot = resolve(projectRoot); + const gitignorePath = join(targetRoot, ".gitignore"); + + let existing = ""; + try { + existing = await readFile(gitignorePath, "utf8"); + } catch (error) { + const code = (error as NodeJS.ErrnoException).code; + if (code && code !== "ENOENT") { + throw new Error( + `Failed to read ${gitignorePath}. Ensure the file is writable and not locked.` + ); + } + } + + const lines = existing.split(/\r?\n/).map((line) => line.trim()); + if (lines.includes(".vesai/")) { + return; + } + + const next = existing + ? `${existing.replace(/\s*$/, "")}\n.vesai/\n` + : ".vesai/\n"; + + try { + await writeFile(gitignorePath, next, "utf8"); + } catch { + throw new Error( + `Failed to update ${gitignorePath}. It may be locked or read-only. Add '.vesai/' manually and fix file permissions.` + ); + } } -export async function tryLoadConfig( +function combineConfig(core: CoreConfig, project: ProjectConfig): VesaiConfig { + return { + projectId: project.projectId, + posthog: project.posthog, + gcloud: core.gcloud, + vertex: core.vertex, + runtime: { + maxRenderMemoryMb: core.runtime.maxRenderMemoryMb, + lookbackDays: project.daemon.lookbackDays, + }, + daemon: project.daemon, + product: project.product, + createdAt: project.createdAt, + updatedAt: project.updatedAt, + core, + project, + }; +} + +export async function loadCoreConfig(homeDir?: string): Promise { + const paths = getVesaiCorePaths(homeDir ?? resolveVesaiHome()); + const raw = await readFile(paths.coreConfigFile, "utf8"); + const parsed = JSON.parse(raw) as Record; + const runtime = + parsed.runtime && typeof parsed.runtime === "object" + ? (parsed.runtime as Record) + : null; + + if ( + runtime && + runtime.maxRenderMemoryMb === undefined && + Number.isFinite(Number(runtime.maxConcurrentRenders)) + ) { + runtime.maxRenderMemoryMb = legacyConcurrencyToRenderMemoryMb( + Number(runtime.maxConcurrentRenders) + ); + } + + return CoreConfigSchema.parse(parsed); +} + +export async function tryLoadCoreConfig( homeDir?: string -): Promise { +): Promise { try { - return await loadConfig(homeDir); + return await loadCoreConfig(homeDir); } catch { return null; } } -export async function saveConfig( - config: Omit | VesaiConfig, +export async function saveCoreConfig( + config: Omit | CoreConfig, homeDir?: string -): Promise { - const paths = getVesaiPaths(homeDir ?? resolveVesaiHome()); - await ensureVesaiDirectories(homeDir); +): Promise { + const paths = getVesaiCorePaths(homeDir ?? resolveVesaiHome()); + await ensureCoreDirectories(homeDir); const now = new Date().toISOString(); - const existing = await tryLoadConfig(homeDir); + const existing = await tryLoadCoreConfig(homeDir); - const fullConfig = VesaiConfigSchema.parse({ + const fullConfig = CoreConfigSchema.parse({ ...config, - createdAt: existing?.createdAt ?? (config as VesaiConfig).createdAt ?? now, + createdAt: existing?.createdAt ?? (config as CoreConfig).createdAt ?? now, updatedAt: now, }); await writeFile( - paths.configFile, + paths.coreConfigFile, JSON.stringify(fullConfig, null, 2), "utf8" ); return fullConfig; } -export async function updateConfig(params: { - updater: (config: VesaiConfig) => VesaiConfig; +export async function updateCoreConfig(params: { + updater: (config: CoreConfig) => CoreConfig; homeDir?: string; -}): Promise { - const current = await requireConfig(params.homeDir); - const next = params.updater(current); - return saveConfig(next, params.homeDir); +}): Promise { + const current = await requireCoreConfig(params.homeDir); + return saveCoreConfig(params.updater(current), params.homeDir); } -export async function requireConfig(homeDir?: string): Promise { - const config = await tryLoadConfig(homeDir); +export async function requireCoreConfig(homeDir?: string): Promise { + const config = await tryLoadCoreConfig(homeDir); + if (!config) { + const paths = getVesaiCorePaths(homeDir ?? resolveVesaiHome()); + throw new Error( + `Missing core config at ${paths.coreConfigFile}. Run \`vesai quickstart\` first.` + ); + } + return config; +} + +export async function loadProjectConfig( + projectRoot?: string +): Promise { + const paths = getVesaiPaths(projectRoot ?? resolveProjectRoot()); + const raw = await readFile(paths.configFile, "utf8"); + return ProjectConfigSchema.parse(JSON.parse(raw)); +} + +export async function tryLoadProjectConfig( + projectRoot?: string +): Promise { + try { + return await loadProjectConfig(projectRoot); + } catch { + return null; + } +} + +export async function saveProjectConfig(params: { + config: Omit | ProjectConfig; + projectRoot?: string; +}): Promise { + const root = params.projectRoot ?? resolveProjectRoot(); + const paths = getVesaiPaths(root); + await ensureProjectDirectories(root); + + const now = new Date().toISOString(); + const existing = await tryLoadProjectConfig(root); + + const fullConfig = ProjectConfigSchema.parse({ + ...params.config, + createdAt: + existing?.createdAt ?? (params.config as ProjectConfig).createdAt ?? now, + updatedAt: now, + }); + + await writeFile( + paths.configFile, + JSON.stringify(fullConfig, null, 2), + "utf8" + ); + return fullConfig; +} + +export async function updateProjectConfig(params: { + updater: (config: ProjectConfig) => ProjectConfig; + projectRoot?: string; +}): Promise { + const root = params.projectRoot ?? resolveProjectRoot(); + const current = await requireProjectConfig(root); + return saveProjectConfig({ + config: params.updater(current), + projectRoot: root, + }); +} + +export async function requireProjectConfig( + projectRoot?: string +): Promise { + const root = projectRoot ?? resolveProjectRoot(); + const config = await tryLoadProjectConfig(root); + if (!config) { + const paths = getVesaiPaths(root); + throw new Error( + `Missing project config at ${paths.configFile}. Run \`vesai init\` in your project root first.` + ); + } + return config; +} + +export async function loadConfig(projectRoot?: string): Promise { + const root = projectRoot ?? resolveProjectRoot(); + const [core, project] = await Promise.all([ + requireCoreConfig(), + loadProjectConfig(root), + ]); + return combineConfig(core, project); +} + +export async function tryLoadConfig( + projectRoot?: string +): Promise { + try { + return await loadConfig(projectRoot); + } catch { + return null; + } +} + +export async function requireConfig( + projectRoot?: string +): Promise { + const config = await tryLoadConfig(projectRoot); if (!config) { throw new Error( - "Missing config at ~/.vesai/vesai.json. Run `vesai quickstart` first." + "Missing core and/or project config. Run `vesai quickstart` (global) and `vesai init` (project) first." ); } return config; diff --git a/config/paths.ts b/config/paths.ts index 0374955..faf1f82 100644 --- a/config/paths.ts +++ b/config/paths.ts @@ -1,19 +1,31 @@ +import { existsSync } from "node:fs"; import { homedir } from "node:os"; -import { join } from "node:path"; +import { dirname, join, resolve } from "node:path"; -export type VesaiPaths = { +export type VesaiCorePaths = { home: string; + coreConfigFile: string; + logsDir: string; + tmpDir: string; + appDir: string; + renderLocksDir: string; +}; + +export type VesaiPaths = { + projectRoot: string; + vesaiDir: string; configFile: string; workspace: string; sessionsDir: string; usersDir: string; groupsDir: string; + researchDir: string; jobsDir: string; cacheDir: string; logsDir: string; tmpDir: string; - appDir: string; daemonPidFile: string; + daemonStateFile: string; }; export function resolveVesaiHome(): string { @@ -24,20 +36,73 @@ export function resolveVesaiHome(): string { return join(homedir(), ".vesai"); } -export function getVesaiPaths(homeDir = resolveVesaiHome()): VesaiPaths { - const workspace = join(homeDir, "workspace"); +export function getVesaiCorePaths( + homeDir = resolveVesaiHome() +): VesaiCorePaths { return { home: homeDir, - configFile: join(homeDir, "vesai.json"), + coreConfigFile: join(homeDir, "core.json"), + logsDir: join(homeDir, "logs"), + tmpDir: join(homeDir, "tmp"), + appDir: join(homeDir, "app", "vesai"), + renderLocksDir: join(homeDir, "render-locks"), + }; +} + +function hasProjectConfig(projectRoot: string): boolean { + return existsSync(join(projectRoot, ".vesai", "project.json")); +} + +export function findProjectRoot(startDir = process.cwd()): string | null { + let current = resolve(startDir); + while (true) { + if (hasProjectConfig(current)) { + return current; + } + + const parent = dirname(current); + if (parent === current) { + return null; + } + current = parent; + } +} + +export function resolveProjectRoot(startDir = process.cwd()): string { + const override = process.env.VESAI_PROJECT_ROOT?.trim(); + if (override) { + return resolve(override); + } + + const found = findProjectRoot(startDir); + if (found) { + return found; + } + + throw new Error( + `No VES AI project found from ${resolve(startDir)}. Run \`vesai init\` in your project root first.` + ); +} + +export function getVesaiPaths(projectRoot = resolveProjectRoot()): VesaiPaths { + const root = resolve(projectRoot); + const vesaiDir = join(root, ".vesai"); + const workspace = join(vesaiDir, "workspace"); + + return { + projectRoot: root, + vesaiDir, + configFile: join(vesaiDir, "project.json"), workspace, sessionsDir: join(workspace, "sessions"), usersDir: join(workspace, "users"), groupsDir: join(workspace, "groups"), - jobsDir: join(homeDir, "jobs"), - cacheDir: join(homeDir, "cache"), - logsDir: join(homeDir, "logs"), - tmpDir: join(homeDir, "tmp"), - appDir: join(homeDir, "app", "vesai"), - daemonPidFile: join(homeDir, "daemon.pid"), + researchDir: join(workspace, "research"), + jobsDir: join(vesaiDir, "jobs"), + cacheDir: join(vesaiDir, "cache"), + logsDir: join(vesaiDir, "logs"), + tmpDir: join(vesaiDir, "tmp"), + daemonPidFile: join(vesaiDir, "daemon.pid"), + daemonStateFile: join(vesaiDir, "daemon-state.json"), }; } diff --git a/config/runtime.ts b/config/runtime.ts new file mode 100644 index 0000000..66fdb19 --- /dev/null +++ b/config/runtime.ts @@ -0,0 +1,104 @@ +import { freemem } from "node:os"; + +export const BYTES_PER_MIB = 1024 * 1024; +export const BYTES_PER_GIB = 1024 * 1024 * 1024; + +export const RENDER_MEMORY_PER_SERVICE_MB = 512; +export const MIN_RENDER_MEMORY_MB = RENDER_MEMORY_PER_SERVICE_MB; + +const DEFAULT_RENDER_MEMORY_FRACTION = 0.5; +const DYNAMIC_FREE_MEMORY_UTILIZATION = 0.9; + +export function formatGiB(bytes: number): string { + if (!Number.isFinite(bytes) || bytes <= 0) { + return "0.0"; + } + return (bytes / BYTES_PER_GIB).toFixed(1); +} + +export function normalizePositiveIntegerValue( + value: string | number | undefined, + fallback: number, + flag: string +): number { + if (value === undefined || value === null || String(value).trim() === "") { + return fallback; + } + + const parsed = Number(value); + if (!(Number.isFinite(parsed) && Number.isInteger(parsed) && parsed >= 1)) { + throw new Error( + `Invalid ${flag} value "${value}". Must be a positive integer.` + ); + } + return parsed; +} + +export function normalizeRenderMemoryMbValue( + value: string | number | undefined, + fallbackMb: number, + flag: string +): number { + const parsed = normalizePositiveIntegerValue(value, fallbackMb, flag); + if (parsed < MIN_RENDER_MEMORY_MB) { + throw new Error( + `Invalid ${flag} value "${value}". Must be at least ${MIN_RENDER_MEMORY_MB} MiB.` + ); + } + return parsed; +} + +function normalizeRenderMemoryMb(value: number): number { + if (!Number.isFinite(value) || value <= 0) { + return MIN_RENDER_MEMORY_MB; + } + return Math.max(MIN_RENDER_MEMORY_MB, Math.floor(value)); +} + +export function computeDefaultRenderMemoryMb( + availableRamBytes: number = freemem() +): number { + if (!Number.isFinite(availableRamBytes) || availableRamBytes <= 0) { + return MIN_RENDER_MEMORY_MB; + } + const budgetBytes = availableRamBytes * DEFAULT_RENDER_MEMORY_FRACTION; + const budgetMb = Math.floor(budgetBytes / BYTES_PER_MIB); + return normalizeRenderMemoryMb(budgetMb); +} + +export function estimateRenderServiceCapacity( + maxRenderMemoryMb: number +): number { + const normalizedMb = normalizeRenderMemoryMb(maxRenderMemoryMb); + return Math.max(1, Math.floor(normalizedMb / RENDER_MEMORY_PER_SERVICE_MB)); +} + +export function computeDynamicRenderServiceCapacity(params: { + maxRenderMemoryMb: number; + availableRamBytes?: number; +}): number { + const configuredMb = normalizeRenderMemoryMb(params.maxRenderMemoryMb); + const availableRamBytes = + params.availableRamBytes === undefined + ? freemem() + : params.availableRamBytes; + + if (!Number.isFinite(availableRamBytes) || availableRamBytes <= 0) { + return estimateRenderServiceCapacity(configuredMb); + } + + const dynamicBudgetMb = Math.floor( + (availableRamBytes * DYNAMIC_FREE_MEMORY_UTILIZATION) / BYTES_PER_MIB + ); + const effectiveBudgetMb = Math.max( + MIN_RENDER_MEMORY_MB, + Math.min(configuredMb, dynamicBudgetMb) + ); + + return estimateRenderServiceCapacity(effectiveBudgetMb); +} + +export function legacyConcurrencyToRenderMemoryMb(concurrency: number): number { + const normalized = Math.max(1, Math.floor(concurrency)); + return normalized * RENDER_MEMORY_PER_SERVICE_MB; +} diff --git a/config/schema.ts b/config/schema.ts index 9e15e89..bdd972e 100644 --- a/config/schema.ts +++ b/config/schema.ts @@ -1,16 +1,10 @@ import { z } from "zod"; +import { MIN_RENDER_MEMORY_MB } from "./runtime"; export const DEFAULT_VERTEX_MODEL = "gemini-3-pro-preview"; -export const VesaiConfigSchema = z.object({ +export const CoreConfigSchema = z.object({ version: z.number().int().positive().default(1), - posthog: z.object({ - host: z.string().url().default("https://us.posthog.com"), - apiKey: z.string().min(1), - projectId: z.string().min(1), - groupKey: z.string().min(1), - domainFilter: z.string().min(1), - }), gcloud: z.object({ projectId: z.string().min(1), region: z.string().min(1).default("us-central1"), @@ -21,7 +15,28 @@ export const VesaiConfigSchema = z.object({ location: z.string().min(1).default("us-central1"), }), runtime: z.object({ - maxConcurrentRenders: z.number().int().min(1).default(2), + maxRenderMemoryMb: z + .number() + .int() + .min(MIN_RENDER_MEMORY_MB) + .default(MIN_RENDER_MEMORY_MB * 2), + }), + createdAt: z.string().datetime(), + updatedAt: z.string().datetime(), +}); + +export const ProjectConfigSchema = z.object({ + version: z.number().int().positive().default(1), + projectId: z.string().uuid(), + posthog: z.object({ + host: z.string().url().default("https://us.posthog.com"), + apiKey: z.string().min(1), + projectId: z.string().min(1), + groupKey: z.string().min(1), + domainFilter: z.string().min(1), + }), + daemon: z.object({ + lookbackDays: z.number().int().min(1).default(180), }), product: z.object({ description: z.string().min(1), @@ -30,6 +45,25 @@ export const VesaiConfigSchema = z.object({ updatedAt: z.string().datetime(), }); -export type VesaiConfig = z.infer; +export type CoreConfig = z.infer; +export type ProjectConfig = z.infer; + +export type VesaiConfig = { + projectId: string; + posthog: ProjectConfig["posthog"]; + gcloud: CoreConfig["gcloud"]; + vertex: CoreConfig["vertex"]; + runtime: { + maxRenderMemoryMb: number; + lookbackDays: number; + }; + daemon: ProjectConfig["daemon"]; + product: ProjectConfig["product"]; + createdAt: string; + updatedAt: string; + core: CoreConfig; + project: ProjectConfig; +}; -export const PartialVesaiConfigSchema = VesaiConfigSchema.partial(); +export const PartialCoreConfigSchema = CoreConfigSchema.partial(); +export const PartialProjectConfigSchema = ProjectConfigSchema.partial(); diff --git a/connectors/gemini.ts b/connectors/gemini.ts index 654971f..59ff87c 100644 --- a/connectors/gemini.ts +++ b/connectors/gemini.ts @@ -32,6 +32,13 @@ export type AggregateAnalysis = { score: number; }; +export type ResearchAnswer = { + answer: string; + findings: string[]; + confidence: "low" | "medium" | "high"; + supportingSessionIds: string[]; +}; + function getSessionSchema() { return { type: Type.OBJECT, @@ -110,6 +117,28 @@ function getAggregateSchema() { }; } +function getResearchSchema() { + return { + type: Type.OBJECT, + properties: { + answer: { type: Type.STRING }, + findings: { + type: Type.ARRAY, + items: { type: Type.STRING }, + }, + confidence: { + type: Type.STRING, + enum: ["low", "medium", "high"], + }, + supportingSessionIds: { + type: Type.ARRAY, + items: { type: Type.STRING }, + }, + }, + required: ["answer", "findings", "confidence", "supportingSessionIds"], + }; +} + function parseJsonText(text: string | undefined): T { if (!text) { throw new Error("Gemini returned empty response text"); @@ -250,3 +279,43 @@ export async function analyzeGroupAggregate(params: { return parseJsonText(response.text); } + +export async function answerResearchQuestion(params: { + ai: GoogleGenAI; + model: string; + productDescription: string; + question: string; + sessions: Array<{ + sessionId: string; + startTime: string | null; + score: number | null; + markdownPath: string; + summary: string; + }>; +}): Promise { + const prompt = { + productDescription: params.productDescription, + question: params.question, + availableSessionIds: params.sessions.map((session) => session.sessionId), + sessions: params.sessions, + instructions: + "Answer the question using only provided sessions. Cite only session IDs from availableSessionIds. If evidence is weak, say so.", + }; + + const response = await params.ai.models.generateContent({ + model: params.model, + contents: [ + createUserContent( + "You are a replay research analyst. Use only supplied evidence. Return JSON only." + ), + createUserContent(JSON.stringify(prompt)), + ], + config: { + thinkingConfig: { thinkingLevel: ThinkingLevel.HIGH }, + responseMimeType: "application/json", + responseSchema: getResearchSchema(), + }, + }); + + return parseJsonText(response.text); +} diff --git a/connectors/posthog.ts b/connectors/posthog.ts index 245f41e..2bf6772 100644 --- a/connectors/posthog.ts +++ b/connectors/posthog.ts @@ -26,73 +26,11 @@ export type PostHogRecording = { [key: string]: unknown; }; -export type PostHogQueryFilters = { - text?: string; - email?: string; - groupId?: string; - groupKey?: string; - domain?: string; - urlContains?: string; - sessionId?: string; - sessionContains?: string; - distinctId?: string; - startsAfter?: string; - startsBefore?: string; - minActiveSeconds?: number; - maxActiveSeconds?: number; - includeOngoing?: boolean; - requirePerson?: boolean; - properties?: Record; - limit?: number; -}; - type PostHogRecordingsResponse = { results: PostHogRecording[]; has_next: boolean; }; -export type PostHogEventDefinition = { - id?: number; - name?: string; - description?: string | null; - verified?: boolean; - hidden?: boolean; - tags?: string[]; - [key: string]: unknown; -}; - -export type PostHogPropertyDefinition = { - id?: number; - name?: string; - property_type?: string; - type?: string; - is_numerical?: boolean; - hidden?: boolean; - [key: string]: unknown; -}; - -export type PostHogLogsQueryParams = { - dateFrom: string; - dateTo: string; - severityLevels?: Array< - "trace" | "debug" | "info" | "warn" | "error" | "fatal" - >; - serviceNames?: string[]; - searchTerm?: string; - orderBy?: "latest" | "earliest"; - limit?: number; - after?: string; -}; - -export type PostHogErrorListParams = { - orderBy?: "occurrences" | "first_seen" | "last_seen" | "users" | "sessions"; - dateFrom?: string; - dateTo?: string; - orderDirection?: "ASC" | "DESC"; - filterTestAccounts?: boolean; - status?: "active" | "resolved" | "all" | "suppressed"; -}; - async function posthogRequest(params: { host?: string; apiKey: string; @@ -280,163 +218,6 @@ export async function findRecordingsByGroupId(params: { }); } -export async function findRecordingsByQuery(params: { - host?: string; - apiKey: string; - projectId: string; - query?: string; - domainFilter?: string; - filters?: PostHogQueryFilters; -}): Promise { - // PostHog's recording list endpoint does not support this full filter surface. - // We fetch pages and apply one deterministic local filter pass so CLI behavior - // stays stable for both humans and agents. - const effectiveFilters: PostHogQueryFilters = { - ...(params.filters || {}), - text: params.filters?.text || params.query, - domain: params.filters?.domain || params.domainFilter, - }; - const normalizedTextNeedle = effectiveFilters.text?.trim().toLowerCase(); - const allRecordings = await listAllRecordings(params); - - const filtered = allRecordings.filter((recording) => { - if (!effectiveFilters.includeOngoing && recording.ongoing) { - return false; - } - - if (effectiveFilters.requirePerson && !recording.person) { - return false; - } - - if (effectiveFilters.email) { - const email = getRecordingUserEmail(recording); - if (email !== effectiveFilters.email.trim().toLowerCase()) { - return false; - } - } - - if (effectiveFilters.groupId) { - const groupKey = effectiveFilters.groupKey || "group_id"; - const props = recording.person?.properties || {}; - const value = props[groupKey] ?? props[`$${groupKey}`]; - if (String(value ?? "") !== effectiveFilters.groupId) { - return false; - } - } - - const url = String(recording.start_url ?? "").toLowerCase(); - if ( - effectiveFilters.domain && - url && - !url.includes(effectiveFilters.domain.toLowerCase()) - ) { - return false; - } - - if ( - effectiveFilters.urlContains && - !url.includes(effectiveFilters.urlContains.toLowerCase()) - ) { - return false; - } - - if ( - effectiveFilters.sessionId && - recording.id !== effectiveFilters.sessionId - ) { - return false; - } - - const normalizedSessionId = String(recording.id).toLowerCase(); - if ( - effectiveFilters.sessionContains && - !normalizedSessionId.includes( - effectiveFilters.sessionContains.toLowerCase() - ) - ) { - return false; - } - - if (effectiveFilters.distinctId) { - const distinctId = String(recording.distinct_id ?? "").toLowerCase(); - if (distinctId !== effectiveFilters.distinctId.toLowerCase()) { - return false; - } - } - - const activeSeconds = Number(recording.active_seconds ?? 0); - if ( - effectiveFilters.minActiveSeconds !== undefined && - activeSeconds < effectiveFilters.minActiveSeconds - ) { - return false; - } - - if ( - effectiveFilters.maxActiveSeconds !== undefined && - activeSeconds > effectiveFilters.maxActiveSeconds - ) { - return false; - } - - const startTime = recording.start_time - ? Date.parse(recording.start_time) - : Number.NaN; - if ( - effectiveFilters.startsAfter && - Number.isFinite(startTime) && - startTime < Date.parse(effectiveFilters.startsAfter) - ) { - return false; - } - - if ( - effectiveFilters.startsBefore && - Number.isFinite(startTime) && - startTime > Date.parse(effectiveFilters.startsBefore) - ) { - return false; - } - - if (effectiveFilters.properties) { - const props = recording.person?.properties || {}; - for (const [key, expected] of Object.entries( - effectiveFilters.properties - )) { - const raw = props[key] ?? props[`$${key}`]; - if (String(raw ?? "").toLowerCase() !== expected.toLowerCase()) { - return false; - } - } - } - - const personPropertiesText = JSON.stringify( - recording.person?.properties ?? {} - ).toLowerCase(); - if (!normalizedTextNeedle) { - return true; - } - const distinctId = String(recording.distinct_id ?? "").toLowerCase(); - const email = getRecordingUserEmail(recording) ?? ""; - return ( - url.includes(normalizedTextNeedle) || - personPropertiesText.includes(normalizedTextNeedle) || - normalizedSessionId.includes(normalizedTextNeedle) || - distinctId.includes(normalizedTextNeedle) || - email.includes(normalizedTextNeedle) - ); - }); - - const sortedNewestFirst = filtered.sort((a, b) => - String(b.start_time || "").localeCompare(String(a.start_time || "")) - ); - - if (!effectiveFilters.limit) { - return sortedNewestFirst; - } - return sortedNewestFirst.slice(0, effectiveFilters.limit); -} - export async function findRecordingById(params: { host?: string; apiKey: string; @@ -446,352 +227,3 @@ export async function findRecordingById(params: { const all = await listAllRecordings(params); return all.find((recording) => recording.id === params.sessionId) ?? null; } - -function normalizeDateToIso(value: string | undefined): string | undefined { - if (!value) { - return undefined; - } - const parsed = Date.parse(value); - if (Number.isNaN(parsed)) { - return undefined; - } - return new Date(parsed).toISOString(); -} - -function parseStringContentMaybeJson(content: string): unknown { - try { - return JSON.parse(content) as unknown; - } catch { - return content; - } -} - -type PostHogMcpToolResult = { - success: boolean; - content: string; -}; - -export async function invokePostHogMcpTool(params: { - host?: string; - apiKey: string; - projectId: string; - toolName: string; - args: Record; -}): Promise { - const result = await posthogRequest({ - host: params.host, - apiKey: params.apiKey, - method: "POST", - path: `/api/environments/${encodeURIComponent(params.projectId)}/mcp_tools/${encodeURIComponent(params.toolName)}/`, - body: { - args: params.args, - }, - }); - - if (!result.success) { - throw new Error( - `PostHog MCP tool '${params.toolName}' failed: ${result.content}` - ); - } - - return parseStringContentMaybeJson(result.content); -} - -export async function readDataSchema(params: { - host?: string; - apiKey: string; - projectId: string; - query: Record; -}): Promise { - return invokePostHogMcpTool({ - host: params.host, - apiKey: params.apiKey, - projectId: params.projectId, - toolName: "read_taxonomy", - args: { - query: params.query, - }, - }); -} - -export async function readDataWarehouseSchema(params: { - host?: string; - apiKey: string; - projectId: string; -}): Promise { - // PostHog MCP currently validates for a `query` object even for schema reads. - return invokePostHogMcpTool({ - host: params.host, - apiKey: params.apiKey, - projectId: params.projectId, - toolName: "read_data_warehouse_schema", - args: { - query: {}, - }, - }); -} - -export async function executeSqlQuery(params: { - host?: string; - apiKey: string; - projectId: string; - query: string; -}): Promise { - return invokePostHogMcpTool({ - host: params.host, - apiKey: params.apiKey, - projectId: params.projectId, - toolName: "execute_sql", - args: { - query: params.query, - }, - }); -} - -export async function listEventDefinitions(params: { - host?: string; - apiKey: string; - projectId: string; - search?: string; - limit?: number; - offset?: number; -}): Promise { - const response = await posthogRequest<{ results: PostHogEventDefinition[] }>({ - host: params.host, - apiKey: params.apiKey, - path: `/api/projects/${encodeURIComponent(params.projectId)}/event_definitions/`, - query: { - search: params.search, - limit: params.limit ?? 50, - offset: params.offset ?? 0, - }, - }); - return response.results || []; -} - -export async function listPropertyDefinitions(params: { - host?: string; - apiKey: string; - projectId: string; - type: "event" | "person"; - eventName?: string; - includePredefinedProperties?: boolean; - limit?: number; - offset?: number; -}): Promise { - const eventNames = - params.type === "event" && params.eventName - ? JSON.stringify([params.eventName]) - : undefined; - const response = await posthogRequest<{ - results: PostHogPropertyDefinition[]; - }>({ - host: params.host, - apiKey: params.apiKey, - path: `/api/projects/${encodeURIComponent(params.projectId)}/property_definitions/`, - query: { - type: params.type, - event_names: eventNames, - filter_by_event_names: params.type === "event", - exclude_core_properties: !(params.includePredefinedProperties ?? false), - exclude_hidden: true, - is_feature_flag: false, - limit: params.limit ?? 50, - offset: params.offset ?? 0, - }, - }); - return response.results || []; -} - -export async function runInsightQuery(params: { - host?: string; - apiKey: string; - projectId: string; - query: Record; -}): Promise<{ - results?: unknown; - columns?: unknown; - [key: string]: unknown; -}> { - return posthogRequest({ - host: params.host, - apiKey: params.apiKey, - method: "POST", - path: `/api/environments/${encodeURIComponent(params.projectId)}/query/`, - body: { - query: params.query, - }, - }); -} - -export async function generateHogQLFromQuestion(params: { - host?: string; - apiKey: string; - projectId: string; - question: string; -}): Promise { - const response = await posthogRequest({ - host: params.host, - apiKey: params.apiKey, - method: "POST", - path: `/api/environments/${encodeURIComponent(params.projectId)}/max_tools/create_and_query_insight/`, - body: { - query: params.question, - insight_type: "sql", - }, - }); - - return response.filter((item) => { - if (!(item && typeof item === "object")) { - return true; - } - const candidate = item as Record; - if (candidate.type !== "message") { - return true; - } - const data = candidate.data as Record | undefined; - return data?.type !== "ack"; - }); -} - -export async function listErrors(params: { - host?: string; - apiKey: string; - projectId: string; - input?: PostHogErrorListParams; -}): Promise { - const now = new Date(); - const sevenDaysAgo = new Date(Date.now() - 7 * 24 * 60 * 60 * 1000); - const input = params.input || {}; - - const response = await runInsightQuery({ - host: params.host, - apiKey: params.apiKey, - projectId: params.projectId, - query: { - kind: "ErrorTrackingQuery", - orderBy: input.orderBy || "occurrences", - dateRange: { - date_from: - normalizeDateToIso(input.dateFrom) || sevenDaysAgo.toISOString(), - date_to: normalizeDateToIso(input.dateTo) || now.toISOString(), - }, - volumeResolution: 1, - orderDirection: input.orderDirection || "DESC", - filterTestAccounts: input.filterTestAccounts ?? true, - status: input.status || "active", - }, - }); - - const results = response.results; - return Array.isArray(results) ? results : []; -} - -export async function getErrorDetails(params: { - host?: string; - apiKey: string; - projectId: string; - issueId: string; - dateFrom?: string; - dateTo?: string; -}): Promise { - const now = new Date(); - const sevenDaysAgo = new Date(Date.now() - 7 * 24 * 60 * 60 * 1000); - const response = await runInsightQuery({ - host: params.host, - apiKey: params.apiKey, - projectId: params.projectId, - query: { - kind: "ErrorTrackingQuery", - orderBy: "occurrences", - dateRange: { - date_from: - normalizeDateToIso(params.dateFrom) || sevenDaysAgo.toISOString(), - date_to: normalizeDateToIso(params.dateTo) || now.toISOString(), - }, - volumeResolution: 0, - issueId: params.issueId, - }, - }); - - const results = response.results; - return Array.isArray(results) ? results : []; -} - -export async function queryLogs(params: { - host?: string; - apiKey: string; - projectId: string; - input: PostHogLogsQueryParams; -}): Promise<{ - results: unknown[]; - hasMore: boolean; - nextCursor: string | null; -}> { - return posthogRequest({ - host: params.host, - apiKey: params.apiKey, - method: "POST", - path: `/api/projects/${encodeURIComponent(params.projectId)}/logs/query/`, - body: { - query: { - dateRange: { - date_from: params.input.dateFrom, - date_to: params.input.dateTo, - }, - severityLevels: params.input.severityLevels ?? [], - serviceNames: params.input.serviceNames ?? [], - searchTerm: params.input.searchTerm ?? null, - orderBy: params.input.orderBy ?? "latest", - limit: params.input.limit ?? 100, - after: params.input.after ?? null, - filterGroup: { type: "AND", values: [] }, - }, - }, - }); -} - -export async function listLogAttributes(params: { - host?: string; - apiKey: string; - projectId: string; - search?: string; - attributeType?: "log" | "resource"; - limit?: number; - offset?: number; -}): Promise<{ - results: Array<{ name: string; propertyFilterType: string }>; - count: number; -}> { - return posthogRequest({ - host: params.host, - apiKey: params.apiKey, - path: `/api/projects/${encodeURIComponent(params.projectId)}/logs/attributes/`, - query: { - search: params.search, - attribute_type: params.attributeType ?? "log", - limit: params.limit ?? 100, - offset: params.offset ?? 0, - }, - }); -} - -export async function listLogAttributeValues(params: { - host?: string; - apiKey: string; - projectId: string; - key: string; - attributeType?: "log" | "resource"; - search?: string; -}): Promise> { - return posthogRequest({ - host: params.host, - apiKey: params.apiKey, - path: `/api/projects/${encodeURIComponent(params.projectId)}/logs/values/`, - query: { - key: params.key, - attribute_type: params.attributeType ?? "log", - value: params.search, - }, - }); -} diff --git a/daemon/index.ts b/daemon/index.ts index f9b081f..5955e49 100644 --- a/daemon/index.ts +++ b/daemon/index.ts @@ -1,8 +1,9 @@ -import { ensureVesaiDirectories } from "../config"; +import { ensureCoreDirectories, ensureProjectDirectories } from "../config"; import { startDaemon } from "./runner"; async function main() { - await ensureVesaiDirectories(); + await ensureCoreDirectories(); + await ensureProjectDirectories(); console.log("Starting VES AI daemon..."); await startDaemon(); } diff --git a/daemon/jobs/store.ts b/daemon/jobs/store.ts index 4c8de50..816b60c 100644 --- a/daemon/jobs/store.ts +++ b/daemon/jobs/store.ts @@ -1,11 +1,11 @@ import { randomUUID } from "node:crypto"; import { readdir, readFile, writeFile } from "node:fs/promises"; import { join } from "node:path"; -import { getVesaiPaths, resolveVesaiHome } from "../../config"; +import { getVesaiPaths } from "../../config"; import type { JobRecord, JobStatus, JobType } from "./types"; function jobPath(id: string, homeDir?: string): string { - const paths = getVesaiPaths(homeDir ?? resolveVesaiHome()); + const paths = getVesaiPaths(homeDir); return join(paths.jobsDir, `${id}.json`); } @@ -111,7 +111,7 @@ export async function updateJobStatus(params: { } export async function listJobs(homeDir?: string): Promise { - const paths = getVesaiPaths(homeDir ?? resolveVesaiHome()); + const paths = getVesaiPaths(homeDir); const files = await readdir(paths.jobsDir).catch(() => []); const jobs = await Promise.all( diff --git a/daemon/jobs/types.ts b/daemon/jobs/types.ts index a966030..a0c80d5 100644 --- a/daemon/jobs/types.ts +++ b/daemon/jobs/types.ts @@ -1,35 +1,16 @@ -import type { PostHogQueryFilters } from "../../connectors/posthog"; +import type { PostHogRecording } from "../../connectors/posthog"; export type JobStatus = "queued" | "running" | "complete" | "failed"; export type AnalyzeSessionPayload = { sessionId: string; + recording?: PostHogRecording; }; -export type AnalyzeUserPayload = { - email: string; -}; - -export type AnalyzeGroupPayload = { - groupId: string; -}; - -export type AnalyzeQueryPayload = { - query?: string; - filters?: PostHogQueryFilters; -}; - -export type JobType = - | "analyze_session" - | "analyze_user" - | "analyze_group" - | "analyze_query"; +export type JobType = "analyze_session"; export type JobPayloadMap = { analyze_session: AnalyzeSessionPayload; - analyze_user: AnalyzeUserPayload; - analyze_group: AnalyzeGroupPayload; - analyze_query: AnalyzeQueryPayload; }; export type JobRecord = { diff --git a/daemon/runner.ts b/daemon/runner.ts index 60e2e3e..f6122c7 100644 --- a/daemon/runner.ts +++ b/daemon/runner.ts @@ -1,27 +1,52 @@ import { readFile, rm, writeFile } from "node:fs/promises"; import { analyzeGroupById, - analyzeQuery, analyzeUserByEmail, renderAndAnalyzeSession, } from "../cli/analysis"; import { getVesaiPaths, requireConfig } from "../config"; -import { findRecordingById } from "../connectors"; +import { computeDynamicRenderServiceCapacity } from "../config/runtime"; +import { + findRecordingById, + getRecordingUserEmail, + listAllRecordings, + type PostHogRecording, +} from "../connectors"; import { appendJobLog, + createJob, + listJobs, listQueuedJobs, readJob, updateJobStatus, } from "./jobs/store"; import type { JobPayloadMap, JobRecord } from "./jobs/types"; +import { loadDaemonState, saveDaemonState, updateDaemonState } from "./state"; export type DaemonOptions = { homeDir?: string; intervalMs?: number; }; +const DAY_MS = 24 * 60 * 60 * 1000; let shouldStop = false; +function sleep(ms: number): Promise { + return new Promise((resolve) => { + setTimeout(resolve, ms); + }); +} + +function extractGroupIdFromRecording( + recording: PostHogRecording, + groupKey: string +): string | null { + const props = recording.person?.properties || {}; + const candidate = props[groupKey] ?? props[`$${groupKey}`]; + const value = String(candidate ?? "").trim(); + return value || null; +} + async function writePidFile(homeDir?: string): Promise { const paths = getVesaiPaths(homeDir); await writeFile(paths.daemonPidFile, String(process.pid), "utf8"); @@ -32,10 +57,119 @@ async function removePidFile(homeDir?: string): Promise { await rm(paths.daemonPidFile, { force: true }); } -async function processJob( +async function queueSessionsFromHeartbeat(params: { + homeDir?: string; + nowIso: string; +}): Promise<{ queued: number; fromIso: string; bootstrapRun: boolean }> { + const config = await requireConfig(params.homeDir); + const state = await loadDaemonState(params.homeDir); + + const bootstrapRun = !state.lastPulledAt; + const fromIso = + state.lastPulledAt || + new Date( + Date.now() - Math.max(1, config.runtime.lookbackDays) * DAY_MS + ).toISOString(); + + const fromMs = Date.parse(fromIso); + const nowMs = Date.parse(params.nowIso); + const hasCursor = Boolean(state.lastPulledAt); + + const recordings = await listAllRecordings({ + host: config.posthog.host, + apiKey: config.posthog.apiKey, + projectId: config.posthog.projectId, + dateFrom: fromIso, + }); + + const existingJobs = await listJobs(params.homeDir); + const alreadyTracked = new Set( + existingJobs + .filter( + (job) => job.type === "analyze_session" && job.status !== "failed" + ) + .map((job) => (job.payload as JobPayloadMap["analyze_session"]).sessionId) + ); + + const candidates = recordings + .filter((recording) => { + if (recording.ongoing) { + return false; + } + + if ( + config.posthog.domainFilter && + typeof recording.start_url === "string" && + !recording.start_url.includes(config.posthog.domainFilter) + ) { + return false; + } + + const startedMs = recording.start_time + ? Date.parse(recording.start_time) + : Number.NaN; + if (!Number.isFinite(startedMs)) { + return true; + } + + if (startedMs > nowMs) { + return false; + } + + if (hasCursor) { + return startedMs > fromMs; + } + + return startedMs >= fromMs; + }) + .sort((a, b) => + String(a.start_time || "").localeCompare(String(b.start_time || "")) + ); + + let queued = 0; + for (const recording of candidates) { + if (alreadyTracked.has(recording.id)) { + continue; + } + + await createJob({ + type: "analyze_session", + payload: { + sessionId: recording.id, + recording, + }, + homeDir: params.homeDir, + }); + alreadyTracked.add(recording.id); + queued += 1; + } + + await updateDaemonState({ + homeDir: params.homeDir, + updater: (current) => ({ + ...current, + backfillStartedAt: + current.backfillStartedAt || (bootstrapRun ? params.nowIso : undefined), + lastPulledAt: params.nowIso, + }), + }); + + return { + queued, + fromIso, + bootstrapRun, + }; +} + +type ProcessSessionOutcome = { + userEmail?: string; + groupId?: string; +} | null; + +async function processSessionJob( job: JobRecord, configHomeDir?: string -): Promise { +): Promise { const config = await requireConfig(configHomeDir); await updateJobStatus({ @@ -47,161 +181,174 @@ async function processJob( try { const context = { config, homeDir: configHomeDir }; + const payload = job.payload as JobPayloadMap["analyze_session"]; - if (job.type === "analyze_session") { - const payload = job.payload as JobPayloadMap["analyze_session"]; - const recording = await findRecordingById({ + const recording = + payload.recording || + (await findRecordingById({ host: config.posthog.host, apiKey: config.posthog.apiKey, projectId: config.posthog.projectId, sessionId: payload.sessionId, - }); + })); - if (!recording) { - throw new Error( - `Session ${payload.sessionId} was not found in PostHog` - ); - } + if (!recording) { + throw new Error(`Session ${payload.sessionId} was not found in PostHog`); + } - await appendJobLog( - job.id, - `Rendering and analyzing session ${recording.id}`, - configHomeDir - ); - const result = await renderAndAnalyzeSession(recording, context); - - await updateJobStatus({ - id: job.id, - status: "complete", - homeDir: configHomeDir, - result: { - sessionId: recording.id, - score: result.analysis.score, - markdownPath: result.markdownPath, - videoUri: result.render.videoUri, - }, - }); + await appendJobLog( + job.id, + `Rendering and analyzing session ${recording.id}`, + configHomeDir + ); + const result = await renderAndAnalyzeSession(recording, context); - await appendJobLog( - job.id, - `Completed session ${recording.id}`, - configHomeDir - ); - return; - } + await updateJobStatus({ + id: job.id, + status: "complete", + homeDir: configHomeDir, + result: { + sessionId: recording.id, + score: result.analysis.score, + markdownPath: result.markdownPath, + videoUri: result.render.videoUri, + }, + }); - if (job.type === "analyze_user") { - const payload = job.payload as JobPayloadMap["analyze_user"]; - await appendJobLog( - job.id, - `Fetching sessions and analyzing user ${payload.email}`, - configHomeDir - ); - const result = await analyzeUserByEmail({ - email: payload.email, - context, - }); + await appendJobLog( + job.id, + `Completed session ${recording.id}`, + configHomeDir + ); + + return { + userEmail: getRecordingUserEmail(recording) || undefined, + groupId: + extractGroupIdFromRecording(recording, config.posthog.groupKey) || + undefined, + }; + } catch (error) { + const message = error instanceof Error ? error.message : String(error); + await updateJobStatus({ + id: job.id, + status: "failed", + homeDir: configHomeDir, + error: message, + }); + await appendJobLog(job.id, `Failed: ${message}`, configHomeDir); + return null; + } +} - await updateJobStatus({ - id: job.id, - status: "complete", - homeDir: configHomeDir, - result: { - email: result.email, - sessionCount: result.sessionCount, - score: result.userScore, - markdownPath: result.markdownPath, - }, - }); - await appendJobLog( - job.id, - `Completed user ${payload.email}`, - configHomeDir - ); - return; - } +async function queueDirtyRollups(params: { + homeDir?: string; + userEmail?: string; + groupId?: string; +}): Promise { + if (!(params.userEmail || params.groupId)) { + return; + } - if (job.type === "analyze_group") { - const payload = job.payload as JobPayloadMap["analyze_group"]; - await appendJobLog( - job.id, - `Fetching sessions and analyzing group ${payload.groupId}`, - configHomeDir - ); - const result = await analyzeGroupById({ - groupId: payload.groupId, - context, - }); + await updateDaemonState({ + homeDir: params.homeDir, + updater: (state) => ({ + ...state, + pendingUserEmails: params.userEmail + ? [...state.pendingUserEmails, params.userEmail] + : state.pendingUserEmails, + pendingGroupIds: params.groupId + ? [...state.pendingGroupIds, params.groupId] + : state.pendingGroupIds, + }), + }); +} + +async function runPendingRollups(homeDir?: string): Promise { + const state = await loadDaemonState(homeDir); + if (!(state.pendingUserEmails.length || state.pendingGroupIds.length)) { + return; + } + + const config = await requireConfig(homeDir); + const context = { config, homeDir }; + const sessionConcurrency = computeDynamicRenderServiceCapacity({ + maxRenderMemoryMb: config.runtime.maxRenderMemoryMb, + }); - await updateJobStatus({ - id: job.id, - status: "complete", - homeDir: configHomeDir, - result: { - groupId: result.groupId, - usersAnalyzed: result.usersAnalyzed, - score: result.score, - markdownPath: result.markdownPath, - }, + const remainingUsers: string[] = []; + for (const email of state.pendingUserEmails) { + try { + await analyzeUserByEmail({ + email, + context, + sessionConcurrency, }); - await appendJobLog( - job.id, - `Completed group ${payload.groupId}`, - configHomeDir + } catch (error) { + console.error( + `Failed user rollup ${email}: ${ + error instanceof Error ? error.message : String(error) + }` ); - return; + remainingUsers.push(email); } + } - if (job.type === "analyze_query") { - const payload = job.payload as JobPayloadMap["analyze_query"]; - const queryLabel = - payload.query || - (payload.filters ? JSON.stringify(payload.filters) : ""); - await appendJobLog( - job.id, - `Analyzing query ${queryLabel}`, - configHomeDir - ); - const result = await analyzeQuery({ - query: payload.query, - filters: payload.filters, + const remainingGroups: string[] = []; + for (const groupId of state.pendingGroupIds) { + try { + await analyzeGroupById({ + groupId, context, + sessionConcurrency, }); - - await updateJobStatus({ - id: job.id, - status: "complete", - homeDir: configHomeDir, - result: { - query: result.query, - filters: result.filters, - sessionCount: result.sessionCount, - averageScore: result.averageScore, - }, - }); - await appendJobLog( - job.id, - `Completed query ${queryLabel}`, - configHomeDir + } catch (error) { + console.error( + `Failed group rollup ${groupId}: ${ + error instanceof Error ? error.message : String(error) + }` ); - return; + remainingGroups.push(groupId); } + } - throw new Error(`Unsupported job type ${job.type}`); - } catch (error) { - const message = error instanceof Error ? error.message : String(error); - await updateJobStatus({ - id: job.id, - status: "failed", - homeDir: configHomeDir, - error: message, - }); - await appendJobLog(job.id, `Failed: ${message}`, configHomeDir); + await saveDaemonState( + { + ...state, + pendingUserEmails: remainingUsers, + pendingGroupIds: remainingGroups, + }, + homeDir + ); +} + +async function maybeMarkBackfillComplete(params: { + homeDir?: string; + hasQueuedSessions: boolean; + hasActiveSessions: boolean; +}): Promise { + const state = await loadDaemonState(params.homeDir); + if ( + !state.backfillStartedAt || + state.backfillCompletedAt || + params.hasQueuedSessions || + params.hasActiveSessions || + state.pendingUserEmails.length > 0 || + state.pendingGroupIds.length > 0 + ) { + return; } + + await saveDaemonState( + { + ...state, + backfillCompletedAt: new Date().toISOString(), + }, + params.homeDir + ); } export async function startDaemon(options?: DaemonOptions): Promise { - const intervalMs = options?.intervalMs ?? 1000; + const intervalMs = options?.intervalMs ?? 30_000; const homeDir = options?.homeDir; const paths = getVesaiPaths(homeDir); @@ -228,12 +375,6 @@ export async function startDaemon(options?: DaemonOptions): Promise { const parsedWorkerCount = workerCountOverride ? Number(workerCountOverride) : Number.NaN; - const maxWorkers = - workerCountOverride && - Number.isFinite(parsedWorkerCount) && - parsedWorkerCount > 0 - ? Math.floor(parsedWorkerCount) - : Number.POSITIVE_INFINITY; const onSignal = async () => { shouldStop = true; @@ -244,11 +385,39 @@ export async function startDaemon(options?: DaemonOptions): Promise { process.on(signal, onSignal); } - // Poll queued jobs and dispatch up to the configured worker limit. while (!shouldStop) { - const queued = await listQueuedJobs(homeDir); + const heartbeatNow = new Date().toISOString(); + try { + const heartbeat = await queueSessionsFromHeartbeat({ + homeDir, + nowIso: heartbeatNow, + }); + if (heartbeat.queued > 0) { + console.log( + `Heartbeat queued ${heartbeat.queued} sessions (from ${heartbeat.fromIso} to ${heartbeatNow}).` + ); + } + } catch (error) { + console.error( + `Heartbeat failed: ${error instanceof Error ? error.message : String(error)}` + ); + } - for (const job of queued) { + const queued = await listQueuedJobs(homeDir); + const queuedSessionJobs = queued.filter( + (job) => job.type === "analyze_session" + ); + const maxWorkers = + workerCountOverride && + Number.isFinite(parsedWorkerCount) && + parsedWorkerCount > 0 + ? Math.floor(parsedWorkerCount) + : computeDynamicRenderServiceCapacity({ + maxRenderMemoryMb: (await requireConfig(homeDir)).runtime + .maxRenderMemoryMb, + }); + + for (const job of queuedSessionJobs) { if (activeJobIds.size >= maxWorkers) { break; } @@ -262,20 +431,42 @@ export async function startDaemon(options?: DaemonOptions): Promise { } activeJobIds.add(job.id); - processJob(job, homeDir) + processSessionJob(job, homeDir) + .then(async (outcome) => { + if (!outcome) { + return; + } + await queueDirtyRollups({ + homeDir, + userEmail: outcome.userEmail, + groupId: outcome.groupId, + }); + }) .catch(() => { - // processJob already persists failures. + // processSessionJob already persists failures. }) .finally(() => { activeJobIds.delete(job.id); }); } - await new Promise((resolve) => setTimeout(resolve, intervalMs)); + const hasQueuedSessions = queuedSessionJobs.length > 0; + const hasActiveSessions = activeJobIds.size > 0; + + if (!(hasQueuedSessions || hasActiveSessions)) { + await runPendingRollups(homeDir); + await maybeMarkBackfillComplete({ + homeDir, + hasQueuedSessions, + hasActiveSessions, + }); + } + + await sleep(intervalMs); } while (activeJobIds.size > 0) { - await new Promise((resolve) => setTimeout(resolve, 250)); + await sleep(250); } for (const signal of signals) { diff --git a/daemon/state.ts b/daemon/state.ts new file mode 100644 index 0000000..8164925 --- /dev/null +++ b/daemon/state.ts @@ -0,0 +1,83 @@ +import { mkdir, readFile, writeFile } from "node:fs/promises"; +import { dirname } from "node:path"; +import { getVesaiPaths } from "../config"; + +export type DaemonState = { + version: 1; + lastPulledAt?: string; + backfillStartedAt?: string; + backfillCompletedAt?: string; + pendingUserEmails: string[]; + pendingGroupIds: string[]; + updatedAt: string; +}; + +function getDefaultState(): DaemonState { + return { + version: 1, + pendingUserEmails: [], + pendingGroupIds: [], + updatedAt: new Date().toISOString(), + }; +} + +function normalizeUnique(values: string[]): string[] { + return Array.from( + new Set(values.map((value) => value.trim()).filter(Boolean)) + ); +} + +export async function loadDaemonState(homeDir?: string): Promise { + const paths = getVesaiPaths(homeDir); + try { + const raw = await readFile(paths.daemonStateFile, "utf8"); + const parsed = JSON.parse(raw) as Partial; + + return { + version: 1, + lastPulledAt: parsed.lastPulledAt, + backfillStartedAt: parsed.backfillStartedAt, + backfillCompletedAt: parsed.backfillCompletedAt, + pendingUserEmails: normalizeUnique(parsed.pendingUserEmails || []), + pendingGroupIds: normalizeUnique(parsed.pendingGroupIds || []), + updatedAt: + typeof parsed.updatedAt === "string" + ? parsed.updatedAt + : new Date().toISOString(), + }; + } catch { + return getDefaultState(); + } +} + +export async function saveDaemonState( + state: DaemonState, + homeDir?: string +): Promise { + const paths = getVesaiPaths(homeDir); + await mkdir(dirname(paths.daemonStateFile), { recursive: true }); + + const normalized: DaemonState = { + ...state, + version: 1, + pendingUserEmails: normalizeUnique(state.pendingUserEmails), + pendingGroupIds: normalizeUnique(state.pendingGroupIds), + updatedAt: new Date().toISOString(), + }; + + await writeFile( + paths.daemonStateFile, + JSON.stringify(normalized, null, 2), + "utf8" + ); +} + +export async function updateDaemonState(params: { + homeDir?: string; + updater: (state: DaemonState) => DaemonState; +}): Promise { + const current = await loadDaemonState(params.homeDir); + const next = params.updater(current); + await saveDaemonState(next, params.homeDir); + return loadDaemonState(params.homeDir); +} diff --git a/render/events.ts b/render/events.ts index b3102c8..ad41912 100644 --- a/render/events.ts +++ b/render/events.ts @@ -1075,7 +1075,7 @@ async function uploadEventsToGCS(params: { }): Promise { const { projectId, sessionId, eventsJson, eventsCount, bucketName } = params; const fileName = `${sessionId}.json`; - const filePath = `${projectId}/${fileName}`; + const filePath = `projects/${projectId}/events/${fileName}`; console.log(` πŸ—‚οΈ Bucket: ${bucketName}`); console.log(` πŸ“ File path: ${filePath}`); diff --git a/render/global-render-slot.ts b/render/global-render-slot.ts new file mode 100644 index 0000000..0a54e08 --- /dev/null +++ b/render/global-render-slot.ts @@ -0,0 +1,176 @@ +import { randomUUID } from "node:crypto"; +import { mkdir, open, readFile, rm, stat } from "node:fs/promises"; +import { join } from "node:path"; +import { getVesaiCorePaths, resolveVesaiHome } from "../config"; +import { computeDynamicRenderServiceCapacity } from "../config/runtime"; + +const LOCK_RETRY_MS = 200; +const STALE_LOCK_MAX_AGE_MS = 5 * 60 * 1000; + +type LockMetadata = { + pid?: number; + token?: string; +}; + +type SlotLease = { + lockPath: string; + token: string; +}; + +function sleep(ms: number): Promise { + return new Promise((resolve) => { + setTimeout(resolve, ms); + }); +} + +function normalizeLimit(value: number): number { + if (!Number.isFinite(value) || value < 1) { + return 1; + } + return Math.floor(value); +} + +function processExists(pid: number): boolean { + try { + process.kill(pid, 0); + return true; + } catch (error) { + return (error as NodeJS.ErrnoException).code === "EPERM"; + } +} + +async function readLockMetadata( + lockPath: string +): Promise { + try { + const raw = await readFile(lockPath, "utf8"); + return JSON.parse(raw) as LockMetadata; + } catch { + return null; + } +} + +async function tryReleaseStaleLock(lockPath: string): Promise { + let raw = ""; + try { + raw = await readFile(lockPath, "utf8"); + } catch { + return; + } + + let metadata: LockMetadata | null = null; + try { + metadata = JSON.parse(raw) as LockMetadata; + } catch { + // Avoid racing on actively-written lock files. Only clear malformed locks + // once they are old enough to be considered stale. + try { + const fileStats = await stat(lockPath); + const ageMs = Date.now() - fileStats.mtimeMs; + if (ageMs > STALE_LOCK_MAX_AGE_MS) { + await rm(lockPath, { force: true }); + } + } catch { + // Ignore lock cleanup failures and retry on next polling pass. + } + return; + } + + const pid = Number(metadata?.pid); + if (Number.isFinite(pid) && pid > 0 && processExists(pid)) { + return; + } + + await rm(lockPath, { force: true }); +} + +async function tryAcquireSlot(params: { + locksDir: string; + limit: number; + token: string; +}): Promise { + for (let slot = 0; slot < params.limit; slot++) { + const lockPath = join(params.locksDir, `slot-${slot}.lock`); + try { + const handle = await open(lockPath, "wx"); + try { + await handle.writeFile( + JSON.stringify( + { + pid: process.pid, + token: params.token, + acquiredAt: new Date().toISOString(), + }, + null, + 2 + ), + "utf8" + ); + } finally { + await handle.close(); + } + + return { + lockPath, + token: params.token, + }; + } catch (error) { + const code = (error as NodeJS.ErrnoException).code; + if (code === "EEXIST") { + await tryReleaseStaleLock(lockPath); + continue; + } + + throw error; + } + } + + return null; +} + +async function releaseSlot(lease: SlotLease): Promise { + try { + const metadata = await readLockMetadata(lease.lockPath); + if (metadata?.token && metadata.token !== lease.token) { + return; + } + } catch { + // Continue and best-effort remove lock. + } + + await rm(lease.lockPath, { force: true }); +} + +export async function withGlobalRenderSlot(params: { + maxRenderMemoryMb: number; + task: () => Promise; +}): Promise { + const corePaths = getVesaiCorePaths(resolveVesaiHome()); + await mkdir(corePaths.renderLocksDir, { recursive: true }); + + const token = randomUUID(); + let lease: SlotLease | null = null; + + while (!lease) { + const dynamicLimit = normalizeLimit( + computeDynamicRenderServiceCapacity({ + maxRenderMemoryMb: params.maxRenderMemoryMb, + }) + ); + + lease = await tryAcquireSlot({ + locksDir: corePaths.renderLocksDir, + limit: dynamicLimit, + token, + }); + if (!lease) { + await sleep(LOCK_RETRY_MS); + } + } + + try { + return await params.task(); + } finally { + await releaseSlot(lease); + } +} diff --git a/render/replay.ts b/render/replay.ts index f1c71b0..7aa82ff 100644 --- a/render/replay.ts +++ b/render/replay.ts @@ -845,7 +845,7 @@ export default async function constructVideo(params: { const fileName = `${params.sessionId}.webm`; const bucketName = params.bucketName || process.env.VESAI_GCS_BUCKET || "ves.ai"; - const filePath = `${params.projectId}/${fileName}`; + const filePath = `projects/${params.projectId}/videos/${fileName}`; console.log(` πŸ—‚οΈ Bucket: ${bucketName}`); console.log(` πŸ“ File path: ${filePath}`); diff --git a/tests/cli-help.test.ts b/tests/cli-help.test.ts index 505606a..02784a2 100644 --- a/tests/cli-help.test.ts +++ b/tests/cli-help.test.ts @@ -2,101 +2,65 @@ import { describe, expect, it } from "bun:test"; import { runCommandOrThrow } from "../connectors/shell"; describe("cli help", () => { - it("shows rich top-level help", async () => { + it("shows replay-focused top-level help", async () => { const result = await runCommandOrThrow("bun", ["cli/index.ts", "--help"]); - expect(result.stdout).toContain("VES AI: AI-ready product analytics"); - expect(result.stdout).toContain("Replay Workflows"); - expect(result.stdout).toContain("Agent Workflows"); - expect(result.stdout).toContain("JSON is default for data commands"); - expect(result.stdout).toContain("replays query"); - expect(result.stdout).toContain("insights sql"); - expect(result.stdout).not.toContain("tui"); - }); - - it("shows robust query filter help", async () => { - const result = await runCommandOrThrow("bun", [ - "cli/index.ts", - "replays", - "query", - "--help", - ]); - - expect(result.stdout).toContain("--email "); - expect(result.stdout).toContain("--from "); - expect(result.stdout).toContain("--where "); - expect(result.stdout).toContain("--dry-run"); - expect(result.stdout).toContain("--no-json"); expect(result.stdout).toContain( - "literal search over replay/session metadata" + "VES AI: session replay intelligence for agents" ); - expect(result.stdout).toContain("Examples:"); + expect(result.stdout).toContain("vesai init"); + expect(result.stdout).toContain("Replay Story Workflows"); + expect(result.stdout).toContain("vesai user"); + expect(result.stdout).toContain("vesai group"); + expect(result.stdout).toContain("vesai research"); + expect(result.stdout).not.toContain("vesai events"); + expect(result.stdout).not.toContain("vesai insights"); }); - it("teaches in-context learning for replay entity subcommands", async () => { - const userHelp = await runCommandOrThrow("bun", [ + it("shows user command help", async () => { + const result = await runCommandOrThrow("bun", [ "cli/index.ts", - "replays", "user", "--help", ]); - const listHelp = await runCommandOrThrow("bun", [ - "cli/index.ts", - "replays", - "list", - "--help", - ]); - expect(userHelp.stdout).toContain("User analysis contract"); - expect(userHelp.stdout).toContain("Learning flow"); - expect(listHelp.stdout).toContain("discover candidates"); - expect(listHelp.stdout).toContain("Next step"); + expect(result.stdout).toContain("Analyze one user story"); + expect(result.stdout).toContain("--max-concurrent "); }); - it("shows PostHog analytics help surfaces", async () => { - const events = await runCommandOrThrow("bun", [ - "cli/index.ts", - "events", - "--help", - ]); - const insights = await runCommandOrThrow("bun", [ + it("shows group command help", async () => { + const result = await runCommandOrThrow("bun", [ "cli/index.ts", - "insights", + "group", "--help", ]); - expect(events.stdout).toContain("event definitions"); - expect(insights.stdout).toContain("hogql"); - expect(insights.stdout).toContain("sql"); + expect(result.stdout).toContain("Analyze one group story"); + expect(result.stdout).toContain(""); }); - it("teaches logs query workflow in help output", async () => { + it("shows research command help", async () => { const result = await runCommandOrThrow("bun", [ "cli/index.ts", - "logs", - "query", + "research", "--help", ]); - expect(result.stdout).toContain("vesai logs attributes"); - expect(result.stdout).toContain("vesai logs values "); + expect(result.stdout).toContain("already analyzed sessions"); + expect(result.stdout).toContain("--limit "); }); - it("shows robust quickstart help", async () => { + it("shows init lookback option", async () => { const result = await runCommandOrThrow("bun", [ "cli/index.ts", - "quickstart", + "init", "--help", ]); - expect(result.stdout).toContain("--posthog-api-key "); - expect(result.stdout).toContain("--non-interactive"); - expect(result.stdout).toContain("--max-concurrent-renders "); - expect(result.stdout).toContain("All access + MCP server scope"); - expect(result.stdout).toContain("Examples:"); + expect(result.stdout).toContain("--lookback-days "); }); - it("shows daemon lifecycle help for background and foreground modes", async () => { + it("shows daemon lifecycle help", async () => { const result = await runCommandOrThrow("bun", [ "cli/index.ts", "daemon", @@ -105,18 +69,6 @@ describe("cli help", () => { expect(result.stdout).toContain("start"); expect(result.stdout).toContain("watch"); - expect(result.stdout).toContain("background"); - expect(result.stdout).toContain("foreground"); - }); - - it("shows safe config display options", async () => { - const result = await runCommandOrThrow("bun", [ - "cli/index.ts", - "config", - "show", - "--help", - ]); - - expect(result.stdout).toContain("--show-secrets"); + expect(result.stdout).toContain("stop"); }); }); diff --git a/tests/config.test.ts b/tests/config.test.ts index 57d9b99..a0d2014 100644 --- a/tests/config.test.ts +++ b/tests/config.test.ts @@ -1,33 +1,52 @@ import { describe, expect, it } from "bun:test"; -import { mkdtemp, readFile, rm, stat } from "node:fs/promises"; +import { mkdtemp, readFile, rm, stat, writeFile } from "node:fs/promises"; import { tmpdir } from "node:os"; import { join } from "node:path"; import { - ensureVesaiDirectories, + ensureCoreDirectories, + ensureProjectDirectories, + ensureProjectGitignore, ensureWorkspaceGitRepo, + getVesaiCorePaths, getVesaiPaths, loadConfig, - saveConfig, - updateConfig, + loadCoreConfig, + saveCoreConfig, + saveProjectConfig, + updateCoreConfig, + updateProjectConfig, } from "../config"; import { makeConfig } from "./helpers"; -async function withTempHome(run: (homeDir: string) => Promise) { - const homeDir = await mkdtemp(join(tmpdir(), "vesai-config-test-")); +async function withTempDir( + prefix: string, + run: (dir: string) => Promise +): Promise { + const dir = await mkdtemp(join(tmpdir(), prefix)); try { - await run(homeDir); + await run(dir); } finally { - await rm(homeDir, { recursive: true, force: true }); + await rm(dir, { recursive: true, force: true }); } } describe("config", () => { - it("creates required ~/.vesai directories", async () => { - await withTempHome(async (homeDir) => { - await ensureVesaiDirectories(homeDir); - const paths = getVesaiPaths(homeDir); + it("creates required core and project directories", async () => { + await withTempDir("vesai-core-test-", async (homeDir) => { + await ensureCoreDirectories(homeDir); + const corePaths = getVesaiCorePaths(homeDir); - await stat(paths.home); + await stat(corePaths.home); + await stat(corePaths.logsDir); + await stat(corePaths.tmpDir); + await stat(corePaths.renderLocksDir); + }); + + await withTempDir("vesai-project-test-", async (projectRoot) => { + await ensureProjectDirectories(projectRoot); + const paths = getVesaiPaths(projectRoot); + + await stat(paths.vesaiDir); await stat(paths.workspace); await stat(paths.sessionsDir); await stat(paths.usersDir); @@ -39,44 +58,101 @@ describe("config", () => { }); }); - it("saves, loads, and updates vesai config", async () => { - await withTempHome(async (homeDir) => { - const config = makeConfig(); - const { - createdAt: _createdAt, - updatedAt: _updatedAt, - ...withoutTimestamps - } = config; - await saveConfig(withoutTimestamps, homeDir); - - const loaded = await loadConfig(homeDir); - expect(loaded.posthog.projectId).toBe("12345"); - expect(loaded.createdAt.length).toBeGreaterThan(0); - expect(loaded.updatedAt.length).toBeGreaterThan(0); - - const updated = await updateConfig({ - homeDir, - updater: (current) => ({ - ...current, - runtime: { - ...current.runtime, - maxConcurrentRenders: 4, - }, - }), + it("saves, loads, and updates merged core + project config", async () => { + await withTempDir("vesai-core-test-", async (homeDir) => { + await withTempDir("vesai-project-test-", async (projectRoot) => { + const config = makeConfig(); + + const { + createdAt: _coreCreatedAt, + updatedAt: _coreUpdatedAt, + ...coreWithoutTimestamps + } = config.core; + await saveCoreConfig(coreWithoutTimestamps, homeDir); + + const { + createdAt: _projectCreatedAt, + updatedAt: _projectUpdatedAt, + ...projectWithoutTimestamps + } = config.project; + await saveProjectConfig({ + config: projectWithoutTimestamps, + projectRoot, + }); + + process.env.VESAI_HOME = homeDir; + const loaded = await loadConfig(projectRoot); + expect(loaded.posthog.projectId).toBe("12345"); + expect(loaded.runtime.lookbackDays).toBe(180); + expect(loaded.runtime.maxRenderMemoryMb).toBe(1024); + + const updatedCore = await updateCoreConfig({ + homeDir, + updater: (current) => ({ + ...current, + runtime: { ...current.runtime, maxRenderMemoryMb: 2048 }, + }), + }); + expect(updatedCore.runtime.maxRenderMemoryMb).toBe(2048); + + const updatedProject = await updateProjectConfig({ + projectRoot, + updater: (current) => ({ + ...current, + daemon: { ...current.daemon, lookbackDays: 90 }, + }), + }); + expect(updatedProject.daemon.lookbackDays).toBe(90); + + const reloaded = await loadConfig(projectRoot); + expect(reloaded.runtime.maxRenderMemoryMb).toBe(2048); + expect(reloaded.runtime.lookbackDays).toBe(90); + delete process.env.VESAI_HOME; }); + }); + }); + + it("migrates legacy maxConcurrentRenders core config on load", async () => { + await withTempDir("vesai-core-test-", async (homeDir) => { + const corePaths = getVesaiCorePaths(homeDir); + await ensureCoreDirectories(homeDir); + await writeFile( + corePaths.coreConfigFile, + JSON.stringify( + { + version: 1, + gcloud: { + projectId: "legacy-project", + region: "us-central1", + bucket: "legacy-bucket", + }, + vertex: { + model: "gemini-3-pro-preview", + location: "us-central1", + }, + runtime: { + maxConcurrentRenders: 3, + }, + createdAt: "2026-01-01T00:00:00.000Z", + updatedAt: "2026-01-01T00:00:00.000Z", + }, + null, + 2 + ), + "utf8" + ); - expect(updated.runtime.maxConcurrentRenders).toBe(4); - const reloaded = await loadConfig(homeDir); - expect(reloaded.runtime.maxConcurrentRenders).toBe(4); + const loaded = await loadCoreConfig(homeDir); + expect(loaded.runtime.maxRenderMemoryMb).toBe(1536); }); }); - it("initializes git-ready workspace", async () => { - await withTempHome(async (homeDir) => { - await ensureVesaiDirectories(homeDir); - await ensureWorkspaceGitRepo(homeDir); + it("initializes git-ready workspace inside project .vesai", async () => { + await withTempDir("vesai-project-test-", async (projectRoot) => { + await ensureProjectDirectories(projectRoot); + await ensureWorkspaceGitRepo(projectRoot); - const paths = getVesaiPaths(homeDir); + const paths = getVesaiPaths(projectRoot); const head = await readFile( join(paths.workspace, ".git", "HEAD"), "utf8" @@ -90,4 +166,18 @@ describe("config", () => { expect(gitignore).toContain(".DS_Store"); }); }); + + it("ensures .vesai is added to project root .gitignore", async () => { + await withTempDir("vesai-project-test-", async (projectRoot) => { + const gitignorePath = join(projectRoot, ".gitignore"); + await writeFile(gitignorePath, "node_modules/\n", "utf8"); + + await ensureProjectGitignore(projectRoot); + await ensureProjectGitignore(projectRoot); + + const gitignore = await readFile(gitignorePath, "utf8"); + expect(gitignore).toContain(".vesai/"); + expect(gitignore.match(/\.vesai\//g)?.length).toBe(1); + }); + }); }); diff --git a/tests/gemini.test.ts b/tests/gemini.test.ts index aa9e198..2fe6c87 100644 --- a/tests/gemini.test.ts +++ b/tests/gemini.test.ts @@ -3,6 +3,7 @@ import { analyzeGroupAggregate, analyzeSessionVideo, analyzeUserAggregate, + answerResearchQuestion, } from "../connectors/gemini"; describe("gemini connector", () => { @@ -140,4 +141,44 @@ describe("gemini connector", () => { expect(thrown).toBe(true); }); + + it("parses research answer JSON", async () => { + const generateContent = mock(async () => ({ + text: JSON.stringify({ + answer: "Most drop-off follows payment errors.", + findings: [ + "Card validation loops appear repeatedly", + "Users abandon after 2-3 retries", + ], + confidence: "medium", + supportingSessionIds: ["s1", "s2"], + }), + })); + + const ai = { + models: { + generateContent, + }, + } as never; + + const result = await answerResearchQuestion({ + ai, + model: "gemini-3-pro-preview", + productDescription: "Demo product", + question: "Why do users drop at payment?", + sessions: [ + { + sessionId: "s1", + startTime: "2026-01-01T00:00:00.000Z", + score: 60, + markdownPath: "/tmp/s1.md", + summary: "Payment validation errors observed.", + }, + ], + }); + + expect(result.confidence).toBe("medium"); + expect(result.supportingSessionIds).toEqual(["s1", "s2"]); + expect(generateContent).toHaveBeenCalledTimes(1); + }); }); diff --git a/tests/global-render-slot.test.ts b/tests/global-render-slot.test.ts new file mode 100644 index 0000000..b5b8270 --- /dev/null +++ b/tests/global-render-slot.test.ts @@ -0,0 +1,86 @@ +import { describe, expect, it } from "bun:test"; +import { mkdtemp, readdir, rm } from "node:fs/promises"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; +import { getVesaiCorePaths } from "../config"; +import { withGlobalRenderSlot } from "../render/global-render-slot"; + +function sleep(ms: number): Promise { + return new Promise((resolve) => { + setTimeout(resolve, ms); + }); +} + +async function withTempVesaiHome( + run: (homeDir: string) => Promise +): Promise { + const homeDir = await mkdtemp(join(tmpdir(), "vesai-lock-test-")); + const previous = process.env.VESAI_HOME; + process.env.VESAI_HOME = homeDir; + + try { + await run(homeDir); + } finally { + if (previous === undefined) { + delete process.env.VESAI_HOME; + } else { + process.env.VESAI_HOME = previous; + } + await rm(homeDir, { recursive: true, force: true }); + } +} + +describe("global render slot", () => { + it("releases lock files after task completes", async () => { + await withTempVesaiHome(async (homeDir) => { + const paths = getVesaiCorePaths(homeDir); + + await withGlobalRenderSlot({ + maxRenderMemoryMb: 512, + task: async () => { + const files = await readdir(paths.renderLocksDir); + expect(files.length).toBe(1); + expect(files[0]).toBe("slot-0.lock"); + }, + }); + + const filesAfter = await readdir(paths.renderLocksDir); + expect(filesAfter.length).toBe(0); + }); + }); + + it("serializes work when limit is one", async () => { + await withTempVesaiHome(async () => { + let releaseFirst: () => void = () => {}; + const firstGate = new Promise((resolve) => { + releaseFirst = resolve; + }); + + const first = withGlobalRenderSlot({ + maxRenderMemoryMb: 512, + task: async () => { + await firstGate; + }, + }); + + await sleep(120); + + let secondEnteredAt = 0; + const second = withGlobalRenderSlot({ + maxRenderMemoryMb: 512, + task: async () => { + secondEnteredAt = Date.now(); + }, + }); + + await sleep(120); + expect(secondEnteredAt).toBe(0); + + releaseFirst(); + await first; + await second; + + expect(secondEnteredAt).toBeGreaterThan(0); + }); + }); +}); diff --git a/tests/helpers.ts b/tests/helpers.ts index b6ea85f..92dba44 100644 --- a/tests/helpers.ts +++ b/tests/helpers.ts @@ -4,9 +4,27 @@ type DeepPartial = { [K in keyof T]?: T[K] extends object ? DeepPartial : T[K]; }; +function deepMerge(base: T, overrides?: DeepPartial): T { + if (!overrides) { + return base; + } + const out: Record = { ...(base as Record) }; + for (const [key, value] of Object.entries(overrides)) { + if (value && typeof value === "object" && !Array.isArray(value)) { + out[key] = deepMerge( + (base as Record)[key] as Record, + value as Record + ); + } else { + out[key] = value as unknown; + } + } + return out as T; +} + export function makeConfig(overrides?: DeepPartial): VesaiConfig { const base: VesaiConfig = { - version: 1, + projectId: "5ab526cb-718f-4d6b-9226-0fdc9f31d8ef", posthog: { host: "https://us.posthog.com", apiKey: "phx_test", @@ -24,22 +42,54 @@ export function makeConfig(overrides?: DeepPartial): VesaiConfig { location: "us-central1", }, runtime: { - maxConcurrentRenders: 2, + maxRenderMemoryMb: 1024, + lookbackDays: 180, + }, + daemon: { + lookbackDays: 180, }, product: { description: "Test product", }, createdAt: "2026-01-01T00:00:00.000Z", updatedAt: "2026-01-01T00:00:00.000Z", + core: { + version: 1, + gcloud: { + projectId: "test-project", + region: "us-central1", + bucket: "test-bucket", + }, + vertex: { + model: "gemini-3-pro-preview", + location: "us-central1", + }, + runtime: { + maxRenderMemoryMb: 1024, + }, + createdAt: "2026-01-01T00:00:00.000Z", + updatedAt: "2026-01-01T00:00:00.000Z", + }, + project: { + version: 1, + projectId: "5ab526cb-718f-4d6b-9226-0fdc9f31d8ef", + posthog: { + host: "https://us.posthog.com", + apiKey: "phx_test", + projectId: "12345", + groupKey: "company_id", + domainFilter: "example.com", + }, + daemon: { + lookbackDays: 180, + }, + product: { + description: "Test product", + }, + createdAt: "2026-01-01T00:00:00.000Z", + updatedAt: "2026-01-01T00:00:00.000Z", + }, }; - return { - ...base, - ...overrides, - posthog: { ...base.posthog, ...(overrides?.posthog || {}) }, - gcloud: { ...base.gcloud, ...(overrides?.gcloud || {}) }, - vertex: { ...base.vertex, ...(overrides?.vertex || {}) }, - runtime: { ...base.runtime, ...(overrides?.runtime || {}) }, - product: { ...base.product, ...(overrides?.product || {}) }, - }; + return deepMerge(base, overrides); } diff --git a/tests/insights-format.test.ts b/tests/insights-format.test.ts deleted file mode 100644 index 1b97076..0000000 --- a/tests/insights-format.test.ts +++ /dev/null @@ -1,95 +0,0 @@ -import { describe, expect, it } from "bun:test"; -import { - extractFirstPipeTable, - normalizeHogqlInsight, - normalizeSqlInsight, -} from "../cli/commands/insights-format"; - -describe("insights format helpers", () => { - it("extracts first pipe table from fenced text", () => { - const input = ` -Some intro text -\`\`\` -event|count() -$pageview|42 -$autocapture|11 -\`\`\` -`; - - const table = extractFirstPipeTable(input); - expect(table).not.toBeNull(); - expect(table?.columns).toEqual(["event", "count()"]); - expect(table?.rows).toEqual([ - ["$pageview", "42"], - ["$autocapture", "11"], - ]); - }); - - it("prefers richer results table over placeholder examples", () => { - const input = ` -\`\`\` -column1|column2 -value1|value2 -\`\`\` - -\`\`\` -event|count() -$pageview|42 -$autocapture|11 -$groupidentify|7 -\`\`\` -`; - - const table = extractFirstPipeTable(input); - expect(table?.columns).toEqual(["event", "count()"]); - expect(table?.rows.length).toBe(3); - }); - - it("normalizes SQL insight payload to structured rows", () => { - const raw = ` -You are given a table: -\`\`\` -event|count() -$pageview|99 -$groupidentify|50 -\`\`\` -`; - - const normalized = normalizeSqlInsight(raw); - expect(normalized.kind).toBe("table"); - expect(normalized.rowCount).toBe(2); - expect(normalized.columns).toEqual(["event", "count()"]); - expect(normalized.rows[0]).toEqual(["$pageview", "99"]); - }); - - it("normalizes HogQL insight payload to query + table", () => { - const raw = [ - { - type: "message", - data: { - answer: { - kind: "TrendsQuery", - interval: "week", - }, - }, - }, - { - type: "message", - data: { - content: `Result: -\`\`\` -Date|active_users -2026-02-01|120 -\`\`\``, - artifact_id: "abc123", - }, - }, - ]; - - const normalized = normalizeHogqlInsight(raw); - expect(normalized.query).not.toBeNull(); - expect(normalized.artifactId).toBe("abc123"); - expect(normalized.table?.columns).toEqual(["Date", "active_users"]); - expect(normalized.table?.rows).toEqual([["2026-02-01", "120"]]); - }); -}); diff --git a/tests/jobs.store.test.ts b/tests/jobs.store.test.ts index 1bd6fb3..e9f3587 100644 --- a/tests/jobs.store.test.ts +++ b/tests/jobs.store.test.ts @@ -2,7 +2,7 @@ import { describe, expect, it } from "bun:test"; import { mkdtemp, rm } from "node:fs/promises"; import { tmpdir } from "node:os"; import { join } from "node:path"; -import { ensureVesaiDirectories } from "../config"; +import { ensureProjectDirectories } from "../config"; import { appendJobLog, createJob, @@ -15,7 +15,7 @@ import { async function withTempHome(run: (homeDir: string) => Promise) { const homeDir = await mkdtemp(join(tmpdir(), "vesai-jobs-test-")); try { - await ensureVesaiDirectories(homeDir); + await ensureProjectDirectories(homeDir); await run(homeDir); } finally { await rm(homeDir, { recursive: true, force: true }); @@ -26,8 +26,8 @@ describe("job store", () => { it("creates and reads jobs", async () => { await withTempHome(async (homeDir) => { const job = await createJob({ - type: "analyze_user", - payload: { email: "user@example.com" }, + type: "analyze_session", + payload: { sessionId: "session_123" }, homeDir, }); @@ -41,8 +41,8 @@ describe("job store", () => { it("tracks running and completion lifecycle", async () => { await withTempHome(async (homeDir) => { const job = await createJob({ - type: "analyze_query", - payload: { query: "checkout" }, + type: "analyze_session", + payload: { sessionId: "session_abc" }, homeDir, }); @@ -67,13 +67,13 @@ describe("job store", () => { it("lists queued jobs and appends logs", async () => { await withTempHome(async (homeDir) => { const queued = await createJob({ - type: "analyze_group", - payload: { groupId: "acme" }, + type: "analyze_session", + payload: { sessionId: "s1" }, homeDir, }); const notQueued = await createJob({ type: "analyze_session", - payload: { sessionId: "s1" }, + payload: { sessionId: "s2" }, homeDir, }); await updateJobStatus({ id: notQueued.id, status: "running", homeDir }); diff --git a/tests/paths.test.ts b/tests/paths.test.ts index 3f1dea9..4bd56ae 100644 --- a/tests/paths.test.ts +++ b/tests/paths.test.ts @@ -1,17 +1,26 @@ import { describe, expect, it } from "bun:test"; -import { getVesaiPaths, resolveVesaiHome } from "../config/paths"; +import { mkdir, mkdtemp, rm, writeFile } from "node:fs/promises"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; +import { + findProjectRoot, + getVesaiCorePaths, + getVesaiPaths, + resolveProjectRoot, + resolveVesaiHome, +} from "../config/paths"; describe("paths", () => { - it("resolves VESAI_HOME override", () => { + it("resolves VESAI_HOME override for core paths", () => { const previous = process.env.VESAI_HOME; process.env.VESAI_HOME = "/tmp/custom-vesai-home"; const resolved = resolveVesaiHome(); - const paths = getVesaiPaths(resolved); + const paths = getVesaiCorePaths(resolved); expect(resolved).toBe("/tmp/custom-vesai-home"); - expect(paths.configFile).toBe("/tmp/custom-vesai-home/vesai.json"); - expect(paths.sessionsDir).toBe("/tmp/custom-vesai-home/workspace/sessions"); + expect(paths.coreConfigFile).toBe("/tmp/custom-vesai-home/core.json"); + expect(paths.renderLocksDir).toBe("/tmp/custom-vesai-home/render-locks"); if (previous === undefined) { delete process.env.VESAI_HOME; @@ -19,4 +28,43 @@ describe("paths", () => { process.env.VESAI_HOME = previous; } }); + + it("finds and resolves project root from nested directory", async () => { + const projectRoot = await mkdtemp(join(tmpdir(), "vesai-paths-project-")); + const nested = join(projectRoot, "apps", "web"); + + try { + await mkdir(join(projectRoot, ".vesai"), { recursive: true }); + await mkdir(nested, { recursive: true }); + await writeFile( + join(projectRoot, ".vesai", "project.json"), + "{}", + "utf8" + ); + + const found = findProjectRoot(nested); + expect(found).toBe(projectRoot); + expect(resolveProjectRoot(nested)).toBe(projectRoot); + } finally { + await rm(projectRoot, { recursive: true, force: true }); + } + }); + + it("builds project-scoped .vesai paths", async () => { + const projectRoot = await mkdtemp(join(tmpdir(), "vesai-paths-project-")); + try { + const paths = getVesaiPaths(projectRoot); + expect(paths.configFile).toBe( + join(projectRoot, ".vesai", "project.json") + ); + expect(paths.sessionsDir).toBe( + join(projectRoot, ".vesai", "workspace", "sessions") + ); + expect(paths.daemonStateFile).toBe( + join(projectRoot, ".vesai", "daemon-state.json") + ); + } finally { + await rm(projectRoot, { recursive: true, force: true }); + } + }); }); diff --git a/tests/posthog.test.ts b/tests/posthog.test.ts index 65edc37..ef19276 100644 --- a/tests/posthog.test.ts +++ b/tests/posthog.test.ts @@ -1,11 +1,9 @@ import { afterEach, describe, expect, it, mock } from "bun:test"; import { findRecordingsByGroupId, - findRecordingsByQuery, findRecordingsByUserEmail, getRecordingUserEmail, listAllRecordings, - readDataWarehouseSchema, } from "../connectors/posthog"; const originalFetch = globalThis.fetch; @@ -106,7 +104,7 @@ describe("posthog connector", () => { expect(results.map((recording) => recording.id)).toEqual(["keep"]); }); - it("filters by group id and query", async () => { + it("filters by group id", async () => { globalThis.fetch = mock(async () => { return new Response( JSON.stringify({ @@ -117,12 +115,7 @@ describe("posthog connector", () => { person: { properties: { company_id: "acme" } }, ongoing: false, }, - { - id: "g2", - start_url: "https://app.example.com/checkout", - person: { properties: { company_id: "acme", role: "buyer" } }, - ongoing: false, - }, + { id: "g2", person: { properties: { company_id: "acme" } } }, ], has_next: false, }), @@ -138,98 +131,6 @@ describe("posthog connector", () => { domainFilter: "example.com", }); - const byQuery = await findRecordingsByQuery({ - apiKey: "key", - projectId: "123", - query: "checkout", - domainFilter: "example.com", - }); - expect(byGroup.length).toBe(2); - expect(byQuery.map((recording) => recording.id)).toEqual(["g2"]); - }); - - it("supports advanced query filters (date, activity, properties, limit)", async () => { - globalThis.fetch = mock(async () => { - return new Response( - JSON.stringify({ - results: [ - { - id: "s1", - start_time: "2026-01-10T00:00:00.000Z", - active_seconds: 20, - distinct_id: "d1", - start_url: "https://app.example.com/checkout", - person: { properties: { email: "a@example.com", plan: "pro" } }, - ongoing: false, - }, - { - id: "s2", - start_time: "2026-01-20T00:00:00.000Z", - active_seconds: 90, - distinct_id: "d2", - start_url: "https://app.example.com/checkout", - person: { - properties: { email: "target@example.com", plan: "enterprise" }, - }, - ongoing: false, - }, - { - id: "s3", - start_time: "2026-02-01T00:00:00.000Z", - active_seconds: 120, - distinct_id: "d3", - start_url: "https://app.example.com/reports", - person: { - properties: { email: "target@example.com", plan: "enterprise" }, - }, - ongoing: false, - }, - ], - has_next: false, - }), - { status: 200 } - ); - }) as unknown as typeof fetch; - - const results = await findRecordingsByQuery({ - apiKey: "key", - projectId: "123", - filters: { - email: "target@example.com", - urlContains: "checkout", - minActiveSeconds: 60, - startsAfter: "2026-01-15T00:00:00.000Z", - startsBefore: "2026-01-31T23:59:59.000Z", - properties: { plan: "enterprise" }, - limit: 1, - }, - }); - - expect(results.map((recording) => recording.id)).toEqual(["s2"]); - }); - - it("sends query payload for warehouse schema MCP tool", async () => { - let body: unknown; - globalThis.fetch = mock( - async (_url: string | URL | Request, init?: RequestInit) => { - body = init?.body ? JSON.parse(String(init.body)) : null; - return new Response( - JSON.stringify({ - success: true, - content: '{"ok":true}', - }), - { status: 200 } - ); - } - ) as unknown as typeof fetch; - - const result = await readDataWarehouseSchema({ - apiKey: "key", - projectId: "123", - }); - - expect(body).toEqual({ args: { query: {} } }); - expect(result).toEqual({ ok: true }); }); }); diff --git a/tests/query-filters.test.ts b/tests/query-filters.test.ts deleted file mode 100644 index d2b8280..0000000 --- a/tests/query-filters.test.ts +++ /dev/null @@ -1,93 +0,0 @@ -import { describe, expect, it } from "bun:test"; -import { - buildQueryFilters, - parseWhereAssignments, -} from "../cli/commands/query-filters"; -import { makeConfig } from "./helpers"; - -describe("query filter parser", () => { - it("builds filters with config defaults", () => { - const config = makeConfig({ - posthog: { - domainFilter: "app.example.com", - groupKey: "company_id", - }, - }); - - const filters = buildQueryFilters({ - text: "checkout", - config, - options: { - group: "acme", - }, - }); - - expect(filters.text).toBe("checkout"); - expect(filters.domain).toBe("app.example.com"); - expect(filters.groupId).toBe("acme"); - expect(filters.groupKey).toBe("company_id"); - }); - - it("supports all-domains and where parsing", () => { - const config = makeConfig(); - const filters = buildQueryFilters({ - text: undefined, - config, - options: { - allDomains: true, - where: ["plan=enterprise", "region=us"], - email: "user@example.com", - }, - }); - - expect(filters.domain).toBeUndefined(); - expect(filters.email).toBe("user@example.com"); - expect(filters.properties).toEqual({ - plan: "enterprise", - region: "us", - }); - }); - - it("validates ranges and empty query", () => { - const config = makeConfig(); - - expect(() => - buildQueryFilters({ - text: undefined, - config, - options: {}, - }) - ).toThrow("No query filters provided"); - - expect(() => - buildQueryFilters({ - text: "checkout", - config, - options: { - minActive: 100, - maxActive: 50, - }, - }) - ).toThrow("--min-active cannot be greater than --max-active"); - - expect(() => - buildQueryFilters({ - text: "checkout", - config, - options: { - limit: 0, - }, - }) - ).toThrow("--limit must be a positive integer"); - }); - - it("parses and validates where assignments", () => { - expect(parseWhereAssignments(["plan=enterprise"])).toEqual({ - plan: "enterprise", - }); - - expect(() => parseWhereAssignments(["broken"])).toThrow( - 'Invalid --where value "broken"' - ); - }); -}); diff --git a/tests/quickstart-cli.test.ts b/tests/quickstart-cli.test.ts index 559d49d..39c90a6 100644 --- a/tests/quickstart-cli.test.ts +++ b/tests/quickstart-cli.test.ts @@ -1,12 +1,16 @@ import { describe, expect, it } from "bun:test"; import { - computeDefaultRenderConcurrency, defaultBucketLocationFromVertex, - formatGiB, normalizeBucket, normalizeConcurrencyValue, parseProjectSelectionIndex, } from "../cli/commands/quickstart"; +import { + computeDefaultRenderMemoryMb, + estimateRenderServiceCapacity, + formatGiB, + normalizeRenderMemoryMbValue, +} from "../config/runtime"; describe("quickstart cli helpers", () => { it("normalizes bucket values", () => { @@ -37,33 +41,41 @@ describe("quickstart cli helpers", () => { ); }); - it("normalizes valid concurrency values and rejects invalid ones", () => { - expect( - normalizeConcurrencyValue(undefined, 2, "--max-concurrent-renders") - ).toBe(2); - expect(normalizeConcurrencyValue("4", 2, "--max-concurrent-renders")).toBe( - 4 - ); - expect(normalizeConcurrencyValue(16, 2, "--max-concurrent-renders")).toBe( - 16 - ); + it("normalizes positive integer values", () => { + expect(normalizeConcurrencyValue(undefined, 2, "--lookback-days")).toBe(2); + expect(normalizeConcurrencyValue("4", 2, "--lookback-days")).toBe(4); + expect(normalizeConcurrencyValue(16, 2, "--lookback-days")).toBe(16); + expect(() => normalizeConcurrencyValue("0", 2, "--lookback-days")).toThrow( + "Must be a positive integer" + ); expect(() => - normalizeConcurrencyValue("0", 2, "--max-concurrent-renders") - ).toThrow("Must be a positive integer"); - expect(() => - normalizeConcurrencyValue("1.5", 2, "--max-concurrent-renders") + normalizeConcurrencyValue("1.5", 2, "--lookback-days") ).toThrow("Must be a positive integer"); + expect(() => normalizeConcurrencyValue("x", 2, "--lookback-days")).toThrow( + "Must be a positive integer" + ); + }); + + it("normalizes max render memory values", () => { + expect( + normalizeRenderMemoryMbValue(undefined, 4096, "--max-render-memory-mb") + ).toBe(4096); + expect( + normalizeRenderMemoryMbValue("8192", 4096, "--max-render-memory-mb") + ).toBe(8192); + expect(() => - normalizeConcurrencyValue("x", 2, "--max-concurrent-renders") - ).toThrow("Must be a positive integer"); + normalizeRenderMemoryMbValue("256", 4096, "--max-render-memory-mb") + ).toThrow("Must be at least 512 MiB"); }); - it("computes RAM-based default render concurrency", () => { - // 8 GiB available -> 4 GiB budget -> 8 render workers at 512MB each - expect(computeDefaultRenderConcurrency(8 * 1024 * 1024 * 1024)).toBe(8); - // Always at least one - expect(computeDefaultRenderConcurrency(0)).toBe(1); + it("computes RAM-based default render memory and capacity", () => { + // 8 GiB available -> 4 GiB budget + expect(computeDefaultRenderMemoryMb(8 * 1024 * 1024 * 1024)).toBe(4096); + expect(estimateRenderServiceCapacity(4096)).toBe(8); + // Always at least one 512 MiB renderer budget + expect(computeDefaultRenderMemoryMb(0)).toBe(512); expect(formatGiB(8 * 1024 * 1024 * 1024)).toBe("8.0"); }); }); diff --git a/tests/workspace.test.ts b/tests/workspace.test.ts index 068ccdf..664db7c 100644 --- a/tests/workspace.test.ts +++ b/tests/workspace.test.ts @@ -2,7 +2,7 @@ import { describe, expect, it } from "bun:test"; import { mkdtemp, readFile, rm } from "node:fs/promises"; import { tmpdir } from "node:os"; import { join } from "node:path"; -import { ensureVesaiDirectories } from "../config"; +import { ensureProjectDirectories } from "../config"; import { writeGroupMarkdown, writeSessionMarkdown, @@ -12,7 +12,7 @@ import { async function withTempHome(run: (homeDir: string) => Promise) { const homeDir = await mkdtemp(join(tmpdir(), "vesai-workspace-test-")); try { - await ensureVesaiDirectories(homeDir); + await ensureProjectDirectories(homeDir); await run(homeDir); } finally { await rm(homeDir, { recursive: true, force: true }); diff --git a/website/components/agent-cards.tsx b/website/components/agent-cards.tsx index f071fbf..d7500af 100644 --- a/website/components/agent-cards.tsx +++ b/website/components/agent-cards.tsx @@ -11,19 +11,19 @@ const cards = [ { title: "Durable Workspace Artifacts", description: - "Session, user, and query analyses persist as git-friendly markdown in ~/.vesai/workspace/ for long-lived agent context.", + "Session, user, group, and research artifacts persist as git-friendly markdown in .vesai/workspace/ for long-lived agent context.", icon: FolderOpen, }, { title: "Local-First, Self-Hosted", description: - "Your PostHog keys, your GCP project, your machine. No data leaves your infrastructure.", + "Global machine config stays in ~/.vesai while project credentials and artifacts stay in each repo's .vesai directory.", icon: Server, }, { title: "Ships with a SKILL.md", description: - "A comprehensive skill file teaches Claude Code, Codex, and other coding agents how to use every vesai command, workflow, and HogQL pattern out of the box.", + "A comprehensive skill file teaches Claude Code, Codex, and other coding agents how to run quickstart/init plus user, group, and research workflows out of the box.", icon: BookOpen, }, ]; diff --git a/website/components/getting-started.tsx b/website/components/getting-started.tsx index 6f30281..d1de9d4 100644 --- a/website/components/getting-started.tsx +++ b/website/components/getting-started.tsx @@ -17,13 +17,18 @@ const steps = [ }, { step: "2", - label: "Run quickstart", - code: "vesai quickstart", + label: "Set up global runtime", + code: "vesai quickstart --max-render-memory-mb 8192", }, { step: "3", - label: "Analyze your first session", - code: "vesai replays session ", + label: "Initialize this project", + code: "vesai init --lookback-days 180", + }, + { + step: "4", + label: "Run replay intelligence", + code: "vesai user ", }, ]; @@ -57,15 +62,20 @@ export function GettingStarted() {

Authenticate gcloud:

" + } />

- Three Steps + Four Steps

+

+ Every command auto-syncs VES AI to latest main before execution. +

    {steps.map((s) => (
  1. diff --git a/website/components/hero.tsx b/website/components/hero.tsx index 21cbdd4..ee1d02f 100644 --- a/website/components/hero.tsx +++ b/website/components/hero.tsx @@ -45,7 +45,7 @@ export function Hero() {
    - AI-ready product analytics CLI + Session replay intelligence CLI

    @@ -53,8 +53,11 @@ export function Hero() {

    - Connect PostHog. Render session recordings. Analyze with Gemini - vision. Output structured markdown your agents can act on. + Run global quickstart once, initialize each repo with{" "} + vesai init, then use + replay evidence to generate user stories, group stories, and + research answers your agents can execute. Commands auto-sync the CLI + to latest main before running.

    diff --git a/website/components/pipeline.tsx b/website/components/pipeline.tsx index fe2aea2..76a61fc 100644 --- a/website/components/pipeline.tsx +++ b/website/components/pipeline.tsx @@ -3,23 +3,23 @@ import { Section } from "./section"; const steps = [ { - name: "Capture", - detail: "Pull replays & events from PostHog", + name: "Quickstart", + detail: "Configure global runtime + render memory budget", icon: Download, }, { - name: "Render", - detail: "Replay sessions in Playwright", + name: "Init", + detail: "Create project-scoped .vesai config and workspace", icon: Play, }, { - name: "Analyze", - detail: "Extract insights with Gemini Vision", + name: "Heartbeat", + detail: "Daemon backfills and continuously analyzes new sessions", icon: BrainCircuit, }, { - name: "Act", - detail: "Output markdown agents can use", + name: "Ask", + detail: "Use user, group, and research commands for decisions", icon: Rocket, }, ]; @@ -31,8 +31,7 @@ export function Pipeline() { How It Works

    - From raw PostHog data to structured, agent-ready workspace artifacts in - four steps. + From machine setup to project intelligence artifacts in four steps.

    diff --git a/website/lib/terminal-content.ts b/website/lib/terminal-content.ts index 4ed701b..e3b0f2c 100644 --- a/website/lib/terminal-content.ts +++ b/website/lib/terminal-content.ts @@ -11,164 +11,165 @@ export interface TerminalTab { export const heroTerminalLines: TerminalLine[] = [ { type: "prompt", text: "curl -fsSL https://ves.ai/install | bash" }, { type: "output", text: "Installing VES AI..." }, - { type: "output", text: "Installed vesai to ~/.vesai/bin/vesai" }, + { type: "output", text: "Linked vesai to ~/.local/bin/vesai" }, { type: "blank", text: "" }, - { type: "prompt", text: "vesai quickstart" }, - { type: "output", text: "Connected to PostHog project: My App" }, - { type: "output", text: "GCS bucket: gs://vesai-renders ready" }, - { type: "output", text: "Quickstart complete." }, + { type: "prompt", text: "vesai quickstart --max-render-memory-mb 8192" }, + { type: "output", text: "Auto-updating VES AI from origin/main..." }, + { type: "output", text: "VES AI quickstart (global core setup)" }, + { + type: "output", + text: "Render budget set to 8192 MiB (dynamic scaling at 512 MiB/service)", + }, + { type: "output", text: "Global quickstart complete." }, { type: "blank", text: "" }, - { type: "prompt", text: "vesai replays session ph_abc123" }, - { type: "output", text: "Fetching session ph_abc123..." }, - { type: "output", text: "Rendering 47 events via Playwright..." }, - { type: "output", text: "Uploading 12 frames to GCS..." }, - { type: "output", text: "Analyzing with Gemini Vision..." }, + { type: "prompt", text: "vesai init --lookback-days 180" }, + { type: "output", text: "Created .vesai/project.json" }, + { type: "output", text: "Added .vesai/ to .gitignore" }, + { type: "output", text: "Project init complete." }, { type: "blank", text: "" }, - { type: "comment", text: '{ "session_id": "ph_abc123",' }, - { type: "comment", text: ' "score": 72,' }, - { type: "comment", text: ' "key_findings": [' }, - { type: "comment", text: ' "User rage-clicked checkout button 3x",' }, - { type: "comment", text: ' "Payment form validation blocked submit"' }, - { type: "comment", text: " ] }" }, + { type: "prompt", text: "vesai user bryce@company.com" }, + { type: "output", text: "Found 12 sessions for bryce@company.com" }, + { type: "output", text: "Rendering + analyzing sessions..." }, + { type: "blank", text: "" }, + { type: "comment", text: "{" }, + { type: "comment", text: ' "email": "bryce@company.com",' }, + { type: "comment", text: ' "sessionCount": 12,' }, + { type: "comment", text: ' "userScore": 78,' }, + { + type: "comment", + text: ' "markdownPath": ".vesai/workspace/users/bryce-company-com-bryce-company-com.md"', + }, + { type: "comment", text: "}" }, ]; export const terminalTabs: TerminalTab[] = [ { - label: "Session Analysis", + label: "Setup", lines: [ - { type: "prompt", text: "vesai replays session ph_abc123" }, - { type: "output", text: "Fetching session ph_abc123..." }, - { type: "output", text: "Rendering 47 events via Playwright..." }, - { type: "output", text: "Uploading 12 frames to GCS..." }, - { type: "output", text: "Analyzing with Gemini Vision..." }, + { type: "prompt", text: "vesai quickstart --max-render-memory-mb 8192" }, + { type: "output", text: "Auto-updating VES AI from origin/main..." }, + { type: "output", text: "VES AI quickstart (global core setup)" }, + { + type: "output", + text: "Configured core runtime at ~/.vesai/core.json", + }, + { + type: "output", + text: "Render budget: 8192 MiB (dynamic service scaling enabled)", + }, + { type: "output", text: "Global quickstart complete." }, { type: "blank", text: "" }, - { type: "comment", text: "{" }, - { type: "comment", text: ' "session_id": "ph_abc123",' }, - { type: "comment", text: ' "duration_seconds": 184,' }, - { type: "comment", text: ' "score": 72,' }, - { type: "comment", text: ' "severity": "medium",' }, - { type: "comment", text: ' "key_findings": [' }, - { type: "comment", text: ' "User rage-clicked checkout button 3x",' }, + { type: "prompt", text: "vesai init --lookback-days 180" }, + { type: "output", text: "VES AI init (project setup)" }, { - type: "comment", - text: ' "Payment form validation blocked submit",', + type: "output", + text: "Created .vesai/project.json with UUID projectId", }, - { type: "comment", text: ' "Session ended without conversion"' }, - { type: "comment", text: " ]," }, { - type: "comment", - text: ' "workspace_artifact": "~/.vesai/workspace/sessions/ph_abc123.md"', + type: "output", + text: "Created .vesai/workspace/{sessions,users,groups,research}", }, - { type: "comment", text: "}" }, + { type: "output", text: "Added .vesai/ to project .gitignore" }, + { type: "output", text: "Project init complete." }, ], }, { - label: "User Story", + label: "Daemon", lines: [ - { type: "prompt", text: "vesai replays user bryce@company.com" }, - { type: "output", text: "Found 8 sessions for bryce@company.com" }, + { type: "prompt", text: "vesai daemon start" }, + { type: "output", text: "Auto-updating VES AI from origin/main..." }, + { type: "output", text: "Daemon started in background (pid 49122)." }, { type: "output", - text: "Processing sessions... β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆ 8/8", + text: "Heartbeat queued 36 sessions (from 2025-08-20T00:00:00Z to now).", }, - { type: "blank", text: "" }, - { type: "comment", text: "{" }, - { type: "comment", text: ' "user": "bryce@company.com",' }, - { type: "comment", text: ' "sessions_analyzed": 8,' }, - { type: "comment", text: ' "date_range": "2025-01-10 to 2025-01-17",' }, { - type: "comment", - text: ' "summary": "Power user exploring advanced features.', + type: "output", + text: "Running session jobs, then rerunning impacted user/group stories.", }, + { type: "blank", text: "" }, + { type: "prompt", text: "vesai daemon status" }, + { type: "comment", text: '{ "running": true, "pid": 49122 }' }, + ], + }, + { + label: "User Story", + lines: [ + { type: "prompt", text: "vesai user bryce@company.com" }, + { type: "output", text: "Found 12 sessions for bryce@company.com" }, { - type: "comment", - text: " Encountered recurring friction in export flow.", + type: "output", + text: "Processing sessions... β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆ 12/12", }, + { type: "blank", text: "" }, + { type: "comment", text: "{" }, + { type: "comment", text: ' "email": "bryce@company.com",' }, + { type: "comment", text: ' "sessionCount": 12,' }, + { type: "comment", text: ' "averageSessionScore": 74,' }, + { type: "comment", text: ' "userScore": 78,' }, { type: "comment", - text: ' 3 of 8 sessions ended at the same error modal.",', + text: ' "health": "Stable with conversion friction at checkout",', }, - { type: "comment", text: ' "top_issues": [' }, - { type: "comment", text: ' "Export CSV timeout on large datasets",' }, - { type: "comment", text: ' "Filter reset after navigating back"' }, - { type: "comment", text: " ]," }, { type: "comment", - text: ' "workspace_artifact": "~/.vesai/workspace/users/bryce_company_com.md"', + text: ' "markdownPath": ".vesai/workspace/users/bryce-company-com-bryce-company-com.md"', }, { type: "comment", text: "}" }, ], }, { - label: "Analytics Query", + label: "Group Story", lines: [ - { - type: "prompt", - text: 'vesai insights sql "SELECT event, count() FROM events WHERE timestamp > now() - INTERVAL 7 DAY GROUP BY event ORDER BY count() DESC LIMIT 5"', - }, + { type: "prompt", text: "vesai group acme-co" }, + { type: "output", text: "Resolved 42 sessions across 9 users" }, + { type: "output", text: "Building user stories and group synthesis..." }, { type: "blank", text: "" }, { type: "comment", text: "{" }, - { type: "comment", text: ' "columns": ["event", "count"],' }, - { type: "comment", text: ' "results": [' }, - { type: "comment", text: ' ["$pageview", 48291],' }, - { type: "comment", text: ' ["$autocapture", 31044],' }, - { type: "comment", text: ' ["checkout_started", 8712],' }, - { type: "comment", text: ' ["item_added_to_cart", 6203],' }, - { type: "comment", text: ' ["payment_completed", 2847]' }, - { type: "comment", text: " ]" }, + { type: "comment", text: ' "groupId": "acme-co",' }, + { type: "comment", text: ' "usersAnalyzed": 9,' }, + { type: "comment", text: ' "score": 69,' }, + { + type: "comment", + text: ' "health": "Adoption is strong, onboarding friction remains for mobile users",', + }, + { + type: "comment", + text: ' "markdownPath": ".vesai/workspace/groups/acme-co-acme-co.md"', + }, { type: "comment", text: "}" }, ], }, { - label: "Query Replays", + label: "Research", lines: [ { type: "prompt", - text: 'vesai replays query "checkout" --url /checkout', + text: 'vesai research "What drives checkout abandonment?"', }, { type: "output", - text: "Querying replays matching: checkout (url=/checkout)", + text: "Selecting relevant analyzed sessions from .vesai/workspace/sessions", }, - { type: "output", text: "Found 14 matching sessions" }, { type: "output", - text: "Processing sessions... β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆ 14/14", + text: "Answering with Gemini using replay evidence...", }, - { type: "output", text: "Synthesizing cross-session analysis..." }, { type: "blank", text: "" }, { type: "comment", text: "{" }, - { type: "comment", text: ' "query": "checkout",' }, - { type: "comment", text: ' "sessions_analyzed": 14,' }, - { - type: "comment", - text: ' "synthesis": "Checkout flow has a 62% completion rate.', - }, - { - type: "comment", - text: " Primary drop-off occurs at the shipping address step.", - }, - { - type: "comment", - text: ' Mobile users are 3x more likely to abandon.",', - }, - { type: "comment", text: ' "common_patterns": [' }, - { - type: "comment", - text: ' "Address autocomplete fails on mobile Safari",', - }, { type: "comment", - text: ' "Shipping cost surprise causes back-navigation",', + text: ' "question": "What drives checkout abandonment?",', }, + { type: "comment", text: ' "confidence": "high",' }, + { type: "comment", text: ' "sessionsUsed": 12,' }, { type: "comment", - text: ' "Promo code field draws attention away from CTA"', + text: ' "supportingSessionIds": ["ph_112","ph_245","ph_318"],', }, - { type: "comment", text: " ]," }, { type: "comment", - text: ' "workspace_artifact": "~/.vesai/workspace/queries/checkout.md"', + text: ' "findings": ["Validation errors on postal code", "Shipping fee surprise", "Promo code distraction"]', }, { type: "comment", text: "}" }, ], diff --git a/workspace/index.ts b/workspace/index.ts index 6956614..3f0205a 100644 --- a/workspace/index.ts +++ b/workspace/index.ts @@ -1,6 +1,6 @@ import { mkdir, writeFile } from "node:fs/promises"; import { join } from "node:path"; -import { getVesaiPaths, resolveVesaiHome } from "../config"; +import { getVesaiPaths } from "../config"; export type SessionMarkdown = { id: string; @@ -39,7 +39,7 @@ async function writeMarkdown(params: { body: string; homeDir?: string; }): Promise { - const paths = getVesaiPaths(params.homeDir ?? resolveVesaiHome()); + const paths = getVesaiPaths(params.homeDir); const targetDir = join(paths.workspace, params.folder); await mkdir(targetDir, { recursive: true });