Skip to content
Merged
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -55,6 +55,7 @@ next-env.d.ts
.claude/settings.local.json
.claude/worktrees/
.claude/review-loop.log
.claude/scheduled_tasks.lock

# supabase cli runtime cache (machine-local)
supabase/.temp/
Expand Down
129 changes: 129 additions & 0 deletions .mission/plan.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,129 @@
# Mission: improve Straude activation

**Started:** 2026-05-04
**Goal:** Drive CLI activation (% of users who push at least once) from ~47% toward ~75% by removing the four error classes surfaced in PostHog, plus add tracking, plus a scheduled 7-day check-in.

## Context

PostHog analysis (last 7 days, 103 CLI users):

- 53% of users (55) install Straude and never push once
- `$exception` is the #1 event with 1,723 occurrences across 73 users
- Three errors explain almost all failures:
- `ccusage is not installed or not on PATH` — 1,627 events / 14 users
- `write EPIPE` — 48 events / 48 users (every active user once)
- `Session expired or invalid` — 33 events / 9 users
- `Date X is outside the 30-day backfill window` — 13 events
- Among activated users, retention is strong: 69% pushed 2+ days, 17% pushed 6 of 7 days

## Out of scope

- Re-engaging the 55 stuck users (deferred)
- Any non-CLI / non-web code
- New features unrelated to activation

## Verification commands

- `bun run typecheck` (monorepo)
- `bun run test` (monorepo)
- `bun run build` (monorepo)
- `bun run --cwd packages/cli test` (CLI vitest)

## Milestones

### 1. CLI resilience + activation tracking — M (~25–35 min)

**Goal:** Stop swallowing EPIPE on every active user; add events needed to measure activation.

**Changes:**
- `packages/cli/src/index.ts` `main()`: register `process.stdout.on('error', …)` and stderr handlers — exit 0 on EPIPE.
- `packages/cli/src/lib/auth.ts`: add `installed_at` to `StraudeConfig`. If config doesn't exist yet at first run, write a separate marker (`~/.straude/.first-run`) so we don't lose the signal pre-login.
- `packages/cli/src/index.ts`: capture `cli_first_run` once per machine (gated by marker). Capture `cli_authenticated` on every invocation where a valid config loads.
- Tests in `packages/cli/__tests__/` for first-run gating and EPIPE handler.

**Acceptance:**
- `bun run --cwd packages/cli test` passes.
- Manual: `node packages/cli/dist/index.js --help | head -1` exits 0.
- Manual: deleting `~/.straude/` and running CLI captures exactly one `cli_first_run`; subsequent runs capture zero additional first-run events.

### 2. ccusage detect & install on first run — M (~30–45 min)

**Goal:** Replace the hard throw at `ccusage.ts:49-51` with a one-time interactive install prompt.

**Changes:**
- `packages/cli/src/lib/ccusage.ts`: when `isOnPath('ccusage')` is false, prompt user (TTY-only). On consent, run `npm install -g ccusage` (prefer `bun add -g` if bun present). Capture `ccusage_install_attempted` and `ccusage_install_succeeded`/`_failed`.
- Non-TTY (auto-push, CI): keep current throw with improved message.
- Tests for TTY/non-TTY branches, install success/failure, declined prompt.

**Acceptance:**
- `bun run --cwd packages/cli test` passes.
- Manual on a system without ccusage: `npx straude@latest` prompts; accepting installs ccusage; push proceeds.
- Manual non-TTY: `npx straude@latest push < /dev/null` throws cleanly with the install command.

### 3. Filter out-of-window backfill dates client-side — S (~10–15 min)

**Goal:** Stop submitting dates the server will reject.

**Changes:**
- `packages/cli/src/commands/push.ts`: after merging entries (~line 342), filter out entries older than `MAX_BACKFILL_DAYS`. If any dropped, log a single warning listing the dates. Do NOT throw.

**Acceptance:**
- New unit test: 60-day input → only last 30 days reach submit body.
- Existing push tests still pass.

### 4. Silent re-auth on 401 + sliding token refresh — L (~60–90 min)

**Goal:** Eliminate "Session expired" failures.

**Changes:**
- **Server** (`apps/web/app/api/usage/submit/route.ts` and `/api/cli/dashboard`): if JWT is older than 7 days, mint a fresh one via `createCliToken` and set `X-Straude-Refreshed-Token` response header.
- **Server helper** (`apps/web/lib/api/cli-auth.ts`): export `tokenAgeDays(token)`.
- **Client** (`packages/cli/src/lib/api.ts`):
- On every successful response, read `X-Straude-Refreshed-Token`; persist via `saveConfig` and update in-memory config.
- On 401: run `loginCommand(config.api_url)`, reload config, retry the request once. If retry also 401s, throw existing message.
- Skip auto-relogin when stdin is non-TTY OR `process.env.STRAUDE_AUTO === '1'`.
- Tests: refresh-header path, 401-retry path, 401-then-401-throws path.

**Acceptance:**
- `bun run --cwd packages/cli test` and any web tests pass.
- `bun run typecheck` passes.
- Manual: token with `iat` >7 days old gets refreshed in `~/.straude/config.json` after a push.
- Manual: invalidated token triggers browser login automatically and push completes.

### 5. Schedule weekly activation check-in via PostHog — S (~15–20 min)

**Goal:** Auto-deliver an activation snapshot 7 days post-merge.

**Changes (PostHog only, via MCP):**
- Create a saved insight: activation funnel `cli_first_run` → `usage_pushed` (filtered to `cli_version >= 0.1.24`).
- Create a weekly subscription delivered to `oscar.hong7@gmail.com`.
- Test delivery via `subscriptions-test-delivery-create`.
- Note the insight URL + baseline values in the PR description.

**Acceptance:**
- Insight URL included in PR description.
- Test delivery succeeds.

### 6. Integration, verification & PR — S (~15–20 min)

**Goal:** Full sweep + open the PR.

**Changes:**
- Run from repo root: `bun run typecheck && bun run test && bun run build`.
- Smoke-test: full login → push → simulate 401 → confirm auto-reauth.
- Update `docs/CHANGELOG.md` under `## Unreleased` (Added/Changed/Fixed).
- Update `docs/DECISIONS.md`: (a) ccusage auto-install reverses prior security stance — note trade-off; (b) sliding token refresh design.
- Bump `packages/cli/package.json` version → `0.1.24`.
- Open PR titled `fix(cli): unblock activation — ccusage install, EPIPE, silent reauth, backfill filter`.

**Acceptance:**
- All three commands above exit 0.
- PR open with green CI.
- PR description includes activation baseline + scheduled-insight link + before/after summary per error class.

## Risk register

- **R1**: `npm install -g ccusage` requires sudo on some setups. → On EACCES, fall through to manual instruction. Don't escalate.
- **R2**: Server-side token refresh changes auth contract. → Header-based refresh is purely additive; older clients ignore it.
- **R3**: Auto-relogin opens browser unexpectedly during `--auto` background push. → Skip when stdin is non-TTY OR `STRAUDE_AUTO=1`.
- **R4**: PostHog scheduled subscription requires email integration. → Confirm via MCP; if unavailable, fall back to a notebook + manual reminder.
23 changes: 23 additions & 0 deletions .mission/progress.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
# Mission Progress

**Mission:** improve Straude activation
**Started:** 2026-05-04
**Status:** Complete. PR https://github.com/ohong/straude/pull/114 opened 2026-05-04.

## Follow-ups

- 7-day check-in scheduled (one-shot cron in this Claude session, plus the manual notebook): query insight `DV22QC1d` on 2026-05-11 and confirm activation rate moved from the 47% baseline.
- Re-engage the 55 install-but-never-pushed users once the PR ships and the new CLI propagates via npm.
- Consider relaxing or quarantining the flaky `authenticated-100ms.test.tsx > messages optimistic send` perf budget — flaked at 1018ms / 1068ms under concurrent monorepo load.

## Milestones

- [x] Milestone 1: CLI resilience + activation tracking
- [x] Milestone 2: ccusage detect & install on first run
- [x] Milestone 3: Filter out-of-window backfill dates client-side
- [x] Milestone 4: Silent re-auth on 401 + sliding token refresh
- [x] Milestone 5: Schedule weekly activation check-in via PostHog
- Insight: https://us.posthog.com/project/374497/insights/DV22QC1d
- Notebook: https://us.posthog.com/project/374497/notebooks/QQ7eCe7G
- PostHog Subscriptions are paid tier — using a notebook + manual reminder as the no-cost equivalent.
- [x] Milestone 6: Integration, verification & PR
9 changes: 8 additions & 1 deletion apps/web/__tests__/api/usage-submit.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@ vi.mock("@/lib/supabase/server", () => ({

vi.mock("@/lib/api/cli-auth", () => ({
verifyCliToken: vi.fn(),
verifyCliTokenWithRefresh: vi.fn(),
}));

vi.mock("@/lib/supabase/service", () => ({
Expand All @@ -18,7 +19,7 @@ vi.mock("@supabase/supabase-js", () => ({

import { POST, aggregateDeviceRows } from "@/app/api/usage/submit/route";
import { createClient } from "@/lib/supabase/server";
import { verifyCliToken } from "@/lib/api/cli-auth";
import { verifyCliToken, verifyCliTokenWithRefresh } from "@/lib/api/cli-auth";
import { getServiceClient } from "@/lib/supabase/service";

function makeEntry(dateStr: string, overrides: Record<string, any> = {}) {
Expand Down Expand Up @@ -124,6 +125,12 @@ beforeEach(() => {
process.env.SUPABASE_SECRET_KEY = "secret";
process.env.NEXT_PUBLIC_APP_URL = "https://straude.com";
(verifyCliToken as any).mockReturnValue(null);
// Auto-derive verifyCliTokenWithRefresh from verifyCliToken so existing
// tests can keep setting `verifyCliToken.mockReturnValue("cli-user-id")`.
(verifyCliTokenWithRefresh as any).mockImplementation((header: string | null) => {
const userId = (verifyCliToken as any)(header);
return userId ? { userId, username: null, refreshedToken: null } : null;
});
});

describe("POST /api/usage/submit", () => {
Expand Down
18 changes: 17 additions & 1 deletion apps/web/__tests__/flows/cli-push-flow.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,7 @@ vi.mock("@/lib/supabase/server", () => ({
vi.mock("@/lib/api/cli-auth", () => ({
createCliToken: vi.fn(() => "mock-cli-jwt-token"),
verifyCliToken: vi.fn(),
verifyCliTokenWithRefresh: vi.fn(),
}));

const mockServiceClient = {
Expand All @@ -26,7 +27,21 @@ vi.mock("@/lib/supabase/service", () => ({
getServiceClient: vi.fn(() => mockServiceClient),
}));

import { verifyCliToken } from "@/lib/api/cli-auth";
import { verifyCliToken, verifyCliTokenWithRefresh } from "@/lib/api/cli-auth";

/**
* Auto-derive verifyCliTokenWithRefresh from verifyCliToken so existing
* tests that set verifyCliToken's return value continue to work. Must be
* called after vi.clearAllMocks() in each beforeEach.
*/
function autoDeriveCliAuthMocks() {
(verifyCliTokenWithRefresh as ReturnType<typeof vi.fn>).mockImplementation(
(header: string | null) => {
const userId = (verifyCliToken as ReturnType<typeof vi.fn>)(header);
return userId ? { userId, username: null, refreshedToken: null } : null;
},
);
}

// ---------------------------------------------------------------------------
// Helpers
Expand Down Expand Up @@ -65,6 +80,7 @@ describe("Flow: CLI Push", () => {
beforeEach(() => {
vi.useFakeTimers({ now: new Date('2026-03-13T12:00:00Z'), toFake: ['Date'] });
vi.clearAllMocks();
autoDeriveCliAuthMocks();
mockServiceClient.rpc.mockResolvedValue({ data: null, error: null });
vi.stubEnv("NEXT_PUBLIC_APP_URL", "https://straude.com");
vi.stubEnv("NEXT_PUBLIC_SUPABASE_URL", "https://test.supabase.co");
Expand Down
1 change: 1 addition & 0 deletions apps/web/__tests__/flows/web-import-flow.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,7 @@ vi.mock("@/lib/supabase/server", () => ({

vi.mock("@/lib/api/cli-auth", () => ({
verifyCliToken: vi.fn(() => null), // web flow — no CLI token
verifyCliTokenWithRefresh: vi.fn(() => null),
}));

vi.mock("@/lib/supabase/service", () => ({
Expand Down
45 changes: 44 additions & 1 deletion apps/web/__tests__/unit/cli-auth.test.ts
Original file line number Diff line number Diff line change
@@ -1,5 +1,10 @@
import { describe, it, expect, beforeEach, afterEach, vi } from "vitest";
import { createCliToken, verifyCliToken } from "@/lib/api/cli-auth";
import {
createCliToken,
verifyCliToken,
verifyCliTokenWithRefresh,
TOKEN_REFRESH_AFTER_DAYS,
} from "@/lib/api/cli-auth";

const TEST_SECRET = "test-secret-key-for-jwt";

Expand Down Expand Up @@ -140,4 +145,42 @@ describe("cli-auth", () => {
expect(verifyCliToken(`Bearer ${token}`)).toBeNull();
});
});

describe("verifyCliTokenWithRefresh", () => {
it("returns userId, username, and no refresh on a fresh token", () => {
const token = createCliToken("user-1", "alice");
const result = verifyCliTokenWithRefresh(`Bearer ${token}`);
expect(result).not.toBeNull();
expect(result!.userId).toBe("user-1");
expect(result!.username).toBe("alice");
expect(result!.refreshedToken).toBeNull();
});

it("emits a refreshed token once iat is older than the threshold", () => {
// Mint a token "now" then jump the clock past the refresh threshold.
const token = createCliToken("user-1", "alice");
vi.setSystemTime(
new Date(Date.now() + (TOKEN_REFRESH_AFTER_DAYS + 1) * 24 * 60 * 60 * 1000),
);
const result = verifyCliTokenWithRefresh(`Bearer ${token}`);
expect(result).not.toBeNull();
expect(result!.refreshedToken).toBeTypeOf("string");
expect(result!.refreshedToken!.split(".")).toHaveLength(3);
expect(result!.refreshedToken).not.toBe(token);
});

it("does not refresh if the token is just barely under the threshold", () => {
const token = createCliToken("user-1", "alice");
vi.setSystemTime(
new Date(Date.now() + (TOKEN_REFRESH_AFTER_DAYS - 1) * 24 * 60 * 60 * 1000),
);
const result = verifyCliTokenWithRefresh(`Bearer ${token}`);
expect(result!.refreshedToken).toBeNull();
});

it("returns null for invalid tokens", () => {
expect(verifyCliTokenWithRefresh("Bearer not.a.token")).toBeNull();
expect(verifyCliTokenWithRefresh(null)).toBeNull();
});
});
});
37 changes: 23 additions & 14 deletions apps/web/app/api/cli/dashboard/route.ts
Original file line number Diff line number Diff line change
@@ -1,14 +1,15 @@
import { NextResponse } from "next/server";
import { verifyCliToken } from "@/lib/api/cli-auth";
import { verifyCliTokenWithRefresh } from "@/lib/api/cli-auth";
import { getServiceClient } from "@/lib/supabase/service";

export async function GET(request: Request) {
// Auth: verify CLI JWT from Authorization header
const authHeader = request.headers.get("authorization");
const userId = verifyCliToken(authHeader);
if (!userId) {
const auth = verifyCliTokenWithRefresh(authHeader);
if (!auth) {
return NextResponse.json({ error: "Unauthorized" }, { status: 401 });
}
const userId = auth.userId;

const db = getServiceClient();

Expand Down Expand Up @@ -179,15 +180,23 @@ export async function GET(request: Request) {
leaderboard = { rank, total_users: totalUsers, above, below };
}

return NextResponse.json({
username: profile.username,
level: levelRow ? Number(levelRow.level) : null,
streak,
daily,
week_cost,
prev_week_cost,
leaderboard,
model_breakdown,
total_output_tokens,
});
const headers: Record<string, string> = {};
if (auth.refreshedToken) {
headers["X-Straude-Refreshed-Token"] = auth.refreshedToken;
}

return NextResponse.json(
{
username: profile.username,
level: levelRow ? Number(levelRow.level) : null,
streak,
daily,
week_cost,
prev_week_cost,
leaderboard,
model_breakdown,
total_output_tokens,
},
{ headers },
);
}
23 changes: 18 additions & 5 deletions apps/web/app/api/usage/submit/route.ts
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
import { NextResponse } from "next/server";
import { createClient } from "@/lib/supabase/server";
import { verifyCliToken } from "@/lib/api/cli-auth";
import { verifyCliTokenWithRefresh } from "@/lib/api/cli-auth";
import { getServiceClient } from "@/lib/supabase/service";
import { checkAndAwardAchievements } from "@/lib/achievements";
import { rateLimit } from "@/lib/rate-limit";
Expand Down Expand Up @@ -51,13 +51,21 @@ function hasNonCodexModels(models: unknown): boolean {
interface AuthContext {
userId: string;
source: "cli" | "web";
/** When set, the response should include X-Straude-Refreshed-Token. */
refreshedToken?: string | null;
}

async function resolveAuthContext(request: Request): Promise<AuthContext | null> {
// Try CLI JWT first
const authHeader = request.headers.get("authorization");
const cliUserId = verifyCliToken(authHeader);
if (cliUserId) return { userId: cliUserId, source: "cli" };
const cliAuth = verifyCliTokenWithRefresh(authHeader);
if (cliAuth) {
return {
userId: cliAuth.userId,
source: "cli",
refreshedToken: cliAuth.refreshedToken,
};
}

// Fall back to Supabase session (web)
try {
Expand Down Expand Up @@ -482,9 +490,14 @@ export async function POST(request: Request) {
})
.catch(() => {});

const responseHeaders: Record<string, string> = {};
if (auth.source === "cli" && auth.refreshedToken) {
responseHeaders["X-Straude-Refreshed-Token"] = auth.refreshedToken;
}

const response: UsageSubmitResponse = { results };
if (errors.length > 0) {
return NextResponse.json({ ...response, errors }, { status: 207 });
return NextResponse.json({ ...response, errors }, { status: 207, headers: responseHeaders });
}
return NextResponse.json(response);
return NextResponse.json(response, { headers: responseHeaders });
}
Loading
Loading