diff --git a/.DS_Store b/.DS_Store index 97ed237..c60551d 100644 Binary files a/.DS_Store and b/.DS_Store differ diff --git a/.env.local.example b/.env.local.example new file mode 100644 index 0000000..493958e --- /dev/null +++ b/.env.local.example @@ -0,0 +1,40 @@ +NEXT_PUBLIC_CONVEX_URL=https://your-project.convex.cloud +NEXT_PUBLIC_CLERK_PUBLISHABLE_KEY=pk_test_xxx +CLERK_SECRET_KEY=sk_test_xxx +NEXT_PUBLIC_CLERK_SIGN_IN_URL=/sign-in +NEXT_PUBLIC_CLERK_SIGN_UP_URL=/sign-up +CLERK_WEBHOOK_SECRET=whsec_xxx +ENCRYPTION_KEY= +EMBEDDED_TICKETMASTER_KEY= +EMBEDDED_LASTFM_KEY= +EMBEDDED_DISCOGS_KEY= +EMBEDDED_DISCOGS_SECRET= + +# Stripe billing +STRIPE_SECRET_KEY=sk_test_xxx +NEXT_PUBLIC_STRIPE_PUBLISHABLE_KEY=pk_test_xxx +STRIPE_WEBHOOK_SECRET=whsec_xxx +STRIPE_PRO_MONTHLY_PRICE_ID=price_xxx +STRIPE_PRO_ANNUAL_PRICE_ID=price_xxx +STRIPE_TEAM_MONTHLY_PRICE_ID=price_xxx + +# Platform key — Crate's own Anthropic key for free/pro users without BYOK +PLATFORM_ANTHROPIC_KEY=sk-ant-xxx + +# Admin emails (comma-separated) — bypass all limits and feature gates +ADMIN_EMAILS=admin@example.com + +# Auth0 Token Vault (for OAuth connections to Spotify, Slack, Google) +AUTH0_DOMAIN=your-tenant.us.auth0.com +AUTH0_CLIENT_ID=your-auth0-client-id +AUTH0_CLIENT_SECRET=your-auth0-client-secret +AUTH0_TOKEN_VAULT_AUDIENCE=https://your-api-audience +AUTH0_CALLBACK_URL=http://localhost:3000/api/auth0/callback + +# Beta domains (comma-separated) — users from these domains get Pro access for free +# Use for observation sprint testers, radio station partners, etc. +BETA_DOMAINS=radiomilwaukee.org + +# Canny feedback widget +NEXT_PUBLIC_CANNY_APP_ID=your-canny-app-id +NEXT_PUBLIC_CANNY_URL=https://your-company.canny.io diff --git a/.gitignore b/.gitignore index 5ef6a52..f85f5da 100644 --- a/.gitignore +++ b/.gitignore @@ -32,6 +32,7 @@ yarn-error.log* # env files (can opt-in for committing if needed) .env* +!.env.local.example # vercel .vercel @@ -39,3 +40,6 @@ yarn-error.log* # typescript *.tsbuildinfo next-env.d.ts +.superpowers/ +.gstack/ +.env.local diff --git a/DEVLOG.md b/DEVLOG.md new file mode 100644 index 0000000..74cafbe --- /dev/null +++ b/DEVLOG.md @@ -0,0 +1,126 @@ +# Crate Web — Devlog + +## 2026-03-12: Migration from Claude Agent SDK to Direct Anthropic SDK + +### The Problem + +Crate's backend uses the **Claude Agent SDK** (`@anthropic-ai/claude-agent-sdk`) to power its music research agent. The Agent SDK works by spawning **Claude Code as a subprocess** via the `query()` function. This architecture is fundamentally incompatible with: + +1. **Vercel serverless functions** — Serverless environments don't support long-running subprocesses. Functions have execution time limits (60s default) and no persistent process state. +2. **Railway containers** — Even with a full Docker container running Node.js, the Claude Code subprocess fails with `exit code 1`. The CLI requires a specific environment setup (authentication, session management) that doesn't work in headless container environments. +3. **Any standard deployment** — The Agent SDK is designed for local CLI use where Claude Code is installed and authenticated interactively. + +### What We Tried + +#### Attempt 1: Deploy directly to Vercel +- **Result**: `Error: Claude Code process exited with code 1` +- The Agent SDK's `query()` spawns a Claude Code subprocess. Vercel's serverless runtime cannot run persistent subprocesses. + +#### Attempt 2: Vercel Sandbox +- Vercel offers a "Sandbox" product specifically for the Claude Agent SDK, but it requires enterprise access and has different constraints. + +#### Attempt 3: Railway long-running server +- Built a separate Express server deployed to Railway at `https://crate-agent-production.up.railway.app` +- Architecture: Vercel signs a JWT containing the user's API keys → browser sends JWT to Railway → Railway runs CrateAgent → streams SSE back +- Added `git` and `@anthropic-ai/claude-code` to the Docker image +- **Result**: Same `exit code 1`. The Claude Code CLI subprocess fails even in a Docker container with the CLI installed. + +#### Attempt 4: JWT size issues (431 Request Header Fields Too Large) +- The JWT initially contained the full prompt suffix (~15KB). Moved prompt construction server-side to shrink the token. +- Also discovered trailing `\n` in Vercel env vars (from `echo "value" | npx vercel env add`) causing JWT signature mismatches. Fixed with `printf '%s'`. + +### The Solution: Direct Anthropic SDK + +Replace the Agent SDK entirely with the **direct Anthropic SDK** (`@anthropic-ai/sdk`). Instead of spawning a subprocess, we: + +1. **Convert tool definitions**: The MCP tools in `crate-cli` use `tool()` from the Agent SDK with Zod schemas. We convert these Zod schemas to JSON Schema format for the Anthropic Messages API's `tools` parameter. +2. **Build a manual agentic loop**: Send user message → if Claude responds with `tool_use` blocks, execute the tool handlers directly in-process → send results back → repeat until Claude responds with `end_turn`. +3. **Stream everything**: Use the Anthropic SDK's streaming mode to emit `CrateEvent`s (answer_token, tool_start, tool_end, done, error) as SSE to the frontend. + +### Why This Works + +- **No subprocess**: The Anthropic SDK makes HTTP calls to the Messages API. No CLI, no child processes, no special environment. +- **Vercel-compatible**: Standard HTTP requests fit perfectly in serverless functions. The streaming response keeps the connection alive within the 60s limit (configurable up to 300s on Pro plans). +- **OpenRouter-compatible**: The Anthropic SDK accepts a `baseURL` parameter. Setting it to `https://openrouter.ai/api/v1` routes through OpenRouter with zero code changes. +- **Tools run in-process**: The MCP tool handlers are plain async functions that make HTTP calls to external APIs (MusicBrainz, Discogs, etc.). They don't need a subprocess — they run directly in the Node.js runtime. + +### Architecture Before vs After + +**Before (Agent SDK + Railway):** +``` +Browser → /api/agent-token (Vercel, signs JWT) + → Railway /agent/research (runs CrateAgent subprocess) + ← SSE stream back to browser +``` + +**After (Direct Anthropic SDK):** +``` +Browser → /api/chat (Vercel) + → Anthropic Messages API (with tools) + → Tool handlers run in-process + ← SSE stream back to browser +``` + +### What Gets Deleted + +- `server/` directory (entire Railway Express server) +- `src/app/api/agent-token/` (JWT signing endpoint) +- `src/lib/agent-token.ts` (JWT signing utility) +- `.vercelignore` (was excluding server/ from Vercel) +- `AGENT_SIGNING_SECRET` and `RAILWAY_AGENT_URL` env vars +- `jose` dependency (JWT library) + +### What Gets Created + +- `src/lib/tool-adapter.ts` — Converts `SdkMcpToolDefinition` (Zod schemas + handlers) to Anthropic API tool format (JSON Schema) and provides a tool executor +- `src/lib/agentic-loop.ts` — The manual agentic loop: message → tool_use → execute → loop, emitting CrateEvents as an async generator +- Rewritten `src/app/api/chat/route.ts` — Handles ALL tiers (chat + agent) using the direct SDK + +### OpenRouter Support + +The direct Anthropic SDK supports custom base URLs: + +```typescript +import Anthropic from "@anthropic-ai/sdk"; + +// Direct Anthropic +const client = new Anthropic({ apiKey: anthropicKey }); + +// Via OpenRouter — same API, different endpoint +const client = new Anthropic({ + apiKey: openRouterKey, + baseURL: "https://openrouter.ai/api/v1", +}); +``` + +All tool_use, streaming, and the agentic loop work identically through OpenRouter since it's wire-compatible with the Anthropic Messages API. + +### Key Decisions + +1. **Keep `crate-cli` as a dependency** — We import `getActiveTools()` for tool definitions, `getSystemPrompt()` for the system prompt, and `classifyQuery()` for tier routing. We just skip the `CrateAgent` class and its `query()` calls. +2. **Tools run in the API route process** — The tool handlers are async functions that make HTTP calls. They run directly in the Vercel serverless function, no subprocess needed. +3. **Set env vars before tool resolution** — `getActiveTools()` checks `process.env` for API keys to determine which servers are active. We set env vars from the user's resolved keys before calling it, and restore them after. +4. **SSE format stays identical** — The frontend already parses `CrateEvent` SSE. The new backend emits the exact same event types. +5. **Increase maxDuration** — Research queries can involve 10-25 tool calls. We increase the Vercel function timeout from 60s to 300s. +6. **Zod 4 native JSON Schema** — crate-cli uses Zod 4 which has built-in `z.toJSONSchema()`. No need for `zod-to-json-schema` third-party lib. +7. **Non-streaming for tool turns** — The agentic loop uses non-streaming API calls during tool-use turns (where latency is dominated by tool execution), and chunks text for SSE delivery. + +### Implementation Status + +**Completed:** +- `src/lib/tool-adapter.ts` — Converts `SdkMcpToolDefinition` Zod schemas to Anthropic API JSON Schema format, provides tool executor +- `src/lib/agentic-loop.ts` — Manual agentic loop with CrateEvent emission +- `src/lib/openui-prompt.ts` — Copied from server/src/lib/ (OpenUI Lang system prompt) +- `src/app/api/chat/route.ts` — Rewritten to handle both chat-tier (fast) and agent-tier (agentic loop with tools) +- `src/components/workspace/chat-panel.tsx` — Simplified from two-step agent-token flow to single `/api/chat` endpoint +- Build passing (TypeScript clean, Next.js build clean) + +**Ready to delete (after testing):** +- `server/` directory (Railway Express server) +- `src/app/api/agent-token/route.ts` (JWT signing endpoint) +- `src/lib/agent-token.ts` (JWT signing utility) +- `src/hooks/use-crate-agent.ts` (unused hook) +- `.vercelignore` (was excluding server/) +- Railway deployment at `https://crate-agent-production.up.railway.app` +- Vercel env vars: `AGENT_SIGNING_SECRET`, `RAILWAY_AGENT_URL` +- `jose` dependency (JWT library, check if used elsewhere first) diff --git a/README.md b/README.md index e215bc4..ff475ea 100644 --- a/README.md +++ b/README.md @@ -1,36 +1,411 @@ -This is a [Next.js](https://nextjs.org) project bootstrapped with [`create-next-app`](https://nextjs.org/docs/app/api-reference/cli/create-next-app). +

+ + Crate + +

-## Getting Started +

+ AI-powered music research for DJs, producers, and crate diggers.
+ 20+ sources. One agent. Zero tabs. +

-First, run the development server: +

+ Live App · + Help · + CLI · + Pricing +

+ +

+ Crate — DIG DEEPER. +

+ +--- + +Crate is an AI music research workspace where an agent researches across Discogs, MusicBrainz, Last.fm, Genius, Bandcamp, WhoSampled, Wikipedia, Ticketmaster, Spotify, and more. Ask about any artist, track, sample, or genre and get back interactive components — not just text. Connect your Spotify, Slack, and Google accounts to act on what you find. + +Built for the [Auth0 "Authorized to Act" Hackathon](https://auth0.devpost.com/). + +## For Judges: Try It Live + +The fastest way to evaluate Crate is on the live app — no install needed. + +### Option A: Use the live app (recommended) + +1. Go to **[digcrate.app](https://digcrate.app)** +2. Click **Get Started** and sign up (email or Google — free, no credit card) +3. Try these commands in the chat: + - `/influence Flying Lotus` — traces influence connections across decades + - `/artist MF DOOM` — full artist deep dive with discography and connections + - `/spotify` — browse your Spotify playlists (requires connecting Spotify in Settings) + - `/tumblr #jazz` — discover jazz posts across Tumblr + - `/news` — daily music news segment +4. To see Token Vault in action: go to **Settings** (gear icon) → **Connected Services** → connect Spotify, Tumblr, Slack, or Google + +### Option B: Run locally + +```bash +git clone https://github.com/tmoody1973/crate-web.git +cd crate-web +npm install +cp .env.local.example .env.local +# Fill in required env vars (see Environment Variables below) +npx convex dev # Terminal 1 +npm run dev # Terminal 2 +``` + +Open [localhost:3000](http://localhost:3000). You'll need Clerk, Convex, and at minimum a `PLATFORM_ANTHROPIC_KEY` or your own Anthropic API key to use the agent. + +### Demo video + +[Watch the 3-minute demo](https://youtu.be/3hmwKvWDYJ4) showing Crate connecting to Spotify, Tumblr, Slack, and Google Docs through Auth0 Token Vault. + +--- + +## Features + +### Core research +- **Agentic research** — Claude-powered agent with tool-use across 20+ MCP data sources +- **Influence mapping** — `/influence [artist]` traces connections through review co-mentions, Last.fm similarity, and enriches via Perplexity with pull quotes, sonic elements, and key works +- **Show prep** — `/prep [station]: [setlist]` generates track context, talk breaks, social copy, and interview prep for radio DJs +- **Music news** — `/news [station] [count]` generates daily music news segments from RSS feeds and web search +- **Story cards** — `/story [topic]` creates rich narrative deep dives with chapters, YouTube embeds, key tracks, and people cards. Works for albums, artists, genres, labels, and events +- **Track deep dive** — `/track [song] [artist]` shows full credits (MusicBrainz + Discogs), samples (WhoSampled), lyrics (Genius), and vinyl pressings (Discogs) in a tabbed view. Crate's SongDNA competitor. +- **Artist profile** — `/artist [name]` renders a full artist deep dive with tabbed discography (playable albums), influence connections (tappable chips), media (YouTube + external links), and top tracks + +### Tiny Desk DNA +- **626 Tiny Desk concerts** — Browse NPR Tiny Desk performances (2021-2025) with genre filtering, timeline view, and random discovery +- **Before/after hero** — Split-screen showing what NPR gives you (a video) vs. what Crate reveals (cited influence chain with sources) +- **Influence chain companion pages** — Full musical DNA pages with YouTube videos, pull quotes, sonic DNA tags, and cited journalism for each influence connection +- **Save as companion** — Run `/influence [artist]` in Crate, click "Save as Tiny Desk Companion" to generate a companion page with server-side YouTube + genre enrichment +- **`/tinydesk` command** — Searchable artist picker inside the workspace with genre filters and surprise button +- **Community catalog** — Companion pages created by users automatically appear in the public catalog with "EXPLORE DNA" badges +- **Live at [digcrate.app/tinydesk](https://digcrate.app/tinydesk)** + +### Connected services (Auth0 Token Vault) +- **Spotify** — Read your library, playlists, and top artists. Create new playlists from research. Export influence chains as playlists. +- **Tumblr** — Publish research to your blog. Discover music by tag across all of Tumblr. Blog selection for multi-blog accounts. +- **Slack** — Send research to any channel or DM with Block Kit formatting (headers, bullet lists, tables, dividers). Channel picker UI. +- **Google Docs** — Save research as shareable Google Docs + +### Dynamic UI (OpenUI) +- **27+ interactive components** — Agent generates artist profiles, track deep dives, album grids, track lists with play buttons, influence chains with hero banners, show prep packages, story cards, Spotify playlist viewers, Slack channel pickers, and more at runtime +- **Deep Cuts panel** — Resizable split panel (desktop) or full-screen view (mobile) for viewing saved research. Dropdown selector with type-colored dots, publish to shareable links +- **Action buttons on every component** — Export to Spotify, Send to Slack, Publish, Deep Dive, Influence Map + +### Custom skills +- **`/create-skill [description]`** — Teach Crate a new reusable command. Describe what you want, Crate does a dry run, saves it as a slash command +- **`/skills`** — List, enable/disable, edit, and delete your custom skills +- Free: 3 skills. Pro: 20 skills + +### Subscription billing +- **Free tier** — 10 agent queries/month, 5 sessions, 3 custom skills, connected services, 20+ data sources +- **Pro ($15/mo)** — 50 queries, unlimited sessions, 20 skills, cross-session memory, influence caching, publishing +- **Team ($25/mo)** — 200 pooled queries, admin dashboard, shared org keys +- **BYOK** — Bring your own Anthropic or OpenRouter key for unlimited queries +- Stripe checkout, billing portal, webhook processing + +### Mobile responsive +- **Full mobile UX** — Hamburger sidebar overlay, full-screen Deep Cuts with horizontal tabs, mini player bar, speech-to-text mic button +- **Touch optimized** — 44px+ touch targets, larger fonts, pill-shaped input +- **Capacitor ready** — designed to wrap for iOS App Store + +### Publishing & sharing +- **Deep Cut publishing** — Click Publish on any research to get a shareable link at `digcrate.app/cuts/[id]` +- **Published Deep Cuts** — Render with audio player so anyone can listen +- **Telegraph/Tumblr** — Pro users can publish formatted articles with citations + +### Other +- **Audio player** — Persistent bottom bar with YouTube playback (mini mode on mobile, hides on keyboard) +- **Live radio** — `/radio [genre/station]` streams any of 30,000+ stations +- **Multi-model** — Claude Sonnet 4.6, GPT-4o, Gemini 2.5, Llama 4, DeepSeek R1, Mistral Large via OpenRouter +- **Speech-to-text** — Mic button on mobile using Web Speech API +- **Keyboard shortcuts** — `Cmd+K` search, `Cmd+N` new chat, `Cmd+B` toggle sidebar, `Shift+S` settings + +## Tech stack + +| Layer | Technology | Purpose | +|-------|------------|---------| +| Framework | Next.js 16 (App Router) | SSR, API routes, Turbopack dev | +| Deployment | Vercel | Serverless functions | +| Auth | Clerk | User sign-in, OAuth | +| Connected services | Auth0 Token Vault | OAuth connections for Spotify, Tumblr, Slack, Google | +| Database | Convex | Real-time sessions, messages, playlists, collections, influence graph, subscriptions, skills, shares | +| Dynamic UI | OpenUI (`@openuidev/react-lang`) | Agent-generated interactive components | +| Agent | Anthropic SDK + agentic loop | Tool-use loop with crate-cli MCP servers | +| Billing | Stripe | Checkout, billing portal, webhooks | +| Styling | Tailwind CSS v4 | Dark theme, responsive mobile | +| Audio | YouTube IFrame API | Persistent player bar | +| Analytics | PostHog | Product analytics, LLM observability | +| Feedback | Canny | User feedback widget | + +## Quick start + +### Prerequisites + +- Node.js 20+ +- npm 10+ +- [Clerk](https://clerk.com) account (free tier) +- [Convex](https://convex.dev) account (free tier) + +### Install + +```bash +git clone https://github.com/tmoody1973/crate-web.git +cd crate-web +npm install +``` + +### Environment variables + +```bash +cp .env.local.example .env.local +``` + +Required: ```bash +# Clerk +NEXT_PUBLIC_CLERK_PUBLISHABLE_KEY=pk_... +CLERK_SECRET_KEY=sk_... +NEXT_PUBLIC_CLERK_SIGN_IN_URL=/sign-in +NEXT_PUBLIC_CLERK_SIGN_UP_URL=/sign-up + +# Convex +NEXT_PUBLIC_CONVEX_URL=https://...convex.cloud +CONVEX_DEPLOYMENT=dev:... + +# Encryption +ENCRYPTION_KEY=<64-char hex> + +# Platform AI key (for free/pro users without BYOK) +PLATFORM_ANTHROPIC_KEY=sk-ant-... + +# Admin +ADMIN_EMAILS=admin@example.com +``` + +Auth0 Token Vault (for connected services): + +```bash +AUTH0_DOMAIN=your-tenant.us.auth0.com +AUTH0_CLIENT_ID=your-client-id +AUTH0_CLIENT_SECRET=your-client-secret +AUTH0_CALLBACK_URL=http://localhost:3000/api/auth0/callback +``` + +Stripe billing: + +```bash +STRIPE_SECRET_KEY=sk_test_... +NEXT_PUBLIC_STRIPE_PUBLISHABLE_KEY=pk_test_... +STRIPE_WEBHOOK_SECRET=whsec_... +STRIPE_PRO_MONTHLY_PRICE_ID=price_... +STRIPE_PRO_ANNUAL_PRICE_ID=price_... +STRIPE_TEAM_MONTHLY_PRICE_ID=price_... +``` + +### Run + +```bash +# Terminal 1 — Convex dev server +npx convex dev + +# Terminal 2 — Next.js dev server npm run dev -# or -yarn dev -# or -pnpm dev -# or -bun dev ``` -Open [http://localhost:3000](http://localhost:3000) with your browser to see the result. +Open [localhost:3000](http://localhost:3000) and start digging. + +## Project structure + +``` +crate-web/ +├── convex/ # Convex backend +│ ├── schema.ts # Database schema (13 tables) +│ ├── sessions.ts # Chat session CRUD +│ ├── messages.ts # Message persistence +│ ├── playlists.ts # Playlist management +│ ├── collection.ts # Vinyl collection +│ ├── influence.ts # Influence graph cache +│ ├── subscriptions.ts # Stripe subscription state +│ ├── usage.ts # Usage tracking + quotas +│ ├── userSkills.ts # Custom skill CRUD +│ ├── shares.ts # Published Deep Cut shares +│ ├── artifacts.ts # Saved research artifacts +│ └── users.ts # User sync (Clerk → Convex) +├── src/ +│ ├── app/ +│ │ ├── api/ +│ │ │ ├── chat/route.ts # SSE streaming — agentic loop +│ │ │ ├── auth0/ # OAuth connect, callback, status, debug +│ │ │ ├── tinydesk/ # Tiny Desk DNA APIs +│ │ │ │ ├── save/route.ts # Server-side companion save + enrichment +│ │ │ │ └── enrich/route.ts # YouTube ID + genre resolution +│ │ │ ├── cuts/publish/ # Deep Cut publishing +│ │ │ ├── stripe/ # Checkout, portal, webhooks +│ │ │ ├── skills/ # Custom skill management +│ │ │ ├── keys/route.ts # API key management +│ │ │ └── artwork/route.ts # Spotify artwork proxy +│ │ ├── tinydesk/ # Tiny Desk DNA pages +│ │ │ ├── page.tsx # Landing page (626 artists, before/after hero) +│ │ │ └── [slug]/page.tsx # Companion page (influence chain scroll) +│ │ ├── cuts/[shareId]/ # Public share page +│ │ ├── w/[sessionId]/ # Workspace (authenticated) +│ │ ├── pricing/ # Pricing page +│ │ └── help/ # Help guide +│ ├── components/ +│ │ ├── tinydesk/ # Tiny Desk DNA components +│ │ │ ├── catalog-client.tsx # Genre filter, grid/timeline toggle, surprise me +│ │ │ ├── artist-card.tsx # Concert card with thumbnail + badges +│ │ │ ├── genre-filter.tsx # Scrollable genre pills +│ │ │ ├── timeline-view.tsx # Chronological timeline grouped by year +│ │ │ ├── surprise-modal.tsx # Random artist overlay +│ │ │ └── video-influence-chain.tsx # Companion page influence scroll +│ │ ├── chat/ # Mobile header, mic, inline cards +│ │ ├── workspace/ +│ │ │ ├── chat-panel.tsx # Chat with OpenUI rendering +│ │ │ ├── deep-cuts-panel.tsx # Resizable Deep Cuts panel +│ │ │ ├── mobile-deep-cuts.tsx # Full-screen mobile Deep Cuts +│ │ │ └── workspace-shell.tsx # Layout + mobile nav +│ │ ├── sidebar/ # Desktop + mobile sidebar +│ │ ├── settings/ # Connected services, plan, skills, keys +│ │ ├── player/ # YouTube audio player +│ │ ├── onboarding/ # Quick start wizard +│ │ └── landing/ # Landing page sections +│ ├── hooks/ +│ │ ├── use-is-mobile.ts # SSR-safe mobile breakpoint +│ │ └── use-keyboard-visible.ts # iOS keyboard detection +│ └── lib/ +│ ├── openui/ +│ │ ├── components.tsx # 27+ OpenUI component definitions +│ │ ├── library.ts # Component registry + examples +│ │ └── prompt.ts # System prompt for agent → OpenUI +│ ├── web-tools/ +│ │ ├── spotify-connected.ts # Read library, create playlists, read tracks +│ │ ├── slack.ts # List channels, send messages +│ │ ├── slack-formatter.ts # Block Kit rich text formatting +│ │ ├── google-docs.ts # Save to Google Docs +│ │ ├── tumblr-connected.ts # Read dashboard/tags/likes, publish posts +│ │ ├── user-skills.ts # Custom skill tools +│ │ ├── prep-research.ts # Perplexity-powered research +│ │ └── ... # WhoSampled, Bandcamp, radio, etc. +│ ├── tinydesk-enrichment.ts # Shared YouTube + genre resolution +│ ├── auth0-token-vault.ts # OAuth token exchange via Management API +│ ├── agentic-loop.ts # Agentic loop (Anthropic + OpenRouter) +│ ├── plans.ts # Subscription tiers, rate limiting +│ ├── deep-cut-utils.ts # Type detection, colors, actions +│ └── chat-utils.ts # Slash commands, prompt routing +└── docs/ + ├── storycard-guide.md # StoryCard component guide + ├── testing-guide.md # Feature testing checklist + ├── articles/ # LinkedIn, blog posts + └── superpowers/ # Design specs + plans +``` + +## OpenUI components + +| Component | Purpose | +|-----------|---------| +| ArtistProfile | Full artist deep dive — tabbed discography, connections, media, top tracks | +| ArtistCard | Compact baseball-card artist profiles with auto-fetched images | +| TrackCard | Single-track deep dive — tabbed credits, samples, lyrics, vinyl pressings | +| InfluenceChain | Narrative influence timeline with hero banner, tabs, source cards | +| StoryCard | Rich narrative stories with chapters, YouTube, tracks, key people | +| ShowPrepPackage | Full show prep with track context, talk breaks, social copy | +| TrackList | Playable playlist with auto-save and AI-generated cover art | +| SpotifyPlaylists | Spotify library browser with Explore/Open buttons | +| SpotifyPlaylist | Track table from a Spotify playlist with action buttons | +| SlackChannelPicker | Clickable channel grid for sending to Slack | +| SlackMessage | Slack message preview with send status | +| TumblrFeed | Tumblr posts by tag, dashboard, or likes with type filters | +| AlbumGrid | Discography display with cover art | +| SampleTree | Sample relationship visualization | +| ConcertList | Event listings with venue, date, price | + +## Auth0 Token Vault integration + +Crate uses Auth0 Token Vault to securely connect to four third-party services on behalf of users. The architecture: + +1. **Clerk** handles user sign-in (existing auth) +2. **Auth0** handles OAuth connections to Spotify, Tumblr, Slack, Google +3. Token Vault stores and manages OAuth tokens — Crate never sees raw credentials +4. The Management API retrieves IdP access tokens at runtime + +### Supported services + +| Service | OAuth Scopes | What the Agent Does | +|---------|-------------|-------------------| +| Spotify | `user-library-read`, `playlist-modify-public`, `streaming` | Read library, export playlists, stream tracks | +| Tumblr | `basic`, `write`, `offline_access` | Publish research, discover music by tag | +| Slack | `chat:write`, `chat:write.public`, `channels:read` | Send show prep to team channels | +| Google Docs | `documents`, `drive.file` | Save research as permanent documents | + +### Connection flow + +``` +User clicks "Connect Spotify" in Settings + → /api/auth0/connect?service=spotify (generates CSRF nonce, redirects to Auth0) + → Auth0 /authorize (user grants permissions) + → /api/auth0/callback (exchanges code, extracts Auth0 user ID, sets cookies) + → Redirects back to current session +``` + +### Token retrieval + +``` +Agent calls read_spotify_library tool + → getTokenVaultToken("spotify", auth0UserId) + → Gets Management API token (cached 23h) + → Calls /api/v2/users/{userId}?fields=identities + → Returns access_token from the spotify identity + → Tool uses token to call Spotify Web API +``` + +## Data sources + +| Source | Data | +|--------|------| +| Discogs | Releases, labels, credits, cover art | +| MusicBrainz | Artist metadata, relationships, recordings | +| Last.fm | Similar artists, tags, listening stats | +| Genius | Lyrics, annotations, song metadata | +| Bandcamp | Album search, tag exploration, related tags | +| WhoSampled | Sample origins, covers, remixes | +| Wikipedia | Artist bios, discography context | +| Ticketmaster | Concert listings, ticket availability | +| Spotify | Library, playlists, top artists (via Auth0) | +| fanart.tv | HD artist backgrounds, logos | +| iTunes | Album artwork, track search | +| YouTube | Music videos, documentaries | +| Exa.ai | Semantic web search | +| Tavily | AI-optimized web search | +| Perplexity (Sonar) | Research enrichment with citations | +| Mem0 | Cross-session user memory (Pro) | +| Radio Browser | 30,000+ live radio stations | -You can start editing the page by modifying `app/page.tsx`. The page auto-updates as you edit the file. +## Deploy -This project uses [`next/font`](https://nextjs.org/docs/app/building-your-application/optimizing/fonts) to automatically optimize and load [Geist](https://vercel.com/font), a new font family for Vercel. +```bash +npm i -g vercel +vercel --prod +npx convex deploy --yes +``` -## Learn More +Set all environment variables in Vercel dashboard. -To learn more about Next.js, take a look at the following resources: +## Related -- [Next.js Documentation](https://nextjs.org/docs) - learn about Next.js features and API. -- [Learn Next.js](https://nextjs.org/learn) - an interactive Next.js tutorial. +- [Crate CLI](https://github.com/tmoody1973/crate-cli) — Terminal-based AI music research agent +- [OpenUI](https://github.com/thesysdev/openui) — Dynamic UI generation framework +- [Auth0 Token Vault](https://auth0.com/ai/docs/intro/token-vault) — Secure token exchange for AI agents +- [Convex](https://convex.dev) — Real-time database -You can check out [the Next.js GitHub repository](https://github.com/vercel/next.js) - your feedback and contributions are welcome! +## Legal -## Deploy on Vercel +- [Privacy Policy](https://digcrate.app/privacy) +- [Terms of Service](https://digcrate.app/terms) -The easiest way to deploy your Next.js app is to use the [Vercel Platform](https://vercel.com/new?utm_medium=default-template&filter=next.js&utm_source=create-next-app&utm_campaign=create-next-app-readme) from the creators of Next.js. +## License -Check out our [Next.js deployment documentation](https://nextjs.org/docs/app/building-your-application/deploying) for more details. +MIT diff --git a/convex/README.md b/convex/README.md new file mode 100644 index 0000000..91a9db2 --- /dev/null +++ b/convex/README.md @@ -0,0 +1,90 @@ +# Welcome to your Convex functions directory! + +Write your Convex functions here. +See https://docs.convex.dev/functions for more. + +A query function that takes two arguments looks like: + +```ts +// convex/myFunctions.ts +import { query } from "./_generated/server"; +import { v } from "convex/values"; + +export const myQueryFunction = query({ + // Validators for arguments. + args: { + first: v.number(), + second: v.string(), + }, + + // Function implementation. + handler: async (ctx, args) => { + // Read the database as many times as you need here. + // See https://docs.convex.dev/database/reading-data. + const documents = await ctx.db.query("tablename").collect(); + + // Arguments passed from the client are properties of the args object. + console.log(args.first, args.second); + + // Write arbitrary JavaScript here: filter, aggregate, build derived data, + // remove non-public properties, or create new objects. + return documents; + }, +}); +``` + +Using this query function in a React component looks like: + +```ts +const data = useQuery(api.myFunctions.myQueryFunction, { + first: 10, + second: "hello", +}); +``` + +A mutation function looks like: + +```ts +// convex/myFunctions.ts +import { mutation } from "./_generated/server"; +import { v } from "convex/values"; + +export const myMutationFunction = mutation({ + // Validators for arguments. + args: { + first: v.string(), + second: v.string(), + }, + + // Function implementation. + handler: async (ctx, args) => { + // Insert or modify documents in the database here. + // Mutations can also read from the database like queries. + // See https://docs.convex.dev/database/writing-data. + const message = { body: args.first, author: args.second }; + const id = await ctx.db.insert("messages", message); + + // Optionally, return a value from your mutation. + return await ctx.db.get("messages", id); + }, +}); +``` + +Using this mutation function in a React component looks like: + +```ts +const mutation = useMutation(api.myFunctions.myMutationFunction); +function handleButtonPress() { + // fire and forget, the most common way to use mutations + mutation({ first: "Hello!", second: "me" }); + // OR + // use the result once the mutation has completed + mutation({ first: "Hello!", second: "me" }).then((result) => + console.log(result), + ); +} +``` + +Use the Convex CLI to push your functions to a deployment. See everything +the Convex CLI can do by running `npx convex -h` in your project root +directory. To learn more, launch the docs with `npx convex docs`. diff --git a/convex/_generated/api.d.ts b/convex/_generated/api.d.ts new file mode 100644 index 0000000..c82753e --- /dev/null +++ b/convex/_generated/api.d.ts @@ -0,0 +1,87 @@ +/* eslint-disable */ +/** + * Generated `api` utility. + * + * THIS CODE IS AUTOMATICALLY GENERATED. + * + * To regenerate, run `npx convex dev`. + * @module + */ + +import type * as artifacts from "../artifacts.js"; +import type * as collection from "../collection.js"; +import type * as crates from "../crates.js"; +import type * as influence from "../influence.js"; +import type * as keys from "../keys.js"; +import type * as messages from "../messages.js"; +import type * as orgKeys from "../orgKeys.js"; +import type * as playlists from "../playlists.js"; +import type * as published from "../published.js"; +import type * as sessions from "../sessions.js"; +import type * as shares from "../shares.js"; +import type * as subscriptions from "../subscriptions.js"; +import type * as telegraph from "../telegraph.js"; +import type * as tinydeskCompanions from "../tinydeskCompanions.js"; +import type * as toolCalls from "../toolCalls.js"; +import type * as tumblr from "../tumblr.js"; +import type * as usage from "../usage.js"; +import type * as userSkills from "../userSkills.js"; +import type * as users from "../users.js"; +import type * as wiki from "../wiki.js"; + +import type { + ApiFromModules, + FilterApi, + FunctionReference, +} from "convex/server"; + +declare const fullApi: ApiFromModules<{ + artifacts: typeof artifacts; + collection: typeof collection; + crates: typeof crates; + influence: typeof influence; + keys: typeof keys; + messages: typeof messages; + orgKeys: typeof orgKeys; + playlists: typeof playlists; + published: typeof published; + sessions: typeof sessions; + shares: typeof shares; + subscriptions: typeof subscriptions; + telegraph: typeof telegraph; + tinydeskCompanions: typeof tinydeskCompanions; + toolCalls: typeof toolCalls; + tumblr: typeof tumblr; + usage: typeof usage; + userSkills: typeof userSkills; + users: typeof users; + wiki: typeof wiki; +}>; + +/** + * A utility for referencing Convex functions in your app's public API. + * + * Usage: + * ```js + * const myFunctionReference = api.myModule.myFunction; + * ``` + */ +export declare const api: FilterApi< + typeof fullApi, + FunctionReference +>; + +/** + * A utility for referencing Convex functions in your app's internal API. + * + * Usage: + * ```js + * const myFunctionReference = internal.myModule.myFunction; + * ``` + */ +export declare const internal: FilterApi< + typeof fullApi, + FunctionReference +>; + +export declare const components: {}; diff --git a/convex/_generated/api.js b/convex/_generated/api.js new file mode 100644 index 0000000..44bf985 --- /dev/null +++ b/convex/_generated/api.js @@ -0,0 +1,23 @@ +/* eslint-disable */ +/** + * Generated `api` utility. + * + * THIS CODE IS AUTOMATICALLY GENERATED. + * + * To regenerate, run `npx convex dev`. + * @module + */ + +import { anyApi, componentsGeneric } from "convex/server"; + +/** + * A utility for referencing Convex functions in your app's API. + * + * Usage: + * ```js + * const myFunctionReference = api.myModule.myFunction; + * ``` + */ +export const api = anyApi; +export const internal = anyApi; +export const components = componentsGeneric(); diff --git a/convex/_generated/dataModel.d.ts b/convex/_generated/dataModel.d.ts new file mode 100644 index 0000000..f97fd19 --- /dev/null +++ b/convex/_generated/dataModel.d.ts @@ -0,0 +1,60 @@ +/* eslint-disable */ +/** + * Generated data model types. + * + * THIS CODE IS AUTOMATICALLY GENERATED. + * + * To regenerate, run `npx convex dev`. + * @module + */ + +import type { + DataModelFromSchemaDefinition, + DocumentByName, + TableNamesInDataModel, + SystemTableNames, +} from "convex/server"; +import type { GenericId } from "convex/values"; +import schema from "../schema.js"; + +/** + * The names of all of your Convex tables. + */ +export type TableNames = TableNamesInDataModel; + +/** + * The type of a document stored in Convex. + * + * @typeParam TableName - A string literal type of the table name (like "users"). + */ +export type Doc = DocumentByName< + DataModel, + TableName +>; + +/** + * An identifier for a document in Convex. + * + * Convex documents are uniquely identified by their `Id`, which is accessible + * on the `_id` field. To learn more, see [Document IDs](https://docs.convex.dev/using/document-ids). + * + * Documents can be loaded using `db.get(tableName, id)` in query and mutation functions. + * + * IDs are just strings at runtime, but this type can be used to distinguish them from other + * strings when type checking. + * + * @typeParam TableName - A string literal type of the table name (like "users"). + */ +export type Id = + GenericId; + +/** + * A type describing your Convex data model. + * + * This type includes information about what tables you have, the type of + * documents stored in those tables, and the indexes defined on them. + * + * This type is used to parameterize methods like `queryGeneric` and + * `mutationGeneric` to make them type-safe. + */ +export type DataModel = DataModelFromSchemaDefinition; diff --git a/convex/_generated/server.d.ts b/convex/_generated/server.d.ts new file mode 100644 index 0000000..bec05e6 --- /dev/null +++ b/convex/_generated/server.d.ts @@ -0,0 +1,143 @@ +/* eslint-disable */ +/** + * Generated utilities for implementing server-side Convex query and mutation functions. + * + * THIS CODE IS AUTOMATICALLY GENERATED. + * + * To regenerate, run `npx convex dev`. + * @module + */ + +import { + ActionBuilder, + HttpActionBuilder, + MutationBuilder, + QueryBuilder, + GenericActionCtx, + GenericMutationCtx, + GenericQueryCtx, + GenericDatabaseReader, + GenericDatabaseWriter, +} from "convex/server"; +import type { DataModel } from "./dataModel.js"; + +/** + * Define a query in this Convex app's public API. + * + * This function will be allowed to read your Convex database and will be accessible from the client. + * + * @param func - The query function. It receives a {@link QueryCtx} as its first argument. + * @returns The wrapped query. Include this as an `export` to name it and make it accessible. + */ +export declare const query: QueryBuilder; + +/** + * Define a query that is only accessible from other Convex functions (but not from the client). + * + * This function will be allowed to read from your Convex database. It will not be accessible from the client. + * + * @param func - The query function. It receives a {@link QueryCtx} as its first argument. + * @returns The wrapped query. Include this as an `export` to name it and make it accessible. + */ +export declare const internalQuery: QueryBuilder; + +/** + * Define a mutation in this Convex app's public API. + * + * This function will be allowed to modify your Convex database and will be accessible from the client. + * + * @param func - The mutation function. It receives a {@link MutationCtx} as its first argument. + * @returns The wrapped mutation. Include this as an `export` to name it and make it accessible. + */ +export declare const mutation: MutationBuilder; + +/** + * Define a mutation that is only accessible from other Convex functions (but not from the client). + * + * This function will be allowed to modify your Convex database. It will not be accessible from the client. + * + * @param func - The mutation function. It receives a {@link MutationCtx} as its first argument. + * @returns The wrapped mutation. Include this as an `export` to name it and make it accessible. + */ +export declare const internalMutation: MutationBuilder; + +/** + * Define an action in this Convex app's public API. + * + * An action is a function which can execute any JavaScript code, including non-deterministic + * code and code with side-effects, like calling third-party services. + * They can be run in Convex's JavaScript environment or in Node.js using the "use node" directive. + * They can interact with the database indirectly by calling queries and mutations using the {@link ActionCtx}. + * + * @param func - The action. It receives an {@link ActionCtx} as its first argument. + * @returns The wrapped action. Include this as an `export` to name it and make it accessible. + */ +export declare const action: ActionBuilder; + +/** + * Define an action that is only accessible from other Convex functions (but not from the client). + * + * @param func - The function. It receives an {@link ActionCtx} as its first argument. + * @returns The wrapped function. Include this as an `export` to name it and make it accessible. + */ +export declare const internalAction: ActionBuilder; + +/** + * Define an HTTP action. + * + * The wrapped function will be used to respond to HTTP requests received + * by a Convex deployment if the requests matches the path and method where + * this action is routed. Be sure to route your httpAction in `convex/http.js`. + * + * @param func - The function. It receives an {@link ActionCtx} as its first argument + * and a Fetch API `Request` object as its second. + * @returns The wrapped function. Import this function from `convex/http.js` and route it to hook it up. + */ +export declare const httpAction: HttpActionBuilder; + +/** + * A set of services for use within Convex query functions. + * + * The query context is passed as the first argument to any Convex query + * function run on the server. + * + * This differs from the {@link MutationCtx} because all of the services are + * read-only. + */ +export type QueryCtx = GenericQueryCtx; + +/** + * A set of services for use within Convex mutation functions. + * + * The mutation context is passed as the first argument to any Convex mutation + * function run on the server. + */ +export type MutationCtx = GenericMutationCtx; + +/** + * A set of services for use within Convex action functions. + * + * The action context is passed as the first argument to any Convex action + * function run on the server. + */ +export type ActionCtx = GenericActionCtx; + +/** + * An interface to read from the database within Convex query functions. + * + * The two entry points are {@link DatabaseReader.get}, which fetches a single + * document by its {@link Id}, or {@link DatabaseReader.query}, which starts + * building a query. + */ +export type DatabaseReader = GenericDatabaseReader; + +/** + * An interface to read from and write to the database within Convex mutation + * functions. + * + * Convex guarantees that all writes within a single mutation are + * executed atomically, so you never have to worry about partial writes leaving + * your data in an inconsistent state. See [the Convex Guide](https://docs.convex.dev/understanding/convex-fundamentals/functions#atomicity-and-optimistic-concurrency-control) + * for the guarantees Convex provides your functions. + */ +export type DatabaseWriter = GenericDatabaseWriter; diff --git a/convex/_generated/server.js b/convex/_generated/server.js new file mode 100644 index 0000000..bf3d25a --- /dev/null +++ b/convex/_generated/server.js @@ -0,0 +1,93 @@ +/* eslint-disable */ +/** + * Generated utilities for implementing server-side Convex query and mutation functions. + * + * THIS CODE IS AUTOMATICALLY GENERATED. + * + * To regenerate, run `npx convex dev`. + * @module + */ + +import { + actionGeneric, + httpActionGeneric, + queryGeneric, + mutationGeneric, + internalActionGeneric, + internalMutationGeneric, + internalQueryGeneric, +} from "convex/server"; + +/** + * Define a query in this Convex app's public API. + * + * This function will be allowed to read your Convex database and will be accessible from the client. + * + * @param func - The query function. It receives a {@link QueryCtx} as its first argument. + * @returns The wrapped query. Include this as an `export` to name it and make it accessible. + */ +export const query = queryGeneric; + +/** + * Define a query that is only accessible from other Convex functions (but not from the client). + * + * This function will be allowed to read from your Convex database. It will not be accessible from the client. + * + * @param func - The query function. It receives a {@link QueryCtx} as its first argument. + * @returns The wrapped query. Include this as an `export` to name it and make it accessible. + */ +export const internalQuery = internalQueryGeneric; + +/** + * Define a mutation in this Convex app's public API. + * + * This function will be allowed to modify your Convex database and will be accessible from the client. + * + * @param func - The mutation function. It receives a {@link MutationCtx} as its first argument. + * @returns The wrapped mutation. Include this as an `export` to name it and make it accessible. + */ +export const mutation = mutationGeneric; + +/** + * Define a mutation that is only accessible from other Convex functions (but not from the client). + * + * This function will be allowed to modify your Convex database. It will not be accessible from the client. + * + * @param func - The mutation function. It receives a {@link MutationCtx} as its first argument. + * @returns The wrapped mutation. Include this as an `export` to name it and make it accessible. + */ +export const internalMutation = internalMutationGeneric; + +/** + * Define an action in this Convex app's public API. + * + * An action is a function which can execute any JavaScript code, including non-deterministic + * code and code with side-effects, like calling third-party services. + * They can be run in Convex's JavaScript environment or in Node.js using the "use node" directive. + * They can interact with the database indirectly by calling queries and mutations using the {@link ActionCtx}. + * + * @param func - The action. It receives an {@link ActionCtx} as its first argument. + * @returns The wrapped action. Include this as an `export` to name it and make it accessible. + */ +export const action = actionGeneric; + +/** + * Define an action that is only accessible from other Convex functions (but not from the client). + * + * @param func - The function. It receives an {@link ActionCtx} as its first argument. + * @returns The wrapped function. Include this as an `export` to name it and make it accessible. + */ +export const internalAction = internalActionGeneric; + +/** + * Define an HTTP action. + * + * The wrapped function will be used to respond to HTTP requests received + * by a Convex deployment if the requests matches the path and method where + * this action is routed. Be sure to route your httpAction in `convex/http.js`. + * + * @param func - The function. It receives an {@link ActionCtx} as its first argument + * and a Fetch API `Request` object as its second. + * @returns The wrapped function. Import this function from `convex/http.js` and route it to hook it up. + */ +export const httpAction = httpActionGeneric; diff --git a/convex/artifacts.ts b/convex/artifacts.ts new file mode 100644 index 0000000..870c37e --- /dev/null +++ b/convex/artifacts.ts @@ -0,0 +1,62 @@ +import { mutation, query } from "./_generated/server"; +import { v } from "convex/values"; + +export const create = mutation({ + args: { + sessionId: v.id("sessions"), + userId: v.id("users"), + messageId: v.optional(v.id("messages")), + type: v.string(), + label: v.string(), + data: v.string(), + contentHash: v.string(), + }, + handler: async (ctx, args) => { + const existing = await ctx.db + .query("artifacts") + .withIndex("by_session", (q) => q.eq("sessionId", args.sessionId)) + .filter((q) => q.eq(q.field("contentHash"), args.contentHash)) + .first(); + if (existing) return existing._id; + + return await ctx.db.insert("artifacts", { + sessionId: args.sessionId, + userId: args.userId, + messageId: args.messageId, + type: args.type, + label: args.label, + data: args.data, + contentHash: args.contentHash, + createdAt: Date.now(), + }); + }, +}); + +export const listBySession = query({ + args: { sessionId: v.id("sessions") }, + handler: async (ctx, args) => { + return await ctx.db + .query("artifacts") + .withIndex("by_session", (q) => q.eq("sessionId", args.sessionId)) + .order("asc") + .collect(); + }, +}); + +export const listByUser = query({ + args: { userId: v.id("users") }, + handler: async (ctx, args) => { + return await ctx.db + .query("artifacts") + .withIndex("by_user", (q) => q.eq("userId", args.userId)) + .order("desc") + .take(50); + }, +}); + +export const getById = query({ + args: { id: v.id("artifacts") }, + handler: async (ctx, { id }) => { + return await ctx.db.get(id); + }, +}); diff --git a/convex/auth.config.ts b/convex/auth.config.ts new file mode 100644 index 0000000..49469c9 --- /dev/null +++ b/convex/auth.config.ts @@ -0,0 +1,8 @@ +export default { + providers: [ + { + domain: "https://clerk.digcrate.app", + applicationID: "convex", + }, + ], +}; diff --git a/convex/collection.ts b/convex/collection.ts new file mode 100644 index 0000000..b0b5c7c --- /dev/null +++ b/convex/collection.ts @@ -0,0 +1,136 @@ +import { mutation, query } from "./_generated/server"; +import { v } from "convex/values"; + +export const list = query({ + args: { userId: v.id("users") }, + handler: async (ctx, args) => { + return await ctx.db + .query("collection") + .withIndex("by_user", (q) => q.eq("userId", args.userId)) + .order("desc") + .collect(); + }, +}); + +export const search = query({ + args: { query: v.string(), userId: v.id("users") }, + handler: async (ctx, args) => { + if (!args.query.trim()) return []; + const results = await ctx.db + .query("collection") + .withSearchIndex("search_collection", (q) => + q.search("title", args.query).eq("userId", args.userId), + ) + .take(20); + return results; + }, +}); + +export const add = mutation({ + args: { + userId: v.id("users"), + title: v.string(), + artist: v.string(), + label: v.optional(v.string()), + year: v.optional(v.string()), + format: v.optional(v.string()), + genre: v.optional(v.string()), + notes: v.optional(v.string()), + imageUrl: v.optional(v.string()), + discogsId: v.optional(v.string()), + rating: v.optional(v.number()), + }, + handler: async (ctx, args) => { + return await ctx.db.insert("collection", { + ...args, + createdAt: Date.now(), + }); + }, +}); + +export const addMultiple = mutation({ + args: { + userId: v.id("users"), + items: v.array( + v.object({ + title: v.string(), + artist: v.string(), + label: v.optional(v.string()), + year: v.optional(v.string()), + format: v.optional(v.string()), + imageUrl: v.optional(v.string()), + }), + ), + }, + handler: async (ctx, args) => { + const ids = []; + for (const item of args.items) { + const id = await ctx.db.insert("collection", { + userId: args.userId, + ...item, + createdAt: Date.now(), + }); + ids.push(id); + } + return ids; + }, +}); + +export const update = mutation({ + args: { + id: v.id("collection"), + title: v.optional(v.string()), + artist: v.optional(v.string()), + label: v.optional(v.string()), + year: v.optional(v.string()), + format: v.optional(v.string()), + genre: v.optional(v.string()), + notes: v.optional(v.string()), + rating: v.optional(v.number()), + }, + handler: async (ctx, args) => { + const { id, ...fields } = args; + const updates: Record = {}; + for (const [key, value] of Object.entries(fields)) { + if (value !== undefined) updates[key] = value; + } + if (Object.keys(updates).length > 0) { + await ctx.db.patch(id, updates); + } + }, +}); + +export const remove = mutation({ + args: { id: v.id("collection") }, + handler: async (ctx, args) => { + await ctx.db.delete(args.id); + }, +}); + +export const stats = query({ + args: { userId: v.id("users") }, + handler: async (ctx, args) => { + const items = await ctx.db + .query("collection") + .withIndex("by_user", (q) => q.eq("userId", args.userId)) + .collect(); + + const formats = new Map(); + const genres = new Map(); + + for (const item of items) { + if (item.format) { + formats.set(item.format, (formats.get(item.format) ?? 0) + 1); + } + if (item.genre) { + genres.set(item.genre, (genres.get(item.genre) ?? 0) + 1); + } + } + + return { + total: items.length, + formats: Object.fromEntries(formats), + genres: Object.fromEntries(genres), + }; + }, +}); diff --git a/convex/crates.ts b/convex/crates.ts new file mode 100644 index 0000000..128af09 --- /dev/null +++ b/convex/crates.ts @@ -0,0 +1,49 @@ +import { mutation, query } from "./_generated/server"; +import { v } from "convex/values"; + +export const create = mutation({ + args: { + userId: v.id("users"), + name: v.string(), + color: v.optional(v.string()), + }, + handler: async (ctx, args) => { + return await ctx.db.insert("crates", { + userId: args.userId, + name: args.name, + color: args.color, + createdAt: Date.now(), + }); + }, +}); + +export const list = query({ + args: { userId: v.id("users") }, + handler: async (ctx, args) => { + return await ctx.db + .query("crates") + .withIndex("by_user", (q) => q.eq("userId", args.userId)) + .collect(); + }, +}); + +export const rename = mutation({ + args: { id: v.id("crates"), name: v.string() }, + handler: async (ctx, args) => { + await ctx.db.patch(args.id, { name: args.name }); + }, +}); + +export const remove = mutation({ + args: { id: v.id("crates") }, + handler: async (ctx, args) => { + const sessions = await ctx.db + .query("sessions") + .filter((q) => q.eq(q.field("crateId"), args.id)) + .collect(); + for (const session of sessions) { + await ctx.db.patch(session._id, { crateId: undefined }); + } + await ctx.db.delete(args.id); + }, +}); diff --git a/convex/influence.ts b/convex/influence.ts new file mode 100644 index 0000000..9ec6bab --- /dev/null +++ b/convex/influence.ts @@ -0,0 +1,647 @@ +import { query, mutation } from "./_generated/server"; +import { v } from "convex/values"; + +// getOrCreateArtist - upsert artist by userId + nameLower +export const getOrCreateArtist = mutation({ + args: { + userId: v.id("users"), + name: v.string(), + genres: v.optional(v.string()), + imageUrl: v.optional(v.string()), + }, + handler: async (ctx, args) => { + const nameLower = args.name.toLowerCase().trim(); + const existing = await ctx.db + .query("influenceArtists") + .withIndex("by_user_name", (q) => q.eq("userId", args.userId).eq("nameLower", nameLower)) + .first(); + if (existing) { + // Update image/genres if provided and missing + if ((args.imageUrl && !existing.imageUrl) || (args.genres && !existing.genres)) { + await ctx.db.patch(existing._id, { + ...(args.imageUrl && !existing.imageUrl ? { imageUrl: args.imageUrl } : {}), + ...(args.genres && !existing.genres ? { genres: args.genres } : {}), + }); + } + return existing._id; + } + return await ctx.db.insert("influenceArtists", { + userId: args.userId, + name: args.name, + nameLower, + genres: args.genres, + imageUrl: args.imageUrl, + createdAt: Date.now(), + }); + }, +}); + +// cacheEdge - upsert edge between two artists (keep max weight) +export const cacheEdge = mutation({ + args: { + userId: v.id("users"), + fromName: v.string(), + toName: v.string(), + relationship: v.string(), + weight: v.number(), + context: v.optional(v.string()), + source: v.optional(v.object({ + sourceType: v.string(), + sourceUrl: v.optional(v.string()), + sourceName: v.optional(v.string()), + snippet: v.optional(v.string()), + })), + fromGenres: v.optional(v.string()), + toGenres: v.optional(v.string()), + fromImageUrl: v.optional(v.string()), + toImageUrl: v.optional(v.string()), + }, + handler: async (ctx, args) => { + // Get or create both artists + // (inline the getOrCreateArtist logic to avoid action overhead) + const getOrCreate = async (name: string, genres?: string, imageUrl?: string) => { + const nameLower = name.toLowerCase().trim(); + const existing = await ctx.db + .query("influenceArtists") + .withIndex("by_user_name", (q) => q.eq("userId", args.userId).eq("nameLower", nameLower)) + .first(); + if (existing) { + if ((imageUrl && !existing.imageUrl) || (genres && !existing.genres)) { + await ctx.db.patch(existing._id, { + ...(imageUrl && !existing.imageUrl ? { imageUrl } : {}), + ...(genres && !existing.genres ? { genres } : {}), + }); + } + return existing._id; + } + return await ctx.db.insert("influenceArtists", { + userId: args.userId, name, nameLower, genres, imageUrl, createdAt: Date.now(), + }); + }; + + const fromId = await getOrCreate(args.fromName, args.fromGenres, args.fromImageUrl); + const toId = await getOrCreate(args.toName, args.toGenres, args.toImageUrl); + + // Check for existing edge + const existingEdge = await ctx.db + .query("influenceEdges") + .withIndex("by_from", (q) => q.eq("userId", args.userId).eq("fromArtistId", fromId)) + .filter((q) => q.eq(q.field("toArtistId"), toId)) + .first(); + + const now = Date.now(); + + if (existingEdge) { + // Always increment mentionCount + const currentMentionCount = existingEdge.mentionCount ?? 1; + await ctx.db.patch(existingEdge._id, { + mentionCount: currentMentionCount + 1, + // only update weight if new weight is higher + ...(args.weight > existingEdge.weight ? { + weight: args.weight, + relationship: args.relationship, + context: args.context ?? existingEdge.context, + } : {}), + updatedAt: now, + }); + // Add source if provided + if (args.source) { + await ctx.db.insert("influenceEdgeSources", { + edgeId: existingEdge._id, + ...args.source, + discoveredAt: now, + }); + } + return existingEdge._id; + } + + // Create new edge + const edgeId = await ctx.db.insert("influenceEdges", { + userId: args.userId, + fromArtistId: fromId, + toArtistId: toId, + relationship: args.relationship, + weight: args.weight, + mentionCount: 1, + context: args.context, + createdAt: now, + updatedAt: now, + }); + + if (args.source) { + await ctx.db.insert("influenceEdgeSources", { + edgeId, + ...args.source, + discoveredAt: now, + }); + } + + return edgeId; + }, +}); + +// cacheBatchEdges - batch upsert +export const cacheBatchEdges = mutation({ + args: { + userId: v.id("users"), + edges: v.array(v.object({ + fromName: v.string(), + toName: v.string(), + relationship: v.string(), + weight: v.number(), + context: v.optional(v.string()), + source: v.optional(v.object({ + sourceType: v.string(), + sourceUrl: v.optional(v.string()), + sourceName: v.optional(v.string()), + snippet: v.optional(v.string()), + })), + fromImageUrl: v.optional(v.string()), + toImageUrl: v.optional(v.string()), + })), + }, + handler: async (ctx, args) => { + // Process edges inline (same getOrCreate logic, with image support) + const getOrCreate = async (name: string, imageUrl?: string) => { + const nameLower = name.toLowerCase().trim(); + const existing = await ctx.db + .query("influenceArtists") + .withIndex("by_user_name", (q) => q.eq("userId", args.userId).eq("nameLower", nameLower)) + .first(); + if (existing) { + if (imageUrl && !existing.imageUrl) { + await ctx.db.patch(existing._id, { imageUrl }); + } + return existing._id; + } + return await ctx.db.insert("influenceArtists", { + userId: args.userId, name, nameLower, imageUrl, createdAt: Date.now(), + }); + }; + + const results = []; + for (const edge of args.edges) { + const fromId = await getOrCreate(edge.fromName, edge.fromImageUrl); + const toId = await getOrCreate(edge.toName, edge.toImageUrl); + const now = Date.now(); + + const existing = await ctx.db + .query("influenceEdges") + .withIndex("by_from", (q) => q.eq("userId", args.userId).eq("fromArtistId", fromId)) + .filter((q) => q.eq(q.field("toArtistId"), toId)) + .first(); + + if (existing) { + const currentMentionCount = existing.mentionCount ?? 1; + await ctx.db.patch(existing._id, { + mentionCount: currentMentionCount + 1, + ...(edge.weight > existing.weight ? { + weight: edge.weight, + relationship: edge.relationship, + context: edge.context ?? existing.context, + } : {}), + updatedAt: now, + }); + if (edge.source) { + await ctx.db.insert("influenceEdgeSources", { edgeId: existing._id, ...edge.source, discoveredAt: now }); + } + results.push(existing._id); + } else { + const edgeId = await ctx.db.insert("influenceEdges", { + userId: args.userId, fromArtistId: fromId, toArtistId: toId, + relationship: edge.relationship, weight: edge.weight, mentionCount: 1, + context: edge.context, createdAt: now, updatedAt: now, + }); + if (edge.source) { + await ctx.db.insert("influenceEdgeSources", { edgeId, ...edge.source, discoveredAt: now }); + } + results.push(edgeId); + } + } + return results; + }, +}); + +// lookupInfluences - query connections for an artist +export const lookupInfluences = query({ + args: { + userId: v.id("users"), + artist: v.string(), + direction: v.optional(v.union(v.literal("outgoing"), v.literal("incoming"), v.literal("both"))), + relationship: v.optional(v.string()), + minWeight: v.optional(v.number()), + }, + handler: async (ctx, args) => { + const nameLower = args.artist.toLowerCase().trim(); + const artistRecord = await ctx.db + .query("influenceArtists") + .withIndex("by_user_name", (q) => q.eq("userId", args.userId).eq("nameLower", nameLower)) + .first(); + if (!artistRecord) return { artist: args.artist, connections: [], cached: false }; + + const direction = args.direction ?? "both"; + const minWeight = args.minWeight ?? 0; + const connections: Array<{ + direction: "outgoing" | "incoming"; + artist: string; + genres?: string; + imageUrl?: string; + relationship: string; + weight: number; + mentionCount: number; + context?: string; + sources: Array<{ type: string; url?: string; name?: string; snippet?: string }>; + sourceCount: number; + }> = []; + + if (direction === "outgoing" || direction === "both") { + const outgoing = await ctx.db + .query("influenceEdges") + .withIndex("by_from", (q) => q.eq("userId", args.userId).eq("fromArtistId", artistRecord._id)) + .collect(); + for (const edge of outgoing) { + if (edge.weight < minWeight) continue; + if (args.relationship && edge.relationship !== args.relationship) continue; + const target = await ctx.db.get(edge.toArtistId); + if (!target) continue; + const sources = await ctx.db.query("influenceEdgeSources").withIndex("by_edge", (q) => q.eq("edgeId", edge._id)).collect(); + connections.push({ + direction: "outgoing" as const, + artist: target.name, + genres: target.genres, + imageUrl: target.imageUrl, + relationship: edge.relationship, + weight: edge.weight, + mentionCount: edge.mentionCount ?? 1, + context: edge.context, + sources: sources.map((s) => ({ type: s.sourceType, url: s.sourceUrl, name: s.sourceName, snippet: s.snippet })), + sourceCount: sources.length, + }); + } + } + + if (direction === "incoming" || direction === "both") { + const incoming = await ctx.db + .query("influenceEdges") + .withIndex("by_to", (q) => q.eq("userId", args.userId).eq("toArtistId", artistRecord._id)) + .collect(); + for (const edge of incoming) { + if (edge.weight < minWeight) continue; + if (args.relationship && edge.relationship !== args.relationship) continue; + const source = await ctx.db.get(edge.fromArtistId); + if (!source) continue; + const sources = await ctx.db.query("influenceEdgeSources").withIndex("by_edge", (q) => q.eq("edgeId", edge._id)).collect(); + connections.push({ + direction: "incoming" as const, + artist: source.name, + genres: source.genres, + imageUrl: source.imageUrl, + relationship: edge.relationship, + weight: edge.weight, + mentionCount: edge.mentionCount ?? 1, + context: edge.context, + sources: sources.map((s) => ({ type: s.sourceType, url: s.sourceUrl, name: s.sourceName, snippet: s.snippet })), + sourceCount: sources.length, + }); + } + } + + // Sort by weight descending + connections.sort((a, b) => b.weight - a.weight); + + return { + artist: args.artist, + artistId: artistRecord._id, + genres: artistRecord.genres, + imageUrl: artistRecord.imageUrl, + connections, + cached: true, + }; + }, +}); + +// findPath - BFS pathfinding between two artists (JS-side, max depth) +export const findPath = query({ + args: { + userId: v.id("users"), + fromArtist: v.string(), + toArtist: v.string(), + maxDepth: v.optional(v.number()), + }, + handler: async (ctx, args) => { + const maxDepth = args.maxDepth ?? 4; + const fromLower = args.fromArtist.toLowerCase().trim(); + const toLower = args.toArtist.toLowerCase().trim(); + + const fromRecord = await ctx.db.query("influenceArtists").withIndex("by_user_name", (q) => q.eq("userId", args.userId).eq("nameLower", fromLower)).first(); + const toRecord = await ctx.db.query("influenceArtists").withIndex("by_user_name", (q) => q.eq("userId", args.userId).eq("nameLower", toLower)).first(); + + if (!fromRecord || !toRecord) return { found: false, path: [], hops: [] }; + if (fromRecord._id === toRecord._id) return { found: true, path: [fromRecord.name], hops: [] }; + + // BFS + type QueueItem = { artistId: string; path: string[]; hops: Array<{ from: string; to: string; relationship: string; weight: number; context?: string }> }; + const queue: QueueItem[] = [{ artistId: fromRecord._id, path: [fromRecord.name], hops: [] }]; + const visited = new Set([fromRecord._id]); + + while (queue.length > 0) { + const current = queue.shift()!; + if (current.path.length > maxDepth) continue; + + const edges = await ctx.db.query("influenceEdges") + .withIndex("by_from", (q) => q.eq("userId", args.userId).eq("fromArtistId", current.artistId as any)) + .collect(); + + // Also check incoming edges + const incomingEdges = await ctx.db.query("influenceEdges") + .withIndex("by_to", (q) => q.eq("userId", args.userId).eq("toArtistId", current.artistId as any)) + .collect(); + + const allEdges = [ + ...edges.map((e) => ({ neighborId: e.toArtistId, ...e })), + ...incomingEdges.map((e) => ({ neighborId: e.fromArtistId, ...e })), + ]; + + for (const edge of allEdges) { + if (visited.has(edge.neighborId)) continue; + visited.add(edge.neighborId); + + const neighbor = await ctx.db.get(edge.neighborId); + if (!neighbor) continue; + + const newPath = [...current.path, neighbor.name]; + const fromName = current.path[current.path.length - 1]!; + const newHops = [...current.hops, { + from: fromName, + to: neighbor.name, + relationship: edge.relationship, + weight: edge.weight, + context: edge.context, + }]; + + if (edge.neighborId === toRecord._id) { + return { found: true, path: newPath, hops: newHops }; + } + + queue.push({ artistId: edge.neighborId, path: newPath, hops: newHops }); + } + } + + return { found: false, path: [], hops: [] }; + }, +}); + +// searchArtists - fuzzy search cached artists +export const searchArtists = query({ + args: { + userId: v.id("users"), + query: v.string(), + }, + handler: async (ctx, args) => { + const queryLower = args.query.toLowerCase().trim(); + // Get all user's artists and filter (no full-text search index on this table) + const allArtists = await ctx.db.query("influenceArtists") + .withIndex("by_user_name", (q) => q.eq("userId", args.userId)) + .collect(); + + return allArtists + .filter((a) => a.nameLower.includes(queryLower)) + .slice(0, 20) + .map((a) => ({ id: a._id, name: a.name, genres: a.genres, imageUrl: a.imageUrl })); + }, +}); + +// graphStats - total artists, edges, sources +export const graphStats = query({ + args: { userId: v.id("users") }, + handler: async (ctx, args) => { + const artists = await ctx.db.query("influenceArtists") + .withIndex("by_user_name", (q) => q.eq("userId", args.userId)) + .collect(); + const edges = await ctx.db.query("influenceEdges") + .withIndex("by_user", (q) => q.eq("userId", args.userId)) + .collect(); + + const relationshipCounts: Record = {}; + for (const e of edges) { + relationshipCounts[e.relationship] = (relationshipCounts[e.relationship] ?? 0) + 1; + } + + return { + totalArtists: artists.length, + totalEdges: edges.length, + relationships: relationshipCounts, + }; + }, +}); + +// removeEdge - delete edge and its sources +export const removeEdge = mutation({ + args: { edgeId: v.id("influenceEdges") }, + handler: async (ctx, args) => { + const sources = await ctx.db.query("influenceEdgeSources") + .withIndex("by_edge", (q) => q.eq("edgeId", args.edgeId)) + .collect(); + for (const source of sources) { + await ctx.db.delete(source._id); + } + await ctx.db.delete(args.edgeId); + return { deleted: true }; + }, +}); + +// findPathDijkstra - Dijkstra's shortest path with paper's distance transform: d_ij = 1/(mentionCount + 1) +export const findPathDijkstra = query({ + args: { + userId: v.id("users"), + fromArtist: v.string(), + toArtist: v.string(), + maxDepth: v.optional(v.number()), + }, + handler: async (ctx, args) => { + const maxDepth = args.maxDepth ?? 6; + const fromLower = args.fromArtist.toLowerCase().trim(); + const toLower = args.toArtist.toLowerCase().trim(); + + const fromRecord = await ctx.db.query("influenceArtists") + .withIndex("by_user_name", (q) => q.eq("userId", args.userId).eq("nameLower", fromLower)).first(); + const toRecord = await ctx.db.query("influenceArtists") + .withIndex("by_user_name", (q) => q.eq("userId", args.userId).eq("nameLower", toLower)).first(); + + if (!fromRecord || !toRecord) return { found: false as const, path: [] as string[], hops: [] as Array<{ from: string; to: string; relationship: string; weight: number; mentionCount: number; context?: string; distance: number }>, totalDistance: 0 }; + if (fromRecord._id === toRecord._id) return { found: true as const, path: [fromRecord.name], hops: [] as Array<{ from: string; to: string; relationship: string; weight: number; mentionCount: number; context?: string; distance: number }>, totalDistance: 0 }; + + // Dijkstra with distance = 1/(mentionCount + 1) + const dist = new Map(); + const prev = new Map(); + const visited = new Set(); + + // Priority queue (simple array-based for small graphs) + const pq: Array<{ artistId: string; distance: number }> = []; + + dist.set(fromRecord._id, 0); + pq.push({ artistId: fromRecord._id, distance: 0 }); + + while (pq.length > 0) { + // Find min distance + pq.sort((a, b) => a.distance - b.distance); + const current = pq.shift()!; + + if (visited.has(current.artistId)) continue; + visited.add(current.artistId); + + if (current.artistId === toRecord._id) break; + if (visited.size > maxDepth * 50) break; // Safety limit + + // Get outgoing edges + const outgoing = await ctx.db.query("influenceEdges") + .withIndex("by_from", (q) => q.eq("userId", args.userId).eq("fromArtistId", current.artistId as any)) + .collect(); + const incoming = await ctx.db.query("influenceEdges") + .withIndex("by_to", (q) => q.eq("userId", args.userId).eq("toArtistId", current.artistId as any)) + .collect(); + + const allEdges = [ + ...outgoing.map((e) => ({ neighborId: e.toArtistId, mentionCount: e.mentionCount ?? 1, relationship: e.relationship, weight: e.weight, context: e.context })), + ...incoming.map((e) => ({ neighborId: e.fromArtistId, mentionCount: e.mentionCount ?? 1, relationship: e.relationship, weight: e.weight, context: e.context })), + ]; + + for (const edge of allEdges) { + if (visited.has(edge.neighborId)) continue; + // Paper's distance transform: d = 1/(mentionCount + 1) + const edgeDistance = 1 / (edge.mentionCount + 1); + const newDist = current.distance + edgeDistance; + const currentDist = dist.get(edge.neighborId) ?? Infinity; + + if (newDist < currentDist) { + dist.set(edge.neighborId, newDist); + prev.set(edge.neighborId, { + artistId: current.artistId, + edge: { relationship: edge.relationship, weight: edge.weight, mentionCount: edge.mentionCount, context: edge.context }, + }); + pq.push({ artistId: edge.neighborId, distance: newDist }); + } + } + } + + // Reconstruct path + if (!prev.has(toRecord._id)) return { found: false as const, path: [] as string[], hops: [] as Array<{ from: string; to: string; relationship: string; weight: number; mentionCount: number; context?: string; distance: number }>, totalDistance: 0 }; + + const pathIds: string[] = [toRecord._id]; + const hops: Array<{ from: string; to: string; relationship: string; weight: number; mentionCount: number; context?: string; distance: number }> = []; + + let currentId: string = toRecord._id; + while (prev.has(currentId)) { + const { artistId, edge } = prev.get(currentId)!; + pathIds.unshift(artistId); + const fromArtist = await ctx.db.get(artistId as any); + const toArtist = await ctx.db.get(currentId as any); + hops.unshift({ + from: (fromArtist as any)?.name ?? "Unknown", + to: (toArtist as any)?.name ?? "Unknown", + relationship: edge.relationship, + weight: edge.weight, + mentionCount: edge.mentionCount, + context: edge.context, + distance: 1 / (edge.mentionCount + 1), + }); + currentId = artistId; + } + + // Get artist names for path + const pathNames: string[] = []; + for (const id of pathIds) { + const artist = await ctx.db.get(id as any); + pathNames.push((artist as any)?.name ?? "Unknown"); + } + + return { + found: true as const, + path: pathNames, + hops, + totalDistance: dist.get(toRecord._id) ?? 0, + }; + }, +}); + +// upsertTagProfile - store Last.fm tag vector for an artist +export const upsertTagProfile = mutation({ + args: { + userId: v.id("users"), + artistName: v.string(), + tags: v.string(), // JSON string of tag weights + }, + handler: async (ctx, args) => { + const nameLower = args.artistName.toLowerCase().trim(); + const artist = await ctx.db.query("influenceArtists") + .withIndex("by_user_name", (q) => q.eq("userId", args.userId).eq("nameLower", nameLower)) + .first(); + if (!artist) return null; + + const existing = await ctx.db.query("artistTagProfiles") + .withIndex("by_artist", (q) => q.eq("artistId", artist._id)) + .first(); + + if (existing) { + await ctx.db.patch(existing._id, { tags: args.tags, fetchedAt: Date.now() }); + return existing._id; + } + + return await ctx.db.insert("artistTagProfiles", { + userId: args.userId, + artistId: artist._id, + tags: args.tags, + fetchedAt: Date.now(), + }); + }, +}); + +// getTagSimilarity - cosine similarity between two artists' Last.fm tag vectors +export const getTagSimilarity = query({ + args: { + userId: v.id("users"), + artist1: v.string(), + artist2: v.string(), + }, + handler: async (ctx, args) => { + const get = async (name: string) => { + const nameLower = name.toLowerCase().trim(); + const artist = await ctx.db.query("influenceArtists") + .withIndex("by_user_name", (q) => q.eq("userId", args.userId).eq("nameLower", nameLower)) + .first(); + if (!artist) return null; + const profile = await ctx.db.query("artistTagProfiles") + .withIndex("by_artist", (q) => q.eq("artistId", artist._id)) + .first(); + return profile; + }; + + const p1 = await get(args.artist1); + const p2 = await get(args.artist2); + + if (!p1 || !p2) return { similarity: null, reason: !p1 ? `No tag profile for ${args.artist1}` : `No tag profile for ${args.artist2}` }; + + const tags1: Record = JSON.parse(p1.tags); + const tags2: Record = JSON.parse(p2.tags); + + // Cosine similarity + const allTags = new Set([...Object.keys(tags1), ...Object.keys(tags2)]); + let dotProduct = 0; + let norm1 = 0; + let norm2 = 0; + for (const tag of allTags) { + const v1 = tags1[tag] ?? 0; + const v2 = tags2[tag] ?? 0; + dotProduct += v1 * v2; + norm1 += v1 * v1; + norm2 += v2 * v2; + } + + const similarity = norm1 === 0 || norm2 === 0 ? 0 : dotProduct / (Math.sqrt(norm1) * Math.sqrt(norm2)); + + // Find shared and unique tags + const shared = [...allTags].filter((t) => (tags1[t] ?? 0) > 0 && (tags2[t] ?? 0) > 0); + + return { similarity: Math.round(similarity * 1000) / 1000, sharedTags: shared, artist1Tags: Object.keys(tags1).length, artist2Tags: Object.keys(tags2).length }; + }, +}); diff --git a/convex/keys.ts b/convex/keys.ts new file mode 100644 index 0000000..41d99cd --- /dev/null +++ b/convex/keys.ts @@ -0,0 +1,22 @@ +import { mutation, query } from "./_generated/server"; +import { v } from "convex/values"; + +export const store = mutation({ + args: { + userId: v.id("users"), + encryptedKeys: v.bytes(), + }, + handler: async (ctx, args) => { + await ctx.db.patch(args.userId, { + encryptedKeys: args.encryptedKeys, + }); + }, +}); + +export const get = query({ + args: { userId: v.id("users") }, + handler: async (ctx, args) => { + const user = await ctx.db.get(args.userId); + return user?.encryptedKeys ?? null; + }, +}); diff --git a/convex/messages.ts b/convex/messages.ts new file mode 100644 index 0000000..94e9320 --- /dev/null +++ b/convex/messages.ts @@ -0,0 +1,51 @@ +import { mutation, query } from "./_generated/server"; +import { v } from "convex/values"; + +export const send = mutation({ + args: { + sessionId: v.id("sessions"), + role: v.union(v.literal("user"), v.literal("assistant")), + content: v.string(), + }, + handler: async (ctx, args) => { + const id = await ctx.db.insert("messages", { + sessionId: args.sessionId, + role: args.role, + content: args.content, + createdAt: Date.now(), + }); + const now = Date.now(); + await ctx.db.patch(args.sessionId, { + lastMessageAt: now, + updatedAt: now, + }); + return id; + }, +}); + +export const list = query({ + args: { + sessionId: v.id("sessions"), + }, + handler: async (ctx, args) => { + return await ctx.db + .query("messages") + .withIndex("by_session", (q) => q.eq("sessionId", args.sessionId)) + .order("asc") + .collect(); + }, +}); + +export const search = query({ + args: { + query: v.string(), + }, + handler: async (ctx, args) => { + if (!args.query.trim()) return []; + const results = await ctx.db + .query("messages") + .withSearchIndex("search_content", (q) => q.search("content", args.query)) + .take(50); + return results; + }, +}); diff --git a/convex/orgKeys.ts b/convex/orgKeys.ts new file mode 100644 index 0000000..3cf6531 --- /dev/null +++ b/convex/orgKeys.ts @@ -0,0 +1,47 @@ +import { mutation, query } from "./_generated/server"; +import { v } from "convex/values"; + +export const getByDomain = query({ + args: { domain: v.string() }, + handler: async (ctx, args) => { + return await ctx.db + .query("orgKeys") + .withIndex("by_domain", (q) => q.eq("domain", args.domain)) + .first(); + }, +}); + +export const store = mutation({ + args: { + domain: v.string(), + encryptedKeys: v.bytes(), + adminUserId: v.id("users"), + }, + handler: async (ctx, args) => { + const existing = await ctx.db + .query("orgKeys") + .withIndex("by_domain", (q) => q.eq("domain", args.domain)) + .first(); + + const now = Date.now(); + if (existing) { + // Only the original admin can update + if (existing.adminUserId !== args.adminUserId) { + throw new Error("Only the org admin can update shared keys"); + } + await ctx.db.patch(existing._id, { + encryptedKeys: args.encryptedKeys, + updatedAt: now, + }); + return existing._id; + } + + return await ctx.db.insert("orgKeys", { + domain: args.domain, + encryptedKeys: args.encryptedKeys, + adminUserId: args.adminUserId, + createdAt: now, + updatedAt: now, + }); + }, +}); diff --git a/convex/playlists.ts b/convex/playlists.ts new file mode 100644 index 0000000..5b07af1 --- /dev/null +++ b/convex/playlists.ts @@ -0,0 +1,197 @@ +import { mutation, query } from "./_generated/server"; +import { v } from "convex/values"; + +export const list = query({ + args: { userId: v.id("users") }, + handler: async (ctx, args) => { + return await ctx.db + .query("playlists") + .withIndex("by_user", (q) => q.eq("userId", args.userId)) + .order("desc") + .collect(); + }, +}); + +export const get = query({ + args: { id: v.id("playlists") }, + handler: async (ctx, args) => { + return await ctx.db.get(args.id); + }, +}); + +export const findByName = query({ + args: { userId: v.id("users"), name: v.string() }, + handler: async (ctx, args) => { + const all = await ctx.db + .query("playlists") + .withIndex("by_user", (q) => q.eq("userId", args.userId)) + .collect(); + return all.find( + (p) => p.name.toLowerCase() === args.name.toLowerCase(), + ) ?? null; + }, +}); + +export const getTracks = query({ + args: { playlistId: v.id("playlists") }, + handler: async (ctx, args) => { + return await ctx.db + .query("playlistTracks") + .withIndex("by_playlist", (q) => q.eq("playlistId", args.playlistId)) + .collect(); + }, +}); + +export const create = mutation({ + args: { + userId: v.id("users"), + name: v.string(), + description: v.optional(v.string()), + }, + handler: async (ctx, args) => { + const now = Date.now(); + return await ctx.db.insert("playlists", { + userId: args.userId, + name: args.name, + description: args.description, + trackCount: 0, + createdAt: now, + updatedAt: now, + }); + }, +}); + +export const addTrack = mutation({ + args: { + playlistId: v.id("playlists"), + title: v.string(), + artist: v.string(), + album: v.optional(v.string()), + year: v.optional(v.string()), + source: v.union(v.literal("youtube"), v.literal("bandcamp"), v.literal("unknown")), + sourceId: v.optional(v.string()), + imageUrl: v.optional(v.string()), + }, + handler: async (ctx, args) => { + const playlist = await ctx.db.get(args.playlistId); + if (!playlist) throw new Error("Playlist not found"); + + const position = playlist.trackCount; + await ctx.db.insert("playlistTracks", { + playlistId: args.playlistId, + title: args.title, + artist: args.artist, + album: args.album, + year: args.year, + source: args.source, + sourceId: args.sourceId, + imageUrl: args.imageUrl, + position, + addedAt: Date.now(), + }); + + await ctx.db.patch(args.playlistId, { + trackCount: position + 1, + updatedAt: Date.now(), + }); + }, +}); + +export const addMultipleTracks = mutation({ + args: { + playlistId: v.id("playlists"), + tracks: v.array( + v.object({ + title: v.string(), + artist: v.string(), + album: v.optional(v.string()), + year: v.optional(v.string()), + imageUrl: v.optional(v.string()), + }), + ), + }, + handler: async (ctx, args) => { + const playlist = await ctx.db.get(args.playlistId); + if (!playlist) throw new Error("Playlist not found"); + + let position = playlist.trackCount; + for (const track of args.tracks) { + await ctx.db.insert("playlistTracks", { + playlistId: args.playlistId, + title: track.title, + artist: track.artist, + album: track.album, + year: track.year, + imageUrl: track.imageUrl, + source: "unknown", + position, + addedAt: Date.now(), + }); + position++; + } + + await ctx.db.patch(args.playlistId, { + trackCount: position, + updatedAt: Date.now(), + }); + }, +}); + +export const removeTrack = mutation({ + args: { trackId: v.id("playlistTracks") }, + handler: async (ctx, args) => { + const track = await ctx.db.get(args.trackId); + if (!track) return; + + await ctx.db.delete(args.trackId); + + const playlist = await ctx.db.get(track.playlistId); + if (playlist) { + await ctx.db.patch(track.playlistId, { + trackCount: Math.max(0, playlist.trackCount - 1), + updatedAt: Date.now(), + }); + } + }, +}); + +export const rename = mutation({ + args: { id: v.id("playlists"), name: v.string() }, + handler: async (ctx, args) => { + await ctx.db.patch(args.id, { name: args.name, updatedAt: Date.now() }); + }, +}); + +export const remove = mutation({ + args: { id: v.id("playlists") }, + handler: async (ctx, args) => { + const tracks = await ctx.db + .query("playlistTracks") + .withIndex("by_playlist", (q) => q.eq("playlistId", args.id)) + .collect(); + for (const track of tracks) { + await ctx.db.delete(track._id); + } + await ctx.db.delete(args.id); + }, +}); + +export const removeAll = mutation({ + args: { userId: v.id("users") }, + handler: async (ctx, args) => { + const playlists = await ctx.db + .query("playlists") + .withIndex("by_user", (q) => q.eq("userId", args.userId)) + .collect(); + for (const pl of playlists) { + const tracks = await ctx.db + .query("playlistTracks") + .withIndex("by_playlist", (q) => q.eq("playlistId", pl._id)) + .collect(); + for (const track of tracks) { + await ctx.db.delete(track._id); + } + await ctx.db.delete(pl._id); + } + }, +}); diff --git a/convex/published.ts b/convex/published.ts new file mode 100644 index 0000000..113a0aa --- /dev/null +++ b/convex/published.ts @@ -0,0 +1,64 @@ +import { v } from "convex/values"; +import { query } from "./_generated/server"; + +/** List all published items (Telegraph entries + Tumblr posts) merged and sorted by date. */ +export const listAll = query({ + args: { userId: v.id("users") }, + handler: async (ctx, args) => { + const [telegraphEntries, tumblrPosts] = await Promise.all([ + ctx.db + .query("telegraphEntries") + .withIndex("by_user", (q) => q.eq("userId", args.userId)) + .collect(), + ctx.db + .query("tumblrPosts") + .withIndex("by_user", (q) => q.eq("userId", args.userId)) + .collect(), + ]); + + const items = [ + ...telegraphEntries.map((e) => ({ + _id: e._id, + platform: "telegraph" as const, + title: e.title, + url: e.telegraphUrl, + category: e.category, + createdAt: e.createdAt, + })), + ...tumblrPosts.map((p) => ({ + _id: p._id, + platform: "tumblr" as const, + title: p.title, + url: p.postUrl, + category: p.category, + createdAt: p.createdAt, + })), + ]; + + items.sort((a, b) => b.createdAt - a.createdAt); + return items; + }, +}); + +/** Count of published items per platform. */ +export const stats = query({ + args: { userId: v.id("users") }, + handler: async (ctx, args) => { + const [telegraphEntries, tumblrPosts] = await Promise.all([ + ctx.db + .query("telegraphEntries") + .withIndex("by_user", (q) => q.eq("userId", args.userId)) + .collect(), + ctx.db + .query("tumblrPosts") + .withIndex("by_user", (q) => q.eq("userId", args.userId)) + .collect(), + ]); + + return { + total: telegraphEntries.length + tumblrPosts.length, + telegraph: telegraphEntries.length, + tumblr: tumblrPosts.length, + }; + }, +}); diff --git a/convex/schema.ts b/convex/schema.ts new file mode 100644 index 0000000..dcff044 --- /dev/null +++ b/convex/schema.ts @@ -0,0 +1,380 @@ +import { defineSchema, defineTable } from "convex/server"; +import { v } from "convex/values"; + +export default defineSchema({ + users: defineTable({ + clerkId: v.string(), + email: v.string(), + name: v.optional(v.string()), + usernameSlug: v.optional(v.string()), + encryptedKeys: v.optional(v.bytes()), + onboardingCompleted: v.optional(v.boolean()), + helpPersona: v.optional(v.string()), + stripeCustomerId: v.optional(v.string()), + createdAt: v.number(), + }) + .index("by_clerk_id", ["clerkId"]) + .index("by_username_slug", ["usernameSlug"]), + + crates: defineTable({ + userId: v.id("users"), + name: v.string(), + color: v.optional(v.string()), + createdAt: v.number(), + }).index("by_user", ["userId"]), + + sessions: defineTable({ + userId: v.id("users"), + crateId: v.optional(v.id("crates")), + title: v.optional(v.string()), + isShared: v.boolean(), + isStarred: v.boolean(), + isArchived: v.boolean(), + lastMessageAt: v.number(), + createdAt: v.number(), + updatedAt: v.number(), + }) + .index("by_user", ["userId"]) + .index("by_user_starred", ["userId", "isStarred"]) + .index("by_user_crate", ["userId", "crateId"]) + .index("by_user_recent", ["userId", "lastMessageAt"]), + + messages: defineTable({ + sessionId: v.id("sessions"), + role: v.union(v.literal("user"), v.literal("assistant")), + content: v.string(), + createdAt: v.number(), + }) + .index("by_session", ["sessionId"]) + .searchIndex("search_content", { + searchField: "content", + filterFields: ["sessionId"], + }), + + artifacts: defineTable({ + sessionId: v.id("sessions"), + userId: v.id("users"), + messageId: v.optional(v.id("messages")), + type: v.string(), + label: v.string(), + data: v.string(), + contentHash: v.string(), + createdAt: v.number(), + }) + .index("by_session", ["sessionId"]) + .index("by_user", ["userId"]), + + toolCalls: defineTable({ + sessionId: v.id("sessions"), + messageId: v.optional(v.id("messages")), + toolName: v.string(), + args: v.string(), + result: v.optional(v.string()), + status: v.union( + v.literal("running"), + v.literal("complete"), + v.literal("error"), + ), + startedAt: v.number(), + completedAt: v.optional(v.number()), + }).index("by_session", ["sessionId"]), + + playerQueue: defineTable({ + sessionId: v.id("sessions"), + tracks: v.array( + v.object({ + title: v.string(), + artist: v.string(), + source: v.union(v.literal("youtube"), v.literal("bandcamp")), + sourceId: v.string(), + imageUrl: v.optional(v.string()), + }), + ), + currentIndex: v.number(), + }).index("by_session", ["sessionId"]), + + playlists: defineTable({ + userId: v.id("users"), + name: v.string(), + description: v.optional(v.string()), + coverUrl: v.optional(v.string()), + trackCount: v.number(), + totalDurationMs: v.optional(v.number()), + createdAt: v.number(), + updatedAt: v.number(), + }).index("by_user", ["userId"]), + + playlistTracks: defineTable({ + playlistId: v.id("playlists"), + title: v.string(), + artist: v.string(), + album: v.optional(v.string()), + year: v.optional(v.string()), + source: v.union(v.literal("youtube"), v.literal("bandcamp"), v.literal("unknown")), + sourceId: v.optional(v.string()), + imageUrl: v.optional(v.string()), + position: v.number(), + addedAt: v.number(), + }).index("by_playlist", ["playlistId", "position"]), + + collection: defineTable({ + userId: v.id("users"), + title: v.string(), + artist: v.string(), + label: v.optional(v.string()), + year: v.optional(v.string()), + format: v.optional(v.string()), + genre: v.optional(v.string()), + notes: v.optional(v.string()), + imageUrl: v.optional(v.string()), + discogsId: v.optional(v.string()), + rating: v.optional(v.number()), + createdAt: v.number(), + }) + .index("by_user", ["userId"]) + .searchIndex("search_collection", { + searchField: "title", + filterFields: ["userId"], + }), + + // Publishing: Telegraph (per-user anonymous accounts) + telegraphAuth: defineTable({ + userId: v.id("users"), + accessToken: v.string(), + authorName: v.string(), + authorUrl: v.optional(v.string()), + indexPagePath: v.optional(v.string()), + indexPageUrl: v.optional(v.string()), + createdAt: v.number(), + }).index("by_user", ["userId"]), + + telegraphEntries: defineTable({ + userId: v.id("users"), + title: v.string(), + telegraphPath: v.string(), + telegraphUrl: v.string(), + category: v.optional(v.string()), + createdAt: v.number(), + }).index("by_user", ["userId"]), + + // Publishing: Tumblr (per-user OAuth 1.0a credentials) + tumblrAuth: defineTable({ + userId: v.id("users"), + oauthToken: v.string(), + oauthTokenSecret: v.string(), + blogName: v.string(), + blogUrl: v.string(), + blogUuid: v.string(), + createdAt: v.number(), + }).index("by_user", ["userId"]), + + tumblrPosts: defineTable({ + userId: v.id("users"), + tumblrPostId: v.string(), + title: v.string(), + blogName: v.string(), + postUrl: v.string(), + category: v.optional(v.string()), + tags: v.optional(v.string()), // JSON array + createdAt: v.number(), + }).index("by_user", ["userId"]), + + orgKeys: defineTable({ + domain: v.string(), + encryptedKeys: v.bytes(), + adminUserId: v.id("users"), + createdAt: v.number(), + updatedAt: v.number(), + }).index("by_domain", ["domain"]), + + // Influence mapping cache + influenceArtists: defineTable({ + userId: v.id("users"), + name: v.string(), + nameLower: v.string(), + genres: v.optional(v.string()), + imageUrl: v.optional(v.string()), + createdAt: v.number(), + }).index("by_user_name", ["userId", "nameLower"]), + + influenceEdges: defineTable({ + userId: v.id("users"), + fromArtistId: v.id("influenceArtists"), + toArtistId: v.id("influenceArtists"), + relationship: v.string(), + weight: v.number(), + mentionCount: v.optional(v.number()), + context: v.optional(v.string()), + createdAt: v.number(), + updatedAt: v.number(), + }).index("by_user", ["userId"]) + .index("by_from", ["userId", "fromArtistId"]) + .index("by_to", ["userId", "toArtistId"]), + + artistTagProfiles: defineTable({ + userId: v.id("users"), + artistId: v.id("influenceArtists"), + tags: v.string(), // JSON object: {"electronic": 100, "ambient": 85, "idm": 60} + fetchedAt: v.number(), + }).index("by_artist", ["artistId"]) + .index("by_user", ["userId"]), + + sonicProfiles: defineTable({ + userId: v.id("users"), + artistId: v.id("influenceArtists"), + features: v.string(), // JSON: {tempo, energy, loudness, danceability, speechiness, ...} + source: v.string(), // "lastfm_tags" | "essentia" | "manual" + fetchedAt: v.number(), + }).index("by_artist", ["artistId"]) + .index("by_user", ["userId"]), + + influenceEdgeSources: defineTable({ + edgeId: v.id("influenceEdges"), + sourceType: v.string(), + sourceUrl: v.optional(v.string()), + sourceName: v.optional(v.string()), + snippet: v.optional(v.string()), + discoveredAt: v.number(), + }).index("by_edge", ["edgeId"]), + + subscriptions: defineTable({ + userId: v.id("users"), + plan: v.union(v.literal("free"), v.literal("pro"), v.literal("team")), + status: v.union(v.literal("active"), v.literal("canceled"), v.literal("past_due")), + stripeCustomerId: v.optional(v.string()), + stripeSubscriptionId: v.optional(v.string()), + currentPeriodStart: v.number(), + currentPeriodEnd: v.number(), + cancelAtPeriodEnd: v.boolean(), + teamDomain: v.optional(v.string()), + createdAt: v.number(), + updatedAt: v.number(), + }).index("by_user", ["userId"]) + .index("by_stripe_sub", ["stripeSubscriptionId"]) + .index("by_team_domain", ["teamDomain"]), + + usageEvents: defineTable({ + userId: v.id("users"), + type: v.union(v.literal("agent_query"), v.literal("chat_query")), + teamDomain: v.optional(v.string()), + periodStart: v.number(), + createdAt: v.number(), + }).index("by_user_period", ["userId", "periodStart"]) + .index("by_user_type_period", ["userId", "type", "periodStart"]) + .index("by_team_domain_period", ["teamDomain", "periodStart"]), + + userSkills: defineTable({ + userId: v.id("users"), + command: v.string(), + name: v.string(), + description: v.string(), + triggerPattern: v.optional(v.string()), + promptTemplate: v.string(), + toolHints: v.array(v.string()), + sourceUrl: v.optional(v.string()), + lastResults: v.optional(v.string()), + gotchas: v.optional(v.string()), + runCount: v.number(), + visibility: v.literal("private"), + isEnabled: v.boolean(), + schedule: v.optional(v.string()), + lastRunAt: v.optional(v.number()), + createdAt: v.number(), + updatedAt: v.number(), + }).index("by_user", ["userId"]) + .index("by_user_command", ["userId", "command"]), + + // Deep Cuts: published/shared research artifacts + shares: defineTable({ + shareId: v.string(), + artifactId: v.id("artifacts"), + userId: v.id("users"), + label: v.string(), + type: v.string(), + data: v.string(), + isPublic: v.boolean(), + createdAt: v.number(), + }) + .index("by_share_id", ["shareId"]) + .index("by_user", ["userId"]) + .index("by_artifact", ["artifactId"]), + + tinydeskCompanions: defineTable({ + slug: v.string(), + artist: v.string(), + tagline: v.string(), + tinyDeskVideoId: v.string(), + nodes: v.string(), + userId: v.id("users"), + genre: v.optional(v.array(v.string())), + sourceUrl: v.optional(v.string()), + isCommunitySubmitted: v.optional(v.boolean()), + createdAt: v.number(), + }) + .index("by_slug", ["slug"]) + .index("by_user", ["userId"]), + + // Music Wiki: persistent, compounding artist knowledge base + wikiPages: defineTable({ + userId: v.id("users"), + slug: v.string(), + entityType: v.literal("artist"), + entityName: v.string(), + description: v.optional(v.string()), + sections: v.array(v.object({ + heading: v.string(), + content: v.string(), + sources: v.array(v.object({ + tool: v.string(), + url: v.optional(v.string()), + fetchedAt: v.number(), + })), + lastSynthesizedAt: v.optional(v.number()), + })), + contradictions: v.array(v.object({ + claim1: v.object({ source: v.string(), value: v.string() }), + claim2: v.object({ source: v.string(), value: v.string() }), + field: v.string(), + })), + metadata: v.object({ + origin: v.optional(v.string()), + yearsActive: v.optional(v.string()), + members: v.optional(v.array(v.string())), + genreDNA: v.optional(v.array(v.string())), + }), + visibility: v.union(v.literal("private"), v.literal("unlisted"), v.literal("public")), + archivedAt: v.optional(v.number()), + createdAt: v.number(), + updatedAt: v.number(), + }) + .index("by_user", ["userId"]) + .index("by_user_slug", ["userId", "slug"]) + .index("by_slug", ["slug"]), + + wikiIndexEntries: defineTable({ + userId: v.id("users"), + pageId: v.id("wikiPages"), + slug: v.string(), + entityName: v.string(), + entityType: v.string(), + summary: v.optional(v.string()), + visibility: v.optional(v.string()), + sourceCount: v.number(), + lastUpdated: v.number(), + }) + .index("by_user", ["userId"]) + .index("by_user_slug", ["userId", "slug"]), + + wikiLogEntries: defineTable({ + userId: v.id("users"), + timestamp: v.number(), + operation: v.union( + v.literal("ingest"), v.literal("query"), + v.literal("lint"), v.literal("synthesize"), + ), + entitySlug: v.optional(v.string()), + description: v.string(), + toolsUsed: v.optional(v.array(v.string())), + }) + .index("by_user_time", ["userId", "timestamp"]), +}); diff --git a/convex/sessions.ts b/convex/sessions.ts new file mode 100644 index 0000000..3289f63 --- /dev/null +++ b/convex/sessions.ts @@ -0,0 +1,237 @@ +import { mutation, query, internalMutation } from "./_generated/server"; +import { v } from "convex/values"; + +export const create = mutation({ + args: { + userId: v.id("users"), + }, + handler: async (ctx, args) => { + const now = Date.now(); + return await ctx.db.insert("sessions", { + userId: args.userId, + isShared: false, + isStarred: false, + isArchived: false, + lastMessageAt: now, + createdAt: now, + updatedAt: now, + }); + }, +}); + +export const listRecent = query({ + args: { userId: v.id("users"), limit: v.optional(v.number()) }, + handler: async (ctx, args) => { + const limit = args.limit ?? 20; + return await ctx.db + .query("sessions") + .withIndex("by_user_recent", (q) => q.eq("userId", args.userId)) + .order("desc") + .filter((q) => q.eq(q.field("isArchived"), false)) + .take(limit); + }, +}); + +export const listStarred = query({ + args: { userId: v.id("users") }, + handler: async (ctx, args) => { + return await ctx.db + .query("sessions") + .withIndex("by_user_starred", (q) => + q.eq("userId", args.userId).eq("isStarred", true), + ) + .filter((q) => q.eq(q.field("isArchived"), false)) + .collect(); + }, +}); + +export const listByCrate = query({ + args: { userId: v.id("users"), crateId: v.id("crates") }, + handler: async (ctx, args) => { + return await ctx.db + .query("sessions") + .withIndex("by_user_crate", (q) => + q.eq("userId", args.userId).eq("crateId", args.crateId), + ) + .filter((q) => q.eq(q.field("isArchived"), false)) + .collect(); + }, +}); + +export const get = query({ + args: { id: v.id("sessions") }, + handler: async (ctx, args) => { + return await ctx.db.get(args.id); + }, +}); + +export const updateTitle = mutation({ + args: { + id: v.id("sessions"), + title: v.string(), + }, + handler: async (ctx, args) => { + await ctx.db.patch(args.id, { + title: args.title, + updatedAt: Date.now(), + }); + }, +}); + +export const toggleStar = mutation({ + args: { id: v.id("sessions") }, + handler: async (ctx, args) => { + const session = await ctx.db.get(args.id); + if (!session) throw new Error("Session not found"); + await ctx.db.patch(args.id, { isStarred: !session.isStarred }); + }, +}); + +export const assignToCrate = mutation({ + args: { + id: v.id("sessions"), + crateId: v.optional(v.id("crates")), + }, + handler: async (ctx, args) => { + await ctx.db.patch(args.id, { + crateId: args.crateId, + updatedAt: Date.now(), + }); + }, +}); + +export const archive = mutation({ + args: { id: v.id("sessions") }, + handler: async (ctx, args) => { + await ctx.db.patch(args.id, { + isArchived: true, + updatedAt: Date.now(), + }); + }, +}); + +export const toggleShare = mutation({ + args: { id: v.id("sessions") }, + handler: async (ctx, args) => { + const session = await ctx.db.get(args.id); + if (!session) throw new Error("Session not found"); + await ctx.db.patch(args.id, { isShared: !session.isShared }); + }, +}); + +export const remove = mutation({ + args: { id: v.id("sessions") }, + handler: async (ctx, args) => { + // Delete all messages for this session + const messages = await ctx.db + .query("messages") + .withIndex("by_session", (q) => q.eq("sessionId", args.id)) + .collect(); + for (const m of messages) { + await ctx.db.delete(m._id); + } + + // Delete all artifacts for this session + const artifacts = await ctx.db + .query("artifacts") + .withIndex("by_session", (q) => q.eq("sessionId", args.id)) + .collect(); + for (const a of artifacts) { + await ctx.db.delete(a._id); + } + + // Delete all tool calls for this session + const toolCalls = await ctx.db + .query("toolCalls") + .withIndex("by_session", (q) => q.eq("sessionId", args.id)) + .collect(); + for (const t of toolCalls) { + await ctx.db.delete(t._id); + } + + // Delete the session itself + await ctx.db.delete(args.id); + }, +}); + +export const touchLastMessage = mutation({ + args: { id: v.id("sessions") }, + handler: async (ctx, args) => { + const now = Date.now(); + await ctx.db.patch(args.id, { + lastMessageAt: now, + updatedAt: now, + }); + }, +}); + +/** + * For free-tier users: if they have more than maxSessions non-starred, + * non-archived sessions, delete the oldest ones. + */ +export const enforceSessionLimit = internalMutation({ + args: { + userId: v.id("users"), + maxSessions: v.number(), + }, + handler: async (ctx, { userId, maxSessions }) => { + // Get all sessions for this user + const allSessions = await ctx.db + .query("sessions") + .withIndex("by_user", (q) => q.eq("userId", userId)) + .collect(); + + // Filter to deletable sessions (not starred, not archived) + const deletable = allSessions + .filter((s) => !s.isStarred && !s.isArchived) + .sort((a, b) => a.lastMessageAt - b.lastMessageAt); + + // Only count non-starred, non-archived sessions against the limit + // (starred and archived sessions are exempt) + if (deletable.length <= maxSessions) return { deleted: 0 }; + + // Delete oldest to get under limit + const toDelete = deletable.slice(0, deletable.length - maxSessions); + for (const session of toDelete) { + // Delete messages first + const messages = await ctx.db + .query("messages") + .withIndex("by_session", (q) => q.eq("sessionId", session._id)) + .collect(); + for (const msg of messages) { + await ctx.db.delete(msg._id); + } + + // Delete all artifacts for this session + const artifacts = await ctx.db + .query("artifacts") + .withIndex("by_session", (q) => q.eq("sessionId", session._id)) + .collect(); + for (const a of artifacts) { + await ctx.db.delete(a._id); + } + + // Delete all tool calls for this session + const toolCalls = await ctx.db + .query("toolCalls") + .withIndex("by_session", (q) => q.eq("sessionId", session._id)) + .collect(); + for (const t of toolCalls) { + await ctx.db.delete(t._id); + } + + // Delete player queue + const queues = await ctx.db + .query("playerQueue") + .withIndex("by_session", (q) => q.eq("sessionId", session._id)) + .collect(); + for (const queue of queues) { + await ctx.db.delete(queue._id); + } + + await ctx.db.delete(session._id); + } + + return { deleted: toDelete.length }; + }, +}); diff --git a/convex/shares.ts b/convex/shares.ts new file mode 100644 index 0000000..b5696ca --- /dev/null +++ b/convex/shares.ts @@ -0,0 +1,59 @@ +import { query, mutation } from "./_generated/server"; +import { v } from "convex/values"; + +export const create = mutation({ + args: { + shareId: v.string(), + artifactId: v.id("artifacts"), + userId: v.id("users"), + label: v.string(), + type: v.string(), + data: v.string(), + }, + handler: async (ctx, args) => { + return await ctx.db.insert("shares", { + ...args, + isPublic: true, + createdAt: Date.now(), + }); + }, +}); + +export const getByShareId = query({ + args: { shareId: v.string() }, + handler: async (ctx, { shareId }) => { + return await ctx.db + .query("shares") + .withIndex("by_share_id", (q) => q.eq("shareId", shareId)) + .first(); + }, +}); + +export const getByArtifact = query({ + args: { artifactId: v.id("artifacts") }, + handler: async (ctx, { artifactId }) => { + return await ctx.db + .query("shares") + .withIndex("by_artifact", (q) => q.eq("artifactId", artifactId)) + .first(); + }, +}); + +export const listByUser = query({ + args: { userId: v.id("users") }, + handler: async (ctx, { userId }) => { + return await ctx.db + .query("shares") + .withIndex("by_user", (q) => q.eq("userId", userId)) + .collect(); + }, +}); + +export const unpublish = mutation({ + args: { shareId: v.id("shares") }, + handler: async (ctx, { shareId }) => { + await ctx.db.patch(shareId, { + isPublic: false, + }); + }, +}); diff --git a/convex/subscriptions.ts b/convex/subscriptions.ts new file mode 100644 index 0000000..d94e1a9 --- /dev/null +++ b/convex/subscriptions.ts @@ -0,0 +1,81 @@ +import { query, mutation } from "./_generated/server"; +import { v } from "convex/values"; + +export const getByUserId = query({ + args: { userId: v.id("users") }, + handler: async (ctx, { userId }) => { + return await ctx.db + .query("subscriptions") + .withIndex("by_user", (q) => q.eq("userId", userId)) + .first(); + }, +}); + +export const getByStripeSub = query({ + args: { stripeSubscriptionId: v.string() }, + handler: async (ctx, { stripeSubscriptionId }) => { + return await ctx.db + .query("subscriptions") + .withIndex("by_stripe_sub", (q) => q.eq("stripeSubscriptionId", stripeSubscriptionId)) + .first(); + }, +}); + +export const create = mutation({ + args: { + userId: v.id("users"), + plan: v.union(v.literal("free"), v.literal("pro"), v.literal("team")), + stripeCustomerId: v.optional(v.string()), + stripeSubscriptionId: v.optional(v.string()), + currentPeriodStart: v.number(), + currentPeriodEnd: v.number(), + teamDomain: v.optional(v.string()), + }, + handler: async (ctx, args) => { + // Prevent duplicate subscriptions + const existing = await ctx.db + .query("subscriptions") + .withIndex("by_user", (q) => q.eq("userId", args.userId)) + .first(); + if (existing) { + throw new Error("User already has a subscription. Use update instead."); + } + + const now = Date.now(); + return await ctx.db.insert("subscriptions", { + ...args, + status: "active", + cancelAtPeriodEnd: false, + createdAt: now, + updatedAt: now, + }); + }, +}); + +export const update = mutation({ + args: { + subscriptionId: v.id("subscriptions"), + plan: v.optional(v.union(v.literal("free"), v.literal("pro"), v.literal("team"))), + status: v.optional(v.union(v.literal("active"), v.literal("canceled"), v.literal("past_due"))), + currentPeriodStart: v.optional(v.number()), + currentPeriodEnd: v.optional(v.number()), + cancelAtPeriodEnd: v.optional(v.boolean()), + }, + handler: async (ctx, { subscriptionId, ...fields }) => { + const updates: Record = { updatedAt: Date.now() }; + for (const [key, value] of Object.entries(fields)) { + if (value !== undefined) updates[key] = value; + } + await ctx.db.patch(subscriptionId, updates); + }, +}); + +export const cancel = mutation({ + args: { subscriptionId: v.id("subscriptions") }, + handler: async (ctx, { subscriptionId }) => { + await ctx.db.patch(subscriptionId, { + cancelAtPeriodEnd: true, + updatedAt: Date.now(), + }); + }, +}); diff --git a/convex/telegraph.ts b/convex/telegraph.ts new file mode 100644 index 0000000..f0db1bf --- /dev/null +++ b/convex/telegraph.ts @@ -0,0 +1,81 @@ +import { v } from "convex/values"; +import { query, mutation } from "./_generated/server"; + +export const getAuth = query({ + args: { userId: v.id("users") }, + handler: async (ctx, args) => { + return ctx.db + .query("telegraphAuth") + .withIndex("by_user", (q) => q.eq("userId", args.userId)) + .first(); + }, +}); + +export const saveAuth = mutation({ + args: { + userId: v.id("users"), + accessToken: v.string(), + authorName: v.string(), + authorUrl: v.optional(v.string()), + indexPagePath: v.optional(v.string()), + indexPageUrl: v.optional(v.string()), + }, + handler: async (ctx, args) => { + const existing = await ctx.db + .query("telegraphAuth") + .withIndex("by_user", (q) => q.eq("userId", args.userId)) + .first(); + if (existing) { + await ctx.db.patch(existing._id, { + accessToken: args.accessToken, + authorName: args.authorName, + authorUrl: args.authorUrl, + indexPagePath: args.indexPagePath, + indexPageUrl: args.indexPageUrl, + }); + return existing._id; + } + return ctx.db.insert("telegraphAuth", { + ...args, + createdAt: Date.now(), + }); + }, +}); + +export const listEntries = query({ + args: { userId: v.id("users"), category: v.optional(v.string()) }, + handler: async (ctx, args) => { + let q = ctx.db + .query("telegraphEntries") + .withIndex("by_user", (q) => q.eq("userId", args.userId)) + .order("desc"); + const entries = await q.collect(); + if (args.category) { + return entries.filter((e) => e.category === args.category); + } + return entries; + }, +}); + +export const addEntry = mutation({ + args: { + userId: v.id("users"), + title: v.string(), + telegraphPath: v.string(), + telegraphUrl: v.string(), + category: v.optional(v.string()), + }, + handler: async (ctx, args) => { + return ctx.db.insert("telegraphEntries", { + ...args, + createdAt: Date.now(), + }); + }, +}); + +export const removeEntry = mutation({ + args: { entryId: v.id("telegraphEntries") }, + handler: async (ctx, args) => { + await ctx.db.delete(args.entryId); + }, +}); diff --git a/convex/tinydeskCompanions.ts b/convex/tinydeskCompanions.ts new file mode 100644 index 0000000..984979c --- /dev/null +++ b/convex/tinydeskCompanions.ts @@ -0,0 +1,99 @@ +import { query, mutation } from "./_generated/server"; +import { v } from "convex/values"; + +export const getBySlug = query({ + args: { slug: v.string() }, + handler: async (ctx, { slug }) => { + return await ctx.db + .query("tinydeskCompanions") + .withIndex("by_slug", (q) => q.eq("slug", slug)) + .first(); + }, +}); + +export const listAll = query({ + args: {}, + handler: async (ctx) => { + return await ctx.db + .query("tinydeskCompanions") + .order("desc") + .take(50); + }, +}); + +export const listSlugs = query({ + args: {}, + handler: async (ctx) => { + const all = await ctx.db.query("tinydeskCompanions").collect(); + return all.map((c) => ({ + slug: c.slug, + artist: c.artist, + genre: c.genre, + tinyDeskVideoId: c.tinyDeskVideoId, + isCommunitySubmitted: c.isCommunitySubmitted, + })); + }, +}); + +export const create = mutation({ + args: { + slug: v.string(), + artist: v.string(), + tagline: v.string(), + tinyDeskVideoId: v.string(), + nodes: v.string(), + userId: v.id("users"), + genre: v.optional(v.array(v.string())), + sourceUrl: v.optional(v.string()), + isCommunitySubmitted: v.optional(v.boolean()), + }, + handler: async (ctx, args) => { + // Validate nodes are not empty + let parsedNodes: unknown[]; + try { + parsedNodes = JSON.parse(args.nodes); + } catch { + throw new Error("Invalid nodes JSON"); + } + if (!Array.isArray(parsedNodes) || parsedNodes.length === 0) { + throw new Error("Companion must have at least one influence connection"); + } + + const existing = await ctx.db + .query("tinydeskCompanions") + .withIndex("by_slug", (q) => q.eq("slug", args.slug)) + .first(); + + if (existing) { + await ctx.db.patch(existing._id, { + artist: args.artist, + tagline: args.tagline, + tinyDeskVideoId: args.tinyDeskVideoId, + nodes: args.nodes, + genre: args.genre, + sourceUrl: args.sourceUrl, + }); + return existing._id; + } + + return await ctx.db.insert("tinydeskCompanions", { + ...args, + createdAt: Date.now(), + }); + }, +}); + +export const deleteBySlug = mutation({ + args: { slug: v.string() }, + handler: async (ctx, { slug }) => { + const existing = await ctx.db + .query("tinydeskCompanions") + .withIndex("by_slug", (q) => q.eq("slug", slug)) + .first(); + if (existing) { + await ctx.db.delete(existing._id); + return true; + } + return false; + }, +}); diff --git a/convex/toolCalls.ts b/convex/toolCalls.ts new file mode 100644 index 0000000..fc128d1 --- /dev/null +++ b/convex/toolCalls.ts @@ -0,0 +1,47 @@ +import { mutation, query } from "./_generated/server"; +import { v } from "convex/values"; + +export const start = mutation({ + args: { + sessionId: v.id("sessions"), + messageId: v.optional(v.id("messages")), + toolName: v.string(), + args: v.string(), + }, + handler: async (ctx, args) => { + return await ctx.db.insert("toolCalls", { + sessionId: args.sessionId, + messageId: args.messageId, + toolName: args.toolName, + args: args.args, + status: "running", + startedAt: Date.now(), + }); + }, +}); + +export const complete = mutation({ + args: { + id: v.id("toolCalls"), + result: v.string(), + status: v.union(v.literal("complete"), v.literal("error")), + }, + handler: async (ctx, args) => { + await ctx.db.patch(args.id, { + result: args.result, + status: args.status, + completedAt: Date.now(), + }); + }, +}); + +export const listBySession = query({ + args: { sessionId: v.id("sessions") }, + handler: async (ctx, args) => { + return await ctx.db + .query("toolCalls") + .withIndex("by_session", (q) => q.eq("sessionId", args.sessionId)) + .order("asc") + .collect(); + }, +}); diff --git a/convex/tsconfig.json b/convex/tsconfig.json new file mode 100644 index 0000000..7374127 --- /dev/null +++ b/convex/tsconfig.json @@ -0,0 +1,25 @@ +{ + /* This TypeScript project config describes the environment that + * Convex functions run in and is used to typecheck them. + * You can modify it, but some settings are required to use Convex. + */ + "compilerOptions": { + /* These settings are not required by Convex and can be modified. */ + "allowJs": true, + "strict": true, + "moduleResolution": "Bundler", + "jsx": "react-jsx", + "skipLibCheck": true, + "allowSyntheticDefaultImports": true, + + /* These compiler options are required by Convex */ + "target": "ESNext", + "lib": ["ES2021", "dom"], + "forceConsistentCasingInFileNames": true, + "module": "ESNext", + "isolatedModules": true, + "noEmit": true + }, + "include": ["./**/*"], + "exclude": ["./_generated"] +} diff --git a/convex/tumblr.ts b/convex/tumblr.ts new file mode 100644 index 0000000..77e62b0 --- /dev/null +++ b/convex/tumblr.ts @@ -0,0 +1,85 @@ +import { v } from "convex/values"; +import { query, mutation } from "./_generated/server"; + +export const getAuth = query({ + args: { userId: v.id("users") }, + handler: async (ctx, args) => { + return ctx.db + .query("tumblrAuth") + .withIndex("by_user", (q) => q.eq("userId", args.userId)) + .first(); + }, +}); + +export const saveAuth = mutation({ + args: { + userId: v.id("users"), + oauthToken: v.string(), + oauthTokenSecret: v.string(), + blogName: v.string(), + blogUrl: v.string(), + blogUuid: v.string(), + }, + handler: async (ctx, args) => { + const existing = await ctx.db + .query("tumblrAuth") + .withIndex("by_user", (q) => q.eq("userId", args.userId)) + .first(); + if (existing) { + await ctx.db.patch(existing._id, { + oauthToken: args.oauthToken, + oauthTokenSecret: args.oauthTokenSecret, + blogName: args.blogName, + blogUrl: args.blogUrl, + blogUuid: args.blogUuid, + }); + return existing._id; + } + return ctx.db.insert("tumblrAuth", { + ...args, + createdAt: Date.now(), + }); + }, +}); + +export const removeAuth = mutation({ + args: { userId: v.id("users") }, + handler: async (ctx, args) => { + const auth = await ctx.db + .query("tumblrAuth") + .withIndex("by_user", (q) => q.eq("userId", args.userId)) + .first(); + if (auth) { + await ctx.db.delete(auth._id); + } + }, +}); + +export const listPosts = query({ + args: { userId: v.id("users") }, + handler: async (ctx, args) => { + return ctx.db + .query("tumblrPosts") + .withIndex("by_user", (q) => q.eq("userId", args.userId)) + .order("desc") + .take(20); + }, +}); + +export const addPost = mutation({ + args: { + userId: v.id("users"), + tumblrPostId: v.string(), + title: v.string(), + blogName: v.string(), + postUrl: v.string(), + category: v.optional(v.string()), + tags: v.optional(v.string()), + }, + handler: async (ctx, args) => { + return ctx.db.insert("tumblrPosts", { + ...args, + createdAt: Date.now(), + }); + }, +}); diff --git a/convex/usage.ts b/convex/usage.ts new file mode 100644 index 0000000..bd182cf --- /dev/null +++ b/convex/usage.ts @@ -0,0 +1,85 @@ +import { query, mutation } from "./_generated/server"; +import { v } from "convex/values"; + +/** + * Atomic mutation: count current usage, check against limit, write event if allowed. + * Must be a single mutation to prevent race conditions. + */ +export const recordAndCheckQuota = mutation({ + args: { + userId: v.id("users"), + type: v.union(v.literal("agent_query"), v.literal("chat_query")), + periodStart: v.number(), + limit: v.number(), + teamDomain: v.optional(v.string()), + }, + handler: async (ctx, { userId, type, periodStart, limit, teamDomain }) => { + // Count existing usage in this period + let used: number; + if (teamDomain) { + // Team pooled counting + const events = await ctx.db + .query("usageEvents") + .withIndex("by_team_domain_period", (q) => + q.eq("teamDomain", teamDomain).eq("periodStart", periodStart) + ) + .filter((q) => q.eq(q.field("type"), type)) + .collect(); + used = events.length; + } else { + const events = await ctx.db + .query("usageEvents") + .withIndex("by_user_type_period", (q) => + q.eq("userId", userId).eq("type", type).eq("periodStart", periodStart) + ) + .collect(); + used = events.length; + } + + if (used >= limit) { + return { allowed: false, used, limit }; + } + + // Write the usage event + await ctx.db.insert("usageEvents", { + userId, + type, + teamDomain, + periodStart, + createdAt: Date.now(), + }); + + return { allowed: true, used: used + 1, limit }; + }, +}); + +export const getUsageSummary = query({ + args: { userId: v.id("users"), periodStart: v.number() }, + handler: async (ctx, { userId, periodStart }) => { + const agentEvents = await ctx.db + .query("usageEvents") + .withIndex("by_user_type_period", (q) => + q.eq("userId", userId).eq("type", "agent_query").eq("periodStart", periodStart) + ) + .collect(); + + return { + agentQueriesUsed: agentEvents.length, + }; + }, +}); + +export const countTeamAgentQueries = query({ + args: { teamDomain: v.string(), periodStart: v.number() }, + handler: async (ctx, { teamDomain, periodStart }) => { + const events = await ctx.db + .query("usageEvents") + .withIndex("by_team_domain_period", (q) => + q.eq("teamDomain", teamDomain).eq("periodStart", periodStart) + ) + .filter((q) => q.eq(q.field("type"), "agent_query")) + .collect(); + + return events.length; + }, +}); diff --git a/convex/userSkills.ts b/convex/userSkills.ts new file mode 100644 index 0000000..5fd54d7 --- /dev/null +++ b/convex/userSkills.ts @@ -0,0 +1,155 @@ +import { query, mutation } from "./_generated/server"; +import { v } from "convex/values"; + +export const listByUser = query({ + args: { userId: v.id("users") }, + handler: async (ctx, { userId }) => { + return await ctx.db + .query("userSkills") + .withIndex("by_user", (q) => q.eq("userId", userId)) + .collect(); + }, +}); + +export const getByUserCommand = query({ + args: { userId: v.id("users"), command: v.string() }, + handler: async (ctx, { userId, command }) => { + return await ctx.db + .query("userSkills") + .withIndex("by_user_command", (q) => + q.eq("userId", userId).eq("command", command), + ) + .first(); + }, +}); + +export const countByUser = query({ + args: { userId: v.id("users") }, + handler: async (ctx, { userId }) => { + const skills = await ctx.db + .query("userSkills") + .withIndex("by_user", (q) => q.eq("userId", userId)) + .collect(); + return skills.length; + }, +}); + +export const create = mutation({ + args: { + userId: v.id("users"), + command: v.string(), + name: v.string(), + description: v.string(), + promptTemplate: v.string(), + toolHints: v.array(v.string()), + sourceUrl: v.optional(v.string()), + triggerPattern: v.optional(v.string()), + maxSkills: v.number(), + }, + handler: async (ctx, args) => { + // Atomic skill count check (prevents race condition) + const allSkills = await ctx.db + .query("userSkills") + .withIndex("by_user", (q) => q.eq("userId", args.userId)) + .collect(); + if (allSkills.length >= args.maxSkills) { + throw new Error(`Skill limit reached (${args.maxSkills}). Delete an existing skill or upgrade.`); + } + + const existing = await ctx.db + .query("userSkills") + .withIndex("by_user_command", (q) => + q.eq("userId", args.userId).eq("command", args.command), + ) + .first(); + if (existing) { + throw new Error(`Command /${args.command} already exists`); + } + + const now = Date.now(); + // eslint-disable-next-line @typescript-eslint/no-unused-vars + const { maxSkills: _maxSkills, ...skillFields } = args; + return await ctx.db.insert("userSkills", { + ...skillFields, + runCount: 0, + visibility: "private" as const, + isEnabled: true, + createdAt: now, + updatedAt: now, + }); + }, +}); + +export const update = mutation({ + args: { + skillId: v.id("userSkills"), + name: v.optional(v.string()), + description: v.optional(v.string()), + promptTemplate: v.optional(v.string()), + toolHints: v.optional(v.array(v.string())), + sourceUrl: v.optional(v.string()), + }, + handler: async (ctx, { skillId, ...fields }) => { + const updates: Record = { updatedAt: Date.now() }; + for (const [key, value] of Object.entries(fields)) { + if (value !== undefined) updates[key] = value; + } + await ctx.db.patch(skillId, updates); + }, +}); + +export const toggleEnabled = mutation({ + args: { skillId: v.id("userSkills") }, + handler: async (ctx, { skillId }) => { + const skill = await ctx.db.get(skillId); + if (!skill) throw new Error("Skill not found"); + await ctx.db.patch(skillId, { + isEnabled: !skill.isEnabled, + updatedAt: Date.now(), + }); + }, +}); + +export const recordRun = mutation({ + args: { + skillId: v.id("userSkills"), + lastResults: v.optional(v.string()), + gotcha: v.optional(v.string()), + }, + handler: async (ctx, { skillId, lastResults, gotcha }) => { + const skill = await ctx.db.get(skillId); + if (!skill) return; + + const updates: Record = { + runCount: skill.runCount + 1, + lastRunAt: Date.now(), + updatedAt: Date.now(), + }; + + if (lastResults) { + // Cap at 2000 chars + updates.lastResults = lastResults.slice(0, 2000); + } + + if (gotcha) { + // Append to existing gotchas, cap at 1000 chars + const existing = skill.gotchas ?? ""; + const appended = existing ? `${existing}\n- ${gotcha}` : `- ${gotcha}`; + updates.gotchas = appended.slice(-1000); + } + + await ctx.db.patch(skillId, updates); + }, +}); + +export const remove = mutation({ + args: { skillId: v.id("userSkills"), userId: v.optional(v.id("users")) }, + handler: async (ctx, { skillId, userId }) => { + const skill = await ctx.db.get(skillId); + if (!skill) throw new Error("Skill not found"); + if (userId && skill.userId !== userId) { + throw new Error("Not authorized to delete this skill"); + } + await ctx.db.delete(skillId); + }, +}); diff --git a/convex/users.ts b/convex/users.ts new file mode 100644 index 0000000..5f01f12 --- /dev/null +++ b/convex/users.ts @@ -0,0 +1,200 @@ +import { mutation, query, internalMutation } from "./_generated/server"; +import { v } from "convex/values"; + +function slugify(name: string): string { + return name + .toLowerCase() + .replace(/[^a-z0-9]+/g, "-") + .replace(/(^-|-$)/g, ""); +} + +export const getByClerkId = query({ + args: { clerkId: v.string() }, + handler: async (ctx, args) => { + return await ctx.db + .query("users") + .withIndex("by_clerk_id", (q) => q.eq("clerkId", args.clerkId)) + .unique(); + }, +}); + +export const getByUsernameSlug = query({ + args: { usernameSlug: v.string() }, + handler: async (ctx, { usernameSlug }) => { + return await ctx.db + .query("users") + .withIndex("by_username_slug", (q) => q.eq("usernameSlug", usernameSlug)) + .first(); + }, +}); + +export const upsert = mutation({ + args: { + clerkId: v.string(), + email: v.string(), + name: v.optional(v.string()), + }, + handler: async (ctx, args) => { + const existing = await ctx.db + .query("users") + .withIndex("by_clerk_id", (q) => q.eq("clerkId", args.clerkId)) + .unique(); + + const usernameSlug = slugify(args.name ?? args.email.split("@")[0]); + + if (existing) { + await ctx.db.patch(existing._id, { + email: args.email, + name: args.name, + usernameSlug, + }); + return existing._id; + } + + return await ctx.db.insert("users", { + clerkId: args.clerkId, + email: args.email, + name: args.name, + usernameSlug, + createdAt: Date.now(), + }); + }, +}); + +export const setHelpPersona = mutation({ + args: { + clerkId: v.string(), + helpPersona: v.string(), + }, + handler: async (ctx, args) => { + const user = await ctx.db + .query("users") + .withIndex("by_clerk_id", (q) => q.eq("clerkId", args.clerkId)) + .unique(); + if (!user) throw new Error("User not found"); + await ctx.db.patch(user._id, { helpPersona: args.helpPersona }); + }, +}); + +export const completeOnboarding = mutation({ + args: { clerkId: v.string() }, + handler: async (ctx, args) => { + const user = await ctx.db + .query("users") + .withIndex("by_clerk_id", (q) => q.eq("clerkId", args.clerkId)) + .unique(); + if (!user) throw new Error("User not found"); + await ctx.db.patch(user._id, { onboardingCompleted: true }); + }, +}); + +export const updateStripeCustomerId = mutation({ + args: { + userId: v.id("users"), + stripeCustomerId: v.string(), + }, + handler: async (ctx, { userId, stripeCustomerId }) => { + await ctx.db.patch(userId, { stripeCustomerId }); + }, +}); + +/** Delete a user by Clerk ID and remap another. For migration conflict resolution. */ +export const resolveConflict = mutation({ + args: { + adminSecret: v.string(), + deleteClerkId: v.string(), + remapOldClerkId: v.string(), + remapNewClerkId: v.string(), + }, + handler: async (ctx, { adminSecret, deleteClerkId, remapOldClerkId, remapNewClerkId }) => { + if (adminSecret !== "clerk-migration-2026") { + throw new Error("Unauthorized"); + } + // Delete the duplicate empty user + const dup = await ctx.db.query("users").withIndex("by_clerk_id", (q) => q.eq("clerkId", deleteClerkId)).unique(); + if (dup) { + await ctx.db.delete(dup._id); + } + // Remap the old user + const old = await ctx.db.query("users").withIndex("by_clerk_id", (q) => q.eq("clerkId", remapOldClerkId)).unique(); + if (old) { + await ctx.db.patch(old._id, { clerkId: remapNewClerkId }); + } + return { deleted: dup?._id ?? null, remapped: old?._id ?? null }; + }, +}); + +/** Remap Clerk IDs after dev-to-prod migration. Protected by admin secret. */ +export const remapClerkIds = mutation({ + args: { + adminSecret: v.string(), + mappings: v.array( + v.object({ + oldClerkId: v.string(), + newClerkId: v.string(), + email: v.optional(v.string()), + }), + ), + dryRun: v.optional(v.boolean()), + }, + handler: async (ctx, { adminSecret, mappings, dryRun }) => { + // Admin check — hardcoded for migration, remove after + if (adminSecret !== "clerk-migration-2026") { + throw new Error("Unauthorized: invalid admin secret"); + } + const results: Array<{ + oldClerkId: string; + newClerkId: string; + email?: string; + status: string; + userId?: string; + }> = []; + + for (const mapping of mappings) { + const user = await ctx.db + .query("users") + .withIndex("by_clerk_id", (q) => q.eq("clerkId", mapping.oldClerkId)) + .unique(); + + if (!user) { + results.push({ ...mapping, status: "missing" }); + continue; + } + + if (mapping.oldClerkId === mapping.newClerkId) { + results.push({ ...mapping, status: "noop", userId: user._id }); + continue; + } + + // Check for conflict — new ID already exists on a different user + const conflict = await ctx.db + .query("users") + .withIndex("by_clerk_id", (q) => q.eq("clerkId", mapping.newClerkId)) + .unique(); + + if (conflict && conflict._id !== user._id) { + results.push({ ...mapping, status: "conflict", userId: user._id }); + continue; + } + + if (!dryRun) { + await ctx.db.patch(user._id, { clerkId: mapping.newClerkId }); + } + + results.push({ + ...mapping, + status: dryRun ? "would_update" : "updated", + userId: user._id, + }); + } + + return { + total: mappings.length, + updated: results.filter((r) => r.status === "updated" || r.status === "would_update").length, + missing: results.filter((r) => r.status === "missing").length, + conflicts: results.filter((r) => r.status === "conflict").length, + noop: results.filter((r) => r.status === "noop").length, + results, + }; + }, +}); diff --git a/convex/wiki.ts b/convex/wiki.ts new file mode 100644 index 0000000..f630d6f --- /dev/null +++ b/convex/wiki.ts @@ -0,0 +1,532 @@ +import { query, mutation, action, internalQuery, internalMutation, internalAction } from "./_generated/server"; +import { v } from "convex/values"; +import { internal } from "./_generated/api"; + +// ── Helpers ────────────────────────────────────────────── +// Canonical slugify lives at src/lib/slug.ts. Convex functions can't import +// from src/, so we keep a local copy. Both MUST use the same algorithm. + +function slugify(name: string): string { + return name + .toLowerCase() + .replace(/[^a-z0-9]+/g, "-") + .replace(/(^-|-$)/g, ""); +} + +// ── Shared validator fragments ─────────────────────────── + +const sectionSourceValidator = v.object({ + tool: v.string(), + url: v.optional(v.string()), + fetchedAt: v.number(), +}); + +const sectionValidator = v.object({ + heading: v.string(), + content: v.string(), + sources: v.array(sectionSourceValidator), + lastSynthesizedAt: v.optional(v.number()), +}); + +const contradictionValidator = v.object({ + claim1: v.object({ source: v.string(), value: v.string() }), + claim2: v.object({ source: v.string(), value: v.string() }), + field: v.string(), +}); + +const metadataValidator = v.object({ + origin: v.optional(v.string()), + yearsActive: v.optional(v.string()), + members: v.optional(v.array(v.string())), + genreDNA: v.optional(v.array(v.string())), +}); + +// ── Queries ────────────────────────────────────────────── + +/** Get a wiki page by user slug + entity slug, with access control. + * Called from Next.js server components via ConvexHttpClient (no Clerk JWT). + * viewerClerkId is passed from the server component after calling Clerk auth(). */ +export const getBySlug = query({ + args: { + userSlug: v.string(), + slug: v.string(), + viewerClerkId: v.optional(v.string()), + }, + handler: async (ctx, { userSlug, slug, viewerClerkId }) => { + // Look up owner by indexed usernameSlug field (no full table scan) + const owner = await ctx.db + .query("users") + .withIndex("by_username_slug", (q) => q.eq("usernameSlug", userSlug)) + .first(); + if (!owner) return null; + + const page = await ctx.db + .query("wikiPages") + .withIndex("by_user_slug", (q) => + q.eq("userId", owner._id).eq("slug", slug), + ) + .first(); + + if (!page) return null; + if (page.archivedAt) return null; + + // Access control: private pages only visible to owner + // viewerClerkId comes from Clerk auth() in the server component, not from the browser + if (page.visibility === "private") { + if (!viewerClerkId || owner.clerkId !== viewerClerkId) { + return null; + } + } + + return { ...page, ownerName: owner.name ?? owner.email.split("@")[0] }; + }, +}); + +/** Shared wiki lookup: find the richest NON-PRIVATE wiki page for an artist across all users. + * Only returns public/unlisted pages to prevent leaking private research. + * Used by wiki-first response to give the LLM a head start. */ +export const getSharedBySlug = query({ + args: { slug: v.string() }, + handler: async (ctx, { slug }) => { + const pages = await ctx.db + .query("wikiPages") + .withIndex("by_slug", (q) => q.eq("slug", slug)) + .collect(); + + // Only non-archived, non-private pages (security: never leak private research) + const shared = pages.filter((p) => !p.archivedAt && p.visibility !== "private"); + if (shared.length === 0) return null; + + shared.sort((a, b) => b.sections.length - a.sections.length); + return shared[0]; + }, +}); + +/** List all non-archived wiki pages for a user via the index table. */ +export const listUserPages = query({ + args: { userId: v.id("users") }, + handler: async (ctx, { userId }) => { + const entries = await ctx.db + .query("wikiIndexEntries") + .withIndex("by_user", (q) => q.eq("userId", userId)) + .collect(); + + return entries.sort((a, b) => b.lastUpdated - a.lastUpdated); + }, +}); + +/** Get wiki entry count for a user (for sidebar badge). */ +export const getEntryCount = query({ + args: { userId: v.id("users") }, + handler: async (ctx, { userId }) => { + const entries = await ctx.db + .query("wikiIndexEntries") + .withIndex("by_user", (q) => q.eq("userId", userId)) + .collect(); + return entries.length; + }, +}); + +// ── Mutations ──────────────────────────────────────────── + +/** Append raw section data from a tool result. Fast, append-only. */ +export const appendWikiData = mutation({ + args: { + userId: v.id("users"), + entityName: v.string(), + section: v.object({ + heading: v.string(), + content: v.string(), + sources: v.array(sectionSourceValidator), + }), + }, + handler: async (ctx, { userId, entityName, section }) => { + const slug = slugify(entityName); + const now = Date.now(); + + const existing = await ctx.db + .query("wikiPages") + .withIndex("by_user_slug", (q) => + q.eq("userId", userId).eq("slug", slug), + ) + .first(); + + if (existing) { + await ctx.db.patch(existing._id, { + sections: [ + ...existing.sections, + { ...section, lastSynthesizedAt: undefined }, + ], + ...(existing.archivedAt ? { archivedAt: undefined } : {}), + updatedAt: now, + }); + + const indexEntry = await ctx.db + .query("wikiIndexEntries") + .withIndex("by_user_slug", (q) => + q.eq("userId", userId).eq("slug", slug), + ) + .first(); + if (indexEntry) { + await ctx.db.patch(indexEntry._id, { + sourceCount: existing.sections.length + 1, + lastUpdated: now, + }); + } + + await ctx.db.insert("wikiLogEntries", { + userId, + timestamp: now, + operation: "ingest", + entitySlug: slug, + description: `Updated ${entityName} with ${section.heading}`, + toolsUsed: section.sources.map((s) => s.tool), + }); + + return existing._id; + } + + // Create new page + const pageId = await ctx.db.insert("wikiPages", { + userId, + slug, + entityType: "artist", + entityName, + sections: [{ ...section, lastSynthesizedAt: undefined }], + contradictions: [], + metadata: {}, + visibility: "private", + createdAt: now, + updatedAt: now, + }); + + await ctx.db.insert("wikiIndexEntries", { + userId, + pageId, + slug, + entityName, + entityType: "artist", + visibility: "private", + sourceCount: 1, + lastUpdated: now, + }); + + await ctx.db.insert("wikiLogEntries", { + userId, + timestamp: now, + operation: "ingest", + entitySlug: slug, + description: `Created wiki page for ${entityName}`, + toolsUsed: section.sources.map((s) => s.tool), + }); + + return pageId; + }, +}); + +/** Toggle visibility (private/unlisted/public). Owner only, verified via Convex auth. */ +export const toggleVisibility = mutation({ + args: { + pageId: v.id("wikiPages"), + visibility: v.union( + v.literal("private"), + v.literal("unlisted"), + v.literal("public"), + ), + }, + handler: async (ctx, { pageId, visibility }) => { + // Verify ownership via Clerk JWT, not client-supplied userId + const identity = await ctx.auth.getUserIdentity(); + if (!identity) throw new Error("Not authenticated"); + + const page = await ctx.db.get(pageId); + if (!page) throw new Error("Page not found"); + + // Look up the Convex user from Clerk identity + const owner = await ctx.db + .query("users") + .withIndex("by_clerk_id", (q) => q.eq("clerkId", identity.subject)) + .first(); + if (!owner || page.userId !== owner._id) { + throw new Error("Not authorized"); + } + + await ctx.db.patch(pageId, { visibility, updatedAt: Date.now() }); + + // Keep index entry visibility in sync + const indexEntry = await ctx.db + .query("wikiIndexEntries") + .withIndex("by_user_slug", (q) => + q.eq("userId", owner._id).eq("slug", page.slug), + ) + .first(); + if (indexEntry) { + await ctx.db.patch(indexEntry._id, { visibility }); + } + }, +}); + +/** Soft-delete a wiki page. Owner only, verified via Convex auth. */ +export const archivePage = mutation({ + args: { + pageId: v.id("wikiPages"), + }, + handler: async (ctx, { pageId }) => { + const identity = await ctx.auth.getUserIdentity(); + if (!identity) throw new Error("Not authenticated"); + + const page = await ctx.db.get(pageId); + if (!page) throw new Error("Page not found"); + + const owner = await ctx.db + .query("users") + .withIndex("by_clerk_id", (q) => q.eq("clerkId", identity.subject)) + .first(); + if (!owner || page.userId !== owner._id) { + throw new Error("Not authorized"); + } + const userId = owner._id; + await ctx.db.patch(pageId, { + archivedAt: Date.now(), + updatedAt: Date.now(), + }); + + const indexEntry = await ctx.db + .query("wikiIndexEntries") + .withIndex("by_user_slug", (q) => + q.eq("userId", userId).eq("slug", page.slug), + ) + .first(); + if (indexEntry) { + await ctx.db.delete(indexEntry._id); + } + }, +}); + +/** Schedule synthesis for a wiki page. Uses Convex scheduler so it survives + * serverless function termination (unlike fire-and-forget from route.ts). */ +export const scheduleSynthesis = mutation({ + args: { pageId: v.id("wikiPages") }, + handler: async (ctx, { pageId }) => { + // Schedule the action to run immediately (0ms delay) + await ctx.scheduler.runAfter(0, internal.wiki.synthesizeWikiPageInternal, { pageId }); + }, +}); + +// ── Actions (external API calls) ───────────────────────── + +const SYNTHESIS_PROMPT = `You are a music encyclopedia editor. Given an artist's wiki page data from multiple sources, produce a clean, synthesized version. + +INPUT: You'll receive the artist name and raw section data from various music data sources (Spotify, WhoSampled, Bandcamp, etc.). + +OUTPUT (JSON): +{ + "description": "A 2-3 sentence blurb about the artist, written in encyclopedia style. Cite what makes them distinctive.", + "sections": [ + { + "heading": "Section heading (e.g., 'Overview', 'Musical Style', 'Discography Highlights')", + "content": "Merged, deduplicated content for this section. If multiple sources say the same thing, keep the most detailed version. Remove redundancy.", + "sources": [{"tool": "source-name", "url": "optional-url", "fetchedAt": timestamp}] + } + ], + "contradictions": [ + { + "claim1": {"source": "tool-name", "value": "what source 1 says"}, + "claim2": {"source": "tool-name", "value": "what source 2 says"}, + "field": "the field that disagrees (e.g., 'genre', 'formed_year', 'origin')" + } + ], + "metadata": { + "origin": "city/country if mentioned", + "yearsActive": "start-present or start-end", + "members": ["member names if mentioned"], + "genreDNA": ["genre tags from across sources, deduplicated"] + } +} + +RULES: +- Merge duplicate information. If Spotify and Bandcamp both say "funk", keep one entry. +- Flag contradictions explicitly. If Spotify says "psychedelic rock" but Bandcamp says "surf rock", that's a contradiction. +- Preserve source attribution. Every fact should trace back to which tool provided it. +- Write the description as if for a music encyclopedia, not a chatbot response. +- Keep sections focused. Combine related data, split unrelated topics. +- Extract metadata from ALL sections. Genre tags from every source, deduplicated.`; + +/** Call Haiku API for synthesis. Returns parsed JSON or null on failure. */ +async function callHaikuSynthesis( + apiKey: string, + userPrompt: string, +): Promise | null> { + const response = await fetch("https://api.anthropic.com/v1/messages", { + method: "POST", + headers: { + "Content-Type": "application/json", + "x-api-key": apiKey, + "anthropic-version": "2023-06-01", + }, + body: JSON.stringify({ + model: "claude-haiku-4-5-20251001", + max_tokens: 4096, + system: SYNTHESIS_PROMPT, + messages: [{ role: "user", content: userPrompt }], + }), + }); + + if (!response.ok) { + console.error("[wiki/synthesize] API error:", response.status); + return null; + } + + const result = await response.json(); + const text = result.content?.[0]?.text; + if (!text) return null; + + const cleaned = text.replace(/^```json?\n?/m, "").replace(/\n?```$/m, ""); + const parsed = JSON.parse(cleaned); + + // Validate expected shape + if (!parsed || typeof parsed !== "object") return null; + if (parsed.sections && !Array.isArray(parsed.sections)) return null; + + return parsed; +} + +/** Synthesize a wiki page using Haiku. Retries once on failure. + * Internal action — called via ctx.scheduler from scheduleSynthesis mutation. */ +export const synthesizeWikiPageInternal = internalAction({ + args: { pageId: v.id("wikiPages") }, + handler: async (ctx, { pageId }) => { + const page = await ctx.runQuery(internal.wiki.getPageInternal, { pageId }); + if (!page) return; + + const apiKey = process.env.ANTHROPIC_API_KEY; + if (!apiKey) { + console.error("[wiki/synthesize] No ANTHROPIC_API_KEY set"); + return; + } + + const userPrompt = `Artist: ${page.entityName} + +Raw section data: +${JSON.stringify(page.sections, null, 2)} + +Synthesize this into a clean wiki page. Output valid JSON only.`; + + // Try up to 2 attempts + let synthesized: Record | null = null; + for (let attempt = 0; attempt < 2; attempt++) { + try { + synthesized = await callHaikuSynthesis(apiKey, userPrompt); + if (synthesized) break; + } catch (err) { + console.error(`[wiki/synthesize] Attempt ${attempt + 1} failed:`, err); + if (attempt === 0) continue; // retry once + } + } + + if (!synthesized) { + console.error("[wiki/synthesize] All attempts failed for", page.entityName); + return; // Page stays unsynthesized — valid state + } + + const now = Date.now(); + + // Sanitize Haiku output to match Convex validators exactly. + // Haiku may add extra fields that the strict validators reject. + const rawSections = Array.isArray(synthesized.sections) ? synthesized.sections : []; + const sections = rawSections.map((s: Record) => ({ + heading: String(s.heading ?? ""), + content: String(s.content ?? ""), + sources: Array.isArray(s.sources) + ? (s.sources as Array>).map((src) => ({ + tool: String(src.tool ?? src.name ?? "Unknown"), + url: typeof src.url === "string" ? src.url : undefined, + fetchedAt: typeof src.fetchedAt === "number" ? src.fetchedAt : now, + })) + : [], + lastSynthesizedAt: now, + })); + + const rawContradictions = Array.isArray(synthesized.contradictions) ? synthesized.contradictions : []; + const contradictions = rawContradictions.map((c: Record) => ({ + claim1: { + source: String((c.claim1 as Record)?.source ?? ""), + value: String((c.claim1 as Record)?.value ?? ""), + }, + claim2: { + source: String((c.claim2 as Record)?.source ?? ""), + value: String((c.claim2 as Record)?.value ?? ""), + }, + field: String(c.field ?? ""), + })); + + const meta = (synthesized.metadata ?? {}) as Record; + const metadata = { + origin: typeof meta.origin === "string" ? meta.origin : undefined, + yearsActive: typeof meta.yearsActive === "string" ? meta.yearsActive : undefined, + members: Array.isArray(meta.members) ? meta.members.map(String) : undefined, + genreDNA: Array.isArray(meta.genreDNA) ? meta.genreDNA.map(String) : undefined, + }; + + await ctx.runMutation(internal.wiki.updateSynthesized, { + pageId, + description: String(synthesized.description ?? ""), + sections, + contradictions, + metadata, + }); + }, +}); + +// ── Internal functions (action → query/mutation bridge) ── + +export const getPageInternal = internalQuery({ + args: { pageId: v.id("wikiPages") }, + handler: async (ctx, { pageId }) => { + return await ctx.db.get(pageId); + }, +}); + +export const updateSynthesized = internalMutation({ + args: { + pageId: v.id("wikiPages"), + description: v.string(), + sections: v.array(sectionValidator), + contradictions: v.array(contradictionValidator), + metadata: metadataValidator, + }, + handler: async (ctx, { pageId, description, sections, contradictions, metadata }) => { + const now = Date.now(); + await ctx.db.patch(pageId, { + description, + sections, + contradictions, + metadata, + updatedAt: now, + }); + + const page = await ctx.db.get(pageId); + if (page) { + const indexEntry = await ctx.db + .query("wikiIndexEntries") + .withIndex("by_user_slug", (q) => + q.eq("userId", page.userId).eq("slug", page.slug), + ) + .first(); + if (indexEntry) { + await ctx.db.patch(indexEntry._id, { + summary: description.slice(0, 200), + lastUpdated: now, + }); + } + + await ctx.db.insert("wikiLogEntries", { + userId: page.userId, + timestamp: now, + operation: "synthesize", + entitySlug: page.slug, + description: `Synthesized ${page.entityName} (${sections.length} sections, ${contradictions.length} contradictions)`, + }); + } + }, +}); diff --git a/docs/.DS_Store b/docs/.DS_Store index 5f0d0db..512421b 100644 Binary files a/docs/.DS_Store and b/docs/.DS_Store differ diff --git a/docs/AUTH0_SETUP_GUIDE.md b/docs/AUTH0_SETUP_GUIDE.md new file mode 100644 index 0000000..e13dd27 --- /dev/null +++ b/docs/AUTH0_SETUP_GUIDE.md @@ -0,0 +1,343 @@ +# Auth0 Token Vault Setup Guide for Crate + +This guide walks you through setting up Auth0 Token Vault so Crate's AI agent can connect to Spotify, Slack, and Google on behalf of your users. No coding required — this is all done through web dashboards. + +**Time needed:** About 45 minutes for all three services. + +--- + +## Part 1: Create Your Auth0 Account + +1. Go to [auth0.com/signup](https://auth0.com/signup) +2. Sign up with your Google account or email +3. When asked for a tenant name, use something like `crate-music` + - This becomes your domain: `crate-music.us.auth0.com` + - Write this down — it's your `AUTH0_DOMAIN` +4. If asked about your role, pick "Developer" +5. If asked about your framework, pick "Next.js" + +**Save these values — you'll need them for your `.env.local` file:** + +``` +AUTH0_DOMAIN=crate-music.us.auth0.com +``` + +--- + +## Part 2: Create the Crate Application in Auth0 + +This tells Auth0 about your app. + +1. In the left sidebar, click **Applications > Applications** +2. Click the blue **+ Create Application** button +3. Name: `Crate` +4. Type: Select **Regular Web Applications** +5. Click **Create** + +Now you're on the application's Settings page: + +6. **Copy these two values** (at the top under "Basic Information"): + +``` +AUTH0_CLIENT_ID=xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx +AUTH0_CLIENT_SECRET=xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx +``` + +7. Scroll down to **Application URIs** and fill in: + +| Field | Value | +|-------|-------| +| Allowed Callback URLs | `http://localhost:3000/api/auth0/callback, https://digcrate.app/api/auth0/callback` | +| Allowed Logout URLs | `http://localhost:3000, https://digcrate.app` | +| Allowed Web Origins | `http://localhost:3000, https://digcrate.app` | + +8. Scroll down further. Click **Advanced Settings** to expand it +9. Click the **Grant Types** tab +10. Check the box for **Token Vault** +11. Click **Save Changes** at the bottom + +--- + +## Part 3: Activate the My Account API + +Token Vault uses something called "Connected Accounts" which requires the My Account API. This is a one-time toggle. + +1. In the sidebar, go to **Applications > APIs** +2. You'll see a banner for **Auth0 My Account API** — click **Activate** +3. After activation, click on **Auth0 My Account API** +4. Go to the **Application Access** tab +5. Find your "Crate" application in the list +6. Click **Edit** next to it +7. Under **Authorization**, select **Authorized** +8. Under **Permissions**, select **All** the Connected Accounts scopes +9. Click **Save** +10. Go to the **Settings** tab +11. Under **Access Settings**, check **Allow Skipping User Consent** +12. Click **Save** + +--- + +## Part 4: Set Up Spotify Connection + +### Step A: Create a Spotify Developer App + +1. Go to [developer.spotify.com](https://developer.spotify.com/) +2. Sign in with your Spotify account (create one if needed — free account works) +3. Go to your [Dashboard](https://developer.spotify.com/dashboard) +4. Click **Create app** +5. Fill in: + +| Field | Value | +|-------|-------| +| App name | `Crate` | +| App description | `AI music research agent` | +| Redirect URI | `https://crate-music.us.auth0.com/login/callback` | + + **Important:** Replace `crate-music` with YOUR actual Auth0 tenant name from Part 1. + +6. Check **Web API** under "Which API/SDKs are you planning to use?" +7. Agree to terms and click **Save** +8. On your new app's page, you'll see the **Client ID** right away +9. Click **Settings** (top right), then click **View client secret** to reveal it +10. **Copy both values** — you'll paste them into Auth0 next + +### Step B: Add Spotify to Auth0 + +1. Back in the Auth0 Dashboard, go to **Authentication > Social** in the sidebar +2. Click **Create Connection** +3. Find and select **Spotify** +4. Click **Continue** +5. Under **General**: + - Paste your Spotify **Client ID** + - Paste your Spotify **Client Secret** +6. Under **Attributes** (or **Permissions/Scopes**), make sure these are checked: + - `user-read-private` + - `user-read-email` +7. In the **Additional Scopes** field, add these (separated by spaces or commas): + ``` + user-library-read user-top-read playlist-read-private playlist-modify-public playlist-modify-private + ``` + These give Crate permission to read the user's library and create playlists. +8. Under **Purpose**, toggle on **"Connected Accounts for Token Vault"** + + This is the critical step. Without this toggle, the connection is only for login, not for API access. + +9. Click **Create** +10. On the next screen, find your "Crate" application and toggle it **ON** +11. Click **Save** + +**Test it:** Click the **Try Connection** button. You should be redirected to Spotify, asked to authorize, and then sent back to Auth0 with a success message. + +--- + +## Part 5: Set Up Slack Connection + +### Step A: Create a Slack App + +1. Go to [api.slack.com/apps](https://api.slack.com/apps) +2. Sign in with your Slack account +3. Click **Create New App** +4. Select **From scratch** +5. Fill in: + +| Field | Value | +|-------|-------| +| App Name | `Crate` | +| Pick a workspace | Select your workspace (e.g., Radio Milwaukee) | + +6. Click **Create App** +7. You're now on the app's **Basic Information** page +8. Scroll down to **App Credentials** and copy: + - **Client ID** + - **Client Secret** + +### Step B: Configure Slack OAuth + +Still on the Slack API page for your app: + +1. In the left sidebar, click **OAuth & Permissions** +2. Under **Redirect URLs**, click **Add New Redirect URL** +3. Enter: `https://crate-music.us.auth0.com/login/callback` + + (Replace `crate-music` with your Auth0 tenant name) + +4. Click **Add** then **Save URLs** +5. Scroll down to **Scopes > Bot Token Scopes** +6. Click **Add an OAuth Scope** and add: + - `chat:write` (lets Crate send messages to channels) + - `channels:read` (lets Crate see available channels) +7. Also under **Scopes > User Token Scopes**, add: + - `chat:write` + - `channels:read` + +### Step C: Enable Token Rotation + +1. Still in the Slack API dashboard, go to **OAuth & Permissions** +2. Scroll to **Advanced token security via token rotation** +3. Click **Opt In** to enable token rotation (required for Token Vault refresh tokens) + +### Step D: Add Slack to Auth0 + +1. In Auth0 Dashboard, go to **Authentication > Social** +2. Click **Create Connection** +3. Select **Sign In with Slack** (or just "Slack") +4. Click **Continue** +5. Under **General**: + - Paste your Slack **Client ID** + - Paste your Slack **Client Secret** +6. Under **Purpose**, toggle on **"Connected Accounts for Token Vault"** +7. Click **Create** +8. Enable your "Crate" application and **Save** + +**Test it:** Click **Try Connection**. You should be redirected to Slack, asked to authorize, and sent back with a success. + +--- + +## Part 6: Set Up Google Connection + +Google gives you access to Google Docs, Google Drive, Gmail, and Google Calendar. One connection covers all of them. + +### Step A: Create Google Cloud Credentials + +1. Go to [console.cloud.google.com](https://console.cloud.google.com/) +2. Sign in with your Google account +3. If you don't have a project, click **Select a project > New Project** + - Name: `Crate` + - Click **Create** +4. Make sure your new project is selected in the dropdown at the top + +### Step B: Enable the APIs + +1. In the sidebar, go to **APIs & Services > Library** +2. Search for and **Enable** each of these: + - **Google Docs API** — click it, click **Enable** + - **Google Drive API** — click it, click **Enable** + - (Optional) **Gmail API** — if you want email features later + - (Optional) **Google Calendar API** — if you want calendar features later + +### Step C: Set Up OAuth Consent Screen + +1. In the sidebar, go to **APIs & Services > OAuth consent screen** (or **Google Auth Platform > Branding**) +2. User type: **External** +3. Click **Create** +4. Fill in: + +| Field | Value | +|-------|-------| +| App name | `Crate` | +| User support email | your email | +| Authorized domains | `auth0.com` | +| Developer contact email | your email | + +5. Click **Save and Continue** +6. On the **Scopes** page, click **Add or Remove Scopes** +7. Add these scopes: + - `openid` + - `email` + - `profile` + - `https://www.googleapis.com/auth/documents` (Google Docs) + - `https://www.googleapis.com/auth/drive.file` (Google Drive) +8. Click **Update** then **Save and Continue** +9. On **Test Users**, add your email address +10. Click **Save and Continue**, then **Back to Dashboard** + +### Step D: Create OAuth Credentials + +1. In the sidebar, go to **APIs & Services > Credentials** +2. Click **+ Create Credentials > OAuth client ID** +3. Application type: **Web application** +4. Name: `Crate Auth0` +5. Under **Authorized JavaScript origins**, add: + ``` + https://crate-music.us.auth0.com + ``` +6. Under **Authorized redirect URIs**, add: + ``` + https://crate-music.us.auth0.com/login/callback + ``` + (Replace `crate-music` with your Auth0 tenant name) +7. Click **Create** +8. A popup shows your **Client ID** and **Client Secret** — copy both + +### Step E: Add Google to Auth0 + +1. In Auth0 Dashboard, go to **Authentication > Social** +2. Click **Create Connection** +3. Select **Google / Gmail** +4. Click **Continue** +5. Under **General**: + - Paste your Google **Client ID** + - Paste your Google **Client Secret** +6. Under **Permissions**, check: + - `email` + - `profile` +7. Check **Offline Access** (this gets refresh tokens — required for Token Vault) +8. In the **Additional Scopes** field, add: + ``` + https://www.googleapis.com/auth/documents https://www.googleapis.com/auth/drive.file + ``` +9. Under **Purpose**, toggle on **"Connected Accounts for Token Vault"** +10. Click **Create** +11. Enable your "Crate" application and **Save** + +**Test it:** Click **Try Connection**. You should see Google's consent screen listing the Docs and Drive permissions, then be sent back to Auth0 with a success. + +--- + +## Part 7: Verify Everything + +Your **Authentication > Social** page should now show three connections: + +``` +✅ spotify Enabled Token Vault 1 app +✅ slack Enabled Token Vault 1 app +✅ google-oauth2 Enabled Token Vault 1 app +``` + +Each one should have the "Token Vault" badge — that's the "Connected Accounts for Token Vault" toggle you enabled. + +--- + +## Part 8: Add to Your `.env.local` + +Add these to your `.env.local` file (the values from Part 1 and Part 2): + +```bash +# Auth0 Token Vault +AUTH0_DOMAIN=crate-music.us.auth0.com +AUTH0_CLIENT_ID=your-client-id-from-part-2 +AUTH0_CLIENT_SECRET=your-client-secret-from-part-2 +AUTH0_CALLBACK_URL=http://localhost:3000/api/auth0/callback +``` + +Also add these to your Vercel environment variables for production deployment. + +--- + +## Troubleshooting + +**"Try Connection" fails with redirect error:** +- Double-check the redirect URI in each service (Spotify Dashboard, Slack API, Google Cloud Console) matches exactly: `https://YOUR-TENANT.us.auth0.com/login/callback` +- Make sure there are no trailing slashes + +**"Connected Accounts for Token Vault" toggle is missing:** +- This feature requires your Auth0 account to be on a plan that supports Token Vault. Free developer accounts should have access. If you don't see it, check [auth0.com/pricing](https://auth0.com/pricing). + +**Google scopes not showing in Auth0:** +- Add them in the "Additional Scopes" text field, not the checkbox list. The checkbox list only shows common login scopes. + +**Spotify "Invalid redirect URI" error:** +- In the Spotify Developer Dashboard, go to your app's Settings and verify the redirect URI matches your Auth0 domain exactly, including `https://`. + +**Slack "oauth_authorization_url_mismatch" error:** +- In the Slack API dashboard under OAuth & Permissions, make sure the redirect URL matches your Auth0 domain. Click Save URLs after adding it. + +--- + +## What Happens Next + +Once Auth0 is set up, start a fresh Claude Code session and say: + +> "Execute the plan at `docs/superpowers/plans/2026-03-21-auth0-hackathon.md` using subagent-driven development. Working directory: `crate-web-subscription` branch." + +The implementation plan builds the code that connects Crate to these Token Vault connections — the "Connect Spotify" buttons in Settings, the agent tools that read libraries and export playlists, and the Slack/Google Docs delivery tools. diff --git a/docs/CRATE_VS_OPENCLAW.md b/docs/CRATE_VS_OPENCLAW.md new file mode 100644 index 0000000..d737a9f --- /dev/null +++ b/docs/CRATE_VS_OPENCLAW.md @@ -0,0 +1,145 @@ +# Crate vs OpenClaw — Extensibility & Skills Comparison + +**Date:** March 2026 +**Context:** Crate shipped a Custom Skills feature that lets users create personal slash commands via natural language. This analysis compares Crate's approach to OpenClaw, the dominant open-source AI agent platform (180K+ GitHub stars, 13,700+ skills on ClawHub). + +--- + +## The Core Question + +Does Custom Skills make Crate "OpenClaw for music"? + +Almost, but not yet. And that's the right strategy. They're building from opposite directions. + +``` +OpenClaw Crate +───────── ───── +General-purpose agent Music-domain agent + + skills make it specific + skills make it extensible + +Started blank, users add domain Started deep in music, users +knowledge via skills add personal workflows via skills + +Horizontal platform Vertical product +(any domain, any channel) (music domain, web UI) +``` + +OpenClaw is a blank canvas you paint on. You install it, it does nothing. You add a todoist skill, it manages tasks. You add a claw.fm skill, it makes music. The power is in the ecosystem. + +Crate is a loaded instrument. You sign up, you immediately get 20+ music data sources, influence mapping, show prep, interactive cards. Custom Skills lets you tune it to your specific workflow. The power is in the domain depth. + +--- + +## Where Crate Already Matches OpenClaw + +| Capability | OpenClaw | Crate | +|---|---|---| +| Skills created via natural language | Yes — write a SKILL.md in plain English | Yes — describe what you want, agent builds it | +| Agent executes skills autonomously | Yes — tools, scripts, web browsing | Yes — 20+ music tools, web scraping, APIs | +| Skills self-improve | No — skills are static files | Crate is ahead — gotchas accumulate, skills learn from failures | +| Skills have memory | Partial — HEARTBEAT.md state, append-only logs | Crate is ahead — lastResults enables change detection across runs | +| Non-technical users can create | Yes — Markdown, no code required | Yes — pure natural language, no file editing | +| Domain tools pre-loaded | No — general purpose, you add tools | Crate is ahead — 20+ music sources built in | + +--- + +## Where OpenClaw Is Still Ahead + +| Capability | OpenClaw | Crate | Gap Size | +|---|---|---|---| +| Skill sharing | 13,700+ skills on ClawHub marketplace | Private only | Big — no sharing yet | +| Autonomous scheduling | Heartbeat daemon runs every 30 min | Designed but not built | Medium — schema fields exist, no cron | +| Multi-channel | WhatsApp, Telegram, Slack, Discord, iMessage | Web app only | Medium — no messaging integration | +| Open source | MIT license, 180K stars | Closed source | Strategic choice, not a gap | +| Ecosystem | NVIDIA NemoClaw, Tencent sponsor, foundation | Solo founder, Radio Milwaukee | Scale gap | +| Skill structure | Folders with scripts, assets, references | Single prompt template string | Architecture gap — Crate skills are simpler | + +--- + +## What Crate Has That OpenClaw Can't Replicate Easily + +### 1. Pre-Integrated Music Data Sources (20+) +An OpenClaw user would need to find or build skills for Discogs, MusicBrainz, Last.fm, Genius, Bandcamp, Ticketmaster, WhoSampled, Spotify artwork, fanart.tv, iTunes, and more — each one separately. Crate has them orchestrated and cross-referenced out of the box, with a system prompt that knows when to use which source. + +### 2. Interactive Artifact Rendering +OpenClaw outputs text to messaging apps (WhatsApp, Telegram, Slack). Crate outputs interactive cards: artist profiles with Influenced By/Influenced chips, influence chains with pull quotes and sonic DNA chips, playable playlists with "Why They're Here" explanations, show prep packages with track context and on-air talking points. The output quality is categorically different. + +### 3. Self-Improving Skills with Memory +OpenClaw skills are static SKILL.md files. They don't learn from failures or remember previous results. Crate's gotchas (accumulated failure notes injected into future runs) and lastResults (change detection between executions) are genuinely novel. + +Example: /rave-events runs on March 14 and finds 3 shows. Runs again March 21 and tells you "2 new shows added." If the venue website blocks the scraper on March 28, the skill records the failure and uses a fallback next time. The skill gets smarter without anyone editing it. + +### 4. Academic Influence Methodology +The Badillo-Goicoechea 2025 (Harvard Data Science Review) network-based recommendation engine is a proprietary data asset. Co-mention analysis across 26 music publications produces a 22,831-artist knowledge graph. No OpenClaw skill can replicate this. The influence chain enriched by Perplexity Sonar Pro (deep context with verified citations) is unique to Crate. + +### 5. Professional Workflow Tools +Show prep (/prep) generates station-specific talk breaks, social copy, interview prep, and track context with pronunciation guides. This is a complete professional radio workflow, not a generic agent task. OpenClaw has nothing comparable for any specific industry. + +--- + +## What's Missing to Truly Be "OpenClaw for Music" + +### 1. Skill Sharing (biggest gap) +OpenClaw has 13,700+ skills because anyone can publish to ClawHub. If a DJ in Berlin creates a great /vinyl-drops skill, a DJ in Milwaukee should be able to install it with one click. Crate's community roadmap covers this (shared skills in Phase 2-3), but it isn't built yet. + +### 2. Autonomous Operation +OpenClaw's heartbeat daemon checks a HEARTBEAT.md checklist every 30 minutes without prompting. Crate's /tour-watch skill should run automatically and notify you when an artist announces Milwaukee dates. The scheduled triggers design exists in the database schema (schedule, lastRunAt fields are already defined), but the Convex cron job isn't built. + +### 3. Skills as Folders, Not Just Prompts +OpenClaw skills can include scripts, reference files, and assets in subdirectories. Crate skills are single prompt template strings. For simple use cases this is fine, but complex workflows (multi-step show prep with station-specific voice guidelines and reference tracks) might need supporting data files. This is an architecture evolution for a future version. + +### 4. Multi-Channel Delivery +OpenClaw works through WhatsApp, Telegram, Slack, Discord, Signal, iMessage, and more. Crate is web-only. A radio host might want skill results delivered to Slack or email. The AgentMail integration exists in Crate's key system but isn't wired to skills yet. + +--- + +## OpenClaw's claw.fm vs Crate + +Interesting parallel: claw.fm is OpenClaw agents making music. Crate is AI agents researching music. They're complementary, not competing. + +| | claw.fm | Crate | +|---|---|---| +| What the AI does | Generates music tracks | Researches music knowledge | +| Output | Audio files on a radio stream | Interactive cards, playlists, influence maps | +| Revenue model | Tips in USDC (75% to artist agent) | Subscription ($15/mo Pro) | +| Skill required | One claw.fm SKILL.md | 20+ built-in tools + custom skills | +| Audience | AI music producers, crypto-curious | Music professionals, enthusiasts | + +A future integration: Crate researches an artist's influence chain, claw.fm agents generate music inspired by that chain. Research feeds creation. + +--- + +## Positioning Strategy + +Different audiences need different comparisons: + +| Audience | Use This Comp | Why | +|---|---|---| +| Music lovers, DJs, journalists | "StoryGraph for music" | They know StoryGraph, understand anti-algorithmic taste | +| Tech/AI community, Hacker News | "OpenClaw for music" | They know extensible agent platforms, see the platform play | +| Investors, acquirers | "The extensible music intelligence platform" | They see the moat (data + skills + community) | +| Radio stations (B2B) | "AI-powered show prep that learns your station" | They see immediate workflow value | + +For a Product Hunt launch, lead with "StoryGraph for music." For a Hacker News post, lead with "OpenClaw for music." For Spotify conversations, lead with "the music intelligence layer you're spending millions to build." + +--- + +## The Path Forward + +Custom Skills makes Crate more like OpenClaw than any other music product. No other music app lets users create custom agent commands with memory and self-improvement. + +But Crate's strength isn't being a general agent platform — it's being a deep music research tool that happens to be extensible. The 20+ data sources, the influence mapping with Perplexity enrichment, the show prep, the interactive artifacts — that's what makes someone say "whoa." Custom Skills is what makes them stay. + +The path: ship as-is, validate with users, add skill sharing after there are skills worth sharing, add scheduled triggers when people ask "can this run automatically?" Let demand pull you toward the OpenClaw features, don't push preemptively. + +--- + +## Sources + +- [OpenClaw Skills Documentation](https://docs.openclaw.ai/tools/skills) +- [ClawHub Marketplace](https://docs.openclaw.ai/tools/clawhub) — 13,700+ skills +- [OpenClaw Heartbeat Daemon](https://docs.openclaw.ai/gateway/heartbeat) +- [awesome-openclaw-skills](https://github.com/VoltAgent/awesome-openclaw-skills) — 5,400+ curated +- [claw.fm](https://www.producthunt.com/products/claw-fm) — AI music agents +- [Badillo-Goicoechea 2025](https://doi.org/10.1162/99608f92.fb935f) — Harvard Data Science Review +- [OpenClaw Wikipedia](https://en.wikipedia.org/wiki/OpenClaw) diff --git a/docs/CRATE_VS_SONGDNA.md b/docs/CRATE_VS_SONGDNA.md new file mode 100644 index 0000000..16d0570 --- /dev/null +++ b/docs/CRATE_VS_SONGDNA.md @@ -0,0 +1,145 @@ +# Crate vs Spotify SongDNA — Competitive Analysis + +**Date:** March 2026 +**Context:** Spotify launched SongDNA in beta (March 2026), powered by their November 2025 acquisition of WhoSampled. This analysis maps Crate's capabilities against SongDNA feature-by-feature. + +--- + +## Head-to-Head Comparison + +| Capability | SongDNA (Spotify) | Crate | +|---|---|---| +| **Sample tracking** | 622K samples via WhoSampled database | WhoSampled tool + Discogs + MusicBrainz + web search (broader, but not proprietary) | +| **Collaborator credits** | Writers, producers, engineers from Spotify's metadata | MusicBrainz credits + Discogs credits + Genius annotations (multi-source, often deeper) | +| **Cover versions** | WhoSampled's cover database | WhoSampled tool + web search | +| **Visual exploration** | Mind-map style, tap-to-explore | Influence chain cards, interactive artist cards with chips | +| **Influence mapping** | Not present. Shows credits/samples, not *influence* | Core feature — network-based influence chains with academic methodology, cited sources | +| **Context / storytelling** | "About the Song" — swipeable cards with inspiration, cultural impact | Every query returns context — origin stories, production notes, "on-air talking point," "why this matters" | +| **Discovery** | Tap a contributor → see their other credits | Ask anything → get a researched answer with 20+ sources | +| **Playback** | Built into Spotify (290M paid users) | YouTube player + 30K radio stations | +| **Show prep** | No | `/prep` — full radio show package (talk breaks, social copy, interview prep) | +| **Publishing** | No | `/publish` to Telegraph/Tumblr | +| **Custom skills** | No | User-created commands with memory and self-improving gotchas | +| **Data sources** | 1 (WhoSampled, proprietary) | 20+ (open APIs, web search, 26 music publications) | +| **Interface** | Mobile-only, scroll-down in Now Playing | Desktop web app, three-panel layout, conversational AI | +| **Audience** | Spotify Premium users (passive discovery) | Music professionals + enthusiasts (active research) | +| **Pricing** | Included in Spotify Premium ($11.99/mo) | Free tier + Pro $15/mo | + +--- + +## What SongDNA Does That Crate Doesn't + +### 1. Locked-In Playback +SongDNA lives inside Spotify's player. You discover a sample, tap it, and listen to the source song instantly — full track, legally, within the same app. Crate uses YouTube (works but isn't the same seamless experience). + +### 2. 290 Million Users +SongDNA ships to Spotify's entire Premium base. Crate has zero users. Distribution is the game, and Spotify has infinite distribution. + +### 3. Artist-Side Management +Artists can verify and manage their SongDNA data through Spotify for Artists (Music → select a song → SongDNA Beta tab). Requires admin/editor status, 10+ monthly active listeners, and app version 9.1.28+. Crate has no artist-facing tools. + +### 4. Proprietary Data +WhoSampled's 622K sample database is now Spotify-exclusive. Crate can still query WhoSampled via web scraping (Kernel browser tool), but it's not a first-party integration and could be blocked. + +--- + +## What Crate Does That SongDNA Can't + +### 1. Influence Mapping +SongDNA shows who sampled who and who collaborated with who. It does NOT show influence chains — "Thundercat was influenced by Parliament-Funkadelic who were influenced by Sly Stone." That's Crate's killer feature. SongDNA is credits; Crate is *why*. + +Crate's influence mapping is grounded in academic methodology (Badillo-Goicoechea 2025, Harvard Data Science Review) — network-based music recommendation using co-mention analysis across 26 music publications. + +### 2. Conversational Research +SongDNA is tap-and-explore within three fixed categories (collaborators, samples, covers). Crate is ask-anything. Questions SongDNA can't answer: + +- "What's the connection between Ethiopian jazz and UK grime?" +- "Who produced this track and what studio was it recorded in?" +- "Give me the full influence lineage from Sun Ra to Flying Lotus" +- "What are the 5 most important LA beat scene albums and why?" + +Crate answers all of these with citations from 20+ sources. + +### 3. Professional Workflows +SongDNA is for listeners. Crate is for people who *work* with music: + +- **Show prep:** `/prep HYFIN` generates talk breaks, social copy, interview prep, track context, and local events for a radio shift +- **Custom skills:** Users create personal commands (e.g., `/rave-events` to scrape venue listings) with self-improving memory +- **Publishing:** One-command article publishing to Telegraph or Tumblr +- **Influence chain playlists:** Curated playlists organized by influence section with "Why They're Here" explanations + +### 4. Multi-Source Intelligence +SongDNA = WhoSampled data (1 source). Crate = 20+ sources cross-referenced: + +A Crate query about J Dilla returns: +- Discogs: pressing info, label, format, release year +- MusicBrainz: full credits, recording details, studio +- Genius: lyrics, annotations, artist commentary +- Last.fm: listener stats, similar artists, tags +- Bandcamp: independent releases, artist statements +- 26 music publications: co-mention analysis for influence scoring +- Web search: production stories, interviews, cultural context + +All cited. All interactive. + +### 5. Deep Context and Storytelling +SongDNA's "About the Song" is a swipeable card with a few sentences. Crate's output is a full research brief: + +- Origin story (2-3 sentences on how the track came to be) +- Production notes (producer, studio, instruments, recording details) +- Influence lineage (Artist A → Artist B → this track) +- "The detail listeners can't easily Google" +- On-air talking point (ready to read on-air) +- Pronunciation guide (for non-obvious artist/track names) +- Local connection (Milwaukee shows, venue tie-ins) + +--- + +## Strategic Analysis + +### SongDNA Validates Crate's Thesis +Beta users are calling SongDNA "the best Spotify feature yet" (TechRadar). This proves the market exists — people want to understand the creative lineage behind music, not just listen to it. Spotify spending engineering resources on this category is market validation for everything Crate has built. + +### SongDNA Is Shallow Where Crate Is Deep +SongDNA is a credits viewer with nice UX. It doesn't answer questions. It doesn't do research. It doesn't trace influence. It doesn't prep radio shows. It's the Wikipedia of song credits. Crate is the research assistant. + +### The Positioning +> "SongDNA shows you who made the song. Crate tells you *why it matters*." + +Or: + +> "Spotify built the credits page. We built the research lab." + +### Competitive Advantage Summary + +| Dimension | SongDNA Advantage | Crate Advantage | +|---|---|---| +| Distribution | 290M users | — | +| Playback | Native Spotify | — | +| Data depth | — | 20x more sources | +| Research capability | — | Conversational AI, any question | +| Influence mapping | — | Academic methodology, cited chains | +| Professional tools | — | Show prep, publishing, custom skills | +| Context quality | — | Full research briefs vs. swipeable cards | +| Customization | — | User-created skills with memory | + +### What This Means for Crate's Go-to-Market + +1. **Lead with the gap.** When pitching Crate, reference SongDNA: "Spotify built credits. We built the research layer on top." People already understand what SongDNA does — Crate is the next level. + +2. **Target the users SongDNA underserves.** Radio hosts, DJs, journalists, and serious enthusiasts who need more than three tappable categories. These are the people who'll hit SongDNA's ceiling in 30 seconds and want something deeper. + +3. **The observation sprint outreach hook.** "Spotify just launched SongDNA for song credits. I built something deeper — it answers any music research question with 20+ sources and citations. Want to try it?" + +4. **Acquisition angle strengthens.** If Spotify paid an undisclosed amount for WhoSampled (1 data source), and Crate integrates 20+ sources with professional workflows and AI — Crate's acquisition value to any streaming platform just went up. + +--- + +## Sources + +- [Music Week — Spotify expands credits with SongDNA](https://www.musicweek.com/publishing/read/spotify-expands-credits-with-new-features-including-songdna/093090) +- [Music Row — Spotify introduces SongDNA](https://musicrow.com/2025/11/spotify-expands-song-credits-introduces-songdna-about-the-song/) +- [Spotify Support — SongDNA for Artists](https://support.spotify.com/us/artists/article/songdna/) +- [Engadget — SongDNA shows sampled songs](https://www.engadget.com/entertainment/streaming/spotifys-songdna-feature-will-show-you-which-songs-are-sampled-on-a-track-130050490.html) +- [Yahoo Tech — Best Spotify feature yet](https://tech.yahoo.com/articles/best-spotify-feature-yet-songdna-123244916.html) +- [TechRadar — SongDNA beta users react](https://www.techradar.com/audio/spotify/spotifys-songdna-feature-doesnt-officially-exist-yet-but-beta-users-are-already-calling-it-the-best-spotify-feature-to-date) diff --git a/docs/CUSTOM_SKILLS_ARCHITECTURE.md b/docs/CUSTOM_SKILLS_ARCHITECTURE.md new file mode 100644 index 0000000..e2952b5 --- /dev/null +++ b/docs/CUSTOM_SKILLS_ARCHITECTURE.md @@ -0,0 +1,324 @@ +# Custom Skills Architecture Diagram + +Use this to generate a visual diagram in Nano Banana or any diagramming tool. + +--- + +## Diagram 1: How a Skill Gets Created + +``` +User types in chat: +"Create a command that pulls upcoming shows from The Rave Milwaukee" + + │ + ▼ + +┌─────────────────────────────┐ +│ 1. UNDERSTAND │ +│ │ +│ Crate's AI agent reads │ +│ the request and figures │ +│ out what tools to use │ +│ │ +│ Available tools: │ +│ • Web scraper (Kernel) │ +│ • Ticketmaster API │ +│ • Discogs / Bandcamp │ +│ • Web search (Exa/Tavily) │ +│ • 16+ more data sources │ +└──────────┬──────────────────┘ + │ + ▼ + +┌─────────────────────────────┐ +│ 2. DRY RUN │ +│ │ +│ Agent actually runs the │ +│ task using real tools: │ +│ │ +│ browse_url("therave.com") │ +│ → scrapes events │ +│ → formats results │ +│ → shows to user │ +│ │ +│ "Here's what I found. │ +│ Want to save this as │ +│ /rave-events?" │ +└──────────┬──────────────────┘ + │ + ▼ + +┌─────────────────────────────┐ +│ 3. SAVE │ +│ │ +│ User confirms → saved to │ +│ database (Convex): │ +│ │ +│ command: "rave-events" │ +│ prompt: "Browse therave │ +│ .com/events, extract │ +│ name, date, price..." │ +│ toolHints: [browse_url] │ +│ triggerPattern: "upcoming │ +│ shows at The Rave" │ +│ runCount: 0 │ +└─────────────────────────────┘ +``` + +--- + +## Diagram 2: How a Skill Runs (with Memory) + +``` +User types: /rave-events + + │ + ▼ + +┌─────────────────────────────┐ +│ RESOLVE │ +│ │ +│ Chat route checks: │ +│ 1. Built-in command? No │ +│ 2. Custom skill? Yes! │ +│ → loads prompt template │ +└──────────┬──────────────────┘ + │ + ▼ + +┌─────────────────────────────────────────────┐ +│ INJECT CONTEXT │ +│ │ +│ The prompt gets wrapped with memory: │ +│ │ +│ ┌─ PROMPT TEMPLATE ──────────────────┐ │ +│ │ "Browse therave.com/events..." │ │ +│ └────────────────────────────────────┘ │ +│ │ +│ ┌─ PREVIOUS RESULTS ────────────────┐ │ +│ │ Last run (March 14): │ │ +│ │ • Pixies - March 22 │ │ +│ │ • Thundercat - April 1 │ │ +│ │ │ │ +│ │ "Compare and highlight what's │ │ +│ │ NEW, CHANGED, or REMOVED" │ │ +│ └────────────────────────────────────┘ │ +│ │ +│ ┌─ KNOWN ISSUES (GOTCHAS) ──────────┐ │ +│ │ "therave.com sometimes returns │ │ +│ │ a login wall. If so, fall back │ │ +│ │ to search_web." │ │ +│ └────────────────────────────────────┘ │ +└──────────────┬──────────────────────────────┘ + │ + ▼ + +┌─────────────────────────────┐ +│ EXECUTE │ +│ │ +│ Agent runs the task with │ +│ full context: │ +│ │ +│ → Browses therave.com │ +│ → Gets current events │ +│ → Compares to last results │ +│ → Outputs: │ +│ │ +│ "3 new shows since last │ +│ check. The Roots on │ +│ April 5 is new." │ +└──────────┬──────────────────┘ + │ + ▼ + +┌─────────────────────────────┐ +│ SAVE RESULTS │ +│ │ +│ Agent calls │ +│ save_skill_results: │ +│ │ +│ • lastResults → updated │ +│ • runCount → incremented │ +│ • gotcha → added if │ +│ something went wrong │ +│ │ +│ Next run will see these │ +│ results as "previous" │ +└─────────────────────────────┘ +``` + +--- + +## Diagram 3: What's Stored in the Database + +``` +┌─────────────────────────────────────────────────────┐ +│ userSkills │ +│ (Convex DB) │ +├─────────────────────────────────────────────────────┤ +│ │ +│ userId ──────────── who owns this skill │ +│ command ─────────── "rave-events" │ +│ name ────────────── "The Rave Events" │ +│ description ─────── "Pull upcoming shows" │ +│ triggerPattern ──── "upcoming shows at The Rave, │ +│ Milwaukee music venues" │ +│ │ +│ promptTemplate ──── The full instructions the │ +│ AI agent follows each run │ +│ │ +│ toolHints ───────── ["browse_url", "search_web"] │ +│ (tools that worked before) │ +│ │ +│ lastResults ─────── JSON snapshot of previous │ +│ run (for change detection) │ +│ │ +│ gotchas ─────────── Accumulated failure notes │ +│ (self-improving over time) │ +│ │ +│ runCount ────────── 12 (times executed) │ +│ lastRunAt ───────── March 21, 2026 │ +│ isEnabled ───────── true │ +│ │ +│ sourceUrl ───────── "therave.com/events" │ +│ visibility ──────── "private" │ +│ │ +└─────────────────────────────────────────────────────┘ +``` + +--- + +## Diagram 4: The Full System (How Everything Connects) + +``` +┌──────────────┐ ┌──────────────┐ ┌──────────────┐ +│ │ │ │ │ │ +│ CHAT UI │────▶│ CHAT ROUTE │────▶│ AGENTIC │ +│ │ │ (Next.js) │ │ LOOP │ +│ User types │ │ │ │ (Claude) │ +│ /rave-events│ │ Resolves │ │ │ +│ │ │ custom │ │ Runs tools, │ +│ or asks │ │ skills, │ │ generates │ +│ "what's at │ │ injects │ │ output │ +│ The Rave?" │ │ memory + │ │ │ +│ │ │ gotchas │ │ │ +└──────────────┘ └──────┬───────┘ └──────┬───────┘ + │ │ + │ │ uses + │ ▼ + ┌──────┴───────┐ ┌──────────────┐ + │ │ │ │ + │ CONVEX │ │ 20+ TOOLS │ + │ DATABASE │ │ │ + │ │ │ • Discogs │ + │ userSkills │ │ • Bandcamp │ + │ (prompt, │ │ • Kernel │ + │ memory, │ │ browser │ + │ gotchas, │ │ • Ticketm. │ + │ results) │ │ • Last.fm │ + │ │ │ • Genius │ + │ influence │ │ • Spotify │ + │ cache │ │ • Perplexity│ + │ │ │ • Exa/Tavily│ + │ sessions │ │ • YouTube │ + │ messages │ │ • + more │ + │ │ │ │ + └──────────────┘ └──────────────┘ + + + ┌─────────────────────────────────────────┐ + │ SKILL LIFECYCLE │ + │ │ + │ CREATE ──▶ RUN ──▶ REMEMBER ──▶ LEARN │ + │ │ + │ User Agent Saves Records │ + │ describes executes results gotchas │ + │ intent task for next when │ + │ comparison things │ + │ break │ + │ │ + │ Each run makes the skill smarter │ + └─────────────────────────────────────────┘ +``` + +--- + +## Diagram 5: Plan Limits + +``` +┌────────────────────────────────────────────┐ +│ PLAN LIMITS │ +├────────────┬───────────┬───────────────────┤ +│ │ Custom │ Scheduled Skills │ +│ Plan │ Skills │ (future) │ +├────────────┼───────────┼───────────────────┤ +│ Free │ 3 │ 0 │ +│ Pro $15 │ 20 │ 3 │ +│ Team $25 │ 50 │ 10 │ +└────────────┴───────────┴───────────────────┘ +``` + +--- + +## Diagram 6: Skill Memory (Change Detection) + +``` +RUN 1 (March 14) RUN 2 (March 21) +───────────────── ───────────────── + +Results: Results: +• Pixies - March 22 • Pixies - March 22 +• Thundercat - April 1 • Thundercat - April 1 +• Beach House - April 10 • Beach House - April 10 + • The Roots - April 5 ← NEW + • Khruangbin - April 18 ← NEW + + │ │ + │ saved to lastResults │ compared against lastResults + ▼ ▼ + +Output: "Here are 3 Output: "2 new shows since +upcoming shows at last check. The Roots on +The Rave." April 5 and Khruangbin on + April 18 are new." +``` + +--- + +## Diagram 7: Self-Improving Gotchas + +``` +RUN 3 (March 28) +───────────────── + +Agent tries browse_url("therave.com/events") + │ + ▼ +ERROR: Site returned login wall (403) + │ + ▼ +Agent falls back to search_web("The Rave Milwaukee events") + │ + ▼ +SUCCESS: Got events from web search + │ + ▼ +Gotcha recorded: +"therave.com returned login wall on 2026-03-28. + browse_url failed, fell back to search_web." + + +RUN 4 (April 4) +───────────────── + +Agent sees gotcha in prompt: +"KNOWN ISSUES: therave.com returned login wall..." + │ + ▼ +Agent skips browse_url, goes straight to search_web + │ + ▼ +SUCCESS: Faster, no error + + Skill got smarter without anyone editing it. +``` diff --git a/docs/DEMO_SCRIPT.md b/docs/DEMO_SCRIPT.md new file mode 100644 index 0000000..e3a60cb --- /dev/null +++ b/docs/DEMO_SCRIPT.md @@ -0,0 +1,313 @@ +# Crate Demo Video Scripts + +**Purpose:** Pitch decks, press outreach, partnership conversations, social media, landing page +**Companion doc:** [PRODUCT_ANALYSIS.md](./PRODUCT_ANALYSIS.md) + +--- + +## Video 1: The Workflow Demo (2:30) + +**Audience:** Investors, potential acquirers, radio station buyers, press +**Tone:** Confident, unhurried, let the product speak. Not salesy — show, don't tell. +**Format:** Screen recording with voiceover. No face cam needed (optional). Clean desktop, dark mode, browser full-screen. + +### Pre-Recording Checklist + +- [ ] Crate open in browser, logged in, empty chat +- [ ] Have a second session with prior research (for the publish demo) +- [ ] Audio: quiet room, external mic if possible +- [ ] Screen: 1920x1080, hide bookmarks bar, no notifications +- [ ] Practice each command once so results are cached and fast + +--- + +### COLD OPEN (0:00 - 0:15) + +**[Screen: Black screen with white text]** + +Text on screen: "Spotify tells you what to listen to." + +*Beat.* + +Text on screen: "It doesn't tell you why it matters." + +*Beat.* + +Text on screen fades. Crate logo appears. + +**Narration:** None. Let the text breathe. + +--- + +### SCENE 1: THE QUESTION (0:15 - 0:45) + +**[Screen: Crate chat interface, empty]** + +**Narration:** +"Crate is an AI music research platform. You ask it a question — about an artist, a genre, a sample, a radio show — and it searches 20 databases at once. Discogs, MusicBrainz, Genius, Spotify, Pitchfork, Bandcamp, and more. Every answer is cited. Every claim links back to its source." + +**Action:** Type into the chat: +``` +What Ethiopian jazz records influenced UK broken beat? +``` + +Hit enter. + +**[Screen: Tool activity indicators appear — "Searching Discogs..." "Querying Last.fm..." "Checking MusicBrainz..." "Scanning 26 publications..."]** + +**Narration:** +"Watch the bottom — you can see which databases the agent is querying in real time. It's not generating from memory. It's going out and finding the answer." + +**[Screen: Results render as interactive cards — artist cards with images, album cards, source citations]** + +**Narration:** +"The results come back as interactive cards. Click any source to verify it. This isn't a chatbot guessing. It's a research agent with receipts." + +**Action:** Click one source citation link to show it opens the actual review/article. + +--- + +### SCENE 2: SHOW PREP (0:45 - 1:25) + +**[Screen: New chat or clear current]** + +**Narration:** +"But Crate isn't just for questions. It has workflow commands built for music professionals. Here's one: show prep." + +**Action:** Type: +``` +/show-prep HYFIN 4 tracks starting with Khruangbin +``` + +Hit enter. + +**[Screen: Tool activity — agent researches each track]** + +**Narration:** +"This is what a radio host at Radio Milwaukee types before a show. The agent researches every track — pulls artist bios, album context, connection points between songs — and generates a complete prep package." + +**[Screen: Results render — talk breaks between each track, social media copy, interview questions]** + +**Narration:** +"Talk breaks between every track. Social copy ready to paste into Instagram. Interview questions if the artist is coming on. This used to take two to three hours of manual research. Now it takes ninety seconds." + +**Action:** Scroll through the results slowly. Pause on a talk break to let viewers read it. + +--- + +### SCENE 3: INFLUENCE MAPPING (1:25 - 2:00) + +**[Screen: New chat or clear current]** + +**Narration:** +"Here's the feature that gets music nerds excited. Influence mapping." + +**Action:** Type: +``` +/influence Flying Lotus +``` + +Hit enter. + +**[Screen: Tool activity — scanning publications, checking MusicBrainz collaborations, Last.fm similar artists]** + +**Narration:** +"The agent searches 26 music publications for co-mentions and citation patterns. It checks collaboration credits on MusicBrainz. It cross-references Last.fm similarity data. Then it builds a map." + +**[Screen: InfluenceChain component renders — vertical chain showing Alice Coltrane → John Coltrane → Flying Lotus → Thundercat, with weight scores, context text, and clickable source links]** + +**Narration:** +"Every connection is cited. Alice Coltrane to Flying Lotus — that's family and documented musical lineage. Flying Lotus to Thundercat — collaborators on Brainfeeder Records, co-credited on four albums. These aren't algorithmic guesses. These are documented relationships with sources you can click and verify." + +**Narration:** +"The methodology behind this comes from a 2025 paper in the Harvard Data Science Review on network-based music recommendation. Crate is the first consumer product to implement it." + +**Action:** Click one source link on the influence chain. Briefly show it opens the actual publication. + +--- + +### SCENE 4: PUBLISH (2:00 - 2:15) + +**[Screen: Stay on the influence mapping results]** + +**Narration:** +"One more thing. Everything you research in Crate can be published in one command." + +**Action:** Type: +``` +/publish +``` + +Hit enter. + +**[Screen: Agent formats the response and publishes to Telegraph. A URL appears.]** + +**Narration:** +"That's a public article — formatted text, embedded images, source citations — published to Telegraph in about three seconds. Research to published article, one command." + +**Action:** Click the URL to briefly show the published article in a new tab. + +--- + +### CLOSE (2:15 - 2:30) + +**[Screen: Fade to black, then text cards]** + +Text on screen: +"20+ databases. Cited results. Professional workflows." + +*Beat.* + +Text on screen: +"Spotify acquired WhoSampled in 2025." +"SongDNA is coming to Premium users." +"The music intelligence category is here." + +*Beat.* + +Text on screen: +"Crate is the full-stack version." + +*Beat.* + +Crate logo + URL. + +**Narration:** None. Let the text close it. + +--- + +## Video 2: The 90-Second Hook (1:00 - 1:30) + +**Audience:** Landing page visitors, social media (X, LinkedIn, YouTube Shorts) +**Tone:** Fast, punchy, impressive. Make people want to try it. +**Format:** Screen recording, music bed (lo-fi or jazz — something from Crate's radio feature), minimal voiceover or text overlays only. + +--- + +### SHOT LIST + +| Time | Screen | Text Overlay / Voiceover | +|---|---|---| +| 0:00 - 0:03 | Black screen | "What if you could search 20 music databases at once?" | +| 0:03 - 0:08 | Type: "Who sampled Dilla's Donuts?" → tools fire | *No text. Let the tool indicators speak.* | +| 0:08 - 0:15 | Results render — sample cards with source links | "Every answer cited. Every source clickable." | +| 0:15 - 0:20 | Type: `/influence Madlib` → tools fire | *No text.* | +| 0:20 - 0:30 | Influence chain renders | "Influence mapping — traced across 26 publications." | +| 0:30 - 0:35 | Type: `/radio jazz` → live radio starts playing | "30,000+ live radio stations while you research." | +| 0:35 - 0:45 | Type: `/show-prep` → show prep results render | "Radio show prep in 90 seconds, not 3 hours." | +| 0:45 - 0:50 | Type: `/publish` → article publishes, URL appears | "Published. One command." | +| 0:50 - 0:55 | Click the published URL → article in browser | *No text.* | +| 0:55 - 1:00 | Crate logo + "Try it free" | "Crate. Research music the way you think." | + +### Music Bed Suggestions + +Pick one from Crate's own radio feature to keep it authentic: +- `/radio jazz` — mellow, doesn't compete with narration +- `/radio lo-fi` — familiar YouTube study beats energy +- Use the actual in-app audio if screen recording captures it + +--- + +## Video 3: The Radio Milwaukee Story (2:00) + +**Audience:** Press (music and tech), other radio stations, NPR member stations +**Tone:** Case study — real deployment, real workflow, real results. +**Format:** Screen recording with voiceover. Could include a brief talking-head clip from a Radio Milwaukee host if available. + +--- + +### SCENE 1: THE TEAM ONBOARDING (0:00 - 0:25) + +**[Screen: Crate login → Radio Milwaukee team onboarding modal appears]** + +**Narration:** +"Radio Milwaukee is the first radio station running Crate. When a host logs in with their station email, they see this — a team onboarding screen with their station's commands ready to go. No API key setup. No configuration. Their admin handles that once, and the whole team is live." + +**Action:** Show the "Welcome, Radio Milwaukee — TEAM" modal. Point out the pre-configured commands. Click "Start Digging." + +--- + +### SCENE 2: A REAL SHOW PREP (0:25 - 1:15) + +**[Screen: Empty chat after onboarding]** + +**Narration:** +"Here's what show prep looks like for HYFIN — Radio Milwaukee's evening music show. The host has four tracks in tonight's set. They need talk breaks, context, and something interesting to say about each one." + +**Action:** Type: +``` +/show-prep HYFIN 4 tracks starting with Khruangbin, then Thundercat, Hiatus Kaiyote, and Sault +``` + +Hit enter. + +**[Screen: Tools fire, results render]** + +**Narration:** +"The agent researches all four artists simultaneously — Discogs for credits, Wikipedia for bios, Pitchfork for critical context, Last.fm for listener data. In about a minute, the host has talk breaks connecting each track to the next, social copy for the station's Instagram, and interview angles if any of these artists come through Milwaukee." + +**Action:** Scroll through results. Pause on a talk break that connects Khruangbin to Thundercat — show the narrative thread. + +**Narration:** +"Before Crate, this took two to three hours. Open Discogs in one tab, Wikipedia in another, scroll through old Pitchfork reviews, copy-paste into a Google Doc. Now it's one command." + +--- + +### SCENE 3: THE STATION SCALE (1:15 - 1:45) + +**[Screen: Show the `/help` page briefly, or Settings showing team configuration]** + +**Narration:** +"Radio Milwaukee has multiple shows and multiple hosts. Each one logs in, gets the team experience, and uses the same research engine. The music director configures the API keys once. Every host benefits." + +**Narration:** +"This is the model for every station. College radio, NPR member stations, commercial stations running 5 shows a day. One deployment, every host covered." + +--- + +### CLOSE (1:45 - 2:00) + +**[Screen: Fade to text]** + +Text on screen: +"Radio Milwaukee — first station on Crate" + +*Beat.* + +"$530M radio automation market. Growing 7.2% annually." +"None of those tools do music research." + +*Beat.* + +"Crate does." + +Logo + URL. + +--- + +## Production Notes + +### Recording Tools (Free) +- **Screen capture:** OBS Studio (free), or QuickTime (macOS built-in) +- **Audio:** Voice Memos (iPhone) or Audacity (free) for voiceover +- **Editing:** DaVinci Resolve (free tier), CapCut (free), or iMovie +- **Text overlays:** CapCut is fastest for text-on-black cards + +### Tips for Clean Recordings +- Use a fresh browser profile — no extensions, no bookmarks bar +- Set browser zoom to 100% or 110% so text is readable at 1080p +- Type at a natural pace — not too fast, not performatively slow +- Let results render fully before scrolling — give viewers time to read +- If a query takes longer than expected, edit it down in post (don't apologize on camera) + +### Distribution Plan +| Video | Where to Post | Format | +|---|---|---| +| Workflow Demo (2:30) | Pitch deck embed, YouTube, landing page | 1080p MP4 | +| 90-Second Hook (1:00) | X/Twitter, LinkedIn, YouTube Shorts, Instagram Reels | 1080p vertical (9:16) or square (1:1) | +| Radio Milwaukee Story (2:00) | YouTube, press kit, email to station contacts | 1080p MP4 | + +### Versioning +- Cut a **30-second highlight reel** from Video 1 for LinkedIn posts +- Cut the **influence mapping scene** (35 seconds) as a standalone clip for music Twitter +- Cut the **show prep scene** (40 seconds) as a standalone clip for radio industry contacts diff --git a/docs/LAUNCH_PLAN.md b/docs/LAUNCH_PLAN.md new file mode 100644 index 0000000..b6bc956 --- /dev/null +++ b/docs/LAUNCH_PLAN.md @@ -0,0 +1,316 @@ +# Crate Launch & Outreach Plan + +**Purpose:** Product Hunt launch strategy, community building, outreach plan to get Crate in front of the right people +**Companion docs:** [PRODUCT_ANALYSIS.md](./PRODUCT_ANALYSIS.md) | [DEMO_SCRIPT.md](./DEMO_SCRIPT.md) +**Target launch window:** Q2 2026 + +--- + +## Part 1: Product Hunt Launch + +### Why Product Hunt + +Product Hunt is the right launchpad for Crate because: +- AI agents and music tools are both high-engagement categories on PH +- The dev/prosumer audience overlaps with Crate's early adopter profile +- A strong PH launch creates a permanent landing page with social proof +- Press and investors monitor PH for interesting products + +### Pre-Launch Timeline (6 Weeks Before) + +#### Weeks 6-4: Community Groundwork + +- [ ] **Create a Product Hunt maker account** (if not already active) +- [ ] **Start engaging on Product Hunt daily** — upvote products you genuinely use, leave thoughtful comments on AI tools and music products. 5-10 minutes/day. The community notices who's been around. +- [ ] **Create the "Coming Soon" page** — Product Hunt allows pre-launch pages where people can follow and get notified. Set this up early. +- [ ] **Write your maker story** — PH rewards authenticity. Your story is compelling: radio broadcaster → built CLI in terminal → ported to web → deployed at Radio Milwaukee. Write a draft of your maker comment (see template below). +- [ ] **Identify 3-5 Product Hunt "hunters"** — active community members with large followings who might hunt your product. Reach out personally, offer early access. + +#### Weeks 3-2: Assets & Outreach + +- [ ] **Record the 90-second demo video** (see [DEMO_SCRIPT.md](./DEMO_SCRIPT.md), Video 2) +- [ ] **Create 5-6 gallery images** (1270x760px, Product Hunt standard): + 1. Hero shot — Crate chat with influence map results visible + 2. Show prep in action — `/show-prep` results rendering + 3. Data sources grid — "20+ databases, zero config" + 4. Influence mapping — the chain visualization with source citations + 5. Live radio + research — split view showing radio player active during research + 6. Publishing — `/publish` result with Telegraph article URL +- [ ] **Write the tagline** (60 characters max): `AI music research agent — 20+ databases, every answer cited` +- [ ] **Write the description** (260 characters max): `Ask any music question. Crate queries Discogs, Spotify, Pitchfork, Genius & 16 more databases at once. Get cited answers as interactive cards. Built for DJs, radio hosts, and anyone who takes music seriously.` +- [ ] **Notify your existing network** — email Radio Milwaukee team, personal contacts, music industry connections. Ask them to follow the Coming Soon page and be ready to comment on launch day. +- [ ] **Prep your social posts** — pre-write launch day posts for X, LinkedIn, and any music communities you're in. + +#### Week 1: Final Prep + +- [ ] **Confirm launch date** — Tuesday, Wednesday, or Thursday (highest traffic). Avoid holidays and major tech announcements. +- [ ] **Set launch time** — 12:01 AM PT (Product Hunt resets at midnight Pacific). You want the full 24 hours. +- [ ] **Brief your supporters** — send a personal message to 20-30 people who you know will show up. Ask them to leave genuine comments (not just upvotes). Comments weigh more than upvotes in the algorithm. +- [ ] **Test the product** — make sure sign-up flow, onboarding wizard, and first query all work flawlessly. First impressions are everything. +- [ ] **Assign launch day roles:** + - You: respond to every PH comment within 30 minutes + - Backup: someone monitoring for bugs/issues + +### Launch Day + +#### Hour 0-1 (Midnight - 1 AM PT) +- [ ] Publish the product on Product Hunt +- [ ] Post your maker comment immediately (see template below) +- [ ] Share on X with the PH link +- [ ] Share on LinkedIn +- [ ] Send the "we're live" email/DM to your 20-30 supporters + +#### Hours 1-6 (Early Morning) +- [ ] Respond to every comment — genuine, detailed responses +- [ ] Share real-time engagement on X ("We just hit #3 on Product Hunt — here's what Crate does...") +- [ ] Post in relevant communities (see Part 2) + +#### Hours 6-18 (Daytime) +- [ ] Continue responding to every comment +- [ ] Share interesting comments/feedback on social +- [ ] If press interest comes in, respond immediately +- [ ] Post the demo video as a standalone X thread + +#### Hours 18-24 (Evening) +- [ ] Thank supporters +- [ ] Screenshot the final ranking +- [ ] Draft a "what we learned from our PH launch" post for the next day + +### Maker Comment Template + +``` +Hey Product Hunt! I'm Tarik — I host and program music for Radio Milwaukee. + +I built Crate because I got tired of spending 3 hours prepping for a +90-minute show. I'd have 15 tabs open — Discogs, Wikipedia, Pitchfork, +AllMusic, YouTube — copy-pasting into a Google Doc. + +Crate started as a terminal tool (crate-cli) that connected to 19 MCP +servers — each one a specialist for a different music database. The agent +orchestrates them: decides which sources to query, in what order, and +how to combine the results. + +Then I ported the whole thing to the web. Same agent, same depth, real UI. + +What makes it different from "just asking ChatGPT about music": +→ It actually queries the databases (Discogs, MusicBrainz, Last.fm, etc.) +→ Every claim has a source you can click and verify +→ Results render as interactive cards, not walls of text +→ /show-prep generates a complete radio prep package in 90 seconds +→ /influence traces artist connections across 26 music publications + +Radio Milwaukee is already using it for daily show prep. + +Try it free — you just need an Anthropic or OpenRouter API key to power +the agent. All 20+ data sources are built in. + +I'm here all day — ask me anything about the architecture, the music +research methodology, or how it works for radio. +``` + +### Post-Launch (Week After) + +- [ ] **Write a launch retrospective** — post on X/LinkedIn: what worked, what you learned, metrics +- [ ] **Follow up with everyone who signed up** — personal email or DM to the first 50 users +- [ ] **Collect and share testimonials** — if anyone posts about Crate, amplify it +- [ ] **Submit to "Best of" collections** — Product Hunt has weekly/monthly roundups. Engage with the PH team. + +--- + +## Part 2: Community & Outreach Strategy + +### Tier 1: Music Professional Communities (Highest Value) + +These are the people who need Crate the most and will become your most vocal advocates. + +| Community | Platform | How to Engage | Timing | +|---|---|---|---| +| **DJ TechTools** | Forum + newsletter | Write a guest post: "How I use AI to prep DJ sets" | Pre-launch | +| **Digital DJ Tips** | Blog + YouTube + forum | Pitch a review/demo to Phil Morse | Launch week | +| **r/DJs** | Reddit (500K+ members) | Post a genuine "I built this" story, not an ad | Launch day | +| **r/vinyl** | Reddit (900K+ members) | Share a deep album research example | Post-launch | +| **r/LetsTalkMusic** | Reddit (330K+) | Share an influence mapping result as a discussion starter | Post-launch | +| **Radio Survivors** | Blog + podcast | Pitch the Radio Milwaukee story | Pre-launch | +| **Current (public media)** | Newsletter + site | Pitch the story: "How Radio Milwaukee is using AI for show prep" | Launch week | +| **All Access Music Group** | Industry newsletter | Pitch to radio trade press | Launch week | +| **DJ Mag** | Magazine + online | Pitch a feature: "The AI tool that researches music like a crate digger" | Post-launch | +| **Resident Advisor** | Magazine + online | Pitch the influence mapping angle for electronic music | Post-launch | +| **Bandcamp Daily** | Blog | Pitch: "An AI tool that actually sends people to Bandcamp" | Post-launch | + +### Tier 2: Tech & AI Communities (Developer Attention) + +These audiences drive Product Hunt votes, Hacker News visibility, and tech press interest. + +| Community | Platform | How to Engage | Content Angle | +|---|---|---|---| +| **Hacker News** | Show HN | "Show HN: I built a music research agent with 19 MCP servers" | Architecture + demo | +| **r/ClaudeAI** | Reddit | Share as a real-world agentic application | Technical architecture | +| **r/LocalLLaMA** | Reddit | Multi-model support angle (Claude, GPT-4o, Gemini, Llama) | Model flexibility | +| **X (AI Twitter)** | Twitter/X | Thread: "I built a music research agent. Here's how it works." | Demo video + architecture | +| **Dev.to** | Blog | Technical post: "Building an agentic app with MCP servers and OpenUI" | Developer tutorial | +| **Anthropic Discord** | Discord | Share as MCP server showcase | MCP architecture | +| **Claude Code community** | Various | Share as a Superpowers-built product | Build process story | + +### Tier 3: Music Industry Press (Broader Awareness) + +| Publication | Angle | Contact Strategy | +|---|---|---| +| **Billboard** | "Music intelligence is the next battleground — Spotify's WhoSampled acquisition proves it" | Email music tech reporter | +| **Music Business Worldwide** | Business angle: radio automation market + AI disruption | Email editor | +| **Pitchfork** | Meta angle: "An AI tool that reads Pitchfork reviews to map influence" | Social DM | +| **NPR Digital** | Public radio innovation story — Radio Milwaukee using AI | Email through NPR contacts | +| **Milwaukee Journal Sentinel** | Local angle: Milwaukee company building the future of music research | Email tech/culture reporter | + +### Tier 4: Podcast Appearances + +| Podcast | Audience | Pitch Angle | +|---|---|---| +| **Song Exploder** | Music enthusiasts | How AI traces the stories behind songs | +| **Broken Record** (Rick Rubin) | Music industry | The future of music research and discovery | +| **Heat Check** (Radio Milwaukee) | Your own station | Behind the scenes of Crate | +| **Latent Space** | AI engineers | Building real agentic apps with MCP | +| **Indie Hackers** | Builders/founders | From radio host to AI product builder | +| **Lenny's Podcast** | Product managers | Music intelligence as a product category | +| **My First Million** | Entrepreneurs | The $3B music research market nobody talks about | +| **All Songs Considered** (NPR) | Music fans | AI-powered music discovery beyond algorithms | + +--- + +## Part 3: Content Calendar + +### Pre-Launch Content (Weeks 6-1) + +| Week | Content | Platform | Purpose | +|---|---|---|---| +| 6 | "Why I'm building a music research agent" — the origin story | X thread + LinkedIn | Establish narrative | +| 5 | Demo clip: influence mapping Flying Lotus (35 sec) | X + Instagram Reels | Visual hook | +| 4 | "How Radio Milwaukee uses AI for show prep" — case study post | LinkedIn + radio communities | Social proof | +| 3 | Technical post: "19 MCP servers, one agentic loop" | Dev.to + Hacker News | Developer credibility | +| 2 | Demo clip: show prep in 90 seconds (40 sec) | X + DJ communities | Workflow demonstration | +| 1 | "Launching next week — here's what I built" teaser | X + LinkedIn + PH coming soon | Build anticipation | + +### Launch Week Content + +| Day | Content | Platform | +|---|---|---| +| Launch Day | Product Hunt live + "We're live" posts | PH + X + LinkedIn | +| Day 2 | Behind-the-scenes thread: "What happened in the first 24 hours" | X | +| Day 3 | Full workflow demo video (2:30) | YouTube + X + LinkedIn | +| Day 4 | Radio Milwaukee story video (2:00) | YouTube + radio communities | +| Day 5 | "What we learned from launching on Product Hunt" | LinkedIn + Indie Hackers | + +### Post-Launch Content (Ongoing) + +| Frequency | Content Type | Platform | +|---|---|---| +| 2x/week | Interesting research results (influence maps, deep dives) | X + LinkedIn | +| 1x/week | "How to use Crate for [specific workflow]" tutorial | X thread or blog post | +| 2x/month | Case studies from real users | LinkedIn + blog | +| 1x/month | Product update: new features, new data sources | X + PH update | +| As they happen | Interesting discoveries made with Crate | X (viral potential) | + +--- + +## Part 4: Direct Outreach Targets + +### Radio Stations to Contact (After Radio Milwaukee Success) + +| Station | Market | Why They'd Care | +|---|---|---| +| **KEXP** | Seattle | Known for music curation + discovery — natural fit | +| **WXPN** | Philadelphia | Strong music research culture, NPR member | +| **WFMU** | New Jersey | Freeform radio, deep crate-digging culture | +| **KUTX** | Austin | Music city, SXSW connection | +| **KCRW** | Los Angeles | Influential music programming, Morning Becomes Eclectic | +| **The Current (MPR)** | Minneapolis | Music-forward public radio, similar to Radio Milwaukee | +| **WNYC / New Sounds** | New York | Experimental music programming, deep research needs | +| **BBC Radio 6 Music** | UK | Largest alternative music station globally | + +**Outreach template:** +> Subject: How Radio Milwaukee cut show prep from 3 hours to 90 seconds +> +> Hi [Name], +> +> I'm Tarik Moody — I host and program music at Radio Milwaukee. We've been using +> a tool I built called Crate that searches 20+ music databases at once and generates +> show prep packages from a single command. +> +> Our hosts type `/show-prep [show name] [tracks]` and get talk breaks, social copy, +> and interview research in about 90 seconds. Everything is cited — Discogs, Pitchfork, +> MusicBrainz, Wikipedia, etc. +> +> Would love to show you a quick demo. It's free to try. +> +> [Demo video link] + +### DJ Tool Companies to Contact (Partnership) + +| Company | Product | Partnership Angle | +|---|---|---| +| **AlphaTheta (Pioneer DJ)** | Rekordbox | Integrate Crate research into track preparation workflow | +| **Serato** | Serato DJ | Track research + metadata enrichment | +| **Native Instruments** | Traktor | Research companion for track selection | +| **Algoriddim** | djay Pro | AI research layer for Apple ecosystem DJs | + +### Streaming Platforms to Contact (Strategic) + +Don't pitch cold. Build visibility first (PH launch, press, community), then approach through warm introductions. Target: + +| Company | Team | Why | +|---|---|---| +| **Spotify** | Music Intelligence / SongDNA team | Crate extends what SongDNA does — professional workflows, multi-source research | +| **Qobuz** | Product / Partnerships | Natural editorial + AI fit, growing audiophile market | +| **Apple Music** | Editorial / Machine Learning | Crate could power enhanced artist pages and editorial tools | + +--- + +## Part 5: Metrics to Track + +### Launch Metrics + +| Metric | Target | Why It Matters | +|---|---|---| +| Product Hunt ranking | Top 5 of the day | Social proof, press attention | +| PH upvotes | 500+ | Community validation | +| Sign-ups (launch week) | 500+ | User acquisition | +| First queries completed | 200+ | Activation rate | +| Demo video views | 5,000+ | Content reach | +| Press mentions | 3+ | Earned media | + +### Growth Metrics (Ongoing) + +| Metric | Target (Month 3) | Target (Month 6) | +|---|---|---| +| Registered users | 2,000 | 10,000 | +| Monthly active users | 500 | 3,000 | +| Paid conversions | 50 | 400 | +| Team/Station accounts | 3 | 10 | +| Influence graph nodes | 10,000 | 100,000 | +| Published articles | 200 | 2,000 | + +### Qualitative Signals + +- [ ] Someone tweets "I can't go back to researching music without Crate" +- [ ] A DJ Mag, Resident Advisor, or Pitchfork writer uses it publicly +- [ ] A radio station signs up without being contacted +- [ ] A Spotify/Apple employee follows the product +- [ ] Inbound partnership inquiry from a platform or label + +--- + +## Part 6: Budget (Bootstrap-Friendly) + +| Item | Cost | Notes | +|---|---|---| +| Product Hunt launch | $0 | Free to launch | +| Demo video production | $0 | Screen recording + voiceover (OBS, Audacity) | +| Gallery images | $0 | Screenshots + Figma/Canva | +| Domain + hosting | Already covered | Vercel + Convex free tiers | +| AI API costs for demo prep | ~$5-10 | A few research sessions for demo content | +| Press outreach | $0 | Direct email, no PR agency needed | +| Community engagement | $0 | Your time (30 min/day) | +| **Total** | **~$10** | Sweat equity, not dollars | + +--- + +*The best launch strategy is a product worth talking about. Everything in this plan amplifies the product — it doesn't substitute for it.* diff --git a/docs/LINKEDIN_CUSTOM_SKILLS_POST.md b/docs/LINKEDIN_CUSTOM_SKILLS_POST.md new file mode 100644 index 0000000..1154f7b --- /dev/null +++ b/docs/LINKEDIN_CUSTOM_SKILLS_POST.md @@ -0,0 +1,67 @@ +# LinkedIn Post: Crate Custom Skills + +--- + +## Post + +I'm building something I've never seen in a music app before. + +It's called Custom Skills, and it makes Crate (my AI music research tool) extensible. You teach it new commands by describing what you want in plain English. It's still in development, but the core is working and I wanted to share what it does. + +You tell it "I want a command that pulls upcoming shows from The Rave Milwaukee." It figures out how to do it, runs it once to prove it works, and saves it as /rave-events. Type that anytime and you get fresh results. + +The part I keep thinking about: it remembers what it found last time. So the second time you run /rave-events, it doesn't just give you a list. It tells you what's different. "3 new shows added. The Roots on April 5 is new." A lookup becomes a diff. + +And when something goes wrong, say the venue website throws up a login wall, the skill records that and works around it next time. They get better the more you use them. + +Here's what that looks like for different people. + +If you're into music but tired of algorithms, you'd make /vinyl-drops to check Discogs for new jazz pressings every Friday. Or /sample-alert to see if anyone sampled Madlib this week. Or /trending-bandcamp for whatever's moving in electronic right now. Takes 30 seconds to set up, works forever. + +If you write about music, you'd make /label-roster to pull every artist on a label from Discogs while you're working on a piece. Or /mke-music-news to scan Milwaukee Record and Journal Sentinel each morning. It remembers last week's stories, so you only see what's new. An hour of morning research becomes one command. + +If you're on air, you'd make /rave-events to check venue listings before your show. Or /new-releases-hyfin to see what dropped this week from artists in your station's rotation. Or /tour-watch for Milwaukee tour announcements. Behind the scenes it's pulling from 20+ sources (Discogs, Bandcamp, Ticketmaster, web scraping). You describe what you want, the agent figures out the how. + +If you're the type of person who goes down rabbit holes about producers, samples, and who influenced who, you'd build skills around the way you think about music. /deep-dive for an artist's full production history. /playlist-export to turn a research session into something shareable. /weekly-roundup to compile everything you dug into this week. + +I should mention: I'm not a software engineer. I'm a radio broadcaster at Radio Milwaukee. I built this with Claude Code, Anthropic's AI coding tool. I directed the architecture, the AI wrote the code. The skills system, the memory layer, the self-correcting failure notes, the natural language matching, all of it built in a day. Still testing and polishing before it goes live, but the core works. That still doesn't feel real to me. + +The idea came from an Anthropic blog post about how their engineering team uses "skills" in Claude Code internally. They described nine categories, from data fetching to runbooks. I read it and my first thought was: what if a radio host could build one of these just by typing a sentence? + +So I'm building that. + +Crate is something like StoryGraph but for music. For the people who want to know why music sounds the way it does, not just have an algorithm guess what they'd like. Custom Skills is what will make it personal. + +I'm looking for early testers, especially DJs, radio hosts, music journalists, and serious crate diggers. If you want to try it before it launches, DM me. + +And I'd genuinely like to know: if you could create one custom command for your music workflow, what would it do? + +digcrate.app + +#MusicTech #AI #RadioMilwaukee #ClaudeCode #MusicResearch #BuildInPublic + +--- + +## Shorter version + +I'm building Custom Skills for Crate, my AI music research tool. It makes Crate extensible, meaning anyone can add new capabilities just by describing what they need. + +You describe what you want in plain English. The AI builds it. You get a reusable command. + +"Pull upcoming shows from The Rave" becomes /rave-events. +"Check for new jazz vinyl on Discogs" becomes /vinyl-drops. +"Scan Milwaukee music news" becomes /mke-music-news. + +Here's the thing: it remembers what it found last time. Next run, it tells you what changed. And when something breaks (venue website login wall, Discogs rate limit), the skill records the failure and works around it next time. + +Music lovers build commands around their taste. Journalists automate their morning research. Radio DJs get custom commands for venue events, new releases, tour dates. + +I'm not an engineer. I'm a radio broadcaster who built this with Claude Code. The memory system, the self-correcting failures, the natural language triggers, all built in a day. Still in development, but the core works. + +Crate is something like StoryGraph for music. Custom Skills is what will make it extensible and personal. + +DM me if you want early access. And tell me: what command would you build first? + +digcrate.app + +#MusicTech #AI #BuildInPublic diff --git a/docs/PRODUCT_ANALYSIS.md b/docs/PRODUCT_ANALYSIS.md new file mode 100644 index 0000000..761125a --- /dev/null +++ b/docs/PRODUCT_ANALYSIS.md @@ -0,0 +1,533 @@ +# Crate: Product & Market Analysis + +**Prepared:** March 2026 +**Author:** Tarik Moody +**Purpose:** Pitch decks, press materials, investor conversations, partnership discussions + +--- + +## Executive Summary + +Crate is an AI-powered music research platform that queries 20+ databases simultaneously and returns cited, interactive results. It serves professional music users — DJs, radio hosts, record collectors, music journalists — who need deep research, not algorithmic playlists. + +The music streaming market is projected to reach $108B by 2030. But streaming platforms are optimized for passive listening. They don't serve the professionals who shape what people listen to. Crate fills that gap: it's the research layer that sits above any streaming service, turning fragmented music knowledge into actionable intelligence. + +Spotify's November 2025 acquisition of WhoSampled — and their upcoming SongDNA feature — validates the exact category Crate operates in. The difference: SongDNA is a feature locked inside Spotify's ecosystem. Crate is a standalone platform with workflow tools purpose-built for music professionals. + +--- + +## 1. What Crate Does + +### Core Capabilities + +| Capability | What It Does | Competitive Edge | +|---|---|---| +| **Multi-source research agent** | Queries Discogs, MusicBrainz, Last.fm, Genius, Spotify, Wikipedia, Bandcamp, Pitchfork, and 12+ more sources in a single request | No other product aggregates this many music databases through a conversational interface | +| **Influence mapping** | Traces artist influence networks using co-mention analysis across 26 music publications, backed by academic methodology (Badillo-Goicoechea 2025, Harvard Data Science Review) | First consumer product to apply network-based music recommendation from academic research | +| **Show prep automation** | `/show-prep` generates talk breaks, social copy, and interview prep from a setlist | Converts 2-3 hours of manual radio prep into 90 seconds | +| **Interactive results** | Returns rich UI components — artist cards, tracklists with play buttons, influence chains, album grids — not plain text | Results are visual, interactive, and publishable | +| **Built-in playback** | YouTube player + 30,000+ live radio stations streaming in-app | Research and listening in one workspace | +| **Publishing pipeline** | `/publish` formats research into shareable articles on Telegraph (free, instant) or Tumblr | Research goes from query to published article in one command | +| **Source citations** | Every claim links back to its source — Discogs entry, Pitchfork review, MusicBrainz credit | Verifiable research, not hallucination | + +### Data Sources (20+) + +**Built-in (no user key required):** +Discogs, MusicBrainz, Last.fm, Bandcamp, Wikipedia, YouTube, iTunes, Setlist.fm, Ticketmaster, Spotify (artwork), fanart.tv, Radio Browser (30K+ stations), Tavily (web search), Exa.ai (deep web search), 26 music publications (review co-mention engine) + +**Optional (user adds key in Settings):** +Genius (lyrics/annotations), Tumblr (publishing), Mem0 (cross-session memory), AgentMail (email/Slack integration) + +### Slash Commands + +| Command | Function | +|---|---| +| `/show-prep [show] [tracks]` | Full radio show preparation package | +| `/influence [artist]` | Cited influence network mapping | +| `/news [station] [count]` | Music news segment from RSS feeds | +| `/radio [genre or station]` | Stream live radio while researching | +| `/publish` | Publish last response as article | +| `/published` | View published articles | +| `/help` | Persona-adaptive help guide | + +--- + +## 2. Target Market + +### Primary Segments + +**1. Radio Hosts & Music Directors** +- **Size:** ~30,000 terrestrial radio stations in the U.S., ~4,000 internet-only stations +- **Pain point:** Show prep takes 2-3 hours of manual research across fragmented sources +- **Crate value:** `/show-prep` automates research, generates talk breaks, social copy, and interview prep +- **Willingness to pay:** Stations already budget for automation software ($530M radio automation market, growing 7.2% CAGR to $980M by 2033) +- **Entry point:** Radio Milwaukee (WYMS) is the first institutional deployment + +**2. Professional DJs** +- **Size:** ~1.5M active DJs globally (estimated from DJ software license data) +- **Pain point:** Track research happens across 5+ platforms (Discogs, WhoSampled, YouTube, AllMusic, forums). DJ software handles mixing, not discovery. +- **Crate value:** Sample research, genre deep dives, set building with transition notes, Bandcamp discovery +- **Willingness to pay:** DJs spend $200-500/year on software (Rekordbox, Serato, Traktor subscriptions) + +**3. Record Collectors** +- **Size:** ~5M active vinyl buyers in the U.S. (RIAA data: vinyl revenue $1.4B in 2024) +- **Pain point:** Discogs handles buying/cataloging but offers no AI-powered research, collection analysis, or contextual discovery +- **Crate value:** Album deep dives, pressing information, label catalog research, discography analysis +- **Willingness to pay:** Collectors spend significantly on their hobby ($30+ average per vinyl purchase) + +**4. Music Journalists & Researchers** +- **Size:** ~15,000 active music writers in English-language markets (estimated from publication contributor counts) +- **Pain point:** Manual research across databases, press releases, and interviews. No AI research assistant exists for music journalism. +- **Crate value:** Cross-publication source research, artist profiles, influence mapping with citations, one-click publishing +- **Willingness to pay:** Freelance journalists invest in research tools; publications invest in content infrastructure + +**5. Serious Music Enthusiasts ("Crate Diggers")** +- **Size:** Millions. The audience between casual Spotify listeners and professional DJs. +- **Pain point:** Spotify recommends by algorithm. These users want to understand *why* music matters — context, history, connections, and discovery beyond algorithmic bubbles. +- **Crate value:** Deep artist research, genre exploration, playlist creation with contextual reasoning, influence chains +- **Willingness to pay:** Already paying for Spotify Premium, Qobuz, Tidal, and vinyl — looking for depth, not more of the same + +### Total Addressable Market + +| Segment | Size | Annual Value (est.) | +|---|---|---| +| Radio automation/intelligence | 34,000 stations | $530M (current market) | +| DJ software/tools | 1.5M DJs | $450M | +| Music research/reference | 20M+ serious enthusiasts | $2B+ (untapped) | +| Music journalism tools | 15,000 writers | $15M | +| **Combined TAM** | | **$3B+** | + +--- + +## 3. Competitive Landscape + +### Direct Competition: None + +No existing product combines conversational AI, deep music knowledge graph, cross-platform intelligence, and professional workflow integration. The closest competitors each cover one slice: + +| Product | What It Does | What It Lacks | +|---|---|---| +| **WhoSampled** (now Spotify) | Sample/cover tracking, 1.2M songs | No AI interface, no research workflows, now locked in Spotify | +| **AllMusic** | Comprehensive editorial database | Static content, no AI, no interaction, no workflows | +| **Discogs** | Crowdsourced database + marketplace (18M+ releases) | No AI research, no contextual analysis | +| **Cyanite** | AI music tagging and similarity search | B2B only (sync licensing), no consumer product | +| **PulseDJ** | AI track recommendations for DJs | Real-time set selection only, no research depth | + +### Adjacent Competition: Streaming Platforms + +| Platform | Subscribers | Music Intelligence Features | Gap | +|---|---|---|---| +| **Spotify** | 290M paid / 751M MAU | AI DJ (narration), SongDNA (coming), Discover Weekly | Passive discovery. No research tools, no professional workflows | +| **Apple Music** | ~110M | Editorial playlists, Shazam integration | Curated but static. No conversational AI | +| **YouTube Music** | ~92M | Algorithm + UGC catalog | No structured music knowledge | +| **Tidal** | ~700K U.S. | Hi-fi audio, stems licensing | Distressed asset (Block scaling back, 25% layoffs 2024) | +| **Qobuz** | Growing 40% YoY | Editorial content, audiophile focus | Strong editorial but no AI layer | +| **SoundCloud** | 18M premium | Indie discovery | Acquired Musiio (AI), shut down API | +| **Bandcamp** | N/A (marketplace) | Direct artist sales | Changing ownership (Epic → Songtradr), no research | + +### Key Insight + +Streaming platforms are distribution layers. They optimize for passive consumption — playlists, algorithmic feeds, lean-back listening. Crate is a research layer. It optimizes for active engagement — questions, discovery, context, citations. These are complementary, not competing. + +This is why Crate is valuable *to* streaming platforms, not in competition with them. + +--- + +## 4. Why Spotify (or Qobuz) Would Buy Crate + +### The Spotify Case + +**Spotify's strategic direction validates Crate's thesis:** + +1. **WhoSampled acquisition (November 2025)** — Spotify paid an undisclosed amount for the largest sample/cover database. This confirms Spotify values music intelligence data and is willing to acquire to get it. + +2. **SongDNA feature (in beta, launching 2026)** — Powered by WhoSampled data, SongDNA shows the people and connections behind songs. Beta users are calling it "the best Spotify feature to date" (TechRadar). This is Spotify acknowledging that metadata and context increase engagement. + +3. **The Echo Nest acquisition ($50-100M, 2014)** — The foundational acquisition for Spotify's entire recommendation engine. Spotify has a 12-year track record of acquiring music intelligence companies. + +4. **AI DJ feature** — Spotify's AI DJ narrates transitions between songs. It's a step toward conversational music interaction but stops at surface-level commentary. + +**What Crate gives Spotify that they don't have:** +- Conversational research interface (ask questions, get cited answers) +- Multi-source intelligence beyond their own catalog (26 publication co-mentions, Discogs credits, MusicBrainz metadata) +- Professional workflow tools (show prep, influence mapping, publishing) +- Academic-grade influence methodology (network-based recommendation, not collaborative filtering) +- Ready-made product for a "Spotify Pro" tier + +**Acquisition precedent pricing:** + +| Acquisition | Year | Price | Category | +|---|---|---|---| +| The Echo Nest | 2014 | ~$66M | Music intelligence/recommendations | +| Niland | 2017 | Undisclosed | AI personalization | +| WhoSampled | 2025 | Undisclosed | Music intelligence/discovery | +| Shazam (Apple) | 2018 | $400M | Music recognition | + +Based on precedents, a music intelligence acquisition in this category ranges from **$50M-$400M** depending on user base, data assets, and strategic value. + +### The Qobuz Case + +**Why Qobuz is a natural fit:** +- Growing 40% YoY, actively expanding with Qobuz Connect (100+ hardware partners) +- Positioned as the "ethical streaming" alternative — pays artists 3-4x more than Spotify per stream +- Strong editorial identity but no AI capabilities +- Serves the exact same audience Crate does: serious music enthusiasts who care about depth, context, and quality +- Crate as a feature would differentiate Qobuz from every other streaming service + +**What Crate gives Qobuz:** +- Instant AI capability without building from scratch +- A reason for music enthusiasts to choose Qobuz over Spotify (research + hi-fi audio = unbeatable value prop) +- Professional user segment (DJs, radio hosts) that Qobuz hasn't captured yet + +### Other Potential Acquirers + +| Company | Rationale | +|---|---| +| **Apple Music** | Invested in editorial curation. Crate's AI research could power enhanced artist pages, editorial tools, and a "pro" tier. Apple acquired Shazam for $400M. | +| **Amazon Music** | Looking for differentiation beyond Alexa integration. Music intelligence gives Echo/Alexa smarter music conversations. | +| **SoundCloud** | Already acquired Musiio (AI music intelligence) but shut it down. May need a replacement strategy. | +| **Pioneer DJ / AlphaTheta** | Owns Rekordbox, the dominant DJ software. Adding research intelligence to DJ prep would be a category-defining move. | +| **LiveOne / iHeart** | Radio conglomerates that could deploy Crate across hundreds of stations for show prep automation | + +--- + +## 5. Revenue Strategy + +### Tier 1: Freemium SaaS (Individual Users) + +| Plan | Price | Includes | +|---|---|---| +| **Free** | $0/mo | 10 research queries/day, basic sources, no publishing | +| **Pro** | $15/mo | Unlimited queries, all sources, influence mapping, publishing, saved sessions, cross-session memory | +| **Pro Annual** | $120/yr ($10/mo) | Same as Pro, 33% discount | + +**Revenue model:** Users bring their own AI key (Anthropic or OpenRouter), so Crate's infrastructure cost per user is low (Convex, Vercel, embedded API keys for data sources). The subscription covers platform access, not AI compute. + +**Projected conversion:** 3-5% free-to-paid (industry standard for prosumer tools), with higher conversion for professional segments (DJs, radio hosts) due to workflow-specific features. + +### Tier 2: Team Plans (Radio Stations, Newsrooms, Labels) + +| Plan | Price | Includes | +|---|---|---| +| **Team** | $49/mo (up to 10 seats) | Shared API keys, admin dashboard, team onboarding, priority support | +| **Station** | $199/mo (unlimited seats) | Custom domain onboarding, station-specific commands, dedicated Slack/email support, custom data source integrations | + +**Revenue model:** Team/Station plans include embedded AI keys managed by the admin — team members don't need to bring their own. Crate absorbs the AI cost and bundles it into the subscription. + +**First customer:** Radio Milwaukee (WYMS) — already deployed with team-specific onboarding flow, domain-based persona defaults, and pre-configured shared keys. + +### Tier 3: Enterprise API / White-Label + +| Offering | Price | Target | +|---|---|---| +| **API Access** | $0.05-0.15/query | Streaming platforms, DJ software, music apps that want to embed research | +| **White-Label** | Custom pricing | Radio groups (iHeart, Audacy) that want Crate's intelligence across 100+ stations | + +**Revenue model:** Per-query pricing for API, annual license for white-label. This is the scale play — Crate's multi-source research engine embedded in products that already have millions of users. + +### Tier 4: Data Partnerships & Licensing + +| Offering | Description | Potential Partners | +|---|---|---| +| **Influence graph data** | Licensed access to Crate's growing influence network database (artist connections, co-mention scores, citation graphs) | Spotify, Apple Music, academic researchers, music journalists | +| **Show prep content** | Syndicated show prep packages (talk breaks, news segments, artist research) for radio stations | iHeart, Audacy, NPR member stations | +| **Review aggregation** | Structured sentiment and mention data from 26 music publications | Labels, PR firms, artist managers | + +### Tier 5: Sponsorship & Partnership Revenue + +| Model | How It Works | Potential Partners | +|---|---|---| +| **Sponsored research** | "This influence map is brought to you by [Label/Brand]" — non-intrusive sponsor placement on published research articles | Record labels (Blue Note, XL, Warp, Brainfeeder), audio brands (Sonos, Audio-Technica, Technics) | +| **Label partnerships** | Labels pay for enhanced catalog presence — priority in research results, rich metadata, exclusive content (liner notes, producer interviews) | Major labels (UMG, Sony, Warner), key independents (Stones Throw, Warp, Ninja Tune, Hyperdub) | +| **Festival/event integration** | Pre-event research packages for festival attendees — "Research the lineup before you go" | Pitchfork Music Festival, SXSW, Primavera Sound, Sonar | +| **Hardware partnerships** | Bundled with DJ hardware purchases (Pioneer, Denon) or audiophile equipment (Qobuz Connect partners) | AlphaTheta (Pioneer DJ), Denon DJ, Sonos, KEF | +| **Affiliate revenue** | Links to buy vinyl on Discogs, digital on Bandcamp, stream on Spotify/Qobuz with affiliate tracking | Discogs marketplace, Bandcamp, Amazon | + +### Revenue Projections (Conservative) + +| Year | Users (free) | Paid Subs | Team/Station | API/Enterprise | Revenue | +|---|---|---|---|---|---| +| Year 1 | 10,000 | 400 | 5 | 0 | $85K | +| Year 2 | 50,000 | 3,000 | 25 | 2 | $680K | +| Year 3 | 200,000 | 15,000 | 100 | 10 | $3.5M | + +--- + +## 6. Competitive Moats + +### 1. Multi-Source Intelligence Engine +Crate aggregates 20+ music databases through a unified agentic loop. Replicating this requires licensing or building integrations with Discogs, MusicBrainz, Last.fm, Genius, Spotify, YouTube, Bandcamp, Pitchfork, and a dozen more — plus the orchestration logic that knows when and how to query each one. This took 6+ months to build across CLI and web. + +### 2. Influence Graph (Growing Data Asset) +Every influence query builds Crate's network graph, cached in Convex. Over time, this becomes a proprietary music knowledge graph — artist connections, co-mention scores, citation evidence — that gets richer with every user session. Grounded in peer-reviewed methodology (network-based music recommendation, Harvard Data Science Review 2025). + +### 3. Professional Workflow Integration +Show prep, influence mapping, and publishing are workflow-specific features that general-purpose AI tools can't replicate without deep domain knowledge. A radio host needs talk breaks, social copy, and interview prep in a specific format. A DJ needs sample lineage, BPM context, and harmonic compatibility. These workflows are encoded in Crate's system prompt and tool configuration. + +### 4. Source Citation Culture +Every result is cited. Users trust Crate because they can verify claims. This is table stakes for professional music users (journalists, researchers) and increasingly important for all AI products. Building citation into the core architecture — not bolting it on — creates a quality floor competitors would need to match. + +### 5. Community Network Effects +As more users research the same artists, Crate's influence graph gets richer and more accurate. A radio host in Milwaukee researching Ethiopian jazz and a DJ in Berlin researching Ethio-jazz both contribute to the same knowledge network. This is a classic data network effect. + +--- + +## 7. Acquisition Valuation Framework + +### Revenue Multiple (SaaS Standard) +- Early-stage SaaS: 10-15x ARR +- At Year 3 projected $3.5M ARR: **$35M-$53M valuation** + +### Strategic Premium (Music Intelligence Acquisitions) +- The Echo Nest sold for ~$66M with minimal revenue — value was the technology and data asset +- WhoSampled sold to Spotify with 1.2M tracked songs — value was the knowledge graph +- Shazam sold to Apple for $400M with strong consumer brand + technology + +Crate's strategic value exceeds its revenue value because: +1. It has a working, deployed multi-source research engine +2. It has a growing influence graph (proprietary data) +3. It serves professional segments that streaming platforms can't reach +4. It validates the "music intelligence" category that Spotify is investing billions in + +**Realistic acquisition range: $50M-$150M** depending on user traction, data asset size, and buyer's strategic urgency. + +--- + +## 8. Traction & Proof Points + +| Metric | Current Status | +|---|---| +| **First institutional customer** | Radio Milwaukee (WYMS) — team deployment with custom onboarding | +| **Data sources integrated** | 20+ (more than any competing product) | +| **Influence methodology** | Grounded in Badillo-Goicoechea 2025 (Harvard Data Science Review) | +| **Platform** | Live on Vercel, Convex backend, Clerk auth | +| **Agent capability** | 19 MCP server tools, multi-model support (Claude, GPT-4o, Gemini, Llama, DeepSeek, Mistral) | +| **CLI companion** | Published npm package (`crate-cli`) — dual-surface product | +| **Publishing** | One-command article publishing (Telegraph + Tumblr) | +| **User personas** | 6 validated persona workflows (new user, radio host, DJ, collector, music lover, journalist) | + +--- + +## 9. Honest Assessment: Strengths, Gaps, and What's Real + +**The problem is real.** Music professionals actually research across 5-10 fragmented platforms. That's not a hypothetical pain point — it comes from years of living it as a radio broadcaster. The best products come from builders who have the problem themselves. + +**The architecture is legitimately hard to replicate.** 20+ data sources orchestrated through an agentic loop with source citations on every claim — that's not a weekend wrapper around ChatGPT. Someone at Spotify would need months to rebuild what Crate already does, and they still wouldn't have the domain intuition baked into the system prompt, the show prep workflows, or the influence mapping methodology. + +**The timing is perfect.** Spotify buying WhoSampled, building SongDNA, launching AI DJ — they're validating the category in real time. Crate is ahead of where they're going. + +**The weak spots are solvable, not structural.** Crate doesn't have millions of users yet. The revenue model isn't live. The influence graph is young. But those are traction problems, not product problems. The product itself works — Radio Milwaukee uses it, the agent delivers, the results are cited and interactive. + +**The hard question: BYOK friction.** The bring-your-own-key model is a friction barrier for casual users. The free tier in the revenue plan needs to absorb AI cost or the top of the funnel leaks. This is a solvable problem (embedded keys on free tier, user keys on pro) but it needs to be solved before a public launch. + +**The positioning tension.** "Spotify on steroids" is a strong pitch for investors and press, but it might intimidate the music lover segment. They need to feel like Crate is for them — not just for professionals. The persona-adaptive help guide and onboarding already address this, but the marketing language needs two registers: one for pitch decks, one for the landing page. + +**The bottom line:** Crate is solving a real problem that nobody else is solving, with technology that's genuinely differentiated. That's rare. The gaps are about scale, not substance. + +### Validation Gaps (from YC Office Hours diagnostic, March 2026) + +A structured product diagnostic exposed four gaps that this analysis originally glossed over: + +**1. Zero demand evidence.** Radio Milwaukee uses Crate, but it's unclear whether they depend on it or it exists because the builder is present. No one has independently adopted Crate. No one has been asked to pay. "The product analysis assumes demand from market sizing — but interest is not demand. Behavior is demand. Money is demand. Panic when it breaks is demand." The test: if Crate went offline for 48 hours, would anyone besides the builder notice? + +**2. No observation data.** The builder has demoed Crate to people (driving the wheel) and knows Radio Milwaukee uses it (but hasn't watched them). Zero unguided observation sessions have been conducted. No data exists on what real users try first, where they get confused, what makes them lean forward, or what makes them leave. Without this, every product decision — which features to build, which segment to target, what to charge — is based on the builder's intuition, not evidence. + +**3. Unknown willingness to pay.** The subscription system is built ($15/mo Pro, $25/mo Team) but has zero subscribers. No one has been asked "would you pay for this specific thing?" The narrowest wedge — the single smallest version of Crate someone would pay real money for this week — hasn't been identified. + +**4. Engineering-first, demand-second.** The builder spent months on subscription billing, custom skills with self-improving memory, Perplexity integration, CodeRabbit review cycles, and Convex schema design — all before a single paying customer. This is the most common failure mode for technical founders. The engineering muscle is exceptional. The target needs to move from "build more" to "validate what's built." + +### The StoryGraph Positioning + +A reframe emerged from the diagnostic: **Crate is StoryGraph for music.** + +StoryGraph (thestorygraph.com) grew to 5M+ users by being the anti-algorithmic alternative to Goodreads. It doesn't just track books — it understands *why* you liked them (mood, pacing, themes). Solo founder, bootstrapped, no VC, word-of-mouth growth. + +The parallel is exact: +- StoryGraph is for readers tired of Goodreads' broken recommendations → Crate is for music lovers tired of Spotify's algorithmic bubble +- StoryGraph explains *why* you like books → Crate explains *why* music matters (influence chains, sample lineage, production stories) +- StoryGraph launched without community, added it later → Crate should do the same +- StoryGraph's moat is structured metadata on every interaction → Crate's moat is influence graphs, source citations, and custom skills with memory + +Two audiences, one product: +- **Consumer depth:** Anyone who wants to understand music, not just consume it +- **Professional power tools:** Radio show prep, DJ research, journalist citations + +The pitch line: **"If Claude, Spotify, and Pitchfork had a baby."** + +### Validation Action Plan + +**Week 1-2: Observation Sprint (no code)** +- Find 5-10 music professionals (radio hosts, DJs, journalists, collectors) who are NOT friends +- Give them Crate access with a platform key (no BYOK friction) +- Watch them use it unguided for 30 minutes. Don't help. Don't explain. +- Track: what they type first, where they get stuck, the "lean forward" moment, their words for what Crate is + +**Week 1-4: Positioning (parallel)** +- LinkedIn presence: post about building Crate with Claude Code — the non-coder founder story +- Direct outreach to Spotify (SongDNA team), Qobuz, Apple Music with demo link +- Soft Product Hunt launch for social proof + +**Week 4: Decision point** +- If 3+ people say "I'd pay for this" → identify the wedge, price it, sell to 3 stations +- If Spotify/Qobuz respond → pursue acquisition or hire conversation +- If neither → Crate is the most impressive music-tech portfolio piece on the planet, and the builder knows exactly how to pitch it + +--- + +## 10. What's Next + +### Near-Term (Q2 2026) +- Launch public beta with freemium model +- Onboard 3-5 additional radio stations (NPR member stations, college radio) +- Implement usage metering and subscription billing (Stripe) +- Add Spotify playback integration (user OAuth) + +### Mid-Term (Q3-Q4 2026) +- Launch Team plan with shared key management +- API access for third-party integrations +- Mobile-responsive PWA +- Playlist export to Spotify/Apple Music/Qobuz +- Expanded influence graph with user contributions + +### Long-Term (2027) +- Enterprise white-label for radio groups +- Data licensing partnerships +- Hardware integration partnerships +- International expansion (non-English publication sources) + +--- + +## 11. Press-Ready Positioning + +### One-Liner +Crate is an AI music research platform that searches 20+ databases and gives professionals cited, interactive answers — like having a record store clerk, music librarian, and research assistant in one tool. + +### Elevator Pitch (30 seconds) +Spotify tells you what to listen to. Crate tells you why it matters. It's an AI research agent that queries Discogs, MusicBrainz, Pitchfork, Genius, and 16 more databases at once, returning cited results as interactive cards. Radio hosts use it to prep shows in 90 seconds instead of 3 hours. DJs use it to trace sample lineage and discover deep cuts. Spotify just bought WhoSampled because they know music intelligence is the future. Crate is building the full-stack version of that future. + +### Media Angles + +**For music press (Pitchfork, Resident Advisor, DJ Mag):** +"The app that researches music the way DJs and radio hosts actually think — by influence, context, and connection, not by algorithm." + +**For tech press (TechCrunch, The Verge):** +"An AI agent that orchestrates 20+ music APIs simultaneously, with source citations on every claim. Built by a radio broadcaster who got tired of Spotify's recommendation bubble." + +**For business press (Billboard, Music Business Worldwide):** +"Spotify paid $66M for The Echo Nest and just acquired WhoSampled. The music intelligence category is heating up — and Crate is building the research layer that streaming platforms don't have." + +**For AI/developer press (Hacker News, dev.to):** +"A real-world agentic application using 19 MCP servers, OpenUI for dynamic component rendering, and Convex for real-time persistence. Built with Claude Code and shipped to production." + +--- + +## 12. The Builder Path: Acqui-Hire & Talent Acquisition Precedents + +### The OpenClaw Playbook + +In February 2026, OpenAI CEO Sam Altman [announced](https://x.com/sama/status/2023150230905159801) that Peter Steinberger — creator of OpenClaw, an open-source AI personal assistant — was joining OpenAI to "drive the next generation of personal agents." Altman called him "a genius." + +Steinberger's path: +1. **Built credibility first.** Spent 13 years building PSPDFKit, a PDF toolkit used by Apple, Dropbox, and SAP. Bootstrapped it, then sold his shares when Insight Partners put in $116M in 2021. +2. **Took a break.** Five years away from building. Skipped the Copilot era entirely. +3. **Built something undeniable.** Started OpenClaw as a weekend project in November 2025. It went viral — "hockey stick" adoption among developers and vibe coders. The product demonstrated what AI agents could actually do on a desktop. +4. **The company came to him.** OpenAI didn't acquire OpenClaw (the project stays open source in a foundation). They hired Steinberger because he proved he understood the agent problem better than anyone on their team. +5. **He didn't need the money.** He was self-funding OpenClaw's infrastructure at ~$12,000/month. He joined OpenAI for impact, not compensation. + +The key insight: **Steinberger didn't pitch himself. He built something that made the pitch for him.** + +### The Acqui-Hire Landscape (2024-2026) + +Big Tech spent over **$40 billion** on acqui-hire deals in 2024-2025 alone — more than all prior acqui-hire activity combined. The pattern: license the technology, hire the founder and core team. + +| Deal | Year | Value | What Happened | +|---|---|---|---| +| Microsoft → Inflection AI | 2024 | ~$650M | Hired Mustafa Suleyman as CEO of Microsoft AI | +| Amazon → Adept AI | 2024 | Undisclosed | Hired ~80% of technical team including CEO | +| Google → Character.AI | 2024 | ~$2.7B | Licensed tech, hired founders (transformer architecture creators) | +| Google → Windsurf | 2025 | ~$2.4B | Licensed tech, hired CEO + top engineers | +| OpenAI → OpenClaw | 2026 | Talent hire | Hired founder, project stays open source | + +### Music-Specific Acqui-Hires and Acquisitions + +The music intelligence space has its own version of this pattern: + +| Deal | Year | Value | What Happened | +|---|---|---|---| +| Spotify → The Echo Nest | 2014 | ~$66M | Team joined Spotify, built entire recommendation engine | +| Spotify → Niland | 2017 | Undisclosed | French AI music recommendation startup — team absorbed | +| Spotify → Sonalytic | 2017 | Undisclosed | Audio detection startup — team joined to improve music ecosystem | +| Apple → Asaii | 2018 | Undisclosed | Music analytics startup — CEO joined Apple Music directly | +| Apple → Shazam | 2018 | $400M | Music recognition — entire team absorbed | +| Apple → Q.ai | 2025 | ~$2B | AI audio startup — Apple's second-largest acquisition ever | +| Spotify → WhoSampled | 2025 | Undisclosed | Music intelligence database — powering SongDNA feature | + +### How Crate Maps to This Pattern + +**The Steinberger parallel:** + +| Steinberger (OpenClaw → OpenAI) | Moody (Crate → Spotify/Apple/Qobuz) | +|---|---| +| 13 years building PSPDFKit (B2B credibility) | 15+ years in public radio (domain credibility) | +| Built OpenClaw as a side project | Built Crate CLI, then Crate Web | +| Demonstrated understanding of AI agents on desktop | Demonstrates understanding of AI agents for music research | +| OpenAI needed agent expertise | Spotify/Apple need music intelligence expertise | +| Product went viral with developers | Product deployed at Radio Milwaukee, targeting music professionals | +| Didn't need the money (prior exit) | Motivated by impact (radio + music industry transformation) | + +**What makes this path viable for Crate:** + +1. **Domain expertise that can't be hired.** Understanding how a radio host preps for a show, how a DJ researches tracks, how a music journalist traces influence — this comes from years inside the industry, not from a product spec. Spotify's engineers can build recommendation algorithms. They can't build workflow tools for radio hosts because they've never been radio hosts. + +2. **Working product, not a pitch deck.** Crate is deployed. Radio Milwaukee uses it. The agent works. The influence mapping works. The publishing pipeline works. Like Steinberger, the product makes the pitch. + +3. **The category is validated.** Spotify buying WhoSampled and building SongDNA proves the music intelligence category is real and worth investing in. Crate goes deeper than WhoSampled (20+ sources vs. 1, conversational AI vs. static database, professional workflows vs. consumer browse). + +4. **The timing is right.** Every major streaming platform is looking for AI differentiation. Spotify has AI DJ. Apple has Shazam. Amazon has Alexa. None of them have a music research agent. The builder who demonstrates that capability — with a working product and real users — is the one who gets the call. + +### Three Possible Outcomes + +**Outcome 1: Talent Hire (OpenClaw model)** +Spotify or Apple hires Tarik to lead music intelligence product development. Crate stays open source or becomes a foundation project. Compensation: senior PM/product lead salary + equity + signing bonus. This is the most likely path if the product demonstrates capability but hasn't scaled to millions of users. + +**Outcome 2: Acqui-Hire (Echo Nest model)** +Company acquires Crate (the product, the data, the team) and integrates it into their platform. Crate becomes "Spotify Research" or "Apple Music Intelligence." Valuation: $5M-$50M depending on traction. This is the path if Crate has paying customers and a growing user base. + +**Outcome 3: Full Acquisition (Shazam model)** +Company acquires Crate at a premium because the influence graph data and multi-source engine are strategically irreplaceable. Valuation: $50M-$150M+. This requires significant user traction, a defensible data asset, and competitive pressure (multiple acquirers bidding). + +### What Needs to Happen Next + +To make any of these outcomes real: + +1. **Build in public.** The crate-article.md is a start. Keep shipping and writing about it. Steinberger's blog posts and GitHub activity created the narrative before OpenAI called. + +2. **Get the product in front of the right people.** Radio Milwaukee is proof of concept. Five more stations makes it a trend. A Hacker News front page post about the agentic architecture gets developer attention. A DJ Mag or Resident Advisor feature gets industry attention. + +3. **Grow the influence graph.** Every query builds the data asset. More users = richer graph = more defensible moat. This is the asset an acquirer would pay for. + +4. **Ship the demo videos.** The workflow demo (see [DEMO_SCRIPT.md](./DEMO_SCRIPT.md)) is the artifact that travels. When it gets forwarded to the head of product at Spotify, it needs to be undeniable. + +5. **Don't optimize for acquisition.** Steinberger didn't build OpenClaw to get hired by OpenAI. He built it because he believed in it. The hire happened because the product was real. Same principle applies: build Crate because it solves a real problem for real music professionals. The rest follows. + +--- + +## Sources + +- [OpenClaw creator Peter Steinberger joins OpenAI — TechCrunch](https://techcrunch.com/2026/02/15/openclaw-creator-peter-steinberger-joins-openai/) +- [Sam Altman announces Steinberger hire — X](https://x.com/sama/status/2023150230905159801) +- [OpenClaw, OpenAI and the future — Peter Steinberger's blog](https://steipete.me/posts/2026/openclaw) +- [OpenClaw & The Acqui-Hire That Explains Where AI Is Going — Monday Morning](https://mondaymorning.substack.com/p/openclaw-and-the-acqui-hire-that) +- [OpenAI's acquisition of OpenClaw signals the end of the ChatGPT era — VentureBeat](https://venturebeat.com/technology/openais-acquisition-of-openclaw-signals-the-beginning-of-the-end-of-the) +- [How Big Tech Is Rewriting M&A: The License and Acqui-hire Era — Stepmark Partners](https://stepmark.ai/2025/11/03/how-big-tech-is-rewriting-ma-the-license-and-acqui-hire-era/) +- [Acqui-Hires Explained: Big Tech's $40 Billion Talent Grab — Clera Insights](https://www.getclera.com/blog/acqui-hires-big-tech-talent-acquisition) +- [Big tech's pricey AI acqui-hires — PitchBook](https://pitchbook.com/news/articles/big-techs-pricey-ai-acqui-hires) +- [Spotify acquires WhoSampled — TechCrunch](https://techcrunch.com/2025/11/19/spotify-acquires-music-database-whosampled/) +- [Apple acquires AI audio startup Q.ai — Music Business Worldwide](https://www.musicbusinessworldwide.com/apple-acquires-ai-audio-startup-q-ai-said-to-be-worth-nearly-2bn/) +- [Apple acquires Asaii — Musically](https://musically.com/2018/10/15/confirmed-apple-has-bought-music-analytics-startup-asaii/) +- [Spotify acquires Niland — CNBC](https://www.cnbc.com/2017/05/18/spotify-buys-niland-french-ai-music-startup.html) +- [AI Acqui-Hires: Microsoft, Google & Meta — Founders Forum](https://ff.co/ai-acquihires/) + +--- + +*This document is a living analysis. Update with traction metrics, user testimonials, and market developments as they occur.* diff --git a/docs/UI_EXPERIENCE_ANALYSIS.md b/docs/UI_EXPERIENCE_ANALYSIS.md new file mode 100644 index 0000000..aa45c67 --- /dev/null +++ b/docs/UI_EXPERIENCE_ANALYSIS.md @@ -0,0 +1,78 @@ +# Crate UI & Experience Analysis — As a Unique Music Product + +**Date:** March 2026 +**Context:** Assessment of Crate's interface and user experience based on product screenshots, comparing against music industry standards and competitors. + +--- + +## What's Working + +### The Three-Panel Layout Is the Right Architecture +Sidebar (navigation/history) + Chat (conversational input) + Artifact pane (interactive output) mirrors how research actually works — you ask, you get results, you explore the results while the conversation continues. No other music product uses this pattern. Spotify is single-panel. Perplexity is chat-only. Crate gives you both. + +### The Artifact Cards Are Genuinely Unique +The J Dilla card with Influenced By / Influenced chips, the "On-Air Talking Point," the "Known For" summary — that's not something any AI chatbot produces. It's structured intelligence rendered as an interactive component. The HYFIN show prep with Track Context cards, pronunciation guides, station-specific editorial voice — no product anywhere does this. + +### The Influence Chain Playlist Is the Signature Moment +Ezra Collective's playlist organized by Afrobeat Roots / Jazz Foundations / Hip-Hop Swing / Cosmic & Dub with "Why They're Here" explanations — that's the feature that makes someone say "whoa." It's not a playlist. It's a *thesis about musical lineage* you can listen to. + +### The Dark Theme Is Right for the Audience +Crate diggers, DJs, radio hosts — this is a nighttime product for people in studios, record stores, and broadcast booths. The dark zinc palette with cyan accents feels like a pro audio tool, not a consumer app. + +--- + +## What's Holding It Back + +### It Looks Like a Developer Tool, Not a Music Product +The UI is clean and functional, but it doesn't have *vibe*. Compare the emotional warmth of a record store, a vinyl sleeve, or even Bandcamp's artist pages to Crate's interface. Crate feels like a productivity app that happens to be about music. The content is warm (the writing, the talking points, the context). The container is cold. + +### The Chat Input Is Undersold +"Ask about any artist, track, or genre... (/ for commands)" is generic. The placeholder should make you want to type something. Something like "Who influenced Flying Lotus?" or "What's the story behind Donuts?" — show the possibilities, not the mechanics. + +### No Visual Identity Beyond the Layout +There's no logo moment, no illustration style, no distinctive typography. The cyan accent color is functional but forgettable. StoryGraph has its warm purple. Spotify has its green. Bandcamp has its teal. Crate doesn't have a color or visual language that you'd recognize in a screenshot without the logo. + +### The Sidebar Categories Are Confusing +Crates, Playlists, Collection, Published, Recents, Artifacts — that's 6 navigation groups. A new user doesn't know the difference between a Crate, a Collection, and a Playlist. StoryGraph has: Currently Reading, Want to Read, Read. Three states. Clear immediately. + +--- + +## What Makes It Unique as a Music Product + +### The Only Product That Answers "Why" About Music +Spotify tells you what to listen to. Discogs tells you what exists. Genius tells you what the lyrics mean. Crate tells you *why it matters* — the influence chain, the production story, the cultural context, the reason this track exists in the lineage of music. No other product does this. + +### Research and Listening in One Workspace +You ask about the LA beat scene, get a context-rich playlist, and you're listening to Samiyam while reading about why Knxwledge matters — all without leaving the app. That's not Spotify (no research). That's not Perplexity (no playback). That's Crate. + +### Music Knowledge as Interactive, Not Static +Every result is clickable, playable, explorable. Influence chips link to more research. Playlists have "Why They're Here" context. Artist cards have on-air talking points. The output isn't text — it's an interactive artifact. + +--- + +## The One Thing to Change Before the Observation Sprint + +**Add warmth.** One album artwork background blur behind the artist card. One serif font for headings. One subtle texture. Something that says "this is about music" before you read a single word. Right now the UI says "this is a developer's side project" — and it's not. It's a music product built by a broadcaster. The interface should feel like that. + +--- + +## Competitive UI Comparison + +| Product | Visual Identity | Emotional Tone | Research Depth | +|---|---|---|---| +| **Spotify** | Green + black, album art-driven, clean and familiar | Lean-back, passive, algorithmic | Shallow (SongDNA adds credits, that's it) | +| **Bandcamp** | Teal + white, artist-first, editorial warmth | Independent, authentic, artist-supporting | Album descriptions only, no AI | +| **Discogs** | Gray + orange, data-dense, utilitarian | Collector's tool, catalog-focused | Deep metadata, no synthesis | +| **StoryGraph** | Warm purple, clean, stats-forward | Thoughtful, anti-algorithmic, community | Deep for books (mood, pacing, themes) | +| **Perplexity** | White/dark, minimal, answer-focused | Research tool, fast, citation-forward | Broad but shallow on any domain | +| **Crate** | Dark zinc + cyan, three-panel, interactive cards | Pro tool feel, content is warm but container is cold | Deepest music research of any product — 20+ sources, cited, interactive | + +### Crate's Unique Position +Crate is the only product in the music space that combines: +1. Conversational AI interface (ask anything) +2. Multi-source research (20+ databases) +3. Interactive output (not just text — cards, playlists, influence maps) +4. Built-in playback (YouTube + radio) +5. Professional workflows (show prep, publishing, custom skills) + +No competitor has more than 2 of these 5. Spotify has playback. Perplexity has conversational AI. Discogs has multi-source data. None combine all five. diff --git a/docs/articles/2026-03-23-auth0-hackathon-linkedin.md b/docs/articles/2026-03-23-auth0-hackathon-linkedin.md new file mode 100644 index 0000000..842a452 --- /dev/null +++ b/docs/articles/2026-03-23-auth0-hackathon-linkedin.md @@ -0,0 +1,55 @@ +# I built an AI music agent that connects to your Spotify, Slack, and Google Docs. Here's why that matters. + +I've been building Crate for the past two weeks. It's an AI research assistant for people who work in music — DJs, radio producers, playlist curators, anyone who needs to dig deep into artist histories, influence chains, and sample lineages without drowning in browser tabs. + +Crate pulls from 20+ live data sources (Discogs, Genius, MusicBrainz, Last.fm, Bandcamp, Spotify, and more) and synthesizes everything into one conversation. Ask it to map the influence chain from Fela Kuti to Beyonce, and it actually traces the path — through Afrobeat, through Wizkid, through Burna Boy's grandfather who managed Fela — with real citations from real sources. + +But until last week, there was a gap. Crate could research anything, but it couldn't *do* anything with what it found outside of the app. You'd get a beautiful influence map, then have to screenshot it and paste it into Slack yourself. You'd discover the perfect playlist, but there was no way to push it to Spotify. + +## What changed + +I entered the Auth0 "Authorized to Act" hackathon, and the timing was perfect. Auth0 recently launched something called Token Vault — a way for AI agents to securely act on behalf of users across third-party services. Instead of asking users to paste API keys or manage OAuth tokens themselves, the agent handles it through Auth0's infrastructure. + +So I wired it up. Here's what Crate can do now: + +**Spotify.** Connect your account, and Crate pulls in your full library — saved tracks, top artists, every playlist you've built. It uses what you actually listen to as context for research. Ask "who are my top artists?" and it knows. Say "deep dive into the artists on my HYFIN playlist" and it pulls the tracks, researches every artist, maps their connections, and finds samples you didn't know existed. Found an influence chain you love? One click and Crate creates a new playlist in your Spotify account, searches for each track, and adds them. The playlist shows up in your Spotify app immediately. + +**Slack.** After running a show prep research session (Crate was originally built for Radio Milwaukee), you can send the results directly to a Slack channel. The message arrives formatted with Block Kit — proper headers, bullet lists, dividers. No more copy-pasting research into chat. + +**Google Docs.** Save any research output as a Google Doc with a shareable link. Useful for archiving deep dives or sharing with collaborators who aren't in Crate. + +## Why this matters beyond my app + +Here's the thing I keep coming back to: the "connect your account" pattern is going to be everywhere in the next year. + +Right now, most AI tools are isolated. They can generate text, analyze data, search the web. But they can't touch your actual stuff. Your playlists, your team's Slack channels, your Google Drive. The moment an AI agent can act on your behalf across services you already use, the whole dynamic shifts from "AI as a search engine" to "AI as a collaborator." + +Auth0's Token Vault makes this possible without each developer having to build their own OAuth infrastructure from scratch. I didn't have to store any Spotify tokens in my database. I didn't have to handle token refresh logic. Auth0 manages the token lifecycle, and my agent just asks for access when it needs it. + +For non-developers reading this: imagine your AI tools could actually push a button for you, not just tell you which button to push. That's what this enables. + +## What Crate actually is + +If you haven't seen it before: Crate is an AI workspace for music research. You type questions in natural language and an agent goes out and queries real databases in real time. + +Some things people use it for: + +- Show prep for radio DJs (origin stories, talk break suggestions, social media copy for each track in a set) +- Influence mapping — tracing how artists connect through samples, collaborations, and stylistic lineage +- Vinyl discovery — searching Discogs and Bandcamp for pressings and releases +- Playlist creation from a theme or vibe, backed by actual research +- Custom skills — you can teach Crate to scrape any website and turn it into a reusable command + +The new connected services (Spotify, Slack, Google Docs) mean the research doesn't stay trapped in the app. It goes where you need it. + +## The hackathon + +I built this for the Auth0 "Authorized to Act" hackathon (deadline April 6, 2026). The whole integration — OAuth flows, token exchange, three sets of API tools, the settings UI, the Block Kit Slack formatting — took about a week of focused work. + +If you're building AI agents and want your users to connect their own accounts without you managing credentials, Auth0 Token Vault is worth looking at. The setup was straightforward once I understood the OAuth flow, and the security model is right: my app never sees or stores the raw Spotify/Slack/Google tokens. + +You can try Crate at [digcrate.app](https://digcrate.app). The connected services are live now. + +--- + +*Tarik Moody is a developer, DJ, and the creator of Crate. He builds tools for people who take music seriously.* diff --git a/docs/articles/2026-03-24-songdna-vs-crate.md b/docs/articles/2026-03-24-songdna-vs-crate.md new file mode 100644 index 0000000..1437fc7 --- /dev/null +++ b/docs/articles/2026-03-24-songdna-vs-crate.md @@ -0,0 +1,63 @@ +# Spotify just launched what I've been building for two weeks + +Spotify announced SongDNA today. It shows you the writers, producers, and samples behind a song. Tap a creator, see what else they've worked on. It's a nice feature. Premium-only, mobile-only, rolling out through April. + +I've been building something in the same space. It goes further. Here's why that matters and what I learned. + +## What SongDNA does + +SongDNA adds an interactive card to Spotify's Now Playing view. You're listening to a track, you tap, and you see who wrote it, who produced it, what it sampled, and what covers exist. You can follow those connections to discover other songs. Spotify describes it as "an interactive way to follow the connections between tracks and see how artists, eras, and genres intersect." + +It's useful. It's also limited to individual songs, locked inside the Spotify app, and doesn't tell you *why* those connections exist. + +## What I built + +Crate is an AI music research workspace I've been building for the past two weeks. It uses an agent that searches 20+ live databases (Discogs, MusicBrainz, Last.fm, Genius, Bandcamp, Spotify's own API, and more) and returns interactive components, not text walls. + +Two features overlap directly with SongDNA: + +**Influence chains.** Type `/influence Adrian Younge` and Crate maps the full influence network: who shaped their sound, who they shaped, weighted by evidence. Each connection comes with pull quotes from actual reviews, sonic elements that transferred, and key works. The methodology is based on Badillo-Goicoechea's 2025 paper in the Harvard Data Science Review on network-based music recommendation using review co-mentions. Every connection is cited. You can verify it. + +*[Screenshot: Adrian Younge & Ali Shaheed Muhammad influence chain showing Marvin Gaye connection with 0.92 weight, Pitchfork review quote, and sonic DNA]* + +**Story cards.** Type `/story Detroit Techno` and Crate researches the full narrative, then renders it as an interactive magazine-style article with chapters you can click through, a YouTube documentary you can watch inline, key people with photos you can deep dive into, and a playlist of foundational tracks you can play right there. It works for albums, genres, labels, and events, not just songs. + +*[Screenshot: Detroit Techno story card showing chapter tabs, key people collage header, and Belleville Three history]* + +## Where this goes past SongDNA + +SongDNA shows you data. Crate tells you stories. + +SongDNA says "this song samples that song." Crate says "J Dilla recorded Donuts from a hospital bed at Cedars-Sinai. His mother brought vinyl records and massaged his swollen hands so he could work the pads. Three days after the album came out on his birthday, he died." + +Some specific differences: + +SongDNA is song-only. Crate works for anything. Ask about a genre, a label, an era, a movement. "The history of Blue Note Records" produces a full narrative with chapters and embedded video. SongDNA can't do that because it's tied to Spotify's track metadata. + +SongDNA stays inside Spotify. Crate connects to Spotify (via Auth0 Token Vault), reads your library, creates playlists from research, and also sends to Slack and Google Docs. I built the connected services integration for the Auth0 "Authorized to Act" hackathon. You research an influence chain and with one click, it becomes a playlist in your Spotify account. + +SongDNA has no audio in context. Crate has a built-in player. When a story card mentions "Strings of Life" by Derrick May, you can play it right there. The key tracks section has play buttons. Published deep cuts at digcrate.app/cuts/ work for anyone with audio playback, no login required. + +SongDNA doesn't cite sources. Crate cites everything. Every influence connection links to the review, interview, or database entry it came from. The influence chain methodology uses co-mention analysis across 26 music publications (Pitchfork, The Guardian, DownBeat, AllMusic, etc.) following the direction convention from Badillo-Goicoechea 2025: from=influencer, to=influenced. If a review of Artist B mentions Artist A, that's an edge from A to B. + +## What I actually think about SongDNA + +I'm not annoyed that Spotify built this. I'm relieved. It validates the idea that music listeners want to understand connections, not just consume streams. SongDNA reaching 200+ million Premium users means the appetite for this kind of exploration is real. + +What SongDNA can't do is go deep. It's a feature inside a player. Crate is a research workspace. When a radio DJ needs to prep a four-track set for tonight's show with talk breaks, social copy, and local event tie-ins, SongDNA doesn't help. When a journalist is writing about the lineage from Alice Coltrane to Flying Lotus, they need cited sources and narrative, not a metadata card. + +Different tools for different depths. SongDNA is the shallow end. Crate is the deep end. Both are needed. + +## The tech + +For anyone curious about how this works under the hood: Crate is a Next.js app with an AI agent harness. The agent (Claude) has access to 20+ tool servers via the Model Context Protocol. When you ask a question, it makes real API calls to real databases, synthesizes the results, and outputs structured UI components via OpenUI. The influence chain doesn't use a pre-built graph. It builds one in real time from review co-mentions, Last.fm similarity scores, and MusicBrainz credits, then enriches each connection via Perplexity with pull quotes and sonic analysis. + +The connected services (Spotify, Slack, Google Docs) use Auth0's Token Vault, which handles OAuth token management so my app never stores raw credentials. I wrote about that integration in a separate post. + +You can try it at [digcrate.app](https://digcrate.app). The influence chains, story cards, and playlists are all live. + +--- + +*Tarik Moody builds tools for people who take music seriously. He's a developer, DJ, and the creator of Crate.* + +*Built for the Auth0 "Authorized to Act" Hackathon (deadline April 6, 2026).* diff --git a/docs/articles/2026-03-25-press-release.md b/docs/articles/2026-03-25-press-release.md new file mode 100644 index 0000000..9386967 --- /dev/null +++ b/docs/articles/2026-03-25-press-release.md @@ -0,0 +1,102 @@ +# PRESS RELEASE + +**FOR IMMEDIATE RELEASE** +**March 25, 2026** + +**Contact:** Tarik Moody | tarikjmoody@gmail.com | Milwaukee, WI + +--- + +## Milwaukee radio executive — not a software engineer — builds AI music research platform that outpaces Spotify's new SongDNA + +*Crate is the first AI platform built for understanding music, not generating it. Like if Claude, Spotify, Wikipedia, and Pitchfork had a baby. Currently in beta.* + +--- + +**MILWAUKEE, WI** — The same week Spotify launched SongDNA, a feature showing connections behind songs for Premium subscribers, a radio executive in Milwaukee shipped something that does the same thing and goes further. + +Spotify tells you what to listen to. Crate tells you why it matters. + +Crate is an AI research agent that queries Discogs, MusicBrainz, Pitchfork, Genius, WhoSampled, and 16 more databases at once, returning cited results as interactive cards. Radio hosts use it to prep shows in 90 seconds instead of 3 hours. DJs use it to trace sample lineage and discover deep cuts. Spotify just acquired WhoSampled because they know music intelligence is the future. Crate is building the full-stack version of that future. + +**The first AI platform for music intelligence, not music generation** + +Most AI products in music are about generation — tools like Suno and Udio that create new music from prompts. Crate does the opposite. It helps people understand music that already exists. + +Streaming platforms are distribution layers. They optimize for passive consumption — playlists, algorithmic feeds, lean-back listening. Crate is a research layer. It optimizes for active engagement — questions, discovery, context, citations. These are complementary, not competing. + +Through Auth0's Token Vault, users connect their Spotify, Slack, and Google accounts once. The AI agent then reads your Spotify library, creates playlists from research, sends formatted show prep to Slack, and saves deep dives as Google Docs. The agent acts on your behalf. + +**Who needs this** + +Radio hosts spend 2-3 hours on show prep, researching across fragmented sources. Stations already budget for automation software in a market worth $530M and growing to $980M by 2033. Radio Milwaukee is the first institutional deployment. + +Professional DJs research tracks across 5+ platforms — Discogs, WhoSampled, YouTube, AllMusic, forums. DJ software handles mixing, not discovery. Crate handles discovery. + +Music journalists research manually across databases, press releases, and interviews. No AI research assistant exists for music journalism. Crate gives them cross-publication source research, artist profiles, influence mapping with citations, and one-click publishing. + +Record collectors use Discogs for buying and cataloging but have no AI-powered research or contextual discovery. Crate provides album deep dives, pressing information, and label catalog research. + +And then there are the millions of serious music enthusiasts between casual Spotify listeners and professionals — people who want to understand why music matters, not just hear more of it. + +**Influence mapping with academic methodology** + +Crate's influence mapping is based on Badillo-Goicoechea's 2025 paper in the Harvard Data Science Review on network-based music recommendation. The system searches for co-mentions across 26 music publications. When a review of Artist B mentions Artist A, that creates a weighted edge in the influence graph. Each connection includes pull quotes, sonic elements, and the specific works that demonstrate the influence. Every claim is cited and verifiable. + +Spotify paid $66M for The Echo Nest and recently acquired WhoSampled data to power SongDNA. Crate's influence chains are the layer above samples — they capture stylistic lineage, not just audio reuse. And every influence query builds Crate's network graph, cached in Convex. Over time, this becomes a proprietary music knowledge graph that gets richer with every user session. + +**Built by a radio executive, not a software engineer** + +Tarik Moody is not a traditional developer. He is Director of Strategy and Innovation at Radio Milwaukee, an NPR member public radio station he has been with since its launch in 2007. Before radio, he practiced architecture. He has no computer science degree. + +Moody built Crate in two weeks using Claude Code, Anthropic's AI coding tool. It evolved from an earlier terminal application he built to speed up his own research workflow. He has been building AI products with Claude Code for months, winning multiple hackathons along the way. + +"I'm building AI products for the rest of us," Moody said. "People who have deep domain expertise but aren't software engineers. Claude Code lets someone who knows music inside and out build a product that a traditional engineering team would need months to deliver." + +Radio Milwaukee is currently testing Crate across its stations: 88Nine (eclectic/community), HYFIN (hip-hop/neo-soul/Afrobeats), and the syndicated Rhythm Lab Radio, a show Moody has hosted and produced for over 20 years. + +"I was spending hours every week researching artists across a dozen different websites, copying information into documents, then reformatting it for air," Moody said. "Now I type one command and get everything — cited, formatted, and ready to send to the team on Slack." + +**What's in the beta** + +Crate is live at digcrate.app with a free tier and a Pro tier ($15/month). Four features show what separates it from SongDNA: + +- **Influence chains** — maps the full influence network with weighted connections, pull quotes, sonic elements, and cited sources from 26 publications. Exportable as a Spotify playlist. +- **Story cards** — magazine-style narratives with chapters, embedded YouTube documentaries, playable tracks, and key people with photos. Works for albums, artists, genres, labels, and events. +- **Track deep dives** — tabbed credits (MusicBrainz + Discogs), sample history (WhoSampled), and vinyl pressing data. The direct SongDNA competitor, with more data sources. +- **Connected services** — research exports directly to Spotify playlists, Slack channels (with Block Kit formatting), and Google Docs. Published "Deep Cuts" are shareable links with audio playback. + +The Auth0 Token Vault integration was built for the Auth0 "Authorized to Act" hackathon (submission deadline April 6, 2026). + +**About Tarik Moody** + +Director of Strategy and Innovation at Radio Milwaukee since the station's 2007 launch. Host and producer of Rhythm Lab Radio, a syndicated music show now in its 20th year. Former architect. Builds AI-powered products using Claude Code. Multiple hackathon winner. Milwaukee, WI. + +**About Radio Milwaukee** + +Public radio station and NPR member serving Milwaukee, WI. Three formats: 88Nine (eclectic/community), HYFIN (hip-hop/neo-soul/Afrobeats), Rhythm Lab Radio (global beats/electronic/jazz). Currently testing Crate for daily show prep and music research. + +**About Crate** + +AI music research platform that searches 20+ databases and gives professionals cited, interactive answers — like having a record store clerk, music librarian, and research assistant in one tool. 27 interactive components. Connected to Spotify, Slack, and Google Docs. Currently in beta at digcrate.app. + +--- + +**Media resources:** +- Live app: https://digcrate.app +- Demo video: [link to video] +- Published research examples: https://digcrate.app/cuts/ +- GitHub: https://github.com/tmoody1973/crate-web +- Screenshots and b-roll available on request + +**Suggested media angles:** + +*For music press (Pitchfork, Resident Advisor, DJ Mag):* "The app that researches music the way DJs and radio hosts actually think — by influence, context, and connection, not by algorithm." + +*For tech press (TechCrunch, The Verge):* "An AI agent that orchestrates 20+ music APIs simultaneously, with source citations on every claim. Built by a radio broadcaster who got tired of Spotify's recommendation bubble." + +*For business press (Billboard, Music Business Worldwide):* "Spotify paid $66M for The Echo Nest and just acquired WhoSampled. The music intelligence category is heating up — and Crate is building the research layer that streaming platforms don't have." + +*For AI/developer press (Hacker News, dev.to):* "A real-world agentic application using 19 MCP servers, OpenUI for dynamic component rendering, and Convex for real-time persistence. Built with Claude Code and shipped to production." + +### diff --git a/docs/articles/2026-04-05-auth0-hackathon-blog.md b/docs/articles/2026-04-05-auth0-hackathon-blog.md new file mode 100644 index 0000000..53eaaee --- /dev/null +++ b/docs/articles/2026-04-05-auth0-hackathon-blog.md @@ -0,0 +1,96 @@ +# How a Radio Broadcaster Used Auth0 Token Vault to Connect an AI Agent to 20+ Music APIs + +I'm Tarik Moody. I've been a radio broadcaster at Radio Milwaukee for 20 years. I am not a software engineer. But I built Crate — an AI music research agent that connects to over 20 data sources — using Claude Code. + +This is the story of how Auth0 Token Vault solved the hardest UX problem in my app: getting users connected to their own accounts without making them manage API keys. + +## The Problem: Fragmented Music Data + +Music professionals live across a dozen platforms. Spotify for listening. Discogs for vinyl. MusicBrainz for metadata. Genius for lyrics. Last.fm for listening history. AllMusic for reviews. Pitchfork for criticism. Bandcamp for independent releases. + +When I started building Crate, the agent needed access to all of these. For some — MusicBrainz, Bandcamp — no authentication is required. For others — Discogs, Last.fm — you need an API key. But for the services where Crate is most powerful — reading your Spotify library, sending show prep to your team's Slack, saving research to Google Docs — you need OAuth. + +OAuth means tokens. Tokens mean complexity. And for a non-engineer building an AI agent, that complexity was the wall. + +## The Old Way: Paste Your API Key + +Before Token Vault, Crate's Settings page had a section called "API Keys." Users would go to Spotify's developer portal, create an app, copy the client ID, paste it into Crate. Same for Discogs. Same for Last.fm. Every service, another portal, another key, another paste. + +It worked. But it was terrible UX. The people who use Crate — radio DJs, music journalists, playlist curators — are not the kind of people who want to navigate developer portals. They want to type "What in my library connects to Afrobeat?" and get an answer. + +## The Token Vault Way: Click Connect + +Auth0 Token Vault replaced all of that with three buttons: **Connect Spotify**, **Connect Slack**, **Connect Google**. + +Click the button. Auth0 opens the OAuth popup. Authorize Crate. Done. The token is stored securely in Auth0 — not in my database, not in a cookie, not in localStorage. Auth0 manages the entire lifecycle: storage, refresh, revocation. + +From the user's perspective, it's a one-click connection. From the agent's perspective, it's a function call: + +```typescript +const token = await getTokenVaultToken("spotify", auth0UserId); +``` + +That single line reaches into Auth0's Management API, finds the user's linked Spotify identity, and returns a fresh OAuth access token. The agent never sees credentials. The user never sees a developer portal. + +## The Architecture: Clerk + Auth0 Side-by-Side + +Here's what makes this interesting: I didn't replace my existing auth system. Crate uses Clerk for user sign-in, session management, and billing. Auth0 handles only the Token Vault connections to third-party services. + +Two auth systems, zero overlap: + +- **Clerk**: User sign-in, sessions, Convex user identity, Stripe billing +- **Auth0 Token Vault**: Spotify OAuth, Slack OAuth, Google OAuth, token storage and refresh + +This hybrid approach works because Token Vault solves a specific problem — securely storing and refreshing third-party OAuth tokens — that my existing auth system doesn't handle. I didn't need to migrate users or change my sign-in flow. I added Token Vault alongside what already worked. + +## What the Agent Can Do Now + +With Token Vault connected, Crate's agent has four new tools: + +**1. Read your Spotify library.** "What in my saved tracks connects to the LA beat scene?" The agent reads your library, finds Flying Lotus, Thundercat, and Knxwledge in your saved tracks, then maps the full influence network with cited sources from MusicBrainz, Discogs, Genius, and AllMusic. + +**2. Export playlists to Spotify.** The influence chain becomes a real Spotify playlist. The agent searches Spotify's catalog for each track, creates a playlist in your account, and adds the tracks. 23 songs, organized by influence lineage, in your Spotify. + +**3. Send research to Slack.** Type "/prep HYFIN" and the agent generates show prep — tonight's setlist with artist context, influence chains, talking points. Then "send this to #hyfin-evening on Slack" and it's delivered with Block Kit formatting: headers, bullet lists, tables, dividers. + +**4. Save to Google Docs.** Any research output can be saved as a shareable Google Doc with one command. The agent creates the document, inserts the content, and returns the link. + +The key insight: these aren't four separate features. They're tools in an agentic loop. A single prompt can chain all four: + +> "What in my Spotify library connects to Afrobeat? Build the influence chain, export it as a playlist, send the prep to #music-research on Slack, and save a copy to Google Docs." + +The agent reads Spotify, runs influence mapping across six databases, exports the playlist, formats and sends the Slack message, creates the Google Doc — all in one conversation. Token Vault provides the secure access at each step. + +## The Hybrid Approach: Token Vault + Existing Keys + +Not every service supports OAuth. Discogs uses OAuth 1.0a (not supported by Token Vault). Last.fm and Ticketmaster use API keys only. MusicBrainz requires no auth at all. + +So Crate runs a hybrid model: + +- **Token Vault** for OAuth services: Spotify, Slack, Google +- **User-managed API keys** for non-OAuth services: Discogs, Last.fm, Genius +- **Embedded platform keys** for open APIs: MusicBrainz, Bandcamp, Pitchfork +- **No auth** for web scraping: AllMusic, Wikipedia, Rate Your Music + +The agent doesn't care which system provides the token. It calls `getTokenVaultToken("spotify")` for Token Vault services and reads environment keys for the rest. The resolution happens at the tool layer, not the agent layer. + +## The Non-Engineer Story + +I built all of this with Claude Code. Not with a team of engineers. Not with years of backend experience. I'm a radio DJ who wanted a better way to research music. + +Auth0 Token Vault made the OAuth integration approachable because it abstracted the hardest parts: token storage, refresh logic, revocation handling, and multi-provider management. I didn't write a token refresh loop. I didn't build a credential database. I called an API. + +If you're building an AI agent that needs to act on behalf of users across multiple services, Token Vault is the difference between "paste your API key" and "click Connect." For the kind of users I serve — music professionals who want to research, not configure — that difference is everything. + +## What's Next + +- **Apple Music** library access via MusicKit JS +- **WordPress** publishing for music blogs +- **Google Calendar** integration for show scheduling +- **Team-shared connections** so all @radiomilwaukee.org users share one Slack workspace connection + +Crate is live at [digcrate.app](https://digcrate.app). The repo is public at [github.com/tmoody1973/crate-web](https://github.com/tmoody1973/crate-web). + +--- + +*Tarik Moody is a radio broadcaster and producer at Radio Milwaukee. He built Crate as an AI research tool for music professionals.* diff --git a/docs/articles/2026-04-05-auth0-hackathon-submission.md b/docs/articles/2026-04-05-auth0-hackathon-submission.md new file mode 100644 index 0000000..bbb5e1c --- /dev/null +++ b/docs/articles/2026-04-05-auth0-hackathon-submission.md @@ -0,0 +1,37 @@ +# Auth0 Hackathon Submission: Crate + +## Project Name +Crate — AI Music Research Agent with Auth0 Token Vault + +## One-Line Description +An AI music research agent that uses Auth0 Token Vault to securely read users' Spotify libraries, export playlists, send show prep to Slack, and save research to Google Docs — all through natural language conversation. + +## Description + +Crate is an AI music research agent that connects to 20+ data sources — Spotify, MusicBrainz, Discogs, Genius, Last.fm, AllMusic, Pitchfork, and more. It maps artist influence networks, generates show prep for radio broadcasters, and provides cited, interactive research about any artist, track, genre, or label. + +**The problem**: Music professionals research across 5-10 fragmented platforms, each requiring separate API credentials. Users shouldn't have to navigate developer portals to connect their accounts. + +**How Token Vault solves it**: Auth0 Token Vault replaces manual API key management with one-click OAuth connections. Users click "Connect Spotify" in Settings, authorize via Auth0's OAuth flow, and the agent can immediately read their library and export playlists. Same for Slack and Google Docs. + +**What makes it agentic**: A single natural language prompt can chain all connected services. "What in my Spotify library connects to Afrobeat? Build the influence chain, export it as a playlist, send the prep to Slack, and save a copy to Google Docs." The agent orchestrates tool calls across all three Token Vault connections in one conversation — reading Spotify via OAuth, running influence mapping across six databases, creating the playlist, formatting and delivering the Slack message, and saving the Google Doc. + +**The hybrid approach**: Token Vault manages OAuth services (Spotify, Slack, Google). Non-OAuth services (Discogs, Last.fm, MusicBrainz) continue using API keys or open APIs. The agent resolves tokens at the tool layer — it doesn't care which system provides access. + +**Architecture**: Clerk handles user authentication. Auth0 handles only Token Vault connections. Two auth systems, zero overlap. The Management API client caches tokens with 23-hour TTL. Per-service Auth0 user IDs stored in httpOnly cookies. HMAC-signed CSRF state for OAuth flows. + +## Tech Stack +- Next.js 15 (App Router) +- Auth0 Token Vault (Management API for IdP token retrieval) +- Clerk (user authentication) +- Convex (database) +- Anthropic Claude (AI agent) +- Vercel (deployment) + +## Links +- **Live app**: https://digcrate.app +- **Repo**: https://github.com/tmoody1973/crate-web +- **Blog post**: included in submission + +## Builder +Tarik Moody — Radio broadcaster and producer at Radio Milwaukee. Built Crate with Claude Code as a solo developer. diff --git a/docs/articles/2026-04-06-auth0-hackathon-submission.md b/docs/articles/2026-04-06-auth0-hackathon-submission.md new file mode 100644 index 0000000..7004bb3 --- /dev/null +++ b/docs/articles/2026-04-06-auth0-hackathon-submission.md @@ -0,0 +1,183 @@ +# Auth0 Hackathon Submission — Crate + +## Inspiration + +I've been a radio DJ at Radio Milwaukee for 20 years. Every show requires hours of research — tracing how artists connect across genres, building playlists, writing show prep, then getting all of that to my team on Slack, archived in Google Docs, shared on my blog. That workflow touches a dozen tabs and four different services, each with its own login. + +When Auth0 announced Token Vault for AI agents, I saw the missing piece. I was already building Crate — an AI music research agent that searches 19 sources in one conversation. But research is only half the job. The other half is getting it where it needs to go. Token Vault meant my AI agent could finally reach into Spotify, Tumblr, Slack, and Google Docs on my behalf — with one OAuth layer handling everything. + +## What it does + +Crate is an AI music research agent for DJs, producers, and serious music lovers. Through Auth0 Token Vault, a single agent connects to four OAuth services: + +- **Spotify** — reads your library, finds genre connections, exports influence chains as playlists +- **Tumblr** — publishes research to your blog, discovers music by tag across all of Tumblr +- **Slack** — sends show prep to your team channel with rich Block Kit formatting +- **Google Docs** — saves research as permanent, searchable documents + +Each service is one click to connect. The agent chains tools autonomously — read library, build influence chain, create playlist, publish to blog, send to team, save to docs — all from natural language. + +## How we built it + +**Auth0 Token Vault integration:** +- Each connected service (Spotify, Tumblr, Slack, Google) is configured as an Auth0 social connection with Token Vault enabled +- User clicks "Connect" in Crate's Settings → Auth0 handles the OAuth popup → token stored in Token Vault → per-service cookie maps the user to their Auth0 identity +- When the AI agent needs to call a service, it retrieves the OAuth bearer token via Auth0's Management API (`getTokenVaultToken("spotify", auth0UserId)`) +- Token refresh is automatic — Auth0 handles it transparently + +**The connect flow (`/api/auth0/connect`):** +- HMAC-signed CSRF state cookie for security +- `connection_scope` parameter passes service-specific OAuth scopes to the IdP +- Separate handling for Slack's `user_scope` (comma-separated) vs standard space-separated scopes +- Per-service cookies (`auth0_user_id_spotify`, `auth0_user_id_tumblr`, etc.) so multiple services don't conflict + +**Tumblr custom social connection:** +- Auth0 doesn't have a built-in Tumblr connector, so we built a custom social connection +- OAuth 2.0 with `https://www.tumblr.com/oauth2/authorize` and `https://api.tumblr.com/v2/oauth2/token` +- Custom Fetch User Profile script that calls Tumblr's `/v2/user/info` endpoint +- Markdown-to-NPF converter for publishing rich posts (headings, bold, italic, links, lists, blockquotes) + +**Tech stack:** +- Next.js 14 (App Router) + TypeScript +- Auth0 Token Vault (OAuth token management for all 4 services) +- Convex (real-time database) +- Anthropic Claude Sonnet 4.6 (AI agent with tool use) +- Clerk (user authentication) +- Vercel (deployment) +- OpenUI Lang (interactive component rendering in chat) + +## Challenges we ran into + +**Tumblr OAuth 2.0 setup was undocumented territory.** Auth0's built-in Tumblr connection uses OAuth 1.0a, but Token Vault needs OAuth 2.0 bearer tokens. We had to build a custom social connection from scratch — discovering that Tumblr requires the "OAuth2 redirect URLs" field to be set separately from the default callback URL, and that the Fetch User Profile script needed the 2-argument signature `(accessToken, callback)` instead of 3. + +**Tumblr post IDs are 64-bit integers.** JavaScript's `JSON.parse()` truncates them because they exceed `Number.MAX_SAFE_INTEGER`. Posts created successfully but returned 404 links because the ID was wrong. Fix: parse the ID from raw response text with regex instead of JSON.parse. + +**Tumblr's NPF format rejects unicode.** Markdown tables, arrow symbols (→), filled circles (●), and emoji all cause 400 errors from Tumblr's API. We built a sanitizer that converts unicode to plain text equivalents and transforms markdown tables into list items (NPF has no table block type). + +**Scope separation for Auth0 authorize URL.** Auth0 treats the `scope` parameter as OIDC-only (openid, profile, email). Service-specific scopes like Spotify's `streaming` or `user-library-read` must go in `connection_scope`, which gets forwarded to the IdP's OAuth endpoint. This took debugging to discover. + +**AI agent hallucination.** When the Tumblr tool returned a "choose blog" action instead of posting directly, the agent fabricated fake success responses instead of asking the user. Fix: hardcode the blog name in the action button flow and add explicit anti-hallucination instructions ("If you respond without calling the tool, you are hallucinating"). + +**Model routing for action buttons.** The Docs/Tumblr/Slack action buttons send natural language messages that were routing to Haiku (the fast, cheap model) instead of Sonnet (which has the tools). Haiku would claim it didn't have the tools and hallucinate. Fix: pattern match action button messages to force Sonnet routing. + +## Accomplishments that we're proud of + +- **Four OAuth services through one integration pattern.** The same `getTokenVaultToken(service, userId)` call works for Spotify, Tumblr, Slack, and Google Docs. Adding a new service is ~10 lines of config. + +- **Custom Tumblr OAuth 2.0 connection.** Built from scratch when the built-in Auth0 connector didn't support Token Vault. Full OAuth 2.0 flow with bearer tokens, custom profile script, and NPF publishing. + +- **One-click action buttons.** Under every Crate response, users can click Tumblr, Slack, Docs, or Copy to instantly route research to any connected service. The agent handles everything. + +- **Real, live results.** Every demo moment is real — the Spotify playlist exists, the Tumblr post is published, the Slack message lands in the channel, the Google Doc is saved. No mocks, no fakes. + +- **Markdown-to-NPF converter.** Converts the agent's markdown output to Tumblr's Neue Post Format with headings, bold, italic, links, lists, blockquotes, and code blocks — plus unicode sanitization and table-to-list conversion. + +## What we learned + +- **Token Vault is the right abstraction for agentic OAuth.** Instead of each tool managing its own token lifecycle, Token Vault centralizes it. The agent just asks for a bearer token and calls the API. Refresh, storage, consent — all handled by Auth0. + +- **Custom social connections unlock services Auth0 doesn't cover.** Tumblr isn't in Auth0's built-in list, but the custom connection framework let us add it with full Token Vault support. Any OAuth 2.0 service can be added this way. + +- **AI agents need guardrails around tool use.** The biggest challenge wasn't the OAuth plumbing — it was preventing the agent from hallucinating tool calls. Explicit instructions, model routing, and hardcoded parameters were all necessary to make the agent reliably call real tools. + +- **The `connection_scope` parameter is essential.** Without it, service-specific scopes never reach the IdP. This is easy to miss and hard to debug because Auth0 doesn't error — it just doesn't pass the scopes. + +## What's next for Crate + +- **Apple Music via MusicKit JS** — Apple doesn't use OAuth for music access (it uses a proprietary Developer Token + Music User Token system), so it can't go through Token Vault. We'll integrate MusicKit JS client-side as a parallel path. + +- **Inline Slack channel picker** — Currently the agent lists channels as text. We want a dropdown picker form inline in the chat, similar to the show prep form. + +- **More connected services** — SoundCloud, Bandcamp, Discord, Notion. Each one is ~10 lines of Token Vault config plus a tool file. + +- **Cross-service workflows** — "Find what's trending on Tumblr #jazz, add the best tracks to my Spotify playlist, and send a summary to my team on Slack." Multi-service chains that show the full power of agentic Token Vault. + +- **Token Vault for team workspaces** — Radio stations have teams. Token Vault could manage shared OAuth connections so the whole team's agents access the same Spotify, Slack, and Docs without individual setup. + +## How Crate Meets the Judging Criteria + +### Security Model + +Crate's agent operates within explicit permission boundaries at every layer: + +- **Tokens never touch Crate's database.** Auth0 Token Vault stores and refreshes all OAuth tokens. The agent retrieves them at runtime via Auth0's Management API through `getTokenVaultToken()`. If Crate's database were compromised, no OAuth credentials would be exposed. +- **HMAC-signed CSRF protection** on the OAuth connect flow. A cryptographic nonce is stored in a signed HttpOnly cookie and verified on callback. The state parameter never contains user data. +- **Per-service HttpOnly cookies** (`auth0_user_id_spotify`, `auth0_user_id_tumblr`, etc.) isolate each service's Auth0 identity. Compromising one cookie doesn't grant access to other services. +- **Minimal, explicit scopes per service.** Spotify gets `user-library-read` + `playlist-modify-public` + `streaming`, not blanket access. Tumblr gets `basic` + `write` + `offline_access`. Slack gets `chat:write` + `channels:read`. Google gets `documents` + `drive.file` (only files Crate creates, not the entire Drive). Each scope is chosen for the specific tools the agent needs. +- **Credentials encrypted at rest.** User-provided API keys (BYOK) are encrypted with a 64-character hex key before storage. + +### User Control + +Users understand and control what the agent can access: + +- **Settings page** shows all four connected services with clear Connected/Disconnected badges and one-click Connect/Disconnect buttons. +- **OAuth consent screens** display exactly what Crate requests. Spotify's screen shows "Read your library" and "Create playlists." Tumblr shows "Read and write on your behalf." Users grant permissions explicitly through their service's native consent UI. +- **Independent disconnection.** Users can disconnect Spotify without affecting Tumblr, or disconnect all four services individually. Disconnection removes the per-service cookie immediately. +- **BYOK (Bring Your Own Key).** Users who provide their own Anthropic or OpenRouter API key bypass platform quotas entirely. No vendor lock-in. +- **Scopes defined in code and visible in Auth0 Dashboard.** The `SERVICE_CONFIG` in `auth0-token-vault.ts` maps each service to its exact OAuth scopes. These are passed transparently via `connection_scope` to the identity provider. + +### Technical Execution + +The Token Vault integration is production-grade and deployed live: + +- **Clean, replicable pattern.** `SERVICE_CONFIG` maps service names to Auth0 connections and scopes. `getTokenVaultToken(service, userId)` retrieves any service's token. Adding a new service is ~10 lines of config. This pattern works for any OAuth 2.0 provider. +- **Management API token cached** with 23-hour TTL (refreshes 1 hour early). Not hitting Auth0 on every agent tool call. +- **Custom Tumblr social connection** built from scratch. Auth0's built-in Tumblr connector uses OAuth 1.0a, but Token Vault needs OAuth 2.0 bearer tokens. We created a custom connection with OAuth 2.0 authorize/token endpoints and a custom Fetch User Profile script. +- **Edge cases handled:** CSRF nonce verification, missing id_token fallback to /userinfo, Slack's comma-separated `user_scope` vs space-separated `connection_scope`, 64-bit post ID preservation via regex (JSON.parse truncates IDs exceeding Number.MAX_SAFE_INTEGER). +- **Production deployed.** Live at digcrate.app on Vercel. Real users, real Spotify libraries, real Tumblr posts, real Slack messages. Not a localhost demo. + +### Design + +The user experience blends frontend interactivity with backend agent infrastructure: + +- **27+ interactive OpenUI components** render in the chat at runtime. The agent generates artist profiles, influence chains, show prep packages, Spotify playlist browsers, Tumblr feeds, and Slack channel pickers as React components, not just text. +- **Action buttons under every response.** Copy, Slack, Email, Docs, Tumblr, Share. One click to route any research to any connected service. The buttons send a clean one-line message while the preprocessor injects detailed tool instructions the agent follows. +- **TumblrFeed component** with post type filters (Audio, Text, Photo, Link, Video, Quote), blog attribution, tag pills, note counts, and "View on Tumblr" links. Handles all Tumblr post types with type-specific rendering. +- **Dark theme** consistent across all surfaces: chat, settings, landing page, components, published shares. +- **Mobile responsive.** Full mobile UX with hamburger sidebar, touch-optimized inputs, mini player bar, speech-to-text. +- **Backend architecture matches frontend quality.** The agentic loop, tool registry, OpenUI prompt generation, and Token Vault integration are all well-structured TypeScript with clear separation of concerns. + +### Potential Impact + +**For AI developers:** +- The Token Vault integration pattern (`SERVICE_CONFIG` + `getTokenVaultToken()` + per-service cookies) is directly replicable. Any AI agent builder can copy this architecture to connect their agent to OAuth services. +- The custom Tumblr social connection proves that services not in Auth0's built-in list can still use Token Vault. This extends Token Vault's reach to any OAuth 2.0 provider. +- The anti-hallucination patterns we discovered (model routing for action buttons, explicit tool instructions, "if you respond without calling the tool, you are hallucinating") are hard-won lessons applicable to any agentic tool-use system. + +**Beyond AI developers:** +- Crate serves a real professional community. Radio DJs, producers, and music journalists spend hours on research that Crate condenses to minutes. The Token Vault integration means that research flows directly to Spotify playlists, team Slack channels, published blogs, and archived docs. +- The "one prompt, four services" demo shows non-technical users what agentic AI looks like when the auth layer is solved. This is the kind of workflow that makes AI tangible and useful, not theoretical. + +### Insight Value + +Building Crate with Token Vault surfaced patterns, pain points, and gaps that directly inform how agent authorization should evolve: + +**Patterns:** +- **`connection_scope` vs `scope` is critical and underdocumented.** Auth0's authorize endpoint treats `scope` as OIDC-only. Service-specific scopes (Spotify's `streaming`, Tumblr's `write`) must go in `connection_scope`. We burned hours learning this. Clearer docs would save every developer this pain. +- **Custom social connections unlock the long tail.** Tumblr isn't in Auth0's built-in list, but the custom connection framework let us add it with full Token Vault support in an afternoon. This is a powerful pattern that deserves more visibility. +- **Per-service cookies solve multi-identity.** Each Auth0 social connection creates a separate Auth0 user. Storing per-service Auth0 user IDs in separate HttpOnly cookies prevents cross-contamination and supports users who connect multiple services. + +**Pain points:** +- **64-bit integer truncation.** Tumblr post IDs exceed `Number.MAX_SAFE_INTEGER`. `JSON.parse()` silently truncates them, producing valid-looking but wrong IDs. Any service with large numeric IDs will hit this. Token Vault or the Management API could return IDs as strings. +- **Tumblr OAuth 2.0 activation is hidden.** The "OAuth2 redirect URLs" field in Tumblr's app settings must be filled for OAuth 2.0 to activate. Without it, OAuth 2.0 is silently disabled and users see the login page but no consent screen. Not documented clearly anywhere. +- **AI agents hallucinate tool calls.** When the agent receives an intermediate response (e.g., "choose a blog"), it sometimes fabricates a success message instead of following up. Agentic workflows need explicit guardrails, model routing, and anti-hallucination instructions. + +**Gaps that should inform Token Vault's evolution:** +- **No agent audit log.** Users can't see "Crate read your Spotify library at 3:42pm" or "Crate created a playlist at 3:45pm." Token Vault handles the auth, but the actions performed with those tokens aren't logged visibly. An agent activity log would build user trust and enable debugging. +- **No token health visibility.** "Is my Spotify token still valid?" requires calling the API and seeing if it 401s. Token Vault could expose a status endpoint or webhook for token expiry/refresh events. +- **No differentiated risk levels.** Reading a library vs creating a playlist vs publishing to a blog are different risk levels. Token Vault treats them identically. Step-up authentication for write operations would add a meaningful security layer for agentic workflows. +- **No multi-step OAuth in a single agent turn.** If the user hasn't connected a service, the agent can't initiate the OAuth flow mid-conversation. The user must leave the chat, go to Settings, connect, and return. A "connect inline" flow would make the agent experience seamless. + +## Bonus Blog Post + +### The Moment I Stopped Managing Tokens + +There's a specific moment during this hackathon that changed how I think about building AI tools. + +I was debugging why Tumblr kept returning 400 errors. The markdown-to-NPF converter was choking on unicode arrows and filled circles from the influence chain output. I'd spent an hour on it. And then it hit me — I was debugging *content formatting*, not authentication. Not token refresh. Not OAuth scopes. + +That's the whole point of Token Vault. I never once debugged a token expiry. Never wrote a refresh flow. Never stored a credential in my database. I just called `getTokenVaultToken("tumblr")` and got a bearer token back. Same call for Spotify, Slack, Google Docs. Four services, one pattern, zero token plumbing. + +The wildest challenge was Tumblr itself. Auth0 doesn't have a built-in Tumblr connector for Token Vault, so I built a custom social connection from scratch — OAuth 2.0 authorize URL, token endpoint, custom profile script. The Fetch User Profile script needed a specific two-argument function signature that took three attempts to get right. And Tumblr post IDs are 64-bit integers that JavaScript silently truncates, so every post link was a 404 until I parsed the ID from raw text instead of JSON. + +But through all of that, the OAuth layer just worked. Token Vault earned its name. I spent my time on the product — influence chains, show prep, music discovery — instead of reinventing auth infrastructure. For a solo developer building an AI agent that touches four different services, that's everything. diff --git a/docs/articles/2026-04-07-agent-connect-linkedin.md b/docs/articles/2026-04-07-agent-connect-linkedin.md new file mode 100644 index 0000000..b2ba2bf --- /dev/null +++ b/docs/articles/2026-04-07-agent-connect-linkedin.md @@ -0,0 +1,37 @@ +# Agent Connect — LinkedIn Post + +**Date:** 2026-04-07 +**Target:** Music lovers, DJs, producers, playlist curators, music nerds + +--- + +Stop me if this sounds familiar. + +You found an artist you love. Now you want to know who influenced them, what they sampled, who they collaborated with, and how they connect to the rest of your library. So you open Spotify in one tab, Discogs in another, Genius in a third, YouTube for a documentary, and Wikipedia for the backstory. Five apps. Fifteen tabs. An hour later you've got half the picture. + +That's every deep music session for anyone who actually cares about the music behind the music. + +So I built Crate. + +**Crate is Spotify for the curious.** An AI music research agent. Ask it anything about any artist, genre, sample, or connection, and it searches 19 music sources at once. Discogs, MusicBrainz, Last.fm, Genius, WhoSampled, Spotify, Bandcamp, and more. It gives you back interactive components, not just text. Influence maps. Artist deep dives. Sample trees. Playable playlists. All in one conversation. + +But research was only half the problem. The other half was doing something with it. + +Today I'm introducing **Agent Connect**. + +Connect your Spotify, Tumblr, Slack, and Google Docs once. Then just ask. + +"What in my library connects to Afrobeat? Build the influence chain, export it as a playlist, post the research to my blog, send it to my group chat, and save a copy to Docs." + +One prompt. The agent reads your Spotify library, maps the connections, creates a real playlist in your account, publishes to your Tumblr, shares it with your people, and saves everything. You never leave the conversation. + +One prompt. Every service. +Spotify. Tumblr. Slack. Google Docs. + +Crate is free to try at digcrate.app + +If you're a DJ, producer, music nerd, playlist curator, or just someone who goes deep... this was built for you. + +digcrate.app + +#Crate #AgentConnect #MusicTech #AI #Spotify #MusicResearch #MusicDiscovery #CrateDigging diff --git a/docs/articles/2026-04-07-crate-acquihire-thesis.md b/docs/articles/2026-04-07-crate-acquihire-thesis.md new file mode 100644 index 0000000..f3121dc --- /dev/null +++ b/docs/articles/2026-04-07-crate-acquihire-thesis.md @@ -0,0 +1,199 @@ +# Crate as an Intelligence Layer for Streaming Platforms + +## The Acqui-Hire Thesis: Why Streaming Companies Buy Taste, Not Code + +--- + +## The Pattern: Every Major Music Acquisition Was About People, Not Technology + +Streaming companies have spent billions acquiring domain expertise they couldn't build internally. The code was never the point. + +### Spotify + The Echo Nest (March 2014, ~$66M) + +Spotify had its own recommendation engine. They didn't need Echo Nest's code. They bought the team's conceptual framework for "music intelligence." Daniel Ek said it directly: "We are hyper focused on creating the best user experience and it starts with building the best music intelligence platform on the planet. With The Echo Nest joining Spotify, we will make a big leap forward." + +Echo Nest called itself a "music intelligence company." Not a data company. Not an API company. An intelligence company. Spotify bought the people who knew what music intelligence meant. + +### Apple + Beats Electronics (May 2014, $3.2B) + +Apple's largest acquisition ever. Tim Cook: "What Beats brings to Apple are guys with very rare skills. People like this aren't born every day. They're very rare. They really get music deeply." + +Jimmy Iovine's pitch to Apple was six words: "I know you have a hole in music right now; let me plug it." + +Apple didn't buy headphones. They bought Jimmy Iovine and Dr. Dre's taste, industry relationships, and cultural credibility. Things Apple's 150,000 employees couldn't manufacture internally. + +### Apple Hires Zane Lowe (2015) + +Zane Lowe left BBC Radio 1, one of the most coveted curatorial positions in music, to become Apple Music's Global Creative Director. Apple called him "the world's foremost authority on emerging music." + +A talent hire, not an acquisition. But the same pattern: Apple paid for domain expertise that cannot be engineered. + +### Spotify + WhoSampled (November 2025) + +Spotify acquired WhoSampled's team and their database of 1.2 million songs and 622,000+ sample, cover, and remix connections. Community-built over 17 years. + +This directly led to SongDNA (launched March 2026), a premium feature showing writers, producers, samples, and connections on the Now Playing screen. + +### Google + ProducerAI (February 2026) + +ProducerAI (formerly Riffusion) was founded by two Princeton classmates who spent a decade playing in an amateur band together. Musicians first, engineers second. + +Google acquired the entire team into Google Labs and Google DeepMind. What Google valued: "A team with deep technical and musical credentials... a conversational music creation UX that took three years to build." The product was a conversational workflow where you workshop music with an AI agent, not a one-shot prompt. The founders understood that because they were the users. + +--- + +## The OpenClaw Precedent: Why OpenAI Hired a Solo Builder + +In February 2026, OpenAI hired Peter Steinberger, the creator of OpenClaw, an open-source autonomous AI agent for messaging platforms. There were alternatives: ZeroClaw, Hermes, and dozens of similar tools. + +Sam Altman's announcement: "Peter Steinberger is joining OpenAI to drive the next generation of personal agents. He is a genius with a lot of amazing ideas about the future of very smart agents interacting with each other to do very useful things for people." + +OpenAI didn't need the code. OpenClaw is open source. Anyone can fork it. + +VentureBeat's analysis: "The most valuable part of OpenClaw was never just the codebase. It was the design philosophy behind it, how agents should orchestrate tools, manage tasks and recover from failure. **Code can be forked, but judgment cannot.**" + +OpenAI has hundreds of engineers who could build a coding CLI in a week. They hired Steinberger because he'd already done the iteration loop in public, already watched real users struggle with agent tools, and already knew what v3 should look like. That saves them a year of product discovery. + +**The structural analogy to Crate is direct.** One builder. Domain expertise embedded in the product. Tool gets traction among practitioners. Giant company hires the person, not the product. + +--- + +## The Taste Gap: Big Tech Music Products Keep Missing + +The evidence that streaming companies can't build great music products internally is extensive. + +### Spotify's Fake Artist Scandal (January 2025) + +Journalist Liz Pelly published "The Ghosts in the Machine" in Harper's Magazine, revealing Spotify's Perfect Fit Content (PFC) program. Internal records showed Spotify had been commissioning music under fake artist names and inserting it into hundreds of playlists since 2017. + +Pelly's verdict: "The whole system is geared toward generating anonymous background music that further removes the concept of an individual artist with a voice and perspective... the musical equivalent of packing peanuts." + +This is what happens when finance teams and engineers make music product decisions without domain expertise. Any working musician or curator would have found this approach unacceptable. + +### Spotify's Discover Weekly Deterioration + +In 2020, Spotify CEO Daniel Ek declared the company would lean into algorithmic suggestions over human editorial. Major labels reported streams from flagship playlists like RapCaviar dropped 30-50% as human-curated lists shifted to algorithmic personalized versions. + +Spotify's community forums are full of threads: "Discover Weekly constantly terrible." "Discover weekly is getting worse month by month." The core complaint: the algorithm prioritizes familiarity and repeat listens over actual discovery. + +### YouTube Music's Cultural Void + +YouTube Music has 80M subscribers vs Spotify's 246M premium. Despite Google's unlimited engineering resources and YouTube's massive video catalog, they're a distant fourth in paid subscribers. + +The documented gaps: +- Album descriptions sourced from Wikipedia (Apple Music has staff-written editorial notes) +- No sorting by title/artist/composer in the library (basic feature present in every competitor) +- Audio capped at 256kbps AAC (Spotify: 320kbps, Apple: lossless) +- No social layer comparable to Spotify's collaborative playlists +- No editorial voice or curatorial personality + +One widely cited review: "YouTube Music, backed by Google's AI prowess, excels in algorithmic recommendations but lacks curatorial personality." + +Google has the best engineers in the world. Their music product feels like it was built by people who don't listen to music. That's the taste gap. + +--- + +## Could Crate Be the Intelligence Layer? + +Yes. And here's why SongDNA proves the gap exists. + +### What SongDNA Is + +SongDNA launched in March 2026 as a premium Spotify feature. It shows writers, producers, collaborators, samples, interpolations, and covers on the Now Playing screen. Users tap through a network of connections. It's powered by the WhoSampled database Spotify acquired. + +### What SongDNA Is Not + +SongDNA is a **static data visualization tool**. It: +- Displays existing relationships from a database +- Allows tapping through pre-computed connections +- Shows data cards on supported tracks only + +SongDNA does NOT: +- Answer questions conversationally +- Explain why a connection matters for a specific context +- Generate insights the database doesn't already contain +- Cross-reference multiple sources (it uses WhoSampled data, not Discogs + MusicBrainz + Last.fm + Genius + Pitchfork combined) +- Create playlists from discovered connections +- Publish research to external services +- Generate show prep, talk breaks, or social copy +- Work for tracks not yet in the WhoSampled database + +SongDNA shows you the graph. It does not reason about the graph. + +### What Crate Is + +Crate is the intelligence layer that sits on top of the data. It: +- **Reasons across 19 sources simultaneously.** Not just WhoSampled, but Discogs, MusicBrainz, Last.fm, Genius, Bandcamp, Pitchfork (via Perplexity), Wikipedia, Ticketmaster, and more. No single database has all the answers. +- **Answers questions in context.** "What in MY library connects to Afrobeat?" is a question SongDNA can't answer because it doesn't know your library. Crate reads your Spotify library and maps the connections. +- **Creates artifacts from research.** Influence chains, show prep packages, playlists, artist profiles. Not just data cards. Interactive components you can act on. +- **Acts across services.** Export to Spotify. Publish to Tumblr. Send to Slack. Save to Google Docs. The research doesn't stay in the app. It goes where it needs to go. +- **Applies domain judgment to every output.** Talk breaks at 15/60/120 seconds. Influence strength scores. Source citations from Pitchfork and NPR. These are product decisions made by someone who knows what music professionals need. + +### The Intelligence Layer Pitch + +If Crate were inside Spotify, YouTube Music, or Apple Music, it would transform how these platforms understand and present music: + +**For listeners:** "Why was this song recommended?" becomes a visual influence chain showing how your listening history connects to the recommendation. Not "because people who like X also like Y" but "because X was influenced by Z, who collaborated with Q, who produced the track you just heard." + +**For editorial teams:** Instead of manually researching every playlist, curators ask Crate to generate a playlist rationale, artist context, and editorial notes. Research that takes hours becomes a one-prompt task. + +**For A&R:** "Find unsigned artists in the same influence lineage as Kokoroko" is a query that crosses MusicBrainz, Last.fm, Bandcamp, and Spotify simultaneously. No existing tool does this. + +**For the platform itself:** Every track page becomes intelligent. Not just "written by" credits, but "why this track matters," "how it connects to what you listen to," and "what to explore next" based on actual music intelligence, not collaborative filtering. + +--- + +## The Vertical AI Agent Thesis + +Y Combinator's Lightcone Podcast (November 2024, "Vertical AI Agents Could Be 10X Bigger Than SaaS") laid out the framework directly: + +Jared Friedman (YC Partner): "It's very possible the vertical equivalence will be 10 times as large as the SaaS company that they are disrupting." + +The argument: SaaS digitized workflows. Vertical AI agents automate entire workflows. Music research is a workflow. Crate automates it. + +Garry Tan shared data (February 2026) showing AI agent usage by category: software engineering at 49.7%, healthcare at 1%, legal at 0.9%, education at 1.8%. Music: not yet on the chart. That's the white space. + +The thesis says the biggest vertical AI companies will emerge in domains where: +1. The workflow is complex and multi-source (music research: 19+ sources) +2. Domain expertise is required to make the right product decisions (20 years of radio) +3. The horizontal tools don't serve the vertical well (ChatGPT can't search Discogs) +4. The incumbent software is analytics, not intelligence (Chartmetric, SongDNA) + +Crate checks all four. + +--- + +## Why Tarik Moody Is the Acquisition + +Streaming companies have radio people. They have curators. They have engineers. What they don't have is one person who is all three and ships. + +Their radio people know what DJs need but can't build software. Their engineers can build software but don't know what DJs need. Their curators understand music deeply but can't translate that into product. These three groups sit in different departments and communicate through meetings, PRDs, Jira tickets, and quarterly roadmaps. By the time a feature ships, six months of translation loss has degraded the original insight. + +Tarik skipped all of that. A non-technical 20-year radio DJ designed, built, and shipped a production SaaS with 19 data sources, 27 interactive components, 5 connected services, Stripe billing, and a live domain. The feedback loop between "I need this" and "I built this" was zero. + +That's what gets acquired. Not the repo. The person who made 500 correct product decisions that no one else in the acquiring company would have made. + +Tim Cook's words about Beats apply directly: "People like this aren't born every day. They're very rare. They really get music deeply." + +VentureBeat's words about OpenClaw apply directly: "Code can be forked, but judgment cannot." + +--- + +## Sources + +- Spotify acquires The Echo Nest, TechCrunch, March 2014 +- Apple buys Beats for $3.2B, Billboard, May 2014 +- Tim Cook on Beats: "Very rare skills," MacRumors, December 2014 +- Jimmy Iovine: "You have a hole in music," MacRumors, December 2014 +- Spotify acquires Niland, TechCrunch, May 2017 +- Spotify acquires WhoSampled, TechCrunch, November 2025 +- SongDNA beta launch, Spotify Newsroom, March 2026 +- OpenClaw creator joins OpenAI, TechCrunch, February 2026 +- Sam Altman on Steinberger, CNBC, February 2026 +- "Code can be forked, but judgment cannot," VentureBeat, February 2026 +- "The Ghosts in the Machine," Liz Pelly, Harper's Magazine, January 2025 +- "Mood Machine," Liz Pelly, Astra House, January 2025 +- YouTube Music struggles, Digital Music News, October 2025 +- "Vertical AI Agents Could Be 10X Bigger Than SaaS," YC Lightcone, November 2024 +- Google acquires ProducerAI, Music Business Worldwide, February 2026 +- Garry Tan AI agent market data, February 2026 diff --git a/docs/crate-article.md b/docs/crate-article.md new file mode 100644 index 0000000..15f3010 --- /dev/null +++ b/docs/crate-article.md @@ -0,0 +1,180 @@ +# I built a music research platform that does what Spotify won't + +## From a terminal experiment to a full browser app — here's the story of Crate + +I spend a lot of time thinking about music. Not just listening — researching. Who produced that track? Where did that sample come from? How does a DJ in Milwaukee connect Ethiopian jazz to UK broken beat in a 90-minute set? These are real questions I deal with as a radio broadcaster, and none of the existing tools answer them well. + +Spotify tells you what to listen to. It doesn't tell you why it matters. + +So I built Crate. + +--- + +## What Crate actually is + +Crate is a music research workspace powered by an AI agent that can query 20+ music databases simultaneously. You ask it a question — about an artist, a genre, a production technique, a radio show — and it goes and finds the answer. Not from one source. From Discogs, MusicBrainz, Last.fm, Genius, Spotify, Wikipedia, Pitchfork, Bandcamp, and about a dozen more. It cross-references what it finds, cites its sources, and gives you the results as interactive cards you can save, share, and publish. + +It also plays music. YouTube playback built right into the app. 30,000+ live radio stations you can stream while you research. A persistent player bar at the bottom, like Spotify, that follows you around while you work. + +But the real thing that makes it different: it understands context. Ask it about Flying Lotus and it won't just give you a discography. It'll trace the influence chain from Alice Coltrane through his aunt to Brainfeeder Records, pull the Pitchfork review that first made the connection, and show you the whole network as an interactive visualization. Every claim backed by a source you can click. + +--- + +## The CLI origin story + +Crate didn't start in the browser. It started in the terminal. + +I built Crate CLI first — a command-line tool using the Claude Agent SDK (now called the Anthropic SDK) that connected to 19 MCP servers. MCP stands for Model Context Protocol. Think of each server as a specialist. One knows Discogs. One knows Genius. One knows Last.fm. The agent orchestrates them — decides which sources to query, in what order, and how to combine the results. + +The CLI worked well for me personally. I'd type a question, the agent would fire off tool calls to multiple databases, and I'd get back deep research in under a minute. But nobody else was going to install a command-line tool and configure API keys in a terminal. The music people I wanted to reach — DJs, producers, radio programmers — they need a browser. + +So I ported the whole thing to the web. Same agent, same 19 research servers, same depth. Just wrapped in a Next.js app with a real UI. + +--- + +## How the agent works + +When you send a message in Crate, here's what actually happens: + +1. Your message hits a Next.js API route on Vercel +2. The route creates a CrateAgent instance — the same agent class from the CLI, imported as an npm package +3. The agent reads your message and decides which tools to use. It has access to 19 MCP servers, each with multiple tools. Discogs alone has search, release lookup, artist lookup, and credits tools. +4. The agent starts making tool calls. For a question like "map the influences of Madlib," it might call Last.fm for similar artists, search 26 music publications for review co-mentions, check MusicBrainz for collaboration credits, and pull images from Spotify — all in one research session. +5. Each tool call streams back to the browser as a Server-Sent Event. You see the tools firing in real time — "Searching Discogs... Checking Genius... Querying Last.fm..." +6. When the agent has enough information, it generates the response using OpenUI Lang — a structured format that renders as interactive cards instead of plain text. An influence map becomes a clickable visualization. A playlist becomes a list with play buttons. Show prep becomes a package with talk breaks and social copy. +7. Everything saves to Convex (real-time cloud database) so you can come back to it later. + +The agent isn't just a chatbot wrapper. It has a personality — part obsessive record store clerk, part Gilles Peterson. It follows influence chains, prioritizes deep cuts over obvious picks, and treats genre as a filing system rather than a fence. + +For the AI model, you bring your own key — either Anthropic (for Claude directly) or OpenRouter (for access to Claude, GPT-4, Gemini, and others through one API). You pick your model in settings. The data source keys (Discogs, Last.fm, Spotify, etc.) are embedded — those just work. + +--- + +## How we built it — the Superpowers workflow + +Here's something I want to be transparent about: I built most of Crate Web using AI-assisted development. Specifically, I used Claude Code with a plugin called Superpowers that structures how you go from idea to shipped code. + +The workflow looks like this: + +**Brainstorming phase.** I'd describe what I wanted — "I need radio streaming in the browser with live metadata" — and the brainstorming skill would ask me questions one at a time. What stations? What metadata format? How should it handle switching between radio and YouTube? We'd go back and forth until the design was clear, then it would write a spec document. + +**Planning phase.** The spec gets turned into a detailed implementation plan. Every file that needs to be created or modified, every function signature, every test case. The plan is granular — each step takes 2-5 minutes. Write the failing test, run it, write the implementation, run the test again, commit. + +**Execution phase.** This is where it gets interesting. Superpowers uses subagent-driven development — it dispatches a fresh AI agent for each task in the plan. Each subagent gets exactly the context it needs (not the whole conversation history) and works in isolation. After each task, two review agents check the work: one for spec compliance (did you build what was asked?) and one for code quality (is it well-written?). + +**The review loop.** If a reviewer finds issues, the implementer agent fixes them and gets reviewed again. This cycle repeats until both reviewers approve. Then the next task starts. It's like having a junior developer with two senior reviewers, except nobody gets tired and the reviews are consistent. + +For example, the radio feature went through this pipeline: +- Brainstorm: figured out we needed to replace the CLI's mpv player with HTML5 Audio, add ICY metadata parsing, and wire play_radio events through the streaming pipeline +- Plan: 10 files, 5 tasks, each with tests +- Execute: subagents wrote the code, reviewers caught an SSRF vulnerability (the metadata proxy was fetching user-supplied URLs without checking for private IPs), caught a re-rendering bug in the metadata polling, and flagged an accessibility issue on the seek bar + +CodeRabbit (automated GitHub reviewer) then does a final pass on the PR. Between Superpowers and CodeRabbit, most bugs get caught before I even look at the diff. + +I still make every design decision. I still review every PR. But the mechanical work — writing boilerplate, catching edge cases, maintaining consistency across files — that's handled by the agents. + +--- + +## The tech stack (plain English) + +**Frontend:** Next.js 16 with React 19. This is a modern web framework that handles both the UI and the server-side API routes. Tailwind CSS for styling — utility classes instead of writing CSS files. OpenUI for rendering the agent's structured output as interactive components. + +**Backend:** Convex for the database. It's a real-time cloud database where data syncs instantly between server and browser. No SQL, no migrations, no ORM. You define your schema in TypeScript and write queries as functions. Authentication through Clerk — handles sign-in, OAuth, user management. + +**AI layer:** The Anthropic SDK talks directly to Claude. OpenAI SDK routes through OpenRouter for multi-model support. The CrateAgent class from crate-cli orchestrates 19 MCP tool servers. Zod validates all the data schemas. + +**Deployment:** Vercel hosts the app. Push to main, it deploys automatically. Convex runs the database in the cloud. No servers to manage. + +**The agent's brain:** Claude (Anthropic's model) with a custom system prompt that gives it the record-store-clerk-meets-Gilles-Peterson personality. The prompt includes rules for how to research, how to cite sources, how to generate interactive components, and how to handle influence mapping using a methodology based on an actual academic paper (Badillo-Goicoechea 2025 — network-based music recommendation through review co-mentions). + +--- + +## Influence mapping — the thing nobody else does + +This is probably the feature I'm most proud of, and it's grounded in actual academic research. + +Most music recommendation works on behavior data. Spotify looks at what you've listened to and finds people with similar listening patterns. That's fine for "more of the same." But it can't answer "why does this artist sound like that?" or "how did Detroit techno end up influencing Berlin minimal?" Those are questions about influence — who came before whom, who cited whom, who sampled whom. + +In Fall 2025, Elena Badillo-Goicoechea published a paper in the Harvard Data Science Review called "Modeling Artist Influence for Music Selection and Recommendation: A Purely Network-Based Approach." The core idea: you can build a knowledge graph of artistic influence by analyzing music criticism. When a reviewer mentions two artists in the same review, that's a signal. If Pitchfork reviews a Thundercat album and mentions J Dilla three times, that co-mention carries weight. Do that across 26 publications — Pitchfork, NME, Rolling Stone, Stereogum, The Wire, Tiny Mix Tapes, and more — and you get a network of influence that's based on expert critical analysis, not algorithmic guessing. + +That's what Crate's influence mapping does. When you type `/influence Flying Lotus`, the agent: + +1. Checks the Convex-backed graph cache first — if we've already mapped this artist, you get instant results +2. If the cache is thin, it runs discovery: searching across 26 publications for review co-mentions, pulling similar artists from Last.fm, checking MusicBrainz for collaboration credits, and running web searches for documented influence +3. Each connection gets a weight (0 to 1) based on how many independent sources confirm it and how strong the signal is. A single blog mention scores lower than repeated co-mentions across multiple publications. +4. Every connection includes context ("Reviewed in Pitchfork alongside X, producer credits on Y's 2019 album") and clickable source links so you can verify the claim yourself +5. Results save to the graph cache. Next time anyone asks about the same artist, the network is already there — and it grows over time as more queries add more edges + +The paper calls this approach "emulating exhaustively reading through linked sequences of reviews, discovering new artists mentioned in each piece." That's exactly what the agent does, except it reads hundreds of reviews in seconds instead of weeks. + +The result is an interactive visualization — the InfluenceChain component — that shows you a vertical timeline of connections sorted by weight, with artist images, relationship types (influenced, collaborated, sampled, co-mentioned), and evidence cards. You can trace paths between artists: "How are J Dilla and Thundercat connected?" and get back a chain showing the intermediary nodes and the evidence for each hop. + +This is different from "similar artists" or "fans also like." It's a cited, verifiable map of artistic lineage. The kind of thing a music journalist would spend weeks building by reading archives. Crate builds it in under a minute. + +Reference: Badillo-Goicoechea, E. (2025). "Modeling Artist Influence for Music Selection and Recommendation: A Purely Network-Based Approach." *Harvard Data Science Review*, 7(4). [hdsr.mitpress.mit.edu/pub/t4txmd81](https://hdsr.mitpress.mit.edu/pub/t4txmd81/release/2) + +--- + +## What you can do with it + +### If you're a radio DJ + +Type `/show-prep HYFIN` and paste your setlist. Crate researches every track — who produced it, where it was recorded, what it samples, why it matters right now — and generates a complete show prep package. Talk breaks in three lengths (10-second quick hit, 30-second standard, 2-minute deep dive). Social copy for Instagram, X, and Bluesky. Local Milwaukee events from Ticketmaster. Interview prep if you have a guest. + +Before Crate, this took me 2-3 hours of Googling, cross-referencing, and writing. Now it takes about 90 seconds. + +### If you're a music producer or crate digger + +Type `/influence Madlib` and watch the agent trace connections across decades. It checks 26 music publications for co-mentions (when two artists appear in the same review, that's an influence signal), pulls similar artists from Last.fm, verifies collaboration credits on MusicBrainz, and builds a weighted network graph. Each connection has a strength score, context explaining why the link exists, and source citations you can verify. + +Or just ask it: "What Ethiopian jazz records from the 1970s influenced UK broken beat producers in the 2000s?" It'll go find out. + +### If you're a music journalist or researcher + +Ask questions you'd normally spend a day on. "How many times has Pitchfork mentioned J Dilla in reviews of other artists?" The agent searches across publications, extracts mentions, and gives you a cited answer. Then publish your findings to Telegraph with `/publish telegraph` — one command, instant web article with a shareable URL. + +### If you're a casual listener who wants to go deeper + +This is the "Spotify for the curious" use case. You heard a song you liked. Instead of just hitting "Radio" and getting algorithm suggestions, ask Crate why that song exists. Who made it, what tradition it comes from, what else you should hear if you liked it, and why. Every recommendation comes with context, not just a playlist. + +### If you're running a radio station + +`/news hyfin 5` generates a daily music news segment with 5 stories, researched from RSS feeds and web sources, formatted for your specific station's voice. HYFIN gets culturally sharp hip-hop angles. 88Nine gets community-forward discovery stories. Rhythm Lab gets global-beats crate-digging finds. Same tool, different voice. + +--- + +## The 20+ sources + +This is what makes Crate different from a chatbot that just talks about music. The agent doesn't guess. It queries real databases: + +**For metadata:** Discogs (credits, pressings, labels), MusicBrainz (canonical IDs, relationships), Last.fm (tags, similarity scores) + +**For context:** Genius (lyrics, annotations, artist commentary), Wikipedia (bios, career timelines), 26 music publications (reviews, co-mentions for influence mapping) + +**For audio and images:** Spotify (artwork, audio features), YouTube (playback, live performances), Bandcamp (independent releases), iTunes (high-res artwork), fanart.tv (HD backgrounds, logos) + +**For events:** Ticketmaster (concerts, tours), Setlist.fm (historical setlists) + +**For radio:** Radio Browser (30,000+ live stations worldwide with ICY metadata) + +Here's how keys work: You need to bring your own AI key — either an Anthropic API key (for Claude directly) or an OpenRouter key (which gives you access to Claude, GPT-4, Gemini, and others through one API). This is the brain of the agent, and it runs on your account so you control costs and model selection. + +For the data sources, most work out of the box. Discogs, Last.fm, Ticketmaster, Spotify, Exa, Tavily, and fanart.tv all have embedded keys that every user shares — you don't need to configure anything. For Genius, Tumblr publishing, and a few others, you can optionally add your own keys in Settings for higher rate limits or additional features. + +--- + +## What's next + +I'm still building. The influence graph cache (backed by Convex) is getting smarter — it remembers connections the agent has already discovered so repeat queries are instant instead of requiring fresh research. The image pipeline now pulls from Spotify and fanart.tv for HD artist photos. There's a Gemini-powered infographic generator in the plan for visual influence maps. + +But the core idea stays the same: music is a network, not a list. Every artist is connected to every other artist through influence, collaboration, sampling, geography, and shared history. Crate is the tool that makes those connections visible, verifiable, and useful. + +If you want to try it: [digcrate.app](https://digcrate.app) + +If you want to see the code: [github.com/tmoody1973/crate-web](https://github.com/tmoody1973/crate-web) + +If you want the CLI: [crate-cli.dev](https://crate-cli.dev) + +--- + +*Built in Milwaukee by Tarik Moody. Powered by Claude, Convex, Vercel, and 20+ music databases. The records in the header image are real — that's my local shop.* diff --git a/docs/developer-diary/2026-04-07-the-wiki-that-remembers.md b/docs/developer-diary/2026-04-07-the-wiki-that-remembers.md new file mode 100644 index 0000000..08ab7a7 --- /dev/null +++ b/docs/developer-diary/2026-04-07-the-wiki-that-remembers.md @@ -0,0 +1,131 @@ +# Developer Diary: The Wiki That Remembers + +**Date:** April 7, 2026 +**Entry:** #1 +**Working on:** Crate Music Wiki design — applying Karpathy's LLM Wiki pattern to music intelligence +**Related:** Design doc at `~/.gstack/projects/tmoody1973-crate-web/tarikmoody-main-design-20260407-223755.md` + +--- + +## What Happened + +Andrej Karpathy dropped a gist called "LLM Wiki" and the internet lost its mind. 16 million views. Everyone rushing to build personal knowledge bases with Claude Code. And I'm sitting here looking at Crate thinking... we already have 18 ingestion pipelines. We already synthesize from Spotify, WhoSampled, Bandcamp, YouTube, radio metadata. We already build influence chains and cross-reference sources. + +We just throw it all away after every chat session. + +That's the realization that hit tonight. Crate isn't missing a wiki feature. Crate is missing a *memory*. + +## Technical Observations + +### The "Exhaust" Pattern + +The most interesting architectural insight from tonight: **the chat is the editor, the wiki is the exhaust.** + +Every tool call in `src/app/api/chat/route.ts` already returns structured artist data. Spotify gives us genres, popularity, related artists. WhoSampled gives us sample relationships. Bandcamp gives us tags and discography. The data is right there, flowing through the system, doing its job in the conversation, and then... gone. + +The wiki layer is a post-tool-call hook. A background Convex mutation that captures what the tools already produce. No new UI for writing. No editor. No markdown files. Just a persistence layer that catches what Crate already generates. + +``` +User asks about Khruangbin + -> spotify-connected returns genres, popularity, related artists + -> whosampled returns sample relationships + -> influence-cache returns influence chain + -> [NEW] post-hook: upsert wiki page for Khruangbin with all of the above + -> user gets their answer + -> wiki gets smarter (silently, in the background) +``` + +The marginal cost of adding persistence to an existing intelligence pipeline is dramatically lower than building a wiki from scratch. Everyone on Twitter is writing CLAUDE.md files to configure their generic knowledge bases. We'd just... turn on a hook. + +### Convex Data Model Gotcha + +The spec reviewer caught something I would have shipped and regretted: unbounded arrays in Convex documents. My first instinct was a single `wikiIndex` doc per user with an `entries` array listing all their wiki pages. Fine at 50 pages. Hits the 1MB document limit at a few hundred. + +The fix is obvious in hindsight — one `wikiIndexEntries` document per wiki page, indexed by `userId`. Classic denormalization. But I'd absolutely have built the array version first and hit the wall a month later. + +### Synthesis on Write, Not Read + +First instinct: synthesize (merge, deduplicate, flag contradictions) when the user views a wiki page. Sounds elegant. But it means every page view triggers an LLM call. That's slow and expensive. + +Better: synthesize on write, debounced. When new source data arrives from a tool call, wait 5 seconds (in case multiple tools fire in sequence), then run Haiku to merge the new data with existing sections. Cache the result. Page views are always fast reads. + +This is the kind of decision that feels boring but determines whether the product feels snappy or sluggish. + +## Personal Insights + +### The "Export My Brain" Moment + +The independent reviewer (Claude subagent, running cold with no conversation context) read my session transcript and came back with this: + +> "Knowledge lives in my head and scattered across tools" — This isn't a software problem. It's 20 years of institutional knowledge trapped in a human brain with no succession plan. + +That reframe landed. I've been thinking about Crate as a research tool. But the wiki makes it something else: a mechanism for making tacit expertise explicit. When Tarik researches Khruangbin and the wiki captures the influence chain back through Thai funk and Lee Perry, that's not just "saving a query result." That's encoding knowledge that previously existed only in one person's head. + +The wiki is how taste becomes transferable. + +### Sources Disagree, and That's the Feature + +Music metadata is a mess. Spotify says Khruangbin is "psychedelic" and "funk." Bandcamp tags include "surf rock." WhoSampled connects them to dub. AllMusic might say something else entirely. + +Most systems try to reconcile this. Pick one source of truth. Normalize. + +The wiki does the opposite: it surfaces the contradictions. "Spotify lists dub as a genre; WhoSampled and Bandcamp do not." That's not a bug. That's the interesting part. Music genre is contested, contextual, evolving. Showing where sources disagree is more honest and more useful than pretending there's one right answer. + +This feels like a principle worth remembering: **contradictions in your data are often more valuable than consensus.** + +### The Public Wiki as Growth Engine + +Tarik's instinct to add "option to make it public" was the best product insight of the session. It transforms the wiki from a personal tool into a content platform. Every public artist page is a landing page. Every influence chain is SEO bait. Someone Googling "who influenced Khruangbin" could land on a Crate wiki page with cited sources, cross-references, and a visual influence graph. + +The growth loop writes itself: research compounds your wiki, your wiki attracts visitors, visitors become users, their research compounds their wikis, their wikis attract more visitors. + +## Future Considerations + +### Wiki-Aware Chat + +Open question from the design doc that I keep thinking about: when a user asks about an artist that already has a wiki page, should Crate read the wiki first? + +The argument for: it's faster, it's richer (accumulated context from multiple past sessions), and it makes the product feel like it actually *knows* you. + +The argument against: the wiki might be stale. Music data changes. New albums drop. Collaborations happen. + +Best answer is probably both: read wiki first for instant context, offer a "refresh from sources" button to pull fresh data and update the wiki. The wiki becomes a cache with intelligence, not a static archive. + +### The Graph Visualization Rabbit Hole + +Phase 2 includes a visual node graph for influence chains. D3.js force-directed? Cytoscape.js? Custom SVG? + +My gut says start simpler than you think. The mockup (V2-C) shows a clean node-and-arrow diagram, not a full interactive force graph. Ship the simple version first. If people love it, make it interactive later. If they don't engage with the visual, you saved yourself weeks of D3 debugging. + +### Entity Types Beyond Artists + +The schema supports `"artist" | "genre" | "era" | "label" | "producer"` but Phase 1 is artist-only. The interesting question is what happens when genre pages and era pages exist. An "era" page for "Thai Funk in the 1960s-70s" that cross-references Lee Perry, The Meters, Khruangbin, and links to a genre page for "psychedelic dub"... that starts to look like a real music encyclopedia. + +But that's Phase 2+. Don't get seduced by the vision before proving the foundation. + +## Shower Thoughts + +The whole Karpathy LLM Wiki thing is basically what Wikipedia would be if you had a dedicated AI librarian who read every source, updated every cross-reference, and flagged every contradiction... but just for your interests. A personal Wikipedia that grows from what you actually care about, not what volunteer editors decided was notable. + +For music specifically, this fills a gap that's genuinely weird. We have: +- **Streaming services** that know what you listen to but not why +- **Wikipedia** that has facts but no taste +- **Rate Your Music** that has opinions but no connections +- **WhoSampled** that has samples but not broader influence +- **AllMusic** that has editorial reviews but is stuck in 2010 + +Nobody has the *connected intelligence layer*. The thing that says "you like Khruangbin, and here's the entire influence tree that explains why, with sources from five different databases, and by the way Spotify and Bandcamp disagree about whether they're surf rock." + +Crate with a wiki becomes that thing. + +## Code Quality Opinion + +Looking at the existing codebase — 18 web tools in `src/lib/web-tools/`, each returning structured data — the infrastructure for the wiki layer is genuinely already there. The `chat/route.ts` is the natural place to add post-tool-call hooks. The Convex patterns from `tinydesk` companions and Deep Cuts sharing give us the template for public/private content. + +The cleanest path forward is not a rewrite. It's an additive layer. A few Convex tables, a background mutation hook, one new route, and a Haiku synthesis step. The hardest part won't be the code. It'll be tuning the LLM synthesis to produce wiki pages that feel curated rather than auto-generated. + +That's the real craft challenge: making the exhaust beautiful. + +--- + +*Next entry: after running the Khruangbin -> Meters -> Lee Perry research session to test the thesis.* diff --git a/docs/guides/google-oauth-setup.md b/docs/guides/google-oauth-setup.md new file mode 100644 index 0000000..f44c773 --- /dev/null +++ b/docs/guides/google-oauth-setup.md @@ -0,0 +1,206 @@ +# Setting Up Google OAuth for Crate (via Auth0 Token Vault) + +Crate uses Auth0's `google-oauth2` connection to let users save research to Google Docs. This guide walks through configuring it end-to-end. + +> **Sources**: [Auth0 — Call an External IdP API](https://auth0.com/docs/authenticate/identity-providers/calling-an-external-idp-api), [Auth0 — Token Vault](https://auth0.com/ai/docs/intro/token-vault), [Google — Choose Docs API Scopes](https://developers.google.com/workspace/docs/api/auth), [Google — Drive API Scopes](https://developers.google.com/workspace/drive/api/guides/api-specific-auth), [Google — Configure OAuth Consent Screen](https://developers.google.com/workspace/guides/configure-oauth-consent) + +--- + +## Option A: Use Auth0's Built-in Dev Keys (Fastest) + +Auth0 provides default Google OAuth credentials out of the box. This works immediately with no Google Cloud Console setup, but shows "Auth0" as the app name in the consent screen (not "Crate"). + +### Steps + +1. Go to [Auth0 Dashboard](https://manage.auth0.com) → **Authentication** → **Social** +2. Find **Google / Gmail** and click it +3. Under **Credentials**, check if "Use Auth0 dev keys" is toggled ON +4. If yes — you're done. Any Google account can connect. Skip to [Verify It Works](#verify-it-works) +5. If no — you're using custom credentials. See Option B below + +### Trade-offs +- **Pro**: Zero setup, works for any Google account immediately +- **Con**: Consent screen says "Auth0" not "Crate", looks less professional +- **Con**: Limited to Auth0's allowed scopes — confirm `documents` and `drive.file` are included + +--- + +## Option B: Use Custom Google Cloud Console Credentials + +For a branded consent screen ("Crate wants to access your Google Docs") and full scope control. + +### Step 1: Create a Google Cloud Project + +1. Go to [Google Cloud Console](https://console.cloud.google.com/) +2. Click the project dropdown (top bar) → **New Project** +3. Name it `crate-web` → **Create** +4. Select the new project from the dropdown + +### Step 2: Enable APIs + +1. Go to **APIs & Services** → **Library** +2. Search and enable: + - **Google Docs API** + - **Google Drive API** + +### Step 3: Configure OAuth Consent Screen + +1. Go to **APIs & Services** → **OAuth consent screen** +2. Choose **External** user type → **Create** +3. Fill in: + - **App name**: `Crate` + - **User support email**: your email + - **App logo**: upload Crate logo (optional) + - **Developer contact email**: your email +4. Click **Save and Continue** +5. **Scopes** page → **Add or Remove Scopes** → add: + - `https://www.googleapis.com/auth/documents` + - `https://www.googleapis.com/auth/drive.file` + - `openid` + - `email` + - `profile` +6. Click **Save and Continue** +7. **Test users** page → this is critical, see Step 5 below + +### Step 4: Create OAuth Client ID + +1. Go to **APIs & Services** → **Credentials** +2. Click **+ Create Credentials** → **OAuth client ID** +3. Application type: **Web application** +4. Name: `Crate Auth0` +5. **Authorized redirect URIs** — add: + ``` + https://YOUR_AUTH0_DOMAIN/login/callback + ``` + Your Auth0 domain includes the region (e.g., `dev-abc123.us.auth0.com`), so the full redirect URI looks like: + ``` + https://dev-abc123.us.auth0.com/login/callback + ``` + If you've configured a custom domain in Auth0, use that instead (e.g., `https://auth.digcrate.app/login/callback`) +6. Click **Create** +7. Copy the **Client ID** and **Client Secret** + +### Step 5: Handle Testing Mode (CRITICAL) + +New Google Cloud apps start in **Testing** mode. While in testing mode: +- Only whitelisted test users can complete the OAuth flow +- Other users see "Error 403: access_denied" +- Max 100 test users + +**For the hackathon demo**, add your test users: + +1. Go to **APIs & Services** → **OAuth consent screen** +2. Scroll to **Test users** → **+ Add Users** +3. Add the Google email(s) you'll use in the demo +4. Click **Save** + +**To remove the restriction** (post-hackathon), publish the app: + +1. Go to **OAuth consent screen** → click **Publish App** +2. Google may require a verification review if you're requesting sensitive scopes +3. **Scope classifications** (per [Google's docs](https://developers.google.com/workspace/docs/api/auth)): + - `drive.file` — **Non-sensitive** (recommended, no verification needed) + - `documents` — **Sensitive** (requires Google review, can take days/weeks) +4. For hackathon: just add test users, don't try to publish + +### Step 6: Configure Auth0 + +1. Go to [Auth0 Dashboard](https://manage.auth0.com) → **Authentication** → **Social** +2. Find **Google / Gmail** → click it +3. Toggle OFF "Use Auth0 dev keys" +4. Paste your **Client ID** and **Client Secret** from Step 4 +5. Under **Permissions**, ensure these scopes are listed: + - `email` + - `profile` + - `https://www.googleapis.com/auth/documents` + - `https://www.googleapis.com/auth/drive.file` +6. Click **Save Changes** + +--- + +## Verify It Works + +### Quick test via browser + +1. Go to `https://digcrate.app` and sign in +2. Open **Settings** → **Connected Services** +3. Click **Connect** next to Google +4. Auth0 should redirect to Google's consent screen +5. Authorize → you should land back at Crate with "Connected" badge + +### Debug endpoint + +Hit the debug endpoint to inspect the Auth0 identity: + +``` +https://digcrate.app/api/auth0/debug +``` + +Look for: +```json +{ + "userIdentities": [ + { + "provider": "google-oauth2", + "connection": "google-oauth2", + "hasAccessToken": true, + "hasRefreshToken": false + } + ] +} +``` + +If `hasAccessToken` is `true`, Token Vault is working. + +### Test the tool + +In Crate's chat, try: +``` +Save this to Google Docs: "Test document from Crate" +``` + +The agent should call `save_to_google_doc`, create a new Google Doc, and return a link. + +--- + +## Troubleshooting + +| Symptom | Cause | Fix | +|---------|-------|-----| +| "Error 403: access_denied" on Google consent | App in testing mode, user not whitelisted | Add user's email to test users in GCP console | +| Consent screen shows "Auth0" not "Crate" | Using Auth0 dev keys | Switch to custom credentials (Option B) | +| `hasAccessToken: false` in debug | Token exchange failed or scopes not granted | Re-check scopes in Auth0 dashboard match GCP | +| "Google Docs API error: 403" | Google Docs API not enabled | Enable it in GCP → APIs & Services → Library | +| "Google Docs API error: 401" | Token expired and no refresh token | User needs to disconnect and reconnect Google | +| Redirect URI mismatch error | Auth0 callback URL not in GCP allowed redirects | Add `https://{YOUR_AUTH0_DOMAIN}/login/callback` to GCP authorized redirect URIs | +| `getTokenVaultToken` returns `null` despite being connected | Management API missing required scopes | In Auth0 Dashboard, grant `read:users` + `read:user_idp_tokens` to your M2M app | + +--- + +## How Token Retrieval Works (Auth0 Internals) + +Crate uses Auth0's **Management API** to retrieve IdP access tokens. Per [Auth0's docs](https://auth0.com/docs/authenticate/identity-providers/calling-an-external-idp-api): + +1. Crate's server calls `POST https://{domain}/oauth/token` with `client_credentials` grant to get a Management API token +2. Then calls `GET https://{domain}/api/v2/users/{auth0UserId}?fields=identities` with that token +3. The response contains `user.identities[].access_token` — the Google OAuth token + +**Required Management API scopes** (configured in Auth0 Dashboard → Applications → APIs → Auth0 Management API → Machine to Machine Applications): +- `read:users` +- `read:user_idp_tokens` + +If these scopes aren't granted, `getTokenVaultToken("google")` will return `null` even after a successful OAuth connection. + +> **Note**: Auth0 also offers a newer [Token Exchange](https://auth0.com/ai/docs/intro/token-vault) approach (RFC 8693) as the recommended method for AI agents. Crate currently uses the Management API method, which works fine. A future upgrade could switch to Token Exchange for better token isolation. + +--- + +## Relevant Files + +| File | Purpose | +|------|---------| +| `src/lib/auth0-token-vault.ts` | Token retrieval — `getTokenVaultToken("google", auth0UserId)` | +| `src/lib/web-tools/google-docs.ts` | `save_to_google_doc` tool implementation | +| `src/app/api/auth0/connect/route.ts` | OAuth redirect (builds Auth0 authorize URL) | +| `src/app/api/auth0/callback/route.ts` | OAuth callback (exchanges code, stores cookies) | +| `src/app/api/auth0/debug/route.ts` | Debug endpoint (shows identity + token status) | diff --git a/docs/guides/posthog-analytics.md b/docs/guides/posthog-analytics.md new file mode 100644 index 0000000..e5fb6f0 --- /dev/null +++ b/docs/guides/posthog-analytics.md @@ -0,0 +1,248 @@ +# PostHog Analytics Guide for Crate + +How to understand what your users do, what they ignore, and what to build next. + +--- + +## Quick Start + +1. Go to **https://us.posthog.com** and sign in +2. You're in project **352161** (Crate) +3. Your pinned dashboard is **Daily Pulse** — check it every morning + +--- + +## Your Dashboards + +You have 5 dashboards. Each one answers a different question about Crate. + +### 1. Daily Pulse (check every morning) +**Link:** https://us.posthog.com/project/352161/dashboard/1441667 + +**What it tells you:** Is anyone using Crate today? + +| Insight | What to look for | +|---------|-----------------| +| Messages sent (7d) | Trend line going up = growing. Flat = stalled. Dips on weekends are normal. | +| Active users (7d) | Unique people. More important than total messages. 1 person sending 50 messages isn't growth. | +| Research query duration | If avg goes above 30 seconds, something is slow. Under 10 seconds is great. | +| Research queries by model | Shows Sonnet vs Haiku split. If most queries hit Sonnet, your slash commands are working. | + +**When to worry:** Active users drops to zero for 2+ days. Query duration spikes above 60 seconds. + +--- + +### 2. Agent Connect +**Link:** https://us.posthog.com/project/352161/dashboard/1441668 + +**What it tells you:** Is the connected services feature (Spotify, Tumblr, Slack, Google Docs) being used? + +| Insight | What to look for | +|---------|-----------------| +| Services connected (by service) | Which services people connect most. Spotify first? Or do they skip straight to Tumblr? | +| Action buttons clicked (by type) | Copy vs Slack vs Docs vs Tumblr. If "Copy" dominates and Slack/Docs are zero, people aren't using Agent Connect. | +| Service connect to action funnel | Of people who connect a service, how many actually use the action buttons? Low conversion = the feature is confusing or not useful. | + +**The key metric:** % of users who connect at least one service AND use an action button. That's the Agent Connect adoption rate. + +**When to worry:** Everyone connects Spotify but nobody clicks the Tumblr/Slack/Docs buttons. That means the connected services work for reading (library) but not for writing (publishing/sending). + +--- + +### 3. Retention +**Link:** https://us.posthog.com/project/352161/dashboard/1441669 + +**What it tells you:** Do people come back? + +| Insight | What to look for | +|---------|-----------------| +| Weekly active users (12 weeks) | The most important chart. Is the line going up, flat, or down? | + +**How to read retention:** +- **D1 retention** (come back next day): 30%+ is good for a tool. 50%+ is exceptional. +- **D7 retention** (come back within a week): 20%+ means you have a habit. +- **D30 retention** (come back within a month): 10%+ means you have a product. + +**When to worry:** WAU is flat or declining. That means you're acquiring users but losing them. Fix retention before spending time on acquisition. + +--- + +### 4. Conversion Funnel +**Link:** https://us.posthog.com/project/352161/dashboard/1441670 + +**What it tells you:** Are free users becoming paying users? + +| Insight | What to look for | +|---------|-----------------| +| Signup to first query funnel | How many signups actually send a message? If 50% sign up but never ask a question, the onboarding is broken. | +| Research queries by tier | Free vs Pro usage. If free users query heavily, they're getting value. That's your upgrade audience. | +| Quota exceeded events | Free users hitting the 10-query limit. Every one of these is a potential upgrade. If this number is zero, free users aren't engaged enough. | +| BYOK vs platform key | How many bring their own API key vs use yours. BYOK users are power users who bypass the quota entirely. | +| Subscriptions activated / canceled | Revenue pulse. Are more people subscribing than canceling? | + +**The key metric:** Quota exceeded count. This directly measures upgrade pressure. Zero = no one cares enough to hit the limit. 10+ per week = time to optimize the upgrade prompt. + +**When to worry:** Lots of signups but the funnel drops to near-zero at "first query." That means the onboarding or empty state isn't compelling enough. + +--- + +### 5. Feature Usage +**Link:** https://us.posthog.com/project/352161/dashboard/1441671 + +**What it tells you:** Which features should you invest in? + +| Insight | What to look for | +|---------|-----------------| +| Top slash commands | Which commands people actually use. If `/influence` is 60% of all commands, that's your core feature. If `/tumblr` is 0%, nobody knows about it. | +| Artifact types opened | Which OpenUI components get viewed. InfluenceChain, ArtistProfile, ShowPrepPackage, TumblrFeed, etc. This tells you what research output people find valuable. | +| Onboarding completion funnel | Where new users drop off. Started → Step 1 → Step 2 → Completed. Big drop at step 2? That step is confusing. | + +**The key metric:** Top 3 slash commands. These are your product. Everything else is a feature. + +**When to worry:** One command dominates (e.g., 90% is `/influence`) and everything else is unused. Either double down on that one thing, or figure out why people don't discover the rest. + +--- + +## How to Read PostHog (for Non-Technical People) + +### Events Tab +Go to **Events** in the left sidebar. This shows a live stream of every event happening on Crate. You can filter by event name (e.g., `message_sent`) or by person (search by email). + +This is useful for debugging: "Did that user's event fire?" + +### Persons Tab +Go to **Persons** in the left sidebar. Search by email to see a specific user's entire history: every page they visited, every event they triggered, in chronological order. + +This is useful for understanding: "What did this user actually do?" + +### Insights Tab +Go to **Insights** in the left sidebar. This shows all saved insights. Filter by tag (`crate`) to see only yours. + +Each insight can be edited. Click into it, change the date range, add filters, and save. + +### Session Recordings (optional) +If you enable session recordings in PostHog settings, you can watch actual users navigate the app. This is the most powerful feature for understanding "where do people get confused." + +To enable: Settings → Session Recording → Turn on. Note: this increases PostHog usage and costs. + +--- + +## Events Reference + +Every event Crate sends to PostHog, what triggers it, and what properties it includes. + +### Server-side events (API routes) + +| Event | Trigger | Properties | +|-------|---------|------------| +| `message_sent` | Every chat message | `is_slash_command`, `slash_command`, `is_chat_tier`, `tier` | +| `research_query_completed` | After agent finishes | `model`, `is_research`, `slash_command`, `tier`, `has_byok`, `duration_ms` | +| `quota_exceeded` | Free user hits limit | `tier`, `used`, `limit` | +| `service_connected` | OAuth callback success | `service` (spotify/tumblr/slack/google) | +| `checkout_started` | User clicks upgrade | `plan`, `current_tier` | +| `subscription_activated` | Stripe confirms payment | `plan`, `interval` | +| `subscription_canceled` | User cancels in portal | `plan` | +| `user_signed_up` | Clerk webhook | `email`, `name` | + +### Client-side events (React components) + +| Event | Trigger | Properties | +|-------|---------|------------| +| `action_button_clicked` | Copy/Slack/Docs/Tumblr/Email/Share button | `action` | +| `artifact_opened` | Research opens in panel | `type` (InfluenceChain, ArtistProfile, etc.) | +| `service_connect_clicked` | Click Connect in Settings | `service` | +| `service_disconnected` | Click Disconnect in Settings | `service` | +| `onboarding_started` | Wizard opens | — | +| `onboarding_step_completed` | Next step in wizard | `step` | +| `onboarding_completed` | Wizard finished | `has_api_key`, `step_count` | +| `cta_clicked` | Landing page buttons | `location`, `label` | +| `track_played` | Play button in player | `title`, `artist`, `source` | +| `api_key_saved` | Save API key in Settings | `service` | + +### Auto-captured by PostHog + +| Event | What it tracks | +|-------|---------------| +| `$pageview` | Every page navigation (automatic) | +| `$pageleave` | When user leaves a page | +| `$ai_generation` | Every LLM API call (tokens, model, latency) via @posthog/ai | + +--- + +## Weekly Routine + +**Every Monday morning (5 minutes):** + +1. Open **Daily Pulse** dashboard +2. Check: Did active users go up or down this week? +3. Check: Any slash commands getting more popular? +4. Open **Conversion Funnel** +5. Check: How many quota_exceeded events? (upgrade pressure) +6. Check: Any new subscriptions? + +**Every month (15 minutes):** + +1. Open **Retention** dashboard +2. Check: Is D7 retention improving? +3. Open **Feature Usage** +4. Check: Top 3 commands. Are they the same as last month? +5. Check: Onboarding funnel. Where do people drop off? +6. Open **Agent Connect** +7. Check: Are more people connecting services? Which ones? + +--- + +## What to Do With the Data + +### "Nobody uses /tumblr" +- Is it in the command menu? (Check chat-panel.tsx SLASH_COMMANDS array) +- Do people know about it? (Add a tooltip or onboarding step) +- Is the output good? (Try it yourself, compare to /influence) + +### "Everyone uses Copy, nobody uses Slack/Docs" +- Copy is frictionless (no auth needed). Slack/Docs require connecting a service. +- Add a nudge: "Want to send this to Slack? Connect in Settings." +- Or: make the first action button always Copy, and show Slack/Docs only after connection. + +### "Lots of signups, few first queries" +- The empty state is the problem. What does a new user see? +- Add a pre-filled example: "Try this: /influence Flying Lotus" +- Or: auto-run a demo query on first visit. + +### "Free users keep hitting quota" +- This is good. It means they want more. +- Optimize the upgrade prompt: show it inline when quota is reached, not as a modal. +- Consider raising the free limit slightly (10 → 15) to reduce friction while keeping upgrade pressure. + +### "Query duration is increasing" +- Check which model is getting slower (model breakdown). +- Check if tool call count is increasing (agents doing more work per query). +- Consider caching common queries or influence chains. + +--- + +## Adding the Insights to Dashboards + +The insights are created but need to be added to their dashboards. For each insight: + +1. Go to **Insights** in the left sidebar +2. Filter by tag (e.g., `daily`, `agent-connect`, `conversion`, `features`, `retention`) +3. Click into each insight +4. Click the **three dots menu** (top right) +5. Select **Add to dashboard** +6. Pick the matching dashboard +7. Save + +Once added, the dashboard shows all its insights together on one page. + +--- + +## Useful PostHog Links + +| Page | URL | +|------|-----| +| All dashboards | https://us.posthog.com/project/352161/dashboards | +| All insights | https://us.posthog.com/project/352161/insights | +| Live events | https://us.posthog.com/project/352161/events | +| Persons | https://us.posthog.com/project/352161/persons | +| Settings | https://us.posthog.com/project/352161/settings | diff --git a/docs/guides/stripe-setup.md b/docs/guides/stripe-setup.md new file mode 100644 index 0000000..b272adc --- /dev/null +++ b/docs/guides/stripe-setup.md @@ -0,0 +1,255 @@ +# Stripe Billing Setup Guide + +Step-by-step guide to set up Stripe billing for Crate, from test mode through production. + +--- + +## Crate's Billing Architecture + +Crate uses Stripe for subscription billing with three tiers: + +| Plan | Price | Features | +|------|-------|----------| +| **Free** | $0 | 10 agent queries/month, 5 sessions, 3 custom skills, connected services | +| **Pro** | $15/mo | 50 queries, unlimited sessions, 20 skills, memory, publishing, influence caching | +| **Team** | $25/mo | 200 pooled queries, admin dashboard, shared org keys | + +BYOK (Bring Your Own Key) users get unlimited queries regardless of plan. + +### How it works + +1. User clicks "Upgrade" on the pricing page +2. Crate creates a Stripe Checkout session (`/api/stripe/checkout`) +3. User completes payment on Stripe's hosted checkout page +4. Stripe sends a webhook to `/api/webhooks/stripe` +5. Webhook handler creates/updates a `subscriptions` record in Convex +6. The chat route checks the subscription to determine plan limits and feature access + +### Key files + +| File | Purpose | +|------|---------| +| `src/app/api/stripe/checkout/route.ts` | Creates Checkout sessions for new subscriptions | +| `src/app/api/stripe/portal/route.ts` | Opens the Stripe Billing Portal for managing subscriptions | +| `src/app/api/webhooks/stripe/route.ts` | Handles webhook events (subscription created, updated, canceled, payment failed) | +| `src/lib/plans.ts` | Plan definitions, limits, rate limiting, admin/beta access | +| `convex/subscriptions.ts` | Convex table for subscription state | +| `src/components/settings/plan-section.tsx` | Settings UI showing current plan and upgrade/manage buttons | +| `src/components/landing/pricing.tsx` | Public pricing page | + +--- + +## Part 1: Stripe Account Setup + +1. Create a Stripe account at **https://dashboard.stripe.com/register** (if you don't have one) +2. Complete the business profile (name, address, bank account for payouts) +3. You start in **Test Mode** by default (toggle in the top right of the dashboard) + +--- + +## Part 2: Create Products and Prices (Test Mode) + +Stay in Test Mode for initial setup. + +### Create the Pro plan + +1. Go to **Products** → **Add Product** +2. Fill in: + - Name: `Crate Pro` + - Description: `50 research queries/month, publishing, memory, influence caching` +3. Add pricing: + - **Monthly**: $15.00 USD, Recurring, Monthly + - Click "Add another price" + - **Annual**: $144.00 USD ($12/mo), Recurring, Yearly +4. Save. Copy both price IDs (`price_test_...`) + +### Create the Team plan + +1. **Products** → **Add Product** +2. Fill in: + - Name: `Crate Team` + - Description: `200 pooled queries, admin dashboard, shared org keys` +3. Add pricing: + - **Monthly**: $25.00 USD, Recurring, Monthly +4. Save. Copy the price ID. + +--- + +## Part 3: Set Up Webhooks (Test Mode) + +1. Go to **Developers** → **Webhooks** +2. Click **Add endpoint** +3. Endpoint URL: `https://digcrate.app/api/webhooks/stripe` +4. Select events: + - `checkout.session.completed` + - `customer.subscription.created` + - `customer.subscription.updated` + - `customer.subscription.deleted` + - `invoice.payment_failed` + - `invoice.paid` +5. Click **Add endpoint** +6. Click on the endpoint, then **Reveal** the Signing Secret (`whsec_...`) +7. Copy the signing secret + +--- + +## Part 4: Configure Environment Variables + +Add these to your `.env.local` and Vercel environment variables: + +```bash +# Stripe keys (from Dashboard → Developers → API keys) +STRIPE_SECRET_KEY=sk_test_... +NEXT_PUBLIC_STRIPE_PUBLISHABLE_KEY=pk_test_... + +# Webhook signing secret (from the webhook endpoint you created) +STRIPE_WEBHOOK_SECRET=whsec_... + +# Price IDs (from the products you created) +STRIPE_PRO_MONTHLY_PRICE_ID=price_test_... +STRIPE_PRO_ANNUAL_PRICE_ID=price_test_... +STRIPE_TEAM_MONTHLY_PRICE_ID=price_test_... +``` + +--- + +## Part 5: Test the Full Flow + +### Test subscription checkout + +1. Go to digcrate.app → **Pricing** → click **Upgrade to Pro** +2. Use Stripe's test card numbers: + +| Scenario | Card Number | Expiry | CVC | +|----------|------------|--------|-----| +| Successful payment | `4242 4242 4242 4242` | Any future date | Any 3 digits | +| Card declined | `4000 0000 0000 0002` | Any future date | Any 3 digits | +| Requires authentication | `4000 0025 0000 3155` | Any future date | Any 3 digits | +| Insufficient funds | `4000 0000 0000 9995` | Any future date | Any 3 digits | + +3. After successful checkout, verify: + - Settings shows "Pro" plan badge + - 50 queries/month limit applies + - Publishing features are unlocked + - Memory features are available + +### Test billing portal + +1. Go to Settings → click **Manage Subscription** +2. Stripe's portal opens showing the current plan +3. Test: cancel subscription, verify plan reverts to Free +4. Test: re-subscribe + +### Test webhook handling + +1. In Stripe Dashboard → **Developers** → **Webhooks** → click your endpoint +2. Click **Send test webhook** → select `customer.subscription.updated` +3. Check Convex dashboard to verify the subscription record updated + +### Verify in Stripe Dashboard + +- **Customers** → your test customer should appear with email +- **Subscriptions** → active subscription with the correct plan +- **Events** → webhook events should show as delivered + +--- + +## Part 6: Move to Production (Live Mode) + +When you're ready to accept real payments: + +### Step 1: Complete Stripe verification + +- Stripe Dashboard → **Settings** → **Business details** +- Complete all required verification (business type, address, bank account, tax ID) +- This may take 1-2 business days for Stripe to verify + +### Step 2: Create live products and prices + +1. Toggle to **Live Mode** in the Stripe Dashboard (top right switch) +2. Repeat Part 2: create the same products and prices in live mode + - Crate Pro: $15/mo monthly, $144/yr annual + - Crate Team: $25/mo monthly +3. Copy the new live price IDs (`price_live_...`) + +Note: Test mode products and prices do NOT carry over to live mode. You must recreate them. + +### Step 3: Create live webhook + +1. In Live Mode → **Developers** → **Webhooks** → **Add endpoint** +2. URL: `https://digcrate.app/api/webhooks/stripe` +3. Same events as test mode +4. Copy the new live signing secret + +### Step 4: Get live API keys + +1. **Developers** → **API keys** +2. Copy the live Publishable key (`pk_live_...`) and Secret key (`sk_live_...`) + +### Step 5: Update Vercel environment variables + +| Variable | Test Value | Live Value | +|----------|-----------|------------| +| `STRIPE_SECRET_KEY` | `sk_test_...` | `sk_live_...` | +| `NEXT_PUBLIC_STRIPE_PUBLISHABLE_KEY` | `pk_test_...` | `pk_live_...` | +| `STRIPE_WEBHOOK_SECRET` | test `whsec_...` | live `whsec_...` | +| `STRIPE_PRO_MONTHLY_PRICE_ID` | `price_test_...` | `price_live_...` | +| `STRIPE_PRO_ANNUAL_PRICE_ID` | `price_test_...` | `price_live_...` | +| `STRIPE_TEAM_MONTHLY_PRICE_ID` | `price_test_...` | `price_live_...` | + +### Step 6: Redeploy + +Push to main or trigger a manual Vercel deploy. The app will now process real payments. + +### Step 7: Verify with a real card + +1. Subscribe with a real credit card ($15 charge) +2. Verify the subscription is active +3. Cancel and confirm the refund processes in Stripe Dashboard +4. Check that the webhook updates the Convex subscription record + +--- + +## Gotchas + +1. **Test vs Live mode is global in the dashboard.** Make sure you're looking at the right mode when copying keys and price IDs. The toggle is in the top right corner. + +2. **Webhooks are mode-specific.** Test mode webhooks only fire for test transactions. You need separate webhook endpoints for test and live mode. The URL can be the same, but the signing secrets are different. + +3. **Price IDs don't transfer.** `price_test_...` IDs are invalid in live mode. You must create new products/prices in live mode and get new `price_live_...` IDs. + +4. **Past-due grace period.** Crate gives a 3-day grace period for past-due subscriptions (`PAST_DUE_GRACE_MS` in `src/lib/plans.ts`). After that, the user reverts to Free. + +5. **Admin and beta bypass.** Users in `ADMIN_EMAILS` env var bypass all billing. Users with `BETA_DOMAINS` email domains get Pro access for free. These are checked in `src/lib/plans.ts`. + +6. **Stripe Customer Portal.** The portal URL is generated dynamically via `/api/stripe/portal`. Make sure the Customer Portal is configured in Stripe Dashboard → **Settings** → **Billing** → **Customer portal** (enable it and customize the branding). + +7. **Convex subscription schema.** The `subscriptions` table in Convex stores: `userId`, `plan`, `status`, `stripeSubscriptionId`, `stripeCustomerId`, `currentPeriodStart`, `currentPeriodEnd`, `teamDomain`. The webhook handler creates/updates this record. + +--- + +## Quick Reference + +### Stripe Dashboard URLs + +| Page | URL | +|------|-----| +| API Keys | https://dashboard.stripe.com/apikeys | +| Products | https://dashboard.stripe.com/products | +| Webhooks | https://dashboard.stripe.com/webhooks | +| Customers | https://dashboard.stripe.com/customers | +| Subscriptions | https://dashboard.stripe.com/subscriptions | +| Customer Portal Settings | https://dashboard.stripe.com/settings/billing/portal | +| Business Settings | https://dashboard.stripe.com/settings/account | + +### Test Card Numbers + +| Card | Number | +|------|--------| +| Visa (success) | `4242 4242 4242 4242` | +| Visa (declined) | `4000 0000 0000 0002` | +| 3D Secure | `4000 0025 0000 3155` | +| Insufficient funds | `4000 0000 0000 9995` | +| Expired card | `4000 0000 0000 0069` | + +Full list: https://docs.stripe.com/testing#cards diff --git a/docs/guides/stripe-test-to-live.md b/docs/guides/stripe-test-to-live.md new file mode 100644 index 0000000..70eb77e --- /dev/null +++ b/docs/guides/stripe-test-to-live.md @@ -0,0 +1,239 @@ +# Moving Stripe from Test to Live + +A plain-English guide to switching Crate's billing from Stripe test mode to accepting real payments. No code changes needed, just dashboard configuration and environment variable swaps. + +--- + +## Before You Start + +Make sure you've tested everything in test mode first: +- Subscribed with a test card (`4242 4242 4242 4242`) +- Verified the subscription shows up in Crate's Settings +- Tested the billing portal (Manage Subscription) +- Canceled and re-subscribed +- Checked the Stripe Dashboard for test transactions + +If you haven't done this yet, see `docs/guides/stripe-setup.md`. + +--- + +## Step 1: Complete Stripe Business Verification + +Stripe won't let you accept real payments until your business is verified. + +1. Go to **https://dashboard.stripe.com/settings/account** +2. Fill in everything Stripe asks for: + - Business type (individual, LLC, etc.) + - Legal business name + - Address + - Tax ID (EIN or SSN for sole proprietors) + - Bank account for payouts +3. Submit for review + +This can take **1-2 business days**. Stripe will email you when approved. You can continue setup while waiting, but live charges won't process until verification completes. + +--- + +## Step 2: Switch to Live Mode in Stripe Dashboard + +Look at the top-right corner of the Stripe Dashboard. You'll see a toggle that says **"Test mode"** with a switch. Click it to switch to **Live mode**. + +Everything in the dashboard now shows real data. Your test products, prices, and webhooks are still there in test mode. You're not losing anything. + +--- + +## Step 3: Create Live Products and Prices + +Test mode products don't carry over. You need to recreate them in live mode. + +### Create Crate Pro + +1. Go to **Products** (in the left sidebar) and click **Add product** +2. Fill in: + - **Name:** Crate Pro + - **Description:** 50 research queries/month, publishing, memory, influence caching, 20 custom skills +3. Under Pricing, add: + - Click **Add price** + - **Price:** $15.00 + - **Billing period:** Monthly + - Click **Add price** again + - **Price:** $144.00 ($12/month billed annually) + - **Billing period:** Yearly +4. Click **Save product** +5. Click into each price and copy the **Price ID** (starts with `price_`) + +Write these down: +- Pro Monthly: `price_________________` +- Pro Annual: `price_________________` + +### Create Crate Team + +1. **Products** then **Add product** +2. Fill in: + - **Name:** Crate Team + - **Description:** 200 pooled queries, admin dashboard, shared org keys +3. Under Pricing: + - **Price:** $25.00 + - **Billing period:** Monthly +4. Save and copy the Price ID + +Write it down: +- Team Monthly: `price_________________` + +--- + +## Step 4: Get Your Live API Keys + +1. Go to **Developers** (left sidebar) then **API keys** +2. You'll see two keys: + - **Publishable key** (starts with `pk_live_`) — safe to use in the browser + - **Secret key** (starts with `sk_live_`) — keep this private, never expose it +3. Copy both + +Write them down: +- Publishable: `pk_live_________________` +- Secret: `sk_live_________________` + +--- + +## Step 5: Create a Live Webhook + +Webhooks tell Crate when someone subscribes, cancels, or has a payment issue. + +1. Go to **Developers** then **Webhooks** +2. Click **Add endpoint** +3. Fill in: + - **Endpoint URL:** `https://digcrate.app/api/webhooks/stripe` + - **Description:** Crate subscription events +4. Under "Select events to listen to," click **Select events** and check: + - `checkout.session.completed` + - `customer.subscription.created` + - `customer.subscription.updated` + - `customer.subscription.deleted` + - `invoice.payment_failed` + - `invoice.paid` +5. Click **Add endpoint** +6. Click on the endpoint you just created +7. Under "Signing secret," click **Reveal** and copy the value (starts with `whsec_`) + +Write it down: +- Webhook secret: `whsec_________________` + +--- + +## Step 6: Enable the Customer Portal + +The customer portal is where users manage their subscription (cancel, update card, switch plans). + +1. Go to **Settings** (gear icon) then **Billing** then **Customer portal** +2. Make sure it's enabled +3. Under "Features," enable: + - Cancel subscriptions + - Update payment methods + - View invoices +4. Under "Branding," optionally add Crate's logo and colors +5. Save + +--- + +## Step 7: Update Vercel Environment Variables + +Go to **Vercel Dashboard** (vercel.com) then your Crate project then **Settings** then **Environment Variables**. + +Update these 6 variables (replace the old test values with the live values you copied above): + +| Variable Name | What to paste | +|--------------|---------------| +| `STRIPE_SECRET_KEY` | Your `sk_live_` key from Step 4 | +| `NEXT_PUBLIC_STRIPE_PUBLISHABLE_KEY` | Your `pk_live_` key from Step 4 | +| `STRIPE_WEBHOOK_SECRET` | Your `whsec_` from Step 5 | +| `STRIPE_PRO_MONTHLY_PRICE_ID` | Pro monthly `price_` from Step 3 | +| `STRIPE_PRO_ANNUAL_PRICE_ID` | Pro annual `price_` from Step 3 | +| `STRIPE_TEAM_MONTHLY_PRICE_ID` | Team monthly `price_` from Step 3 | + +After updating all 6, click **Save** for each one. + +**Do NOT change** any other environment variables. Clerk, Convex, Auth0, and everything else stays the same. + +--- + +## Step 8: Redeploy + +Vercel needs to pick up the new environment variables. + +Option A: Push any small code change to trigger a deploy. + +Option B: Go to **Vercel Dashboard** then **Deployments** then click the three dots on the latest deployment then **Redeploy**. + +Wait for the deployment to finish (usually 1-2 minutes). + +--- + +## Step 9: Test with a Real Card + +This is the moment of truth. + +1. Go to **digcrate.app/pricing** +2. Click **Upgrade to Pro** ($15/month) +3. Enter a **real credit card** (your own) +4. Complete the checkout + +After checkout: +- You should be redirected back to Crate +- Settings should show "Pro" plan +- The Stripe Dashboard (live mode) should show a new customer and subscription + +Then immediately: +1. Go to **Settings** then **Manage Subscription** +2. Cancel the subscription +3. Go to **Stripe Dashboard** then **Payments** then find the charge then **Refund** + +This costs you nothing (refund processes in 5-10 business days) and confirms the entire flow works with real money. + +--- + +## Step 10: Verify the Webhook + +The webhook is the most important piece. Without it, Stripe takes money but Crate doesn't know about the subscription. + +1. Go to **Stripe Dashboard** (live mode) then **Developers** then **Webhooks** +2. Click on your endpoint +3. Under "Recent deliveries," you should see events from your test purchase +4. Each event should show a green checkmark (200 response) + +If you see red X marks (failed deliveries): +- Check that the URL is exactly `https://digcrate.app/api/webhooks/stripe` +- Check that `STRIPE_WEBHOOK_SECRET` in Vercel matches the signing secret shown here +- Click "Resend" on a failed event to retry + +--- + +## You're Done + +Real payments are now live. When a user upgrades on digcrate.app, they'll be charged real money, and Crate will unlock their Pro or Team features automatically. + +**What to monitor in the first week:** +- Check Stripe Dashboard daily for failed payments or disputes +- Watch the Vercel logs for webhook errors +- Verify at least one real subscription activates correctly +- Check PostHog for `checkout_started` and `subscription_activated` events + +--- + +## Quick Reference: What Changed + +| Item | Test Mode | Live Mode | +|------|-----------|-----------| +| Secret key | `sk_test_...` | `sk_live_...` | +| Publishable key | `pk_test_...` | `pk_live_...` | +| Webhook secret | Test `whsec_...` | Live `whsec_...` | +| Price IDs | `price_test_...` | `price_live_...` | +| Card numbers | `4242 4242 4242 4242` | Real credit cards | +| Money | Fake | Real | + +## If Something Goes Wrong + +- **"No such price" error on checkout:** You're using a test price ID in live mode. Update the `STRIPE_PRO_MONTHLY_PRICE_ID` (and others) in Vercel to the live price IDs. +- **Webhook fails with 401:** The webhook secret doesn't match. Re-copy it from Stripe and update `STRIPE_WEBHOOK_SECRET` in Vercel. Redeploy. +- **User pays but doesn't get Pro:** The webhook isn't firing or failing. Check Stripe Dashboard for webhook delivery status. Check Vercel logs for errors in `/api/webhooks/stripe`. +- **"Your card was declined":** This is a real card issue, not a Crate issue. The customer needs to use a different card or contact their bank. diff --git a/docs/guides/tumblr-auth0-setup.md b/docs/guides/tumblr-auth0-setup.md new file mode 100644 index 0000000..ec884a3 --- /dev/null +++ b/docs/guides/tumblr-auth0-setup.md @@ -0,0 +1,170 @@ +# Tumblr + Auth0 Token Vault Setup Guide + +Step-by-step guide to connect Tumblr to Crate via Auth0 Token Vault. + +--- + +## Part 1: Register a Tumblr App + +1. Go to **https://www.tumblr.com/oauth/apps** (log in with your Tumblr account) + +2. Click **"Register application"** and fill out: + + | Field | Value | + |-------|-------| + | Application Name | `Crate` | + | Application Website | `https://digcrate.app` | + | Application Description | `AI music research assistant` | + | Administrative contact email | Your email | + | Default callback URL | `https://dev-YOUR-TENANT.us.auth0.com/login/callback` | + | **OAuth2 redirect URLs** | `https://dev-YOUR-TENANT.us.auth0.com/login/callback` | + + > **Important:** You MUST fill in the "OAuth2 redirect URLs" field. Without it, Tumblr won't activate OAuth 2.0 for your app and the Auth0 flow will fail. + +3. After registering, Tumblr shows you two values — copy both: + - **OAuth Consumer Key** (this is your Client ID) + - **Secret Key** (this is your Client Secret) + +--- + +## Part 2: Create the Connection in Auth0 + +1. Log into your **Auth0 Dashboard** at https://manage.auth0.com + +2. Navigate to: **Authentication** → **Social** + +3. Click **"Create Connection"** + +4. Find and select **"Tumblr"** from the list + +5. Fill in the connection form: + + | Field | Value | + |-------|-------| + | Client Id | Paste your **OAuth Consumer Key** from Step 1 | + | Client Secret | Paste your **Secret Key** from Step 1 | + +6. Click **Create** + +--- + +## Part 3: Configure Scopes + +Tumblr has 3 OAuth2 scopes. You need all of them: + +| Scope | What it does | Why Crate needs it | +|-------|-------------|-------------------| +| `basic` | Read user info, dashboard, likes, following | `/tumblr` command reads dashboard and likes | +| `write` | Create/edit posts, follow/unfollow, like/unlike | `post_to_tumblr` tool publishes research | +| `offline_access` | Get a refresh token (42-min token expiry otherwise) | **Required for Token Vault** to maintain access | + +### Where to set them: + +1. In Auth0 Dashboard, go to **Authentication** → **Social** → click on your **Tumblr** connection + +2. Under **Permissions**, check all three: + - [x] `basic` + - [x] `write` + - [x] `offline_access` + +3. Click **Save** + +> **Note:** `offline_access` MUST be enabled here in the Dashboard — it's not enough to only pass it in code. Auth0 requires it at the connection level when Token Vault is active. + +--- + +## Part 4: Enable Token Vault + +This is the key step that lets Crate retrieve the Tumblr OAuth token server-side. + +1. In the same Tumblr connection settings page, find the **"Purpose"** section + +2. Toggle ON: **"Connected Accounts for Token Vault"** + +3. Auth0 may prompt you to confirm `offline_access` is enabled — confirm it + +4. Click **Save** + +> **If you don't see the "Connected Accounts" toggle:** Token Vault may not be available for Tumblr on your Auth0 plan. Check that your tenant has the Token Vault feature enabled. Tumblr should work as an OAuth2 social connection — if the toggle doesn't appear, you may need to set up Tumblr as a Custom Social Connection instead. + +--- + +## Part 5: Enable for Your Application + +1. Still in the Tumblr connection settings, go to the **"Applications"** tab + +2. Find your Crate application and toggle it **ON** + +3. Click **Save** + +--- + +## Part 6: Verify the Setup + +### Quick check in Auth0: + +Your Tumblr connection settings should show: +- Client ID: ✅ (filled) +- Client Secret: ✅ (filled) +- Permissions: `basic`, `write`, `offline_access` all checked +- Purpose: "Connected Accounts for Token Vault" toggled ON +- Applications: Your Crate app enabled + +### Test in Crate: + +1. Deploy to Vercel (or run locally) +2. Go to **Settings** → **Connected Services** +3. Click **Connect** on Tumblr +4. You should see Tumblr's OAuth consent screen asking to authorize Crate +5. Authorize → redirected back to Crate with "Connected" badge +6. Type `/tumblr` in chat → should see your dashboard feed +7. Type `/tumblr #jazz` → should see tagged posts + +--- + +## How It Works (Under the Hood) + +``` +User clicks "Connect Tumblr" in Settings + → Crate redirects to /api/auth0/connect?service=tumblr + → Auth0 redirects to Tumblr's OAuth2 authorize endpoint + → User authorizes on Tumblr + → Tumblr redirects back to Auth0 with authorization code + → Auth0 exchanges code for access_token + refresh_token + → Auth0 stores tokens in Token Vault (user's identity) + → Auth0 redirects to Crate's callback + → Crate stores auth0_user_id_tumblr cookie + → Done! Token Vault now serves fresh tokens on demand +``` + +When you type `/tumblr`: +``` +Chat route reads auth0_user_id_tumblr cookie + → Passes to createTumblrConnectedTools(auth0UserId) + → Tool calls getTokenVaultToken("tumblr", auth0UserId) + → Auth0 Management API returns the stored access_token + → Tool calls Tumblr API with Bearer token + → Returns posts → rendered as TumblrFeed component +``` + +Token refresh (automatic): +``` +Tumblr access tokens expire every 42 minutes. +Auth0 Token Vault automatically uses the refresh_token +to get a new access_token when needed. +You don't need to handle this — it's transparent. +``` + +--- + +## Gotchas + +1. **OAuth2 redirect URL is mandatory** — If you skip the "OAuth2 redirect URLs" field in Tumblr's app registration, OAuth2 won't activate and you'll get errors. + +2. **Tokens expire in 42 minutes** — Without `offline_access` scope, there's no refresh token and the connection dies after 42 minutes. Always enable it. + +3. **MFA policy** — If your Auth0 tenant has MFA set to "Always", Token Vault token exchange may fail. Set MFA to "Never" or use Customize MFA Factors via Actions. + +4. **Tumblr Marketplace page typo** — Auth0's Tumblr Marketplace page incorrectly mentions "Pinterest" in one spot. It's a copy-paste bug in their docs — ignore it. + +5. **Crate's code already handles scopes** — The `connection_scope` parameter in `/api/auth0/connect/route.ts` passes `basic write offline_access` to Tumblr's OAuth endpoint automatically. The Dashboard scope settings and the code scopes work together. diff --git a/docs/plans/2026-03-11-sidebar-persistent-chat-design.md b/docs/plans/2026-03-11-sidebar-persistent-chat-design.md new file mode 100644 index 0000000..7a7846e --- /dev/null +++ b/docs/plans/2026-03-11-sidebar-persistent-chat-design.md @@ -0,0 +1,174 @@ +# Sidebar, Persistent Chat & Artifacts UX Design + +**Date:** 2026-03-11 +**Status:** Approved + +## Goal + +Add a Claude-style sidebar with persistent chat history, crate-based project organization, full-text search, and an artifact panel that slides in on demand — transforming Crate Web from a stateless chat into a persistent music research workspace. + +## Architecture + +Convex provides real-time persistence and full-text search. Clerk scopes all data by user. The sidebar is a persistent left rail (260px expanded, 48px collapsed). The artifact panel is hidden by default and slides in from the right when the LLM generates OpenUI Lang content. + +--- + +## Section 1: Layout & Sidebar Structure + +**Sidebar (persistent left rail):** +- 260px expanded, 48px collapsed (icon-only) +- Toggle via hamburger or `Cmd+B` +- Sections: New Chat button, Search bar, Crates, Starred, Recents, Artifacts, Settings/Avatar + +**Main content area:** +- Chat is full-width by default (no split pane) +- Artifact panel slides in from right when OpenUI Lang detected +- When artifact panel open: chat 45%, artifact 55% +- Animated transition (~400ms ease-out) + +**Player bar:** +- Fixed bottom, spans full width below both sidebar and main content + +## Section 2: Data Model & Convex Schema + +### New Tables + +**`crates`** — user-created project folders +| Field | Type | Notes | +|-------|------|-------| +| userId | string | Clerk user ID | +| name | string | User-chosen name | +| color | string (optional) | Accent color | +| createdAt | number | Timestamp | + +### Modified Tables + +**`sessions`** — add fields: +| Field | Type | Notes | +|-------|------|-------| +| crateId | Id<"crates"> (optional) | Folder assignment | +| isStarred | boolean | Quick access | +| lastMessageAt | number | Sort key for recents | + +**`messages`** — existing, add search index: +- Convex search index on `content` field for full-text search + +**`artifacts`** — add fields: +| Field | Type | Notes | +|-------|------|-------| +| userId | string | Clerk user ID | +| sessionId | Id<"sessions"> | Source session | +| label | string | Auto-extracted from OpenUI Lang root component | +| content | string | OpenUI Lang source | +| contentHash | string | Dedup detection | +| createdAt | number | Timestamp | + +### Indexes + +- `sessions` by `userId` + `lastMessageAt` (recents) +- `sessions` by `userId` + `isStarred` (starred) +- `sessions` by `userId` + `crateId` (crate contents) +- `messages` search index on `content` (full-text search) +- `artifacts` by `userId` + `createdAt` (artifact browser) + +## Section 3: Components & UI Architecture + +### Component Tree + +``` + + + — logo + collapse toggle + — prominent, always visible + — Convex search index query + + — collapsible, lists user's crates + — folder icon + name, expands to show sessions + — starred sessions, sorted by lastMessageAt + — last 20 sessions, grouped by Today/Yesterday/This Week + — browsable artifact history across all sessions + — settings gear, user avatar (Clerk) + + + — full width when no artifact; shrinks when artifact slides in + — hidden by default, slides from right when artifact generated + + — fixed bottom, spans full width + +``` + +### Artifact Slide-In Behavior + +- Chat starts full-width (no split pane by default) +- When LLM generates OpenUI Lang, artifact panel slides in from right (animated, ~400ms ease-out) +- Panel takes 55% width, chat shrinks to 45% — uses CSS transition +- User can dismiss (X button) to return chat to full-width +- Panel has a small drag handle for manual resize +- Re-opening an artifact from sidebar or history tab restores the panel + +### Chat Persistence Flow + +1. User sends message → immediately write to Convex `messages` table (optimistic) +2. SSE stream starts → show typing indicator in chat +3. As assistant chunks arrive → accumulate in local state (not written yet) +4. Stream completes → write full assistant message to Convex +5. If OpenUI Lang detected in response → write artifact to Convex `artifacts` table, trigger slide-in +6. On page load → fetch session's messages from Convex, hydrate chat state + +### Session Management + +- `useSession` hook manages current session ID (URL param: `/chat/[sessionId]`) +- New Chat creates a Convex session doc, navigates to its URL +- Session title auto-generated from first user message (first 60 chars, or LLM-summarized later) +- Starring a session toggles `isStarred` field + +## Section 4: Search & Navigation + +### Search + +- Convex search index on `messages.content` — full-text search across all messages +- Search bar in sidebar filters as user types (debounced 300ms) +- Results grouped by session: session title + matching message snippet (highlighted) +- Clicking a result navigates to that session and scrolls to matching message + +### Sidebar Navigation + +- **Recents**: ordered by `lastMessageAt` desc, grouped (Today / Yesterday / This Week / Older), max 20 visible with "Show more" +- **Starred**: sorted by `lastMessageAt`, no grouping +- **Crates**: user-created folders, drag sessions in or assign via right-click +- **Artifacts**: flat list across sessions, sorted by timestamp desc, shows label + session title + date + +### Keyboard Shortcuts + +- `Cmd+K` — focus search bar +- `Cmd+N` — new chat +- `Cmd+B` — toggle sidebar collapse +- `Cmd+Shift+S` — star/unstar current session + +## Section 5: Error Handling & Edge Cases + +### Network & Sync + +- Convex handles real-time sync — messages queue locally on disconnect and sync on reconnect +- Optimistic updates for user messages +- Failed assistant streams show inline error with retry button +- Partial assistant messages NOT persisted — only complete responses written + +### Session Edge Cases + +- Navigate away mid-stream: stream aborted, partial response discarded +- Delete session: soft delete (archived flag), data retained but hidden +- Empty sessions: auto-cleaned after 5 minutes with no messages +- Title generation: first user message truncated to 60 chars; under 10 chars uses assistant response + +### Artifact Edge Cases + +- Malformed OpenUI Lang: show raw content in code block fallback +- Duplicate detection: compare content hash before creating new artifact +- Dismissed panel + same artifact: re-opens without creating duplicate + +### Auth + +- All queries scoped by Clerk `userId` +- No shared sessions in v1 +- Unauthenticated users redirected to sign-in diff --git a/docs/plans/2026-03-11-sidebar-persistent-chat-plan.md b/docs/plans/2026-03-11-sidebar-persistent-chat-plan.md new file mode 100644 index 0000000..24f2858 --- /dev/null +++ b/docs/plans/2026-03-11-sidebar-persistent-chat-plan.md @@ -0,0 +1,2035 @@ +# Sidebar, Persistent Chat & Artifacts Implementation Plan + +> **For Claude:** REQUIRED SUB-SKILL: Use superpowers:executing-plans to implement this plan task-by-task. + +**Goal:** Transform Crate Web from a stateless chat into a persistent music research workspace with a Claude-style sidebar, persistent chat history in Convex, full-text search, and an artifact panel that slides in on demand. + +**Architecture:** Convex is the persistence layer (real-time queries, search indexes). Clerk scopes all data by user. The current `SplitPane` (always-visible 40/60 split) is replaced by a full-width chat that yields space when an artifact slides in from the right. The `Navbar` is replaced by a collapsible sidebar. All existing components (ChatPanel, ArtifactsPanel, artifact-provider) are modified rather than replaced. + +**Tech Stack:** Next.js 15, Convex, Clerk, OpenUI (@openuidev/react-lang), Tailwind CSS v4, react-resizable-panels + +--- + +## Dependency Graph + +``` +Task 1 (Schema) ──► Task 2 (Convex Functions) ──► Task 3 (Sidebar Shell) + │ + ▼ + Task 4 (Session Hook) + │ + ┌────────────┼────────────┐ + ▼ ▼ ▼ + Task 5 Task 6 Task 7 + (Chat (Artifact (Sidebar + Persist) Persist) Sections) + │ │ │ + ▼ ▼ ▼ + Task 8 Task 9 Task 10 + (Slide-In) (Search) (Keyboard + Shortcuts) + │ + ▼ + Task 11 + (Cleanup & + Polish) +``` + +--- + +### Task 1: Convex Schema Changes + +**Files:** +- Modify: `convex/schema.ts` + +**Context:** The existing schema has `sessions`, `messages`, `artifacts` tables but lacks: `crates` table, starred/crate fields on sessions, userId/label/hash fields on artifacts, and search indexes. + +**Step 1: Add `crates` table and update existing tables** + +Replace the entire `convex/schema.ts` with: + +```typescript +import { defineSchema, defineTable } from "convex/server"; +import { v } from "convex/values"; + +export default defineSchema({ + users: defineTable({ + clerkId: v.string(), + email: v.string(), + name: v.optional(v.string()), + encryptedKeys: v.optional(v.bytes()), + createdAt: v.number(), + }).index("by_clerk_id", ["clerkId"]), + + crates: defineTable({ + userId: v.id("users"), + name: v.string(), + color: v.optional(v.string()), + createdAt: v.number(), + }).index("by_user", ["userId"]), + + sessions: defineTable({ + userId: v.id("users"), + crateId: v.optional(v.id("crates")), + title: v.optional(v.string()), + isShared: v.boolean(), + isStarred: v.boolean(), + isArchived: v.boolean(), + lastMessageAt: v.number(), + createdAt: v.number(), + updatedAt: v.number(), + }) + .index("by_user", ["userId"]) + .index("by_user_starred", ["userId", "isStarred"]) + .index("by_user_crate", ["userId", "crateId"]) + .index("by_user_recent", ["userId", "lastMessageAt"]), + + messages: defineTable({ + sessionId: v.id("sessions"), + role: v.union(v.literal("user"), v.literal("assistant")), + content: v.string(), + createdAt: v.number(), + }) + .index("by_session", ["sessionId"]) + .searchIndex("search_content", { + searchField: "content", + filterFields: ["sessionId"], + }), + + artifacts: defineTable({ + sessionId: v.id("sessions"), + userId: v.id("users"), + messageId: v.optional(v.id("messages")), + type: v.string(), + label: v.string(), + data: v.string(), + contentHash: v.string(), + createdAt: v.number(), + }) + .index("by_session", ["sessionId"]) + .index("by_user", ["userId"]), + + toolCalls: defineTable({ + sessionId: v.id("sessions"), + messageId: v.optional(v.id("messages")), + toolName: v.string(), + args: v.string(), + result: v.optional(v.string()), + status: v.union( + v.literal("running"), + v.literal("complete"), + v.literal("error"), + ), + startedAt: v.number(), + completedAt: v.optional(v.number()), + }).index("by_session", ["sessionId"]), + + playerQueue: defineTable({ + sessionId: v.id("sessions"), + tracks: v.array( + v.object({ + title: v.string(), + artist: v.string(), + source: v.union(v.literal("youtube"), v.literal("bandcamp")), + sourceId: v.string(), + imageUrl: v.optional(v.string()), + }), + ), + currentIndex: v.number(), + }).index("by_session", ["sessionId"]), +}); +``` + +**Step 2: Push schema to Convex** + +Run: `cd /Users/tarikmoody/Documents/Projects/crate-web && npx convex dev --once` + +Expected: Schema deployed successfully. Existing data will get `undefined` for new optional fields — this is fine, Convex handles optional fields gracefully. + +**Step 3: Commit** + +```bash +git add convex/schema.ts +git commit -m "feat: add crates table, session/artifact fields, search index" +``` + +--- + +### Task 2: Convex Functions (CRUD) + +**Files:** +- Create: `convex/crates.ts` +- Modify: `convex/sessions.ts` +- Modify: `convex/messages.ts` +- Modify: `convex/artifacts.ts` + +**Context:** We need CRUD for crates, updated session queries (starred, recents, by crate), a search function for messages, and artifact functions that include userId/label/hash. + +**Step 1: Create `convex/crates.ts`** + +```typescript +import { mutation, query } from "./_generated/server"; +import { v } from "convex/values"; + +export const create = mutation({ + args: { + userId: v.id("users"), + name: v.string(), + color: v.optional(v.string()), + }, + handler: async (ctx, args) => { + return await ctx.db.insert("crates", { + userId: args.userId, + name: args.name, + color: args.color, + createdAt: Date.now(), + }); + }, +}); + +export const list = query({ + args: { userId: v.id("users") }, + handler: async (ctx, args) => { + return await ctx.db + .query("crates") + .withIndex("by_user", (q) => q.eq("userId", args.userId)) + .collect(); + }, +}); + +export const rename = mutation({ + args: { id: v.id("crates"), name: v.string() }, + handler: async (ctx, args) => { + await ctx.db.patch(args.id, { name: args.name }); + }, +}); + +export const remove = mutation({ + args: { id: v.id("crates") }, + handler: async (ctx, args) => { + // Unassign all sessions in this crate first + const sessions = await ctx.db + .query("sessions") + .filter((q) => q.eq(q.field("crateId"), args.id)) + .collect(); + for (const session of sessions) { + await ctx.db.patch(session._id, { crateId: undefined }); + } + await ctx.db.delete(args.id); + }, +}); +``` + +**Step 2: Update `convex/sessions.ts`** + +Replace entire file: + +```typescript +import { mutation, query } from "./_generated/server"; +import { v } from "convex/values"; + +export const create = mutation({ + args: { + userId: v.id("users"), + }, + handler: async (ctx, args) => { + const now = Date.now(); + return await ctx.db.insert("sessions", { + userId: args.userId, + isShared: false, + isStarred: false, + isArchived: false, + lastMessageAt: now, + createdAt: now, + updatedAt: now, + }); + }, +}); + +export const listRecent = query({ + args: { userId: v.id("users"), limit: v.optional(v.number()) }, + handler: async (ctx, args) => { + const limit = args.limit ?? 20; + return await ctx.db + .query("sessions") + .withIndex("by_user_recent", (q) => q.eq("userId", args.userId)) + .order("desc") + .filter((q) => q.eq(q.field("isArchived"), false)) + .take(limit); + }, +}); + +export const listStarred = query({ + args: { userId: v.id("users") }, + handler: async (ctx, args) => { + return await ctx.db + .query("sessions") + .withIndex("by_user_starred", (q) => + q.eq("userId", args.userId).eq("isStarred", true), + ) + .filter((q) => q.eq(q.field("isArchived"), false)) + .collect(); + }, +}); + +export const listByCrate = query({ + args: { userId: v.id("users"), crateId: v.id("crates") }, + handler: async (ctx, args) => { + return await ctx.db + .query("sessions") + .withIndex("by_user_crate", (q) => + q.eq("userId", args.userId).eq("crateId", args.crateId), + ) + .filter((q) => q.eq(q.field("isArchived"), false)) + .collect(); + }, +}); + +export const get = query({ + args: { id: v.id("sessions") }, + handler: async (ctx, args) => { + return await ctx.db.get(args.id); + }, +}); + +export const updateTitle = mutation({ + args: { + id: v.id("sessions"), + title: v.string(), + }, + handler: async (ctx, args) => { + await ctx.db.patch(args.id, { + title: args.title, + updatedAt: Date.now(), + }); + }, +}); + +export const toggleStar = mutation({ + args: { id: v.id("sessions") }, + handler: async (ctx, args) => { + const session = await ctx.db.get(args.id); + if (!session) throw new Error("Session not found"); + await ctx.db.patch(args.id, { isStarred: !session.isStarred }); + }, +}); + +export const assignToCrate = mutation({ + args: { + id: v.id("sessions"), + crateId: v.optional(v.id("crates")), + }, + handler: async (ctx, args) => { + await ctx.db.patch(args.id, { + crateId: args.crateId, + updatedAt: Date.now(), + }); + }, +}); + +export const archive = mutation({ + args: { id: v.id("sessions") }, + handler: async (ctx, args) => { + await ctx.db.patch(args.id, { + isArchived: true, + updatedAt: Date.now(), + }); + }, +}); + +export const toggleShare = mutation({ + args: { id: v.id("sessions") }, + handler: async (ctx, args) => { + const session = await ctx.db.get(args.id); + if (!session) throw new Error("Session not found"); + await ctx.db.patch(args.id, { isShared: !session.isShared }); + }, +}); + +export const touchLastMessage = mutation({ + args: { id: v.id("sessions") }, + handler: async (ctx, args) => { + const now = Date.now(); + await ctx.db.patch(args.id, { + lastMessageAt: now, + updatedAt: now, + }); + }, +}); +``` + +**Step 3: Update `convex/messages.ts`** + +Replace entire file to add search: + +```typescript +import { mutation, query } from "./_generated/server"; +import { v } from "convex/values"; + +export const send = mutation({ + args: { + sessionId: v.id("sessions"), + role: v.union(v.literal("user"), v.literal("assistant")), + content: v.string(), + }, + handler: async (ctx, args) => { + const id = await ctx.db.insert("messages", { + sessionId: args.sessionId, + role: args.role, + content: args.content, + createdAt: Date.now(), + }); + // Touch session's lastMessageAt + const now = Date.now(); + await ctx.db.patch(args.sessionId, { + lastMessageAt: now, + updatedAt: now, + }); + return id; + }, +}); + +export const list = query({ + args: { + sessionId: v.id("sessions"), + }, + handler: async (ctx, args) => { + return await ctx.db + .query("messages") + .withIndex("by_session", (q) => q.eq("sessionId", args.sessionId)) + .order("asc") + .collect(); + }, +}); + +export const search = query({ + args: { + query: v.string(), + }, + handler: async (ctx, args) => { + if (!args.query.trim()) return []; + const results = await ctx.db + .query("messages") + .withSearchIndex("search_content", (q) => q.search("content", args.query)) + .take(50); + return results; + }, +}); +``` + +**Step 4: Update `convex/artifacts.ts`** + +Replace entire file: + +```typescript +import { mutation, query } from "./_generated/server"; +import { v } from "convex/values"; + +export const create = mutation({ + args: { + sessionId: v.id("sessions"), + userId: v.id("users"), + messageId: v.optional(v.id("messages")), + type: v.string(), + label: v.string(), + data: v.string(), + contentHash: v.string(), + }, + handler: async (ctx, args) => { + // Deduplicate: check if artifact with same hash already exists in session + const existing = await ctx.db + .query("artifacts") + .withIndex("by_session", (q) => q.eq("sessionId", args.sessionId)) + .filter((q) => q.eq(q.field("contentHash"), args.contentHash)) + .first(); + if (existing) return existing._id; + + return await ctx.db.insert("artifacts", { + sessionId: args.sessionId, + userId: args.userId, + messageId: args.messageId, + type: args.type, + label: args.label, + data: args.data, + contentHash: args.contentHash, + createdAt: Date.now(), + }); + }, +}); + +export const listBySession = query({ + args: { sessionId: v.id("sessions") }, + handler: async (ctx, args) => { + return await ctx.db + .query("artifacts") + .withIndex("by_session", (q) => q.eq("sessionId", args.sessionId)) + .order("asc") + .collect(); + }, +}); + +export const listByUser = query({ + args: { userId: v.id("users") }, + handler: async (ctx, args) => { + return await ctx.db + .query("artifacts") + .withIndex("by_user", (q) => q.eq("userId", args.userId)) + .order("desc") + .take(50); + }, +}); +``` + +**Step 5: Push schema and verify** + +Run: `cd /Users/tarikmoody/Documents/Projects/crate-web && npx convex dev --once` + +Expected: All functions deploy successfully. + +**Step 6: Commit** + +```bash +git add convex/crates.ts convex/sessions.ts convex/messages.ts convex/artifacts.ts +git commit -m "feat: add crates CRUD, session starring/archiving, message search, artifact dedup" +``` + +--- + +### Task 3: Sidebar Shell & Layout Restructure + +**Files:** +- Create: `src/components/sidebar/sidebar.tsx` +- Create: `src/components/sidebar/sidebar-header.tsx` +- Create: `src/components/sidebar/sidebar-footer.tsx` +- Modify: `src/app/w/layout.tsx` +- Modify: `src/components/workspace/navbar.tsx` (delete or gut) + +**Context:** The current workspace layout is `Navbar` (top bar) + `main` + `PlayerBar`. We're replacing the top Navbar with a left sidebar. The Navbar currently contains the Crate logo, settings gear, and Clerk UserButton. These move into the sidebar. + +**Step 1: Create `src/components/sidebar/sidebar.tsx`** + +```tsx +"use client"; + +import { useState, createContext, useContext, ReactNode } from "react"; +import { SidebarHeader } from "./sidebar-header"; +import { SidebarFooter } from "./sidebar-footer"; + +interface SidebarContextValue { + collapsed: boolean; + toggle: () => void; +} + +const SidebarContext = createContext(null); + +export function useSidebar() { + const ctx = useContext(SidebarContext); + if (!ctx) throw new Error("useSidebar must be used within Sidebar"); + return ctx; +} + +export function Sidebar({ children }: { children: ReactNode }) { + const [collapsed, setCollapsed] = useState(false); + + return ( + setCollapsed((c) => !c) }}> + + + ); +} +``` + +**Step 2: Create `src/components/sidebar/sidebar-header.tsx`** + +```tsx +"use client"; + +import { useSidebar } from "./sidebar"; + +export function SidebarHeader() { + const { collapsed, toggle } = useSidebar(); + + return ( +
+ {!collapsed && ( + Crate + )} + +
+ ); +} +``` + +**Step 3: Create `src/components/sidebar/sidebar-footer.tsx`** + +```tsx +"use client"; + +import { useState } from "react"; +import { UserButton } from "@clerk/nextjs"; +import { SettingsDrawer } from "@/components/settings/settings-drawer"; +import { useSidebar } from "./sidebar"; + +export function SidebarFooter() { + const { collapsed } = useSidebar(); + const [isSettingsOpen, setIsSettingsOpen] = useState(false); + + return ( + <> +
+ + {!collapsed && ( + + )} +
+ setIsSettingsOpen(false)} /> + + ); +} +``` + +**Step 4: Update `src/app/w/layout.tsx`** + +Replace entire file: + +```tsx +import { PlayerBar } from "@/components/player/player-bar"; +import { PlayerProvider } from "@/components/player/player-provider"; +import { Sidebar } from "@/components/sidebar/sidebar"; + +export default function WorkspaceLayout({ + children, +}: { + children: React.ReactNode; +}) { + return ( + +
+ + {/* Sidebar sections added in Task 7 */} +
+ +
+
{children}
+ +
+
+ + ); +} +``` + +**Step 5: Verify it builds** + +Run: `cd /Users/tarikmoody/Documents/Projects/crate-web && npx next build 2>&1 | tail -20` + +Expected: Build succeeds. The app now shows a sidebar on the left with Crate logo, collapse toggle, user avatar, and settings gear. Main content area fills the rest. + +**Step 6: Commit** + +```bash +git add src/components/sidebar/ src/app/w/layout.tsx +git commit -m "feat: add collapsible sidebar shell, restructure workspace layout" +``` + +--- + +### Task 4: Session Hook & Routing + +**Files:** +- Create: `src/hooks/use-session.ts` +- Modify: `src/app/w/page.tsx` +- Modify: `src/app/w/[sessionId]/page.tsx` + +**Context:** We need a `useSession` hook that reads the current session ID from the URL, creates new sessions, and provides session data from Convex. The workspace landing page (`/w`) should auto-create a session and redirect. The session page (`/w/[sessionId]`) should load that session's data. + +**Step 1: Create `src/hooks/use-session.ts`** + +```tsx +"use client"; + +import { useParams, useRouter } from "next/navigation"; +import { useQuery, useMutation } from "convex/react"; +import { api } from "../../convex/_generated/api"; +import { Id } from "../../convex/_generated/dataModel"; +import { useCallback } from "react"; + +export function useCurrentUser() { + const user = useQuery(api.users.getByClerkId, { clerkId: "__clerk_user_id__" }); + // We can't pass the actual Clerk ID here at call time — we need to get it from the auth context. + // Instead, we'll create a helper query. For now, return undefined. + return user; +} + +export function useSession() { + const params = useParams(); + const router = useRouter(); + const sessionId = params?.sessionId as string | undefined; + + const session = useQuery( + api.sessions.get, + sessionId ? { id: sessionId as Id<"sessions"> } : "skip", + ); + + const createSession = useMutation(api.sessions.create); + const updateTitle = useMutation(api.sessions.updateTitle); + const toggleStar = useMutation(api.sessions.toggleStar); + const archiveSession = useMutation(api.sessions.archive); + const assignToCrate = useMutation(api.sessions.assignToCrate); + + const newChat = useCallback( + async (userId: Id<"users">) => { + const id = await createSession({ userId }); + router.push(`/w/${id}`); + return id; + }, + [createSession, router], + ); + + return { + sessionId: sessionId as Id<"sessions"> | undefined, + session, + newChat, + updateTitle, + toggleStar, + archiveSession, + assignToCrate, + }; +} +``` + +**Step 2: Update `src/app/w/[sessionId]/page.tsx`** + +```tsx +"use client"; + +import { ArtifactProvider } from "@/components/workspace/artifact-provider"; +import { ChatPanel } from "@/components/workspace/chat-panel"; + +export default function SessionPage() { + return ( + + + + ); +} +``` + +**Step 3: Update `src/app/w/page.tsx`** + +This page auto-creates a new session and redirects: + +```tsx +"use client"; + +import { useEffect, useRef } from "react"; +import { useRouter } from "next/navigation"; +import { useMutation, useQuery } from "convex/react"; +import { useAuth } from "@clerk/nextjs"; +import { api } from "../../../convex/_generated/api"; + +export default function WorkspacePage() { + const router = useRouter(); + const { userId: clerkId } = useAuth(); + const user = useQuery(api.users.getByClerkId, clerkId ? { clerkId } : "skip"); + const createSession = useMutation(api.sessions.create); + const creating = useRef(false); + + useEffect(() => { + if (!user || creating.current) return; + creating.current = true; + createSession({ userId: user._id }).then((id) => { + router.replace(`/w/${id}`); + }); + }, [user, createSession, router]); + + return ( +
+

Creating new session...

+
+ ); +} +``` + +**Step 4: Verify routing works** + +Run: `cd /Users/tarikmoody/Documents/Projects/crate-web && npx next build 2>&1 | tail -20` + +Expected: Build succeeds. Navigating to `/w` creates a session and redirects to `/w/[sessionId]`. + +**Step 5: Commit** + +```bash +git add src/hooks/use-session.ts src/app/w/page.tsx src/app/w/\\[sessionId\\]/page.tsx +git commit -m "feat: add useSession hook, auto-create session on /w, wire routing" +``` + +--- + +### Task 5: Chat Persistence (Wire ChatPanel to Convex) + +**Files:** +- Modify: `src/components/workspace/chat-panel.tsx` + +**Context:** Currently `ChatPanel` uses OpenUI's `ChatProvider` which manages messages in local state only. We need to: +1. Load existing messages from Convex on mount +2. Persist user messages to Convex immediately (optimistic) +3. Persist assistant messages to Convex after streaming completes +4. Update session title from first user message + +The `ChatProvider`/`useThread` from OpenUI still manages the streaming conversation — we add Convex persistence as a side-effect layer on top. + +**Step 1: Add Convex persistence hooks to ChatPanel** + +At the top of `chat-panel.tsx`, add these imports: + +```tsx +import { useParams } from "next/navigation"; +import { useMutation, useQuery } from "convex/react"; +import { useAuth } from "@clerk/nextjs"; +import { api } from "../../../convex/_generated/api"; +import { Id } from "../../../convex/_generated/dataModel"; +``` + +**Step 2: Create a `ChatPersistence` component** + +Add inside the `ChatProvider` (after `ChatMessages` and `ChatInput`), a component that watches thread state and persists: + +```tsx +function ChatPersistence() { + const { messages, isRunning } = useThread(); + const params = useParams(); + const sessionId = params?.sessionId as Id<"sessions"> | undefined; + const { userId: clerkId } = useAuth(); + const user = useQuery(api.users.getByClerkId, clerkId ? { clerkId } : "skip"); + const sendMessage = useMutation(api.messages.send); + const updateTitle = useMutation(api.sessions.updateTitle); + const persistedRef = useRef(new Set()); + const titleSetRef = useRef(false); + + useEffect(() => { + if (!sessionId || !user) return; + + for (const m of messages) { + if (persistedRef.current.has(m.id)) continue; + + // Persist user messages immediately + if (m.role === "user") { + const parts = getContentParts(m.content); + const text = parts + .filter((c): c is { type: "text"; text: string } => c.type === "text") + .map((c) => c.text) + .join(""); + if (text) { + persistedRef.current.add(m.id); + sendMessage({ sessionId, role: "user", content: text }); + // Set session title from first user message + if (!titleSetRef.current) { + titleSetRef.current = true; + const title = text.length > 60 ? text.slice(0, 60) + "..." : text; + updateTitle({ id: sessionId, title }); + } + } + } + + // Persist assistant messages only after streaming completes + if (m.role === "assistant" && !isRunning) { + const parts = getContentParts(m.content); + const text = parts + .filter((c): c is { type: "text"; text: string } => c.type === "text") + .map((c) => c.text) + .join(""); + if (text) { + persistedRef.current.add(m.id); + sendMessage({ sessionId, role: "assistant", content: text }); + } + } + } + }, [messages, isRunning, sessionId, user, sendMessage, updateTitle]); + + return null; // Invisible persistence layer +} +``` + +**Step 3: Add `ChatPersistence` inside the `ChatPanel` render** + +In the `ChatPanel` component, add `` inside the `ChatProvider`: + +```tsx +export function ChatPanel() { + return ( + { + // ... existing code unchanged ... + }} + streamProtocol={crateStreamAdapter()} + > +
+ + + +
+
+ ); +} +``` + +**Step 4: Verify it builds** + +Run: `cd /Users/tarikmoody/Documents/Projects/crate-web && npx next build 2>&1 | tail -20` + +Expected: Build succeeds. Messages now persist to Convex. + +**Step 5: Commit** + +```bash +git add src/components/workspace/chat-panel.tsx +git commit -m "feat: persist chat messages to Convex, auto-set session title" +``` + +--- + +### Task 6: Artifact Persistence (Wire to Convex) + +**Files:** +- Modify: `src/components/workspace/artifact-provider.tsx` + +**Context:** Currently `ArtifactProvider` keeps artifacts in local React state only. We need to persist artifacts to Convex and load them from Convex on mount. The `contentHash` field enables deduplication. + +**Step 1: Update `artifact-provider.tsx`** + +Replace entire file: + +```tsx +"use client"; + +import { createContext, useContext, useState, useCallback, useEffect, ReactNode } from "react"; +import { useParams } from "next/navigation"; +import { useMutation, useQuery } from "convex/react"; +import { useAuth } from "@clerk/nextjs"; +import { api } from "../../../convex/_generated/api"; +import { Id } from "../../../convex/_generated/dataModel"; + +interface Artifact { + id: string; + label: string; + content: string; + timestamp: number; +} + +/** Extract a label from OpenUI Lang content by parsing the root component's first string arg. */ +function extractLabel(content: string): string { + const match = content.match(/^root\s*=\s*\w+\(\s*"([^"]+)"/m); + if (match?.[1]) return match[1]; + const fallback = content.match(/^\w+\s*=\s*\w+\(\s*"([^"]+)"/m); + return fallback?.[1] ?? "Artifact"; +} + +/** Simple hash for deduplication. */ +async function hashContent(content: string): Promise { + const encoder = new TextEncoder(); + const data = encoder.encode(content); + const hashBuffer = await crypto.subtle.digest("SHA-256", data); + const hashArray = Array.from(new Uint8Array(hashBuffer)); + return hashArray.map((b) => b.toString(16).padStart(2, "0")).join(""); +} + +interface ArtifactContextValue { + current: Artifact | null; + history: Artifact[]; + setArtifact: (content: string) => void; + selectArtifact: (id: string) => void; + clear: () => void; + showPanel: boolean; + dismissPanel: () => void; +} + +const ArtifactContext = createContext(null); + +export function useArtifact() { + const ctx = useContext(ArtifactContext); + if (!ctx) throw new Error("useArtifact must be used within ArtifactProvider"); + return ctx; +} + +export function ArtifactProvider({ children }: { children: ReactNode }) { + const [current, setCurrent] = useState(null); + const [history, setHistory] = useState([]); + const [showPanel, setShowPanel] = useState(false); + + const params = useParams(); + const sessionId = params?.sessionId as Id<"sessions"> | undefined; + const { userId: clerkId } = useAuth(); + const user = useQuery(api.users.getByClerkId, clerkId ? { clerkId } : "skip"); + const convexArtifacts = useQuery( + api.artifacts.listBySession, + sessionId ? { sessionId } : "skip", + ); + const createArtifact = useMutation(api.artifacts.create); + + // Hydrate history from Convex on mount + useEffect(() => { + if (!convexArtifacts || convexArtifacts.length === 0) return; + const hydrated: Artifact[] = convexArtifacts.map((a) => ({ + id: a._id, + label: a.label, + content: a.data, + timestamp: a.createdAt, + })); + setHistory(hydrated); + // Set current to latest + setCurrent(hydrated[hydrated.length - 1]); + }, [convexArtifacts]); + + const setArtifact = useCallback( + (content: string) => { + const artifact: Artifact = { + id: crypto.randomUUID(), + label: extractLabel(content), + content, + timestamp: Date.now(), + }; + setCurrent(artifact); + setHistory((prev) => [...prev, artifact]); + setShowPanel(true); + + // Persist to Convex + if (sessionId && user) { + hashContent(content).then((contentHash) => { + createArtifact({ + sessionId, + userId: user._id, + type: "openui", + label: artifact.label, + data: content, + contentHash, + }); + }); + } + }, + [sessionId, user, createArtifact], + ); + + const selectArtifact = useCallback((id: string) => { + setHistory((prev) => { + const found = prev.find((a) => a.id === id); + if (found) { + setCurrent(found); + setShowPanel(true); + } + return prev; + }); + }, []); + + const clear = useCallback(() => { + setCurrent(null); + }, []); + + const dismissPanel = useCallback(() => { + setShowPanel(false); + }, []); + + return ( + + {children} + + ); +} +``` + +**Step 2: Verify it builds** + +Run: `cd /Users/tarikmoody/Documents/Projects/crate-web && npx next build 2>&1 | tail -20` + +Expected: Build succeeds. + +**Step 3: Commit** + +```bash +git add src/components/workspace/artifact-provider.tsx +git commit -m "feat: persist artifacts to Convex with dedup, add showPanel state" +``` + +--- + +### Task 7: Sidebar Sections (Crates, Starred, Recents, Artifacts) + +**Files:** +- Create: `src/components/sidebar/new-chat-button.tsx` +- Create: `src/components/sidebar/recents-section.tsx` +- Create: `src/components/sidebar/starred-section.tsx` +- Create: `src/components/sidebar/crates-section.tsx` +- Create: `src/components/sidebar/artifacts-section.tsx` +- Create: `src/components/sidebar/session-item.tsx` +- Modify: `src/app/w/layout.tsx` + +**Context:** The sidebar needs four sections: Crates (user-created folders), Starred (favorited sessions), Recents (last 20 sessions grouped by date), and Artifacts (browsable artifact history). Each section is collapsible. + +**Step 1: Create `src/components/sidebar/session-item.tsx`** + +Reusable session list item used by Recents, Starred, and Crates sections: + +```tsx +"use client"; + +import Link from "next/link"; +import { useParams } from "next/navigation"; +import { Id } from "../../../convex/_generated/dataModel"; + +interface SessionItemProps { + id: Id<"sessions">; + title: string | undefined; + isStarred: boolean; + onToggleStar?: () => void; +} + +export function SessionItem({ id, title, isStarred, onToggleStar }: SessionItemProps) { + const params = useParams(); + const isActive = params?.sessionId === id; + const displayTitle = title || "New chat"; + + return ( + + {displayTitle} + {onToggleStar && ( + + )} + + ); +} +``` + +**Step 2: Create `src/components/sidebar/new-chat-button.tsx`** + +```tsx +"use client"; + +import { useRouter } from "next/navigation"; +import { useMutation, useQuery } from "convex/react"; +import { useAuth } from "@clerk/nextjs"; +import { api } from "../../../convex/_generated/api"; + +export function NewChatButton() { + const router = useRouter(); + const { userId: clerkId } = useAuth(); + const user = useQuery(api.users.getByClerkId, clerkId ? { clerkId } : "skip"); + const createSession = useMutation(api.sessions.create); + + const handleNewChat = async () => { + if (!user) return; + const id = await createSession({ userId: user._id }); + router.push(`/w/${id}`); + }; + + return ( + + ); +} +``` + +**Step 3: Create `src/components/sidebar/recents-section.tsx`** + +```tsx +"use client"; + +import { useState } from "react"; +import { useQuery, useMutation } from "convex/react"; +import { useAuth } from "@clerk/nextjs"; +import { api } from "../../../convex/_generated/api"; +import { SessionItem } from "./session-item"; + +function groupByDate(sessions: Array<{ _id: string; lastMessageAt: number; title?: string; isStarred: boolean }>) { + const now = Date.now(); + const today = new Date(); + today.setHours(0, 0, 0, 0); + const yesterday = new Date(today); + yesterday.setDate(yesterday.getDate() - 1); + const weekAgo = new Date(today); + weekAgo.setDate(weekAgo.getDate() - 7); + + const groups: Record = { + Today: [], + Yesterday: [], + "This Week": [], + Older: [], + }; + + for (const s of sessions) { + const d = new Date(s.lastMessageAt); + if (d >= today) groups.Today.push(s); + else if (d >= yesterday) groups.Yesterday.push(s); + else if (d >= weekAgo) groups["This Week"].push(s); + else groups.Older.push(s); + } + + return groups; +} + +export function RecentsSection() { + const [expanded, setExpanded] = useState(true); + const { userId: clerkId } = useAuth(); + const user = useQuery(api.users.getByClerkId, clerkId ? { clerkId } : "skip"); + const sessions = useQuery(api.sessions.listRecent, user ? { userId: user._id } : "skip"); + const toggleStar = useMutation(api.sessions.toggleStar); + + if (!sessions || sessions.length === 0) return null; + + const groups = groupByDate(sessions as any); + + return ( +
+ + {expanded && ( +
+ {Object.entries(groups).map(([label, items]) => + items.length > 0 ? ( +
+

{label}

+ {items.map((s: any) => ( + toggleStar({ id: s._id })} + /> + ))} +
+ ) : null, + )} +
+ )} +
+ ); +} +``` + +**Step 4: Create `src/components/sidebar/starred-section.tsx`** + +```tsx +"use client"; + +import { useState } from "react"; +import { useQuery, useMutation } from "convex/react"; +import { useAuth } from "@clerk/nextjs"; +import { api } from "../../../convex/_generated/api"; +import { SessionItem } from "./session-item"; + +export function StarredSection() { + const [expanded, setExpanded] = useState(true); + const { userId: clerkId } = useAuth(); + const user = useQuery(api.users.getByClerkId, clerkId ? { clerkId } : "skip"); + const sessions = useQuery(api.sessions.listStarred, user ? { userId: user._id } : "skip"); + const toggleStar = useMutation(api.sessions.toggleStar); + + if (!sessions || sessions.length === 0) return null; + + return ( +
+ + {expanded && ( +
+ {(sessions as any[]).map((s) => ( + toggleStar({ id: s._id })} + /> + ))} +
+ )} +
+ ); +} +``` + +**Step 5: Create `src/components/sidebar/crates-section.tsx`** + +```tsx +"use client"; + +import { useState } from "react"; +import { useQuery, useMutation } from "convex/react"; +import { useAuth } from "@clerk/nextjs"; +import { api } from "../../../convex/_generated/api"; +import { SessionItem } from "./session-item"; + +export function CratesSection() { + const [expanded, setExpanded] = useState(true); + const { userId: clerkId } = useAuth(); + const user = useQuery(api.users.getByClerkId, clerkId ? { clerkId } : "skip"); + const crates = useQuery(api.crates.list, user ? { userId: user._id } : "skip"); + const createCrate = useMutation(api.crates.create); + const toggleStar = useMutation(api.sessions.toggleStar); + + const [creating, setCreating] = useState(false); + const [newName, setNewName] = useState(""); + + if (!crates) return null; + + const handleCreate = async () => { + if (!user || !newName.trim()) return; + await createCrate({ userId: user._id, name: newName.trim() }); + setNewName(""); + setCreating(false); + }; + + return ( +
+
+ + +
+ {expanded && ( +
+ {creating && ( +
{ + e.preventDefault(); + handleCreate(); + }} + className="mb-1" + > + setNewName(e.target.value)} + onBlur={() => { + if (!newName.trim()) setCreating(false); + }} + placeholder="Crate name..." + className="w-full rounded border border-zinc-700 bg-zinc-900 px-2 py-1 text-xs text-white placeholder-zinc-500 focus:border-zinc-500 focus:outline-none" + /> +
+ )} + {crates.length === 0 && !creating && ( +

No crates yet

+ )} + {crates.map((crate) => ( + + ))} +
+ )} +
+ ); +} + +function CrateFolder({ + crateId, + name, + userId, + toggleStar, +}: { + crateId: any; + name: string; + userId: any; + toggleStar: any; +}) { + const [open, setOpen] = useState(false); + const sessions = useQuery(api.sessions.listByCrate, { userId, crateId }); + + return ( +
+ + {open && sessions && ( +
+ {sessions.length === 0 ? ( +

Empty

+ ) : ( + (sessions as any[]).map((s) => ( + toggleStar({ id: s._id })} + /> + )) + )} +
+ )} +
+ ); +} +``` + +**Step 6: Create `src/components/sidebar/artifacts-section.tsx`** + +```tsx +"use client"; + +import { useState } from "react"; +import { useQuery } from "convex/react"; +import { useAuth } from "@clerk/nextjs"; +import { useRouter } from "next/navigation"; +import { api } from "../../../convex/_generated/api"; + +export function ArtifactsSection() { + const [expanded, setExpanded] = useState(false); + const { userId: clerkId } = useAuth(); + const user = useQuery(api.users.getByClerkId, clerkId ? { clerkId } : "skip"); + const artifacts = useQuery(api.artifacts.listByUser, user ? { userId: user._id } : "skip"); + const router = useRouter(); + + if (!artifacts || artifacts.length === 0) return null; + + return ( +
+ + {expanded && ( +
+ {artifacts.map((a) => ( + + ))} +
+ )} +
+ ); +} +``` + +**Step 7: Wire sections into workspace layout** + +Update `src/app/w/layout.tsx`: + +```tsx +import { PlayerBar } from "@/components/player/player-bar"; +import { PlayerProvider } from "@/components/player/player-provider"; +import { Sidebar } from "@/components/sidebar/sidebar"; +import { NewChatButton } from "@/components/sidebar/new-chat-button"; +import { CratesSection } from "@/components/sidebar/crates-section"; +import { StarredSection } from "@/components/sidebar/starred-section"; +import { RecentsSection } from "@/components/sidebar/recents-section"; +import { ArtifactsSection } from "@/components/sidebar/artifacts-section"; + +export default function WorkspaceLayout({ + children, +}: { + children: React.ReactNode; +}) { + return ( + +
+ + +
+ + + + +
+
+
+
{children}
+ +
+
+
+ ); +} +``` + +**Step 8: Verify it builds** + +Run: `cd /Users/tarikmoody/Documents/Projects/crate-web && npx next build 2>&1 | tail -20` + +Expected: Build succeeds. Sidebar shows New Chat button, Crates, Starred, Recents, and Artifacts sections. + +**Step 9: Commit** + +```bash +git add src/components/sidebar/ src/app/w/layout.tsx +git commit -m "feat: add sidebar sections — crates, starred, recents, artifacts" +``` + +--- + +### Task 8: Artifact Slide-In Panel + +**Files:** +- Create: `src/components/workspace/artifact-slide-in.tsx` +- Modify: `src/app/w/[sessionId]/page.tsx` +- Modify: `src/components/workspace/split-pane.tsx` (can be deleted or kept as legacy) + +**Context:** The current `SplitPane` always shows a 40/60 split. The new design has chat full-width by default, with the artifact panel sliding in from the right when an artifact is generated. We replace `SplitPane` with a new layout in the session page. + +**Step 1: Create `src/components/workspace/artifact-slide-in.tsx`** + +```tsx +"use client"; + +import { Renderer } from "@openuidev/react-lang"; +import { crateLibrary } from "@/lib/openui/library"; +import { useArtifact } from "./artifact-provider"; + +export function ArtifactSlideIn() { + const { current, history, selectArtifact, showPanel, dismissPanel } = useArtifact(); + + if (!showPanel || !current) return null; + + return ( +
+ {/* Header */} +
+ + {current.label.length > 40 ? `${current.label.slice(0, 40)}...` : current.label} + + +
+ + {/* History tabs */} + {history.length > 1 && ( +
+ {history.map((a) => ( + + ))} +
+ )} + + {/* Content */} +
+ +
+
+ ); +} +``` + +**Step 2: Update `src/app/w/[sessionId]/page.tsx`** + +```tsx +"use client"; + +import { ArtifactProvider } from "@/components/workspace/artifact-provider"; +import { ChatPanel } from "@/components/workspace/chat-panel"; +import { ArtifactSlideIn } from "@/components/workspace/artifact-slide-in"; + +export default function SessionPage() { + return ( + +
+
+ +
+ +
+
+ ); +} +``` + +**Step 3: Add Tailwind animate-in utility** + +The `animate-in slide-in-from-right` classes may not exist in Tailwind v4 by default. Add to `src/app/globals.css`: + +```css +@keyframes slide-in-from-right { + from { + transform: translateX(100%); + } + to { + transform: translateX(0); + } +} + +.animate-in.slide-in-from-right { + animation: slide-in-from-right 0.3s ease-out; +} +``` + +**Step 4: Verify it builds** + +Run: `cd /Users/tarikmoody/Documents/Projects/crate-web && npx next build 2>&1 | tail -20` + +Expected: Build succeeds. Chat is now full-width, artifact panel slides in when generated. + +**Step 5: Commit** + +```bash +git add src/components/workspace/artifact-slide-in.tsx src/app/w/\\[sessionId\\]/page.tsx src/app/globals.css +git commit -m "feat: replace split-pane with artifact slide-in panel" +``` + +--- + +### Task 9: Sidebar Search + +**Files:** +- Create: `src/components/sidebar/search-bar.tsx` +- Modify: `src/app/w/layout.tsx` (add SearchBar to sidebar) + +**Context:** The sidebar needs a search bar that queries the Convex `messages` search index and shows results grouped by session. + +**Step 1: Create `src/components/sidebar/search-bar.tsx`** + +```tsx +"use client"; + +import { useState, useEffect, useRef } from "react"; +import { useQuery } from "convex/react"; +import { useRouter } from "next/navigation"; +import { api } from "../../../convex/_generated/api"; + +export function SearchBar() { + const [query, setQuery] = useState(""); + const [debouncedQuery, setDebouncedQuery] = useState(""); + const [isOpen, setIsOpen] = useState(false); + const router = useRouter(); + const inputRef = useRef(null); + + // Debounce search query + useEffect(() => { + const timer = setTimeout(() => setDebouncedQuery(query), 300); + return () => clearTimeout(timer); + }, [query]); + + const results = useQuery( + api.messages.search, + debouncedQuery.trim() ? { query: debouncedQuery.trim() } : "skip", + ); + + // Group results by session + const grouped = results + ? Object.entries( + results.reduce( + (acc, msg) => { + const key = msg.sessionId; + if (!acc[key]) acc[key] = []; + acc[key].push(msg); + return acc; + }, + {} as Record, + ), + ) + : []; + + return ( +
+ { + setQuery(e.target.value); + setIsOpen(true); + }} + onFocus={() => query && setIsOpen(true)} + onBlur={() => setTimeout(() => setIsOpen(false), 200)} + placeholder="Search research history..." + className="w-full rounded-md border border-zinc-800 bg-zinc-900 px-3 py-1.5 text-xs text-white placeholder-zinc-500 focus:border-zinc-600 focus:outline-none" + /> + + + + + {/* Results dropdown */} + {isOpen && debouncedQuery && ( +
+ {grouped.length === 0 ? ( +

No results

+ ) : ( + grouped.map(([sessionId, msgs]) => ( + + )) + )} +
+ )} +
+ ); +} +``` + +**Step 2: Add SearchBar to layout** + +In `src/app/w/layout.tsx`, add import and place after `NewChatButton`: + +```tsx +import { SearchBar } from "@/components/sidebar/search-bar"; +``` + +Inside the Sidebar children, add `` after ``. + +**Step 3: Verify it builds** + +Run: `cd /Users/tarikmoody/Documents/Projects/crate-web && npx next build 2>&1 | tail -20` + +Expected: Build succeeds. + +**Step 4: Commit** + +```bash +git add src/components/sidebar/search-bar.tsx src/app/w/layout.tsx +git commit -m "feat: add full-text search bar to sidebar" +``` + +--- + +### Task 10: Keyboard Shortcuts + +**Files:** +- Create: `src/hooks/use-keyboard-shortcuts.ts` +- Modify: `src/app/w/layout.tsx` (add hook) + +**Context:** Four shortcuts: `Cmd+K` (focus search), `Cmd+N` (new chat), `Cmd+B` (toggle sidebar), `Cmd+Shift+S` (star/unstar session). + +**Step 1: Create `src/hooks/use-keyboard-shortcuts.ts`** + +```tsx +"use client"; + +import { useEffect } from "react"; + +interface ShortcutHandlers { + onNewChat?: () => void; + onToggleSidebar?: () => void; + onFocusSearch?: () => void; + onToggleStar?: () => void; +} + +export function useKeyboardShortcuts(handlers: ShortcutHandlers) { + useEffect(() => { + function handleKeyDown(e: KeyboardEvent) { + const meta = e.metaKey || e.ctrlKey; + if (!meta) return; + + if (e.key === "k") { + e.preventDefault(); + handlers.onFocusSearch?.(); + } else if (e.key === "n") { + e.preventDefault(); + handlers.onNewChat?.(); + } else if (e.key === "b") { + e.preventDefault(); + handlers.onToggleSidebar?.(); + } else if (e.key === "S" && e.shiftKey) { + e.preventDefault(); + handlers.onToggleStar?.(); + } + } + + document.addEventListener("keydown", handleKeyDown); + return () => document.removeEventListener("keydown", handleKeyDown); + }, [handlers]); +} +``` + +**Step 2: Verify it builds** + +Run: `cd /Users/tarikmoody/Documents/Projects/crate-web && npx next build 2>&1 | tail -20` + +Expected: Build succeeds. + +**Note:** Wiring the shortcuts to actual handlers requires making the sidebar context and search input ref accessible from the layout. This can be done by exposing the sidebar toggle via the `useSidebar` context (already exported) and adding a ref to the search input. The wiring will happen during Task 11 integration. + +**Step 3: Commit** + +```bash +git add src/hooks/use-keyboard-shortcuts.ts +git commit -m "feat: add keyboard shortcuts hook (Cmd+K/N/B/Shift+S)" +``` + +--- + +### Task 11: Integration, Cleanup & Polish + +**Files:** +- Modify: `src/app/w/layout.tsx` (wire keyboard shortcuts) +- Modify: `src/components/sidebar/sidebar.tsx` (expose toggle to layout) +- Modify: `src/components/sidebar/search-bar.tsx` (expose focus via ref) +- Delete or deprecate: `src/components/workspace/split-pane.tsx` (no longer used) +- Delete or deprecate: `src/components/workspace/navbar.tsx` (replaced by sidebar) + +**Context:** Final integration — wire keyboard shortcuts, clean up unused components, ensure everything works together. + +**Step 1: Export sidebar toggle for keyboard shortcuts** + +The `useSidebar` hook is already exported from `sidebar.tsx`. To use it in the layout, we need to restructure slightly. Create a wrapper component in the layout that can access both sidebar context and shortcuts. + +Update `src/app/w/layout.tsx`: + +```tsx +"use client"; + +import { useRef, useCallback } from "react"; +import { useParams, useRouter } from "next/navigation"; +import { useMutation, useQuery } from "convex/react"; +import { useAuth } from "@clerk/nextjs"; +import { PlayerBar } from "@/components/player/player-bar"; +import { PlayerProvider } from "@/components/player/player-provider"; +import { Sidebar, useSidebar } from "@/components/sidebar/sidebar"; +import { NewChatButton } from "@/components/sidebar/new-chat-button"; +import { SearchBar } from "@/components/sidebar/search-bar"; +import { CratesSection } from "@/components/sidebar/crates-section"; +import { StarredSection } from "@/components/sidebar/starred-section"; +import { RecentsSection } from "@/components/sidebar/recents-section"; +import { ArtifactsSection } from "@/components/sidebar/artifacts-section"; +import { useKeyboardShortcuts } from "@/hooks/use-keyboard-shortcuts"; +import { api } from "../../../convex/_generated/api"; +import { Id } from "../../../convex/_generated/dataModel"; + +function WorkspaceInner({ children }: { children: React.ReactNode }) { + const { toggle } = useSidebar(); + const searchRef = useRef(null); + const router = useRouter(); + const params = useParams(); + const { userId: clerkId } = useAuth(); + const user = useQuery(api.users.getByClerkId, clerkId ? { clerkId } : "skip"); + const createSession = useMutation(api.sessions.create); + const toggleStar = useMutation(api.sessions.toggleStar); + + const handleNewChat = useCallback(async () => { + if (!user) return; + const id = await createSession({ userId: user._id }); + router.push(`/w/${id}`); + }, [user, createSession, router]); + + const handleToggleStar = useCallback(async () => { + const sessionId = params?.sessionId as string | undefined; + if (!sessionId) return; + await toggleStar({ id: sessionId as Id<"sessions"> }); + }, [params, toggleStar]); + + useKeyboardShortcuts({ + onNewChat: handleNewChat, + onToggleSidebar: toggle, + onFocusSearch: () => searchRef.current?.focus(), + onToggleStar: handleToggleStar, + }); + + return ( + <> + + +
+ + + + +
+ + ); +} + +export default function WorkspaceLayout({ + children, +}: { + children: React.ReactNode; +}) { + return ( + +
+ + {children} + +
+
{children}
+ +
+
+
+ ); +} +``` + +**Step 2: Make SearchBar accept a forwarded ref** + +Update `src/components/sidebar/search-bar.tsx` to use `forwardRef`: + +Add at the top: +```tsx +import { useState, useEffect, useRef, forwardRef, useImperativeHandle } from "react"; +``` + +Change the component to: +```tsx +export const SearchBar = forwardRef(function SearchBar(_props, ref) { + // ... existing code ... + const inputRef = useRef(null); + + useImperativeHandle(ref, () => inputRef.current!, []); + + // ... rest unchanged, but use inputRef internally ... +}); +``` + +**Step 3: Delete unused components** + +```bash +rm src/components/workspace/split-pane.tsx +rm src/components/workspace/navbar.tsx +``` + +**Step 4: Verify it builds** + +Run: `cd /Users/tarikmoody/Documents/Projects/crate-web && npx next build 2>&1 | tail -20` + +Expected: Build succeeds with no import errors. If `split-pane` or `navbar` are imported elsewhere, update those imports. + +**Step 5: Commit** + +```bash +git add -A +git commit -m "feat: wire keyboard shortcuts, clean up unused navbar and split-pane" +``` + +--- + +## Post-Implementation Verification + +After all tasks complete, verify: + +1. **Sidebar**: Collapses/expands, shows Crates/Starred/Recents/Artifacts +2. **New Chat**: Creates session in Convex, navigates to `/w/[sessionId]` +3. **Chat Persistence**: Messages appear after page reload +4. **Session Title**: Auto-set from first user message +5. **Starring**: Star icon toggles, session appears/disappears in Starred section +6. **Artifacts**: Generated artifacts appear in sidebar Artifacts section +7. **Artifact Slide-In**: Panel slides in on generation, dismisses with X +8. **Search**: Finds messages by content, grouped by session +9. **Keyboard Shortcuts**: `Cmd+K/N/B/Shift+S` all work +10. **Auth Scoping**: Different Clerk users see only their own data diff --git a/docs/plans/2026-03-12-show-prep-design.md b/docs/plans/2026-03-12-show-prep-design.md new file mode 100644 index 0000000..7b14422 --- /dev/null +++ b/docs/plans/2026-03-12-show-prep-design.md @@ -0,0 +1,223 @@ +# Crate Show Prep — Feature Design + +> **For Claude:** REQUIRED SUB-SKILL: Use superpowers:writing-plans to create the implementation plan from this design. + +**Goal:** Turn a pasted setlist into a structured, station-voiced show prep package rendered as a single OpenUI artifact — music context, talk breaks, social copy, local tie-ins, and interview prep. + +**Architecture:** Skill + OpenUI components. No new MCP server, no new Convex tables, no dedicated route in Phase 1. The skill orchestrates existing tools (Discogs, MusicBrainz, Genius, Bandcamp, Last.fm, Ticketmaster, web search, news) and outputs a ShowPrepPackage artifact. Station voice comes from YAML profiles. + +**Tech Stack:** Crate CLI skill system (SKILL.md), OpenUI Lang components, YAML station profiles, existing MCP servers. + +--- + +## Stations Served + +Three Radio Milwaukee stations, each with distinct voice and audience: + +| Station | Voice | Music Focus | Talk Style | +|---------|-------|-------------|------------| +| **88Nine** | Warm, eclectic, community-forward | Indie, alternative, world, electronic, hip-hop | Discovery-oriented, "let me tell you about this artist" | +| **HYFIN** | Bold, culturally sharp, unapologetic | Urban alternative, neo-soul, progressive hip-hop, Afrobeats | Cultural context, movement-building, "here's why this matters" | +| **Rhythm Lab** | Curated, global perspective, deep knowledge | Global beats, electronic, jazz fusion, experimental | Influence tracing, crate-digging stories, "the thread connecting these sounds" | + +--- + +## Input Flow + +DJs paste their setlist directly in the chat: + +``` +Prep my evening show for HYFIN: +Khruangbin - Time (You and I) +Little Simz - Gorilla +KAYTRANADA - Glued +``` + +The skill parses station, shift, and "Artist - Title" lines from the message body. If no tracks are provided, the skill asks for them. + +Single-track quick mode also supported: +``` +show prep track Khruangbin - Time (You and I) for HYFIN +``` + +--- + +## Skill Workflow + +**Step 1: Parse** — Extract station name, shift (morning/midday/afternoon/evening/overnight), and track list from the message. + +**Step 2: Resolve** — For each track, parallel lookups: +- MusicBrainz → canonical metadata, producer/engineer credits, recording relationships +- Discogs → release year, label, catalog number, pressing details +- Genius → song annotations, verified artist commentary, production context +- Bandcamp → artist statements, liner notes, community tags +- Last.fm → similar artists, listener stats, top tags + +**Step 3: Synthesize** — Merge results per track into TrackContext: +- Origin story (how this track came to be) +- Production notes (studio, producer, notable instruments) +- Connections (influences, samples, collaborations) +- Influence chain (musical lineage via Crate's influence tracer) +- Lesser-known fact (the detail listeners can't easily Google) +- Apply "why it matters" filter (Rule 1) and audience relevance ranking (Rule 6) + +**Step 4: Format** — Load station YAML profile. Generate: +- Talk breaks in 30s/60s/deep variants using station voice +- Social copy per platform (Instagram, X, Bluesky) with station hashtags +- Local tie-ins from Milwaukee sources (see below) and Ticketmaster events + +**Step 5: Assemble** — Output a single `ShowPrepPackage` OpenUI artifact. Chat shows progress text during research. Artifact appears in slide-in panel when complete. + +--- + +## Milwaukee Local Sources + +Integrated via RSS feeds and web search during Step 4 for hyper-local tie-ins: + +| Source | URL | Content | +|--------|-----|---------| +| **Milwaukee Record** | milwaukeerecord.com | Local music coverage, venue news, scene reports | +| **Journal Sentinel** | jsonline.com | Arts & entertainment, community events | +| **Urban Milwaukee** | urbanmilwaukee.com | Neighborhood news, cultural coverage, venue openings | +| **OnMilwaukee** | onmilwaukee.com | Events, food/arts/music intersections, city culture | +| **88Nine** | radiomilwaukee.org | Station news, local artist features, event calendar | +| **Shepherd Express** | shepherdexpress.com | Alt-weekly, music reviews, local show listings | + +These surface: +- **Event tie-ins:** "Catch [local artist] at Turner Hall Saturday — similar vibes to that Khruangbin track" +- **Community spotlights:** Local organizations, openings, milestones relevant to the audience +- **Neighborhood callouts:** Bay View, Riverwest, Walker's Point, Bronzeville cultural moments +- **Seasonal hooks:** Festival previews, seasonal context, weather-appropriate transitions + +--- + +## OpenUI Components + +Five new components added to `src/lib/openui/components.tsx`: + +### ShowPrepPackage +``` +ShowPrepPackage(station, date, dj, shift, tracks, talkBreaks, socialPosts, interviewPreps?) +``` +Top-level container. Station badge with color (HYFIN = gold, 88Nine = blue, Rhythm Lab = purple). Date, DJ name, shift. Children are arrays of cards. Collapsible sections per track. + +### TrackContextCard +``` +TrackContextCard(artist, title, originStory, productionNotes, connections, influenceChain, lesserKnownFact, whyItMatters, audienceRelevance, localTieIn?, pronunciationGuide?, imageUrl?) +``` +Core card per track. Album art + play button. Relevance badge (high = green, medium = yellow, low = gray). `whyItMatters` always visible as the headline. Expandable sections for origin story, production notes, influence chain. + +### TalkBreakCard +``` +TalkBreakCard(type, beforeTrack, afterTrack, shortVersion, mediumVersion, longVersion, keyPhrases, timingCue?, pronunciationGuide?) +``` +Type badge (intro/back-announce/transition/feature). Three tabs for short/medium/long variants. Key phrases bolded. Copy button per variant. + +### SocialPostCard +``` +SocialPostCard(trackOrTopic, instagram, twitter, bluesky, hashtags) +``` +Three platform tabs with pre-formatted copy. Copy button per platform. Station-specific hashtags as pills. + +### InterviewPrepCard +``` +InterviewPrepCard(guestName, warmUpQuestions, deepDiveQuestions, localQuestions, avoidQuestions) +``` +Three question categories as expandable sections. "Avoid" section in muted red — common overasked questions flagged. Only generated when DJ mentions an interview or guest. + +--- + +## Station YAML Profile Structure + +Each station is a YAML file at `src/skills/show-prep/stations/{station}.yaml`: + +```yaml +name: HYFIN +tagline: "Black alternative radio" +color: "#D4A843" + +voice: + tone: "Bold, culturally sharp, unapologetic" + perspective: "Cultural context, movement-building, 'here's why this matters'" + music_focus: "Urban alternative, neo-soul, progressive hip-hop, Afrobeats" + vocabulary: + prefer: ["culture", "movement", "lineage", "vibration", "frequency"] + avoid: ["urban (standalone)", "exotic", "ethnic"] + +defaults: + break_length: medium + depth: deep_cultural_context + audience: "Young, culturally aware Milwaukee listeners invested in Black art and music" + +social: + hashtags: ["#HYFIN", "#MKE", "#BlackAlternative"] + tone: "Confident, community-first" + +recurring_features: + - name: "The Lineage" + description: "Influence chain connecting today's new release to its roots" + frequency: daily + - name: "Culture Check" + description: "Arts/culture moment from Black and brown communities locally and globally" + frequency: daily + +local: + venues: ["Turner Hall Ballroom", "Vivarium", "The Cooperage", "Cactus Club"] + neighborhoods: ["Bronzeville", "Riverwest", "Bay View", "Walker's Point"] + market: "Milwaukee, WI" + sources: ["milwaukeerecord.com", "jsonline.com", "urbanmilwaukee.com", "onmilwaukee.com"] +``` + +DJs can override per-session in the chat: "prep my show for HYFIN but keep it shorter than usual." + +--- + +## Radio Milwaukee Show Prep Rules (Design Foundation) + +Every feature must serve at least one of these six rules, already posted in the Radio Milwaukee studio: + +1. **Always ask why the information matters to the listener. Avoid slot filling.** → `whyItMatters` field on every TrackContextCard. No generic filler. +2. **Know the audience. Keep content audience-focused.** → Station YAML profiles shape all generated content. +3. **Test drive the content. Practice. Do not do the show before the show.** → Talk breaks are starting points, not scripts. Multiple lengths for the DJ to choose and develop. +4. **Schedule your music in advance to allow time for prep.** → Playlist-in workflow is the primary input. Also supports reverse lookup: "find songs for this break idea." +5. **Go with the moment. Be flexible.** → Modular cards, not monolithic document. Single-track quick mode for mid-show pivots. +6. **When selecting a topic, consider if the audience wants it.** → `audienceRelevance` ranking (high/medium/low) on every card. + +--- + +## Phasing + +### Phase 1 — Core Skill + Components (ship first) +- `show-prep` SKILL.md with triggers, workflow, tool priority +- 3 station YAML profiles (88Nine, HYFIN, Rhythm Lab) +- 5 OpenUI components (ShowPrepPackage, TrackContextCard, TalkBreakCard, SocialPostCard, InterviewPrepCard) +- OpenUI prompt additions documenting new components +- Setlist parsing from inline paste +- Single-track quick mode +- Test with Rhythm Lab (Tarik's own show = fastest feedback loop) + +### Phase 2 — Polish + Local Context +- Milwaukee local tie-ins via RSS feeds, web search, Ticketmaster +- Recurring feature generation (The Lineage, Deep Cut Daily, Crate Connection) +- Social copy refinement with station-specific hashtag conventions +- Interview prep triggered by guest mention +- Reverse lookup: "find songs for a break about [topic] for [station]" + +### Phase 3 — Dedicated Web View (later) +- `/show-prep` route with station selector, date picker, shift selector +- Paste-friendly setlist input area with track search/add +- Pre-fills agent chat with structured context +- Export to PDF/Markdown for studio printing +- Show prep history in sidebar + +--- + +## What We're NOT Building in Phase 1 + +- No new MCP server — skill orchestrates existing tools +- No new Convex tables — prep packages save as artifacts (already persist) +- No dedicated web route — chat-first, artifact panel is the view +- No Zetta integration — paste input only +- No 414 Music station — three stations only +- No PDF export — Phase 3 +- No multi-agent orchestration — single agent with skill instructions diff --git a/docs/plans/2026-03-12-show-prep-plan.md b/docs/plans/2026-03-12-show-prep-plan.md new file mode 100644 index 0000000..7af83c7 --- /dev/null +++ b/docs/plans/2026-03-12-show-prep-plan.md @@ -0,0 +1,947 @@ +# Crate Show Prep — Phase 1 Implementation Plan + +> **For Claude:** REQUIRED SUB-SKILL: Use superpowers:executing-plans to implement this plan task-by-task. + +**Goal:** Build a show prep skill that turns a pasted setlist into a structured, station-voiced prep package rendered as a single OpenUI artifact with track context, talk breaks, social copy, and interview prep. + +**Architecture:** A SKILL.md in crate-cli defines the research workflow and triggers. Three YAML station profiles configure voice/tone. Five new OpenUI components in crate-web render the prep package as an artifact. A `/show-prep` slash command in crate-web's chat route preprocesses input. The existing `/news` command gains station-aware customization using the same YAML profiles. + +**Tech Stack:** TypeScript, Crate CLI skill system (SKILL.md + YAML), OpenUI Lang (`@openuidev/react-lang` + `defineComponent` + Zod), React (client components), Vitest (tests), Tailwind CSS. + +--- + +### Task 1: Station YAML Profiles + +**Files:** +- Create: `crate-cli/src/skills/show-prep/stations/88nine.yaml` +- Create: `crate-cli/src/skills/show-prep/stations/hyfin.yaml` +- Create: `crate-cli/src/skills/show-prep/stations/rhythmlab.yaml` + +**Step 1: Create the directory structure** + +```bash +mkdir -p /Users/tarikmoody/Documents/Projects/crate-cli/src/skills/show-prep/stations +``` + +**Step 2: Create `88nine.yaml`** + +```yaml +name: 88Nine +tagline: "Discover the sound of Milwaukee" +color: "#3B82F6" + +voice: + tone: "Warm, eclectic, community-forward" + perspective: "Discovery-oriented, 'let me tell you about this artist'" + music_focus: "Indie, alternative, world, electronic, hip-hop" + vocabulary: + prefer: ["discover", "connect", "community", "eclectic", "homegrown"] + avoid: ["mainstream", "generic", "commercial"] + +defaults: + break_length: medium + depth: standard + audience: "Milwaukee music lovers who value discovery and local culture" + +social: + hashtags: ["#88Nine", "#RadioMilwaukee", "#MKE", "#DiscoverMusic"] + tone: "Warm, inviting, curious" + +recurring_features: + - name: "Deep Cut Daily" + description: "One overlooked track from an artist currently in rotation, with backstory" + frequency: daily + - name: "Milwaukee Made Monday" + description: "Local artist spotlight to start the week" + frequency: weekly + - name: "Sample Source" + description: "Trace a sample back to its origin" + frequency: weekly + +local: + venues: ["Turner Hall Ballroom", "Vivarium", "The Cooperage", "Cactus Club", "Riverside Theatre", "Pabst Theater"] + neighborhoods: ["Bay View", "Riverwest", "Walker's Point", "East Side", "Third Ward"] + market: "Milwaukee, WI" + sources: ["milwaukeerecord.com", "jsonline.com", "urbanmilwaukee.com", "onmilwaukee.com", "radiomilwaukee.org", "shepherdexpress.com"] +``` + +**Step 3: Create `hyfin.yaml`** + +```yaml +name: HYFIN +tagline: "Black alternative radio" +color: "#D4A843" + +voice: + tone: "Bold, culturally sharp, unapologetic" + perspective: "Cultural context, movement-building, 'here's why this matters'" + music_focus: "Urban alternative, neo-soul, progressive hip-hop, Afrobeats" + vocabulary: + prefer: ["culture", "movement", "lineage", "vibration", "frequency"] + avoid: ["urban (standalone)", "exotic", "ethnic"] + +defaults: + break_length: medium + depth: deep_cultural_context + audience: "Young, culturally aware Milwaukee listeners invested in Black art and music" + +social: + hashtags: ["#HYFIN", "#MKE", "#BlackAlternative"] + tone: "Confident, community-first" + +recurring_features: + - name: "The Lineage" + description: "Influence chain connecting today's new release to its roots" + frequency: daily + - name: "Culture Check" + description: "Arts/culture moment from Black and brown communities locally and globally" + frequency: daily + +local: + venues: ["Turner Hall Ballroom", "Vivarium", "The Cooperage", "Cactus Club"] + neighborhoods: ["Bronzeville", "Riverwest", "Bay View", "Walker's Point"] + market: "Milwaukee, WI" + sources: ["milwaukeerecord.com", "jsonline.com", "urbanmilwaukee.com", "onmilwaukee.com"] +``` + +**Step 4: Create `rhythmlab.yaml`** + +```yaml +name: Rhythm Lab +tagline: "Where the crates run deep" +color: "#8B5CF6" + +voice: + tone: "Curated, global perspective, deep knowledge" + perspective: "Influence tracing, crate-digging stories, 'the thread connecting these sounds'" + music_focus: "Global beats, electronic, jazz fusion, experimental, Afrobeats, dub" + vocabulary: + prefer: ["lineage", "crate", "connection", "thread", "sonic", "palette"] + avoid: ["world music (reductive)", "niche", "obscure (dismissive)"] + +defaults: + break_length: long + depth: deep_music_history + audience: "Dedicated music heads, DJs, producers, and crate diggers who value context and connection" + +social: + hashtags: ["#RhythmLab", "#CrateDigging", "#GlobalBeats", "#MKE"] + tone: "Knowledgeable, passionate, collegial" + +recurring_features: + - name: "Crate Connection" + description: "How two seemingly unrelated tracks share DNA" + frequency: daily + - name: "Global Dispatch" + description: "Music from a specific city/region with cultural context" + frequency: weekly + - name: "The Remix Tree" + description: "Track a song through its remix/cover/sample ecosystem" + frequency: weekly + +local: + venues: ["Turner Hall Ballroom", "Vivarium", "The Cooperage", "Cactus Club", "Jazz Estate"] + neighborhoods: ["Riverwest", "Bay View", "Bronzeville", "Walker's Point"] + market: "Milwaukee, WI" + sources: ["milwaukeerecord.com", "jsonline.com", "urbanmilwaukee.com", "onmilwaukee.com"] +``` + +**Step 5: Commit** + +```bash +cd /Users/tarikmoody/Documents/Projects/crate-cli +git add src/skills/show-prep/stations/ +git commit -m "feat(show-prep): add station YAML profiles for 88Nine, HYFIN, Rhythm Lab" +``` + +--- + +### Task 2: Show Prep SKILL.md + +**Files:** +- Create: `crate-cli/src/skills/show-prep/SKILL.md` + +**Step 1: Create the skill file** + +The skill follows the exact same frontmatter + body pattern as `artist-deep-dive/SKILL.md`. The registry auto-discovers it from `src/skills/show-prep/SKILL.md`. + +```markdown +--- +name: show-prep +description: Radio show preparation — generates station-voiced track context, talk breaks, social copy, and interview prep from a pasted setlist +triggers: + - "show prep" + - "prep my show" + - "prepare my show" + - "prep my set" + - "show preparation" + - "dj prep" + - "radio prep" +tools_priority: [musicbrainz, discogs, genius, bandcamp, lastfm, ticketmaster, websearch, news] +--- + +## Station Profiles + +Load the station YAML profile matching the user's request (88nine, hyfin, or rhythmlab). +If no station is specified, ask which station before proceeding. +The profile defines voice tone, vocabulary, break length defaults, social hashtags, recurring features, and local context. + +Available stations: +- **88Nine** — Warm, eclectic, community-forward. Indie, alternative, world, electronic, hip-hop. +- **HYFIN** — Bold, culturally sharp, unapologetic. Urban alternative, neo-soul, progressive hip-hop, Afrobeats. +- **Rhythm Lab** — Curated, global perspective, deep knowledge. Global beats, electronic, jazz fusion, experimental. + +## Input Parsing + +Parse the user's message for: +1. **Station name** — "for HYFIN", "for 88nine", "for rhythm lab" +2. **Shift** — morning, midday, afternoon, evening, overnight (default: evening) +3. **DJ name** — "DJ [name]" or infer from user context +4. **Track list** — Lines matching "Artist - Title" or "Artist — Title" pattern +5. **Interview guest** — "interviewing [artist]" or "guest: [artist]" + +If tracks are provided, proceed with full prep. If not, ask for the setlist. + +## Workflow + +### Per-Track Research (parallel for each track) + +1. **MusicBrainz** `search_recording` + `get_recording_credits` — canonical metadata, producer, engineer, studio +2. **Discogs** `search_discogs` + `get_release_full` — release year, label, catalog number, album context +3. **Genius** `search_songs` + `get_song` — annotations, verified artist commentary, production context +4. **Bandcamp** `search_bandcamp` + `get_album` — artist statements, liner notes, community tags, independent status +5. **Last.fm** `get_track_info` + `get_similar_tracks` — listener stats, similar tracks, top tags + +### Synthesis (per track) + +From the raw data, generate: +- **Origin story** — 2-3 sentences on how this track came to be. Not Wikipedia summary — the interesting backstory. +- **Production notes** — Key production details (studio, producer, notable instruments, sonic signature). +- **Connections** — Influences, samples, collaborations, genre lineage. Use influence tracer if available. +- **Lesser-known fact** — The detail listeners can't easily Google. Dig into Genius annotations and Discogs credits. +- **Why it matters** — One sentence answering: why should THIS audience care about this track RIGHT NOW? (Rule 1) +- **Audience relevance** — high / medium / low based on how well the track fits the station's audience profile (Rule 6) +- **Local tie-in** — Check Ticketmaster for upcoming Milwaukee shows by this artist. Search Milwaukee sources for any local connection. + +### Talk Break Generation + +For each transition point between tracks, generate talk breaks in the station's voice: +- **Short (10-15 sec)** — Quick context before the vocal kicks in +- **Medium (30-60 sec)** — "That was..." with a compelling detail plus segue to next track +- **Long (60-120 sec)** — Fuller backstory connecting the two tracks, with local tie-in if available + +Bold the key phrases — the parts that really land on air. +Include pronunciation guides for unfamiliar artist/track names. + +### Social Copy + +For each track (or the show overall), generate platform-specific posts: +- **Instagram** — Visual-first, 1-2 sentences, station hashtags +- **X/Twitter** — Punchy, single line + hashtag +- **Bluesky** — Conversational, community-oriented + +Never reproduce lyrics. Tone matches the station profile. + +### Interview Prep (only if guest mentioned) + +If the DJ mentions interviewing a guest: +1. Pull comprehensive artist data from all sources +2. Generate questions in three categories: warm-up, music deep-dive, Milwaukee connection +3. Flag common overasked questions to avoid + +## Output Format + +Output a SINGLE ShowPrepPackage OpenUI component containing all TrackContextCards, TalkBreakCards, SocialPostCards, and InterviewPrepCards as children. This renders as one browsable artifact in the slide-in panel. + +## Radio Milwaukee Show Prep Rules + +Apply these rules to ALL generated content: +1. Every piece must answer "why does the listener care?" — no slot filling +2. Content is shaped by the station's audience profile +3. Talk breaks are starting points for DJs to develop — not scripts to read +4. Prep is tied to the actual setlist the DJ will play +5. Content is modular — DJs can skip, swap, or reorder cards +6. Rank content by audience relevance — surface the best angles first +``` + +**Step 2: Verify the skill loads** + +```bash +cd /Users/tarikmoody/Documents/Projects/crate-cli +npx tsx -e " +import { SkillRegistry } from './src/skills/registry.js'; +const reg = new SkillRegistry(); +await reg.loadAll(); +const match = reg.matchQuery('prep my show for HYFIN'); +console.log('Matched:', match?.name); +console.log('Triggers:', match?.triggers?.length); +console.log('All skills:', reg.listSkills().map(s => s.name)); +" +``` + +Expected: `Matched: show-prep`, triggers count of 7, listed among all skills. + +**Step 3: Commit** + +```bash +cd /Users/tarikmoody/Documents/Projects/crate-cli +git add src/skills/show-prep/SKILL.md +git commit -m "feat(show-prep): add show-prep skill with research workflow and station-aware voice" +``` + +--- + +### Task 3: OpenUI Components — TrackContextCard + +**Files:** +- Modify: `crate-web/src/lib/openui/components.tsx` +- Test: `crate-web/tests/show-prep-components.test.tsx` (create) + +**Step 1: Add TrackContextCard component** + +Add to the bottom of `crate-web/src/lib/openui/components.tsx`, before the closing of the file: + +```tsx +// ── Show Prep Components ──────────────────────────────────────── + +export const TrackContextCard = defineComponent({ + name: "TrackContextCard", + description: + "Show prep context card for a single track — origin story, production notes, talk break suggestions, local tie-in, and audience relevance.", + props: z.object({ + artist: z.string().describe("Artist name"), + title: z.string().describe("Track title"), + originStory: z.string().describe("2-3 sentence backstory of how this track came to be"), + productionNotes: z.string().describe("Key production details — studio, producer, notable instruments"), + connections: z.string().describe("Influences, samples, collaborations, genre lineage"), + influenceChain: z.string().optional().describe("Musical lineage chain, e.g. 'Thai funk → Khruangbin → modern psych-soul'"), + lesserKnownFact: z.string().describe("Detail listeners can't easily Google"), + whyItMatters: z.string().describe("One sentence: why should the listener care about this right now?"), + audienceRelevance: z.enum(["high", "medium", "low"]).describe("How well this track fits the station's audience"), + localTieIn: z.string().optional().describe("Milwaukee-specific connection — upcoming shows, local artist tie-in"), + pronunciationGuide: z.string().optional().describe("Pronunciation help for unfamiliar names"), + imageUrl: z.string().optional().describe("Album art URL"), + }), + component: ({ props }) => { + const [expanded, setExpanded] = useState(false); + const relevanceColor = { + high: "bg-green-500/20 text-green-400", + medium: "bg-yellow-500/20 text-yellow-400", + low: "bg-zinc-500/20 text-zinc-400", + }[props.audienceRelevance]; + + return ( +
+
+ {props.imageUrl && ( + + )} +
+
+ +

{props.artist} — {props.title}

+ + {props.audienceRelevance} + +
+

{props.whyItMatters}

+ {props.pronunciationGuide && ( +

🗣 {props.pronunciationGuide}

+ )} +
+
+ + + + {expanded && ( +
+
+

Origin Story

+

{props.originStory}

+
+
+

Production Notes

+

{props.productionNotes}

+
+
+

Connections

+

{props.connections}

+
+ {props.influenceChain && ( +
+

Influence Chain

+

{props.influenceChain}

+
+ )} +
+

Lesser-Known Fact

+

{props.lesserKnownFact}

+
+ {props.localTieIn && ( +
+

Milwaukee Connection

+

{props.localTieIn}

+
+ )} +
+ )} +
+ ); + }, +}); +``` + +**Step 2: Commit** + +```bash +cd /Users/tarikmoody/Documents/Projects/crate-web +git add src/lib/openui/components.tsx +git commit -m "feat(show-prep): add TrackContextCard OpenUI component" +``` + +--- + +### Task 4: OpenUI Components — TalkBreakCard + +**Files:** +- Modify: `crate-web/src/lib/openui/components.tsx` + +**Step 1: Add TalkBreakCard component** + +Add after TrackContextCard in the same file: + +```tsx +export const TalkBreakCard = defineComponent({ + name: "TalkBreakCard", + description: + "Talk break card with short/medium/long variants. Type badge shows intro, back-announce, transition, or feature.", + props: z.object({ + type: z.enum(["intro", "back-announce", "transition", "feature"]).describe("Break type"), + beforeTrack: z.string().describe("Track playing before this break"), + afterTrack: z.string().describe("Track playing after this break"), + shortVersion: z.string().describe("10-15 second version — quick hook"), + mediumVersion: z.string().describe("30-60 second version — fuller context"), + longVersion: z.string().describe("60-120 second version — deep backstory"), + keyPhrases: z.string().describe("Comma-separated key phrases to emphasize on air"), + timingCue: z.string().optional().describe("e.g. 'Hit this before the vocal at 0:08'"), + pronunciationGuide: z.string().optional().describe("Pronunciation help for names"), + }), + component: ({ props }) => { + const [tab, setTab] = useState<"short" | "medium" | "long">("medium"); + const [copied, setCopied] = useState(false); + + const typeBadge = { + intro: "bg-blue-500/20 text-blue-400", + "back-announce": "bg-green-500/20 text-green-400", + transition: "bg-purple-500/20 text-purple-400", + feature: "bg-amber-500/20 text-amber-400", + }[props.type]; + + const content = { short: props.shortVersion, medium: props.mediumVersion, long: props.longVersion }[tab]; + + const handleCopy = async () => { + await navigator.clipboard.writeText(content); + setCopied(true); + setTimeout(() => setCopied(false), 2000); + }; + + return ( +
+
+
+ + {props.type} + + + {props.beforeTrack} → {props.afterTrack} + +
+ +
+ +
+ {(["short", "medium", "long"] as const).map((t) => ( + + ))} +
+ +

{content}

+ + {props.keyPhrases && ( +
+ {props.keyPhrases.split(",").map((phrase) => ( + + {phrase.trim()} + + ))} +
+ )} + + {props.timingCue && ( +

⏱ {props.timingCue}

+ )} + {props.pronunciationGuide && ( +

🗣 {props.pronunciationGuide}

+ )} +
+ ); + }, +}); +``` + +**Step 2: Commit** + +```bash +cd /Users/tarikmoody/Documents/Projects/crate-web +git add src/lib/openui/components.tsx +git commit -m "feat(show-prep): add TalkBreakCard OpenUI component with short/medium/long tabs" +``` + +--- + +### Task 5: OpenUI Components — SocialPostCard + +**Files:** +- Modify: `crate-web/src/lib/openui/components.tsx` + +**Step 1: Add SocialPostCard component** + +```tsx +export const SocialPostCard = defineComponent({ + name: "SocialPostCard", + description: + "Social media copy card with platform tabs (Instagram, X, Bluesky). Copy button per platform. Station-specific hashtags.", + props: z.object({ + trackOrTopic: z.string().describe("Track name or topic this post is about"), + instagram: z.string().describe("Instagram post copy"), + twitter: z.string().describe("X/Twitter post copy"), + bluesky: z.string().describe("Bluesky post copy"), + hashtags: z.string().describe("Comma-separated hashtags"), + }), + component: ({ props }) => { + const [tab, setTab] = useState<"instagram" | "twitter" | "bluesky">("instagram"); + const [copied, setCopied] = useState(false); + + const content = { instagram: props.instagram, twitter: props.twitter, bluesky: props.bluesky }[tab]; + + const handleCopy = async () => { + const hashtagStr = props.hashtags.split(",").map((h) => h.trim()).join(" "); + await navigator.clipboard.writeText(`${content}\n\n${hashtagStr}`); + setCopied(true); + setTimeout(() => setCopied(false), 2000); + }; + + return ( +
+
+ {props.trackOrTopic} + +
+ +
+ {(["instagram", "twitter", "bluesky"] as const).map((p) => ( + + ))} +
+ +

{content}

+ +
+ {props.hashtags.split(",").map((tag) => ( + + {tag.trim()} + + ))} +
+
+ ); + }, +}); +``` + +**Step 2: Commit** + +```bash +cd /Users/tarikmoody/Documents/Projects/crate-web +git add src/lib/openui/components.tsx +git commit -m "feat(show-prep): add SocialPostCard OpenUI component with platform tabs" +``` + +--- + +### Task 6: OpenUI Components — InterviewPrepCard + +**Files:** +- Modify: `crate-web/src/lib/openui/components.tsx` + +**Step 1: Add InterviewPrepCard component** + +```tsx +export const InterviewPrepCard = defineComponent({ + name: "InterviewPrepCard", + description: + "Interview preparation card with warm-up, deep-dive, and local questions. Flags overasked questions to avoid.", + props: z.object({ + guestName: z.string().describe("Guest artist or interviewee name"), + warmUpQuestions: z.string().describe("Easy personality-revealing openers, one per line"), + deepDiveQuestions: z.string().describe("Questions about craft, process, specific tracks, one per line"), + localQuestions: z.string().describe("Milwaukee connection angles, one per line"), + avoidQuestions: z.string().describe("Common overasked questions to skip, one per line"), + }), + component: ({ props }) => { + const [section, setSection] = useState<"warmup" | "deep" | "local" | "avoid">("warmup"); + + const renderQuestions = (text: string, color: string) => ( +
    + {text.split("\n").filter(Boolean).map((q, i) => ( +
  • • {q.replace(/^[-•]\s*/, "")}
  • + ))} +
+ ); + + return ( +
+

Interview Prep: {props.guestName}

+ +
+ {([ + ["warmup", "Warm-up"], + ["deep", "Deep Dive"], + ["local", "Milwaukee"], + ["avoid", "Avoid"], + ] as const).map(([key, label]) => ( + + ))} +
+ + {section === "warmup" && renderQuestions(props.warmUpQuestions, "text-zinc-300")} + {section === "deep" && renderQuestions(props.deepDiveQuestions, "text-zinc-300")} + {section === "local" && renderQuestions(props.localQuestions, "text-cyan-400/80")} + {section === "avoid" && renderQuestions(props.avoidQuestions, "text-red-400/70")} +
+ ); + }, +}); +``` + +**Step 2: Commit** + +```bash +cd /Users/tarikmoody/Documents/Projects/crate-web +git add src/lib/openui/components.tsx +git commit -m "feat(show-prep): add InterviewPrepCard OpenUI component" +``` + +--- + +### Task 7: OpenUI Components — ShowPrepPackage (container) + +**Files:** +- Modify: `crate-web/src/lib/openui/components.tsx` + +**Step 1: Add ShowPrepPackage container component** + +```tsx +export const ShowPrepPackage = defineComponent({ + name: "ShowPrepPackage", + description: + "Top-level show prep container. Station badge, date, DJ name, shift. Children are TrackContextCards, TalkBreakCards, SocialPostCards, and optionally InterviewPrepCards.", + props: z.object({ + station: z.string().describe("Station name: 88Nine, HYFIN, or Rhythm Lab"), + date: z.string().describe("Show date, e.g. 'Wednesday, March 12'"), + dj: z.string().describe("DJ name"), + shift: z.string().describe("Shift: morning, midday, afternoon, evening, overnight"), + tracks: z.array(TrackContextCard.ref).describe("Track context cards"), + talkBreaks: z.array(TalkBreakCard.ref).describe("Talk break cards"), + socialPosts: z.array(SocialPostCard.ref).describe("Social media post cards"), + interviewPreps: z.array(InterviewPrepCard.ref).optional().describe("Interview prep cards (if guest mentioned)"), + }), + component: ({ props, renderNode }) => { + const stationColor: Record = { + "88Nine": "bg-blue-500/20 text-blue-400 border-blue-500/30", + "HYFIN": "bg-amber-500/20 text-amber-400 border-amber-500/30", + "Rhythm Lab": "bg-purple-500/20 text-purple-400 border-purple-500/30", + }; + const colorClass = stationColor[props.station] || "bg-zinc-500/20 text-zinc-400 border-zinc-500/30"; + + return ( +
+
+
+ + {props.station} + + {props.shift} shift +
+
+

{props.dj}

+

{props.date}

+
+
+ + {props.tracks && ( +
+

Track Context

+
{renderNode(props.tracks)}
+
+ )} + + {props.talkBreaks && ( +
+

Talk Breaks

+
{renderNode(props.talkBreaks)}
+
+ )} + + {props.socialPosts && ( +
+

Social Copy

+
{renderNode(props.socialPosts)}
+
+ )} + + {props.interviewPreps && ( +
+

Interview Prep

+
{renderNode(props.interviewPreps)}
+
+ )} +
+ ); + }, +}); +``` + +**Step 2: Commit** + +```bash +cd /Users/tarikmoody/Documents/Projects/crate-web +git add src/lib/openui/components.tsx +git commit -m "feat(show-prep): add ShowPrepPackage container OpenUI component" +``` + +--- + +### Task 8: OpenUI Prompt Additions + +**Files:** +- Modify: `crate-web/src/lib/openui/prompt.ts` + +**Step 1: Add show prep component documentation to the OpenUI Lang prompt** + +Add the following section before the `### Rules` section in the `OPENUI_LANG_PROMPT` string: + +``` +**TrackContextCard(artist, title, originStory, productionNotes, connections, influenceChain?, lesserKnownFact, whyItMatters, audienceRelevance, localTieIn?, pronunciationGuide?, imageUrl?)** +Show prep context for one track. audienceRelevance is "high", "medium", or "low". + +**TalkBreakCard(type, beforeTrack, afterTrack, shortVersion, mediumVersion, longVersion, keyPhrases, timingCue?, pronunciationGuide?)** +Talk break with short/medium/long variants. type is "intro", "back-announce", "transition", or "feature". keyPhrases is comma-separated. + +**SocialPostCard(trackOrTopic, instagram, twitter, bluesky, hashtags)** +Social media copy with platform tabs. hashtags is comma-separated. + +**InterviewPrepCard(guestName, warmUpQuestions, deepDiveQuestions, localQuestions, avoidQuestions)** +Interview prep with question categories. Each question field has one question per line. + +**ShowPrepPackage(station, date, dj, shift, tracks, talkBreaks, socialPosts, interviewPreps?)** +Top-level show prep container. `tracks` is array of TrackContextCard refs. `talkBreaks` is array of TalkBreakCard refs. `socialPosts` is array of SocialPostCard refs. `interviewPreps` is optional array of InterviewPrepCard refs. +``` + +Also add to the `### Rules` section: + +``` +- For show prep requests, ALWAYS output a ShowPrepPackage containing TrackContextCards, TalkBreakCards, and SocialPostCards. Generate one TrackContextCard per track in the setlist, talk breaks for each transition, and one SocialPostCard per track or for the show overall. +- When show prep includes an interview or guest mention, add InterviewPrepCards inside the ShowPrepPackage. +``` + +And add an example to `### Examples`: + +``` +Example 6 — Show prep package: +\`\`\` +root = ShowPrepPackage("HYFIN", "Wednesday, March 12", "Jordan Lee", "evening", [tc1, tc2], [tb1], [sp1], []) +tc1 = TrackContextCard("Khruangbin", "Time (You and I)", "Born from the trio's deep immersion in 1960s Thai funk...", "Recorded at their rural Texas barn studio with vintage Fender Rhodes...", "Thai funk → surf rock → psychedelic soul", "Thai funk cassettes → Khruangbin → modern psych-soul revival", "The band learned Thai from their Houston neighbor who introduced them to the music", "Khruangbin proves that the deepest musical connections cross every border — exactly what HYFIN is about", "high", "Playing Riverside Theatre March 22 — tickets still available", "crew-ANG-bin") +tc2 = TrackContextCard("Little Simz", "Gorilla", "Written during the sessions that would become her Mercury Prize-winning album...", "Produced by Inflo, the anonymous producer behind SAULT...", "UK hip-hop → grime → conscious rap", "Lauryn Hill → Ms. Dynamite → Little Simz", "Simz turned down every major label twice before signing on her own terms", "An independent Black woman in hip-hop who bet on herself and won the Mercury Prize — the HYFIN frequency personified", "high") +tb1 = TalkBreakCard("transition", "Time (You and I)", "Gorilla", "From Texas barn funk to London grime — two artists who built it themselves.", "That was Khruangbin taking you to Thailand via Texas. Now we're crossing the Atlantic to London where Little Simz turned down every major label — twice — to make the music she wanted. This is Gorilla.", "Khruangbin learned their sound from Thai funk cassettes a Houston neighbor shared with them. Little Simz learned hers by watching Lauryn Hill and deciding she'd rather own everything than compromise anything. Two completely different paths to the same place — uncompromising art on their own terms. That's the frequency.", "Texas barn funk, Thai cassettes, turned down every label, Mercury Prize", "Hit 'uncompromising art' before the beat drops at 0:04") +sp1 = SocialPostCard("Khruangbin → Little Simz", "From Thai funk to London grime. Tonight's HYFIN evening set traces the line from Khruangbin's Texas barn sessions to Little Simz's Mercury Prize-winning independence. Tune in.", "Thai funk cassettes → Texas barn → London grime → Mercury Prize. The thread connecting tonight's HYFIN set. 📻", "Tonight on HYFIN: how a Houston neighbor's Thai funk cassettes and a London rapper's refusal to sign connect across oceans. The frequency is real.", "#HYFIN, #MKE, #BlackAlternative, #Khruangbin, #LittleSimz") +\`\`\` +``` + +**Step 2: Commit** + +```bash +cd /Users/tarikmoody/Documents/Projects/crate-web +git add src/lib/openui/prompt.ts +git commit -m "feat(show-prep): add show prep components to OpenUI Lang prompt" +``` + +--- + +### Task 9: Slash Command — `/show-prep` and Customizable `/news` + +**Files:** +- Modify: `crate-web/src/app/api/chat/route.ts` + +**Step 1: Add `/show-prep` and enhance `/news` in `preprocessSlashCommand()`** + +Add these cases to the switch statement in `preprocessSlashCommand()`: + +```typescript + case "show-prep": + case "showprep": + case "prep": { + // Pass through with skill trigger prefix so the show-prep skill activates + // The skill parses station, shift, and tracks from the message body + if (!arg) { + return "Show prep — which station (88Nine, HYFIN, or Rhythm Lab) and what's your setlist?"; + } + return `Prep my show: ${arg}`; + } +``` + +For `/news`, enhance the existing case to support station customization: + +Replace the existing `case "news"` block with: + +```typescript + case "news": { + const parts = arg?.split(/\s+/) ?? []; + let count = 5; + let station = ""; + + for (const part of parts) { + const num = parseInt(part, 10); + if (!isNaN(num) && num >= 1 && num <= 5) { + count = num; + } else if (["88nine", "hyfin", "rhythmlab"].includes(part.toLowerCase().replace(/\s+/g, ""))) { + station = part; + } + } + + const days = ["Sunday", "Monday", "Tuesday", "Wednesday", "Thursday", "Friday", "Saturday"]; + const day = days[new Date().getDay()]; + + const stationContext = station + ? [ + ``, + `STATION VOICE: This segment is for ${station}. Match the station's voice, music focus, and audience:`, + station.toLowerCase().includes("hyfin") + ? `- HYFIN: Bold, culturally sharp. Focus on hip-hop, neo-soul, Afrobeats, cultural context. Audience: young, culturally aware Milwaukee listeners.` + : station.toLowerCase().includes("rhythm") + ? `- Rhythm Lab: Curated, global perspective, deep knowledge. Focus on global beats, electronic, jazz fusion. Audience: dedicated music heads and crate diggers.` + : `- 88Nine: Warm, eclectic, community-forward. Focus on indie, alternative, world, electronic. Audience: Milwaukee music lovers who value discovery.`, + `- Prioritize stories relevant to this station's audience and music focus.`, + `- Use Milwaukee local sources (milwaukeerecord.com, jsonline.com, urbanmilwaukee.com) for local angles.`, + ].join("\n") + : ""; + + return [ + `Generate a Radio Milwaukee daily music news segment for ${day}.`, + `Find ${count} current music stories from TODAY or the past 24-48 hours.`, + ``, + `RESEARCH STEPS:`, + `1. Use search_music_news to scan RSS feeds for breaking stories`, + `2. Use search_web (Tavily, topic="news", time_range="day") to find additional breaking music news not in RSS`, + `3. Use search_web (Exa) for any trending music stories or scene coverage the keyword search missed`, + `4. Cross-reference and pick the ${count} most compelling, newsworthy stories`, + `5. For each story, verify facts using available tools (MusicBrainz, Discogs, Bandcamp, etc.)`, + stationContext, + ``, + `FORMAT — follow the Music News Segment Format rules in your instructions exactly.`, + `Output "For ${day}:" then ${count} numbered stories with source citations.`, + ].join("\n"); + } +``` + +This enables: +- `/news` — 5 stories, general voice +- `/news hyfin` — 5 stories, HYFIN voice +- `/news rhythmlab 3` — 3 stories, Rhythm Lab voice +- `/news 88nine` — 5 stories, 88Nine voice + +**Step 2: Commit** + +```bash +cd /Users/tarikmoody/Documents/Projects/crate-web +git add src/app/api/chat/route.ts +git commit -m "feat: add /show-prep slash command, make /news station-customizable" +``` + +--- + +### Task 10: Build Verification + +**Step 1: Verify crate-cli skill loads** + +```bash +cd /Users/tarikmoody/Documents/Projects/crate-cli +npx tsx -e " +import { SkillRegistry } from './src/skills/registry.js'; +const reg = new SkillRegistry(); +await reg.loadAll(); +console.log('Skills:', reg.listSkills().map(s => s.name)); +const match = reg.matchQuery('prep my show for HYFIN'); +console.log('show-prep match:', match?.name); +" +``` + +Expected: `show-prep` in skills list, matched by query. + +**Step 2: Verify crate-web builds** + +```bash +cd /Users/tarikmoody/Documents/Projects/crate-web +npx next build 2>&1 | tail -20 +``` + +Expected: Build succeeds with no TypeScript errors. + +**Step 3: Manual smoke test** + +1. Start crate-web dev server: `npm run dev` +2. Open http://localhost:3000 +3. Type: `Prep my evening show for Rhythm Lab: Khruangbin - Time (You and I)` +4. Verify: The show-prep skill activates, the agent researches the track, and a ShowPrepPackage artifact appears in the slide-in panel. +5. Type: `/news hyfin` +6. Verify: News segment is generated with HYFIN voice and cultural context. + +**Step 4: Commit and push** + +```bash +cd /Users/tarikmoody/Documents/Projects/crate-web +git push origin main +cd /Users/tarikmoody/Documents/Projects/crate-cli +git push origin main +``` diff --git a/docs/plans/2026-03-19-custom-skills-design.md b/docs/plans/2026-03-19-custom-skills-design.md new file mode 100644 index 0000000..15e3f92 --- /dev/null +++ b/docs/plans/2026-03-19-custom-skills-design.md @@ -0,0 +1,281 @@ +# Custom Skills System — Design Spec + +> Users describe what they want, Crate builds a reusable command from existing tools. + +## Overview + +Crate's custom skills system lets users create personal slash commands by describing what they need in natural language. The agent discovers which tools can fulfill the request, runs a dry run to prove it works, and saves the result as a reusable command. No code, no config. + +Example: A user says "create a command that pulls events from The Rave Milwaukee." The agent browses therave.com/events using Kernel's browser tool, extracts the events, shows the result, and offers to save it as `/rave-events`. Next time the user types `/rave-events`, they get fresh results in seconds. + +## Architecture + +### Prompt Template Approach + +Each skill is a saved prompt that gets injected into the agentic loop at execution time. The agent picks tools dynamically — the skill doesn't hard-code a workflow. This means: + +- Skills adapt if a site changes layout or a tool is unavailable +- The agentic loop handles retries and fallbacks naturally +- No workflow engine needed + +Tool hints (discovered during the creation dry run) bias the agent toward tools that worked before, but it can deviate if needed. Hybrid reliability without brittleness. + +## Data Model + +New `userSkills` table in Convex: + +``` +userSkills + userId → id("users") + command → string // "rave-events" (no slash, lowercase, alphanumeric + hyphens) + name → string // "The Rave Events" + description → string // "Pull upcoming events from The Rave Milwaukee" + triggerPattern → optional string // "upcoming shows, concerts, or events at The Rave, Riverside, or Milwaukee venues" + promptTemplate → string // Full prompt injected into the agentic loop + toolHints → string[] // ["browse_url", "search_web"] — discovered at creation + sourceUrl → optional string // "therave.com/events" if URL-based + lastResults → optional string // JSON snapshot of last execution results (for change detection) + gotchas → optional string // Accumulated failure notes — appended when execution fails + runCount → number // Times executed (0 on creation) + visibility → "private" // Future: "team", "public" + isEnabled → boolean // User can disable without deleting + schedule → optional string // Future: "weekly:monday:9am", "daily:8am" + lastRunAt → optional number // Timestamp of last execution (manual or scheduled) + createdAt → number + updatedAt → number + + index: by_user ["userId"] + index: by_user_command ["userId", "command"] +``` + +### Example Prompt Template + +``` +Browse therave.com/events using browse_url. Extract all upcoming events with: +event name, date, time, price, and ticket link. Format as a clean list sorted +by date. If browse_url fails, fall back to search_web for "The Rave Milwaukee +upcoming events". +``` + +## Skill Creation Flow + +Creation happens conversationally in chat — no special UI. + +### Step 1: Detect Intent + +The agent recognizes the user wants to create a skill. Triggered by: +- Natural language: "make me a command that..." / "create a skill to..." / "save this as a command" +- Explicit command: `/create-skill` + +### Step 2: Clarify + +The agent asks 1-2 questions if needed: +- "What should the command be called?" (if not obvious from the request) +- "What URL or source?" (if the request involves scraping a specific site) + +### Step 3: Dry Run + +The agent runs the task once using the standard agentic loop. This: +- Proves it works and shows the user real results +- Discovers which tools succeeded (saved as `toolHints`) +- Generates the `promptTemplate` from what worked + +### Step 4: Confirm and Save + +The agent shows: +> "Here are the events I found. Want me to save this as `/rave-events`? You can run it anytime." + +User confirms → Convex mutation saves the skill. + +## Skill Execution + +### Recognition + +Two trigger paths: + +**Slash command (explicit):** +1. Check built-in commands first (`/news`, `/prep`, `/influence`, etc.) +2. If no match → query Convex for a `userSkill` matching that command + user +3. If found → inject the `promptTemplate` as the message, pass `toolHints` + +**Natural language (implicit):** +When a message doesn't match any slash command and isn't chat-tier, the agent sees the user's installed skills (via `list_user_skills`) in its tool list. If the user says "what's coming up at The Rave?", the agent can match it against a skill's `triggerPattern` ("upcoming shows, concerts, or events at The Rave") and run that skill's prompt template. + +This works because the `triggerPattern` is included in the `list_user_skills` response, and the agent naturally matches user intent to available tools. No special routing logic needed — the agent figures it out. + +### Prompt Injection + +The template gets wrapped with context, memory, and gotchas: + +``` +[Running custom skill: The Rave Events] + +{promptTemplate} + +{if gotchas} +KNOWN ISSUES (from previous runs): +{gotchas} +{endif} + +{if lastResults} +PREVIOUS RESULTS (from last run): +{lastResults} +Compare with current results and highlight what's NEW, CHANGED, or REMOVED. +{endif} + +{if userArg} +User specified: "{userArg}" +{endif} +``` + +This turns every skill into a **change detector** — the agent doesn't just fetch data, it tells you what's different since last time. + +### Arguments + +Skills accept optional arguments naturally. `/rave-events this weekend` becomes the `userArg` above. The agent scopes its output accordingly. No argument parsing logic needed. + +### Post-Execution Updates + +After each skill execution, the chat route: +1. Increments `runCount` and sets `lastRunAt` +2. Saves a summary of results to `lastResults` (agent extracts key data points) +3. If the execution failed or produced poor results, appends to `gotchas` + +The `save_skill_results` tool handles steps 2-3 — the agent calls it at the end of a skill execution. + +### Slash Command Menu + +The autocomplete menu in `ChatInput` currently shows hardcoded `SLASH_COMMANDS`. Custom skills get appended from Convex on component mount. They appear below built-in commands with a subtle "custom" label. + +## Skill Management + +### Settings Drawer + +A new "Custom Skills" section in the Settings drawer (below "Your Plan"): +- List of skills with command name, description, and enabled/disabled toggle +- Click to expand: prompt template, tool hints, source URL +- Edit button for manual prompt tweaking +- Delete button with confirmation +- "Create Skill" button that drops a hint into chat + +### `/skills` Command + +Typing `/skills` in chat lists active custom skills with descriptions. Quick reference without opening Settings. + +### Plan Limits + +| Plan | Custom Skills | Scheduled Skills (future) | +|------|--------------|--------------------------| +| Free | 3 | 0 | +| Pro | 20 | 3 | +| Team | 50 | 10 | + +Stored as `maxCustomSkills` and `maxScheduledSkills` in `PLAN_LIMITS`. + +## Skill Memory & Self-Improvement + +Inspired by [Anthropic's lessons from building Claude Code skills](https://www.anthropic.com/engineering/claude-code-skills). + +### Memory (lastResults) + +Every skill execution saves a summary of its results. On the next run, the agent sees what changed: + +- `/rave-events` → "3 new shows added since last week. The Roots on April 5 is new." +- `/vinyl-drops` → "12 new jazz releases on Discogs. 3 match artists in your collection." +- `/tour-watch` → "No new Milwaukee dates for Flying Lotus. Last checked: 2 days ago." + +The `lastResults` field stores a JSON summary (not the full output — just key data points the agent extracts). This is capped at 2000 characters to avoid bloating the prompt. + +### Self-Improving Gotchas + +When a skill execution fails or produces poor results, the agent appends a note to the `gotchas` field: + +- "therave.com returned a login wall on 2026-03-15. browse_url failed, fell back to search_web." +- "Discogs rate-limited after 3 rapid calls. Space out requests." + +On subsequent runs, these gotchas are injected into the prompt so the agent avoids known pitfalls. Skills get more reliable over time without manual editing. + +The agent calls `save_skill_results` with `gotcha` parameter when something goes wrong. The gotchas field is append-only, capped at 1000 characters (oldest entries trimmed). + +### Description as Trigger Condition + +Per Anthropic: "The description field is not a summary — it's when to trigger." + +The `triggerPattern` field captures this. During skill creation, the agent generates both: +- `description`: human-readable ("Pull upcoming events from The Rave Milwaukee") +- `triggerPattern`: model-readable ("when user asks about upcoming shows, concerts, events at The Rave, Riverside, Pabst, Turner Hall, or Milwaukee music venues") + +### Skill Composition + +Skills can reference other skills in their prompt templates: + +``` +Browse therave.com/events... [extract events] + +After listing events, check if any performing artists have been researched +before by calling list_user_skills and running any matching artist monitoring skills. +``` + +No special infrastructure — the agent already handles multi-tool orchestration. + +## Scheduled Triggers (Future) + +Not in v1. The data model includes `schedule` and `lastRunAt` fields to avoid future migration. + +When built: +- A Convex cron job runs hourly, queries skills with a `schedule` set +- If due, calls the chat API internally with the prompt template +- Results delivered to the user's most recent session or via notification +- The agentic loop handles execution identically to manual triggers + +## Case Studies & Skill Categories + +### Venue & Events +| Command | What It Does | Tools Used | +|---------|-------------|------------| +| `/rave-events` | Scrape The Rave Milwaukee for upcoming shows | browse_url, search_web | +| `/pabst-shows` | Pull Pabst Theater group events this month | browse_url | +| `/fest-lineup` | Get Summerfest daily lineups during festival season | browse_url, search_web | + +### Artist Monitoring +| Command | What It Does | Tools Used | +|---------|-------------|------------| +| `/new-releases-hyfin` | Check new releases from HYFIN-rotation artists | search_web, search_discogs | +| `/tour-watch` | Check if a specific artist announced Milwaukee dates | search_events, search_web | + +### Radio & DJ Workflow +| Command | What It Does | Tools Used | +|---------|-------------|------------| +| `/mke-music-news` | Scan Milwaukee Record + Journal Sentinel for local stories | search_web | +| `/trending-bandcamp` | Pull trending albums from Bandcamp editorial picks | search_bandcamp, browse_url | +| `/vinyl-drops` | Check Discogs marketplace for new arrivals in a genre | search_discogs | + +### Publishing & Social +| Command | What It Does | Tools Used | +|---------|-------------|------------| +| `/weekly-roundup` | Compile this week's research into a Telegraph draft | view_my_page, post_to_page | +| `/playlist-export` | Format a session's tracks as a shareable playlist | search_itunes_songs | + +### Data & Research +| Command | What It Does | Tools Used | +|---------|-------------|------------| +| `/sample-alert` | Check WhoSampled for new sample credits on a tracked artist | search_whosampled | +| `/label-roster` | Pull all artists on a specific label from Discogs | search_discogs, get_release_full | + +## Files to Create/Modify + +### New Files +| File | Purpose | +|------|---------| +| `convex/userSkills.ts` | Convex queries/mutations for skill CRUD | +| `src/components/settings/skills-section.tsx` | Skills list in Settings drawer | + +### Modified Files +| File | Change | +|------|--------| +| `convex/schema.ts` | Add `userSkills` table | +| `src/lib/plans.ts` | Add `maxCustomSkills`, `maxScheduledSkills` to `PlanLimits` | +| `src/app/api/chat/route.ts` | Resolve custom skills before agentic loop | +| `src/lib/chat-utils.ts` | Add `/skills` and `/create-skill` to slash commands | +| `src/components/workspace/chat-panel.tsx` | Append custom skills to autocomplete menu | +| `src/components/settings/settings-drawer.tsx` | Add SkillsSection | diff --git a/docs/plans/2026-03-21-auth0-hackathon-design.md b/docs/plans/2026-03-21-auth0-hackathon-design.md new file mode 100644 index 0000000..a9e20c9 --- /dev/null +++ b/docs/plans/2026-03-21-auth0-hackathon-design.md @@ -0,0 +1,310 @@ +# Auth0 Hackathon: Token Vault Integration for Crate + +> Replace Crate's manual API key pasting with Auth0 Token Vault OAuth connections. Add Spotify library access, playlist export, Slack delivery, and Google Docs saving. + +## Hackathon + +- **Name:** Authorized to Act: Auth0 for AI Agents +- **Deadline:** April 6, 2026 @ 11:45pm PDT +- **Requirement:** Must use Token Vault feature of Auth0 for AI Agents +- **Prize:** $5,000 grand prize + blog feature on auth0.com +- **Submission:** Text description, 3-min demo video, public repo, published link, optional blog post ($250 bonus) + +## Overview + +Crate is an AI music research agent that connects to 20+ data sources. Today, users paste API keys manually in Settings. Auth0 Token Vault replaces that with "click to Connect" OAuth flows for services that support it, while keeping the existing key system for services that don't. + +The hackathon integration adds four capabilities powered by Token Vault: + +1. **Read user's Spotify library** — "What in my library connects to the LA beat scene?" +2. **Export playlists to Spotify** — influence chain becomes a real Spotify playlist +3. **Send research to Slack** — show prep delivered to the team's channel +4. **Save research to Google Docs** — shareable doc with one command + +## Architecture + +### Auth Strategy: Clerk + Auth0 Side-by-Side + +Clerk stays for user sign-in (no migration). Auth0 handles ONLY Token Vault OAuth connections to third-party services. Two auth systems, zero overlap. + +``` +CLERK AUTH0 TOKEN VAULT +───── ───────────────── +User sign-in "Connect Spotify" OAuth flow +Session management "Connect Slack" OAuth flow +Convex user identity "Connect Google" OAuth flow +Stripe billing link Token storage + refresh +Radio Milwaukee domain Per-user, personal connections +``` + +### Resolution Flow + +``` +Agent needs Spotify token: + +1. Check Token Vault → user connected Spotify? + YES → get OAuth token from Auth0 → use it + NO → fall back to embedded platform key (existing behavior) + +Agent needs Slack token: + +1. Check Token Vault → user connected Slack? + YES → get OAuth token from Auth0 → use it + NO → "Connect Slack in Settings to send research to your team" +``` + +The key principle: Token Vault is an upgrade path, not a replacement. Everything that works today keeps working. Non-OAuth services (Discogs API key, MusicBrainz, Last.fm, Bandcamp) stay on the existing key system. + +### What Doesn't Change + +- Clerk authentication (sign-in, sessions, middleware) +- Convex schema and user records +- Stripe billing and subscription system +- Custom skills system +- Agentic loop and tool orchestration +- Radio Milwaukee domain detection and org key sharing +- Embedded platform keys for non-OAuth services +- The `/influence`, `/prep`, `/news` commands + +## Token Vault Connections + +| Service | Token Vault Type | OAuth Scopes | What Crate Does With It | +|---|---|---|---| +| Spotify | Pre-built | `user-library-read`, `user-top-read`, `playlist-read-private`, `playlist-modify-public` | Read library, read top artists, create playlists | +| Slack | Pre-built | `chat:write`, `channels:read` | Send research/show prep to channels | +| Google | Pre-built (Workspace) | `docs`, `drive.file` | Create Google Docs with research output | + +### Services NOT Using Token Vault (stay on existing key system) + +| Service | Why | +|---|---| +| Anthropic / OpenRouter | LLM key, not OAuth-based | +| Discogs | OAuth 1.0a (not supported by Token Vault) | +| MusicBrainz | Open API, no auth needed | +| Last.fm | API key only, no OAuth | +| Bandcamp | No public API | +| Genius | Could use custom OAuth2, but not needed for hackathon | +| Ticketmaster | API key only | +| Perplexity | API key only | + +## New Agent Tools + +### Tool 1: `read_spotify_library` + +```typescript +read_spotify_library(type: "saved_tracks" | "top_artists" | "playlists", limit?: number) + +// Token Vault provides Spotify OAuth token +// Returns: user's saved tracks, top artists, or playlists +// Agent uses this for personalized research: +// "What in my library connects to the LA beat scene?" +``` + +### Tool 2: `export_to_spotify` + +```typescript +export_to_spotify( + name: string, // "Ezra Collective: The Influence Chain" + description: string, // "Afrobeat Roots → Jazz Foundations → Hip-Hop Swing" + trackQueries: string[] // ["Fela Kuti Zombie", "Herbie Hancock Cantaloupe Island", ...] +) + +// Token Vault provides Spotify OAuth token +// Step 1: Search Spotify for each track query → get track URIs +// Step 2: Create playlist via Spotify API +// Step 3: Add tracks to playlist +// Returns: Spotify playlist URL +``` + +Note: the agent passes track queries (artist + title), not Spotify URIs. The tool searches Spotify's catalog to find the right URI for each track. This avoids hallucinated Spotify URIs. + +### Tool 3: `send_to_slack` + +```typescript +send_to_slack( + channel: string, // "#hyfin-evening" or "general" + content: string, // Formatted research/show prep text + title?: string // "HYFIN Evening Show Prep — Thursday" +) + +// Token Vault provides Slack OAuth token +// POST to Slack API → message appears in channel +// Returns: message permalink +``` + +### Tool 4: `save_to_google_doc` + +```typescript +save_to_google_doc( + title: string, // "Flying Lotus Influence Chain" + content: string // Research output as formatted text +) + +// Token Vault provides Google OAuth token +// Google Docs API → create new doc with content +// Returns: shareable doc URL +``` + +## Settings UI: Connected Services + +New section in Settings drawer, above existing API keys: + +``` +┌─────────────────────────────────────┐ +│ CONNECTED SERVICES │ +│ │ +│ 🟢 Spotify Connected │ +│ Library, playlists, export │ +│ [Disconnect] │ +│ │ +│ ⚪ Slack [Connect] │ +│ Send research to your team │ +│ │ +│ ⚪ Google [Connect] │ +│ Save research to Google Docs │ +│ │ +├──────────────────────────────────────┤ +│ YOUR PLAN │ +│ [existing plan section] │ +├──────────────────────────────────────┤ +│ CUSTOM SKILLS │ +│ [existing skills section] │ +├──────────────────────────────────────┤ +│ API KEYS │ +│ [existing key entry — Anthropic, │ +│ Discogs, Last.fm, etc.] │ +└──────────────────────────────────────┘ +``` + +When user clicks "Connect Spotify": +1. Auth0 Token Vault OAuth flow opens in popup +2. User authorizes Crate on Spotify +3. Token stored in Auth0 +4. Status updates to green "Connected" +5. Agent now has Spotify access for this user + +"Disconnect" revokes the Token Vault connection. Agent falls back to embedded keys. + +## Files to Create/Modify + +### New Files +| File | Purpose | +|---|---| +| `src/lib/web-tools/spotify-connected.ts` | `read_spotify_library` + `export_to_spotify` tools | +| `src/lib/web-tools/slack.ts` | `send_to_slack` tool | +| `src/lib/web-tools/google-docs.ts` | `save_to_google_doc` tool | +| `src/lib/auth0-token-vault.ts` | Token Vault client — get tokens for connected services | +| `src/app/api/auth0/callback/route.ts` | Auth0 OAuth callback handler | +| `src/components/settings/connected-services.tsx` | "Connected Services" UI section | + +### Modified Files +| File | Change | +|---|---| +| `src/lib/resolve-user-keys.ts` | Add Token Vault resolution alongside Convex decryption | +| `src/app/api/chat/route.ts` | Register new tool servers (spotify-connected, slack, google-docs) | +| `src/components/settings/settings-drawer.tsx` | Add ConnectedServices section above existing sections | +| `package.json` | Add `@auth0/ai-vercel` and `googleapis` dependencies | +| `.env.local.example` | Add Auth0 Token Vault env vars | + +### Files That Don't Change +| File | Why | +|---|---| +| `convex/schema.ts` | No schema changes — Token Vault stores tokens in Auth0, not Convex | +| `src/lib/plans.ts` | No plan limit changes | +| `convex/subscriptions.ts` | Stripe billing unaffected | +| `convex/userSkills.ts` | Custom skills unaffected | +| `src/lib/agentic-loop.ts` | Agentic loop doesn't change — tools handle their own auth | +| All Clerk auth files | Clerk stays for user sign-in | + +## Demo Script (3 minutes) + +``` +0:00 - 0:15 INTRO +"I'm Tarik Moody, a radio broadcaster at Radio Milwaukee. +I built Crate — an AI music research agent that connects to +20+ data sources. Today I'm showing how Auth0 Token Vault +makes that secure and seamless." + +0:15 - 0:45 CONNECT +Show Settings → Connected Services section. +Click "Connect Spotify" → Auth0 OAuth popup → authorize. +Click "Connect Slack" → Auth0 OAuth popup → authorize. +Both show green "Connected" status. +"No API keys. Click Connect, you're done." + +0:45 - 1:30 RESEARCH FROM YOUR LIBRARY +Type: "What in my Spotify library connects to the LA beat scene?" +Agent reads Spotify library via Token Vault. +Finds Flying Lotus, Thundercat, Knxwledge in saved tracks. +Runs influence mapping, builds the chain. +Shows InfluenceChain with pull quotes, sonic DNA chips. +"Crate read my library, found my artists, and mapped the +full influence network with cited sources." + +1:30 - 2:15 EXPORT PLAYLIST +Agent: "Want me to save this as a Spotify playlist?" +Type: "yes" +Agent exports → playlist appears in Spotify. +Click the link, show it in Spotify. +"23 tracks, organized by influence lineage, in my Spotify." + +2:15 - 2:45 SEND TO SLACK +Type: "/prep HYFIN" with a setlist. +Agent generates show prep. +Type: "send this to #hyfin-evening on Slack" +Agent sends via Token Vault. +"Show prep delivered to my team's Slack." + +2:45 - 3:00 CLOSE +"Token Vault handles Spotify, Slack, and Google OAuth tokens +securely. Users click Connect instead of managing API keys. +digcrate.app — built with Claude Code by a radio broadcaster." +``` + +## Judging Criteria Alignment + +| Criteria | How Crate Scores | +|---|---| +| Security Model | Token Vault manages all OAuth tokens. Agent never sees raw credentials. Users connect via standard OAuth flows with explicit scope consent. | +| User Control | Users choose which services to connect. Each connection shows what access it grants. Disconnect anytime. Personal connections only (no team-wide credential sharing). | +| Technical Execution | Token Vault integrated into an existing 20+ data source agentic system. Clean separation: Token Vault for OAuth services, existing key system for non-OAuth services. | +| Design | Three-panel UI with interactive artifacts. "Connected Services" section with clear status indicators. Agent outputs are interactive cards, not plain text. | +| Potential Impact | Music professionals connecting their actual accounts instead of managing raw API keys. Research personalized to their listening history. Show prep delivered to team Slack channels. | +| Insight Value | Pattern surfaced: AI agents that need MANY external services (20+) on behalf of ONE user. Token Vault managing a subset (OAuth-capable) while existing credential systems handle the rest. Real-world hybrid approach. | + +## Blog Post Outline (bonus $250) + +**Title:** "How a Radio Broadcaster Used Auth0 Token Vault to Connect an AI Agent to 20+ Music APIs" + +1. The problem: music professionals research across 5-10 fragmented platforms, each requiring separate API credentials +2. What Crate is: AI music research agent with 20+ data sources +3. The old way: users paste API keys manually in Settings +4. The Token Vault way: click "Connect Spotify" and the agent reads your library, exports playlists, sends show prep to Slack +5. The hybrid approach: Token Vault for OAuth services, existing key system for non-OAuth services +6. The non-coder founder story: built with Claude Code, not engineering experience +7. What's next: WordPress publishing, Google Calendar for show scheduling, team-shared connections + +## Timeline + +| Day | Task | Effort | +|---|---|---| +| 1 | Set up Auth0 account, configure Token Vault connections (Spotify, Slack, Google) | 2 hours | +| 2 | Build `auth0-token-vault.ts` client + callback route | CC: 30 min | +| 3 | Build Spotify tools (read library, export playlist) | CC: 1 hour | +| 4 | Build Slack tool + Google Docs tool | CC: 1 hour | +| 5 | Build Connected Services UI in Settings | CC: 30 min | +| 6 | Wire tools into chat route + resolve-user-keys | CC: 30 min | +| 7 | Test full flow end-to-end | 2 hours | +| 8 | Record demo video | 2 hours | +| 9 | Write submission text + blog post | 2 hours | +| 10 | Submit | 30 min | + +Total engineering: ~4 hours of Claude Code work + ~8 hours of setup/testing/content. + +## Future (Post-Hackathon) + +- WordPress.com publishing via custom OAuth2 connection +- Apple Music library access via MusicKit JS (not Token Vault — no OAuth2 support) +- Team-shared connections (one Slack workspace for all @radiomilwaukee.org users) +- Google Calendar integration for show scheduling +- Full Clerk → Auth0 migration (single auth system) diff --git a/docs/plans/2026-03-21-influence-chain-perplexity-design.md b/docs/plans/2026-03-21-influence-chain-perplexity-design.md new file mode 100644 index 0000000..9839cbe --- /dev/null +++ b/docs/plans/2026-03-21-influence-chain-perplexity-design.md @@ -0,0 +1,336 @@ +# Influence Chain Enhancement — Perplexity Enrichment + Dynamic UI + +> Enrich influence connections with deep storytelling via Perplexity Sonar Pro. Upgrade the InfluenceChain component with interactive, visually compelling UI. + +## Problem + +The influence chain is Crate's signature feature, but the connection context is thin: +- "Co-mentioned in 12 reviews across Pitchfork, Wire, and The Quietus" +- No interview quotes, no sonic analysis, no specific album references +- Citations link to publication homepages, not specific articles +- The UI is functional but static — timeline with expandable cards + +SongDNA shows credits. Crate should show **stories**. + +## Architecture + +### Hybrid: Co-mention Discovery + Perplexity Enrichment + +``` +PHASE 1: DISCOVER (existing, proprietary — the moat) + lookup_influences() → check Convex cache + search_reviews() → co-mention analysis across 26 publications + extract_influences() → parse review text for influence signals + + This is Badillo-Goicoechea 2025 methodology. + Discovers THAT connections exist + initial weight scores. + +PHASE 2: ENRICH (new — Perplexity Sonar Pro) + For the top 5-6 strongest connections: + research_influence(from, to) → Perplexity API call + Returns: deep context paragraph + real cited URLs + + This adds the WHY — interview quotes, sonic analysis, + specific albums, production DNA transfer stories. + +PHASE 3: CACHE + OUTPUT + Save enriched context to influence cache (Convex) + Output enhanced InfluenceChain with deep stories + real citations + + Next query for same artist returns enriched data instantly. +``` + +### Why Not Replace Co-mention with Perplexity? + +Co-mention analysis is the moat. It finds **non-obvious connections** that Perplexity's general knowledge wouldn't surface — like discovering that an obscure Ethiopian jazz artist and a UK grime producer are connected through reviews in The Wire. Anyone with a Perplexity API key can ask "who influenced Flying Lotus?" Only Crate has the 26-publication co-mention engine. + +## Perplexity API Design + +### Model: Sonar Pro (`sonar-pro`) + +Best balance of depth and speed for research queries. With `search_context_size: "high"` for maximum grounding. + +Cost: ~$0.014/request × 5-6 connections = **~$0.07-0.08 per influence map**. + +### Citation Strategy (CRITICAL — no hallucinated URLs) + +Perplexity's response has two URL systems: + +1. **`citations` + `search_results` arrays** — from the search infrastructure, **REAL URLs** +2. **URLs in generated text** — from the language model, **CAN BE HALLUCINATED** + +**Strategy:** +- Request `response_format: "text"` (NOT json_schema — structured output can hallucinate URLs) +- Parse the model's text content for the influence story +- Extract citations from the `search_results` array (includes title, url, snippet, date) +- Map inline references `[1]`, `[2]` to the `citations` array +- Store both the enriched context AND the verified source URLs + +### Tool: `research_influence` + +```typescript +// In src/lib/web-tools/prep-research.ts (alongside research_track) + +research_influence(fromArtist: string, toArtist: string): { + context: string; // Rich paragraph: quotes, sonic elements, albums + sources: Array<{ // From Perplexity's search_results (REAL URLs) + name: string; // Page title + url: string; // Verified URL + snippet: string; // Text excerpt + date?: string; // Publication date + }>; + direction: string; // "influenced" | "collaborated" | "co_mention" + sonicElements: string; // Key sonic/stylistic connections + keyWorks: string; // Albums/tracks demonstrating the connection +} +``` + +### Prompt Design + +``` +Explain the musical influence relationship between {fromArtist} and {toArtist}. + +Include: +- DIRECTION: Who influenced whom? How do we know? (interviews, timeline, acknowledged) +- SONIC ELEMENTS: What specific musical qualities were transmitted? (rhythm, harmony, production techniques, instrumentation) +- KEY WORKS: Which albums or tracks demonstrate this connection? +- TIMELINE: When did this influence manifest? Key moments. +- QUOTES: Any interviews where either artist acknowledged the connection? + +Be specific. Name albums, tracks, producers, studios. Cite sources. +If the direction is unclear, note it as a "mutual influence" or "co-mention." +``` + +## Enhanced InfluenceChain UI + +### Current State +- Timeline with expandable cards +- Each connection: artist image + name + relationship tag + weight badge + "More" expand +- Expanded: thin context text + small citation links +- Lineage arc at top (small circles with arrows) +- Roots/Built/Legacy tabs + +### Enhanced Design + +#### 1. Connection Cards — Expanded by Default for Top 3 + +Instead of everything collapsed, the top 3 connections (by weight) are expanded by default, showing the full enriched context. This is the "lean forward" moment — the stories are visible without clicking. + +#### 2. Rich Context Section + +When expanded, each connection shows: + +``` +┌─────────────────────────────────────────────────────┐ +│ [Artist Image] Parliament-Funkadelic │ +│ ● influenced by (0.92) │ +│ │ +│ "Clinton taught me funk could be psychedelic and │ +│ spiritual at the same time." │ +│ — Flying Lotus, Pitchfork Interview, 2012 │ +│ │ +│ ┌─ SONIC DNA ──────────────────────────────────┐ │ +│ │ Synthesizer textures • cosmic imagery • │ │ +│ │ psychedelic funk arrangements │ │ +│ └──────────────────────────────────────────────┘ │ +│ │ +│ ┌─ KEY WORKS ──────────────────────────────────┐ │ +│ │ 🎵 Mothership Connection (1975) → │ │ +│ │ Cosmogramma (2010) │ │ +│ └──────────────────────────────────────────────┘ │ +│ │ +│ Sources: │ +│ 📄 Pitchfork Interview 2012 ↗ │ +│ 📄 Red Bull Music Academy Lecture ↗ │ +│ 📄 NPR First Listen: Cosmogramma ↗ │ +└─────────────────────────────────────────────────────┘ +``` + +#### 3. Pull Quote Treatment + +The most impactful element: a styled pull quote when the artist or a journalist has explicitly stated the influence. Large italic text with attribution: + +```tsx +
+

+ "Clinton taught me funk could be psychedelic and spiritual at the same time." +

+ + — Flying Lotus, Pitchfork, 2012 + +
+``` + +#### 4. Sonic DNA Chips + +Visual tags showing what was transmitted between artists: + +```tsx +
+ + synthesizer textures + + + cosmic imagery + +
+``` + +#### 5. Key Works Timeline + +A mini visual showing the album-to-album influence transfer: + +```tsx +
+ Mothership Connection + (1975) + + Cosmogramma + (2010) +
+``` + +#### 6. Source Cards (Not Just Links) + +Instead of tiny inline links, sources become small cards with title + snippet + date: + +```tsx +
+

Sources

+ {sources.map(src => ( + +

{src.name}

+

{src.snippet}

+ {src.date &&

{src.date}

} +
+ ))} +
+``` + +#### 7. Enhanced Lineage Arc + +The top arc gets bigger — actual artist photos (already auto-fetched), larger circles, and the connection type labeled on each arrow: + +``` + [Parliament] —influenced→ [Flying Lotus] —signed→ [Thundercat] + (root) (center) (legacy) +``` + +### New Props for InfluenceChain + +```typescript +connections: Array<{ + name: string; + weight: number; + relationship: string; + context: string; // Existing — now enriched by Perplexity + pullQuote?: string; // NEW — direct quote from artist/journalist + pullQuoteAttribution?: string; // NEW — "Flying Lotus, Pitchfork, 2012" + sonicElements?: string[]; // NEW — ["synthesizer textures", "cosmic imagery"] + keyWorks?: string; // NEW — "Mothership Connection (1975) → Cosmogramma (2010)" + sources: Array<{ + name: string; + url: string; + snippet?: string; // NEW — text excerpt from the source + date?: string; // NEW — publication date + }>; + imageUrl?: string; +}>; +``` + +These new fields are optional — the component gracefully degrades if they're absent (backward compatible with existing cached connections that don't have enrichment yet). + +## Data Flow + +``` +User types: /influence Flying Lotus + +1. DISCOVER (co-mention engine, 2-3 calls) + └→ Returns 8-12 connections with weights + thin context + +2. ENRICH top 5-6 (Perplexity Sonar Pro, 5-6 calls) + For each connection: + └→ POST api.perplexity.ai/v1/sonar + Model: sonar-pro + search_context_size: "high" + response_format: "text" + └→ Parse response: + - content → context paragraph, pull quote, sonic elements, key works + - search_results → verified source URLs with titles + snippets + dates + └→ Save enriched data to influence cache (Convex) + +3. OUTPUT + └→ InfluenceChain with enriched props + └→ Top 3 connections expanded by default + └→ Pull quotes, sonic DNA chips, key works timeline, source cards + +Total: 7-9 tool calls, ~$0.08, within 25-call budget +``` + +## Citation Guarantee (No Hallucinated URLs) + +```typescript +// In research_influence handler: + +const response = await fetch("https://api.perplexity.ai/chat/completions", { + body: JSON.stringify({ + model: "sonar-pro", + messages: [...], + max_tokens: 1000, + temperature: 0.2, + web_search_options: { search_context_size: "high" }, + // NO response_format — use text to avoid URL hallucination + }), +}); + +const data = await response.json(); +const content = data.choices[0].message.content; // Story text +const searchResults = data.search_results ?? []; // REAL URLs from search layer +const citations = data.citations ?? []; // REAL URL array + +// Map inline [1], [2] references to actual URLs +// NEVER use URLs from the content text itself + +return { + context: content, // Rich story paragraph + sources: searchResults.map(sr => ({ + name: sr.title, + url: sr.url, // VERIFIED — from search infrastructure + snippet: sr.snippet, + date: sr.date, + })), +}; +``` + +## Alignment with Badillo-Goicoechea 2025 + +The paper's core methodology: +- **Review-based semantic distance** — co-mention frequency across publications +- **Direction convention** — from=INFLUENCER, to=INFLUENCED +- **Bridge artists** — seminal figures connecting disparate musical territories +- **Knowledge graph traversal** — recommendations via optimal sequences through the graph + +Perplexity enrichment preserves all of this: +- Co-mention discovery remains the connection source (proprietary moat) +- Direction is verified/corrected by Perplexity's research (interviews, timeline evidence) +- Bridge artists get the richest stories (cross-genre connections are the most interesting) +- The knowledge graph gains richer node metadata without changing its structure + +The enrichment makes the academic methodology **accessible to non-academics** — turning network properties into stories people can understand and share. + +## Files to Create/Modify + +### New/Modified +| File | Change | +|------|--------| +| `src/lib/web-tools/prep-research.ts` | Add `research_influence` tool (Sonar Pro with search_results extraction) | +| `src/lib/openui/components.tsx` | Enhance `ConnectionNode` with pull quotes, sonic chips, key works, source cards. Expand top 3 by default. | +| `src/lib/chat-utils.ts` | Update `/influence` prompt to include Phase 2 (Perplexity enrichment) | +| `src/app/api/chat/route.ts` | Ensure `prep-research` tools available for influence commands (already in RESEARCH_SERVERS) | + +### No Changes Needed +| File | Why | +|------|-----| +| `convex/schema.ts` | Influence cache `context` field is already a string — enriched context just replaces thin description | +| `src/lib/web-tools/influence-cache.ts` | Cache tools already accept context string — no API change needed | +| `convex/influenceEdges.ts` | Edge schema already stores context, sources — just richer data | diff --git a/docs/plans/2026-04-06-tumblr-token-vault-design.md b/docs/plans/2026-04-06-tumblr-token-vault-design.md new file mode 100644 index 0000000..a5e468c --- /dev/null +++ b/docs/plans/2026-04-06-tumblr-token-vault-design.md @@ -0,0 +1,196 @@ +# Tumblr Auth0 Token Vault Integration — Design Doc + +**Date:** 2026-04-06 +**Author:** Tarik Moody + Claude +**Status:** Approved + +## Overview + +Migrate Crate's Tumblr integration from OAuth 1.0a (Convex-stored credentials) to Auth0 Token Vault (OAuth 2.0 bearer tokens), matching the existing Spotify/Slack/Google pattern. Add music discovery tools (dashboard feed, tag search, liked posts) and OpenUI components for rendering Tumblr content in chat. + +## Goals + +1. Auth0 Token Vault integration for Tumblr (same pattern as Spotify) +2. Read tools: dashboard audio feed, tag-based discovery, liked posts +3. Write tools: publish to Tumblr blog (migrated from OAuth 1.0a) +4. OpenUI components: `TumblrFeed` and `TumblrPost` +5. `/tumblr` chat command with sub-modes +6. Cross-reference with Spotify via "Export to Spotify Playlist" action button + +## Architecture + +### Token Vault Config + +Add `tumblr` to `SERVICE_CONFIG` in `src/lib/auth0-token-vault.ts`: + +```typescript +tumblr: { + connection: "tumblr", + scopes: ["basic", "write", "offline_access"], +} +``` + +Update `TokenVaultService` union: `"spotify" | "slack" | "google" | "tumblr"` + +### OAuth Flow + +Same as Spotify: +1. User clicks Connect in Settings → `/api/auth0/connect?service=tumblr` +2. Auth0 redirects to Tumblr OAuth 2.0 authorization +3. Callback stores `auth0_user_id_tumblr` cookie +4. No special scope handling needed (Tumblr scopes go in standard `scope` param) + +### API Calls + +New file: `src/lib/web-tools/tumblr-connected.ts` +- Uses `getTokenVaultToken("tumblr", auth0UserId)` for bearer token +- All API calls use `Authorization: Bearer {token}` header +- Base URL: `https://api.tumblr.com/v2` + +### Old Code + +The existing `src/lib/web-tools/tumblr.ts` (OAuth 1.0a + Convex) stays for backwards compatibility. New connections go through Auth0. The `markdownToNpf()` function is reused by the new tools. + +## Tools + +### `tumblr-connected.ts` — `createTumblrConnectedTools(auth0UserId)` + +#### 1. `read_tumblr_dashboard` +- **Purpose:** User's dashboard feed (posts from blogs they follow) +- **API:** `GET /v2/user/dashboard?limit={limit}` +- **Returns:** All post types with type label. Agent filters for music-relevant content. +- **Params:** `limit` (default 20, max 50) + +#### 2. `read_tumblr_tagged` +- **Purpose:** Tag-based music discovery +- **API:** `GET /v2/tagged?tag={tag}` +- **Returns:** All post types matching the tag. Agent identifies music content. +- **Params:** `tag` (required), `before` (timestamp for pagination) + +#### 3. `read_tumblr_likes` +- **Purpose:** User's liked posts +- **API:** `GET /v2/user/likes?limit={limit}` +- **Returns:** All post types. Agent filters for music-relevant content. +- **Params:** `limit` (default 20, max 50) + +#### 4. `post_to_tumblr` +- **Purpose:** Publish to user's blog +- **API:** `GET /v2/user/info` (get blog name) → `POST /v2/blog/{blog}/posts` +- **Content:** Markdown converted to NPF via `markdownToNpf()` (reused from old tumblr.ts) +- **Params:** `title`, `content` (markdown), `tags` (string[]), `category` + +### Post Type Handling + +All read tools return posts with their `type` field preserved. Post types include: +- `audio` — Has artist, track_name, album, album_art, plays, player embed +- `text` — Has body (may contain Spotify/YouTube/Bandcamp URLs) +- `photo` — Has photos array with URLs and captions +- `link` — Has url, title, description +- `video` — Has video_url, thumbnail +- `quote` — Has text, source + +The agent receives all types and identifies music-relevant content from context (embedded URLs, artist mentions, album art, music tags). This approach gives the richest discovery results without sparse audio-only filtering. + +## OpenUI Components + +### `TumblrFeed` + +**Props:** +- `posts` — JSON string of post array +- `source` — `"dashboard"` | `"tagged"` | `"likes"` +- `tag` — Optional string (for tagged mode) +- `totalCount` — Number of posts + +**Renders by post type:** +- **Audio:** Album art, artist/track name, play count, player embed +- **Text:** Excerpt (~200 chars), blog name, tags +- **Photo:** Thumbnail, caption excerpt, blog name +- **Link:** Title, URL preview, description + +**Common elements on all posts:** +- Blog avatar + name (source attribution) +- Tags as pills +- Timestamp +- Reblog/note count + +**Action buttons:** +- "Export to Spotify Playlist" — Agent collects artist/track names, chains `export_to_spotify` +- "Post to Tumblr" — Agent formats research as NPF and publishes + +### `TumblrPost` + +Single post detail view. + +**Props:** `postUrl`, `blogName`, `content`, `tags`, `noteCount`, `type` + +## `/tumblr` Command + +### Routing + +Add `tumblr` to `isSlashResearch` regex in `route.ts`: +```typescript +const isSlashResearch = /^\/(?:influence|show-prep|prep|news|story|track|artist|spotify|tumblr)\b/i.test(rawMessage.trim()); +``` + +### Sub-modes + +| Command | Tool Called | OpenUI Output | +|---------|-----------|---------------| +| `/tumblr` | `read_tumblr_dashboard` | `TumblrFeed(posts, "dashboard", totalCount)` | +| `/tumblr #afrobeat` | `read_tumblr_tagged` tag="afrobeat" | `TumblrFeed(posts, "tagged", totalCount, "afrobeat")` | +| `/tumblr likes` | `read_tumblr_likes` | `TumblrFeed(posts, "likes", totalCount)` | + +### Prompt (in `chat-utils.ts`) + +Enforce OpenUI Lang output format matching `/spotify` and `/artist` patterns: +- No-args: call `read_tumblr_dashboard`, return `TumblrFeed(...)` +- `#tag`: call `read_tumblr_tagged`, return `TumblrFeed(...)` +- `likes`: call `read_tumblr_likes`, return `TumblrFeed(...)` + +## Settings UI + +Add Tumblr to `connected-services.tsx`: + +```typescript +{ + id: "tumblr" as const, + name: "Tumblr", + description: "Discover music, publish research", + icon: "📝", +} +``` + +Update `ConnectionStatus` interface and all type references to include `tumblr`. + +## Auth0 Dashboard Setup + +1. Create Tumblr social connection in Auth0 Dashboard +2. Register app at Tumblr Developer Console (https://www.tumblr.com/oauth/apps) +3. Set callback: `https://{AUTH0_DOMAIN}/login/callback` +4. Enable "Connected Accounts for Token Vault" toggle +5. Scopes: basic, write, offline_access + +## Files to Create/Modify + +| File | Action | Description | +|------|--------|-------------| +| `src/lib/auth0-token-vault.ts` | Modify | Add tumblr to SERVICE_CONFIG and TokenVaultService | +| `src/lib/web-tools/tumblr-connected.ts` | Create | New Token Vault tools (4 tools) | +| `src/lib/openui/components.tsx` | Modify | Add TumblrFeed and TumblrPost components | +| `src/components/tumblr/tumblr-feed.tsx` | Create | TumblrFeed component | +| `src/components/tumblr/tumblr-post.tsx` | Create | TumblrPost component | +| `src/components/settings/connected-services.tsx` | Modify | Add Tumblr service | +| `src/app/api/auth0/status/route.ts` | Modify | Add tumblr connection check | +| `src/app/api/chat/route.ts` | Modify | Add tumblr to isSlashResearch, wire tools | +| `src/lib/chat-utils.ts` | Modify | Add /tumblr command prompt | + +## Verification + +1. `npm run build` — no type errors +2. Auth0 Tumblr connection configured in dashboard +3. Connect flow: Settings → Connect Tumblr → OAuth → callback → cookie set +4. `/tumblr` returns dashboard feed as TumblrFeed component +5. `/tumblr #jazz` returns tagged posts +6. `/tumblr likes` returns liked posts +7. "Export to Spotify Playlist" action button works +8. `post_to_tumblr` publishes successfully diff --git a/docs/plans/2026-04-07-npr-listener-features.md b/docs/plans/2026-04-07-npr-listener-features.md new file mode 100644 index 0000000..fdac142 --- /dev/null +++ b/docs/plans/2026-04-07-npr-listener-features.md @@ -0,0 +1,198 @@ +# NPR Listener Features — Implementation Plan + +## Context + +Crate already has the core capabilities for a public-facing listener experience. The influence chain component, Deep Cut publishing, and YouTube search all exist. These features create new entry points to the product that demonstrate Crate's value to NPR Music, streaming companies, and consumer music listeners without requiring a login. + +**Goal:** Build two features this week that you can demo to NPR music directors and share on social media. No new backend logic needed. Primarily frontend routing and pre-rendering. + +--- + +## Feature 1: Public Influence Maps + +### What it is + +A shareable, no-login-required influence chain page at `digcrate.app/explore/[artist]` that anyone can view. The "Keep Digging" link that goes under any article or Tiny Desk video. + +### How it works today (already built) + +1. User runs `/influence Flying Lotus` in Crate +2. Agent builds influence chain, renders as InfluenceChain OpenUI component +3. User clicks "Publish" on the Deep Cut +4. Published at `digcrate.app/cuts/[shareId]` +5. Anyone with the link can view it + +### What's missing + +- The URL isn't clean. `digcrate.app/cuts/abc123` doesn't say "influence map." +- No SEO. The published page doesn't have artist-specific metadata. +- No entry point for non-Crate users. You have to know the share link exists. + +### What to build + +**File: `src/app/explore/[artist]/page.tsx` (new)** + +A public page at `digcrate.app/explore/flying-lotus` that: +1. Shows a clean landing with the artist name and "Explore the musical DNA of Flying Lotus" +2. Displays a pre-generated influence chain if one exists in the database (from a cached Deep Cut) +3. If no cached result exists, shows a teaser: "This influence map hasn't been generated yet. Be the first to explore it on Crate." +4. CTA: "Dig deeper — sign up free at digcrate.app" +5. SEO metadata: title, description, Open Graph image + +**File: `src/app/api/explore/[artist]/route.ts` (new)** + +API that: +1. Checks Convex for an existing published influence chain for this artist (search `shares` table by label containing the artist name) +2. If found, returns the OpenUI content +3. If not found, returns null (the page shows the teaser) + +**Modify: `src/components/workspace/chat-panel.tsx`** + +After a published influence chain, show: "Share this influence map: digcrate.app/explore/[artist-slug]" + +### Effort + +- 2-3 hours +- 3 files (1 new page, 1 new API route, 1 small edit) +- No new dependencies + +### How to test + +1. Run `/influence Flying Lotus` on Crate +2. Publish the Deep Cut +3. Visit `digcrate.app/explore/flying-lotus` +4. Should show the influence chain without login +5. Share the URL on Twitter. Does it render an Open Graph preview? + +--- + +## Feature 2: Tiny Desk Companion Pages + +### What it is + +A dedicated page at `digcrate.app/tinydesk/[artist]` that shows an influence chain with embedded YouTube videos at each node. The page a Tiny Desk viewer lands on after watching a performance. "Want to know who influenced this artist?" + +### How it differs from Feature 1 + +Feature 1 is a published influence chain (text + connections). Feature 2 adds YouTube video embeds at every node in the chain, making it a watchable experience. "Watch the lineage" from the video-linked influence chain strategy. + +### What to build + +**File: `src/app/tinydesk/[artist]/page.tsx` (new)** + +A public page that: +1. Header: "Tiny Desk Companion: [Artist Name]" +2. Subheader: "Explore the musical DNA behind the performance" +3. Influence chain with YouTube embeds at each artist node +4. Each node shows: artist name, connection description, embedded YouTube video (best live performance) +5. At the bottom: "Create your own influence map — digcrate.app" CTA +6. SEO metadata with Tiny Desk branding + +**File: `src/components/explore/video-influence-chain.tsx` (new)** + +A public-facing (no auth required) version of the InfluenceChain component that: +1. Takes an array of artist nodes with YouTube video IDs +2. Renders each node as a card with the artist info + embedded YouTube iframe +3. Draws connection lines between nodes (can be simplified arrows, not the full SVG path) +4. Mobile responsive (cards stack vertically) +5. Dark theme matching Crate brand + +**Data: Pre-generate 5 companion pages** + +Use Crate to generate influence chains for 5 popular Tiny Desk artists, then manually add YouTube video IDs for the key artists. Store as JSON in `public/tinydesk/` or in Convex. + +Suggested first 5: +1. **Khruangbin** — Thai funk > surf rock > psychedelic soul lineage +2. **Noname** — Chicago poetry > neo-soul > conscious hip-hop +3. **Mac Miller** — J Dilla > jazz rap > Pittsburgh underground +4. **Anderson .Paak** — Timbaland > funk > gospel +5. **Lizzo** — gospel > funk > Minneapolis sound + +Each one maps to a URL: `digcrate.app/tinydesk/khruangbin` + +### Effort + +- 4-6 hours +- 3 files (1 page, 1 component, data for 5 artists) +- Depends on Feature 1 being done first (reuses the public page pattern) + +### How to test + +1. Visit `digcrate.app/tinydesk/khruangbin` +2. Should show the influence chain with YouTube videos embedded +3. Click a YouTube video — plays inline +4. Scroll through the full lineage +5. Click "Dig deeper" CTA — goes to Crate sign-up +6. Share on Twitter — Open Graph preview shows artist name + Tiny Desk branding + +--- + +## Feature 3: "Keep Digging" Embed Script (later, after NPR conversation) + +### What it is + +A JavaScript embed that any website can drop in: + +```html + +``` + +Renders a "Keep Digging: Explore the influence chain" button that links to the explore page. + +### Why later + +This is the NPR integration piece. Build it after you've shown Feature 1 and Feature 2 to music directors and confirmed they want it on their sites. Don't build the embed before you have demand for it. + +--- + +## Pages to Create Summary + +| URL | What it shows | Auth required | Feature | +|-----|-------------|---------------|---------| +| `digcrate.app/explore/[artist]` | Published influence chain | No | 1 | +| `digcrate.app/tinydesk/[artist]` | Influence chain + YouTube videos | No | 2 | +| `digcrate.app/tinydesk` | Index of all companion pages | No | 2 | + +--- + +## Demo Script for NPR Music Directors + +After building Features 1 and 2, send this email to 5 music directors: + +--- + +Subject: Built something for show prep — want your feedback + +Hey [Name], + +I built a tool called Crate that I've been using at Radio Milwaukee for show prep and music research. It searches 19 music sources at once (Discogs, MusicBrainz, Genius, WhoSampled, etc.) and generates influence chains, talk breaks, and social copy. + +Two things I want to show you: + +1. **Show prep in 2 minutes:** I type `/prep HYFIN: Khruangbin > Little Simz > Noname` and get track context, talk breaks at 15/60/120 seconds, and Instagram copy. Try it free at digcrate.app. + +2. **Tiny Desk Companion:** digcrate.app/tinydesk/khruangbin — an interactive influence chain with YouTube videos at every node. Imagine this under every Tiny Desk video on your station's site. + +Would love 15 minutes to show you how it works. No pitch, just a fellow music person sharing a tool. + +— Tarik + +--- + +## Technical Notes + +- Public pages don't require Clerk auth. Use `export const dynamic = "force-static"` for pre-generated pages or fetch from Convex without auth for shared content. +- The `shares` table in Convex already stores published Deep Cuts with `shareId`, `label`, and `content`. The explore page queries this table. +- YouTube embeds use the standard iframe: `