DriftLoop AI

Collaborative, crypto-powered music generation. Create infinite, evolving songs from simple prompts (e.g. drifting ambient synth loop about late-night Pomona drives) with AI-generated instrumentals (Lyria) and vocal-style layers (OpenAI TTS). Friends add drifts in real-time; tips and votes steer direction; export stems as NFTs.

Not financial advice. Use at your own risk.

Features

• Weaves: Start a song from a text or voice prompt; get a 30s loop (Lyria + optional TTS).
• Drifts: Use /add <text> to evolve the loop (e.g. "add ethereal vocals").
• Votes / tips: Use /vote <direction> [USDC amount] to steer and tip in USDC (Base/Sepolia). A platform fee (default 10%) is applied; creator and platform addresses must be set (see Env).
• Export: Use /export to mint stems as NFTs (ERC721) via AgentKit. Requires DRIFTLOOP_NFT_CONTRACT and DRIFTLOOP_EXPORT_DEST.
• Infinite drift: Procedural BPM/filter tweaks for non-repeating playback.

Tech stack

• Backend: Node.js + TypeScript (AgentKit scaffold, LangChain).
• AI: Google Lyria (Vertex AI) for instrumentals; OpenAI TTS for vocal-style; FFmpeg for mixing.
• Data: Firestore (weaves, contributions, votes, exports).
• Crypto: AgentKit (CDP wallet, ERC20 USDC, ERC721 mint).
• Deploy: Docker, Google Cloud Run, Cloud Build.

Setup

1. Prerequisites

• Node.js 20+
• FFmpeg (apt install ffmpeg or Homebrew)
• GCP project with Vertex AI (Lyria) and Firestore enabled
• OpenAI API key
• Coinbase CDP API keys and wallet (for AgentKit)
• Telegram Bot Token (@BotFather)

2. Env

Copy .env.example to .env and set:

# GCP
GCP_PROJECT_ID=your-gcp-project-id
VERTEX_AI_LOCATION=us-central1

# OpenAI
OPENAI_API_KEY=sk-...

# AgentKit
CDP_API_KEY_ID=...
CDP_API_KEY_SECRET=...
CDP_WALLET_SECRET=...
NETWORK_ID=base-sepolia

# Telegram
TELEGRAM_BOT_TOKEN=...

# Optional: tip fee in basis points (1000 = 10%); required for tipping
DRIFTLOOP_TIP_FEE_BPS=1000
PLATFORM_WALLET_ADDRESS=0x...
CREATOR_WALLET_ADDRESS=0x...

# For /export: required for NFT mint (real Ethereum addresses)
DRIFTLOOP_NFT_CONTRACT=0x...
DRIFTLOOP_EXPORT_DEST=0x...

3. GCP

• Create a project and enable Vertex AI API and Firestore.
• Create a Firestore database (default mode).
• (Optional) For Cloud Run: enable Cloud Build, Artifact Registry (or use gcr.io), and Cloud Run Admin.

4. Install and run locally

cd driftloop-ai
npm install
npm run dev

Then set your Telegram webhook (if using webhook mode):

# Without secret (development)
curl -X POST "https://api.telegram.org/bot<TOKEN>/setWebhook?url=https://YOUR_PUBLIC_URL/webhook/telegram"

# With secret (production): set TELEGRAM_WEBHOOK_SECRET in .env, then:
curl -X POST "https://api.telegram.org/bot<TOKEN>/setWebhook?url=https://YOUR_PUBLIC_URL/webhook/telegram" \
  -d "secret_token=YOUR_TELEGRAM_WEBHOOK_SECRET"

The server verifies the X-Telegram-Bot-Api-Secret-Token header when TELEGRAM_WEBHOOK_SECRET is set. Or run the bot in polling mode by uncommenting bot.launch() in code and not using the webhook.

5. Docker (build and run)

The image uses a multi-stage build: builder stage compiles TypeScript; final stage runs production deps and FFmpeg.

cd driftloop-ai
docker build -t driftloop-ai:latest .

Run with your env file:

docker run --rm -d -p 8080:8080 --env-file .env --name driftloop driftloop-ai:latest

Or with explicit env vars:

docker run --rm -d -p 8080:8080 --name driftloop \
  -e GCP_PROJECT_ID=... -e OPENAI_API_KEY=... \
  -e CDP_API_KEY_ID=... -e CDP_API_KEY_SECRET=... -e CDP_WALLET_SECRET=... \
  -e TELEGRAM_BOT_TOKEN=... \
  driftloop-ai:latest

Test the health endpoint:

curl http://localhost:8080/health
# {"status":"ok","service":"driftloop-ai"}

Stop the container:

docker stop driftloop

6. Deploy (GCP Cloud Run)

Cloud Build uses the Dockerfile in driftloop-ai/ to build the image, then deploys to Cloud Run. From repo root:

chmod +x driftloop-ai/deploy.sh
./driftloop-ai/deploy.sh YOUR_GCP_PROJECT_ID

Or with Cloud Build config only (from repo root):

gcloud builds submit --config driftloop-ai/cloudbuild.yaml driftloop-ai

After deploy, configure Cloud Run env vars and set the Telegram webhook (see deploy.sh output).

Set Cloud Run env vars (e.g. TELEGRAM_BOT_TOKEN, OPENAI_API_KEY, CDP_*, GCP_PROJECT_ID) in the Cloud Run service or via Secret Manager. Point the Telegram webhook to https://<your-cloud-run-url>/webhook/telegram. Optionally set TELEGRAM_WEBHOOK_SECRET and pass it as secret_token when calling setWebhook.

Rate limits

• Weaves: 5 per user per hour (in-memory; use Redis/Firestore for multi-instance).
• Adds: 20 drifts per user per weave.

When Lyria or OpenAI return 429/503, the bot replies with a user-facing message asking to try again later; vocals are skipped if TTS fails (instrumental-only fallback).

Production checklist

• Set all required env vars (no placeholders). For tips: PLATFORM_WALLET_ADDRESS and CREATOR_WALLET_ADDRESS (or set per-weave creator wallet). For export: DRIFTLOOP_NFT_CONTRACT, DRIFTLOOP_EXPORT_DEST.
• Use Secret Manager or Cloud Run secrets for OPENAI_API_KEY, CDP_*, TELEGRAM_BOT_TOKEN.
• Optional: add TELEGRAM_WEBHOOK_SECRET and verify webhook requests.
• Firestore: create composite index for weaves (participantIds array-contains, createdAt desc). Create index for contributions (weaveId, userId, type) if using drift count query.
• Vertex AI: enable Lyria and grant the Cloud Run service account roles/aiplatform.user.

Defaults (2AM Pomona vibes)

• Lyria negative prompt: harsh noise, vocals
• Suggested prompts: drifting ambient synth loop about late-night Pomona drives, ethereal pad with slow filter sweep
• Slow BPM range and subtle filter movement in drift state

Project layout

• src/index.ts – HTTP server, webhook (optional secret), health
• src/bot.ts – Telegraf commands and voice (Whisper)
• src/ai-pipeline.ts – Lyria + TTS + mix + drift
• src/lyria.ts – Vertex AI Lyria REST client
• src/tts.ts – OpenAI TTS
• src/mix.ts – FFmpeg mixing and drift params
• src/agent.ts – LangChain + AgentKit (tips, votes, mint)
• src/session.ts – Firestore weaves, contributions, votes, exports
• src/weave-service.ts – Orchestration (start/add/vote/export)
• src/rate-limit.ts – In-memory rate limits (weaves/hour, adds/weave)
• src/config.ts – Env validation and config
• Dockerfile – Multi-stage build: Node 20, FFmpeg, production image
• deploy.sh – gcloud build + deploy
• cloudbuild.yaml – Cloud Build steps
• .env.example – All env vars (required and optional)

Firestore indexes

Create these composite indexes in the Firestore console:

1. weaves: participantIds (Array-contains), createdAt (Descending)
2. contributions: weaveId (Ascending), userId (Ascending), type (Ascending) — for drift count rate limit

License

Apache-2.0