Skip to content

AGENTS.md

Latest commit

 

History

History
172 lines (147 loc) · 11.6 KB

AGENTS.md

File metadata and controls

172 lines (147 loc) · 11.6 KB
Raw

AGENTS.md

Guide for AI agents (and humans) working on Continuity. Read this first; it encodes the architecture, commands, conventions, and traps that aren't obvious from the code.

What this is

Native iOS 26 music app. Flagship: transitions so smooth you don't notice the song changed — stem-separated, beatmatched, harmonically-mixed DJ blends. Audio is sourced from YouTube; Spotify contributes tracklists only (its audio is DRM-locked). Personal prototype — not App-Store-shippable as-is (YouTube ToS). See README.md for the feature tour.

Architecture

Layered Swift-package modules plus the app target. Each layer owns a directory and may only depend downward; the compiler blocks reaching into another layer.

Module ownership

Module Directory Owns Depends on
ContinuityCore Packages/ContinuityCore all scraping/parsing + all DSP/math. NO UIKit/AVFoundation/SwiftData. ~114 unit tests. (nothing)
Domain Packages/ContinuityKit/Sources/Domain @Model Track/Playlist, TransitionSettings, AudioCache/StemCache ContinuityCore
Ingest Packages/ContinuityKit/Sources/Ingest resolvers, download, analysis, stems, PreparationQueue, LibraryCleanup Domain, ContinuityCore, YouTubeKit, onnxruntime
Playback Packages/ContinuityKit/Sources/Playback Player, Deck, NowPlayingBridge, PlaybackStateStore, ToneSynth Domain, ContinuityCore
app target App/Continuity (Views/*, Library/SampleData) SwiftUI, app wiring all package products

Playback is a sibling of Ingest and never imports it — they meet only through Domain and callbacks (e.g. Player.onUpcomingTracks → PreparationQueue.ensureStems).

The rule: an agent edits only its module's directory; the compiler enforces the boundary. Anything fragile (page scraping) or mathematical (DSP, transition logic) lives in ContinuityCore, pinned by unit tests — verifiable without a simulator. Big types are split into by-concern extension files (Player+Transport.swift, PreparationQueue+Sync.swift, …) so agents editing different concerns of the same type don't collide; stored properties + init stay in the core file, methods move to the extensions.

Data flow: AddMusicView → PreparationQueue (resolve → download → analyze → stems, bounded concurrency) → SwiftData. Player owns one AVAudioEngine with two Decks; transitions blend via ContinuityCore.TransitionPlan.

Build & test

Requires Xcode 26+. xcode-select points at CommandLineTools, so prefix Xcode tools with DEVELOPER_DIR:

# ContinuityCore tests (fast, no simulator — run for any ContinuityCore change):
cd Packages/ContinuityCore && DEVELOPER_DIR=/Applications/Xcode.app/Contents/Developer swift test

# App + modules build (regenerate the project first if you added/removed/renamed a file):
xcodegen generate
DEVELOPER_DIR=/Applications/Xcode.app/Contents/Developer xcodebuild \
  -project Continuity.xcodeproj -scheme Continuity \
  -destination 'platform=iOS Simulator,id=<UDID>' build

# Module tests (Domain/Ingest/Playback — SwiftData/AVFoundation, so they need the simulator):
DEVELOPER_DIR=/Applications/Xcode.app/Contents/Developer xcodebuild test \
  -project Continuity.xcodeproj -scheme Continuity \
  -destination 'platform=iOS Simulator,id=<UDID>'

The .xcodeproj is generated by XcodeGen from project.yml and is gitignored — never edit or commit it. Bundle id com.sanylax.continuity (share extension com.sanylax.continuity.share; app group group.com.sanylax.continuity).

Conventions

  • Add a Core module for new parsing/DSP; expose a small public API; write tests in Packages/ContinuityCore/Tests.
  • Protocol boundaries live in Ingest/IngestContracts.swift (AudioStreamResolving, PlaylistResolving, VideoMetadataResolving, …). Implement against these so the two sides of a boundary can be worked independently and mocked in tests.
  • Logging: os.Logger(subsystem: "com.continuity.app", category: …). Log failures, not happy paths.
  • Comments: explain the non-obvious why, not the what. Keep them short.
  • Analysis changes that should reach existing libraries: bump TrackAnalyzer.analysisVersion — tracks below it re-analyze at launch.

Gotchas (these have bitten us)

  • New files need xcodegen generate before xcodebuild, or you get "cannot find X in scope." XcodeGen uses explicit file lists.
  • Schemes come only from project.yml. XcodeGen emits no scheme unless a target declares scheme: (the Continuity target does — keep it). xcodebuild on Xcode 26.x does not auto-create schemes the way the Xcode GUI does, so without it -scheme Continuity fails in ~30s with exit 65 "does not contain a scheme named" — which is exactly how every TestFlight run from PR #127 to #137 died.
  • onnxruntime is a static .framework (ar archive), not a dylib. Xcode still embeds a broken ~50 KB stub into Continuity.app/Frameworks. Never "fix" that stub by patching MinimumOSVersion and re-signing — that cured ITMS upload checks while leaving a poison Frameworks entry (and earlier, an invalidated signature → device install 0xe8008001). The app already statically links ORT into its binary. The Continuity target's post-build script must delete onnxruntime.framework / onnxruntime_extensions.framework from the app bundle after Embed Frameworks. Do not reintroduce plist-patch/resign for them.
  • Stem separation jetsams at ~3.4 GB (per-process-limit) if the ORT session for HT-Demucs starts at cold launch. Player.prepare / restore must stay metadata-only — no notifyUpcoming() → ensureStems until the first real play (ensureCurrentLoaded / startCurrentFresh). Keep the stem limiter at 1; never separate the whole library eagerly. Use CPU EP only (never CoreML EP on device — session creation alone has hit the per-process limit). Cap ORT intraOpNumThreads to 1.
  • Stem separation streams — don't reintroduce whole-file buffers. The pipeline (chunked decode → windowed inference → overlap-add → incremental AAC encode) keeps peak memory O(segment) regardless of track length; the old whole-file implementation held ~7 full-length float buffers (>1 GB on an hour-long track) and jetsammed devices. The overlap-add math is StreamingOverlapAdd in ContinuityCore (unit-tested); simulator tests for the pipeline live in ContinuityKit/Tests/IngestTests (fake-inference seam, no model download; run via xcodebuild test -scheme ContinuityKit-Package with a simulator destination).
  • Stem separation on the Simulator is also CPU-only (and slow). Do not "fix" perceived hangs by enabling CoreML on sim — sim CoreML has no ANE/GPU and routes through a ~100× slower serial CPU queue.
  • YouTubeKit is pinned exact: on purpose — bump it deliberately, never float on main. Stream URLs come from whichever InnerTube client the pinned YouTubeKit asks; YouTube retires clients without notice (Aug 2026: ANDROID_VR URLs started 403-ing after the first ~1 MB, so every 1 MiB ranged download died on chunk #2 → streamURLExpired on every track → minutes of spinner, then the orange retry badge). The signature of "YouTube changed again" is every imported track failing with prep failed for …: streamURLExpired (or .network) in Console; playlist/search scrapes still succeed. First response: check upstream YouTubeKit for a newer tag, bump exact: in Packages/ContinuityKit/Package.swift, and re-run the opt-in probe CONTINUITY_LIVE_PROBE=1 … -only-testing:IngestTests/LiveIngestProbeTests (simulator) — it prints the failing stage and exact error per track. Beware: a scratch SwiftPM harness that depends on branch: "main" silently resolves upstream HEAD, not the app's pin.
  • Scrapers are fragile by design. YouTube/Spotify change their embedded JSON shapes without notice (YouTube moved playlists to lockupViewModel mid-project). Parsers handle multiple shapes and are pinned by tests against real fixtures. Resolvers retry transient failures (Ingest/Retry.swift) and classify errors (IngestError.network/.rateLimited/.sourceUnavailable).
  • Stem cache is size-budgeted (8 GB LRU) and demand-driven — never separate the whole library eagerly (that's CPU-hours and gigabytes). Player.onUpcomingTracks drives just-in-time separation of the play-queue neighborhood after playback starts.
  • Deleting tracks: call Player.handleDeleted before the SwiftData delete — a deck/queue reference to a freed @Model crashes.
  • Spotify caps ~50 tracks (anonymous embed API). YouTube playlists paginate to ~500.
  • git push hangs on this machine (GCM GUI dialog). Push with: GIT_TERMINAL_PROMPT=0 git -c credential.helper= -c credential.https://github.com.helper='!/opt/homebrew/bin/gh auth git-credential' push -u origin <branch>

Workflow (multi-agent)

  • GitHub is the source of truth. git fetch origin and base branches on the latest origin/main; don't assume local matches remote.
  • One feature/fix per branch → one PR → origin/main. The org ruleset requires PRs (direct pushes to protected refs are rejected) and you cannot add commits to an already-pushed branch — finish a unit, push a fresh branch, open its PR. The author merges.
  • Pick low-contention work. The layer split means an agent in Ingest/ rarely collides with one in Playback/ or Views/. The former hotspots (Player, PreparationQueue) are now split into by-concern extension files — edit the relevant +Concern.swift, not the core file.
  • Keep PR descriptions and commit messages tight: what changed and why, in a couple of sentences.

Cursor Cloud specific instructions

Cloud agents run on Linux (Ubuntu 24.04), not macOS — so most of this project cannot be built or run here. Plan work accordingly:

  • iOS app target, ContinuityShare, and all of ContinuityKit (Domain/Ingest/Playback) are Linux-unbuildable. They need macOS + Xcode 26, and pull in SwiftData / AVFoundation / CoreML plus the iOS-only ContinuityKit platform pin (platforms: [.iOS("26.0")]). xcodebuild, the iOS Simulator, and xcodegen do not exist here. Validate changes to those modules by code review + the ContinuityCore unit tests they rely on; real build/run must happen on a Mac.
  • A Linux Swift toolchain is preinstalled via swiftly (added to ~/.profile, so swift is on PATH in login shells). Use swift, not DEVELOPER_DIR/xcodebuild, on this VM.
  • swift test in Packages/ContinuityCore FAILS to compile on Linux, even though the README implies it's portable: BeatTracker.swift and KeyDetector.swift import Accelerate (Apple's vDSP — no Linux module). Since SwiftPM compiles the target as a unit, those two files break the whole build/test, including the 15 pure-Foundation files.
  • To run the platform-agnostic tests on Linux, build a throwaway SwiftPM package that includes only the non-Accelerate sources + tests (exclude BeatTracker.swift, KeyDetector.swift, BeatTrackerTests.swift, KeyDetectorTests.swift, KeyDetectorAccuracyTests.swift). That runs 111 of ~114 real XCTest cases (Camelot/flow ordering, crossfade curves, transition plan, loudness, silence trimming, YouTube/Spotify URL + page parsing). The BPM/beat-grid and musical-key suites (Accelerate) can only be exercised on a Mac. Test fixtures are inline string literals — no external resource files needed.
  • The core package has zero external SwiftPM dependencies, so there is nothing to fetch on startup; the update script only warms the build graph.