Feature/vapi

[JasonCaoGit]

Summary by CodeRabbit

• New Features

  • Record a hum from the microphone and save as WAV.
  • Convert recorded WAV to MIDI using Basic Pitch.
  • Real-time audio session: stream mic audio, receive assistant responses, detect instrument cues, then auto-record and convert to MIDI.
  • Server-side script to initiate calls via Vapi.
  • Automated remote workflow to import and play MIDI in a Linux-based DAW environment.

• Chores

  • Pinned repository Python version to 3.13.7.
  • Added project package configuration with required dependencies.

[coderabbitai]

Caution

Review failed

The pull request is closed.

Walkthrough

Adds a real-time voice-to-WS pipeline and audio-to-MIDI conversion, a microphone hum recorder, and a remote computer-use agent to upload/play a MIDI in BandLab. Introduces Node/ESM setup for Vapi server SDK, pins Python version, and writes detected instrument to a text file.

Changes

Cohort / File(s)	Summary
Remote computer automation cua.py	New async orchestrator to upload a local MIDI to a cloud Linux machine and guide a ComputerAgent through BandLab Studio import/playback steps parameterized by instrument.
Voice/WebSocket pipeline (server + client scaffolding) listen.py, listen.js, package.json	Adds WS call creation, live mic capture/streaming, server audio playback, instrument phrase detection with persistence to instrument.txt, session end handling, and post-session hum-to-MIDI conversion. Node script demonstrates Vapi server SDK usage; package marked as ES module with dependencies.
Audio capture and MIDI conversion hum.py, midi.py	New mic recorder detects trailing silence and saves WAV; MIDI conversion wrapper around Basic Pitch generates _basic_pitch.mid.
Configuration and data files .tool-versions, instrument.txt	Pins Python to 3.13.7; adds text file storing detected instrument name.

Sequence Diagram(s)

sequenceDiagram
  autonumber
  actor User
  participant Mic as Microphone
  participant Client as listen.py
  participant API as Vapi API
  participant WS as Assistant WS
  participant SPK as Speaker
  participant Post as Post-Process

  User->>Client: Run main()
  Client->>API: POST create call (API_KEY, ASSISTANT_ID)
  API-->>Client: websocket_url
  Client->>WS: Connect
  par Stream audio up
    Mic->>Client: PCM chunks
    Client->>WS: send(binary audio)
  and Receive responses
    WS-->>Client: audio/text events
    Client->>SPK: play audio frames
    Client->>Client: buffer text
    alt Instrument phrase detected
      Client->>Client: extract instrument
      Client->>Client: write instrument.txt
    end
    alt Speech status == stopped
      WS-->>Client: status: stopped
      Client->>WS: close
    end
  end
  Client->>Post: record_hum()
  Post-->>Client: hum.wav
  Client->>Post: convert_to_midi(hum.wav)
  Post-->>Client: hum_basic_pitch.mid
  Client-->>User: Done

Loading

sequenceDiagram
  autonumber
  actor Dev as Operator
  participant FS as Local FS
  participant CUA as cua.py
  participant Cloud as Remote Computer
  participant Agent as ComputerAgent
  participant App as BandLab Studio

  Dev->>CUA: computer_use_agent(midi_file, instrument)
  CUA->>FS: Read MIDI bytes
  CUA->>Cloud: Connect/start interface
  CUA->>Cloud: Upload ~/Downloads/midi-file-name.midi
  CUA->>Agent: Configure (model, prompts)
  loop Tasks
    Agent->>Cloud: UI actions (open app, import, select instrument)
    Cloud-->>Agent: Screens/feedback
    Agent-->>CUA: Streamed updates
  end
  Agent-->>CUA: Task complete

Loading

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~60 minutes

Poem

I twitched my ears at bytes that sing,
A hum becomes a MIDI string—
Websockets whisper, BandLab plays,
A piano blooms from cloud-bound ways.
I tap my paw: “All set, let’s go!” 🐇🎹
From hush to score, we ship the show.

✨ Finishing touches

• 📝 Generate Docstrings

🧪 Generate unit tests

• Create PR with unit tests
• Post copyable unit tests in a comment
• Commit unit tests in branch feature/vapi

---

📜 Recent review details

Configuration used: CodeRabbit UI

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 1c79a3a and 77be31b.

⛔ Files ignored due to path filters (4)

• hum.wav is excluded by !**/*.wav
• hum_basic_pitch.csv is excluded by !**/*.csv
• instrument.wav is excluded by !**/*.wav
• package-lock.json is excluded by !**/package-lock.json

📒 Files selected for processing (8)

• .tool-versions (1 hunks)
• cua.py (1 hunks)
• hum.py (1 hunks)
• instrument.txt (1 hunks)
• listen.js (1 hunks)
• listen.py (1 hunks)
• midi.py (1 hunks)
• package.json (1 hunks)

Tip

👮 Agentic pre-merge checks are now available in preview!

Pro plan users can now enable pre-merge checks in their settings to enforce checklists before merging PRs.

• Built-in checks – Quickly apply ready-made checks to enforce title conventions, require pull request descriptions that follow templates, validate linked issues for compliance, and more.
• Custom agentic checks – Define your own rules using CodeRabbit’s advanced agentic capabilities to enforce organization-specific policies and workflows. For example, you can instruct CodeRabbit’s agent to verify that API documentation is updated whenever API schema files are modified in a PR. Note: Upto 5 custom checks are currently allowed during the preview period. Pricing for this feature will be announced in a few weeks.

Please see the documentation for more information.

Example:

reviews:
  pre_merge_checks:
    custom_checks:
      - name: "Undocumented Breaking Changes"
        mode: "warning"
        instructions: |
          Pass/fail criteria: All breaking changes to public APIs, CLI flags, environment variables, configuration keys, database schemas, or HTTP/GraphQL endpoints must be documented in the "Breaking Change" section of the PR description and in CHANGELOG.md. Exclude purely internal or private changes (e.g., code not exported from package entry points or explicitly marked as internal).

Please share your feedback with us on this Discord post.

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}