carplayd — Pi Wired CarPlay head unit

Prior wireless/macOS-oriented work has moved to the legacy branch. As of 2026-07-06, main is the proven, device-verified wired implementation below — the Raspberry Pi is the hardware this project is actually built for.

✅ VIDEO + TOUCH + AUDIO WORKING (2026-07-06) — full wired CarPlay session, device-verified.

Root cause of the original long video stall was a one-boolean Rust-port regression: the iAP2 driver ran with declare_wired=true (no 0x4301 sender) → iOS committed to a wired handshake we never complete, abandoned mDNS, and never brought up NCM (usb1 carrier down). Fix: declare_wired=false + bring usb1 admin-up/enable-IPv6. Do NOT send 0x4301 (proven dead end). Full writeup: docs/11_DECLARE_WIRED_ROOT_CAUSE_AND_FIX.md. Since then: a clean-room GTK4 render/touch frontend (docs/13_TOUCH_CONTROL_AND_UI_DONE.md) and a three-lane media/ducking/swap audio backend (docs/14_AUDIO_MEDIA_DUCKING_SWAP.md) shipped as new sibling Rust crates, device-verified against real, live CarPlay sessions — real navigation ducking real music, a real phone call correctly swapping without disturbing it.

A Raspberry Pi 4 acting as a wired-CarPlay head unit: the iPhone plugs into the Pi's USB-C/OTG port, the Pi presents an iAP2+NCM CarPlay accessory, and a baseline Carlinkit CPC200-CCPA is used only as an MFi signing oracle over its NCM link. Clean-room: no Carlinkit binaries reused; the CCPA contributes only its MFi coprocessor.

Ground rule (immutable): the CCPA is the MFi oracle and NOTHING else. The Pi does everything.

Status (2026-07-08)

2026-07-08 — protocol validated against Apple's first-party spec. The iAP2/CarPlay protocol across the whole stack is now checked and corrected against Apple's own authoritative message spec (decoded from Xcode's CarPlay Simulator), with a generated constants module (crates/iap2-core/src/spec.rs) as the single source of truth. New session-start / vehicle-data message planes were built, and Apple's IdentificationRejected retry was implemented. The entire workspace + the C gadget now compile and pass on the Pi's architecture (179 tests, clippy-clean, via the carplay-ui-build Lima VM) — no device changes, zero device-verified wire bytes altered. See docs/23_APPLE_AUTHORITATIVE_SPEC.md and docs/24_FULL_LINUX_BUILD_VERIFICATION.md.

Stage	State
Pi bring-up (OS, eth0 mgmt, toolchain)	✅ done
CCPA MFi oracle (persistent, survives power loss)	✅ done
USB role flip on dwc2 (host→gadget)	✅ SOLVED — deterministic (flip6.sh)
iAP2+NCM accessory gadget (functionfs)	✅ done
iap2_pi — iAP2 link + MFi(oracle) + Identify	✅ IdentifyAccept, device-verified
NCM bearer (usb1) up post-Identify	✅ carrier up
AirPlay receiver → H.264 video	✅ DONE — 45.9 fps device-verified
HEVC enablement + HW decode	✅ SOLVED (enabledFeatures lever + hvcC parser + rpi-hevc-dec)
Modular codec selection (H.264/HEVC)	✅ carplay_codec.sh, pre-warmed, HW-decoded
GTK4 render + touch frontend (carplay-ui/carplay-control)	✅ DONE, device-verified — doc 13
Media audio + navigation ducking + telephony/Siri swap (carplay-audio)	✅ DONE, device-verified — doc 14
Mic/uplink audio (Siri dictation, call mic)	✅ DONE, device-verified — doc 16
Now Playing + nav metadata window (carplay-metadata)	✅ DONE, device-verified — doc 17
Hardware-buttons test window (carplay-buttons)	⚠️ media buttons DONE, device-verified; Siri/nav/call-control need more work — doc 18
Wireless CarPlay research	📋 research done — doc 19
Wired/wireless session arbiter	✅ DONE, device-verified — doc 20
Wireless CarPlay Phase A1 (BT discoverability + pairing + SDP)	✅ DONE, device-verified — doc 21
Wireless CarPlay Phase A2 (iAP2-over-RFCOMM handshake)	⚠️ protocol DONE, device-verified; 3 confirmed shutdown/deployment bugs found by audit, not yet fixed — doc 22
Protocol validated vs Apple's first-party spec (spec.rs)	✅ DONE — message ids/dirs/params/enums corrected; auth planes disambiguated; zero wire-byte changes — doc 23
CarPlayStartSession/Availability + vehicle/location/destination message planes	✅ built + unit-tested (not device-exercised); wired path still must not send 0x4301 — doc 23
IdentificationRejected (0x1D03) parameter-strip retry	✅ implemented, host/VM-tested; not yet device-exercised (needs a forced reject) — doc 23
Full-workspace + C-gadget build/test on Pi arch (Lima VM)	✅ 179 tests green, clippy-clean, gcc-clean — doc 24
Modern Vehicle Data Config (VDC) over AirPlay iAP channel	📋 researched, unimplemented — doc 23 / APPLE_BEHAVIORAL_REFERENCE

The hard, unproven part — making a non-CCPA host do the wired-CarPlay USB role flip — is done. See docs/02_USB_FLIP_SOLVED.md. Video, touch, playback audio, mic uplink, and Now Playing/nav metadata are all device-verified against real CarPlay sessions; media hardware buttons are proven too, with Siri/nav/call-control needing further protocol research.

Architecture

                 USB-C splitter (constant 5V in)
   iPhone  ⇄ ───────────────────────────────── ⇄  Pi dwc2 OTG (fe980000.usb)
      │        role flips: iPhone→host, Pi→iAP2+NCM gadget          │
      │                                                             │  ep1/ep2 (iAP2)  → iap2_pi
      │                                                             │  usb1 (NCM)      → AirPlay IP
      │                                                             │
      └── (wireless radios reserved for future wireless CP) ──┐     ├─ eth0 192.168.4.253  (mgmt)
                                                              │     └─ usb0 192.168.50.100 (CCPA)
   CCPA (MFi ONLY) ⇄ usb0/NCM ⇄ Pi : mfi_oracle TCP 192.168.50.2:5290
                                     op 0x01 CopyCertificate (945B) · op 0x02 CreateSignature (128B)

Repo layout

• docs/ — the numbered writeup series (hardware/network → USB flip → video → HEVC → touch → audio), doc map below
• rust/carplayd/ — the head-unit daemon: USB supervisor, 0x51 flip, iAP2 functionfs gadget, netcfg, session arbiter, plus sibling crates crates/{iap2-core,control,ui,audio,mic,metadata,buttons,wireless} — the transport-agnostic iAP2 protocol core (link framing, identify/auth state machine, TLV builder, MFi oracle client, and spec.rs, the authoritative message catalog generated from Apple's spec — see doc 23), touch UI, audio ducking/swap, mic uplink, Now Playing/nav metadata, hardware-buttons test window, and wireless CarPlay Bluetooth bring-up
• pi/ — the iAP2 gadget C implementation (iap2_pi.c, current; iap2_ffs.c, superseded phase-1 probe) and Python probes
• scripts/ — shell orchestration: USB flip (flip6.sh), gadget bring-up (cp_accessory.sh, cp_gadget_up.sh), codec selection, MFi oracle deploy (ccpa_mfi_up.sh), video/receiver launch
• systemd/ — the 8 unit files (+timer) that run the stack in production
• boot/ — the dwc2-rolesw device-tree overlay (source .dts + compiled .dtbo) that enables the USB role switch
• vendor/ncm_carplayd/ — the AirPlay receiver (receiver_core), rx_connect/touch-input (receiver), and the MFi oracle + iap2_auth.c reference (ccpa/probes) that this project reuses rather than reinvents (see below)

Doc map

• 01_HARDWARE_AND_NETWORK.md — Pi/CCPA hardware, the splitter, network interfaces + the eth0 fix
• 02_USB_FLIP_SOLVED.md — the dwc2 role-flip breakthrough (the whole-session wall)
• 03_BRINGUP_AND_FILES.md — step-by-step bring-up, file inventory, how to reproduce
• 04_VIDEO_PIPELINE.md — receiver→:9001 render, NCM addressing, systemd services
• 05_HEVC_AND_MODULAR.md — HEVC enablement + modular codec/format design + pre-warm rule
• 06_CODEC_THROUGHPUT.md — H.264 vs HEVC bandwidth + CPU/GPU/RAM (device-measured)
• 07_SESSION_LIFECYCLE.md — start/end handling, self-healing, the physical-unplug gap
• 08_PIOS_USB_GRABBERS.md — PiOS USB gadget quirks encountered during bring-up
• 09_USB_LIFECYCLE_STATE.md — supervisor Idle/Active state machine, DETACH/ATTACH edges
• 10_RUST_CONSOLIDATION.md — the bash→Rust carplayd port (USB/iAP2 supervisor)
• 11_DECLARE_WIRED_ROOT_CAUSE_AND_FIX.md — the declare_wired regression + fix (see above)
• 12_TOUCH_CONTROL_STATUS.md — touch protocol research + the A1-vs-A3 decision (superseded by 13)
• 13_TOUCH_CONTROL_AND_UI_DONE.md — the carplay-control/carplay-ui GTK4 crates, DONE
• 14_AUDIO_MEDIA_DUCKING_SWAP.md — the carplay-audio crate: media + ducking + swap, DONE
• 15_SESSION_MGMT_RUST_UI_CUTOVER_BASELINE.md — session-mgmt hardening + status indicator; carplay-ui/carplay-audio made the live default (closing a gap where audio/touch weren't running at all), device-verified baseline
• 16_MIC_UPLINK.md — the carplay-mic crate: Siri dictation + phone-call voice uplink, DONE
• 17_NOW_PLAYING_NAV_METADATA.md — the carplay-metadata crate + metadata window: Now Playing + Route Guidance, DONE, device-verified
• 18_HARDWARE_BUTTONS.md — the carplay-buttons crate + buttons test window: media/Siri/nav/call control, plus the 2026-07-06 /info incident writeup (a bad HID-device capability declaration broke the whole session — root cause + fix + lesson applied to the rest of the feature)
• 19_WIRELESS_CARPLAY_RESEARCH.md — research phase for bringing wireless CarPlay to this Pi using its own onboard BT/WiFi: what's reusable (the AirPlay/RTSP receiver is already the same codebase as the sibling carlink_linux project's proven wireless receiver), what's genuinely new (BT bring-up, WiFi AP hosting, the credential handoff), and the one hard constraint that doesn't change (MFi auth still needs the same CCPA oracle). No implementation yet — research only.
• 20_SESSION_ARBITER.md — the wired/wireless coexistence primitive (arbiter.rs): claim/release/ preempt semantics so only one transport is ever an active connection target, wired into the existing wired supervisor and device-verified; the wireless side is a documented contract awaiting the wireless supervisor from doc 19.
• 21_WIRELESS_PHASE_A1.md — the carplay-wireless crate: Bluetooth discoverability + Just-Works pairing + a from-scratch SDP responder, device-verified end to end against a real iPhone up to the exact intended Phase A1/A2 boundary (RFCOMM data channel not yet implemented). Includes the live packet-capture evidence and what Phase A2 (iAP2-over-RFCOMM) needs next.
• 22_WIRELESS_PHASE_A2.md — the carplay-iap2-core extraction + RFCOMM listener/driver, device- verified end to end through IdentifyAccept with live Now Playing metadata. Includes a 12-agent adversarially-verified code audit's findings: 3 confirmed shutdown/deployment bugs not yet fixed (see the doc for details before further live testing).
• 23_APPLE_AUTHORITATIVE_SPEC.md — the integration of Apple's first-party iAP2/CarPlay spec (decoded from Xcode's CarPlay Simulator) as ground truth: the generated spec.rs constants module, the three refinement passes (validate/correct the message surface → build the session + vehicle-data planes → implement IdentificationRejected retry + harden the parser), the two-vehicle-data-planes finding, and the open/device-gated items. The authoritative reference for the protocol going forward.
• 24_FULL_LINUX_BUILD_VERIFICATION.md — compiling and testing the WHOLE workspace + the C gadget on the Pi's architecture (aarch64) via the carplay-ui-build Lima VM, without a Pi: how to run it, the 179-tests-green result, and the two Linux-only bugs on-target compilation caught (the metadata_server.rs const-initializer break and the control/buttons generation-counter test race).
• APPLE_BEHAVIORAL_REFERENCE.md — behavior recovered from the CarPlay Simulator Swift binary + resources: session sequencing, the video/display + HID + vehicle-data models, I2MRichParser internals, and a ranked next-work list. The evidence base for docs 23/24.
• frontend_plan_A1_carlink.md — the (superseded) carlink_linux-adoption plan doc 12 was weighing
• PI_WIRED_CARPLAY_BREAKTHROUGH.md — the original condensed breakthrough note

Reused (proven) materials — NOT reinvented

Everything but the USB transport is a direct reuse of the CCPA-proven stack, vendored in vendor/ncm_carplayd/: ccpa/probes/iap2_auth.c (reference basis for pi/iap2_pi.c), ccpa/probes/mfi_oracle.{c,armv7} (the oracle pushed onto the CCPA), receiver_core (the AirPlay receiver: RTSP + HomeKit pair-setup/verify + MFi-SAP + ChaCha20 A/V decrypt), receiver (mDNS advertise/connect-out rx_connect + touch_input). The only genuinely new work is the Pi's dwc2 USB flip, because the Pi's Synopsys dwc2 silicon differs from the CCPA's ChipIdea ci_hdrc.

Setting up on new hardware

Prerequisites: Raspberry Pi 4B running Raspberry Pi OS, a baseline Carlinkit CPC200-CCPA adapter, a USB-C splitter/hub that feeds the Pi's OTG port constant 5V while letting the iPhone's data lines reach the Pi, an iPhone, Rust + gcc/cargo/GStreamer + v4l2 HW-decode plugins, and GTK4 dev headers for the UI crate.

⚠️ Path assumption: every script and systemd unit hardcodes an absolute path under the reference deployment's home directory, ~/carplay (concretely /home/zeno/carplay there) — this mirrors the exact deployed layout, not a portable default, and systemd's ExecStart= can't expand ~ at runtime. Either (a) clone this repo to ~/carplay (i.e. carplay directly under your home directory), or (b) grep -rl '/home/zeno/carplay' scripts systemd rust pi | xargs sed -i "s#/home/zeno/carplay#$HOME/carplay#g" before installing units.

Two scripts already use ~ instead: ccpa_mfi_up.sh and video_render.sh, because their services (ccpa-mfi-oracle.service, carplay-render.service) set User=zeno, so HOME is reliably set. Every other script runs as root with no HOME (systemd doesn't set one unless User= is given), where a bare ~ silently resolves to /root instead — rx_launch.sh notes this exact gotcha in its header. Leave those, and every ExecStart=, as literal absolute paths.

1. Device tree + boot config — install boot/dwc2-rolesw.dtbo to /boot/firmware/overlays/, add dtoverlay=dwc2,dr_mode=otg and dtoverlay=dwc2-rolesw under [all] in /boot/firmware/config.txt, and load dwc2/libcomposite via /etc/modules-load.d/carplay-gadget.conf. Reboot.
2. CCPA link — connect the CCPA over its NCM USB port (usb0, Pi side 192.168.50.100, CCPA 192.168.50.2); confirm ping 192.168.50.2.
3. Symlink the vendored dependency so the hardcoded paths resolve: ln -s $(pwd)/vendor/ncm_carplayd ~/carplay/ncm_carplayd (only needed if you cloned elsewhere from ~/carplay itself).
4. Build:
   • cd rust/carplayd && cargo build --release (the head-unit daemon + control/ui/audio crates)
   • cd vendor/ncm_carplayd/receiver_core && cargo build --release (AirPlay receiver)
   • cd vendor/ncm_carplayd/receiver && cargo build --release (rx_connect, touch_input)
   • gcc -O2 -o pi/iap2_pi pi/iap2_pi.c (the iAP2 gadget daemon; see doc 10 for exact flags/libs)
   • ccpa/probes/mfi_oracle.armv7 ships prebuilt (cross-compiling for the CCPA's ARMv7 userspace needs a matching toolchain not assumed to be on hand); rebuild from mfi_oracle.c if needed.
5. Install units — copy systemd/*.service and systemd/ccpa-mfi-oracle.timer to /etc/systemd/system/, then systemctl daemon-reload && systemctl enable --now ccpa-mfi-oracle.timer carplayd.service carplay-netcfg.service carplay-receiver.service carplay-render.service carplay-rxconnect.service carplay-supervisor.service.
6. First bring-up / verification — follow docs/03_BRINGUP_AND_FILES.md: arm host mode, plug in the iPhone, run flip6.sh, and watch /tmp/iap2_ffs.log for IdentifyAccept. Doc 04 covers the video pipeline check, doc 13/14 cover touch and audio verification.

Similar works that inspired this project

RPiPlay · Pi-Carplay · node-Carplay · pycarplay · Carplay - Abuharsky