Skip to content

Map: Native Android media player #371

Closed
12 of 12 issues completed
Closed

Description

@patricktree

Destination

A spec ready for /to-tickets that makes Cup's Android app play an audiobook's narration track reliably, and settles the web player's navigator.mediaSession action handlers and position state for every platform. It names the playback engine and the native pieces the shell gets. It covers how system media controls, headphone disconnects, audio focus and listening-position saves behave, how the result is tested on the emulator and on a physical device, and which ADRs, architecture pages and READMEs change. Its acceptance check is CUP-41's: with the device locked, playback continues through a whole audiobook on the operator's Pixel 10 Pro (Android 17); system media controls show title, cover image, whole duration and position, and their play, pause and seek actions work; playback pauses when headphones disconnect; and the saved listening position follows the same rules as the web player. Releasing the app is out of this map's scope (see Out of scope).

Notes

  • Origin: CUP-41 Native Android media player (Linear, stays in Backlog while this map is open), migrated from Android and iOS apps that play the narration track. Until 2026-11-16 this map and CUP-41 covered iOS too; iOS moved to Map: iOS app that plays the narration track and CUP-81.
  • Starting point: the shell plays through the web player's single <audio> in the WebView, best effort (ADR 0011). The Android manifest declares only INTERNET, with no service, no foreground-service permissions and no POST_NOTIFICATIONS. The web player sets only navigator.mediaSession.metadata and registers no action handlers or position state. The removed per-segment native player (Media3 service, coordinator and plugins; AccountMediaPlugin) is in git history at 0fb4e71d; it played one MP3 per segment, so none of it applies unchanged.
  • Evidence so far, none from a physical device:
    • Narration track playback in the Capacitor WebView (2026-10-27, from source): Android WebView keeps audio playing in a hidden page but has no media notification and disables the Media Session API; it proposes device experiments.
    • Android background network access and freezing (2026-11-14, Android 16 emulator, from CUP-71): about 5 to 7 s after Cup leaves the foreground, requests to cup.audio fail with UnknownHostException (Android 15 background network access); after 60 to 90 s the cached apps freezer stops audio and timers. Playing audio in the WebView does not raise the process state. Loopback is exempt, so the mobile E2E suite cannot reproduce either. CUP-71 saves a failed position again when Cup returns to the foreground.
    • The probe under Decisions so far that ran a foreground service on the emulator.
  • The CUP-41 comment of 2026-11-14 proposes, without deciding, a mediaPlayback foreground service while playing, as a small own Capacitor plugin around Media3's MediaSessionService that drives the WebView <audio>; its notification would be the media notification.
  • The mobile E2E suite runs on host-owned emulators and simulators through Appium (ADR 0018) and is part of the deploy gate.
  • Settled while charting (2026-11-14).
    • The map ends in one spec; it plans only and carries no build.
    • The engine was an open decision until 2026-11-16 and was decided again the same day after the frozen-page probe; see Decisions so far.
    • Changing the web player's media-session behavior on every platform is in scope, so the spec may reverse ADR 0011's "system media controls use browser defaults".
    • Probes run AFK on the emulator wherever it can show the behavior. HITL probes on the operator's Pixel cover only what it cannot, such as headphone routing and calls.
  • Settled at the split (2026-11-16). iOS is out of this map. The web player's navigator.mediaSession handlers and position state stay here, for every platform; the iOS map builds on them.
  • Skills: grilling and domain-modeling for grilling tickets, research for research tickets. Read the glossary, the conversion architecture (Playback; Delivery, positions and lifecycle), the signed media URL, apps/mobile-app/README.md and the mobile E2E suite first.

Decisions so far

  • Can the Capacitor WebView's audio element play a narration track with the device locked? (from the narration-track map): Android needs a mediaPlayback foreground service and a native media session for controls; device behavior unverified.

  • How do iOS Safari and Android Chrome fetch and play one long MP3 while locked? (from the narration-track map): the engine keeps issuing range requests through a locked listen without page JavaScript, so media authorization must stay valid from Play to the end.

  • How is the narration track authorized over a long listening session? (from the narration-track map): a signed track URL valid for the whole session.

  • Which native configuration stays in the Capacitor shells? (from the narration-track map): the shells got removals only; everything that makes them play moved to this increment.

  • Which plugins or players could give Cup's Android and iOS apps a media-playback foreground service and system controls under Capacitor 8?: for the WebView engine, @capgo/capacitor-media-session (Capacitor 8) and an own Media3 MediaSessionService plugin are viable, Capawesome Media Session only if paid, @jofr/capacitor-media-session (Capacitor 6, unmaintained) a reference only; no WebView option handles headphone disconnects, and the community plugins lack FOREGROUND_SERVICE_MEDIA_PLAYBACK; native players (Capawesome Audio Player, hand-written ExoPlayer) are viable but add a second engine behind the web player; from source and docs, not run.

  • Does a mediaPlayback foreground service keep Cup's WebView audio and network alive on the Android emulator?: yes on the Android 16 emulator: started on Play, it kept Cup and its renderer at process state 4 and unfrozen for 11 minutes in the background with the screen off, the MP3 kept playing and refilling its buffer, and every request to cup.audio succeeded, against a frozen, offline baseline without it; signed-URL range requests from https://localhost return 206, authorized by the token alone; page timers still slow to about once a minute once playback stops in the background; one stall under 80 kbps throttling was not reproduced.

  • Which engine plays the narration track in Cup's Android app? (decided again 2026-11-16, after the WebView suspended the hidden page on the Pixel): a native Media3 ExoPlayer in the app-local plugin's MediaPlaybackService, Android only, is the only source of playback state; the page drives it through the player port and creates no <audio>; native listening-position saves through a new signed position URL issued with the track URL; the page renews the track URL on its own Plays, and its expiry or a 403 stops the service; ExoPlayer handles focus (ducks pause) and headphone disconnects; a TWA, the mirroring WebView design and paid plugins ruled out; a new ADR supersedes ADR 0011 for Android.

  • When does Cup's Android media session service start, leave the foreground and stop? (partly superseded by the engine decision: no 2 s foreground wait; track URL expiry also stops it; and by the page–ExoPlayer interface decision: a playing session survives a swipe from recents): it starts on the first Play, not on load. Paused, including at the audiobook's end, it keeps Media3's 10-minute foreground grace, then leaves the foreground with its notification left in place. It stops only when the page unloads or releases the element (sign-out, user change, deletion), when Cup's activity is destroyed, or when the listener swipes away the notification. No POST_NOTIFICATIONS and no playback resumption.

  • Which system commands and position state does the web player's media session provide, and how does the Android plugin mirror them? (Android part superseded by the engine decision: no mirroring player or command path; the web part stands): on every platform the page handles play, pause, stop (as pause), seekto and the skips (system seekOffset, else 15 s) and reports playbackState and position state from the timeline on media events; platform/media-session.ts becomes a port with browser and Android implementations calling the same handlers; the Android SimpleBasePlayer mirrors the reported state with an extrapolating position and back 15 / play / forward 15 buttons, and sends every system command, earbud pauses and the noisy receiver included, to the page as a command event; reverses ADR 0011's browser-defaults bullet.

  • How does Cup's Android app react to earbud pauses, headphone disconnects and audio focus changes? (partly superseded by the engine decision: ExoPlayer handles focus and noisy, and duck requests pause): the plugin's noisy receiver pauses with no resume on reconnect; earbuds send AVRCP Pause/Play to the media session, so Cup needs no in-ear logic; audio focus stays with WebView (pauses on calls and resumes after transient losses, ducks to 20 % on duck requests), the plugin holds none; under Android 15 the plugin must return the service to the foreground before sending play; a call longer than the 10-minute grace may need a manual Play.

  • What is the interface between the page and the ExoPlayer in Cup's Android plugin? (replaceTrackUrl became replaceUrls in the position-save decision): PlayerHostServices.createPlayer is the seam (browser <audio> player vs native-backed player behind the same Player port); the page sends load (no prepare, so no notification before the first Play), play, pause, seekTo, native skipBy, replaceTrackUrl, close; one state event per change (with conversionId and subject), extrapolated in the page, getState() on becoming visible; the web media-session port and feat: native Android media player (#392) #401's command path go on Android; network failures pause with "Press Play to try again", a 403 stops the service and forces renewal; native stops report closed and reload on the next Play; a playing session survives a swipe from recents (Media3 default, reversing part of the lifecycle decision); a starting page adopts a running session or closes it if deleted or another account.

  • How does the server issue and check the signed position URL?: narrationTrack.positionUrl in the audiobook response, issued to account principals only, is PUT /api/audiobooks/{id}/position?token=… with a media-URL-style token (same key and lifetime) bound by "purpose": "position", so track and position tokens aren't interchangeable; the route takes exactly one of bearer or token, keeps the browser-mutation header check, and on both paths now checks an active account that owns the conversion; body unchanged (unit + offset), last write wins; stateless, not revoked by sign-out.

  • Who saves the listening position in Cup's Android app, and how does the page take over from the native player?: only the plugin writes it on Android, by the web player's rules and PlaybackSession's retry, converting with the timeline load carries; replaceTrackUrl becomes replaceUrls({ trackUrl, positionUrl }); state reports positionSave and the page shows the failure message; while a session exists the page shows the plugin's position and writes it into the cached audiobook, else the server's; close("save") on sign-out and user change saves through the position URL, deletion discards; a native stop tries one final save and otherwise drops the position, with no page takeover; no device-local positions on Android, since every app listener is signed in.

  • How is Cup's native Android playback tested on the emulator and the Pixel?: automate everything the emulator can show. The Android mobile E2E suite covers the session and its metadata, system controls (media keys, the notification's skip buttons, the seek bar), a locked listen of about 120 s with isFrozen=false and saves, focus (gsm call, transient focus, duck focus), network loss, track URL expiry or 403, swiping from recents with adoption on relaunch, sign-out and the grace period, through debug-only seams (short URL lifetimes, FocusProbePlugin, a shortened Media3 grace). The plugin's logic lives in plain Java tested with JVM JUnit in the review gate, which then needs JDK 21 and the Android SDK. Web tests go through the Player port. A checked-in manual Pixel protocol, run as CUP-41's acceptance check and again when native code, Media3 or the target Android version changes, keeps only earbud AVRCP, noisy, a real app taking focus, an earbud Play after the grace, a locked listen longer than an hour against production, and a visual check of the lock screen.

  • Which documentation does the native Android engine change?: each slice updates the pages it makes untrue, and the ADR lands with the first slice that contradicts ADR 0011. There is one new ADR, with an "Every platform" media-session section, and ADR 0011 becomes partially superseded by it; ADRs 0018 and 0020 are left as they are. A new docs/architecture/playback.md is carved out of conversion.md. A "Signed position URL" subsection goes into authentication.md. The mobile app and mobile E2E READMEs change, and a new manual protocol goes in qa/mobile-e2e/android-playback-protocol.md. The two research pages get outcome notes. Stale claims in web-app.md are fixed. The glossary does not change.

  • Rewrite the spec for Cup's native Android playback engine: Spec: Native Android media player rewritten for the ExoPlayer engine and ready for /to-tickets; re-cut into six slices on draft PR feat: native Android media player (#392) #401 (web media session port built; signed position URL; ExoPlayer driven by the page; native stops, expiry and adoption; native saves; Pixel protocol), with each documentation change assigned to its slice; Android media session plugin: playback keeps going while locked, and notification commands reach the page #396, The Android media session service ends with the audiobook #398, Headphone disconnect pauses Cup on Android #399 and Physical-device protocol and CUP-41 acceptance check #400 to be superseded and Frozen-page probe on the emulator and the Pixel #397 closed by /to-tickets.

Not yet specified

  • Nothing left: every ticket is closed and the spec is ready for /to-tickets. The map stays open until the build is merged and landed.

Out of scope

Activity

  1. changed the title [-]Map: Android and iOS apps that play the narration track[/-] [+]Map: Android app that plays the narration track[/+] on Nov 16, 2026
  2. patricktree commented on Nov 16, 2026

    @patricktree
    CollaboratorAuthor

    The map's spec: #392

  3. changed the title [-]Map: Android app that plays the narration track[/-] [+]Map: Native Android media player[/+] on Nov 17, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions