feat(sdk): add configurable media3 player builder

[jonathancaudill]

Okie-dokie!

First draft of some more advanced audio capabilities. I can edit the description later as needed before merge.

So as it turns out, I forgot a crucial detail in #119 . The SDK sandbox blocks most stuff regarding Context. So a straight-up handoff isn't really feasible. Below is what I think will work pretty well, and it still honors the split.

SDK's default audio behavior and simple building of LightAudioPlayer instances is unchanged.

SDK will allow more advanced specs to be given when building an ExoPlayer instance via the configure argument. SDK will also allow tools to change aspects of the media3 pipeline live--queueing, etc. Which is good, because streaming requires a lot of small changes to be made as cell service comes and goes, as the user queues up tracks, etc.

SDK will also enforce one LightMediaEnv per tool & one SimpleCache object per directory.

[dupontgu]

Excited to take a look at this! Hopefully will get a chance by EOW

[dupontgu]

I did a quick pass and I know @brunoro is doing the same since he's kinda owning the audio stuff.

I've just got a few little API suggestions that I put up here: 5d073d2

Basically just made a base LightAudioPlayerCore class that can take any arbitrary Player instance (as you added), and then made it so LightAudioPlayer extends that but explicitly takes in an ExoPlayer instance. I think it A) makes it a little clearer that LightAudioPlayer is kind of a safe default (but you can write your own extension of LightAudioPlayerCore if you wanna go nuts) and B) adds a cleaner surface to add those ExoPlayer-specific calls like skipSilenceEnabled.

[brunoro]

Hi @jonathancaudill, thanks for the contribution! I'm glad that you already did some work on Phono, since it's a great real life tool use case that will allow us to properly tune the LightAudioPlayer SDK surface.

I think that the approach your proposed is a good compromise between providing a full Light SDK-wrapped layer for general use cases and leaking Android media objects for advanced usage. Me and @dupontgu both agree it flows well with our thoughts of building an straightforward and opinionated audio library (vs building a even more opinionated audio engine). I'll expand a bit on the next comments on what I see as the two open questions we need cover before merging the PR.

[brunoro]

What it would take to support the proposed API changes on detached audio mode?

This is the trickier part since it involves many aspects of the API. The architecture we're working on for detached (background) audio is to have an SDK-owned foreground service per tool that would own the player instance. That raises some issues:

• LightPlayerConfigurator callbacks wouldn't be able to configure the ExoPlayer instance living in the service → this could be solved by configuring the player object service-side through Binder IPC. We'd need to make sure that the configuration object is serializable but it shouldn't be a huge lift. Same applies to other things that operate on player callbacks.
• State, lifecycle and focus handling might require some additional logic on tool side, since on release the tool just disconnects from the service, and on restart it needs to figure out how to rebuild it's internal state from the current player state → this might involve moving to the service side any eventual player logic that isn't represented in media3.
• Error handling that relies on exception will need to go through an IPC indirection layer → some additional serialization and communication work on tool side would solve this.

[brunoro]

Are the proposed changes enough for a Phono Tidal MVP?

Now I'm getting more on speculative territory because I only scratched the surface on the Phono codebase, so please help me expand the list. I noticed that the Phono Tidal backend uses some features that aren't supported by LightAudio atm:

• DASH playback and downloads: would adding the media3-exoplayer-dash dependency solve it? We would need to wrap it so not all projects depend on it, but it should be easy enough.
• Player error visibility: LightAudio doesn't expose any of that, and having access to the media3 player solves it, but it seems to me like other tools might need, so we might actually pull some of that behavior in the SDK.
• Queue mutation: same as player errors, it's something that wasn't covered but our API that likely other audio tools might need.
• Player state: same as error visibility and queue mutation, it's something we'd want to pull into LightAudioPlayer.
• Track caching: solved in this PR by LightMediaEnv, including any possible lifecycle issues since player and cache aren't released together and cache can survive an eventual player recreation.
• Audio focus handling: I saw that Phono does some of this and it's definitely something we'll want to have the SDK to handle. This needs some care since a tool can override it using the leaked media3 player object.
• Offline downloads: DownloadManager or DownloadService would both require a Context object to be initialized. I'm wondering if we can consider this to be out of the MVP initially, since it opens a bunch of other side quests that we can postpone.
• Reacting to connectivity, battery and power events: ConnectivityManager, BatteryManager and PowerManager all would require Context access. This is something that could also make it into the SDK, not only for audio tools. However, I wonder if the logic on Phono that reacts to these could be simplified to something like caching current track + resolving and prefetching the next one, plus some retry policy.

[jonathancaudill]

@dupontgu @brunoro thanks for all these awesome comments. I'll move in relevant commits and write out my thoughts on @brunoro 's comments when I get some time this evening!

[jonathancaudill]

• Audio focus handling: I saw that Phono does some of this and it's definitely something we'll want to have the SDK to handle. This needs some care since a tool can override it using the leaked media3 player object.

Yes. will definitely need to dedupe this when phono migrates to SDK.

• Offline downloads: DownloadManager or DownloadService would both require a Context object to be initialized. I'm wondering if we can consider this to be out of the MVP initially, since it opens a bunch of other side quests that we can postpone.

Yeah, this one is tricky. My plan was to just run in foreground when tool is open, then ship off as LightWork tasks if tool is closed. Obviously some kind of DownloadManager in the SDK would be good--can maybe mock something up this week or next.

• Reacting to connectivity, battery and power events: ConnectivityManager, BatteryManager and PowerManager all would require Context access. This is something that could also make it into the SDK, not only for audio tools. However, I wonder if the logic on Phono that reacts to these could be simplified to something like caching current track + resolving and prefetching the next one, plus some retry policy.

I would actually push back on this a little bit--Guy had mentioned that it would be good to put these in the SDK, and I definitely agree. And in the short term if we migrate to the SDK before those are implemented, your idea could definitely work.

The main issue is stream quality, since phono will bump stream quality up/down to speed up the caching process per-song. This could probably be replicated with a pretty simple "speed test" of sorts where if caching is moving unusually slow we bump stream quality down. I do need to work on making this more stable for phono as well, so I'm sure the caching will improve and change over time. I do think that the best solution will probably involve some kind of access to these events, though. This also applies to automatically-detected offline mode.

[jonathancaudill]

DASH playback and downloads: would adding the media3-exoplayer-dash dependency solve it? We would need to wrap it so not all projects depend on it, but it should be easy enough.

Yeah, I think this would be a good solution.