[Feature]: Support streaming addon results with NDJSON - #3252
pengu-play wants to merge 6 commits into
Conversation
|
@tapframe @skoruppa some changes here may go beyond the scope of the approved issue so i want your guys' opinion i noticed that when using ndjson the stream results are sorted in the order they come in (expected behavior) but i felt its more intuitive to sort by resolution, i saw logic for this in debrid integration so i took inspiration of that. side effect is that if this were to be merged all addons will have this sorting by resolution, is that okay? or should i drop that and just keep this PR as solely adding ndjson support and not worry about sorting. thanks for the approval guys <3 |
|
after some feedback im either just going to: drop the sorting completely let me know what you think ! |
|
@pengu-play like I wrote on discord, sorting can be only for ndjson addons as we don't want to break addons sorting for people. Maybe even with ability to turn it off (although I'm not sure if it is needed). Be sure to have stable focus for when new entries come from ndjson And it would be nice to have proper PR template filled ;) @tapframe what do you think? |
b2267a7 to
8231a7f
Compare
|
okay pushed commits (loading ui bug, and only sort by resolution for these ndjson results), filled PR template (sorry), added video demo. any other comments before i mark this ready for review? |
|
Just a thought on the sorting, with something like AIOStreams if it ever got implemented, users will probably have more complex sorting configured (quality, visual tags, seadex, etc) so sorting by resolution on the Nuvio side wouldn't be suitable for a lot of people. Could the addons instead send the full, already-sorted result set each time another source finishes, with Nuvio just replacing the current list? (More like stages rather than batches) For example: Source A finishes Source B finishes This keeps all sorting logic on the addon side and Nuvio just replaces the results with the entire latest stage. |
|
i love that idea honestly. would work perfect, i'll wait for skoruppa or tap's input |
|
@DeclanSC I think it is a good idea, but it will no longer be a NDJSON but rather our own new type ;) I'm also afraid that this will significantly increase addon's bandwidth as they need to send the same list each time a new result is ready @pengu-play - if user didn't start any navigation the focus should stay on the first item. We are already checking that as addons have different priorities and can't respond on different times |
|
Could still be NDJSON, but would instead be something like: Rather than: But yeah I agree the addon bandwidth could be the main problem if returning a lot of results, hoping the impact would be minimal though. |
Summary
Add incremental NDJSON support for
/streamso addons can send{"streams":[...]}batches as each source finishes. NDJSON batches are sorted by resolution via sharedStreamResolutionhelper, shown incrementally with throttling, while plain JSON keeps addon order. Chip spinner stays loading until the addon actually finishes, and focus stays at the same visual index when sorting occurs.PR type
Why
Addons that scrape several sources sequentially force users to wait for the slowest source. #3046 proposes NDJSON so results appear as they arrive. Approved for draft PR by @tapframe on 2026-08-26.
Issue or approval
Fixes #3046
Approval: #3046 (comment)... (@tapframe: "I like the approach, you can create a draft PR")
Reproduction steps
Content-Type: application/x-ndjsonfor/stream(or usePenguPlaytest addon with staggered sources).application/jsonkeeps original order and shows once.No reproduction steps for plain JSON path beyond normal stream loading.
UI / behavior impact
Notes: Stream list ordering for NDJSON and loading spinner plus focus anchoring are behavior changes covered by #3046 approval. Plain JSON UI unchanged.
Policy check
CONTRIBUTING.md.Scope boundaries
Intentionally not changed or out of scope for this NDJSON branch:
application/json) addon order is preserved exactly as the addon sent it, no re sorting.DebridStreamPresentationreverted toupstream/devverbatim. Debrid presentation sorting only applies inside the NDJSON read path (StreamRepositoryImpl.fetchStreamsbyResolution()), not globally.StreamResolutionunification (StreamMapper,DirectDebridStreamFilter,parseQualityValueto one parser).Testing
BUILD SUCCESSFULStreamResolutionTest(9 cases: explicit heights, aliases4K,UHD,2K,FHD,HD,SD, token boundariesHDTVnotHD, generic800p, bare800, label priority, intra label highest wins)NdjsonStreamParserTest(5 cases: content type with charset and case, blank and malformed tolerance, field preservation)StreamRepositoryNdjsonTest(5 cases: batches render as they arrivebyResolution["B 2160p","A 1080p"], plain JSON keeps order["A 1080p","B 2160p"], dedup update, 404 toMISSINGvs 500 toREQUEST_FAILED)StreamRepositoryPluginIsolationTest(2 cases: addon results arrive while TMDB lookup pending):app:compileFullDebugKotlinand:app:compileFullDebugUnitTestKotlinBUILD SUCCESSFULontv_ndjsonworktrees for each commit in the series.tv_ndjsonAVDandroid-tvAPI 36x86_641920x1080 (-gpu host -feature -VulkanwithQT_QPA_PLATFORM=xcbunder Wayland). Installedapp-full-x86_64-debug.apk(92 MB) anduniversal(228 MB) viaadb install -r -tfrom5cdc4b1rebased onupstream/dev eca648a86.application/x-ndjsonaddonPenguPlaybatches appear incrementally, sorted4K>1080pas they arrive.LOADINGuntilisFinal=true(spinner visible while streams already shown).Prebuilt APKs (no build needed): https://github.com/pengu-play/NuvioTV/releases/tag/feat-ndjson-streams-5cdc4b197
NuvioTV-universal-feat-ndjson-streams-5cdc4b197.apk(228 MB)NuvioTV-x86_64-feat-ndjson-streams-5cdc4b197.apk(92 MB)Install via
adb install -r -t <apk>ontv_ndjsonAVD, or build from source:Screenshots / Video
recording_20260828_203813.mp4
https://github.com/pengu-play/NuvioTV/releases/download/feat-ndjson-streams-5cdc4b197/recording_20260828_203813.mp4
UI change is limited to stream list ordering, loading spinner, and focus behavior. Screenshot from
tv_ndjson(com.nuviodebug.com/com.nuvio.tv.MainActivity,OBSESSIONdetail):NuvioTV-universalstream list showsPenguPlay 4Kon top,PenguPlay 1080psecond. NDJSON sorted, addon chipPenguPlaystill shows loading spinner while contents are visible (see release asset).application/jsonkeeps sent order (verified viaStreamRepositoryNdjsonTest).tv_ndjson(link above), showing batches arriving and resorting, spinner persisting, and cursor staying at index.Breaking changes
None.
AddonStreams.isFinal: Boolean = trueadded with default. Existing construction sites unchanged, no wire format change forStream, no DataStore or config migration, no addon API contract change. Opt in byContent-Type: application/x-ndjson.Linked issues
Implements #3046