Skip to content

[Feature]: Support streaming addon results with NDJSON - #3252

Draft
pengu-play wants to merge 6 commits into
NuvioMedia:devfrom
pengu-play:feat/ndjson-streams
Draft

pengu-play wants to merge 6 commits into
NuvioMedia:devfrom
pengu-play:feat/ndjson-streams

Conversation

@pengu-play

@pengu-play pengu-play commented Aug 28, 2026

Copy link
Copy Markdown

Summary

Add incremental NDJSON support for /stream so addons can send {"streams":[...]} batches as each source finishes. NDJSON batches are sorted by resolution via shared StreamResolution helper, shown incrementally with throttling, while plain JSON keeps addon order. Chip spinner stays loading until the addon actually finishes, and focus stays at the same visual index when sorting occurs.

PR type

  • Translation/localization only
  • Critical bug fix

This is a feature with maintainer approval, not a bug fix or translation. See Issue or approval.

Why

Addons that scrape several sources sequentially force users to wait for the slowest source. #3046 proposes NDJSON so results appear as they arrive. Approved for draft PR by @tapframe on 2026-08-26.

Issue or approval

Fixes #3046
Approval: #3046 (comment)... (@tapframe: "I like the approach, you can create a draft PR")

Reproduction steps

  1. Install an addon that serves Content-Type: application/x-ndjson for /stream (or use PenguPlay test addon with staggered sources).
  2. Open a detail page and go to stream selection.
  3. Observe: first batch appears quickly, later batches appear and resort by resolution, chip stays loading, cursor stays at index.
  4. Control: same addon with application/json keeps original order and shows once.

No reproduction steps for plain JSON path beyond normal stream loading.

UI / behavior impact

  • No UI change
  • No behavior change
  • UI changed only to fix a documented glitch/bug
  • Behavior changed only to fix a documented bug/regression
  • UI change has explicit maintainer approval
  • Behavior change has explicit maintainer approval

Notes: Stream list ordering for NDJSON and loading spinner plus focus anchoring are behavior changes covered by #3046 approval. Plain JSON UI unchanged.

Policy check

  • I have read and understood CONTRIBUTING.md.
  • This PR fits the current PR policy: localization/translation only or a critical bug fix. This is a feature with explicit maintainer approval for a draft PR as above.
  • This PR does not add features, UI changes, refactors, or other non-critical changes beyond the approved scope.
  • This PR is small, focused, and limited to one issue ([Feature]: Support streaming addon results with NDJSON #3046).
  • This PR does not bundle unrelated refactors, cleanups, formatting, or drive-by changes.
  • This PR includes a linked issue, reproduction steps, and testing notes as a feature with approval.
  • I listed the testing performed below.

Scope boundaries

Intentionally not changed or out of scope for this NDJSON branch:

  • Plain JSON (application/json) addon order is preserved exactly as the addon sent it, no re sorting.
  • DebridStreamPresentation reverted to upstream/dev verbatim. Debrid presentation sorting only applies inside the NDJSON read path (StreamRepositoryImpl.fetchStreams byResolution()), not globally.
  • No new user facing setting or toggle added. Incremental path is content type opt in per [Feature]: Support streaming addon results with NDJSON #3046.
  • No unrelated refactors beyond StreamResolution unification (StreamMapper, DirectDebridStreamFilter, parseQualityValue to one parser).
  • No UI polish beyond required behavior: chip spinner persistence and index anchored cursor.
  • No DataStore, config, or schema migration.

Testing

  • Unit (fullDebug): BUILD SUCCESSFUL
    • StreamResolutionTest (9 cases: explicit heights, aliases 4K, UHD, 2K, FHD, HD, SD, token boundaries HDTV not HD, generic 800p, bare 800, label priority, intra label highest wins)
    • NdjsonStreamParserTest (5 cases: content type with charset and case, blank and malformed tolerance, field preservation)
    • StreamRepositoryNdjsonTest (5 cases: batches render as they arrive byResolution ["B 2160p","A 1080p"], plain JSON keeps order ["A 1080p","B 2160p"], dedup update, 404 to MISSING vs 500 to REQUEST_FAILED)
    • StreamRepositoryPluginIsolationTest (2 cases: addon results arrive while TMDB lookup pending)
  • Compile per commit: :app:compileFullDebugKotlin and :app:compileFullDebugUnitTestKotlin BUILD SUCCESSFUL on tv_ndjson worktrees for each commit in the series.
  • Manual (emulator): tv_ndjson AVD android-tv API 36 x86_64 1920x1080 (-gpu host -feature -Vulkan with QT_QPA_PLATFORM=xcb under Wayland). Installed app-full-x86_64-debug.apk (92 MB) and universal (228 MB) via adb install -r -t from 5cdc4b1 rebased on upstream/dev eca648a86.
    • NDJSON application/x-ndjson addon PenguPlay batches appear incrementally, sorted 4K > 1080p as they arrive.
    • Plain JSON addon order untouched.
    • Chip stays LOADING until isFinal=true (spinner visible while streams already shown).
    • Cursor stays at same visual index (for example 2nd) when resort happens.

Prebuilt APKs (no build needed): https://github.com/pengu-play/NuvioTV/releases/tag/feat-ndjson-streams-5cdc4b197

  • NuvioTV-universal-feat-ndjson-streams-5cdc4b197.apk (228 MB)
  • NuvioTV-x86_64-feat-ndjson-streams-5cdc4b197.apk (92 MB)
    Install via adb install -r -t <apk> on tv_ndjson AVD, or build from source:
NUVIO_RELEASE_STORE_FILE=~/.android/debug.keystore NUVIO_RELEASE_KEY_ALIAS=androiddebugkey \
  NUVIO_RELEASE_KEY_PASSWORD=android NUVIO_RELEASE_STORE_PASSWORD=android \
  ./gradlew :app:testFullDebugUnitTest --tests "com.nuvio.tv.data.repository.StreamRepository*"

Screenshots / Video

recording_20260828_203813.mp4

https://github.com/pengu-play/NuvioTV/releases/download/feat-ndjson-streams-5cdc4b197/recording_20260828_203813.mp4

UI change is limited to stream list ordering, loading spinner, and focus behavior. Screenshot from tv_ndjson (com.nuviodebug.com/com.nuvio.tv.MainActivity, OBSESSION detail):

  • NuvioTV-universal stream list shows PenguPlay 4K on top, PenguPlay 1080p second. NDJSON sorted, addon chip PenguPlay still shows loading spinner while contents are visible (see release asset).
  • Plain JSON control: same addon with application/json keeps sent order (verified via StreamRepositoryNdjsonTest).
  • Video: screen recording of NDJSON incremental batches on tv_ndjson (link above), showing batches arriving and resorting, spinner persisting, and cursor staying at index.

Breaking changes

None.

  • AddonStreams.isFinal: Boolean = true added with default. Existing construction sites unchanged, no wire format change for Stream, no DataStore or config migration, no addon API contract change. Opt in by Content-Type: application/x-ndjson.

Linked issues

Implements #3046

@pengu-play

pengu-play commented Aug 28, 2026

Copy link
Copy Markdown
Author

@tapframe @skoruppa some changes here may go beyond the scope of the approved issue so i want your guys' opinion

i noticed that when using ndjson the stream results are sorted in the order they come in (expected behavior) but i felt its more intuitive to sort by resolution, i saw logic for this in debrid integration so i took inspiration of that.

side effect is that if this were to be merged all addons will have this sorting by resolution, is that okay? or should i drop that and just keep this PR as solely adding ndjson support and not worry about sorting.

thanks for the approval guys <3

@pengu-play

Copy link
Copy Markdown
Author

after some feedback im either just going to:

drop the sorting completely
or
keep the sorting just for ndjson

let me know what you think !

@skoruppa

skoruppa commented Aug 28, 2026

Copy link
Copy Markdown
Member

@pengu-play like I wrote on discord, sorting can be only for ndjson addons as we don't want to break addons sorting for people. Maybe even with ability to turn it off (although I'm not sure if it is needed). Be sure to have stable focus for when new entries come from ndjson

And it would be nice to have proper PR template filled ;)

@tapframe what do you think?

@pengu-play
pengu-play force-pushed the feat/ndjson-streams branch from b2267a7 to 8231a7f Compare August 29, 2026 00:05
@pengu-play

Copy link
Copy Markdown
Author

okay pushed commits (loading ui bug, and only sort by resolution for these ndjson results), filled PR template (sorry), added video demo.

any other comments before i mark this ready for review?

@DeclanSC

DeclanSC commented Aug 29, 2026

Copy link
Copy Markdown
Contributor

Just a thought on the sorting, with something like AIOStreams if it ever got implemented, users will probably have more complex sorting configured (quality, visual tags, seadex, etc) so sorting by resolution on the Nuvio side wouldn't be suitable for a lot of people.

Could the addons instead send the full, already-sorted result set each time another source finishes, with Nuvio just replacing the current list? (More like stages rather than batches)

For example:

Source A finishes
Stage 1
A1
A2
A3

Source B finishes
Stage 2 (containing all of Stage 1 with the new results sorted)
A1
B1
A2
B2
B3
A3

This keeps all sorting logic on the addon side and Nuvio just replaces the results with the entire latest stage.

@pengu-play

Copy link
Copy Markdown
Author

i love that idea honestly. would work perfect, i'll wait for skoruppa or tap's input

@skoruppa

skoruppa commented Aug 29, 2026

Copy link
Copy Markdown
Member

@DeclanSC I think it is a good idea, but it will no longer be a NDJSON but rather our own new type ;) I'm also afraid that this will significantly increase addon's bandwidth as they need to send the same list each time a new result is ready

@pengu-play - if user didn't start any navigation the focus should stay on the first item. We are already checking that as addons have different priorities and can't respond on different times

@DeclanSC

Copy link
Copy Markdown
Contributor

Could still be NDJSON, but would instead be something like:
{"streams":[A1,A2,A3]}
{"streams":[A1,B1,A2,B2,B3,A3]}
{"streams":[A1,C1,B1,A2,B2,C2,B3,A3]}

Rather than:
{"streams":[A1,A2,A3]}
{"streams":[B1,B2,B3]}
{"streams":[C1,C2]}

But yeah I agree the addon bandwidth could be the main problem if returning a lot of results, hoping the impact would be minimal though.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Feature]: Support streaming addon results with NDJSON

3 participants