v0.9.4

CHANGELOG.md

@@ -1,5 +1,80 @@
 # Changelog
 
+## 0.9.4
+
+**After upgrading, restart Dispatcharr, then press Test portals and Re-fetch
+all.** The restart is what puts every worker on the new code, since plugins are
+loaded once per process. Sync on its own would report every portal as unchanged
+and fetch nothing — it compares your settings against what was last published,
+and nothing in them changed, only the code did.
+
+- **Stalker compatibility improvements.**
+- **Security improvements in the Settings panel, which now masks credentials.**
+
+<!-- details -->
+
+**Talking to more portals**
+
+- **A refusal is recognised however the portal words it.** Ministra answers a
+  rejected session with HTTP 200 and a bare line of text, so only one exact
+  phrase was ever understood; the others arrived as "portal returned non-JSON
+  response" and cost a channel its failover. `Access denied.`, `Unauthorized
+  request.`, the stock server's numeric debug counter and the refusals
+  non-Ministra panels put inside the JSON envelope are all read now, and each
+  says which of the three things is actually wrong: the session, the account,
+  or the MAC.
+- **A portal is looked for on every path one is served from.** Two were probed;
+  five are now — the URL as written first, then `/server/load.php`,
+  `/c/portal.php`, `/portal.php` and `/stalker_portal/server/load.php`. An
+  explicit `.php` in your own URL is no longer swapped for a guess, since that
+  path is the address the provider handed out.
+- **The box describes itself the way a real one does.** The handshake and
+  profile now carry `prehash`, `client_type`, `video_out` and the metrics blob
+  a MAG sends, which is what some panels authenticate on and what the admin
+  panel reads to show a box as connected.
+- **A command reaches the portal decoded exactly once.** A `%` inside a channel
+  command was being encoded twice, so those channels asked for a link that
+  never existed.
+- **Channels the portal marks as needing no temporary link now play without one**
+  — the portal's own player skips `create_link` for them, and so does this. It
+  removes the one request known to go wrong on those providers.
+- **The portal's own streams are fetched as the box that authenticated**, with
+  the session and MAC that minted the link. Never to anyone else's CDN: a
+  stream on another host gets no credentials, which is what keeps a
+  subscriber's MAC out of a request that has no business carrying it.
+- Three things the portal was already saying are now read: the expiry date
+  where Ministra puts it, a device-conflict message with the setting that fixes
+  it named, and a channel listed twice in one response counted once.
+
+**Credentials are no longer displayed**
+
+- **The Portals box hides them once saved.** The MAC, the password and the rest
+  of the account identity come back as `••••`: the real line lives in
+  `/data/distalker/portals.txt`, which the settings panel cannot read. Names,
+  URLs and the tuning keys stay, so the box still reads as your own
+  configuration — and a portal is still deleted by deleting its line.
+- This matters beyond the screen. Dispatcharr serves a plugin's settings row to
+  every account on the install, so the credentials were in an API response
+  anyone could read.
+- **Editing works through the bullets.** Renaming a portal and repointing its
+  URL both keep the MAC you cannot see; pasting a line back in full still
+  works. Change a line's name *and* its URL in one edit and there is nothing
+  left to recognise it by — the plugin then quotes that line back and asks you
+  to retype it, rather than parsing a MAC address made of bullets.
+- **Keep your own copy of each portal line in a password manager.** The box will
+  not give a credential back, and if `/data/distalker` is ever lost the last
+  copy goes with it. Nothing is redacted until that file is known to hold the
+  same list, so a registry that could not be written leaves the credentials
+  where they were.
+- Hiding is not encryption. The resolver reads the MAC on every tune in a
+  process with no database, so `portals.txt` and the state mirrors go on
+  holding it in the clear at `0600` — the README says which files those are.
+- **Settings the panel no longer has a field for are dropped.** Removing a field
+  from the manifest stops the panel rendering it and nothing else, so the
+  Add-portal form retired in 0.4.0 left a MAC, a password and a portal URL in
+  that same API response — for a portal you may have deleted months ago. They
+  are cleared on the first action after upgrading.
+
 ## 0.9.3
 
 **After upgrading, restart Dispatcharr, then press Test portals and Re-fetch

CONTRIBUTING.md

@@ -139,10 +139,10 @@ added or deleted would silently rebind existing channels to the other portal.
 
 It runs inside the container with no persistence, so it comes back empty from
 every restart — which used to kill every channel until someone pressed Sync,
-twice observed before it was understood. Everything `save_portal` and
-`save_fallback` publish is therefore mirrored to `/data/distalker/state/*.json`
-(`0600`: credentials), read only when Redis has nothing, and written back to
-Redis on first use. Reads go through `_client_or_none`, so an unreachable Redis
+twice observed before it was understood. Everything `save_portal`,
+`save_static_cmds` and `save_fallback` publish is therefore mirrored to
+`/data/distalker/state/*.json` (`0600`: credentials), read only when Redis has
+nothing, and written back to Redis on first use. Reads go through `_client_or_none`, so an unreachable Redis
 degrades rather than raising. The session token is deliberately *not* mirrored:
 it expires within the hour, and losing it costs one handshake.
 
@@ -151,6 +151,31 @@ before the first sync, or a lost data volume, and there is again nothing to
 read. `Plugin._republish` runs on the assign path instead, which the button,
 every `m3u_refresh` and every `channel_error` all reach.
 
+### Some channels never reach the portal at tune time
+
+A Stalker row carries `use_http_tmp_link` and `use_load_balancing`, and the
+portal's own `player.js` calls `create_link` only when one of them is set —
+everything else plays the `cmd` the listing already gave. pvr.stalker does the
+same and cites that line for it (`ChannelManager::GetStreamURL`).
+
+Sync evaluates `plays_without_create_link` per row and publishes the commands
+that qualify as a set (`save_static_cmds`); `resolver.resolve` checks it before
+doing anything else, and on a hit returns the command's own URL with no
+handshake and no `create_link`. On nearly every portal the set is empty — a
+stock row is a loopback marker, which only the portal can resolve whatever its
+flags say. The family it exists for is the one `undoubled_link` was written
+for: providers that answer the listing with a resolved link and then mangle it
+when handed it back.
+
+Four guards narrow it, and every one of them falls back to asking the portal,
+so none can break an install that works today. Absent flags mean *no evidence*
+rather than "no". The scheme must be one ffmpeg opens (a portal's own
+`ffrt4://` pseudo-URL parses like an address and plays as nothing). The host
+must not be loopback in any of its spellings. And — this one is ours, not the
+reference behaviour — the command must carry **no query string**: a link that
+expires keeps its token there, and by the time a stream fails the resolver has
+already become ffmpeg and Dispatcharr has spent this channel's failover.
+
 ### The ffmpeg defaults carry no `-reconnect`, and that is load-bearing
 
 ffmpeg's own reconnection retries the URL it was handed, which for Stalker is a
@@ -259,6 +284,44 @@ Consequences that shape the code:
   (`Plugin._report`). Anything more urgent goes to Dispatcharr's notification
   centre, which does reach an open browser (`sync.announce`).
 
+### The panel is shown a redacted list
+
+The settings row is served to every account on the install and painted straight
+into a textarea, so it is the one copy of the portal list that must not hold a
+credential. `Plugin._save_settings` therefore writes two different things: the
+whole list to `portals.txt`, and `stalker_api.mask_portals()` of it to the row.
+The MAC, `username`, `password`, `device_id`, `device_id2`, `serial` and
+`signature` become `••••`; names, URLs and the tuning keys stay, because a box
+of nothing but bullets is one nobody can recognise their own portals in.
+
+Redaction is a property of the **write path and nothing else**.
+`_reconcile_registry` runs `unmask_portals()` on whatever the panel sent and
+always returns the real list, so every migration, action and sync downstream
+goes on reading plain lines and never has to know.
+
+Four things are easy to break here:
+
+- **Never mirror redacted text.** `_save_settings` checks `is_masked()` before
+  writing `portals.txt`, because `_failed()` can be reached with settings that
+  never passed through `_reconcile_registry` — and mirroring a row of tokens
+  would write them over the only copy of the credentials.
+- **Never redact before the file holds the list.** The guard is
+  `digest(stored) == digest(text)`. A registry that could not be written leaves
+  the row as the only copy there is, and hiding the only copy loses it.
+- **The token stands where the MAC stands.** `split_portal_line` decides which
+  field is which by *where the MAC sits*, so `_mac_index()` counts the token as
+  one; without that, redacting an unnamed line's MAC shifts its fields by one.
+  For the same reason `mask_portals` writes the derived name out.
+- **A redacted line is paired back up by slug, then by URL.** Renaming a portal
+  and repointing it are both ordinary edits, and each changes the half the other
+  is recognised by. Change both at once and nothing matches: the token survives
+  `unmask_portals`, and `Plugin._portals` quotes the line back and asks for it
+  to be retyped rather than parsing a MAC address made of bullets.
+
+None of this is encryption at rest, and the README says so: the resolver reads
+the MAC on every tune with no Django, so `portals.txt` and the state mirrors go
+on holding it in the clear at `0600`.
+
 ### Actions
 
 `plugin.json` is the single definition of the UI; `plugin.py` reads it at import

README.md

@@ -59,6 +59,7 @@ A plugin that writes credentials to disk should say so plainly:
 | --- | --- |
 | `/data/uploads/m3us/distalker-<slug>.m3u` | The generated playlist |
 | `/data/distalker/portals.txt` | Your portal list verbatim, **credentials included** |
+| Dispatcharr's plugin settings row | The same list with every credential **redacted** — this is the copy the panel renders and the API serves |
 | `/data/distalker/state/*.json` | What the resolver reads at tune time, **credentials included**, `0600` |
 | Redis `distalker:*` | The same, plus the session token |
 
@@ -156,13 +157,14 @@ URL:
 | `http://host:8080/c/` | `http://host:8080/c/portal.php` |
 | `host:8080/c/` | `http://host:8080/c/portal.php` |
 | `http://host` | `http://host/portal.php` |
-| `http://host/…/load.php` | unchanged — explicit endpoints are preserved |
-| `http://host/c/other.php` | `http://host/c/portal.php` |
+| `http://host/…/load.php` | unchanged — any explicit `.php` is preserved |
+| `http://host/cp/api.php` | unchanged — a panel's own path is an address, not a typo |
 
 If that path turns out not to be where the portal answers, Distalker tries the
-other one Ministra uses — `…/c/portal.php` and `…/server/load.php` are swapped
-for each other — and logs which one worked. Putting the working one on the
-portal line saves a failed request on every sync.
+others in turn — `…/server/load.php`, `…/c/portal.php`, `…/portal.php` and
+`…/stalker_portal/server/load.php`, built from the same install root — and logs
+which one worked. Putting the working one on the portal line saves a failed
+request on every sync.
 
 Anything unusual goes in trailing `key=value` pairs, separated by spaces or
 further `|` characters, quoted where a value contains spaces
@@ -182,9 +184,16 @@ further `|` characters, quoted where a value contains spaces
 > Raise it only on what your provider told you: exceeding it is the quickest
 > route to a blocked MAC.
 
-> **Credentials are visible in this box**, and stored unencrypted in the
-> Dispatcharr database like every plugin setting. Treat your backups
-> accordingly.
+> **Credentials are hidden once saved.** The MAC, the password and the rest of
+> the account identity come back as `••••` the moment the list is stored: the
+> real line lives in `/data/distalker/portals.txt`, which the settings panel
+> cannot read. Edit around the bullets and what you leave alone is left alone —
+> renaming a portal or repointing its URL both work with the MAC still hidden.
+>
+> **Keep your own copy of each portal line in a password manager.** The box will
+> not give a credential back, and if `/data/distalker` is ever lost the only
+> remaining copy goes with it. Hidden is also not encrypted — see
+> [What it writes, and where](#what-it-writes-and-where).
 
 ### STB identity
 
@@ -391,8 +400,10 @@ deleting it.
   a stock Dispatcharr can run, so *Refresh every (hours)* drives the M3U
   accounts' own refresh interval and answers the event that follows. It works,
   and it is why the interval cannot usefully go below an hour.
-- **Credentials are stored unencrypted**, in the Dispatcharr database and on
-  disk — see [What it writes, and where](#what-it-writes-and-where).
+- **Credentials are stored unencrypted** on disk. The settings panel no longer
+  shows them, but `/data/distalker/portals.txt` and the state mirrors hold them
+  in the clear at `0600`: the resolver reads them on every tune, in a process
+  with no database — see [What it writes, and where](#what-it-writes-and-where).
 - **No session keep-alive.** A cached token is reused and re-issued on demand.
   Portals that drop idle sessions are untested.
 - **One `ffmpeg` per tuned channel**, which is normal for any non-proxy stream

plugin.json

@@ -1,6 +1,6 @@
 {
   "name": "Distalker",
-  "version": "0.9.3",
+  "version": "0.9.4",
   "description": "Stalker/MAG portal support for Dispatcharr. Syncs portal channels into a native M3U account and resolves short-lived stream links at tune time -- no extra containers, no extra ports.",
   "author": "PiloUnk",
   "license": "AGPL-3.0-only",
@@ -19,7 +19,7 @@
       "type": "text",
       "default": "",
       "placeholder": "http://portal.example.com:8080/c/ | 00:1A:79:AA:BB:CC",
-      "help_text": "Portal URL | MAC address, one line each. The name is taken from the host, so put one in front only if you want a different label -- or if two portals share a host, which the sync will then ask you to do. Trailing key=value pairs cover the rest: username, password, max_streams, model, serial, device_id, device_id2, timezone, and epg=1 to fetch this portal's programme guide (epg_hours=48 for more than a day -- a guide is by far the largest thing a sync downloads, which is why it is off unless asked for). A line starting with '#' is ignored, which is how you suspend a portal without losing its channels. Credentials are stored unencrypted and are visible in this box."
+      "help_text": "Portal URL | MAC address, one line each. The name is taken from the host, so put one in front only if you want a different label -- or if two portals share a host, which the sync will then ask you to do. Trailing key=value pairs cover the rest: username, password, max_streams, model, serial, device_id, device_id2, timezone, and epg=1 to fetch this portal's programme guide (epg_hours=48 for more than a day -- a guide is by far the largest thing a sync downloads, which is why it is off unless asked for). A line starting with '#' is ignored, which is how you suspend a portal without losing its channels. Once saved, the MAC, the password and the rest of the account identity are replaced by bullets here and kept in a file only this plugin reads; edit around the bullets and what you leave alone stays as it was. That hides them from this page, it does not encrypt them -- and the box will not give one back, so keep your own copy of each line in a password manager."
     },
     {
       "id": "status",