[Feature / RFC] Knossos Network — peer-to-peer mod transfer between KNet instances

[Shivansps]

Summary

Add an optional networking layer that lets two Knossos.NET instances transfer mods directly to each other over a LAN (and, optionally, a direct IP/port).

The primary use case: prepare/transcode mods on your desktop (S3TC/BC7 → ETC2, LZ41-compressed) and push them to the KNet Android client on your phone, without re-downloading from Nebula and without transcoding on the (slow) phone. It works both ways (pull from a peer, or push to a peer), and can be used PC↔PC, PC↔phone, or phone↔PC.

This is an RFC as much as a feature request — I'd like feedback on the transport, discovery/pairing, and security model before anyone writes code. Nothing here is final.

Motivation

• Transcoding to ETC2 on a phone is slow and battery-hungry. Doing it once on the desktop and shipping the result is much better.
• Re-downloading large mods over the internet on mobile data / weak Wi-Fi is wasteful when the mod already exists (already transcoded) on a machine on the same network.
• A modder or a LAN party can seed prepared mods to several clients without everyone hitting Nebula.

Why this is feasible (most of the hard parts already exist)

A big reason to file this now is that the codebase already contains almost every "hard" building block. The genuinely new work is the network transport, discovery/pairing, security, and UI — not the mod handling itself.

Already implemented and reusable:

• ETC2 transcoding pipeline — Etc2Transcoder (DDS BC1/2/3/BC7 → KTX1/ETC2) plus the TranscodeMod / TranscodeVP / TranscodeLosseFiles tasks, including .eff patching via PatchEffFiles. "Prepare a mod for a device that lacks S3TC/BC7" is essentially already a solved task.
• Per-device capability detection — AndroidHelper.GpuSupportsBCnTexturesOpenGL() already returns (s3tc, bc7) by querying OpenGL ES extensions; desktop reports full support.
• Robust file transfer client — DownloadFile already supports multiple mirror URLs, throttling (ThrottledStream), progress (ProgressStream), and cancellation. If the server exposes each mod file at an HTTP URL, the existing downloader/installer can consume a peer as if it were just another Nebula mirror, with almost no new client-side transfer code.
• Integrity metadata — ModFile already carries checksum (sha256) and filesize, so received files can be verified with existing data.
• Task queue — TaskViewModel already models every long operation (install, compress, transcode, download) as a queued TaskItemViewModel. "Serve", "Send", "Receive", and "Prepare to stash" fit this pattern directly.
• Settings + background — GlobalSettings is a persisted JSON blob (easy to extend with a network section), and tray mode (-traymode) already exists as a basis for "serve in the background".
• Mod library API — Knossos.AddMod / RemoveMod / GetInstalledMod / GetInstalledModList and Mod.SaveJson() cover reconstructing a received mod in the library.

Key architectural insight

The cleanest design is probably "the server is just a mirror." If the server exposes a mod's mod.json and each of its files over HTTP, the receiving side reuses the normal install path (InstallMod + DownloadFile) with peer URLs substituted for Nebula URLs. That keeps the client side thin and reuses all the resume/verify/throttle logic that already exists.

Proposed behavior

Server (any instance can be one; opt-in via settings)

• Toggle to enable serving; custom port selection.
• Visibility options:
  • Code (default) — autogenerated or manual short code, similar to how many mobile apps pair ("open on PC, get a code, type it on the phone"). The code also doubles as the shared secret / auth token.
  • Open — anyone on the network can see and connect.
• Can run in the background (tray) and, ideally, headless without loading the UI.
• Serve-mods toggle (default off): allow clients to pull mods from this server. Either "all mods" or a manual allow-list of id + version (or all versions of an id).
• Receive-mods toggle (default on): allow clients to push mods to this server.
• Advertise capabilities (S3TC / BC7) and free library space to connected clients.

Client

• Connect via a pairing code (same network), or via direct IP/port.
• Choose direction per session: pull mods from the server, or push mods to it.
• When pushing to a server that lacks S3TC and/or BC7, transcode on the fly locally before/while sending (full transcode config available), so the payload matches what the target device can actually use.

Stash / "prepared mods" (both roles)

• A hidden stash of prepared mods, kept in a special folder in the library and not shown in normal mod lists.
• "Prepare" = run the existing transcode (→ETC2) + LZ41 compression pipeline and drop the result in the stash, so mods can be pre-baked once and then offered/pushed later without transcoding at transfer time.
• Both sides need to know each other's S3TC/BC7 capabilities and free space before transfer.
• Option to clean/empty the stash on demand.

Proposed architecture

• Core lib (Knossos.NET) hosts the transport, protocol, discovery, and stash logic so both Desktop and Android share it.
• Transport: an embedded HTTP server exposing:
  • GET /caps — S3TC/BC7 support, free space, protocol version, whether receive is enabled.
  • GET /mods — list of offered mods (honoring the allow-list / visibility), each with id, version, capability variant (original vs ETC2), and size.
  • GET /mods/{id}/{version}/manifest — the mod.json + file list (sha256 + size), so the client can reuse InstallMod.
  • GET /mods/{id}/{version}/files/{path} — a single file, with HTTP range support so DownloadFile's resume/throttle works unchanged.
  • POST /receive/... — upload endpoint for pushes (only when receive is enabled).
• Client: reuse InstallMod + DownloadFile, pointing file URLs at the peer. Verify sha256 on completion (already available).
• New tasks: ServeNetwork (long-lived), SendModToPeer, ReceiveModFromPeer (or reuse InstallMod with peer URLs), PrepareModToStash (wrapper over the existing transcode/compress tasks).

Open design questions

1. Transport. Raw System.Net.HttpListener (zero deps, but Windows may need a URL ACL / firewall prompt for non-loopback binds), embedded Kestrel / ASP.NET minimal API (native range + streaming, but heavier — how does it behave inside the net10.0-android app?), or a small third-party embedded HTTP server? Or a raw TcpListener with a custom framing protocol (max control, reinvents HTTP range/resume)? Leaning toward "an HTTP server so the existing client is reused as a mirror" — thoughts?
2. Discovery / pairing. For "connect by code on the same network," how do we resolve a code to an endpoint? Options: mDNS/Zeroconf service advertisement, a UDP broadcast/beacon on the LAN, or code = compact encoding of ip:port (offline, no discovery). Note Android 13+ needs NEARBY_WIFI_DEVICES for some local-network discovery.
3. Security. LAN-only trust vs authenticated transfers. Proposed: derive an auth token from the pairing code; optionally TLS (self-signed + pin, or plaintext on trusted LAN). We must protect the receive endpoint against path traversal, disk-fill, and untrusted mod content; sha256 verification is available but content trust is a separate question.
4. Android as a server. Requires a foreground service to stay alive; cleartext HTTP needs usesCleartextTraffic (or a per-address network-security-config); Doze/battery caveats. Is phone-as-server in scope for v1, or should v1 be desktop-serves / phone-pulls only?
5. "Prepared" variant tracking. How should a stashed/transferred ETC2 build be tagged so the receiving library knows it's already transcoded (and shouldn't be re-transcoded, and should be flagged as ETC2 for FSO)? Does the current mod metadata capture "this build targets no-S3TC/no-BC7 / ETC2," or do we need a new field?
6. Headless serving. Tray mode exists but the app still spins up the Avalonia lifetime. Do we want a true headless "server only" launch mode (e.g. -netserver) that never loads UI resources?
7. Free-space / capability negotiation. Where in the handshake do we exchange free space + caps, and how do we present "won't fit" / "device can't use this format, transcode first?" to the user before a transfer starts?

Rough phased plan

• Phase 0 – RFC: settle transport, discovery, security.
• Phase 1 – Stash + prepare: hidden stash folder, "prepare to stash" (reuse transcode + LZ41), clean-stash action, prepared-variant metadata.
• Phase 2 – Server: embedded HTTP server, /caps /mods /manifest /files (+range), allow-list & visibility, background/tray operation.
• Phase 3 – Client (pull): connect by code or direct IP/port, browse remote mods, pull via existing install path with peer URLs, verify sha256.
• Phase 4 – Client (push) + on-the-fly transcode: push to a peer; transcode locally when the target lacks S3TC/BC7.
• Phase 5 – Discovery/pairing: mDNS/UDP beacon, code↔endpoint resolution, code-as-secret auth.
• Phase 6 – Hardening + Android service + UX: foreground service, path-traversal/space guards, settings pages, optional TLS, headless mode.

Scope / non-goals

• Not a replacement for Nebula; strictly a side channel between instances.
• Initial target is same-LAN; direct IP/port is the manual fallback. No relay/broker/internet-wide transfer in v1.