Add galaxy-web: browsable/searchable web UI for galaxy archives

[krzlew]

Summary

• web.cpp currently only supports looking up a single message by Message-ID — there's no way to browse groups or threads through a web UI.
• This adds galaxy-web, a new local web server built on top of libuat/Galaxy/Archive, providing:
  • / — newsgroup listing with description, message and thread counts
  • /group/<name> — paginated thread list (50/page)
  • /group/<name>/thread/<id> — full threaded message view (nested replies)
  • /group/<name>/msg/<id> — single message permalink
  • /search?q=... and /group/<name>/search?q=... — full-text search across the whole galaxy or within one group
  • /msgid/<id> — resolves a bare Message-ID (as found in quoted news: links) to its group+message URL
• Search needs no extra indexing step: each archive's lexicon (lexdata/lexstr/etc.) is already embedded in packaged archives, so SearchEngine works immediately per group; global search loops over all available archives.
• Message rendering (headers, quote-depth coloring, signature, decorations) reuses the same MessageLines-based approach already used by web.cpp.
• Added man/uat-galaxy-web.1, wired into the CMake install list, and listed the new tool (plus the pre-existing but undocumented web) under ### End-user Utilities in the README. Also removed the "Implement pan-group search mechanism" future-work bullet, since this delivers it.

Test plan

• cmake --build build --target galaxy-web builds cleanly
• Ran against a real galaxy (333 newsgroups, built via galaxy-util) and exercised every route with curl: group list, thread list + pagination, thread view, message permalink, group-scoped search, global search, and /msgid/<id> redirect resolution
• Verified rendering of real archived messages (headers, quoting, signatures, Polish UTF-8 text) matches web.cpp's existing styling

[wolfpld]

Auto-review below. Most things actionable.

---

Review: usenetarchive PR #3 — galaxy-web

Verified everything against upstream source at the PR's base (16b4ed1, release 1.0). I built the PR head cleanly, then built a real 2-group galaxy (1055 messages, packaged archives via import-source-maildir → extract-msgid → connectivity → extract-msgmeta → lexicon → lexsort → repack-zstd → package → galaxy-util) and exercised every route — plus an AddressSanitizer build for memory safety. No originals touched (scratch clone in /tmp/ua-review).

What checks out

• Build claim confirmed: cmake --build --target galaxy-web — clean (clang + gcc, exit 0, no warnings from the new TU). Link set mongoose ini common libuat zstd exactly matches the web target (CMakeLists.txt:316).
• Routing is sound: I verified in the vendored Mongoose 6.18 source (contrib/mongoose/mongoose.c:6112-6117) that hm->uri is truncated at ? and query_string set separately — so SplitPath + mg_get_http_var interact correctly. Live-tested ?page=, ?q=, pagination, clamp page=999 → Page 10 of 10, 404s on unknown groups/out-of-range idx/bad msgid.
• API usage correct: GetArchive(idx, false) is the right read-only choice (Galaxy.cpp:84 — change=true sets m_active, write bookkeeping); SearchEngine/SearchResult.postid/.rank/SF_AdjacentWords match SearchEngine.hpp; RepackMsgId direction matches web.cpp:224; man page follows repo conventions and uat-lexicon(1)/uat-web(1) both exist.
• Escaping verified with adversarial data: & < > " in group descriptions and <script> inside message bodies render fully escaped; %2B etc. round-trip through the msgid redirect.

Blocker: unbounded PackMsgId → stack buffer overflow in /msgid/<id>

galaxy-web.cpp Handler, /msgid route:

uint8_t packed[4096];
galaxy->PackMsgId( midStr.c_str(), packed );

IsMsgId validates only charset (printable ASCII, exactly one @) — no length cap. Mongoose accepts requests up to MG_MAX_HTTP_REQUEST_SIZE = 8192, and StringCompress::Pack passes through ~1 byte per input byte (trigram/bigram tables cover only 0-9a-v.$; unknown hosts fall back to raw @host copy). So a printable path segment ≥ ~4096 chars overflows packed (and archivePacked via RepackMsgId).

Proven, not theoretical: ASan galaxy-web died with stack-buffer-overflow ... StringCompress::Pack on GET /msgid/AAAA…(5000)@x.org; the plain Release build also died outright (connection reset, process gone) on the same single request. Reachable at minimum as remote DoS; classic stack smash layout means potential code execution. This pattern is inherited from web.cpp:212-224 (identical 4096 buffers — so upstream itself is vulnerable), but this PR ships a second instance whose config explicitly encourages choosing the bind address (bind = 0.0.0.0 is a natural "expose my archive" move). Minimal fix in the PR: reject midStr.size() > ~2048 before packing; proper fix: cap length in IsMsgId (web.cpp would be fixed with it).

Integration gaps

1. Binary never installed. web is in install(TARGETS ... RUNTIME DESTINATION lib/uat) (CMakeLists.txt:351); the PR adds the man page but not galaxy-web to that list — install a uat-galaxy-web.1 documenting a binary that isn't installed.
2. Not in the dispatcher. uat <tool> execs /usr/lib/uat/<tool> via the hardcoded table in uat/uat.cpp:20-51 — galaxy-web isn't listed, so uat galaxy-web → "No such command".
3. README future-work removal is overreach: "Implement pan-group search mechanism" is the maintainer's explicitly-declined-ideas list, and galaxy-web doesn't actually deliver it — it loops per-archive lexicons with caps, no galaxy-level index. Revert that hunk; let wolfpld strike it.
4. PR body says web is "pre-existing but undocumented" — it isn't: man/uat-web.1 was added in de323aa, an ancestor of the PR base.

Bugs, minor

• /group/x/msg/abc → 200, renders message 0 (strtoul failure silently yields 0; same for thread/). Should 404. Live-confirmed.
• UriDecode maps + → space in path segments (galaxy-web.cpp UriDecode). Per RFC 3986 + is literal in paths; msgids commonly contain + (Gmail-style). Self-generated links are fine (own encoder %2B-escapes), but externally pasted /msgid/<tmbPz+aaa@mail> links resolve to the wrong (nonexistent) msgid. web.cpp doesn't do this.
• Group-search truncation lies: SearchArchive( archive, name, q, SearchGlobalCap ) + always truncated=false — live confirmed exactly 200 results for a query with ~400+ matches, no +.
• Cross-group rank merging compares per-corpus TF/IDF-style scores — apples to oranges; and the per-group cap of 20 is applied before the global merge, biasing galaxy-wide results toward spread rather than quality.
• Silent query drop: GetQueryVar's 4096-byte buffer → mg_get_http_var returns -3 for over-long values → empty query → bare form, no error.
• Global search loops every archive synchronously in the single-threaded Mongoose loop (no caching) — one query blocks all connections on a large galaxy.
• FindRoot has no cycle guard — corrupt connectivity data hangs the server.
• Inherited from web.cpp: no scheme filter on D_Url hrefs (javascript: URLs from message bodies pass through as clickable links); same mg_mgr_poll(INT_MAX) loop.

Side findings (upstream, not PR-caused)

• repack-zstd segfaults on small archives: ZDICT_optimizeTrainFromBuffer_fastCover return value unchecked (repack-zstd/repack-zstd.cpp:129), negative error → huge size_t dict size → crash in ZSTD_createCDict. Confirmed under gdb; worked around with ≥250-message corpora.
• All UAT binaries abort before main on systems without a compiled en_US.UTF-8 locale (common/UTF8.cpp:92, global std::locale utf8("en_US.UTF-8")) — hit this on a stock Arch box; upstream web crashes identically.

Verdict

The feature works as advertised — routes, threading, escaping, pagination, and search all verified live against a real galaxy — and the code faithfully mirrors web.cpp conventions. But the unbounded PackMsgId overflow is a proven memory-safety bug on an externally reachable route and must be guarded before merge; install/dispatcher registration must be added; the README future-work hunk should be reverted.