Skip to content

fix(research,judge,sanitize): drop multi-line result urls, one-line report labels, cap kept joiners per run - #45

Merged
wikithoughts merged 2 commits into
mainfrom
overnight/untrusted-url-and-report-fields
Oct 11, 2026
Merged

wikithoughts merged 2 commits into
mainfrom
overnight/untrusted-url-and-report-fields

Conversation

@wikithoughts

Copy link
Copy Markdown
Owner

Follow-ups from the independent review of #42/#43 (items 1–3 of overnight task W6-passiveworkers-4243fu). Branched from origin/main at f497835, after both #42 and #43 merged. Item 4 (the serve override warning and the README proxy note) is in a separate PR.

What changed

  • passiveworkers/research.py (finding fix(sanitize): keep ZWJ/VS16/ZWNJ/RGI flags in context; strip more invisibles; clean legacy search #42-1, Medium): new _result_url(), used by both _clean() and search_structured(). A result whose href contains whitespace or a non-printable character (newline, CR, U+2028, U+200B, NUL, TAG chars) is dropped. urlparse() ignores those characters (the hostname still parses as good.example), so https://good.example/a\n[S9] Nature … passed the host checks and forged a source line. Plain URLs, including non-ASCII paths, are unchanged.
  • passiveworkers/judge.py (finding fix(serve,judge): exact-origin check, strict Host, PW_SERVE_ALLOWED_HOSTS; country inside spotlight #43-1, Medium): new _report_label(), applied to the node-supplied country and model in compile_report's final report (the byline and the ### country — model headings). It strips invisibles, makes the value one line, drops ` * [ ] < > # | ~ \, breaks :// so the renderers can't auto-link it, and caps the length (country 40, model 60). The editor prompt hunk from fix(serve,judge): exact-origin check, strict Host, PW_SERVE_ALLOWED_HOSTS; country inside spotlight #43 is untouched.
  • passiveworkers/sanitize.py (finding fix(sanitize): keep ZWJ/VS16/ZWNJ/RGI flags in context; strip more invisibles; clean legacy search #42-2, Medium): a run keeps at most _MAX_KEPT_PER_RUN = 8 kept ZWJ/VS16/ZWNJ/RGI-flag forms, and any after that are stripped. A run is an unbroken stretch of emoji, Arabic letters/marks and keycap parts. Stripped invisibles don't break a run, so a ZWSP can't reset the count. The scan stays linear (each gap is checked once).
  • docs/DECISIONS.md D55: the covert-channel figure is corrected. ZWJ and VS16 are independent in each emoji gap, so that is 2 bits per gap, not "about one bit". ZWNJ carries 1 bit per Arabic letter gap, and a black flag up to 2 bits. The text records the cap and what it does not bound: it limits each run, not the whole text, and the positions of the 8 kept forms still encode more than 8 bits. "Not a payload" is now stated as a judgement, not a proven bound.

Known cost (accepted, documented in D55 and pinned by a test)

A row of joined emoji with no spaces and more than 8 kept forms loses its later joiners and shows as separate emoji. Examples: three family emoji back to back, or nine 👩🏽‍💻 in a row. Two sequences back to back (up to 8) and space-separated rows are unaffected.

Verification (run locally, .venv, Python 3)

  • python -m py_compile $(git ls-files '*.py') → exit 0
  • ruff check . && pytest tests/ -q --cov=passiveworkers --cov-fail-under=75 → exit 0, 750 passed, 9 skipped, coverage 80.69%
  • New tests fail on old code (each run with only the source file reverted to origin/main):
    • tests/test_research_url.py (new file): 16 of 17 fail on old research.py. The one that passes checks that plain URLs survive, so it passes on both.
    • tests/test_judge_spotlight.py::test_compile_report_one_lines_node_labels_in_the_final_report: both parametrizations (local and network) fail on old judge.py.
    • tests/test_sanitize.py::test_a_run_keeps_at_most_eight_joiners: all 5 cases fail on old sanitize.py. test_run_cap_leaves_real_text_alone was extended after review (the unspaced-row cost) and was not re-run against old code.
    • Total: 23 new test cases fail on old code.
  • Adversarial review: a separate read-only reviewer probed the first commit. No High or Medium findings. It reported three Low findings (the unspaced-row cost and two D55 wording points), all fixed in the second commit. It also probed URL bypasses (every splitlines separator), label payloads against both md() renderers, the counter-reset attempts and performance (400k-pair inputs ran in under 0.4 s, linear).

Unverified

  • No browser check: the HTML/JS isn't touched. The report change was checked as markdown text in the tests, and the reviewer read md() in serve.py/net/app.py.
  • The run-cap test for "real text left alone" is a best effort. It covers common sequences, not every RGI emoji.
  • CI was not watched (not allowed in this lane).

Merge order: independent of the item-4 PR. Both branch from the same origin/main and touch different files.

🤖 Generated with Claude Code

wikithoughts and others added 2 commits October 11, 2026 08:02
…eport labels, cap kept joiners per run

- research: a search result whose href holds whitespace or an invisible/control
  character is dropped (_result_url). urlparse() ignores an embedded newline,
  U+2028 or U+200B, so "https://good.example/a\n[S9] ..." passed the host checks
  and forged a source line in _clean() and search_structured().
- judge: node-supplied country/model in the final report (byline and per-
  contribution headings) are one plain line: invisibles stripped, markdown/HTML
  control chars dropped, "://" broken, length capped (_report_label).
- sanitize: a run keeps at most 8 ZWJ/VS16/ZWNJ/RGI-flag forms; extra ones are
  stripped. Stripped invisibles don't break a run.
- D55: correct the covert-channel figure (ZWJ and VS16 are independent per
  emoji gap = 2 bits; ZWNJ 1 bit per Arabic letter gap) and record the cap and
  what it does not bound.

Follow-ups from the independent review of #42/#43.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…run-reset wording

From the adversarial review of the previous commit: a row of more than 8 joined
emoji with no space splits (pinned by a test), a black flag carries up to 2 bits,
and any visible character outside the run classes starts a new run.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@wikithoughts
wikithoughts merged commit 2d489bd into main Oct 11, 2026
11 checks passed
@wikithoughts
wikithoughts deleted the overnight/untrusted-url-and-report-fields branch October 11, 2026 08:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant