Repository navigation
fix(research,judge,sanitize): drop multi-line result urls, one-line report labels, cap kept joiners per run - #45
Merged
Conversation
…eport labels, cap kept joiners per run - research: a search result whose href holds whitespace or an invisible/control character is dropped (_result_url). urlparse() ignores an embedded newline, U+2028 or U+200B, so "https://good.example/a\n[S9] ..." passed the host checks and forged a source line in _clean() and search_structured(). - judge: node-supplied country/model in the final report (byline and per- contribution headings) are one plain line: invisibles stripped, markdown/HTML control chars dropped, "://" broken, length capped (_report_label). - sanitize: a run keeps at most 8 ZWJ/VS16/ZWNJ/RGI-flag forms; extra ones are stripped. Stripped invisibles don't break a run. - D55: correct the covert-channel figure (ZWJ and VS16 are independent per emoji gap = 2 bits; ZWNJ 1 bit per Arabic letter gap) and record the cap and what it does not bound. Follow-ups from the independent review of #42/#43. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…run-reset wording From the adversarial review of the previous commit: a row of more than 8 joined emoji with no space splits (pinned by a test), a black flag carries up to 2 bits, and any visible character outside the run classes starts a new run. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Follow-ups from the independent review of #42/#43 (items 1–3 of overnight task W6-passiveworkers-4243fu). Branched from origin/main at f497835, after both #42 and #43 merged. Item 4 (the serve override warning and the README proxy note) is in a separate PR.
What changed
passiveworkers/research.py(finding fix(sanitize): keep ZWJ/VS16/ZWNJ/RGI flags in context; strip more invisibles; clean legacy search #42-1, Medium): new_result_url(), used by both_clean()andsearch_structured(). A result whose href contains whitespace or a non-printable character (newline, CR, U+2028, U+200B, NUL, TAG chars) is dropped.urlparse()ignores those characters (the hostname still parses asgood.example), sohttps://good.example/a\n[S9] Nature …passed the host checks and forged a source line. Plain URLs, including non-ASCII paths, are unchanged.passiveworkers/judge.py(finding fix(serve,judge): exact-origin check, strict Host, PW_SERVE_ALLOWED_HOSTS; country inside spotlight #43-1, Medium): new_report_label(), applied to the node-suppliedcountryandmodelincompile_report's final report (the byline and the### country — modelheadings). It strips invisibles, makes the value one line, drops` * [ ] < > # | ~ \, breaks://so the renderers can't auto-link it, and caps the length (country 40, model 60). The editor prompt hunk from fix(serve,judge): exact-origin check, strict Host, PW_SERVE_ALLOWED_HOSTS; country inside spotlight #43 is untouched.passiveworkers/sanitize.py(finding fix(sanitize): keep ZWJ/VS16/ZWNJ/RGI flags in context; strip more invisibles; clean legacy search #42-2, Medium): a run keeps at most_MAX_KEPT_PER_RUN = 8kept ZWJ/VS16/ZWNJ/RGI-flag forms, and any after that are stripped. A run is an unbroken stretch of emoji, Arabic letters/marks and keycap parts. Stripped invisibles don't break a run, so a ZWSP can't reset the count. The scan stays linear (each gap is checked once).docs/DECISIONS.mdD55: the covert-channel figure is corrected. ZWJ and VS16 are independent in each emoji gap, so that is 2 bits per gap, not "about one bit". ZWNJ carries 1 bit per Arabic letter gap, and a black flag up to 2 bits. The text records the cap and what it does not bound: it limits each run, not the whole text, and the positions of the 8 kept forms still encode more than 8 bits. "Not a payload" is now stated as a judgement, not a proven bound.Known cost (accepted, documented in D55 and pinned by a test)
A row of joined emoji with no spaces and more than 8 kept forms loses its later joiners and shows as separate emoji. Examples: three family emoji back to back, or nine 👩🏽💻 in a row. Two sequences back to back (up to 8) and space-separated rows are unaffected.
Verification (run locally,
.venv, Python 3)python -m py_compile $(git ls-files '*.py')→ exit 0ruff check . && pytest tests/ -q --cov=passiveworkers --cov-fail-under=75→ exit 0,750 passed, 9 skipped, coverage 80.69%origin/main):tests/test_research_url.py(new file): 16 of 17 fail on oldresearch.py. The one that passes checks that plain URLs survive, so it passes on both.tests/test_judge_spotlight.py::test_compile_report_one_lines_node_labels_in_the_final_report: both parametrizations (local and network) fail on oldjudge.py.tests/test_sanitize.py::test_a_run_keeps_at_most_eight_joiners: all 5 cases fail on oldsanitize.py.test_run_cap_leaves_real_text_alonewas extended after review (the unspaced-row cost) and was not re-run against old code.splitlinesseparator), label payloads against bothmd()renderers, the counter-reset attempts and performance (400k-pair inputs ran in under 0.4 s, linear).Unverified
md()inserve.py/net/app.py.Merge order: independent of the item-4 PR. Both branch from the same origin/main and touch different files.
🤖 Generated with Claude Code