Skip to content

Atomic output writes, CSV formula neutralisation, 0600 HTML dumps - #17

Merged
jehrr merged 1 commit into
mainfrom
fix/atomic-writes-csv-escape
Oct 1, 2026
Merged

jehrr merged 1 commit into
mainfrom
fix/atomic-writes-csv-escape

Conversation

@jehrr

@jehrr jehrr commented Sep 30, 2026

Copy link
Copy Markdown
Collaborator

Follow-up to the 2026-10-01 audit of d8660fa, verified against the code first. Family items §27.1 and §27.2 plus a small dump hardening.

Changes

  • Atomic writes for json, csv and the .meta.json sidecar (lifted from snapchat-scraper; mode fix added, since NamedTemporaryFile is 0600 and a rename keeps it).
  • CSV-only formula neutralisation, strings only, after the list join; count in the sidecar under the caller's extra.
  • HTML dumps written 0600 via write_private_text in all three engines.

Evidence: smoke_test passes (engine groups skipped locally: no engine libraries installed, so the three engine edits are covered by compile, the undefined-name walk and a source check, NOT a live run). Eight planted faults (json/sidecar written in place, chmod removed, numbers escaped, escape before join, extra precedence reversed, dump chmod removed) each went red on the named check; one more (dump created 0644) stays green because the following chmod makes it an equivalent mutant.

Not changed, and why: no Bright Data/Oxylabs adapters (2Captcha only), no engine removal (family contract), no NDJSON, no scheduled canary (profiles expire in about a day), no parser split or project.scripts. The SECURITY.md line about cookies was reworded: a DOM snapshot holds no HTTP cookies; not measured what a served page leaks.

Needs CI before merge; engine-smoke will run the three engines.

🤖 Generated with Claude Code

- write_json/write_csv/write_run_meta go through _atomic (temp file in the
  target's directory, fsync, os.replace); mode 0644 for new files, the
  existing mode for rewrites, since NamedTemporaryFile is 0600.
- CSV: strings starting with = + - @ tab CR LF get a leading apostrophe,
  after the list join, numbers untouched; csv_cells_escaped in the sidecar
  (caller's extra wins a collision). JSON keeps the site's bytes.
- HTML dumps in all three engines via write_private_text (0600).
- smoke_test: one new test; eight planted faults, each confirmed red on the
  named check (a ninth was an equivalent mutant, see PR).
Not changed: no NDJSON/streaming, no engine removal, no competitor adapters.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
@jehrr
jehrr marked this pull request as ready for review October 1, 2026 11:45
@jehrr
jehrr merged commit 8c2db1f into main Oct 1, 2026
7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant